Merge branch 'for-linus' of git://git.o-hand.com/linux-rpurdie-backlight
* 'for-linus' of git://git.o-hand.com/linux-rpurdie-backlight:
backlight: Rename the corgi backlight driver to generic
backlight: add support for Toppoly TDO35S series to tdo24m lcd driver
backlight: Add suspend/resume support to the backlight core
bd->props.brightness doesn't reflect the actual backlight level.
backlight: Support VGA/QVGA mode switching in tosa_lcd
backlight: Catch invalid input in sysfs attributes
backlight: Value of ILI9320_RGB_IF2 register should not be hardcoded
backlight: crbllcd_bl - Use platform_device_register_simple()
backlight: progear_bl - Use platform_device_register_simple()
backlight: hp680_bl - Use platform_device_register_simple()
diff --git a/Documentation/ABI/testing/sysfs-class-regulator b/Documentation/ABI/testing/sysfs-class-regulator
index 3731f6f..873ef1f 100644
--- a/Documentation/ABI/testing/sysfs-class-regulator
+++ b/Documentation/ABI/testing/sysfs-class-regulator
@@ -3,8 +3,9 @@
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
- state. This holds the regulator output state.
+ Some regulator directories will contain a field called
+ state. This reports the regulator enable status, for
+ regulators which can report that value.
This will be one of the following strings:
@@ -18,7 +19,8 @@
'disabled' means the regulator output is OFF and is not
supplying power to the system..
- 'unknown' means software cannot determine the state.
+ 'unknown' means software cannot determine the state, or
+ the reported state is invalid.
NOTE: this field can be used in conjunction with microvolts
and microamps to determine regulator output levels.
@@ -53,9 +55,10 @@
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
microvolts. This holds the regulator output voltage setting
- measured in microvolts (i.e. E-6 Volts).
+ measured in microvolts (i.e. E-6 Volts), for regulators
+ which can report that voltage.
NOTE: This value should not be used to determine the regulator
output voltage level as this value is the same regardless of
@@ -67,9 +70,10 @@
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
microamps. This holds the regulator output current limit
- setting measured in microamps (i.e. E-6 Amps).
+ setting measured in microamps (i.e. E-6 Amps), for regulators
+ which can report that current.
NOTE: This value should not be used to determine the regulator
output current level as this value is the same regardless of
@@ -81,8 +85,9 @@
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
- opmode. This holds the regulator operating mode setting.
+ Some regulator directories will contain a field called
+ opmode. This holds the current regulator operating mode,
+ for regulators which can report it.
The opmode value can be one of the following strings:
@@ -92,7 +97,7 @@
'standby'
'unknown'
- The modes are described in include/linux/regulator/regulator.h
+ The modes are described in include/linux/regulator/consumer.h
NOTE: This value should not be used to determine the regulator
output operating mode as this value is the same regardless of
@@ -104,9 +109,10 @@
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
min_microvolts. This holds the minimum safe working regulator
- output voltage setting for this domain measured in microvolts.
+ output voltage setting for this domain measured in microvolts,
+ for regulators which support voltage constraints.
NOTE: this will return the string 'constraint not defined' if
the power domain has no min microvolts constraint defined by
@@ -118,9 +124,10 @@
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
max_microvolts. This holds the maximum safe working regulator
- output voltage setting for this domain measured in microvolts.
+ output voltage setting for this domain measured in microvolts,
+ for regulators which support voltage constraints.
NOTE: this will return the string 'constraint not defined' if
the power domain has no max microvolts constraint defined by
@@ -132,10 +139,10 @@
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
min_microamps. This holds the minimum safe working regulator
output current limit setting for this domain measured in
- microamps.
+ microamps, for regulators which support current constraints.
NOTE: this will return the string 'constraint not defined' if
the power domain has no min microamps constraint defined by
@@ -147,10 +154,10 @@
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
max_microamps. This holds the maximum safe working regulator
output current limit setting for this domain measured in
- microamps.
+ microamps, for regulators which support current constraints.
NOTE: this will return the string 'constraint not defined' if
the power domain has no max microamps constraint defined by
@@ -185,7 +192,7 @@
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
requested_microamps. This holds the total requested load
current in microamps for this regulator from all its consumer
devices.
@@ -204,125 +211,102 @@
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_mem_microvolts. This holds the regulator output
voltage setting for this domain measured in microvolts when
- the system is suspended to memory.
-
- NOTE: this will return the string 'not defined' if
- the power domain has no suspend to memory voltage defined by
- platform code.
+ the system is suspended to memory, for voltage regulators
+ implementing suspend voltage configuration constraints.
What: /sys/class/regulator/.../suspend_disk_microvolts
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_disk_microvolts. This holds the regulator output
voltage setting for this domain measured in microvolts when
- the system is suspended to disk.
-
- NOTE: this will return the string 'not defined' if
- the power domain has no suspend to disk voltage defined by
- platform code.
+ the system is suspended to disk, for voltage regulators
+ implementing suspend voltage configuration constraints.
What: /sys/class/regulator/.../suspend_standby_microvolts
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_standby_microvolts. This holds the regulator output
voltage setting for this domain measured in microvolts when
- the system is suspended to standby.
-
- NOTE: this will return the string 'not defined' if
- the power domain has no suspend to standby voltage defined by
- platform code.
+ the system is suspended to standby, for voltage regulators
+ implementing suspend voltage configuration constraints.
What: /sys/class/regulator/.../suspend_mem_mode
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_mem_mode. This holds the regulator operating mode
setting for this domain when the system is suspended to
- memory.
-
- NOTE: this will return the string 'not defined' if
- the power domain has no suspend to memory mode defined by
- platform code.
+ memory, for regulators implementing suspend mode
+ configuration constraints.
What: /sys/class/regulator/.../suspend_disk_mode
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_disk_mode. This holds the regulator operating mode
- setting for this domain when the system is suspended to disk.
-
- NOTE: this will return the string 'not defined' if
- the power domain has no suspend to disk mode defined by
- platform code.
+ setting for this domain when the system is suspended to disk,
+ for regulators implementing suspend mode configuration
+ constraints.
What: /sys/class/regulator/.../suspend_standby_mode
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_standby_mode. This holds the regulator operating mode
setting for this domain when the system is suspended to
- standby.
-
- NOTE: this will return the string 'not defined' if
- the power domain has no suspend to standby mode defined by
- platform code.
+ standby, for regulators implementing suspend mode
+ configuration constraints.
What: /sys/class/regulator/.../suspend_mem_state
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_mem_state. This holds the regulator operating state
- when suspended to memory.
+ when suspended to memory, for regulators implementing suspend
+ configuration constraints.
- This will be one of the following strings:
-
- 'enabled'
- 'disabled'
- 'not defined'
+ This will be one of the same strings reported by
+ the "state" attribute.
What: /sys/class/regulator/.../suspend_disk_state
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_disk_state. This holds the regulator operating state
- when suspended to disk.
+ when suspended to disk, for regulators implementing
+ suspend configuration constraints.
- This will be one of the following strings:
-
- 'enabled'
- 'disabled'
- 'not defined'
+ This will be one of the same strings reported by
+ the "state" attribute.
What: /sys/class/regulator/.../suspend_standby_state
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_standby_state. This holds the regulator operating
- state when suspended to standby.
+ state when suspended to standby, for regulators implementing
+ suspend configuration constraints.
- This will be one of the following strings:
-
- 'enabled'
- 'disabled'
- 'not defined'
+ This will be one of the same strings reported by
+ the "state" attribute.
diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile
index 0a08126..dc3154e 100644
--- a/Documentation/DocBook/Makefile
+++ b/Documentation/DocBook/Makefile
@@ -12,7 +12,7 @@
kernel-api.xml filesystems.xml lsm.xml usb.xml kgdb.xml \
gadget.xml libata.xml mtdnand.xml librs.xml rapidio.xml \
genericirq.xml s390-drivers.xml uio-howto.xml scsi.xml \
- mac80211.xml debugobjects.xml sh.xml
+ mac80211.xml debugobjects.xml sh.xml regulator.xml
###
# The build process is as follows (targets):
diff --git a/Documentation/DocBook/regulator.tmpl b/Documentation/DocBook/regulator.tmpl
new file mode 100644
index 0000000..53f4f8d
--- /dev/null
+++ b/Documentation/DocBook/regulator.tmpl
@@ -0,0 +1,304 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN"
+ "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd" []>
+
+<book id="regulator-api">
+ <bookinfo>
+ <title>Voltage and current regulator API</title>
+
+ <authorgroup>
+ <author>
+ <firstname>Liam</firstname>
+ <surname>Girdwood</surname>
+ <affiliation>
+ <address>
+ <email>lrg@slimlogic.co.uk</email>
+ </address>
+ </affiliation>
+ </author>
+ <author>
+ <firstname>Mark</firstname>
+ <surname>Brown</surname>
+ <affiliation>
+ <orgname>Wolfson Microelectronics</orgname>
+ <address>
+ <email>broonie@opensource.wolfsonmicro.com</email>
+ </address>
+ </affiliation>
+ </author>
+ </authorgroup>
+
+ <copyright>
+ <year>2007-2008</year>
+ <holder>Wolfson Microelectronics</holder>
+ </copyright>
+ <copyright>
+ <year>2008</year>
+ <holder>Liam Girdwood</holder>
+ </copyright>
+
+ <legalnotice>
+ <para>
+ This documentation is free software; you can redistribute
+ it and/or modify it under the terms of the GNU General Public
+ License version 2 as published by the Free Software Foundation.
+ </para>
+
+ <para>
+ This program is distributed in the hope that it will be
+ useful, but WITHOUT ANY WARRANTY; without even the implied
+ warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+ See the GNU General Public License for more details.
+ </para>
+
+ <para>
+ You should have received a copy of the GNU General Public
+ License along with this program; if not, write to the Free
+ Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,
+ MA 02111-1307 USA
+ </para>
+
+ <para>
+ For more details see the file COPYING in the source
+ distribution of Linux.
+ </para>
+ </legalnotice>
+ </bookinfo>
+
+<toc></toc>
+
+ <chapter id="intro">
+ <title>Introduction</title>
+ <para>
+ This framework is designed to provide a standard kernel
+ interface to control voltage and current regulators.
+ </para>
+ <para>
+ The intention is to allow systems to dynamically control
+ regulator power output in order to save power and prolong
+ battery life. This applies to both voltage regulators (where
+ voltage output is controllable) and current sinks (where current
+ limit is controllable).
+ </para>
+ <para>
+ Note that additional (and currently more complete) documentation
+ is available in the Linux kernel source under
+ <filename>Documentation/power/regulator</filename>.
+ </para>
+
+ <sect1 id="glossary">
+ <title>Glossary</title>
+ <para>
+ The regulator API uses a number of terms which may not be
+ familiar:
+ </para>
+ <glossary>
+
+ <glossentry>
+ <glossterm>Regulator</glossterm>
+ <glossdef>
+ <para>
+ Electronic device that supplies power to other devices. Most
+ regulators can enable and disable their output and some can also
+ control their output voltage or current.
+ </para>
+ </glossdef>
+ </glossentry>
+
+ <glossentry>
+ <glossterm>Consumer</glossterm>
+ <glossdef>
+ <para>
+ Electronic device which consumes power provided by a regulator.
+ These may either be static, requiring only a fixed supply, or
+ dynamic, requiring active management of the regulator at
+ runtime.
+ </para>
+ </glossdef>
+ </glossentry>
+
+ <glossentry>
+ <glossterm>Power Domain</glossterm>
+ <glossdef>
+ <para>
+ The electronic circuit supplied by a given regulator, including
+ the regulator and all consumer devices. The configuration of
+ the regulator is shared between all the components in the
+ circuit.
+ </para>
+ </glossdef>
+ </glossentry>
+
+ <glossentry>
+ <glossterm>Power Management Integrated Circuit</glossterm>
+ <acronym>PMIC</acronym>
+ <glossdef>
+ <para>
+ An IC which contains numerous regulators and often also other
+ subsystems. In an embedded system the primary PMIC is often
+ equivalent to a combination of the PSU and southbridge in a
+ desktop system.
+ </para>
+ </glossdef>
+ </glossentry>
+ </glossary>
+ </sect1>
+ </chapter>
+
+ <chapter id="consumer">
+ <title>Consumer driver interface</title>
+ <para>
+ This offers a similar API to the kernel clock framework.
+ Consumer drivers use <link
+ linkend='API-regulator-get'>get</link> and <link
+ linkend='API-regulator-put'>put</link> operations to acquire and
+ release regulators. Functions are
+ provided to <link linkend='API-regulator-enable'>enable</link>
+ and <link linkend='API-regulator-disable'>disable</link> the
+ reguator and to get and set the runtime parameters of the
+ regulator.
+ </para>
+ <para>
+ When requesting regulators consumers use symbolic names for their
+ supplies, such as "Vcc", which are mapped into actual regulator
+ devices by the machine interface.
+ </para>
+ <para>
+ A stub version of this API is provided when the regulator
+ framework is not in use in order to minimise the need to use
+ ifdefs.
+ </para>
+
+ <sect1 id="consumer-enable">
+ <title>Enabling and disabling</title>
+ <para>
+ The regulator API provides reference counted enabling and
+ disabling of regulators. Consumer devices use the <function><link
+ linkend='API-regulator-enable'>regulator_enable</link></function>
+ and <function><link
+ linkend='API-regulator-disable'>regulator_disable</link>
+ </function> functions to enable and disable regulators. Calls
+ to the two functions must be balanced.
+ </para>
+ <para>
+ Note that since multiple consumers may be using a regulator and
+ machine constraints may not allow the regulator to be disabled
+ there is no guarantee that calling
+ <function>regulator_disable</function> will actually cause the
+ supply provided by the regulator to be disabled. Consumer
+ drivers should assume that the regulator may be enabled at all
+ times.
+ </para>
+ </sect1>
+
+ <sect1 id="consumer-config">
+ <title>Configuration</title>
+ <para>
+ Some consumer devices may need to be able to dynamically
+ configure their supplies. For example, MMC drivers may need to
+ select the correct operating voltage for their cards. This may
+ be done while the regulator is enabled or disabled.
+ </para>
+ <para>
+ The <function><link
+ linkend='API-regulator-set-voltage'>regulator_set_voltage</link>
+ </function> and <function><link
+ linkend='API-regulator-set-current-limit'
+ >regulator_set_current_limit</link>
+ </function> functions provide the primary interface for this.
+ Both take ranges of voltages and currents, supporting drivers
+ that do not require a specific value (eg, CPU frequency scaling
+ normally permits the CPU to use a wider range of supply
+ voltages at lower frequencies but does not require that the
+ supply voltage be lowered). Where an exact value is required
+ both minimum and maximum values should be identical.
+ </para>
+ </sect1>
+
+ <sect1 id="consumer-callback">
+ <title>Callbacks</title>
+ <para>
+ Callbacks may also be <link
+ linkend='API-regulator-register-notifier'>registered</link>
+ for events such as regulation failures.
+ </para>
+ </sect1>
+ </chapter>
+
+ <chapter id="driver">
+ <title>Regulator driver interface</title>
+ <para>
+ Drivers for regulator chips <link
+ linkend='API-regulator-register'>register</link> the regulators
+ with the regulator core, providing operations structures to the
+ core. A <link
+ linkend='API-regulator-notifier-call-chain'>notifier</link> interface
+ allows error conditions to be reported to the core.
+ </para>
+ <para>
+ Registration should be triggered by explicit setup done by the
+ platform, supplying a <link
+ linkend='API-struct-regulator-init-data'>struct
+ regulator_init_data</link> for the regulator containing
+ <link linkend='machine-constraint'>constraint</link> and
+ <link linkend='machine-supply'>supply</link> information.
+ </para>
+ </chapter>
+
+ <chapter id="machine">
+ <title>Machine interface</title>
+ <para>
+ This interface provides a way to define how regulators are
+ connected to consumers on a given system and what the valid
+ operating parameters are for the system.
+ </para>
+
+ <sect1 id="machine-supply">
+ <title>Supplies</title>
+ <para>
+ Regulator supplies are specified using <link
+ linkend='API-struct-regulator-consumer-supply'>struct
+ regulator_consumer_supply</link>. This is done at
+ <link linkend='driver'>driver registration
+ time</link> as part of the machine constraints.
+ </para>
+ </sect1>
+
+ <sect1 id="machine-constraint">
+ <title>Constraints</title>
+ <para>
+ As well as definining the connections the machine interface
+ also provides constraints definining the operations that
+ clients are allowed to perform and the parameters that may be
+ set. This is required since generally regulator devices will
+ offer more flexibility than it is safe to use on a given
+ system, for example supporting higher supply voltages than the
+ consumers are rated for.
+ </para>
+ <para>
+ This is done at <link linkend='driver'>driver
+ registration time</link> by providing a <link
+ linkend='API-struct-regulation-constraints'>struct
+ regulation_constraints</link>.
+ </para>
+ <para>
+ The constraints may also specify an initial configuration for the
+ regulator in the constraints, which is particularly useful for
+ use with static consumers.
+ </para>
+ </sect1>
+ </chapter>
+
+ <chapter id="api">
+ <title>API reference</title>
+ <para>
+ Due to limitations of the kernel documentation framework and the
+ existing layout of the source code the entire regulator API is
+ documented here.
+ </para>
+!Iinclude/linux/regulator/consumer.h
+!Iinclude/linux/regulator/machine.h
+!Iinclude/linux/regulator/driver.h
+!Edrivers/regulator/core.c
+ </chapter>
+</book>
diff --git a/Documentation/RCU/00-INDEX b/Documentation/RCU/00-INDEX
index 7dc0695..9bb62f7 100644
--- a/Documentation/RCU/00-INDEX
+++ b/Documentation/RCU/00-INDEX
@@ -12,6 +12,8 @@
- Reference-count design for elements of lists/arrays protected by RCU
rcu.txt
- RCU Concepts
+rcubarrier.txt
+ - Unloading modules that use RCU callbacks
RTFP.txt
- List of RCU papers (bibliography) going back to 1980.
torture.txt
diff --git a/Documentation/RCU/rcubarrier.txt b/Documentation/RCU/rcubarrier.txt
new file mode 100644
index 0000000..909602d
--- /dev/null
+++ b/Documentation/RCU/rcubarrier.txt
@@ -0,0 +1,304 @@
+RCU and Unloadable Modules
+
+[Originally published in LWN Jan. 14, 2007: http://lwn.net/Articles/217484/]
+
+RCU (read-copy update) is a synchronization mechanism that can be thought
+of as a replacement for read-writer locking (among other things), but with
+very low-overhead readers that are immune to deadlock, priority inversion,
+and unbounded latency. RCU read-side critical sections are delimited
+by rcu_read_lock() and rcu_read_unlock(), which, in non-CONFIG_PREEMPT
+kernels, generate no code whatsoever.
+
+This means that RCU writers are unaware of the presence of concurrent
+readers, so that RCU updates to shared data must be undertaken quite
+carefully, leaving an old version of the data structure in place until all
+pre-existing readers have finished. These old versions are needed because
+such readers might hold a reference to them. RCU updates can therefore be
+rather expensive, and RCU is thus best suited for read-mostly situations.
+
+How can an RCU writer possibly determine when all readers are finished,
+given that readers might well leave absolutely no trace of their
+presence? There is a synchronize_rcu() primitive that blocks until all
+pre-existing readers have completed. An updater wishing to delete an
+element p from a linked list might do the following, while holding an
+appropriate lock, of course:
+
+ list_del_rcu(p);
+ synchronize_rcu();
+ kfree(p);
+
+But the above code cannot be used in IRQ context -- the call_rcu()
+primitive must be used instead. This primitive takes a pointer to an
+rcu_head struct placed within the RCU-protected data structure and
+another pointer to a function that may be invoked later to free that
+structure. Code to delete an element p from the linked list from IRQ
+context might then be as follows:
+
+ list_del_rcu(p);
+ call_rcu(&p->rcu, p_callback);
+
+Since call_rcu() never blocks, this code can safely be used from within
+IRQ context. The function p_callback() might be defined as follows:
+
+ static void p_callback(struct rcu_head *rp)
+ {
+ struct pstruct *p = container_of(rp, struct pstruct, rcu);
+
+ kfree(p);
+ }
+
+
+Unloading Modules That Use call_rcu()
+
+But what if p_callback is defined in an unloadable module?
+
+If we unload the module while some RCU callbacks are pending,
+the CPUs executing these callbacks are going to be severely
+disappointed when they are later invoked, as fancifully depicted at
+http://lwn.net/images/ns/kernel/rcu-drop.jpg.
+
+We could try placing a synchronize_rcu() in the module-exit code path,
+but this is not sufficient. Although synchronize_rcu() does wait for a
+grace period to elapse, it does not wait for the callbacks to complete.
+
+One might be tempted to try several back-to-back synchronize_rcu()
+calls, but this is still not guaranteed to work. If there is a very
+heavy RCU-callback load, then some of the callbacks might be deferred
+in order to allow other processing to proceed. Such deferral is required
+in realtime kernels in order to avoid excessive scheduling latencies.
+
+
+rcu_barrier()
+
+We instead need the rcu_barrier() primitive. This primitive is similar
+to synchronize_rcu(), but instead of waiting solely for a grace
+period to elapse, it also waits for all outstanding RCU callbacks to
+complete. Pseudo-code using rcu_barrier() is as follows:
+
+ 1. Prevent any new RCU callbacks from being posted.
+ 2. Execute rcu_barrier().
+ 3. Allow the module to be unloaded.
+
+Quick Quiz #1: Why is there no srcu_barrier()?
+
+The rcutorture module makes use of rcu_barrier in its exit function
+as follows:
+
+ 1 static void
+ 2 rcu_torture_cleanup(void)
+ 3 {
+ 4 int i;
+ 5
+ 6 fullstop = 1;
+ 7 if (shuffler_task != NULL) {
+ 8 VERBOSE_PRINTK_STRING("Stopping rcu_torture_shuffle task");
+ 9 kthread_stop(shuffler_task);
+10 }
+11 shuffler_task = NULL;
+12
+13 if (writer_task != NULL) {
+14 VERBOSE_PRINTK_STRING("Stopping rcu_torture_writer task");
+15 kthread_stop(writer_task);
+16 }
+17 writer_task = NULL;
+18
+19 if (reader_tasks != NULL) {
+20 for (i = 0; i < nrealreaders; i++) {
+21 if (reader_tasks[i] != NULL) {
+22 VERBOSE_PRINTK_STRING(
+23 "Stopping rcu_torture_reader task");
+24 kthread_stop(reader_tasks[i]);
+25 }
+26 reader_tasks[i] = NULL;
+27 }
+28 kfree(reader_tasks);
+29 reader_tasks = NULL;
+30 }
+31 rcu_torture_current = NULL;
+32
+33 if (fakewriter_tasks != NULL) {
+34 for (i = 0; i < nfakewriters; i++) {
+35 if (fakewriter_tasks[i] != NULL) {
+36 VERBOSE_PRINTK_STRING(
+37 "Stopping rcu_torture_fakewriter task");
+38 kthread_stop(fakewriter_tasks[i]);
+39 }
+40 fakewriter_tasks[i] = NULL;
+41 }
+42 kfree(fakewriter_tasks);
+43 fakewriter_tasks = NULL;
+44 }
+45
+46 if (stats_task != NULL) {
+47 VERBOSE_PRINTK_STRING("Stopping rcu_torture_stats task");
+48 kthread_stop(stats_task);
+49 }
+50 stats_task = NULL;
+51
+52 /* Wait for all RCU callbacks to fire. */
+53 rcu_barrier();
+54
+55 rcu_torture_stats_print(); /* -After- the stats thread is stopped! */
+56
+57 if (cur_ops->cleanup != NULL)
+58 cur_ops->cleanup();
+59 if (atomic_read(&n_rcu_torture_error))
+60 rcu_torture_print_module_parms("End of test: FAILURE");
+61 else
+62 rcu_torture_print_module_parms("End of test: SUCCESS");
+63 }
+
+Line 6 sets a global variable that prevents any RCU callbacks from
+re-posting themselves. This will not be necessary in most cases, since
+RCU callbacks rarely include calls to call_rcu(). However, the rcutorture
+module is an exception to this rule, and therefore needs to set this
+global variable.
+
+Lines 7-50 stop all the kernel tasks associated with the rcutorture
+module. Therefore, once execution reaches line 53, no more rcutorture
+RCU callbacks will be posted. The rcu_barrier() call on line 53 waits
+for any pre-existing callbacks to complete.
+
+Then lines 55-62 print status and do operation-specific cleanup, and
+then return, permitting the module-unload operation to be completed.
+
+Quick Quiz #2: Is there any other situation where rcu_barrier() might
+ be required?
+
+Your module might have additional complications. For example, if your
+module invokes call_rcu() from timers, you will need to first cancel all
+the timers, and only then invoke rcu_barrier() to wait for any remaining
+RCU callbacks to complete.
+
+
+Implementing rcu_barrier()
+
+Dipankar Sarma's implementation of rcu_barrier() makes use of the fact
+that RCU callbacks are never reordered once queued on one of the per-CPU
+queues. His implementation queues an RCU callback on each of the per-CPU
+callback queues, and then waits until they have all started executing, at
+which point, all earlier RCU callbacks are guaranteed to have completed.
+
+The original code for rcu_barrier() was as follows:
+
+ 1 void rcu_barrier(void)
+ 2 {
+ 3 BUG_ON(in_interrupt());
+ 4 /* Take cpucontrol mutex to protect against CPU hotplug */
+ 5 mutex_lock(&rcu_barrier_mutex);
+ 6 init_completion(&rcu_barrier_completion);
+ 7 atomic_set(&rcu_barrier_cpu_count, 0);
+ 8 on_each_cpu(rcu_barrier_func, NULL, 0, 1);
+ 9 wait_for_completion(&rcu_barrier_completion);
+10 mutex_unlock(&rcu_barrier_mutex);
+11 }
+
+Line 3 verifies that the caller is in process context, and lines 5 and 10
+use rcu_barrier_mutex to ensure that only one rcu_barrier() is using the
+global completion and counters at a time, which are initialized on lines
+6 and 7. Line 8 causes each CPU to invoke rcu_barrier_func(), which is
+shown below. Note that the final "1" in on_each_cpu()'s argument list
+ensures that all the calls to rcu_barrier_func() will have completed
+before on_each_cpu() returns. Line 9 then waits for the completion.
+
+This code was rewritten in 2008 to support rcu_barrier_bh() and
+rcu_barrier_sched() in addition to the original rcu_barrier().
+
+The rcu_barrier_func() runs on each CPU, where it invokes call_rcu()
+to post an RCU callback, as follows:
+
+ 1 static void rcu_barrier_func(void *notused)
+ 2 {
+ 3 int cpu = smp_processor_id();
+ 4 struct rcu_data *rdp = &per_cpu(rcu_data, cpu);
+ 5 struct rcu_head *head;
+ 6
+ 7 head = &rdp->barrier;
+ 8 atomic_inc(&rcu_barrier_cpu_count);
+ 9 call_rcu(head, rcu_barrier_callback);
+10 }
+
+Lines 3 and 4 locate RCU's internal per-CPU rcu_data structure,
+which contains the struct rcu_head that needed for the later call to
+call_rcu(). Line 7 picks up a pointer to this struct rcu_head, and line
+8 increments a global counter. This counter will later be decremented
+by the callback. Line 9 then registers the rcu_barrier_callback() on
+the current CPU's queue.
+
+The rcu_barrier_callback() function simply atomically decrements the
+rcu_barrier_cpu_count variable and finalizes the completion when it
+reaches zero, as follows:
+
+ 1 static void rcu_barrier_callback(struct rcu_head *notused)
+ 2 {
+ 3 if (atomic_dec_and_test(&rcu_barrier_cpu_count))
+ 4 complete(&rcu_barrier_completion);
+ 5 }
+
+Quick Quiz #3: What happens if CPU 0's rcu_barrier_func() executes
+ immediately (thus incrementing rcu_barrier_cpu_count to the
+ value one), but the other CPU's rcu_barrier_func() invocations
+ are delayed for a full grace period? Couldn't this result in
+ rcu_barrier() returning prematurely?
+
+
+rcu_barrier() Summary
+
+The rcu_barrier() primitive has seen relatively little use, since most
+code using RCU is in the core kernel rather than in modules. However, if
+you are using RCU from an unloadable module, you need to use rcu_barrier()
+so that your module may be safely unloaded.
+
+
+Answers to Quick Quizzes
+
+Quick Quiz #1: Why is there no srcu_barrier()?
+
+Answer: Since there is no call_srcu(), there can be no outstanding SRCU
+ callbacks. Therefore, there is no need to wait for them.
+
+Quick Quiz #2: Is there any other situation where rcu_barrier() might
+ be required?
+
+Answer: Interestingly enough, rcu_barrier() was not originally
+ implemented for module unloading. Nikita Danilov was using
+ RCU in a filesystem, which resulted in a similar situation at
+ filesystem-unmount time. Dipankar Sarma coded up rcu_barrier()
+ in response, so that Nikita could invoke it during the
+ filesystem-unmount process.
+
+ Much later, yours truly hit the RCU module-unload problem when
+ implementing rcutorture, and found that rcu_barrier() solves
+ this problem as well.
+
+Quick Quiz #3: What happens if CPU 0's rcu_barrier_func() executes
+ immediately (thus incrementing rcu_barrier_cpu_count to the
+ value one), but the other CPU's rcu_barrier_func() invocations
+ are delayed for a full grace period? Couldn't this result in
+ rcu_barrier() returning prematurely?
+
+Answer: This cannot happen. The reason is that on_each_cpu() has its last
+ argument, the wait flag, set to "1". This flag is passed through
+ to smp_call_function() and further to smp_call_function_on_cpu(),
+ causing this latter to spin until the cross-CPU invocation of
+ rcu_barrier_func() has completed. This by itself would prevent
+ a grace period from completing on non-CONFIG_PREEMPT kernels,
+ since each CPU must undergo a context switch (or other quiescent
+ state) before the grace period can complete. However, this is
+ of no use in CONFIG_PREEMPT kernels.
+
+ Therefore, on_each_cpu() disables preemption across its call
+ to smp_call_function() and also across the local call to
+ rcu_barrier_func(). This prevents the local CPU from context
+ switching, again preventing grace periods from completing. This
+ means that all CPUs have executed rcu_barrier_func() before
+ the first rcu_barrier_callback() can possibly execute, in turn
+ preventing rcu_barrier_cpu_count from prematurely reaching zero.
+
+ Currently, -rt implementations of RCU keep but a single global
+ queue for RCU callbacks, and thus do not suffer from this
+ problem. However, when the -rt RCU eventually does have per-CPU
+ callback queues, things will have to change. One simple change
+ is to add an rcu_read_lock() before line 8 of rcu_barrier()
+ and an rcu_read_unlock() after line 8 of this same function. If
+ you can think of a better change, please let me know!
diff --git a/Documentation/bad_memory.txt b/Documentation/bad_memory.txt
new file mode 100644
index 0000000..df84162
--- /dev/null
+++ b/Documentation/bad_memory.txt
@@ -0,0 +1,45 @@
+March 2008
+Jan-Simon Moeller, dl9pf@gmx.de
+
+
+How to deal with bad memory e.g. reported by memtest86+ ?
+#########################################################
+
+There are three possibilities I know of:
+
+1) Reinsert/swap the memory modules
+
+2) Buy new modules (best!) or try to exchange the memory
+ if you have spare-parts
+
+3) Use BadRAM or memmap
+
+This Howto is about number 3) .
+
+
+BadRAM
+######
+BadRAM is the actively developed and available as kernel-patch
+here: http://rick.vanrein.org/linux/badram/
+
+For more details see the BadRAM documentation.
+
+memmap
+######
+
+memmap is already in the kernel and usable as kernel-parameter at
+boot-time. Its syntax is slightly strange and you may need to
+calculate the values by yourself!
+
+Syntax to exclude a memory area (see kernel-parameters.txt for details):
+memmap=<size>$<address>
+
+Example: memtest86+ reported here errors at address 0x18691458, 0x18698424 and
+ some others. All had 0x1869xxxx in common, so I chose a pattern of
+ 0x18690000,0xffff0000.
+
+With the numbers of the example above:
+memmap=64K$0x18690000
+ or
+memmap=0x10000$0x18690000
+
diff --git a/Documentation/cgroups/cgroups.txt b/Documentation/cgroups/cgroups.txt
index d9014aa..e33ee74 100644
--- a/Documentation/cgroups/cgroups.txt
+++ b/Documentation/cgroups/cgroups.txt
@@ -227,7 +227,6 @@
containing the following files describing that cgroup:
- tasks: list of tasks (by pid) attached to that cgroup
- - releasable flag: cgroup currently removeable?
- notify_on_release flag: run the release agent on exit?
- release_agent: the path to use for release notifications (this file
exists in the top cgroup only)
@@ -360,7 +359,7 @@
In this directory you can find several files:
# ls
-notify_on_release releasable tasks
+notify_on_release tasks
(plus whatever files added by the attached subsystems)
Now attach your shell to this cgroup:
@@ -479,7 +478,6 @@
create() method has been called for the new cgroup).
void pre_destroy(struct cgroup_subsys *ss, struct cgroup *cgrp);
-(cgroup_mutex held by caller)
Called before checking the reference count on each subsystem. This may
be useful for subsystems which have some extra references even if
@@ -498,6 +496,7 @@
void attach(struct cgroup_subsys *ss, struct cgroup *cgrp,
struct cgroup *old_cgrp, struct task_struct *task)
+(cgroup_mutex held by caller)
Called after the task has been attached to the cgroup, to allow any
post-attachment activity that requires memory allocations or blocking.
@@ -511,6 +510,7 @@
Called during task exit.
int populate(struct cgroup_subsys *ss, struct cgroup *cgrp)
+(cgroup_mutex held by caller)
Called after creation of a cgroup to allow a subsystem to populate
the cgroup directory with file entries. The subsystem should make
@@ -520,6 +520,7 @@
always handled well.
void post_clone(struct cgroup_subsys *ss, struct cgroup *cgrp)
+(cgroup_mutex held by caller)
Called at the end of cgroup_clone() to do any paramater
initialization which might be required before a task could attach. For
@@ -527,7 +528,7 @@
up.
void bind(struct cgroup_subsys *ss, struct cgroup *root)
-(cgroup_mutex held by caller)
+(cgroup_mutex and ss->hierarchy_mutex held by caller)
Called when a cgroup subsystem is rebound to a different hierarchy
and root cgroup. Currently this will only involve movement between
diff --git a/Documentation/controllers/memcg_test.txt b/Documentation/controllers/memcg_test.txt
new file mode 100644
index 0000000..08d4d3e
--- /dev/null
+++ b/Documentation/controllers/memcg_test.txt
@@ -0,0 +1,342 @@
+Memory Resource Controller(Memcg) Implementation Memo.
+Last Updated: 2008/12/15
+Base Kernel Version: based on 2.6.28-rc8-mm.
+
+Because VM is getting complex (one of reasons is memcg...), memcg's behavior
+is complex. This is a document for memcg's internal behavior.
+Please note that implementation details can be changed.
+
+(*) Topics on API should be in Documentation/controllers/memory.txt)
+
+0. How to record usage ?
+ 2 objects are used.
+
+ page_cgroup ....an object per page.
+ Allocated at boot or memory hotplug. Freed at memory hot removal.
+
+ swap_cgroup ... an entry per swp_entry.
+ Allocated at swapon(). Freed at swapoff().
+
+ The page_cgroup has USED bit and double count against a page_cgroup never
+ occurs. swap_cgroup is used only when a charged page is swapped-out.
+
+1. Charge
+
+ a page/swp_entry may be charged (usage += PAGE_SIZE) at
+
+ mem_cgroup_newpage_charge()
+ Called at new page fault and Copy-On-Write.
+
+ mem_cgroup_try_charge_swapin()
+ Called at do_swap_page() (page fault on swap entry) and swapoff.
+ Followed by charge-commit-cancel protocol. (With swap accounting)
+ At commit, a charge recorded in swap_cgroup is removed.
+
+ mem_cgroup_cache_charge()
+ Called at add_to_page_cache()
+
+ mem_cgroup_cache_charge_swapin()
+ Called at shmem's swapin.
+
+ mem_cgroup_prepare_migration()
+ Called before migration. "extra" charge is done and followed by
+ charge-commit-cancel protocol.
+ At commit, charge against oldpage or newpage will be committed.
+
+2. Uncharge
+ a page/swp_entry may be uncharged (usage -= PAGE_SIZE) by
+
+ mem_cgroup_uncharge_page()
+ Called when an anonymous page is fully unmapped. I.e., mapcount goes
+ to 0. If the page is SwapCache, uncharge is delayed until
+ mem_cgroup_uncharge_swapcache().
+
+ mem_cgroup_uncharge_cache_page()
+ Called when a page-cache is deleted from radix-tree. If the page is
+ SwapCache, uncharge is delayed until mem_cgroup_uncharge_swapcache().
+
+ mem_cgroup_uncharge_swapcache()
+ Called when SwapCache is removed from radix-tree. The charge itself
+ is moved to swap_cgroup. (If mem+swap controller is disabled, no
+ charge to swap occurs.)
+
+ mem_cgroup_uncharge_swap()
+ Called when swp_entry's refcnt goes down to 0. A charge against swap
+ disappears.
+
+ mem_cgroup_end_migration(old, new)
+ At success of migration old is uncharged (if necessary), a charge
+ to new page is committed. At failure, charge to old page is committed.
+
+3. charge-commit-cancel
+ In some case, we can't know this "charge" is valid or not at charging
+ (because of races).
+ To handle such case, there are charge-commit-cancel functions.
+ mem_cgroup_try_charge_XXX
+ mem_cgroup_commit_charge_XXX
+ mem_cgroup_cancel_charge_XXX
+ these are used in swap-in and migration.
+
+ At try_charge(), there are no flags to say "this page is charged".
+ at this point, usage += PAGE_SIZE.
+
+ At commit(), the function checks the page should be charged or not
+ and set flags or avoid charging.(usage -= PAGE_SIZE)
+
+ At cancel(), simply usage -= PAGE_SIZE.
+
+Under below explanation, we assume CONFIG_MEM_RES_CTRL_SWAP=y.
+
+4. Anonymous
+ Anonymous page is newly allocated at
+ - page fault into MAP_ANONYMOUS mapping.
+ - Copy-On-Write.
+ It is charged right after it's allocated before doing any page table
+ related operations. Of course, it's uncharged when another page is used
+ for the fault address.
+
+ At freeing anonymous page (by exit() or munmap()), zap_pte() is called
+ and pages for ptes are freed one by one.(see mm/memory.c). Uncharges
+ are done at page_remove_rmap() when page_mapcount() goes down to 0.
+
+ Another page freeing is by page-reclaim (vmscan.c) and anonymous
+ pages are swapped out. In this case, the page is marked as
+ PageSwapCache(). uncharge() routine doesn't uncharge the page marked
+ as SwapCache(). It's delayed until __delete_from_swap_cache().
+
+ 4.1 Swap-in.
+ At swap-in, the page is taken from swap-cache. There are 2 cases.
+
+ (a) If the SwapCache is newly allocated and read, it has no charges.
+ (b) If the SwapCache has been mapped by processes, it has been
+ charged already.
+
+ This swap-in is one of the most complicated work. In do_swap_page(),
+ following events occur when pte is unchanged.
+
+ (1) the page (SwapCache) is looked up.
+ (2) lock_page()
+ (3) try_charge_swapin()
+ (4) reuse_swap_page() (may call delete_swap_cache())
+ (5) commit_charge_swapin()
+ (6) swap_free().
+
+ Considering following situation for example.
+
+ (A) The page has not been charged before (2) and reuse_swap_page()
+ doesn't call delete_from_swap_cache().
+ (B) The page has not been charged before (2) and reuse_swap_page()
+ calls delete_from_swap_cache().
+ (C) The page has been charged before (2) and reuse_swap_page() doesn't
+ call delete_from_swap_cache().
+ (D) The page has been charged before (2) and reuse_swap_page() calls
+ delete_from_swap_cache().
+
+ memory.usage/memsw.usage changes to this page/swp_entry will be
+ Case (A) (B) (C) (D)
+ Event
+ Before (2) 0/ 1 0/ 1 1/ 1 1/ 1
+ ===========================================
+ (3) +1/+1 +1/+1 +1/+1 +1/+1
+ (4) - 0/ 0 - -1/ 0
+ (5) 0/-1 0/ 0 -1/-1 0/ 0
+ (6) - 0/-1 - 0/-1
+ ===========================================
+ Result 1/ 1 1/ 1 1/ 1 1/ 1
+
+ In any cases, charges to this page should be 1/ 1.
+
+ 4.2 Swap-out.
+ At swap-out, typical state transition is below.
+
+ (a) add to swap cache. (marked as SwapCache)
+ swp_entry's refcnt += 1.
+ (b) fully unmapped.
+ swp_entry's refcnt += # of ptes.
+ (c) write back to swap.
+ (d) delete from swap cache. (remove from SwapCache)
+ swp_entry's refcnt -= 1.
+
+
+ At (b), the page is marked as SwapCache and not uncharged.
+ At (d), the page is removed from SwapCache and a charge in page_cgroup
+ is moved to swap_cgroup.
+
+ Finally, at task exit,
+ (e) zap_pte() is called and swp_entry's refcnt -=1 -> 0.
+ Here, a charge in swap_cgroup disappears.
+
+5. Page Cache
+ Page Cache is charged at
+ - add_to_page_cache_locked().
+
+ uncharged at
+ - __remove_from_page_cache().
+
+ The logic is very clear. (About migration, see below)
+ Note: __remove_from_page_cache() is called by remove_from_page_cache()
+ and __remove_mapping().
+
+6. Shmem(tmpfs) Page Cache
+ Memcg's charge/uncharge have special handlers of shmem. The best way
+ to understand shmem's page state transition is to read mm/shmem.c.
+ But brief explanation of the behavior of memcg around shmem will be
+ helpful to understand the logic.
+
+ Shmem's page (just leaf page, not direct/indirect block) can be on
+ - radix-tree of shmem's inode.
+ - SwapCache.
+ - Both on radix-tree and SwapCache. This happens at swap-in
+ and swap-out,
+
+ It's charged when...
+ - A new page is added to shmem's radix-tree.
+ - A swp page is read. (move a charge from swap_cgroup to page_cgroup)
+ It's uncharged when
+ - A page is removed from radix-tree and not SwapCache.
+ - When SwapCache is removed, a charge is moved to swap_cgroup.
+ - When swp_entry's refcnt goes down to 0, a charge in swap_cgroup
+ disappears.
+
+7. Page Migration
+ One of the most complicated functions is page-migration-handler.
+ Memcg has 2 routines. Assume that we are migrating a page's contents
+ from OLDPAGE to NEWPAGE.
+
+ Usual migration logic is..
+ (a) remove the page from LRU.
+ (b) allocate NEWPAGE (migration target)
+ (c) lock by lock_page().
+ (d) unmap all mappings.
+ (e-1) If necessary, replace entry in radix-tree.
+ (e-2) move contents of a page.
+ (f) map all mappings again.
+ (g) pushback the page to LRU.
+ (-) OLDPAGE will be freed.
+
+ Before (g), memcg should complete all necessary charge/uncharge to
+ NEWPAGE/OLDPAGE.
+
+ The point is....
+ - If OLDPAGE is anonymous, all charges will be dropped at (d) because
+ try_to_unmap() drops all mapcount and the page will not be
+ SwapCache.
+
+ - If OLDPAGE is SwapCache, charges will be kept at (g) because
+ __delete_from_swap_cache() isn't called at (e-1)
+
+ - If OLDPAGE is page-cache, charges will be kept at (g) because
+ remove_from_swap_cache() isn't called at (e-1)
+
+ memcg provides following hooks.
+
+ - mem_cgroup_prepare_migration(OLDPAGE)
+ Called after (b) to account a charge (usage += PAGE_SIZE) against
+ memcg which OLDPAGE belongs to.
+
+ - mem_cgroup_end_migration(OLDPAGE, NEWPAGE)
+ Called after (f) before (g).
+ If OLDPAGE is used, commit OLDPAGE again. If OLDPAGE is already
+ charged, a charge by prepare_migration() is automatically canceled.
+ If NEWPAGE is used, commit NEWPAGE and uncharge OLDPAGE.
+
+ But zap_pte() (by exit or munmap) can be called while migration,
+ we have to check if OLDPAGE/NEWPAGE is a valid page after commit().
+
+8. LRU
+ Each memcg has its own private LRU. Now, it's handling is under global
+ VM's control (means that it's handled under global zone->lru_lock).
+ Almost all routines around memcg's LRU is called by global LRU's
+ list management functions under zone->lru_lock().
+
+ A special function is mem_cgroup_isolate_pages(). This scans
+ memcg's private LRU and call __isolate_lru_page() to extract a page
+ from LRU.
+ (By __isolate_lru_page(), the page is removed from both of global and
+ private LRU.)
+
+
+9. Typical Tests.
+
+ Tests for racy cases.
+
+ 9.1 Small limit to memcg.
+ When you do test to do racy case, it's good test to set memcg's limit
+ to be very small rather than GB. Many races found in the test under
+ xKB or xxMB limits.
+ (Memory behavior under GB and Memory behavior under MB shows very
+ different situation.)
+
+ 9.2 Shmem
+ Historically, memcg's shmem handling was poor and we saw some amount
+ of troubles here. This is because shmem is page-cache but can be
+ SwapCache. Test with shmem/tmpfs is always good test.
+
+ 9.3 Migration
+ For NUMA, migration is an another special case. To do easy test, cpuset
+ is useful. Following is a sample script to do migration.
+
+ mount -t cgroup -o cpuset none /opt/cpuset
+
+ mkdir /opt/cpuset/01
+ echo 1 > /opt/cpuset/01/cpuset.cpus
+ echo 0 > /opt/cpuset/01/cpuset.mems
+ echo 1 > /opt/cpuset/01/cpuset.memory_migrate
+ mkdir /opt/cpuset/02
+ echo 1 > /opt/cpuset/02/cpuset.cpus
+ echo 1 > /opt/cpuset/02/cpuset.mems
+ echo 1 > /opt/cpuset/02/cpuset.memory_migrate
+
+ In above set, when you moves a task from 01 to 02, page migration to
+ node 0 to node 1 will occur. Following is a script to migrate all
+ under cpuset.
+ --
+ move_task()
+ {
+ for pid in $1
+ do
+ /bin/echo $pid >$2/tasks 2>/dev/null
+ echo -n $pid
+ echo -n " "
+ done
+ echo END
+ }
+
+ G1_TASK=`cat ${G1}/tasks`
+ G2_TASK=`cat ${G2}/tasks`
+ move_task "${G1_TASK}" ${G2} &
+ --
+ 9.4 Memory hotplug.
+ memory hotplug test is one of good test.
+ to offline memory, do following.
+ # echo offline > /sys/devices/system/memory/memoryXXX/state
+ (XXX is the place of memory)
+ This is an easy way to test page migration, too.
+
+ 9.5 mkdir/rmdir
+ When using hierarchy, mkdir/rmdir test should be done.
+ Use tests like the following.
+
+ echo 1 >/opt/cgroup/01/memory/use_hierarchy
+ mkdir /opt/cgroup/01/child_a
+ mkdir /opt/cgroup/01/child_b
+
+ set limit to 01.
+ add limit to 01/child_b
+ run jobs under child_a and child_b
+
+ create/delete following groups at random while jobs are running.
+ /opt/cgroup/01/child_a/child_aa
+ /opt/cgroup/01/child_b/child_bb
+ /opt/cgroup/01/child_c
+
+ running new jobs in new group is also good.
+
+ 9.6 Mount with other subsystems.
+ Mounting with other subsystems is a good test because there is a
+ race and lock dependency with other cgroup subsystems.
+
+ example)
+ # mount -t cgroup none /cgroup -t cpuset,memory,cpu,devices
+
+ and do task move, mkdir, rmdir etc...under this.
diff --git a/Documentation/controllers/memory.txt b/Documentation/controllers/memory.txt
index 1c07547..e150196 100644
--- a/Documentation/controllers/memory.txt
+++ b/Documentation/controllers/memory.txt
@@ -137,7 +137,32 @@
page will eventually get charged for it (once it is uncharged from
the cgroup that brought it in -- this will happen on memory pressure).
-2.4 Reclaim
+Exception: If CONFIG_CGROUP_CGROUP_MEM_RES_CTLR_SWAP is not used..
+When you do swapoff and make swapped-out pages of shmem(tmpfs) to
+be backed into memory in force, charges for pages are accounted against the
+caller of swapoff rather than the users of shmem.
+
+
+2.4 Swap Extension (CONFIG_CGROUP_MEM_RES_CTLR_SWAP)
+Swap Extension allows you to record charge for swap. A swapped-in page is
+charged back to original page allocator if possible.
+
+When swap is accounted, following files are added.
+ - memory.memsw.usage_in_bytes.
+ - memory.memsw.limit_in_bytes.
+
+usage of mem+swap is limited by memsw.limit_in_bytes.
+
+Note: why 'mem+swap' rather than swap.
+The global LRU(kswapd) can swap out arbitrary pages. Swap-out means
+to move account from memory to swap...there is no change in usage of
+mem+swap.
+
+In other words, when we want to limit the usage of swap without affecting
+global LRU, mem+swap limit is better than just limiting swap from OS point
+of view.
+
+2.5 Reclaim
Each cgroup maintains a per cgroup LRU that consists of an active
and inactive list. When a cgroup goes over its limit, we first try
@@ -207,12 +232,6 @@
The memory.stat file gives accounting information. Now, the number of
caches, RSS and Active pages/Inactive pages are shown.
-The memory.force_empty gives an interface to drop *all* charges by force.
-
-# echo 1 > memory.force_empty
-
-will drop all charges in cgroup. Currently, this is maintained for test.
-
4. Testing
Balbir posted lmbench, AIM9, LTP and vmmstress results [10] and [11].
@@ -242,10 +261,106 @@
A cgroup can be removed by rmdir, but as discussed in sections 4.1 and 4.2, a
cgroup might have some charge associated with it, even though all
-tasks have migrated away from it. Such charges are automatically dropped at
-rmdir() if there are no tasks.
+tasks have migrated away from it.
+Such charges are freed(at default) or moved to its parent. When moved,
+both of RSS and CACHES are moved to parent.
+If both of them are busy, rmdir() returns -EBUSY. See 5.1 Also.
-5. TODO
+Charges recorded in swap information is not updated at removal of cgroup.
+Recorded information is discarded and a cgroup which uses swap (swapcache)
+will be charged as a new owner of it.
+
+
+5. Misc. interfaces.
+
+5.1 force_empty
+ memory.force_empty interface is provided to make cgroup's memory usage empty.
+ You can use this interface only when the cgroup has no tasks.
+ When writing anything to this
+
+ # echo 0 > memory.force_empty
+
+ Almost all pages tracked by this memcg will be unmapped and freed. Some of
+ pages cannot be freed because it's locked or in-use. Such pages are moved
+ to parent and this cgroup will be empty. But this may return -EBUSY in
+ some too busy case.
+
+ Typical use case of this interface is that calling this before rmdir().
+ Because rmdir() moves all pages to parent, some out-of-use page caches can be
+ moved to the parent. If you want to avoid that, force_empty will be useful.
+
+5.2 stat file
+ memory.stat file includes following statistics (now)
+ cache - # of pages from page-cache and shmem.
+ rss - # of pages from anonymous memory.
+ pgpgin - # of event of charging
+ pgpgout - # of event of uncharging
+ active_anon - # of pages on active lru of anon, shmem.
+ inactive_anon - # of pages on active lru of anon, shmem
+ active_file - # of pages on active lru of file-cache
+ inactive_file - # of pages on inactive lru of file cache
+ unevictable - # of pages cannot be reclaimed.(mlocked etc)
+
+ Below is depend on CONFIG_DEBUG_VM.
+ inactive_ratio - VM inernal parameter. (see mm/page_alloc.c)
+ recent_rotated_anon - VM internal parameter. (see mm/vmscan.c)
+ recent_rotated_file - VM internal parameter. (see mm/vmscan.c)
+ recent_scanned_anon - VM internal parameter. (see mm/vmscan.c)
+ recent_scanned_file - VM internal parameter. (see mm/vmscan.c)
+
+ Memo:
+ recent_rotated means recent frequency of lru rotation.
+ recent_scanned means recent # of scans to lru.
+ showing for better debug please see the code for meanings.
+
+
+5.3 swappiness
+ Similar to /proc/sys/vm/swappiness, but affecting a hierarchy of groups only.
+
+ Following cgroup's swapiness can't be changed.
+ - root cgroup (uses /proc/sys/vm/swappiness).
+ - a cgroup which uses hierarchy and it has child cgroup.
+ - a cgroup which uses hierarchy and not the root of hierarchy.
+
+
+6. Hierarchy support
+
+The memory controller supports a deep hierarchy and hierarchical accounting.
+The hierarchy is created by creating the appropriate cgroups in the
+cgroup filesystem. Consider for example, the following cgroup filesystem
+hierarchy
+
+ root
+ / | \
+ / | \
+ a b c
+ | \
+ | \
+ d e
+
+In the diagram above, with hierarchical accounting enabled, all memory
+usage of e, is accounted to its ancestors up until the root (i.e, c and root),
+that has memory.use_hierarchy enabled. If one of the ancestors goes over its
+limit, the reclaim algorithm reclaims from the tasks in the ancestor and the
+children of the ancestor.
+
+6.1 Enabling hierarchical accounting and reclaim
+
+The memory controller by default disables the hierarchy feature. Support
+can be enabled by writing 1 to memory.use_hierarchy file of the root cgroup
+
+# echo 1 > memory.use_hierarchy
+
+The feature can be disabled by
+
+# echo 0 > memory.use_hierarchy
+
+NOTE1: Enabling/disabling will fail if the cgroup already has other
+cgroups created below it.
+
+NOTE2: This feature can be enabled/disabled per subtree.
+
+7. TODO
1. Add support for accounting huge pages (as a separate controller)
2. Make per-cgroup scanner reclaim not-shared pages first
diff --git a/Documentation/crypto/async-tx-api.txt b/Documentation/crypto/async-tx-api.txt
index c1e9545..9f59fcb 100644
--- a/Documentation/crypto/async-tx-api.txt
+++ b/Documentation/crypto/async-tx-api.txt
@@ -13,9 +13,9 @@
3.6 Constraints
3.7 Example
-4 DRIVER DEVELOPER NOTES
+4 DMAENGINE DRIVER DEVELOPER NOTES
4.1 Conformance points
-4.2 "My application needs finer control of hardware channels"
+4.2 "My application needs exclusive control of hardware channels"
5 SOURCE
@@ -150,6 +150,7 @@
implementation examples.
4 DRIVER DEVELOPMENT NOTES
+
4.1 Conformance points:
There are a few conformance points required in dmaengine drivers to
accommodate assumptions made by applications using the async_tx API:
@@ -158,58 +159,49 @@
3/ Use async_tx_run_dependencies() in the descriptor clean up path to
handle submission of dependent operations
-4.2 "My application needs finer control of hardware channels"
-This requirement seems to arise from cases where a DMA engine driver is
-trying to support device-to-memory DMA. The dmaengine and async_tx
-implementations were designed for offloading memory-to-memory
-operations; however, there are some capabilities of the dmaengine layer
-that can be used for platform-specific channel management.
-Platform-specific constraints can be handled by registering the
-application as a 'dma_client' and implementing a 'dma_event_callback' to
-apply a filter to the available channels in the system. Before showing
-how to implement a custom dma_event callback some background of
-dmaengine's client support is required.
+4.2 "My application needs exclusive control of hardware channels"
+Primarily this requirement arises from cases where a DMA engine driver
+is being used to support device-to-memory operations. A channel that is
+performing these operations cannot, for many platform specific reasons,
+be shared. For these cases the dma_request_channel() interface is
+provided.
-The following routines in dmaengine support multiple clients requesting
-use of a channel:
-- dma_async_client_register(struct dma_client *client)
-- dma_async_client_chan_request(struct dma_client *client)
+The interface is:
+struct dma_chan *dma_request_channel(dma_cap_mask_t mask,
+ dma_filter_fn filter_fn,
+ void *filter_param);
-dma_async_client_register takes a pointer to an initialized dma_client
-structure. It expects that the 'event_callback' and 'cap_mask' fields
-are already initialized.
+Where dma_filter_fn is defined as:
+typedef bool (*dma_filter_fn)(struct dma_chan *chan, void *filter_param);
-dma_async_client_chan_request triggers dmaengine to notify the client of
-all channels that satisfy the capability mask. It is up to the client's
-event_callback routine to track how many channels the client needs and
-how many it is currently using. The dma_event_callback routine returns a
-dma_state_client code to let dmaengine know the status of the
-allocation.
+When the optional 'filter_fn' parameter is set to NULL
+dma_request_channel simply returns the first channel that satisfies the
+capability mask. Otherwise, when the mask parameter is insufficient for
+specifying the necessary channel, the filter_fn routine can be used to
+disposition the available channels in the system. The filter_fn routine
+is called once for each free channel in the system. Upon seeing a
+suitable channel filter_fn returns DMA_ACK which flags that channel to
+be the return value from dma_request_channel. A channel allocated via
+this interface is exclusive to the caller, until dma_release_channel()
+is called.
-Below is the example of how to extend this functionality for
-platform-specific filtering of the available channels beyond the
-standard capability mask:
+The DMA_PRIVATE capability flag is used to tag dma devices that should
+not be used by the general-purpose allocator. It can be set at
+initialization time if it is known that a channel will always be
+private. Alternatively, it is set when dma_request_channel() finds an
+unused "public" channel.
-static enum dma_state_client
-my_dma_client_callback(struct dma_client *client,
- struct dma_chan *chan, enum dma_state state)
-{
- struct dma_device *dma_dev;
- struct my_platform_specific_dma *plat_dma_dev;
-
- dma_dev = chan->device;
- plat_dma_dev = container_of(dma_dev,
- struct my_platform_specific_dma,
- dma_dev);
-
- if (!plat_dma_dev->platform_specific_capability)
- return DMA_DUP;
-
- . . .
-}
+A couple caveats to note when implementing a driver and consumer:
+1/ Once a channel has been privately allocated it will no longer be
+ considered by the general-purpose allocator even after a call to
+ dma_release_channel().
+2/ Since capabilities are specified at the device level a dma_device
+ with multiple channels will either have all channels public, or all
+ channels private.
5 SOURCE
-include/linux/dmaengine.h: core header file for DMA drivers and clients
+
+include/linux/dmaengine.h: core header file for DMA drivers and api users
drivers/dma/dmaengine.c: offload engine channel management routines
drivers/dma/: location for offload engine drivers
include/linux/async_tx.h: core header file for the async_tx api
diff --git a/Documentation/development-process/4.Coding b/Documentation/development-process/4.Coding
index 014aca8..a5a3450 100644
--- a/Documentation/development-process/4.Coding
+++ b/Documentation/development-process/4.Coding
@@ -375,10 +375,10 @@
justification is solid.
When making an incompatible API change, one should, whenever possible,
-ensure that code which has not been updated is caught by the compiler.
+ensure that code which has not been updated is caught by the compiler.
This will help you to be sure that you have found all in-tree uses of that
interface. It will also alert developers of out-of-tree code that there is
a change that they need to respond to. Supporting out-of-tree code is not
something that kernel developers need to be worried about, but we also do
-not have to make life harder for out-of-tree developers than it it needs to
-be.
+not have to make life harder for out-of-tree developers than it needs to
+be.
diff --git a/Documentation/dmaengine.txt b/Documentation/dmaengine.txt
new file mode 100644
index 0000000..0c1c2f6
--- /dev/null
+++ b/Documentation/dmaengine.txt
@@ -0,0 +1 @@
+See Documentation/crypto/async-tx-api.txt
diff --git a/Documentation/filesystems/btrfs.txt b/Documentation/filesystems/btrfs.txt
new file mode 100644
index 0000000..64087c3
--- /dev/null
+++ b/Documentation/filesystems/btrfs.txt
@@ -0,0 +1,91 @@
+
+ BTRFS
+ =====
+
+Btrfs is a new copy on write filesystem for Linux aimed at
+implementing advanced features while focusing on fault tolerance,
+repair and easy administration. Initially developed by Oracle, Btrfs
+is licensed under the GPL and open for contribution from anyone.
+
+Linux has a wealth of filesystems to choose from, but we are facing a
+number of challenges with scaling to the large storage subsystems that
+are becoming common in today's data centers. Filesystems need to scale
+in their ability to address and manage large storage, and also in
+their ability to detect, repair and tolerate errors in the data stored
+on disk. Btrfs is under heavy development, and is not suitable for
+any uses other than benchmarking and review. The Btrfs disk format is
+not yet finalized.
+
+The main Btrfs features include:
+
+ * Extent based file storage (2^64 max file size)
+ * Space efficient packing of small files
+ * Space efficient indexed directories
+ * Dynamic inode allocation
+ * Writable snapshots
+ * Subvolumes (separate internal filesystem roots)
+ * Object level mirroring and striping
+ * Checksums on data and metadata (multiple algorithms available)
+ * Compression
+ * Integrated multiple device support, with several raid algorithms
+ * Online filesystem check (not yet implemented)
+ * Very fast offline filesystem check
+ * Efficient incremental backup and FS mirroring (not yet implemented)
+ * Online filesystem defragmentation
+
+
+
+ MAILING LIST
+ ============
+
+There is a Btrfs mailing list hosted on vger.kernel.org. You can
+find details on how to subscribe here:
+
+http://vger.kernel.org/vger-lists.html#linux-btrfs
+
+Mailing list archives are available from gmane:
+
+http://dir.gmane.org/gmane.comp.file-systems.btrfs
+
+
+
+ IRC
+ ===
+
+Discussion of Btrfs also occurs on the #btrfs channel of the Freenode
+IRC network.
+
+
+
+ UTILITIES
+ =========
+
+Userspace tools for creating and manipulating Btrfs file systems are
+available from the git repository at the following location:
+
+ http://git.kernel.org/?p=linux/kernel/git/mason/btrfs-progs-unstable.git
+ git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs-unstable.git
+
+These include the following tools:
+
+mkfs.btrfs: create a filesystem
+
+btrfsctl: control program to create snapshots and subvolumes:
+
+ mount /dev/sda2 /mnt
+ btrfsctl -s new_subvol_name /mnt
+ btrfsctl -s snapshot_of_default /mnt/default
+ btrfsctl -s snapshot_of_new_subvol /mnt/new_subvol_name
+ btrfsctl -s snapshot_of_a_snapshot /mnt/snapshot_of_new_subvol
+ ls /mnt
+ default snapshot_of_a_snapshot snapshot_of_new_subvol
+ new_subvol_name snapshot_of_default
+
+ Snapshots and subvolumes cannot be deleted right now, but you can
+ rm -rf all the files and directories inside them.
+
+btrfsck: do a limited check of the FS extent trees.
+
+btrfs-debug-tree: print all of the FS metadata in text form. Example:
+
+ btrfs-debug-tree /dev/sda2 >& big_output_file
diff --git a/Documentation/filesystems/ext4.txt b/Documentation/filesystems/ext4.txt
index 174eaff..cec829b 100644
--- a/Documentation/filesystems/ext4.txt
+++ b/Documentation/filesystems/ext4.txt
@@ -58,13 +58,22 @@
# mount -t ext4 /dev/hda1 /wherever
- - When comparing performance with other filesystems, remember that
- ext3/4 by default offers higher data integrity guarantees than most.
- So when comparing with a metadata-only journalling filesystem, such
- as ext3, use `mount -o data=writeback'. And you might as well use
- `mount -o nobh' too along with it. Making the journal larger than
- the mke2fs default often helps performance with metadata-intensive
- workloads.
+ - When comparing performance with other filesystems, it's always
+ important to try multiple workloads; very often a subtle change in a
+ workload parameter can completely change the ranking of which
+ filesystems do well compared to others. When comparing versus ext3,
+ note that ext4 enables write barriers by default, while ext3 does
+ not enable write barriers by default. So it is useful to use
+ explicitly specify whether barriers are enabled or not when via the
+ '-o barriers=[0|1]' mount option for both ext3 and ext4 filesystems
+ for a fair comparison. When tuning ext3 for best benchmark numbers,
+ it is often worthwhile to try changing the data journaling mode; '-o
+ data=writeback,nobh' can be faster for some workloads. (Note
+ however that running mounted with data=writeback can potentially
+ leave stale data exposed in recently written files in case of an
+ unclean shutdown, which could be a security exposure in some
+ situations.) Configuring the filesystem with a large journal can
+ also be helpful for metadata-intensive workloads.
2. Features
===========
@@ -74,7 +83,7 @@
* ability to use filesystems > 16TB (e2fsprogs support not available yet)
* extent format reduces metadata overhead (RAM, IO for access, transactions)
* extent format more robust in face of on-disk corruption due to magics,
-* internal redunancy in tree
+* internal redundancy in tree
* improved file allocation (multi-block alloc)
* fix 32000 subdirectory limit
* nsec timestamps for mtime, atime, ctime, create time
@@ -116,10 +125,11 @@
When mounting an ext4 filesystem, the following option are accepted:
(*) == default
-extents (*) ext4 will use extents to address file data. The
- file system will no longer be mountable by ext3.
-
-noextents ext4 will not use extents for newly created files
+ro Mount filesystem read only. Note that ext4 will
+ replay the journal (and thus write to the
+ partition) even when mounted "read only". The
+ mount options "ro,noload" can be used to prevent
+ writes to the filesystem.
journal_checksum Enable checksumming of the journal transactions.
This will allow the recovery code in e2fsck and the
@@ -134,17 +144,17 @@
journal=update Update the ext4 file system's journal to the current
format.
-journal=inum When a journal already exists, this option is ignored.
- Otherwise, it specifies the number of the inode which
- will represent the ext4 file system's journal file.
-
journal_dev=devnum When the external journal device's major/minor numbers
have changed, this option allows the user to specify
the new journal location. The journal device is
identified through its new major/minor numbers encoded
in devnum.
-noload Don't load the journal on mounting.
+noload Don't load the journal on mounting. Note that
+ if the filesystem was not unmounted cleanly,
+ skipping the journal replay will lead to the
+ filesystem containing inconsistencies that can
+ lead to any number of problems.
data=journal All data are committed into the journal prior to being
written into the main file system.
@@ -219,9 +229,12 @@
debug Extra debugging information is sent to syslog.
-errors=remount-ro(*) Remount the filesystem read-only on an error.
+errors=remount-ro Remount the filesystem read-only on an error.
errors=continue Keep going on a filesystem error.
errors=panic Panic and halt the machine if an error occurs.
+ (These mount options override the errors behavior
+ specified in the superblock, which can be configured
+ using tune2fs)
data_err=ignore(*) Just print an error message if an error occurs
in a file data buffer in ordered mode.
@@ -261,6 +274,42 @@
nodelalloc Disable delayed allocation. Blocks are allocation
when data is copied from user to page cache.
+max_batch_time=usec Maximum amount of time ext4 should wait for
+ additional filesystem operations to be batch
+ together with a synchronous write operation.
+ Since a synchronous write operation is going to
+ force a commit and then a wait for the I/O
+ complete, it doesn't cost much, and can be a
+ huge throughput win, we wait for a small amount
+ of time to see if any other transactions can
+ piggyback on the synchronous write. The
+ algorithm used is designed to automatically tune
+ for the speed of the disk, by measuring the
+ amount of time (on average) that it takes to
+ finish committing a transaction. Call this time
+ the "commit time". If the time that the
+ transactoin has been running is less than the
+ commit time, ext4 will try sleeping for the
+ commit time to see if other operations will join
+ the transaction. The commit time is capped by
+ the max_batch_time, which defaults to 15000us
+ (15ms). This optimization can be turned off
+ entirely by setting max_batch_time to 0.
+
+min_batch_time=usec This parameter sets the commit time (as
+ described above) to be at least min_batch_time.
+ It defaults to zero microseconds. Increasing
+ this parameter may improve the throughput of
+ multi-threaded, synchronous workloads on very
+ fast disks, at the cost of increasing latency.
+
+journal_ioprio=prio The I/O priority (from 0 to 7, where 0 is the
+ highest priorty) which should be used for I/O
+ operations submitted by kjournald2 during a
+ commit operation. This defaults to 3, which is
+ a slightly higher priority than the default I/O
+ priority.
+
Data Mode
=========
There are 3 different data modes:
diff --git a/Documentation/hwmon/abituguru-datasheet b/Documentation/hwmon/abituguru-datasheet
index 4d184f2..d9251ef 100644
--- a/Documentation/hwmon/abituguru-datasheet
+++ b/Documentation/hwmon/abituguru-datasheet
@@ -121,7 +121,7 @@
test for this. Notice that the number of bytes is bank address dependent see
above and below.
-After completing a successfull read it is advised to put the uGuru back in
+After completing a successful read it is advised to put the uGuru back in
ready mode, so that it is ready for the next read / write cycle. This way
if your program / driver is unloaded and later loaded again the detection
algorithm described above will still work.
@@ -141,7 +141,7 @@
Once DATA holds 0x01 read CMD it should hold 0xAC now.
-After completing a successfull write it is advised to put the uGuru back in
+After completing a successful write it is advised to put the uGuru back in
ready mode, so that it is ready for the next read / write cycle. This way
if your program / driver is unloaded and later loaded again the detection
algorithm described above will still work.
diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index 532eacb..fcc48bf 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -141,6 +141,7 @@
ht -- run only enough ACPI to enable Hyper Threading
strict -- Be less tolerant of platforms that are not
strictly ACPI specification compliant.
+ rsdt -- prefer RSDT over (default) XSDT
See also Documentation/power/pm.txt, pci=noacpi
@@ -151,16 +152,20 @@
default: 0
acpi_sleep= [HW,ACPI] Sleep options
- Format: { s3_bios, s3_mode, s3_beep, s4_nohwsig, old_ordering }
- See Documentation/power/video.txt for s3_bios and s3_mode.
+ Format: { s3_bios, s3_mode, s3_beep, s4_nohwsig,
+ old_ordering, s4_nonvs }
+ See Documentation/power/video.txt for information on
+ s3_bios and s3_mode.
s3_beep is for debugging; it makes the PC's speaker beep
as soon as the kernel's real-mode entry point is called.
s4_nohwsig prevents ACPI hardware signature from being
used during resume from hibernation.
old_ordering causes the ACPI 1.0 ordering of the _PTS
- control method, wrt putting devices into low power
- states, to be enforced (the ACPI 2.0 ordering of _PTS is
- used by default).
+ control method, with respect to putting devices into
+ low power states, to be enforced (the ACPI 2.0 ordering
+ of _PTS is used by default).
+ s4_nonvs prevents the kernel from saving/restoring the
+ ACPI NVS memory during hibernation.
acpi_sci= [HW,ACPI] ACPI System Control Interrupt trigger mode
Format: { level | edge | high | low }
@@ -195,7 +200,7 @@
acpi_skip_timer_override [HW,ACPI]
Recognize and ignore IRQ0/pin2 Interrupt Override.
For broken nForce2 BIOS resulting in XT-PIC timer.
- acpi_use_timer_override [HW,ACPI}
+ acpi_use_timer_override [HW,ACPI]
Use timer override. For some broken Nvidia NF5 boards
that require a timer override, but don't have
HPET
@@ -878,17 +883,19 @@
See Documentation/ide/ide.txt.
idle= [X86]
- Format: idle=poll or idle=mwait, idle=halt, idle=nomwait
- Poll forces a polling idle loop that can slightly improves the performance
- of waking up a idle CPU, but will use a lot of power and make the system
- run hot. Not recommended.
- idle=mwait. On systems which support MONITOR/MWAIT but the kernel chose
- to not use it because it doesn't save as much power as a normal idle
- loop use the MONITOR/MWAIT idle loop anyways. Performance should be the same
- as idle=poll.
- idle=halt. Halt is forced to be used for CPU idle.
+ Format: idle=poll, idle=mwait, idle=halt, idle=nomwait
+ Poll forces a polling idle loop that can slightly
+ improve the performance of waking up a idle CPU, but
+ will use a lot of power and make the system run hot.
+ Not recommended.
+ idle=mwait: On systems which support MONITOR/MWAIT but
+ the kernel chose to not use it because it doesn't save
+ as much power as a normal idle loop, use the
+ MONITOR/MWAIT idle loop anyways. Performance should be
+ the same as idle=poll.
+ idle=halt: Halt is forced to be used for CPU idle.
In such case C2/C3 won't be used again.
- idle=nomwait. Disable mwait for CPU C-states
+ idle=nomwait: Disable mwait for CPU C-states
ide-pci-generic.all-generic-ide [HW] (E)IDE subsystem
Claim all unknown PCI IDE storage controllers.
@@ -1074,8 +1081,8 @@
lapic [X86-32,APIC] Enable the local APIC even if BIOS
disabled it.
- lapic_timer_c2_ok [X86-32,x86-64,APIC] trust the local apic timer in
- C2 power state.
+ lapic_timer_c2_ok [X86-32,x86-64,APIC] trust the local apic timer
+ in C2 power state.
libata.dma= [LIBATA] DMA control
libata.dma=0 Disable all PATA and SATA DMA
@@ -1562,6 +1569,9 @@
nosoftlockup [KNL] Disable the soft-lockup detector.
+ noswapaccount [KNL] Disable accounting of swap in memory resource
+ controller. (See Documentation/controllers/memory.txt)
+
nosync [HW,M68K] Disables sync negotiation for all devices.
notsc [BUGS=X86-32] Disable Time Stamp Counter
@@ -2300,7 +2310,8 @@
thermal.psv= [HW,ACPI]
-1: disable all passive trip points
- <degrees C>: override all passive trip points to this value
+ <degrees C>: override all passive trip points to this
+ value
thermal.tzp= [HW,ACPI]
Specify global default ACPI thermal zone polling rate
diff --git a/Documentation/powerpc/dts-bindings/4xx/ndfc.txt b/Documentation/powerpc/dts-bindings/4xx/ndfc.txt
new file mode 100644
index 0000000..869f0b5
--- /dev/null
+++ b/Documentation/powerpc/dts-bindings/4xx/ndfc.txt
@@ -0,0 +1,39 @@
+AMCC NDFC (NanD Flash Controller)
+
+Required properties:
+- compatible : "ibm,ndfc".
+- reg : should specify chip select and size used for the chip (0x2000).
+
+Optional properties:
+- ccr : NDFC config and control register value (default 0).
+- bank-settings : NDFC bank configuration register value (default 0).
+
+Notes:
+- partition(s) - follows the OF MTD standard for partitions
+
+Example:
+
+ndfc@1,0 {
+ compatible = "ibm,ndfc";
+ reg = <0x00000001 0x00000000 0x00002000>;
+ ccr = <0x00001000>;
+ bank-settings = <0x80002222>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ nand {
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ partition@0 {
+ label = "kernel";
+ reg = <0x00000000 0x00200000>;
+ };
+ partition@200000 {
+ label = "root";
+ reg = <0x00200000 0x03E00000>;
+ };
+ };
+};
+
+
diff --git a/Documentation/powerpc/dts-bindings/fsl/board.txt b/Documentation/powerpc/dts-bindings/fsl/board.txt
index 81a917e..6c974d2 100644
--- a/Documentation/powerpc/dts-bindings/fsl/board.txt
+++ b/Documentation/powerpc/dts-bindings/fsl/board.txt
@@ -18,7 +18,7 @@
Required properities:
- compatible : should be "fsl,fpga-pixis".
-- reg : should contain the address and the lenght of the FPPGA register
+- reg : should contain the address and the length of the FPPGA register
set.
Example (MPC8610HPCD):
@@ -27,3 +27,33 @@
compatible = "fsl,fpga-pixis";
reg = <0xe8000000 32>;
};
+
+* Freescale BCSR GPIO banks
+
+Some BCSR registers act as simple GPIO controllers, each such
+register can be represented by the gpio-controller node.
+
+Required properities:
+- compatible : Should be "fsl,<board>-bcsr-gpio".
+- reg : Should contain the address and the length of the GPIO bank
+ register.
+- #gpio-cells : Should be two. The first cell is the pin number and the
+ second cell is used to specify optional paramters (currently unused).
+- gpio-controller : Marks the port as GPIO controller.
+
+Example:
+
+ bcsr@1,0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "fsl,mpc8360mds-bcsr";
+ reg = <1 0 0x8000>;
+ ranges = <0 1 0 0x8000>;
+
+ bcsr13: gpio-controller@d {
+ #gpio-cells = <2>;
+ compatible = "fsl,mpc8360mds-bcsr-gpio";
+ reg = <0xd 1>;
+ gpio-controller;
+ };
+ };
diff --git a/Documentation/scsi/scsi_fc_transport.txt b/Documentation/scsi/scsi_fc_transport.txt
index 38d324d..e5b071d 100644
--- a/Documentation/scsi/scsi_fc_transport.txt
+++ b/Documentation/scsi/scsi_fc_transport.txt
@@ -191,7 +191,7 @@
This is equivalent to a driver "attach" on an adapter, which is
independent of the adapter's link state.
- Instantiation of the vport on the FC link via ELS traffic, etc.
- This is equivalent to a "link up" and successfull link initialization.
+ This is equivalent to a "link up" and successful link initialization.
Further information can be found in the interfaces section below for
Vport Creation.
@@ -320,7 +320,7 @@
This is equivalent to a driver "attach" on an adapter, which is
independent of the adapter's link state.
- Instantiation of the vport on the FC link via ELS traffic, etc.
- This is equivalent to a "link up" and successfull link initialization.
+ This is equivalent to a "link up" and successful link initialization.
The LLDD's vport_create() function will not synchronously wait for both
parts to be fully completed before returning. It must validate that the
diff --git a/Documentation/w1/masters/00-INDEX b/Documentation/w1/masters/00-INDEX
index 7b0ceaa..d63fa02 100644
--- a/Documentation/w1/masters/00-INDEX
+++ b/Documentation/w1/masters/00-INDEX
@@ -4,5 +4,7 @@
- The Maxim/Dallas Semiconductor DS2482 provides 1-wire busses.
ds2490
- The Maxim/Dallas Semiconductor DS2490 builds USB <-> W1 bridges.
+mxc_w1
+ - W1 master controller driver found on Freescale MX2/MX3 SoCs
w1-gpio
- GPIO 1-wire bus master driver.
diff --git a/Documentation/w1/masters/mxc-w1 b/Documentation/w1/masters/mxc-w1
new file mode 100644
index 0000000..97f6199
--- /dev/null
+++ b/Documentation/w1/masters/mxc-w1
@@ -0,0 +1,11 @@
+Kernel driver mxc_w1
+====================
+
+Supported chips:
+ * Freescale MX27, MX31 and probably other i.MX SoCs
+ Datasheets:
+ http://www.freescale.com/files/32bit/doc/data_sheet/MCIMX31.pdf?fpsp=1
+ http://www.freescale.com/files/dsp/MCIMX27.pdf?fpsp=1
+
+Author: Originally based on Freescale code, prepared for mainline by
+ Sascha Hauer <s.hauer@pengutronix.de>
diff --git a/Documentation/w1/w1.netlink b/Documentation/w1/w1.netlink
index 3640c7c..804445f 100644
--- a/Documentation/w1/w1.netlink
+++ b/Documentation/w1/w1.netlink
@@ -5,69 +5,157 @@
=============
There are three types of messages between w1 core and userspace:
-1. Events. They are generated each time new master or slave device found
- either due to automatic or requested search.
-2. Userspace commands. Includes read/write and search/alarm search comamnds.
+1. Events. They are generated each time new master or slave device
+ found either due to automatic or requested search.
+2. Userspace commands.
3. Replies to userspace commands.
Protocol.
========
-[struct cn_msg] - connector header. It's length field is equal to size of the attached data.
+[struct cn_msg] - connector header.
+ Its length field is equal to size of the attached data
[struct w1_netlink_msg] - w1 netlink header.
__u8 type - message type.
- W1_SLAVE_ADD/W1_SLAVE_REMOVE - slave add/remove events.
- W1_MASTER_ADD/W1_MASTER_REMOVE - master add/remove events.
- W1_MASTER_CMD - userspace command for bus master device (search/alarm search).
- W1_SLAVE_CMD - userspace command for slave device (read/write/ search/alarm search
- for bus master device where given slave device found).
+ W1_LIST_MASTERS
+ list current bus masters
+ W1_SLAVE_ADD/W1_SLAVE_REMOVE
+ slave add/remove events
+ W1_MASTER_ADD/W1_MASTER_REMOVE
+ master add/remove events
+ W1_MASTER_CMD
+ userspace command for bus master
+ device (search/alarm search)
+ W1_SLAVE_CMD
+ userspace command for slave device
+ (read/write/touch)
__u8 res - reserved
- __u16 len - size of attached to this header data.
+ __u16 len - size of data attached to this header data
union {
- __u8 id; - slave unique device id
+ __u8 id[8]; - slave unique device id
struct w1_mst {
- __u32 id; - master's id.
+ __u32 id; - master's id
__u32 res; - reserved
} mst;
} id;
-[strucrt w1_netlink_cmd] - command for gived master or slave device.
+[struct w1_netlink_cmd] - command for given master or slave device.
__u8 cmd - command opcode.
- W1_CMD_READ - read command.
- W1_CMD_WRITE - write command.
- W1_CMD_SEARCH - search command.
- W1_CMD_ALARM_SEARCH - alarm search command.
+ W1_CMD_READ - read command
+ W1_CMD_WRITE - write command
+ W1_CMD_TOUCH - touch command
+ (write and sample data back to userspace)
+ W1_CMD_SEARCH - search command
+ W1_CMD_ALARM_SEARCH - alarm search command
__u8 res - reserved
- __u16 len - length of data for this command.
- For read command data must be allocated like for write command.
- __u8 data[0] - data for this command.
+ __u16 len - length of data for this command
+ For read command data must be allocated like for write command
+ __u8 data[0] - data for this command
-Each connector message can include one or more w1_netlink_msg with zero of more attached w1_netlink_cmd messages.
+Each connector message can include one or more w1_netlink_msg with
+zero or more attached w1_netlink_cmd messages.
-For event messages there are no w1_netlink_cmd embedded structures, only connector header
-and w1_netlink_msg strucutre with "len" field being zero and filled type (one of event types)
-and id - either 8 bytes of slave unique id in host order, or master's id, which is assigned
-to bus master device when it is added to w1 core.
+For event messages there are no w1_netlink_cmd embedded structures,
+only connector header and w1_netlink_msg strucutre with "len" field
+being zero and filled type (one of event types) and id:
+either 8 bytes of slave unique id in host order,
+or master's id, which is assigned to bus master device
+when it is added to w1 core.
-Currently replies to userspace commands are only generated for read command request.
-One reply is generated exactly for one w1_netlink_cmd read request.
-Replies are not combined when sent - i.e. typical reply messages looks like the following:
+Currently replies to userspace commands are only generated for read
+command request. One reply is generated exactly for one w1_netlink_cmd
+read request. Replies are not combined when sent - i.e. typical reply
+messages looks like the following:
+
[cn_msg][w1_netlink_msg][w1_netlink_cmd]
-cn_msg.len = sizeof(struct w1_netlink_msg) + sizeof(struct w1_netlink_cmd) + cmd->len;
+cn_msg.len = sizeof(struct w1_netlink_msg) +
+ sizeof(struct w1_netlink_cmd) +
+ cmd->len;
w1_netlink_msg.len = sizeof(struct w1_netlink_cmd) + cmd->len;
w1_netlink_cmd.len = cmd->len;
+Replies to W1_LIST_MASTERS should send a message back to the userspace
+which will contain list of all registered master ids in the following
+format:
+
+ cn_msg (CN_W1_IDX.CN_W1_VAL as id, len is equal to sizeof(struct
+ w1_netlink_msg) plus number of masters multipled by 4)
+ w1_netlink_msg (type: W1_LIST_MASTERS, len is equal to
+ number of masters multiplied by 4 (u32 size))
+ id0 ... idN
+
+ Each message is at most 4k in size, so if number of master devices
+ exceeds this, it will be split into several messages,
+ cn.seq will be increased for each one.
+
+W1 search and alarm search commands.
+request:
+[cn_msg]
+ [w1_netlink_msg type = W1_MASTER_CMD
+ id is equal to the bus master id to use for searching]
+ [w1_netlink_cmd cmd = W1_CMD_SEARCH or W1_CMD_ALARM_SEARCH]
+
+reply:
+ [cn_msg, ack = 1 and increasing, 0 means the last message,
+ seq is equal to the request seq]
+ [w1_netlink_msg type = W1_MASTER_CMD]
+ [w1_netlink_cmd cmd = W1_CMD_SEARCH or W1_CMD_ALARM_SEARCH
+ len is equal to number of IDs multiplied by 8]
+ [64bit-id0 ... 64bit-idN]
+Length in each header corresponds to the size of the data behind it, so
+w1_netlink_cmd->len = N * 8; where N is number of IDs in this message.
+ Can be zero.
+w1_netlink_msg->len = sizeof(struct w1_netlink_cmd) + N * 8;
+cn_msg->len = sizeof(struct w1_netlink_msg) +
+ sizeof(struct w1_netlink_cmd) +
+ N*8;
+
+W1 reset command.
+[cn_msg]
+ [w1_netlink_msg type = W1_MASTER_CMD
+ id is equal to the bus master id to use for searching]
+ [w1_netlink_cmd cmd = W1_CMD_RESET]
+
+
+Command status replies.
+======================
+
+Each command (either root, master or slave with or without w1_netlink_cmd
+structure) will be 'acked' by the w1 core. Format of the reply is the same
+as request message except that length parameters do not account for data
+requested by the user, i.e. read/write/touch IO requests will not contain
+data, so w1_netlink_cmd.len will be 0, w1_netlink_msg.len will be size
+of the w1_netlink_cmd structure and cn_msg.len will be equal to the sum
+of the sizeof(struct w1_netlink_msg) and sizeof(struct w1_netlink_cmd).
+If reply is generated for master or root command (which do not have
+w1_netlink_cmd attached), reply will contain only cn_msg and w1_netlink_msg
+structires.
+
+w1_netlink_msg.status field will carry positive error value
+(EINVAL for example) or zero in case of success.
+
+All other fields in every structure will mirror the same parameters in the
+request message (except lengths as described above).
+
+Status reply is generated for every w1_netlink_cmd embedded in the
+w1_netlink_msg, if there are no w1_netlink_cmd structures,
+reply will be generated for the w1_netlink_msg.
+
+All w1_netlink_cmd command structures are handled in every w1_netlink_msg,
+even if there were errors, only length mismatch interrupts message processing.
+
Operation steps in w1 core when new command is received.
=======================================================
-When new message (w1_netlink_msg) is received w1 core detects if it is master of slave request,
-according to w1_netlink_msg.type field.
+When new message (w1_netlink_msg) is received w1 core detects if it is
+master or slave request, according to w1_netlink_msg.type field.
Then master or slave device is searched for.
-When found, master device (requested or those one on where slave device is found) is locked.
-If slave command is requested, then reset/select procedure is started to select given device.
+When found, master device (requested or those one on where slave device
+is found) is locked. If slave command is requested, then reset/select
+procedure is started to select given device.
Then all requested in w1_netlink_msg operations are performed one by one.
If command requires reply (like read command) it is sent on command completion.
@@ -82,8 +170,8 @@
Each connector message includes two u32 fields as "address".
w1 uses CN_W1_IDX and CN_W1_VAL defined in include/linux/connector.h header.
Each message also includes sequence and acknowledge numbers.
-Sequence number for event messages is appropriate bus master sequence number increased with
-each event message sent "through" this master.
+Sequence number for event messages is appropriate bus master sequence number
+increased with each event message sent "through" this master.
Sequence number for userspace requests is set by userspace application.
Sequence number for reply is the same as was in request, and
acknowledge number is set to seq+1.
@@ -93,6 +181,6 @@
============================================
1. Documentation/connector
-2. http://tservice.net.ru/~s0mbre/archive/w1
-This archive includes userspace application w1d.c which
-uses read/write/search commands for all master/slave devices found on the bus.
+2. http://www.ioremap.net/archive/w1
+This archive includes userspace application w1d.c which uses
+read/write/search commands for all master/slave devices found on the bus.
diff --git a/MAINTAINERS b/MAINTAINERS
index a018844..57e0309 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1360,6 +1360,11 @@
M: macro@linux-mips.org
S: Maintained
+DELL LAPTOP DRIVER
+P: Matthew Garrett
+M: mjg59@srcf.ucam.org
+S: Maintained
+
DELL LAPTOP SMM DRIVER
P: Massimo Dal Zotto
M: dz@debian.org
@@ -3484,6 +3489,12 @@
L: cbe-oss-dev@ozlabs.org
S: Supported
+PS3VRAM DRIVER
+P: Jim Paris
+M: jim@jtan.com
+L: cbe-oss-dev@ozlabs.org
+S: Maintained
+
PVRUSB2 VIDEO4LINUX DRIVER
P: Mike Isely
M: isely@pobox.com
diff --git a/arch/arm/configs/clps7500_defconfig b/arch/arm/configs/clps7500_defconfig
deleted file mode 100644
index 49e9f9d..0000000
--- a/arch/arm/configs/clps7500_defconfig
+++ /dev/null
@@ -1,801 +0,0 @@
-#
-# Automatically generated make config: don't edit
-# Linux kernel version: 2.6.12-rc1-bk2
-# Sun Mar 27 17:20:48 2005
-#
-CONFIG_ARM=y
-CONFIG_MMU=y
-CONFIG_UID16=y
-CONFIG_RWSEM_GENERIC_SPINLOCK=y
-CONFIG_GENERIC_CALIBRATE_DELAY=y
-CONFIG_GENERIC_IOMAP=y
-
-#
-# Code maturity level options
-#
-CONFIG_EXPERIMENTAL=y
-CONFIG_CLEAN_COMPILE=y
-CONFIG_BROKEN_ON_SMP=y
-
-#
-# General setup
-#
-CONFIG_LOCALVERSION=""
-CONFIG_SWAP=y
-CONFIG_SYSVIPC=y
-# CONFIG_POSIX_MQUEUE is not set
-# CONFIG_BSD_PROCESS_ACCT is not set
-# CONFIG_SYSCTL is not set
-# CONFIG_AUDIT is not set
-# CONFIG_HOTPLUG is not set
-CONFIG_KOBJECT_UEVENT=y
-# CONFIG_IKCONFIG is not set
-CONFIG_EMBEDDED=y
-CONFIG_KALLSYMS=y
-# CONFIG_KALLSYMS_EXTRA_PASS is not set
-CONFIG_BASE_FULL=y
-CONFIG_FUTEX=y
-CONFIG_EPOLL=y
-CONFIG_CC_OPTIMIZE_FOR_SIZE=y
-CONFIG_SHMEM=y
-CONFIG_CC_ALIGN_FUNCTIONS=0
-CONFIG_CC_ALIGN_LABELS=0
-CONFIG_CC_ALIGN_LOOPS=0
-CONFIG_CC_ALIGN_JUMPS=0
-# CONFIG_TINY_SHMEM is not set
-CONFIG_BASE_SMALL=0
-
-#
-# Loadable module support
-#
-# CONFIG_MODULES is not set
-
-#
-# System Type
-#
-CONFIG_ARCH_CLPS7500=y
-# CONFIG_ARCH_CLPS711X is not set
-# CONFIG_ARCH_CO285 is not set
-# CONFIG_ARCH_EBSA110 is not set
-# CONFIG_ARCH_FOOTBRIDGE is not set
-# CONFIG_ARCH_INTEGRATOR is not set
-# CONFIG_ARCH_IOP3XX is not set
-# CONFIG_ARCH_IXP4XX is not set
-# CONFIG_ARCH_IXP2000 is not set
-# CONFIG_ARCH_L7200 is not set
-# CONFIG_ARCH_PXA is not set
-# CONFIG_ARCH_RPC is not set
-# CONFIG_ARCH_SA1100 is not set
-# CONFIG_ARCH_S3C2410 is not set
-# CONFIG_ARCH_SHARK is not set
-# CONFIG_ARCH_LH7A40X is not set
-# CONFIG_ARCH_OMAP is not set
-# CONFIG_ARCH_VERSATILE is not set
-# CONFIG_ARCH_IMX is not set
-# CONFIG_ARCH_H720X is not set
-
-#
-# Processor Type
-#
-CONFIG_CPU_32=y
-CONFIG_CPU_ARM710=y
-CONFIG_CPU_32v3=y
-CONFIG_CPU_CACHE_V3=y
-CONFIG_CPU_CACHE_VIVT=y
-CONFIG_CPU_COPY_V3=y
-CONFIG_CPU_TLB_V3=y
-
-#
-# Processor Features
-#
-CONFIG_TIMER_ACORN=y
-
-#
-# Bus support
-#
-CONFIG_ISA=y
-
-#
-# PCCARD (PCMCIA/CardBus) support
-#
-# CONFIG_PCCARD is not set
-
-#
-# Kernel Features
-#
-# CONFIG_PREEMPT is not set
-CONFIG_ALIGNMENT_TRAP=y
-
-#
-# Boot options
-#
-CONFIG_ZBOOT_ROM_TEXT=0x0
-CONFIG_ZBOOT_ROM_BSS=0x0
-CONFIG_CMDLINE="mem=16M root=nfs"
-# CONFIG_XIP_KERNEL is not set
-
-#
-# Floating point emulation
-#
-
-#
-# At least one emulation must be selected
-#
-# CONFIG_FPE_NWFPE is not set
-
-#
-# Userspace binary formats
-#
-CONFIG_BINFMT_ELF=y
-# CONFIG_BINFMT_AOUT is not set
-# CONFIG_BINFMT_MISC is not set
-# CONFIG_ARTHUR is not set
-
-#
-# Power management options
-#
-# CONFIG_PM is not set
-
-#
-# Device Drivers
-#
-
-#
-# Generic Driver Options
-#
-CONFIG_STANDALONE=y
-CONFIG_PREVENT_FIRMWARE_BUILD=y
-# CONFIG_FW_LOADER is not set
-
-#
-# Memory Technology Devices (MTD)
-#
-CONFIG_MTD=y
-# CONFIG_MTD_DEBUG is not set
-# CONFIG_MTD_CONCAT is not set
-# CONFIG_MTD_PARTITIONS is not set
-
-#
-# User Modules And Translation Layers
-#
-# CONFIG_MTD_CHAR is not set
-# CONFIG_MTD_BLOCK is not set
-# CONFIG_MTD_BLOCK_RO is not set
-# CONFIG_FTL is not set
-# CONFIG_NFTL is not set
-# CONFIG_INFTL is not set
-
-#
-# RAM/ROM/Flash chip drivers
-#
-# CONFIG_MTD_CFI is not set
-# CONFIG_MTD_JEDECPROBE is not set
-CONFIG_MTD_MAP_BANK_WIDTH_1=y
-CONFIG_MTD_MAP_BANK_WIDTH_2=y
-CONFIG_MTD_MAP_BANK_WIDTH_4=y
-# CONFIG_MTD_MAP_BANK_WIDTH_8 is not set
-# CONFIG_MTD_MAP_BANK_WIDTH_16 is not set
-# CONFIG_MTD_MAP_BANK_WIDTH_32 is not set
-CONFIG_MTD_CFI_I1=y
-CONFIG_MTD_CFI_I2=y
-# CONFIG_MTD_CFI_I4 is not set
-# CONFIG_MTD_CFI_I8 is not set
-# CONFIG_MTD_RAM is not set
-# CONFIG_MTD_ROM is not set
-# CONFIG_MTD_ABSENT is not set
-
-#
-# Mapping drivers for chip access
-#
-# CONFIG_MTD_COMPLEX_MAPPINGS is not set
-
-#
-# Self-contained MTD device drivers
-#
-# CONFIG_MTD_SLRAM is not set
-# CONFIG_MTD_PHRAM is not set
-# CONFIG_MTD_MTDRAM is not set
-# CONFIG_MTD_BLKMTD is not set
-# CONFIG_MTD_BLOCK2MTD is not set
-
-#
-# Disk-On-Chip Device Drivers
-#
-# CONFIG_MTD_DOC2000 is not set
-# CONFIG_MTD_DOC2001 is not set
-# CONFIG_MTD_DOC2001PLUS is not set
-
-#
-# NAND Flash Device Drivers
-#
-# CONFIG_MTD_NAND is not set
-
-#
-# Parallel port support
-#
-CONFIG_PARPORT=y
-CONFIG_PARPORT_PC=y
-CONFIG_PARPORT_PC_FIFO=y
-# CONFIG_PARPORT_PC_SUPERIO is not set
-# CONFIG_PARPORT_ARC is not set
-# CONFIG_PARPORT_GSC is not set
-CONFIG_PARPORT_1284=y
-
-#
-# Plug and Play support
-#
-# CONFIG_PNP is not set
-
-#
-# Block devices
-#
-# CONFIG_BLK_DEV_FD is not set
-# CONFIG_BLK_DEV_XD is not set
-# CONFIG_PARIDE is not set
-# CONFIG_BLK_DEV_COW_COMMON is not set
-# CONFIG_BLK_DEV_LOOP is not set
-CONFIG_BLK_DEV_NBD=y
-CONFIG_BLK_DEV_RAM=y
-CONFIG_BLK_DEV_RAM_COUNT=16
-CONFIG_BLK_DEV_RAM_SIZE=4096
-# CONFIG_BLK_DEV_INITRD is not set
-CONFIG_INITRAMFS_SOURCE=""
-# CONFIG_CDROM_PKTCDVD is not set
-
-#
-# IO Schedulers
-#
-CONFIG_IOSCHED_NOOP=y
-CONFIG_IOSCHED_AS=y
-CONFIG_IOSCHED_DEADLINE=y
-CONFIG_IOSCHED_CFQ=y
-# CONFIG_ATA_OVER_ETH is not set
-
-#
-# ATA/ATAPI/MFM/RLL support
-#
-# CONFIG_IDE is not set
-
-#
-# SCSI device support
-#
-# CONFIG_SCSI is not set
-
-#
-# Multi-device support (RAID and LVM)
-#
-# CONFIG_MD is not set
-
-#
-# Fusion MPT device support
-#
-
-#
-# IEEE 1394 (FireWire) support
-#
-
-#
-# I2O device support
-#
-
-#
-# Networking support
-#
-CONFIG_NET=y
-
-#
-# Networking options
-#
-# CONFIG_PACKET is not set
-# CONFIG_NETLINK_DEV is not set
-CONFIG_UNIX=y
-# CONFIG_NET_KEY is not set
-CONFIG_INET=y
-# CONFIG_IP_MULTICAST is not set
-# CONFIG_IP_ADVANCED_ROUTER is not set
-CONFIG_IP_PNP=y
-# CONFIG_IP_PNP_DHCP is not set
-CONFIG_IP_PNP_BOOTP=y
-# CONFIG_IP_PNP_RARP is not set
-# CONFIG_NET_IPIP is not set
-# CONFIG_NET_IPGRE is not set
-# CONFIG_ARPD is not set
-# CONFIG_SYN_COOKIES is not set
-# CONFIG_INET_AH is not set
-# CONFIG_INET_ESP is not set
-# CONFIG_INET_IPCOMP is not set
-# CONFIG_INET_TUNNEL is not set
-CONFIG_IP_TCPDIAG=y
-# CONFIG_IP_TCPDIAG_IPV6 is not set
-# CONFIG_IPV6 is not set
-# CONFIG_NETFILTER is not set
-
-#
-# SCTP Configuration (EXPERIMENTAL)
-#
-# CONFIG_IP_SCTP is not set
-# CONFIG_ATM is not set
-# CONFIG_BRIDGE is not set
-# CONFIG_VLAN_8021Q is not set
-# CONFIG_DECNET is not set
-# CONFIG_LLC2 is not set
-# CONFIG_IPX is not set
-# CONFIG_ATALK is not set
-# CONFIG_X25 is not set
-# CONFIG_LAPB is not set
-# CONFIG_NET_DIVERT is not set
-# CONFIG_ECONET is not set
-# CONFIG_WAN_ROUTER is not set
-
-#
-# QoS and/or fair queueing
-#
-# CONFIG_NET_SCHED is not set
-# CONFIG_NET_CLS_ROUTE is not set
-
-#
-# Network testing
-#
-# CONFIG_NET_PKTGEN is not set
-# CONFIG_NETPOLL is not set
-# CONFIG_NET_POLL_CONTROLLER is not set
-# CONFIG_HAMRADIO is not set
-# CONFIG_IRDA is not set
-# CONFIG_BT is not set
-CONFIG_NETDEVICES=y
-CONFIG_DUMMY=y
-# CONFIG_BONDING is not set
-# CONFIG_EQUALIZER is not set
-# CONFIG_TUN is not set
-
-#
-# ARCnet devices
-#
-# CONFIG_ARCNET is not set
-
-#
-# Ethernet (10 or 100Mbit)
-#
-CONFIG_NET_ETHERNET=y
-# CONFIG_MII is not set
-# CONFIG_NET_VENDOR_3COM is not set
-# CONFIG_LANCE is not set
-# CONFIG_NET_VENDOR_SMC is not set
-# CONFIG_SMC91X is not set
-# CONFIG_NET_VENDOR_RACAL is not set
-# CONFIG_AT1700 is not set
-# CONFIG_DEPCA is not set
-# CONFIG_HP100 is not set
-# CONFIG_NET_ISA is not set
-CONFIG_NET_PCI=y
-# CONFIG_AC3200 is not set
-# CONFIG_APRICOT is not set
-CONFIG_CS89x0=y
-# CONFIG_NET_POCKET is not set
-
-#
-# Ethernet (1000 Mbit)
-#
-
-#
-# Ethernet (10000 Mbit)
-#
-
-#
-# Token Ring devices
-#
-# CONFIG_TR is not set
-
-#
-# Wireless LAN (non-hamradio)
-#
-# CONFIG_NET_RADIO is not set
-
-#
-# Wan interfaces
-#
-# CONFIG_WAN is not set
-# CONFIG_PLIP is not set
-CONFIG_PPP=y
-# CONFIG_PPP_MULTILINK is not set
-# CONFIG_PPP_FILTER is not set
-# CONFIG_PPP_ASYNC is not set
-# CONFIG_PPP_SYNC_TTY is not set
-# CONFIG_PPP_DEFLATE is not set
-# CONFIG_PPP_BSDCOMP is not set
-# CONFIG_PPPOE is not set
-CONFIG_SLIP=y
-CONFIG_SLIP_COMPRESSED=y
-# CONFIG_SLIP_SMART is not set
-# CONFIG_SLIP_MODE_SLIP6 is not set
-# CONFIG_SHAPER is not set
-# CONFIG_NETCONSOLE is not set
-
-#
-# ISDN subsystem
-#
-# CONFIG_ISDN is not set
-
-#
-# Input device support
-#
-CONFIG_INPUT=y
-
-#
-# Userland interfaces
-#
-CONFIG_INPUT_MOUSEDEV=y
-CONFIG_INPUT_MOUSEDEV_PSAUX=y
-CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
-CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
-# CONFIG_INPUT_JOYDEV is not set
-# CONFIG_INPUT_TSDEV is not set
-# CONFIG_INPUT_EVDEV is not set
-# CONFIG_INPUT_EVBUG is not set
-
-#
-# Input Device Drivers
-#
-CONFIG_INPUT_KEYBOARD=y
-CONFIG_KEYBOARD_ATKBD=y
-# CONFIG_KEYBOARD_SUNKBD is not set
-# CONFIG_KEYBOARD_LKKBD is not set
-# CONFIG_KEYBOARD_XTKBD is not set
-# CONFIG_KEYBOARD_NEWTON is not set
-CONFIG_INPUT_MOUSE=y
-CONFIG_MOUSE_PS2=y
-# CONFIG_MOUSE_SERIAL is not set
-# CONFIG_MOUSE_INPORT is not set
-# CONFIG_MOUSE_LOGIBM is not set
-# CONFIG_MOUSE_PC110PAD is not set
-# CONFIG_MOUSE_VSXXXAA is not set
-# CONFIG_INPUT_JOYSTICK is not set
-# CONFIG_INPUT_TOUCHSCREEN is not set
-# CONFIG_INPUT_MISC is not set
-
-#
-# Hardware I/O ports
-#
-CONFIG_SERIO=y
-# CONFIG_SERIO_SERPORT is not set
-# CONFIG_SERIO_PARKBD is not set
-CONFIG_SERIO_RPCKBD=y
-CONFIG_SERIO_LIBPS2=y
-# CONFIG_SERIO_RAW is not set
-# CONFIG_GAMEPORT is not set
-CONFIG_SOUND_GAMEPORT=y
-
-#
-# Character devices
-#
-CONFIG_VT=y
-CONFIG_VT_CONSOLE=y
-CONFIG_HW_CONSOLE=y
-# CONFIG_SERIAL_NONSTANDARD is not set
-
-#
-# Serial drivers
-#
-CONFIG_SERIAL_8250=y
-CONFIG_SERIAL_8250_CONSOLE=y
-CONFIG_SERIAL_8250_NR_UARTS=4
-# CONFIG_SERIAL_8250_EXTENDED is not set
-
-#
-# Non-8250 serial port support
-#
-CONFIG_SERIAL_CORE=y
-CONFIG_SERIAL_CORE_CONSOLE=y
-CONFIG_UNIX98_PTYS=y
-CONFIG_LEGACY_PTYS=y
-CONFIG_LEGACY_PTY_COUNT=256
-CONFIG_PRINTER=y
-# CONFIG_LP_CONSOLE is not set
-# CONFIG_PPDEV is not set
-# CONFIG_TIPAR is not set
-
-#
-# IPMI
-#
-# CONFIG_IPMI_HANDLER is not set
-
-#
-# Watchdog Cards
-#
-# CONFIG_WATCHDOG is not set
-# CONFIG_NVRAM is not set
-# CONFIG_RTC is not set
-# CONFIG_DTLK is not set
-# CONFIG_R3964 is not set
-
-#
-# Ftape, the floppy tape device driver
-#
-# CONFIG_DRM is not set
-# CONFIG_RAW_DRIVER is not set
-
-#
-# TPM devices
-#
-# CONFIG_TCG_TPM is not set
-
-#
-# I2C support
-#
-CONFIG_I2C=y
-# CONFIG_I2C_CHARDEV is not set
-
-#
-# I2C Algorithms
-#
-CONFIG_I2C_ALGOBIT=y
-# CONFIG_I2C_ALGOPCF is not set
-# CONFIG_I2C_ALGOPCA is not set
-
-#
-# I2C Hardware Bus support
-#
-# CONFIG_I2C_ELEKTOR is not set
-# CONFIG_I2C_PARPORT is not set
-# CONFIG_I2C_PARPORT_LIGHT is not set
-# CONFIG_I2C_PCA_ISA is not set
-
-#
-# Hardware Sensors Chip support
-#
-# CONFIG_I2C_SENSOR is not set
-# CONFIG_SENSORS_ADM1021 is not set
-# CONFIG_SENSORS_ADM1025 is not set
-# CONFIG_SENSORS_ADM1026 is not set
-# CONFIG_SENSORS_ADM1031 is not set
-# CONFIG_SENSORS_ASB100 is not set
-# CONFIG_SENSORS_DS1621 is not set
-# CONFIG_SENSORS_FSCHER is not set
-# CONFIG_SENSORS_FSCPOS is not set
-# CONFIG_SENSORS_GL518SM is not set
-# CONFIG_SENSORS_GL520SM is not set
-# CONFIG_SENSORS_IT87 is not set
-# CONFIG_SENSORS_LM63 is not set
-# CONFIG_SENSORS_LM75 is not set
-# CONFIG_SENSORS_LM77 is not set
-# CONFIG_SENSORS_LM78 is not set
-# CONFIG_SENSORS_LM80 is not set
-# CONFIG_SENSORS_LM83 is not set
-# CONFIG_SENSORS_LM85 is not set
-# CONFIG_SENSORS_LM87 is not set
-# CONFIG_SENSORS_LM90 is not set
-# CONFIG_SENSORS_MAX1619 is not set
-# CONFIG_SENSORS_PC87360 is not set
-# CONFIG_SENSORS_SMSC47B397 is not set
-# CONFIG_SENSORS_SMSC47M1 is not set
-# CONFIG_SENSORS_W83781D is not set
-# CONFIG_SENSORS_W83L785TS is not set
-# CONFIG_SENSORS_W83627HF is not set
-
-#
-# Other I2C Chip support
-#
-# CONFIG_SENSORS_EEPROM is not set
-# CONFIG_SENSORS_PCF8574 is not set
-# CONFIG_SENSORS_PCF8591 is not set
-# CONFIG_SENSORS_RTC8564 is not set
-# CONFIG_I2C_DEBUG_CORE is not set
-# CONFIG_I2C_DEBUG_ALGO is not set
-# CONFIG_I2C_DEBUG_BUS is not set
-# CONFIG_I2C_DEBUG_CHIP is not set
-
-#
-# Misc devices
-#
-
-#
-# Multimedia devices
-#
-# CONFIG_VIDEO_DEV is not set
-
-#
-# Digital Video Broadcasting Devices
-#
-# CONFIG_DVB is not set
-
-#
-# Graphics support
-#
-CONFIG_FB=y
-CONFIG_FB_CFB_FILLRECT=y
-CONFIG_FB_CFB_COPYAREA=y
-CONFIG_FB_CFB_IMAGEBLIT=y
-CONFIG_FB_SOFT_CURSOR=y
-# CONFIG_FB_MODE_HELPERS is not set
-# CONFIG_FB_TILEBLITTING is not set
-CONFIG_FB_ACORN=y
-# CONFIG_FB_VIRTUAL is not set
-
-#
-# Console display driver support
-#
-# CONFIG_VGA_CONSOLE is not set
-# CONFIG_MDA_CONSOLE is not set
-CONFIG_DUMMY_CONSOLE=y
-CONFIG_FRAMEBUFFER_CONSOLE=y
-CONFIG_FONTS=y
-CONFIG_FONT_8x8=y
-CONFIG_FONT_8x16=y
-# CONFIG_FONT_6x11 is not set
-# CONFIG_FONT_PEARL_8x8 is not set
-# CONFIG_FONT_ACORN_8x8 is not set
-# CONFIG_FONT_MINI_4x6 is not set
-# CONFIG_FONT_SUN8x16 is not set
-# CONFIG_FONT_SUN12x22 is not set
-
-#
-# Logo configuration
-#
-CONFIG_LOGO=y
-CONFIG_LOGO_LINUX_MONO=y
-CONFIG_LOGO_LINUX_VGA16=y
-CONFIG_LOGO_LINUX_CLUT224=y
-# CONFIG_BACKLIGHT_LCD_SUPPORT is not set
-
-#
-# Sound
-#
-# CONFIG_SOUND is not set
-
-#
-# USB support
-#
-CONFIG_USB_ARCH_HAS_HCD=y
-# CONFIG_USB_ARCH_HAS_OHCI is not set
-# CONFIG_USB is not set
-
-#
-# USB Gadget Support
-#
-# CONFIG_USB_GADGET is not set
-
-#
-# MMC/SD Card support
-#
-# CONFIG_MMC is not set
-
-#
-# File systems
-#
-CONFIG_EXT2_FS=y
-# CONFIG_EXT2_FS_XATTR is not set
-# CONFIG_EXT3_FS is not set
-# CONFIG_JBD is not set
-# CONFIG_REISERFS_FS is not set
-# CONFIG_JFS_FS is not set
-
-#
-# XFS support
-#
-# CONFIG_XFS_FS is not set
-CONFIG_MINIX_FS=y
-# CONFIG_ROMFS_FS is not set
-# CONFIG_QUOTA is not set
-CONFIG_DNOTIFY=y
-# CONFIG_AUTOFS_FS is not set
-# CONFIG_AUTOFS4_FS is not set
-
-#
-# CD-ROM/DVD Filesystems
-#
-# CONFIG_ISO9660_FS is not set
-# CONFIG_UDF_FS is not set
-
-#
-# DOS/FAT/NT Filesystems
-#
-# CONFIG_MSDOS_FS is not set
-# CONFIG_VFAT_FS is not set
-# CONFIG_NTFS_FS is not set
-
-#
-# Pseudo filesystems
-#
-CONFIG_PROC_FS=y
-CONFIG_SYSFS=y
-# CONFIG_DEVFS_FS is not set
-# CONFIG_DEVPTS_FS_XATTR is not set
-# CONFIG_TMPFS is not set
-# CONFIG_HUGETLB_PAGE is not set
-CONFIG_RAMFS=y
-
-#
-# Miscellaneous filesystems
-#
-# CONFIG_ADFS_FS is not set
-# CONFIG_AFFS_FS is not set
-# CONFIG_HFS_FS is not set
-# CONFIG_HFSPLUS_FS is not set
-# CONFIG_BEFS_FS is not set
-# CONFIG_BFS_FS is not set
-# CONFIG_EFS_FS is not set
-# CONFIG_JFFS_FS is not set
-# CONFIG_JFFS2_FS is not set
-# CONFIG_CRAMFS is not set
-# CONFIG_VXFS_FS is not set
-# CONFIG_HPFS_FS is not set
-# CONFIG_QNX4FS_FS is not set
-# CONFIG_SYSV_FS is not set
-# CONFIG_UFS_FS is not set
-
-#
-# Network File Systems
-#
-CONFIG_NFS_FS=y
-# CONFIG_NFS_V3 is not set
-# CONFIG_NFS_V4 is not set
-# CONFIG_NFS_DIRECTIO is not set
-# CONFIG_NFSD is not set
-CONFIG_ROOT_NFS=y
-CONFIG_LOCKD=y
-CONFIG_SUNRPC=y
-# CONFIG_RPCSEC_GSS_KRB5 is not set
-# CONFIG_RPCSEC_GSS_SPKM3 is not set
-# CONFIG_SMB_FS is not set
-# CONFIG_CIFS is not set
-# CONFIG_NCP_FS is not set
-# CONFIG_CODA_FS is not set
-# CONFIG_AFS_FS is not set
-
-#
-# Partition Types
-#
-CONFIG_PARTITION_ADVANCED=y
-# CONFIG_ACORN_PARTITION is not set
-# CONFIG_OSF_PARTITION is not set
-# CONFIG_AMIGA_PARTITION is not set
-# CONFIG_ATARI_PARTITION is not set
-# CONFIG_MAC_PARTITION is not set
-# CONFIG_MSDOS_PARTITION is not set
-# CONFIG_LDM_PARTITION is not set
-# CONFIG_SGI_PARTITION is not set
-# CONFIG_ULTRIX_PARTITION is not set
-# CONFIG_SUN_PARTITION is not set
-# CONFIG_EFI_PARTITION is not set
-
-#
-# Native Language Support
-#
-# CONFIG_NLS is not set
-
-#
-# Profiling support
-#
-# CONFIG_PROFILING is not set
-
-#
-# Kernel hacking
-#
-# CONFIG_PRINTK_TIME is not set
-# CONFIG_DEBUG_KERNEL is not set
-CONFIG_LOG_BUF_SHIFT=14
-# CONFIG_DEBUG_BUGVERBOSE is not set
-CONFIG_FRAME_POINTER=y
-# CONFIG_DEBUG_USER is not set
-
-#
-# Security options
-#
-# CONFIG_KEYS is not set
-# CONFIG_SECURITY is not set
-
-#
-# Cryptographic options
-#
-# CONFIG_CRYPTO is not set
-
-#
-# Hardware crypto devices
-#
-
-#
-# Library routines
-#
-# CONFIG_CRC_CCITT is not set
-CONFIG_CRC32=y
-# CONFIG_LIBCRC32C is not set
diff --git a/arch/arm/kernel/isa.c b/arch/arm/kernel/isa.c
index 50a30bc..8ac9b84 100644
--- a/arch/arm/kernel/isa.c
+++ b/arch/arm/kernel/isa.c
@@ -16,6 +16,7 @@
#include <linux/fs.h>
#include <linux/sysctl.h>
#include <linux/init.h>
+#include <linux/io.h>
static unsigned int isa_membase, isa_portbase, isa_portshift;
diff --git a/arch/arm/mach-at91/at91cap9.c b/arch/arm/mach-at91/at91cap9.c
index 0a38c69..7337617 100644
--- a/arch/arm/mach-at91/at91cap9.c
+++ b/arch/arm/mach-at91/at91cap9.c
@@ -15,6 +15,7 @@
#include <linux/module.h>
#include <linux/pm.h>
+#include <asm/irq.h>
#include <asm/mach/arch.h>
#include <asm/mach/map.h>
diff --git a/arch/arm/mach-at91/at91rm9200.c b/arch/arm/mach-at91/at91rm9200.c
index 28594fc..2e9ecad 100644
--- a/arch/arm/mach-at91/at91rm9200.c
+++ b/arch/arm/mach-at91/at91rm9200.c
@@ -12,6 +12,7 @@
#include <linux/module.h>
+#include <asm/irq.h>
#include <asm/mach/arch.h>
#include <asm/mach/map.h>
#include <mach/at91rm9200.h>
diff --git a/arch/arm/mach-at91/at91sam9260.c b/arch/arm/mach-at91/at91sam9260.c
index accb69e..0894f10 100644
--- a/arch/arm/mach-at91/at91sam9260.c
+++ b/arch/arm/mach-at91/at91sam9260.c
@@ -13,6 +13,7 @@
#include <linux/module.h>
#include <linux/pm.h>
+#include <asm/irq.h>
#include <asm/mach/arch.h>
#include <asm/mach/map.h>
#include <mach/cpu.h>
diff --git a/arch/arm/mach-at91/at91sam9261.c b/arch/arm/mach-at91/at91sam9261.c
index 7b51a59..3acd7d7 100644
--- a/arch/arm/mach-at91/at91sam9261.c
+++ b/arch/arm/mach-at91/at91sam9261.c
@@ -13,6 +13,7 @@
#include <linux/module.h>
#include <linux/pm.h>
+#include <asm/irq.h>
#include <asm/mach/arch.h>
#include <asm/mach/map.h>
#include <mach/at91sam9261.h>
diff --git a/arch/arm/mach-at91/at91sam9263.c b/arch/arm/mach-at91/at91sam9263.c
index ada4b67..942792d 100644
--- a/arch/arm/mach-at91/at91sam9263.c
+++ b/arch/arm/mach-at91/at91sam9263.c
@@ -13,6 +13,7 @@
#include <linux/module.h>
#include <linux/pm.h>
+#include <asm/irq.h>
#include <asm/mach/arch.h>
#include <asm/mach/map.h>
#include <mach/at91sam9263.h>
diff --git a/arch/arm/mach-at91/at91sam9rl.c b/arch/arm/mach-at91/at91sam9rl.c
index 252e954..211c5c1 100644
--- a/arch/arm/mach-at91/at91sam9rl.c
+++ b/arch/arm/mach-at91/at91sam9rl.c
@@ -12,6 +12,7 @@
#include <linux/module.h>
#include <linux/pm.h>
+#include <asm/irq.h>
#include <asm/mach/arch.h>
#include <asm/mach/map.h>
#include <mach/cpu.h>
diff --git a/arch/arm/mach-at91/board-sam9rlek.c b/arch/arm/mach-at91/board-sam9rlek.c
index 9b937ee..35e12a4 100644
--- a/arch/arm/mach-at91/board-sam9rlek.c
+++ b/arch/arm/mach-at91/board-sam9rlek.c
@@ -29,6 +29,7 @@
#include <mach/hardware.h>
#include <mach/board.h>
#include <mach/gpio.h>
+#include <mach/at91sam9_smc.h>
#include <mach/at91_shdwc.h>
#include "sam9_smc.h"
diff --git a/arch/arm/mach-clps711x/edb7211-mm.c b/arch/arm/mach-clps711x/edb7211-mm.c
index c58e32e..0bea145 100644
--- a/arch/arm/mach-clps711x/edb7211-mm.c
+++ b/arch/arm/mach-clps711x/edb7211-mm.c
@@ -24,7 +24,6 @@
#include <mach/hardware.h>
#include <asm/page.h>
-#include <asm/pgtable.h>
#include <asm/sizes.h>
#include <asm/mach/map.h>
diff --git a/arch/arm/mach-clps711x/fortunet.c b/arch/arm/mach-clps711x/fortunet.c
index 7122b3d..7430e40 100644
--- a/arch/arm/mach-clps711x/fortunet.c
+++ b/arch/arm/mach-clps711x/fortunet.c
@@ -24,7 +24,6 @@
#include <linux/initrd.h>
#include <mach/hardware.h>
-#include <asm/irq.h>
#include <asm/setup.h>
#include <asm/mach-types.h>
diff --git a/arch/arm/mach-davinci/devices.c b/arch/arm/mach-davinci/devices.c
index 3d4b1de..808633f 100644
--- a/arch/arm/mach-davinci/devices.c
+++ b/arch/arm/mach-davinci/devices.c
@@ -20,6 +20,7 @@
#include <mach/hardware.h>
#include <mach/i2c.h>
+#include <mach/irqs.h>
static struct resource i2c_resources[] = {
{
diff --git a/arch/arm/mach-davinci/include/mach/gpio.h b/arch/arm/mach-davinci/include/mach/gpio.h
index b3a2961..b456f07 100644
--- a/arch/arm/mach-davinci/include/mach/gpio.h
+++ b/arch/arm/mach-davinci/include/mach/gpio.h
@@ -16,6 +16,7 @@
#include <linux/io.h>
#include <asm-generic/gpio.h>
#include <mach/hardware.h>
+#include <mach/irqs.h>
/*
* basic gpio routines
diff --git a/arch/arm/mach-footbridge/common.c b/arch/arm/mach-footbridge/common.c
index 36ff06d..b97f529 100644
--- a/arch/arm/mach-footbridge/common.c
+++ b/arch/arm/mach-footbridge/common.c
@@ -28,12 +28,17 @@
#include "common.h"
-extern void __init isa_init_irq(unsigned int irq);
-
unsigned int mem_fclk_21285 = 50000000;
EXPORT_SYMBOL(mem_fclk_21285);
+static void __init early_fclk(char **arg)
+{
+ mem_fclk_21285 = simple_strtoul(*arg, arg, 0);
+}
+
+__early_param("mem_fclk_21285=", early_fclk);
+
static int __init parse_tag_memclk(const struct tag *tag)
{
mem_fclk_21285 = tag->u.memclk.fmemclk;
diff --git a/arch/arm/mach-footbridge/common.h b/arch/arm/mach-footbridge/common.h
index 580e31b..b05e662 100644
--- a/arch/arm/mach-footbridge/common.h
+++ b/arch/arm/mach-footbridge/common.h
@@ -7,3 +7,4 @@
extern void footbridge_map_io(void);
extern void footbridge_init_irq(void);
+extern void isa_init_irq(unsigned int irq);
diff --git a/arch/arm/mach-footbridge/dc21285.c b/arch/arm/mach-footbridge/dc21285.c
index 1330860..3ffa548 100644
--- a/arch/arm/mach-footbridge/dc21285.c
+++ b/arch/arm/mach-footbridge/dc21285.c
@@ -287,6 +287,9 @@
return pci_scan_bus(0, &dc21285_ops, sys);
}
+#define dc21285_request_irq(_a, _b, _c, _d, _e) \
+ WARN_ON(request_irq(_a, _b, _c, _d, _e) < 0)
+
void __init dc21285_preinit(void)
{
unsigned int mem_size, mem_mask;
@@ -335,16 +338,16 @@
/*
* We don't care if these fail.
*/
- request_irq(IRQ_PCI_SERR, dc21285_serr_irq, IRQF_DISABLED,
- "PCI system error", &serr_timer);
- request_irq(IRQ_PCI_PERR, dc21285_parity_irq, IRQF_DISABLED,
- "PCI parity error", &perr_timer);
- request_irq(IRQ_PCI_ABORT, dc21285_abort_irq, IRQF_DISABLED,
- "PCI abort", NULL);
- request_irq(IRQ_DISCARD_TIMER, dc21285_discard_irq, IRQF_DISABLED,
- "Discard timer", NULL);
- request_irq(IRQ_PCI_DPERR, dc21285_dparity_irq, IRQF_DISABLED,
- "PCI data parity", NULL);
+ dc21285_request_irq(IRQ_PCI_SERR, dc21285_serr_irq, IRQF_DISABLED,
+ "PCI system error", &serr_timer);
+ dc21285_request_irq(IRQ_PCI_PERR, dc21285_parity_irq, IRQF_DISABLED,
+ "PCI parity error", &perr_timer);
+ dc21285_request_irq(IRQ_PCI_ABORT, dc21285_abort_irq, IRQF_DISABLED,
+ "PCI abort", NULL);
+ dc21285_request_irq(IRQ_DISCARD_TIMER, dc21285_discard_irq, IRQF_DISABLED,
+ "Discard timer", NULL);
+ dc21285_request_irq(IRQ_PCI_DPERR, dc21285_dparity_irq, IRQF_DISABLED,
+ "PCI data parity", NULL);
if (cfn_mode) {
static struct resource csrio;
diff --git a/arch/arm/mach-footbridge/isa-irq.c b/arch/arm/mach-footbridge/isa-irq.c
index 9ee80a2..8bfd06a 100644
--- a/arch/arm/mach-footbridge/isa-irq.c
+++ b/arch/arm/mach-footbridge/isa-irq.c
@@ -28,6 +28,8 @@
#include <asm/irq.h>
#include <asm/mach-types.h>
+#include "common.h"
+
static void isa_mask_pic_lo_irq(unsigned int irq)
{
unsigned int mask = 1 << (irq & 7);
diff --git a/arch/arm/mach-h720x/h7202-eval.c b/arch/arm/mach-h720x/h7202-eval.c
index 56161d5..8c0ba99 100644
--- a/arch/arm/mach-h720x/h7202-eval.c
+++ b/arch/arm/mach-h720x/h7202-eval.c
@@ -25,6 +25,7 @@
#include <asm/page.h>
#include <asm/pgtable.h>
#include <asm/mach/arch.h>
+#include <mach/irqs.h>
#include <mach/hardware.h>
#include "common.h"
diff --git a/arch/arm/mach-kirkwood/common.c b/arch/arm/mach-kirkwood/common.c
index 7b8ef97..b3404b7 100644
--- a/arch/arm/mach-kirkwood/common.c
+++ b/arch/arm/mach-kirkwood/common.c
@@ -698,6 +698,7 @@
printk(KERN_INFO "Kirkwood: %s, TCLK=%d.\n",
kirkwood_id(), kirkwood_tclk);
kirkwood_ge00_shared_data.t_clk = kirkwood_tclk;
+ kirkwood_ge01_shared_data.t_clk = kirkwood_tclk;
kirkwood_spi_plat_data.tclk = kirkwood_tclk;
kirkwood_uart0_data[0].uartclk = kirkwood_tclk;
kirkwood_uart1_data[0].uartclk = kirkwood_tclk;
diff --git a/arch/arm/mach-kirkwood/pcie.c b/arch/arm/mach-kirkwood/pcie.c
index f6b08f2..73fccac 100644
--- a/arch/arm/mach-kirkwood/pcie.c
+++ b/arch/arm/mach-kirkwood/pcie.c
@@ -11,6 +11,7 @@
#include <linux/kernel.h>
#include <linux/pci.h>
#include <linux/mbus.h>
+#include <asm/irq.h>
#include <asm/mach/pci.h>
#include <plat/pcie.h>
#include "common.h"
diff --git a/arch/arm/mach-ks8695/devices.c b/arch/arm/mach-ks8695/devices.c
index 36ab0fd..b89fb6d 100644
--- a/arch/arm/mach-ks8695/devices.c
+++ b/arch/arm/mach-ks8695/devices.c
@@ -22,6 +22,7 @@
#include <linux/platform_device.h>
+#include <mach/irqs.h>
#include <mach/regs-wan.h>
#include <mach/regs-lan.h>
#include <mach/regs-hpna.h>
diff --git a/arch/arm/mach-msm/devices.c b/arch/arm/mach-msm/devices.c
index f2a74b9..31b6b30 100644
--- a/arch/arm/mach-msm/devices.c
+++ b/arch/arm/mach-msm/devices.c
@@ -16,6 +16,7 @@
#include <linux/kernel.h>
#include <linux/platform_device.h>
+#include <mach/irqs.h>
#include <mach/msm_iomap.h>
#include "devices.h"
diff --git a/arch/arm/mach-mv78xx0/pcie.c b/arch/arm/mach-mv78xx0/pcie.c
index 430ea84..aad3a7a 100644
--- a/arch/arm/mach-mv78xx0/pcie.c
+++ b/arch/arm/mach-mv78xx0/pcie.c
@@ -11,6 +11,7 @@
#include <linux/kernel.h>
#include <linux/pci.h>
#include <linux/mbus.h>
+#include <asm/irq.h>
#include <asm/mach/pci.h>
#include <plat/pcie.h>
#include "common.h"
diff --git a/arch/arm/mach-mx2/devices.c b/arch/arm/mach-mx2/devices.c
index af121f5..2f9240b 100644
--- a/arch/arm/mach-mx2/devices.c
+++ b/arch/arm/mach-mx2/devices.c
@@ -32,6 +32,7 @@
#include <linux/platform_device.h>
#include <linux/gpio.h>
+#include <mach/irqs.h>
#include <mach/hardware.h>
/*
diff --git a/arch/arm/mach-mx3/devices.c b/arch/arm/mach-mx3/devices.c
index 1d46cb4..f842880 100644
--- a/arch/arm/mach-mx3/devices.c
+++ b/arch/arm/mach-mx3/devices.c
@@ -22,6 +22,7 @@
#include <linux/serial.h>
#include <linux/gpio.h>
#include <mach/hardware.h>
+#include <mach/irqs.h>
#include <mach/imx-uart.h>
static struct resource uart0[] = {
diff --git a/arch/arm/mach-netx/fb.c b/arch/arm/mach-netx/fb.c
index ea8fa88..1d844e2 100644
--- a/arch/arm/mach-netx/fb.c
+++ b/arch/arm/mach-netx/fb.c
@@ -24,6 +24,8 @@
#include <linux/amba/clcd.h>
#include <linux/err.h>
+#include <asm/irq.h>
+
#include <mach/netx-regs.h>
#include <mach/hardware.h>
diff --git a/arch/arm/mach-netx/time.c b/arch/arm/mach-netx/time.c
index d51d627..f201fdd 100644
--- a/arch/arm/mach-netx/time.c
+++ b/arch/arm/mach-netx/time.c
@@ -163,7 +163,7 @@
* Adding some safety ... */
netx_clockevent.min_delta_ns =
clockevent_delta2ns(0xa00, &netx_clockevent);
- netx_clockevent.cpumask = cpumask_of_cpu(0);
+ netx_clockevent.cpumask = cpumask_of(0);
clockevents_register_device(&netx_clockevent);
}
diff --git a/arch/arm/mach-netx/xc.c b/arch/arm/mach-netx/xc.c
index 8fc6205..181a78b 100644
--- a/arch/arm/mach-netx/xc.c
+++ b/arch/arm/mach-netx/xc.c
@@ -24,6 +24,7 @@
#include <linux/io.h>
#include <mach/hardware.h>
+#include <mach/irqs.h>
#include <mach/netx-regs.h>
#include <mach/xc.h>
diff --git a/arch/arm/mach-omap1/mcbsp.c b/arch/arm/mach-omap1/mcbsp.c
index 7de7c69..4474da7 100644
--- a/arch/arm/mach-omap1/mcbsp.c
+++ b/arch/arm/mach-omap1/mcbsp.c
@@ -18,6 +18,7 @@
#include <linux/platform_device.h>
#include <mach/dma.h>
+#include <mach/irqs.h>
#include <mach/mux.h>
#include <mach/cpu.h>
#include <mach/mcbsp.h>
diff --git a/arch/arm/mach-omap2/mcbsp.c b/arch/arm/mach-omap2/mcbsp.c
index cae3ebe..acdc709 100644
--- a/arch/arm/mach-omap2/mcbsp.c
+++ b/arch/arm/mach-omap2/mcbsp.c
@@ -18,6 +18,7 @@
#include <linux/platform_device.h>
#include <mach/dma.h>
+#include <mach/irqs.h>
#include <mach/mux.h>
#include <mach/cpu.h>
#include <mach/mcbsp.h>
diff --git a/arch/arm/mach-orion5x/pci.c b/arch/arm/mach-orion5x/pci.c
index a7b7d77..d0a785a 100644
--- a/arch/arm/mach-orion5x/pci.c
+++ b/arch/arm/mach-orion5x/pci.c
@@ -13,6 +13,7 @@
#include <linux/kernel.h>
#include <linux/pci.h>
#include <linux/mbus.h>
+#include <asm/irq.h>
#include <asm/mach/pci.h>
#include <plat/pcie.h>
#include "common.h"
diff --git a/arch/arm/mach-pnx4008/gpio.c b/arch/arm/mach-pnx4008/gpio.c
index 015cc21..f219914 100644
--- a/arch/arm/mach-pnx4008/gpio.c
+++ b/arch/arm/mach-pnx4008/gpio.c
@@ -18,6 +18,7 @@
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/io.h>
+#include <mach/hardware.h>
#include <mach/platform.h>
#include <mach/gpio.h>
diff --git a/arch/arm/mach-pnx4008/i2c.c b/arch/arm/mach-pnx4008/i2c.c
index 87c0932..f3fea29 100644
--- a/arch/arm/mach-pnx4008/i2c.c
+++ b/arch/arm/mach-pnx4008/i2c.c
@@ -15,6 +15,7 @@
#include <linux/platform_device.h>
#include <linux/err.h>
#include <mach/platform.h>
+#include <mach/irqs.h>
#include <mach/i2c.h>
static int set_clock_run(struct platform_device *pdev)
diff --git a/arch/arm/mach-pxa/corgi.c b/arch/arm/mach-pxa/corgi.c
index c5e28a4..a8d91b6 100644
--- a/arch/arm/mach-pxa/corgi.c
+++ b/arch/arm/mach-pxa/corgi.c
@@ -27,6 +27,7 @@
#include <linux/spi/spi.h>
#include <linux/spi/ads7846.h>
#include <linux/spi/corgi_lcd.h>
+#include <linux/mtd/sharpsl.h>
#include <video/w100fb.h>
#include <asm/setup.h>
@@ -542,6 +543,55 @@
static inline void corgi_init_spi(void) {}
#endif
+static struct mtd_partition sharpsl_nand_partitions[] = {
+ {
+ .name = "System Area",
+ .offset = 0,
+ .size = 7 * 1024 * 1024,
+ },
+ {
+ .name = "Root Filesystem",
+ .offset = 7 * 1024 * 1024,
+ .size = 25 * 1024 * 1024,
+ },
+ {
+ .name = "Home Filesystem",
+ .offset = MTDPART_OFS_APPEND,
+ .size = MTDPART_SIZ_FULL,
+ },
+};
+
+static uint8_t scan_ff_pattern[] = { 0xff, 0xff };
+
+static struct nand_bbt_descr sharpsl_bbt = {
+ .options = 0,
+ .offs = 4,
+ .len = 2,
+ .pattern = scan_ff_pattern
+};
+
+static struct sharpsl_nand_platform_data sharpsl_nand_platform_data = {
+ .badblock_pattern = &sharpsl_bbt,
+ .partitions = sharpsl_nand_partitions,
+ .nr_partitions = ARRAY_SIZE(sharpsl_nand_partitions),
+};
+
+static struct resource sharpsl_nand_resources[] = {
+ {
+ .start = 0x0C000000,
+ .end = 0x0C000FFF,
+ .flags = IORESOURCE_MEM,
+ },
+};
+
+static struct platform_device sharpsl_nand_device = {
+ .name = "sharpsl-nand",
+ .id = -1,
+ .resource = sharpsl_nand_resources,
+ .num_resources = ARRAY_SIZE(sharpsl_nand_resources),
+ .dev.platform_data = &sharpsl_nand_platform_data,
+};
+
static struct mtd_partition sharpsl_rom_parts[] = {
{
.name ="Boot PROM Filesystem",
@@ -577,6 +627,7 @@
&corgifb_device,
&corgikbd_device,
&corgiled_device,
+ &sharpsl_nand_device,
&sharpsl_rom_device,
};
@@ -617,6 +668,9 @@
platform_scoop_config = &corgi_pcmcia_config;
+ if (machine_is_husky())
+ sharpsl_nand_partitions[1].size = 53 * 1024 * 1024;
+
platform_add_devices(devices, ARRAY_SIZE(devices));
}
diff --git a/arch/arm/mach-pxa/e350.c b/arch/arm/mach-pxa/e350.c
index 2511293..edcd9d5 100644
--- a/arch/arm/mach-pxa/e350.c
+++ b/arch/arm/mach-pxa/e350.c
@@ -20,6 +20,7 @@
#include <asm/mach/arch.h>
#include <asm/mach-types.h>
+#include <mach/irqs.h>
#include <mach/mfp-pxa25x.h>
#include <mach/pxa-regs.h>
#include <mach/hardware.h>
diff --git a/arch/arm/mach-pxa/e400.c b/arch/arm/mach-pxa/e400.c
index bed0336..77bb8e2 100644
--- a/arch/arm/mach-pxa/e400.c
+++ b/arch/arm/mach-pxa/e400.c
@@ -28,6 +28,7 @@
#include <mach/eseries-gpio.h>
#include <mach/pxafb.h>
#include <mach/udc.h>
+#include <mach/irqs.h>
#include "generic.h"
#include "eseries.h"
diff --git a/arch/arm/mach-pxa/e740.c b/arch/arm/mach-pxa/e740.c
index b00d670..6d48e00 100644
--- a/arch/arm/mach-pxa/e740.c
+++ b/arch/arm/mach-pxa/e740.c
@@ -30,6 +30,7 @@
#include <mach/eseries-gpio.h>
#include <mach/udc.h>
#include <mach/irda.h>
+#include <mach/irqs.h>
#include "generic.h"
#include "eseries.h"
diff --git a/arch/arm/mach-pxa/e750.c b/arch/arm/mach-pxa/e750.c
index 84d7c1a..be1ab8e 100644
--- a/arch/arm/mach-pxa/e750.c
+++ b/arch/arm/mach-pxa/e750.c
@@ -29,6 +29,7 @@
#include <mach/eseries-gpio.h>
#include <mach/udc.h>
#include <mach/irda.h>
+#include <mach/irqs.h>
#include "generic.h"
#include "eseries.h"
@@ -105,6 +106,57 @@
.resource = e750_fb_resources,
};
+/* -------------------- e750 MFP parameters -------------------- */
+
+static unsigned long e750_pin_config[] __initdata = {
+ /* Chip selects */
+ GPIO15_nCS_1, /* CS1 - Flash */
+ GPIO79_nCS_3, /* CS3 - IMAGEON */
+ GPIO80_nCS_4, /* CS4 - TMIO */
+
+ /* Clocks */
+ GPIO11_3_6MHz,
+
+ /* BTUART */
+ GPIO42_BTUART_RXD,
+ GPIO43_BTUART_TXD,
+ GPIO44_BTUART_CTS,
+
+ /* TMIO controller */
+ GPIO19_GPIO, /* t7l66xb #PCLR */
+ GPIO45_GPIO, /* t7l66xb #SUSPEND (NOT BTUART!) */
+
+ /* UDC */
+ GPIO13_GPIO,
+ GPIO3_GPIO,
+
+ /* IrDA */
+ GPIO38_GPIO | MFP_LPM_DRIVE_HIGH,
+
+ /* PC Card */
+ GPIO8_GPIO, /* CD0 */
+ GPIO44_GPIO, /* CD1 */
+ GPIO11_GPIO, /* IRQ0 */
+ GPIO6_GPIO, /* IRQ1 */
+ GPIO27_GPIO, /* RST0 */
+ GPIO24_GPIO, /* RST1 */
+ GPIO20_GPIO, /* PWR0 */
+ GPIO23_GPIO, /* PWR1 */
+ GPIO48_nPOE,
+ GPIO49_nPWE,
+ GPIO50_nPIOR,
+ GPIO51_nPIOW,
+ GPIO52_nPCE_1,
+ GPIO53_nPCE_2,
+ GPIO54_nPSKTSEL,
+ GPIO55_nPREG,
+ GPIO56_nPWAIT,
+ GPIO57_nIOIS16,
+
+ /* wakeup */
+ GPIO0_GPIO | WAKEUP_ON_EDGE_RISE,
+};
+
/* ----------------- e750 tc6393xb parameters ------------------ */
static struct tc6393xb_platform_data e750_tc6393xb_info = {
@@ -137,6 +189,7 @@
static void __init e750_init(void)
{
+ pxa2xx_mfp_config(ARRAY_AND_SIZE(e750_pin_config));
clk_add_alias("CLK_CK3P6MI", &e750_tc6393xb_device.dev,
"GPIO11_CLK", NULL),
eseries_get_tmio_gpios();
diff --git a/arch/arm/mach-pxa/e800.c b/arch/arm/mach-pxa/e800.c
index 9a86a42..cc9b129 100644
--- a/arch/arm/mach-pxa/e800.c
+++ b/arch/arm/mach-pxa/e800.c
@@ -28,6 +28,7 @@
#include <mach/hardware.h>
#include <mach/eseries-gpio.h>
#include <mach/udc.h>
+#include <mach/irqs.h>
#include "generic.h"
#include "eseries.h"
diff --git a/arch/arm/mach-pxa/include/mach/pxa3xx-regs.h b/arch/arm/mach-pxa/include/mach/pxa3xx-regs.h
index b1fcd10..bcf3fb2 100644
--- a/arch/arm/mach-pxa/include/mach/pxa3xx-regs.h
+++ b/arch/arm/mach-pxa/include/mach/pxa3xx-regs.h
@@ -193,10 +193,8 @@
#define CKEN_MINI_IM 48 /* < Mini-IM */
#define CKEN_MINI_LCD 49 /* < Mini LCD */
-#if defined(CONFIG_CPU_PXA310)
#define CKEN_MMC3 5 /* < MMC3 Clock Enable */
#define CKEN_MVED 43 /* < MVED clock enable */
-#endif
/* Note: GCU clock enable bit differs on PXA300/PXA310 and PXA320 */
#define PXA300_CKEN_GRAPHICS 42 /* Graphics controller clock enable */
diff --git a/arch/arm/mach-pxa/poodle.c b/arch/arm/mach-pxa/poodle.c
index ae88855..f9093be 100644
--- a/arch/arm/mach-pxa/poodle.c
+++ b/arch/arm/mach-pxa/poodle.c
@@ -24,6 +24,7 @@
#include <linux/gpio.h>
#include <linux/spi/spi.h>
#include <linux/spi/ads7846.h>
+#include <linux/mtd/sharpsl.h>
#include <mach/hardware.h>
#include <asm/mach-types.h>
@@ -414,6 +415,55 @@
.lcd_conn = LCD_COLOR_TFT_16BPP,
};
+static struct mtd_partition sharpsl_nand_partitions[] = {
+ {
+ .name = "System Area",
+ .offset = 0,
+ .size = 7 * 1024 * 1024,
+ },
+ {
+ .name = "Root Filesystem",
+ .offset = 7 * 1024 * 1024,
+ .size = 22 * 1024 * 1024,
+ },
+ {
+ .name = "Home Filesystem",
+ .offset = MTDPART_OFS_APPEND,
+ .size = MTDPART_SIZ_FULL,
+ },
+};
+
+static uint8_t scan_ff_pattern[] = { 0xff, 0xff };
+
+static struct nand_bbt_descr sharpsl_bbt = {
+ .options = 0,
+ .offs = 4,
+ .len = 2,
+ .pattern = scan_ff_pattern
+};
+
+static struct sharpsl_nand_platform_data sharpsl_nand_platform_data = {
+ .badblock_pattern = &sharpsl_bbt,
+ .partitions = sharpsl_nand_partitions,
+ .nr_partitions = ARRAY_SIZE(sharpsl_nand_partitions),
+};
+
+static struct resource sharpsl_nand_resources[] = {
+ {
+ .start = 0x0C000000,
+ .end = 0x0C000FFF,
+ .flags = IORESOURCE_MEM,
+ },
+};
+
+static struct platform_device sharpsl_nand_device = {
+ .name = "sharpsl-nand",
+ .id = -1,
+ .resource = sharpsl_nand_resources,
+ .num_resources = ARRAY_SIZE(sharpsl_nand_resources),
+ .dev.platform_data = &sharpsl_nand_platform_data,
+};
+
static struct mtd_partition sharpsl_rom_parts[] = {
{
.name ="Boot PROM Filesystem",
@@ -447,6 +497,7 @@
static struct platform_device *devices[] __initdata = {
&poodle_locomo_device,
&poodle_scoop_device,
+ &sharpsl_nand_device,
&sharpsl_rom_device,
};
diff --git a/arch/arm/mach-pxa/spitz.c b/arch/arm/mach-pxa/spitz.c
index 7299d87..6d447c9 100644
--- a/arch/arm/mach-pxa/spitz.c
+++ b/arch/arm/mach-pxa/spitz.c
@@ -31,6 +31,7 @@
#include <linux/spi/spi.h>
#include <linux/spi/ads7846.h>
#include <linux/spi/corgi_lcd.h>
+#include <linux/mtd/sharpsl.h>
#include <asm/setup.h>
#include <asm/memory.h>
@@ -613,6 +614,54 @@
.lcd_conn = LCD_COLOR_TFT_16BPP | LCD_ALTERNATE_MAPPING,
};
+static struct mtd_partition sharpsl_nand_partitions[] = {
+ {
+ .name = "System Area",
+ .offset = 0,
+ .size = 7 * 1024 * 1024,
+ },
+ {
+ .name = "Root Filesystem",
+ .offset = 7 * 1024 * 1024,
+ },
+ {
+ .name = "Home Filesystem",
+ .offset = MTDPART_OFS_APPEND,
+ .size = MTDPART_SIZ_FULL,
+ },
+};
+
+static uint8_t scan_ff_pattern[] = { 0xff, 0xff };
+
+static struct nand_bbt_descr sharpsl_bbt = {
+ .options = 0,
+ .offs = 4,
+ .len = 2,
+ .pattern = scan_ff_pattern
+};
+
+static struct sharpsl_nand_platform_data sharpsl_nand_platform_data = {
+ .badblock_pattern = &sharpsl_bbt,
+ .partitions = sharpsl_nand_partitions,
+ .nr_partitions = ARRAY_SIZE(sharpsl_nand_partitions),
+};
+
+static struct resource sharpsl_nand_resources[] = {
+ {
+ .start = 0x0C000000,
+ .end = 0x0C000FFF,
+ .flags = IORESOURCE_MEM,
+ },
+};
+
+static struct platform_device sharpsl_nand_device = {
+ .name = "sharpsl-nand",
+ .id = -1,
+ .resource = sharpsl_nand_resources,
+ .num_resources = ARRAY_SIZE(sharpsl_nand_resources),
+ .dev.platform_data = &sharpsl_nand_platform_data,
+};
+
static struct mtd_partition sharpsl_rom_parts[] = {
{
@@ -648,6 +697,7 @@
&spitzscoop_device,
&spitzkbd_device,
&spitzled_device,
+ &sharpsl_nand_device,
&sharpsl_rom_device,
};
@@ -671,6 +721,14 @@
pm_power_off = spitz_poweroff;
arm_pm_restart = spitz_restart;
+ if (machine_is_spitz()) {
+ sharpsl_nand_partitions[1].size = 5 * 1024 * 1024;
+ } else if (machine_is_akita()) {
+ sharpsl_nand_partitions[1].size = 58 * 1024 * 1024;
+ } else if (machine_is_borzoi()) {
+ sharpsl_nand_partitions[1].size = 32 * 1024 * 1024;
+ }
+
PMCR = 0x00;
/* Stop 3.6MHz and drive HIGH to PCMCIA and CS */
@@ -715,10 +773,29 @@
},
};
+static struct nand_bbt_descr sharpsl_akita_bbt = {
+ .options = 0,
+ .offs = 4,
+ .len = 1,
+ .pattern = scan_ff_pattern
+};
+
+static struct nand_ecclayout akita_oobinfo = {
+ .eccbytes = 24,
+ .eccpos = {
+ 0x5, 0x1, 0x2, 0x3, 0x6, 0x7, 0x15, 0x11,
+ 0x12, 0x13, 0x16, 0x17, 0x25, 0x21, 0x22, 0x23,
+ 0x26, 0x27, 0x35, 0x31, 0x32, 0x33, 0x36, 0x37},
+ .oobfree = {{0x08, 0x09}}
+};
+
static void __init akita_init(void)
{
spitz_ficp_platform_data.transceiver_mode = akita_irda_transceiver_mode;
+ sharpsl_nand_platform_data.badblock_pattern = &sharpsl_akita_bbt;
+ sharpsl_nand_platform_data.ecc_layout = &akita_oobinfo;
+
/* We just pretend the second element of the array doesn't exist */
spitz_pcmcia_config.num_devs = 1;
platform_scoop_config = &spitz_pcmcia_config;
diff --git a/arch/arm/mach-realview/platsmp.c b/arch/arm/mach-realview/platsmp.c
index 8fce85f..ea3c755 100644
--- a/arch/arm/mach-realview/platsmp.c
+++ b/arch/arm/mach-realview/platsmp.c
@@ -12,6 +12,7 @@
#include <linux/errno.h>
#include <linux/delay.h>
#include <linux/device.h>
+#include <linux/jiffies.h>
#include <linux/smp.h>
#include <linux/io.h>
diff --git a/arch/arm/mach-s3c2410/include/mach/gpio.h b/arch/arm/mach-s3c2410/include/mach/gpio.h
index e0349af..00476a5 100644
--- a/arch/arm/mach-s3c2410/include/mach/gpio.h
+++ b/arch/arm/mach-s3c2410/include/mach/gpio.h
@@ -14,6 +14,7 @@
#define gpio_get_value __gpio_get_value
#define gpio_set_value __gpio_set_value
#define gpio_cansleep __gpio_cansleep
+#define gpio_to_irq __gpio_to_irq
/* some boards require extra gpio capacity to support external
* devices that need GPIO.
diff --git a/arch/arm/mach-s3c2410/include/mach/irqs.h b/arch/arm/mach-s3c2410/include/mach/irqs.h
index 9565903..49efce8 100644
--- a/arch/arm/mach-s3c2410/include/mach/irqs.h
+++ b/arch/arm/mach-s3c2410/include/mach/irqs.h
@@ -12,10 +12,6 @@
#ifndef __ASM_ARCH_IRQS_H
#define __ASM_ARCH_IRQS_H __FILE__
-#ifndef __ASM_ARM_IRQ_H
-#error "Do not include this directly, instead #include <asm/irq.h>"
-#endif
-
/* we keep the first set of CPU IRQs out of the range of
* the ISA space, so that the PC104 has them to itself
* and we don't end up having to do horrible things to the
diff --git a/arch/arm/mach-s3c2440/mach-at2440evb.c b/arch/arm/mach-s3c2440/mach-at2440evb.c
index 0a6d0a5..315c42e 100644
--- a/arch/arm/mach-s3c2440/mach-at2440evb.c
+++ b/arch/arm/mach-s3c2440/mach-at2440evb.c
@@ -47,7 +47,7 @@
#include <plat/clock.h>
#include <plat/devs.h>
#include <plat/cpu.h>
-#include <asm/plat-s3c24xx/mci.h>
+#include <plat/mci.h>
static struct map_desc at2440evb_iodesc[] __initdata = {
/* Nothing here */
diff --git a/arch/arm/mach-s3c6400/include/mach/irqs.h b/arch/arm/mach-s3c6400/include/mach/irqs.h
index b38c47c..4c97f9a 100644
--- a/arch/arm/mach-s3c6400/include/mach/irqs.h
+++ b/arch/arm/mach-s3c6400/include/mach/irqs.h
@@ -11,10 +11,6 @@
#ifndef __ASM_ARCH_IRQS_H
#define __ASM_ARCH_IRQS_H __FILE__
-#ifndef __ASM_ARM_IRQ_H
-#error "Do not include this directly, instead #include <asm/irq.h>"
-#endif
-
#include <plat/irqs.h>
#endif /* __ASM_ARCH_IRQ_H */
diff --git a/arch/arm/plat-omap/i2c.c b/arch/arm/plat-omap/i2c.c
index 89a6ab0..467531e 100644
--- a/arch/arm/plat-omap/i2c.c
+++ b/arch/arm/plat-omap/i2c.c
@@ -26,6 +26,7 @@
#include <linux/kernel.h>
#include <linux/platform_device.h>
#include <linux/i2c.h>
+#include <mach/irqs.h>
#include <mach/mux.h>
#define OMAP_I2C_SIZE 0x3f
diff --git a/arch/arm/plat-s3c/dev-fb.c b/arch/arm/plat-s3c/dev-fb.c
index 0454b8e..a90198f 100644
--- a/arch/arm/plat-s3c/dev-fb.c
+++ b/arch/arm/plat-s3c/dev-fb.c
@@ -16,6 +16,7 @@
#include <linux/platform_device.h>
#include <linux/fb.h>
+#include <mach/irqs.h>
#include <mach/map.h>
#include <mach/regs-fb.h>
diff --git a/arch/arm/plat-s3c/dev-i2c0.c b/arch/arm/plat-s3c/dev-i2c0.c
index 2c0128c..fe32707 100644
--- a/arch/arm/plat-s3c/dev-i2c0.c
+++ b/arch/arm/plat-s3c/dev-i2c0.c
@@ -15,6 +15,7 @@
#include <linux/string.h>
#include <linux/platform_device.h>
+#include <mach/irqs.h>
#include <mach/map.h>
#include <plat/regs-iic.h>
diff --git a/arch/arm/plat-s3c/dev-i2c1.c b/arch/arm/plat-s3c/dev-i2c1.c
index 9658fb0..2387fbf 100644
--- a/arch/arm/plat-s3c/dev-i2c1.c
+++ b/arch/arm/plat-s3c/dev-i2c1.c
@@ -15,6 +15,7 @@
#include <linux/string.h>
#include <linux/platform_device.h>
+#include <mach/irqs.h>
#include <mach/map.h>
#include <plat/regs-iic.h>
diff --git a/arch/arm/plat-s3c24xx/gpiolib.c b/arch/arm/plat-s3c24xx/gpiolib.c
index f95c6c9..94a341a 100644
--- a/arch/arm/plat-s3c24xx/gpiolib.c
+++ b/arch/arm/plat-s3c24xx/gpiolib.c
@@ -59,6 +59,22 @@
return 0;
}
+static int s3c24xx_gpiolib_bankf_toirq(struct gpio_chip *chip, unsigned offset)
+{
+ if (offset < 4)
+ return IRQ_EINT0 + offset;
+
+ if (offset < 8)
+ return IRQ_EINT4 + offset - 4;
+
+ return -EINVAL;
+}
+
+static int s3c24xx_gpiolib_bankg_toirq(struct gpio_chip *chip, unsigned offset)
+{
+ return IRQ_EINT8 + offset;
+}
+
struct s3c_gpio_chip s3c24xx_gpios[] = {
[0] = {
.base = S3C24XX_GPIO_BASE(S3C2410_GPA0),
@@ -114,6 +130,7 @@
.owner = THIS_MODULE,
.label = "GPIOF",
.ngpio = 8,
+ .to_irq = s3c24xx_gpiolib_bankf_toirq,
},
},
[6] = {
@@ -123,6 +140,7 @@
.owner = THIS_MODULE,
.label = "GPIOG",
.ngpio = 10,
+ .to_irq = s3c24xx_gpiolib_bankg_toirq,
},
},
};
diff --git a/arch/arm/plat-s3c24xx/pwm.c b/arch/arm/plat-s3c24xx/pwm.c
index ec56b88..0120b76 100644
--- a/arch/arm/plat-s3c24xx/pwm.c
+++ b/arch/arm/plat-s3c24xx/pwm.c
@@ -19,6 +19,8 @@
#include <linux/io.h>
#include <linux/pwm.h>
+#include <mach/irqs.h>
+
#include <plat/devs.h>
#include <plat/regs-timer.h>
diff --git a/arch/arm/plat-s3c64xx/include/plat/irqs.h b/arch/arm/plat-s3c64xx/include/plat/irqs.h
index 02e8dd4..2846f55 100644
--- a/arch/arm/plat-s3c64xx/include/plat/irqs.h
+++ b/arch/arm/plat-s3c64xx/include/plat/irqs.h
@@ -191,7 +191,7 @@
#define IRQ_EINT_GROUP8_BASE (IRQ_EINT_GROUP7_BASE + IRQ_EINT_GROUP7_NR)
#define IRQ_EINT_GROUP9_BASE (IRQ_EINT_GROUP8_BASE + IRQ_EINT_GROUP8_NR)
-#define IRQ_EINT_GROUP(group, no) (IRQ_EINT_GROUP##group##__BASE + (x))
+#define IRQ_EINT_GROUP(group, no) (IRQ_EINT_GROUP##group##_BASE + (no))
/* Set the default NR_IRQS */
diff --git a/arch/avr32/mach-at32ap/at32ap700x.c b/arch/avr32/mach-at32ap/at32ap700x.c
index ea7bc1e..3fbfd1e 100644
--- a/arch/avr32/mach-at32ap/at32ap700x.c
+++ b/arch/avr32/mach-at32ap/at32ap700x.c
@@ -1305,7 +1305,7 @@
at32_add_device_mci(unsigned int id, struct mci_platform_data *data)
{
struct platform_device *pdev;
- struct dw_dma_slave *dws;
+ struct dw_dma_slave *dws = &data->dma_slave;
u32 pioa_mask;
u32 piob_mask;
@@ -1324,22 +1324,13 @@
ARRAY_SIZE(atmel_mci0_resource)))
goto fail;
- if (data->dma_slave)
- dws = kmemdup(to_dw_dma_slave(data->dma_slave),
- sizeof(struct dw_dma_slave), GFP_KERNEL);
- else
- dws = kzalloc(sizeof(struct dw_dma_slave), GFP_KERNEL);
-
- dws->slave.dev = &pdev->dev;
- dws->slave.dma_dev = &dw_dmac0_device.dev;
- dws->slave.reg_width = DMA_SLAVE_WIDTH_32BIT;
+ dws->dma_dev = &dw_dmac0_device.dev;
+ dws->reg_width = DW_DMA_SLAVE_WIDTH_32BIT;
dws->cfg_hi = (DWC_CFGH_SRC_PER(0)
| DWC_CFGH_DST_PER(1));
dws->cfg_lo &= ~(DWC_CFGL_HS_DST_POL
| DWC_CFGL_HS_SRC_POL);
- data->dma_slave = &dws->slave;
-
if (platform_device_add_data(pdev, data,
sizeof(struct mci_platform_data)))
goto fail;
diff --git a/arch/ia64/include/asm/acpi-ext.h b/arch/ia64/include/asm/acpi-ext.h
index 734d137..7f8362b 100644
--- a/arch/ia64/include/asm/acpi-ext.h
+++ b/arch/ia64/include/asm/acpi-ext.h
@@ -14,7 +14,6 @@
#define _ASM_IA64_ACPI_EXT_H
#include <linux/types.h>
-#include <acpi/actypes.h>
extern acpi_status hp_acpi_csr_space (acpi_handle, u64 *base, u64 *length);
diff --git a/arch/ia64/include/asm/sn/acpi.h b/arch/ia64/include/asm/sn/acpi.h
index 9ce2801..fd480db2 100644
--- a/arch/ia64/include/asm/sn/acpi.h
+++ b/arch/ia64/include/asm/sn/acpi.h
@@ -9,8 +9,6 @@
#ifndef _ASM_IA64_SN_ACPI_H
#define _ASM_IA64_SN_ACPI_H
-#include "acpi/acglobal.h"
-
extern int sn_acpi_rev;
#define SN_ACPI_BASE_SUPPORT() (sn_acpi_rev >= 0x20101)
diff --git a/arch/ia64/kernel/acpi.c b/arch/ia64/kernel/acpi.c
index 0553648..d541671 100644
--- a/arch/ia64/kernel/acpi.c
+++ b/arch/ia64/kernel/acpi.c
@@ -65,6 +65,7 @@
void (*pm_power_off) (void);
EXPORT_SYMBOL(pm_power_off);
+u32 acpi_rsdt_forced;
unsigned int acpi_cpei_override;
unsigned int acpi_cpei_phys_cpuid;
diff --git a/arch/ia64/sn/kernel/io_acpi_init.c b/arch/ia64/sn/kernel/io_acpi_init.c
index bc610a6..c5a2140 100644
--- a/arch/ia64/sn/kernel/io_acpi_init.c
+++ b/arch/ia64/sn/kernel/io_acpi_init.c
@@ -13,7 +13,6 @@
#include <asm/sn/sn_sal.h>
#include "xtalk/hubdev.h"
#include <linux/acpi.h>
-#include <acpi/acnamesp.h>
/*
@@ -64,6 +63,7 @@
sn_acpi_hubdev_init(acpi_handle handle, u32 depth, void *context, void **ret)
{
struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL };
+ struct acpi_buffer name_buffer = { ACPI_ALLOCATE_BUFFER, NULL };
u64 addr;
struct hubdev_info *hubdev;
struct hubdev_info *hubdev_ptr;
@@ -77,11 +77,12 @@
status = acpi_get_vendor_resource(handle, METHOD_NAME__CRS,
&sn_uuid, &buffer);
if (ACPI_FAILURE(status)) {
+ acpi_get_name(handle, ACPI_FULL_PATHNAME, &name_buffer);
printk(KERN_ERR
"sn_acpi_hubdev_init: acpi_get_vendor_resource() "
- "(0x%x) failed for: ", status);
- acpi_ns_print_node_pathname(handle, NULL);
- printk("\n");
+ "(0x%x) failed for: %s\n", status,
+ (char *)name_buffer.pointer);
+ kfree(name_buffer.pointer);
return AE_OK; /* Continue walking namespace */
}
@@ -89,11 +90,12 @@
vendor = &resource->data.vendor_typed;
if ((vendor->byte_length - sizeof(struct acpi_vendor_uuid)) !=
sizeof(struct hubdev_info *)) {
+ acpi_get_name(handle, ACPI_FULL_PATHNAME, &name_buffer);
printk(KERN_ERR
- "sn_acpi_hubdev_init: Invalid vendor data length: %d for: ",
- vendor->byte_length);
- acpi_ns_print_node_pathname(handle, NULL);
- printk("\n");
+ "sn_acpi_hubdev_init: Invalid vendor data length: "
+ "%d for: %s\n",
+ vendor->byte_length, (char *)name_buffer.pointer);
+ kfree(name_buffer.pointer);
goto exit;
}
@@ -120,6 +122,7 @@
{
u64 addr;
struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL };
+ struct acpi_buffer name_buffer = { ACPI_ALLOCATE_BUFFER, NULL };
acpi_handle handle;
struct pcibus_bussoft *prom_bussoft_ptr;
struct acpi_resource *resource;
@@ -131,11 +134,11 @@
status = acpi_get_vendor_resource(handle, METHOD_NAME__CRS,
&sn_uuid, &buffer);
if (ACPI_FAILURE(status)) {
+ acpi_get_name(handle, ACPI_FULL_PATHNAME, &name_buffer);
printk(KERN_ERR "%s: "
- "acpi_get_vendor_resource() failed (0x%x) for: ",
- __func__, status);
- acpi_ns_print_node_pathname(handle, NULL);
- printk("\n");
+ "acpi_get_vendor_resource() failed (0x%x) for: %s\n",
+ __func__, status, (char *)name_buffer.pointer);
+ kfree(name_buffer.pointer);
return NULL;
}
resource = buffer.pointer;
@@ -168,6 +171,7 @@
{
u64 addr;
struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL };
+ struct acpi_buffer name_buffer = { ACPI_ALLOCATE_BUFFER, NULL };
struct sn_irq_info *irq_info, *irq_info_prom;
struct pcidev_info *pcidev_ptr, *pcidev_prom_ptr;
struct acpi_resource *resource;
@@ -182,11 +186,11 @@
status = acpi_get_vendor_resource(handle, METHOD_NAME__CRS,
&sn_uuid, &buffer);
if (ACPI_FAILURE(status)) {
+ acpi_get_name(handle, ACPI_FULL_PATHNAME, &name_buffer);
printk(KERN_ERR
- "%s: acpi_get_vendor_resource() failed (0x%x) for: ",
- __func__, status);
- acpi_ns_print_node_pathname(handle, NULL);
- printk("\n");
+ "%s: acpi_get_vendor_resource() failed (0x%x) for: %s\n",
+ __func__, status, (char *)name_buffer.pointer);
+ kfree(name_buffer.pointer);
return 1;
}
@@ -194,11 +198,12 @@
vendor = &resource->data.vendor_typed;
if ((vendor->byte_length - sizeof(struct acpi_vendor_uuid)) !=
sizeof(struct pci_devdev_info *)) {
+ acpi_get_name(handle, ACPI_FULL_PATHNAME, &name_buffer);
printk(KERN_ERR
- "%s: Invalid vendor data length: %d for: ",
- __func__, vendor->byte_length);
- acpi_ns_print_node_pathname(handle, NULL);
- printk("\n");
+ "%s: Invalid vendor data length: %d for: %s\n",
+ __func__, vendor->byte_length,
+ (char *)name_buffer.pointer);
+ kfree(name_buffer.pointer);
ret = 1;
goto exit;
}
@@ -239,6 +244,9 @@
acpi_handle parent;
int slot;
acpi_status status;
+ struct acpi_buffer name_buffer = { ACPI_ALLOCATE_BUFFER, NULL };
+
+ acpi_get_name(device_handle, ACPI_FULL_PATHNAME, &name_buffer);
/*
* Do an upward search to find the root bus device, and
@@ -249,9 +257,8 @@
status = acpi_get_parent(child, &parent);
if (ACPI_FAILURE(status)) {
printk(KERN_ERR "%s: acpi_get_parent() failed "
- "(0x%x) for: ", __func__, status);
- acpi_ns_print_node_pathname(child, NULL);
- printk("\n");
+ "(0x%x) for: %s\n", __func__, status,
+ (char *)name_buffer.pointer);
panic("%s: Unable to find host devfn\n", __func__);
}
if (parent == rootbus_handle)
@@ -259,22 +266,20 @@
child = parent;
}
if (!child) {
- printk(KERN_ERR "%s: Unable to find root bus for: ",
- __func__);
- acpi_ns_print_node_pathname(device_handle, NULL);
- printk("\n");
+ printk(KERN_ERR "%s: Unable to find root bus for: %s\n",
+ __func__, (char *)name_buffer.pointer);
BUG();
}
status = acpi_evaluate_integer(child, METHOD_NAME__ADR, NULL, &adr);
if (ACPI_FAILURE(status)) {
- printk(KERN_ERR "%s: Unable to get _ADR (0x%x) for: ",
- __func__, status);
- acpi_ns_print_node_pathname(child, NULL);
- printk("\n");
+ printk(KERN_ERR "%s: Unable to get _ADR (0x%x) for: %s\n",
+ __func__, status, (char *)name_buffer.pointer);
panic("%s: Unable to find host devfn\n", __func__);
}
+ kfree(name_buffer.pointer);
+
slot = (adr >> 16) & 0xffff;
function = adr & 0xffff;
devfn = PCI_DEVFN(slot, function);
@@ -300,27 +305,28 @@
int function;
int slot;
struct sn_pcidev_match *info = context;
+ struct acpi_buffer name_buffer = { ACPI_ALLOCATE_BUFFER, NULL };
status = acpi_evaluate_integer(handle, METHOD_NAME__ADR, NULL,
&adr);
if (ACPI_SUCCESS(status)) {
status = acpi_get_parent(handle, &parent);
if (ACPI_FAILURE(status)) {
+ acpi_get_name(handle, ACPI_FULL_PATHNAME, &name_buffer);
printk(KERN_ERR
- "%s: acpi_get_parent() failed (0x%x) for: ",
- __func__, status);
- acpi_ns_print_node_pathname(handle, NULL);
- printk("\n");
+ "%s: acpi_get_parent() failed (0x%x) for: %s\n",
+ __func__, status, (char *)name_buffer.pointer);
+ kfree(name_buffer.pointer);
return AE_OK;
}
status = acpi_evaluate_integer(parent, METHOD_NAME__BBN,
NULL, &bbn);
if (ACPI_FAILURE(status)) {
+ acpi_get_name(handle, ACPI_FULL_PATHNAME, &name_buffer);
printk(KERN_ERR
- "%s: Failed to find _BBN in parent of: ",
- __func__);
- acpi_ns_print_node_pathname(handle, NULL);
- printk("\n");
+ "%s: Failed to find _BBN in parent of: %s\n",
+ __func__, (char *)name_buffer.pointer);
+ kfree(name_buffer.pointer);
return AE_OK;
}
@@ -350,24 +356,27 @@
acpi_handle rootbus_handle;
unsigned long long segment;
acpi_status status;
+ struct acpi_buffer name_buffer = { ACPI_ALLOCATE_BUFFER, NULL };
rootbus_handle = PCI_CONTROLLER(dev)->acpi_handle;
status = acpi_evaluate_integer(rootbus_handle, METHOD_NAME__SEG, NULL,
&segment);
if (ACPI_SUCCESS(status)) {
if (segment != pci_domain_nr(dev)) {
+ acpi_get_name(rootbus_handle, ACPI_FULL_PATHNAME,
+ &name_buffer);
printk(KERN_ERR
- "%s: Segment number mismatch, 0x%llx vs 0x%x for: ",
- __func__, segment, pci_domain_nr(dev));
- acpi_ns_print_node_pathname(rootbus_handle, NULL);
- printk("\n");
+ "%s: Segment number mismatch, 0x%llx vs 0x%x for: %s\n",
+ __func__, segment, pci_domain_nr(dev),
+ (char *)name_buffer.pointer);
+ kfree(name_buffer.pointer);
return 1;
}
} else {
- printk(KERN_ERR "%s: Unable to get __SEG from: ",
- __func__);
- acpi_ns_print_node_pathname(rootbus_handle, NULL);
- printk("\n");
+ acpi_get_name(rootbus_handle, ACPI_FULL_PATHNAME, &name_buffer);
+ printk(KERN_ERR "%s: Unable to get __SEG from: %s\n",
+ __func__, (char *)name_buffer.pointer);
+ kfree(name_buffer.pointer);
return 1;
}
diff --git a/arch/ia64/sn/kernel/io_common.c b/arch/ia64/sn/kernel/io_common.c
index 8a924a5..0d4ffa4 100644
--- a/arch/ia64/sn/kernel/io_common.c
+++ b/arch/ia64/sn/kernel/io_common.c
@@ -26,7 +26,6 @@
#include <linux/acpi.h>
#include <asm/sn/sn2/sn_hwperf.h>
#include <asm/sn/acpi.h>
-#include "acpi/acglobal.h"
extern void sn_init_cpei_timer(void);
extern void register_sn_procfs(void);
@@ -473,7 +472,7 @@
{
struct acpi_table_header *header = NULL;
- acpi_get_table_by_index(ACPI_TABLE_INDEX_DSDT, &header);
+ acpi_get_table(ACPI_SIG_DSDT, 1, &header);
BUG_ON(header == NULL);
sn_acpi_rev = header->oem_revision;
}
@@ -505,7 +504,7 @@
{
struct acpi_table_header *header;
- (void)acpi_get_table_by_index(ACPI_TABLE_INDEX_DSDT, &header);
+ (void)acpi_get_table(ACPI_SIG_DSDT, 1, &header);
printk(KERN_INFO "ACPI DSDT OEM Rev 0x%x\n",
header->oem_revision);
}
diff --git a/arch/parisc/Makefile b/arch/parisc/Makefile
index 5ddad7b..0d42827 100644
--- a/arch/parisc/Makefile
+++ b/arch/parisc/Makefile
@@ -77,7 +77,7 @@
drivers-$(CONFIG_OPROFILE) += arch/parisc/oprofile/
-PALO := $(shell if which palo; then : ; \
+PALO := $(shell if (which palo 2>&1); then : ; \
elif [ -x /sbin/palo ]; then echo /sbin/palo; \
fi)
diff --git a/arch/parisc/include/asm/Kbuild b/arch/parisc/include/asm/Kbuild
index f88b252..2121d99 100644
--- a/arch/parisc/include/asm/Kbuild
+++ b/arch/parisc/include/asm/Kbuild
@@ -1,3 +1,4 @@
include include/asm-generic/Kbuild.asm
unifdef-y += pdc.h
+unifdef-y += swab.h
diff --git a/arch/parisc/include/asm/byteorder.h b/arch/parisc/include/asm/byteorder.h
index db14831..da66029 100644
--- a/arch/parisc/include/asm/byteorder.h
+++ b/arch/parisc/include/asm/byteorder.h
@@ -1,82 +1,7 @@
#ifndef _PARISC_BYTEORDER_H
#define _PARISC_BYTEORDER_H
-#include <asm/types.h>
-#include <linux/compiler.h>
-
-#ifdef __GNUC__
-
-static __inline__ __attribute_const__ __u16 ___arch__swab16(__u16 x)
-{
- __asm__("dep %0, 15, 8, %0\n\t" /* deposit 00ab -> 0bab */
- "shd %%r0, %0, 8, %0" /* shift 000000ab -> 00ba */
- : "=r" (x)
- : "0" (x));
- return x;
-}
-
-static __inline__ __attribute_const__ __u32 ___arch__swab24(__u32 x)
-{
- __asm__("shd %0, %0, 8, %0\n\t" /* shift xabcxabc -> cxab */
- "dep %0, 15, 8, %0\n\t" /* deposit cxab -> cbab */
- "shd %%r0, %0, 8, %0" /* shift 0000cbab -> 0cba */
- : "=r" (x)
- : "0" (x));
- return x;
-}
-
-static __inline__ __attribute_const__ __u32 ___arch__swab32(__u32 x)
-{
- unsigned int temp;
- __asm__("shd %0, %0, 16, %1\n\t" /* shift abcdabcd -> cdab */
- "dep %1, 15, 8, %1\n\t" /* deposit cdab -> cbab */
- "shd %0, %1, 8, %0" /* shift abcdcbab -> dcba */
- : "=r" (x), "=&r" (temp)
- : "0" (x));
- return x;
-}
-
-
-#if BITS_PER_LONG > 32
-/*
-** From "PA-RISC 2.0 Architecture", HP Professional Books.
-** See Appendix I page 8 , "Endian Byte Swapping".
-**
-** Pretty cool algorithm: (* == zero'd bits)
-** PERMH 01234567 -> 67452301 into %0
-** HSHL 67452301 -> 7*5*3*1* into %1
-** HSHR 67452301 -> *6*4*2*0 into %0
-** OR %0 | %1 -> 76543210 into %0 (all done!)
-*/
-static __inline__ __attribute_const__ __u64 ___arch__swab64(__u64 x) {
- __u64 temp;
- __asm__("permh,3210 %0, %0\n\t"
- "hshl %0, 8, %1\n\t"
- "hshr,u %0, 8, %0\n\t"
- "or %1, %0, %0"
- : "=r" (x), "=&r" (temp)
- : "0" (x));
- return x;
-}
-#define __arch__swab64(x) ___arch__swab64(x)
-#define __BYTEORDER_HAS_U64__
-#elif !defined(__STRICT_ANSI__)
-static __inline__ __attribute_const__ __u64 ___arch__swab64(__u64 x)
-{
- __u32 t1 = ___arch__swab32((__u32) x);
- __u32 t2 = ___arch__swab32((__u32) (x >> 32));
- return (((__u64) t1 << 32) | t2);
-}
-#define __arch__swab64(x) ___arch__swab64(x)
-#define __BYTEORDER_HAS_U64__
-#endif
-
-#define __arch__swab16(x) ___arch__swab16(x)
-#define __arch__swab24(x) ___arch__swab24(x)
-#define __arch__swab32(x) ___arch__swab32(x)
-
-#endif /* __GNUC__ */
-
+#include <asm/swab.h>
#include <linux/byteorder/big_endian.h>
#endif /* _PARISC_BYTEORDER_H */
diff --git a/arch/parisc/include/asm/checksum.h b/arch/parisc/include/asm/checksum.h
index e9639cc..c84b2fc 100644
--- a/arch/parisc/include/asm/checksum.h
+++ b/arch/parisc/include/asm/checksum.h
@@ -182,7 +182,7 @@
#endif
: "=r" (sum), "=r" (saddr), "=r" (daddr), "=r" (len)
: "0" (sum), "1" (saddr), "2" (daddr), "3" (len), "r" (proto)
- : "r19", "r20", "r21", "r22");
+ : "r19", "r20", "r21", "r22", "memory");
return csum_fold(sum);
}
diff --git a/arch/parisc/include/asm/io.h b/arch/parisc/include/asm/io.h
index 55ddb18..d3031d1 100644
--- a/arch/parisc/include/asm/io.h
+++ b/arch/parisc/include/asm/io.h
@@ -4,12 +4,6 @@
#include <linux/types.h>
#include <asm/pgtable.h>
-extern unsigned long parisc_vmerge_boundary;
-extern unsigned long parisc_vmerge_max_size;
-
-#define BIO_VMERGE_BOUNDARY parisc_vmerge_boundary
-#define BIO_VMERGE_MAX_SIZE parisc_vmerge_max_size
-
#define virt_to_phys(a) ((unsigned long)__pa(a))
#define phys_to_virt(a) __va(a)
#define virt_to_bus virt_to_phys
@@ -182,9 +176,9 @@
/* readb can never be const, so use __fswab instead of le*_to_cpu */
#define readb(addr) __raw_readb(addr)
-#define readw(addr) __fswab16(__raw_readw(addr))
-#define readl(addr) __fswab32(__raw_readl(addr))
-#define readq(addr) __fswab64(__raw_readq(addr))
+#define readw(addr) le16_to_cpu(__raw_readw(addr))
+#define readl(addr) le32_to_cpu(__raw_readl(addr))
+#define readq(addr) le64_to_cpu(__raw_readq(addr))
#define writeb(b, addr) __raw_writeb(b, addr)
#define writew(b, addr) __raw_writew(cpu_to_le16(b), addr)
#define writel(b, addr) __raw_writel(cpu_to_le32(b), addr)
diff --git a/arch/parisc/include/asm/mmu_context.h b/arch/parisc/include/asm/mmu_context.h
index 85856c7..354b2ac 100644
--- a/arch/parisc/include/asm/mmu_context.h
+++ b/arch/parisc/include/asm/mmu_context.h
@@ -34,14 +34,19 @@
mm->context = 0;
}
+static inline unsigned long __space_to_prot(mm_context_t context)
+{
+#if SPACEID_SHIFT == 0
+ return context << 1;
+#else
+ return context >> (SPACEID_SHIFT - 1);
+#endif
+}
+
static inline void load_context(mm_context_t context)
{
mtsp(context, 3);
-#if SPACEID_SHIFT == 0
- mtctl(context << 1,8);
-#else
- mtctl(context >> (SPACEID_SHIFT - 1),8);
-#endif
+ mtctl(__space_to_prot(context), 8);
}
static inline void switch_mm(struct mm_struct *prev, struct mm_struct *next, struct task_struct *tsk)
diff --git a/arch/parisc/include/asm/processor.h b/arch/parisc/include/asm/processor.h
index 3c9d348..9d64df8 100644
--- a/arch/parisc/include/asm/processor.h
+++ b/arch/parisc/include/asm/processor.h
@@ -17,6 +17,7 @@
#include <asm/ptrace.h>
#include <asm/types.h>
#include <asm/system.h>
+#include <asm/percpu.h>
#endif /* __ASSEMBLY__ */
#define KERNEL_STACK_SIZE (4*PAGE_SIZE)
@@ -109,8 +110,7 @@
};
extern struct system_cpuinfo_parisc boot_cpu_data;
-extern struct cpuinfo_parisc cpu_data[NR_CPUS];
-#define current_cpu_data cpu_data[smp_processor_id()]
+DECLARE_PER_CPU(struct cpuinfo_parisc, cpu_data);
#define CPU_HVERSION ((boot_cpu_data.hversion >> 4) & 0x0FFF)
diff --git a/arch/parisc/include/asm/swab.h b/arch/parisc/include/asm/swab.h
new file mode 100644
index 0000000..3ff16c5
--- /dev/null
+++ b/arch/parisc/include/asm/swab.h
@@ -0,0 +1,66 @@
+#ifndef _PARISC_SWAB_H
+#define _PARISC_SWAB_H
+
+#include <asm/types.h>
+#include <linux/compiler.h>
+
+#define __SWAB_64_THRU_32__
+
+static inline __attribute_const__ __u16 __arch_swab16(__u16 x)
+{
+ __asm__("dep %0, 15, 8, %0\n\t" /* deposit 00ab -> 0bab */
+ "shd %%r0, %0, 8, %0" /* shift 000000ab -> 00ba */
+ : "=r" (x)
+ : "0" (x));
+ return x;
+}
+#define __arch_swab16 __arch_swab16
+
+static inline __attribute_const__ __u32 __arch_swab24(__u32 x)
+{
+ __asm__("shd %0, %0, 8, %0\n\t" /* shift xabcxabc -> cxab */
+ "dep %0, 15, 8, %0\n\t" /* deposit cxab -> cbab */
+ "shd %%r0, %0, 8, %0" /* shift 0000cbab -> 0cba */
+ : "=r" (x)
+ : "0" (x));
+ return x;
+}
+
+static inline __attribute_const__ __u32 __arch_swab32(__u32 x)
+{
+ unsigned int temp;
+ __asm__("shd %0, %0, 16, %1\n\t" /* shift abcdabcd -> cdab */
+ "dep %1, 15, 8, %1\n\t" /* deposit cdab -> cbab */
+ "shd %0, %1, 8, %0" /* shift abcdcbab -> dcba */
+ : "=r" (x), "=&r" (temp)
+ : "0" (x));
+ return x;
+}
+#define __arch_swab32 __arch_swab32
+
+#if BITS_PER_LONG > 32
+/*
+** From "PA-RISC 2.0 Architecture", HP Professional Books.
+** See Appendix I page 8 , "Endian Byte Swapping".
+**
+** Pretty cool algorithm: (* == zero'd bits)
+** PERMH 01234567 -> 67452301 into %0
+** HSHL 67452301 -> 7*5*3*1* into %1
+** HSHR 67452301 -> *6*4*2*0 into %0
+** OR %0 | %1 -> 76543210 into %0 (all done!)
+*/
+static inline __attribute_const__ __u64 __arch_swab64(__u64 x)
+{
+ __u64 temp;
+ __asm__("permh,3210 %0, %0\n\t"
+ "hshl %0, 8, %1\n\t"
+ "hshr,u %0, 8, %0\n\t"
+ "or %1, %0, %0"
+ : "=r" (x), "=&r" (temp)
+ : "0" (x));
+ return x;
+}
+#define __arch_swab64 __arch_swab64
+#endif /* BITS_PER_LONG > 32 */
+
+#endif /* _PARISC_SWAB_H */
diff --git a/arch/parisc/include/asm/uaccess.h b/arch/parisc/include/asm/uaccess.h
index 4878b95..1c6dbb6 100644
--- a/arch/parisc/include/asm/uaccess.h
+++ b/arch/parisc/include/asm/uaccess.h
@@ -241,4 +241,6 @@
#define __copy_to_user_inatomic __copy_to_user
#define __copy_from_user_inatomic __copy_from_user
+int fixup_exception(struct pt_regs *regs);
+
#endif /* __PARISC_UACCESS_H */
diff --git a/arch/parisc/kernel/drivers.c b/arch/parisc/kernel/drivers.c
index 884b7ce..994bcd9 100644
--- a/arch/parisc/kernel/drivers.c
+++ b/arch/parisc/kernel/drivers.c
@@ -549,6 +549,38 @@
return match_device(to_parisc_driver(drv), to_parisc_device(dev));
}
+static ssize_t make_modalias(struct device *dev, char *buf)
+{
+ const struct parisc_device *padev = to_parisc_device(dev);
+ const struct parisc_device_id *id = &padev->id;
+
+ return sprintf(buf, "parisc:t%02Xhv%04Xrev%02Xsv%08X\n",
+ (u8)id->hw_type, (u16)id->hversion, (u8)id->hversion_rev,
+ (u32)id->sversion);
+}
+
+static int parisc_uevent(struct device *dev, struct kobj_uevent_env *env)
+{
+ const struct parisc_device *padev;
+ char modalias[40];
+
+ if (!dev)
+ return -ENODEV;
+
+ padev = to_parisc_device(dev);
+ if (!padev)
+ return -ENODEV;
+
+ if (add_uevent_var(env, "PARISC_NAME=%s", padev->name))
+ return -ENOMEM;
+
+ make_modalias(dev, modalias);
+ if (add_uevent_var(env, "MODALIAS=%s", modalias))
+ return -ENOMEM;
+
+ return 0;
+}
+
#define pa_dev_attr(name, field, format_string) \
static ssize_t name##_show(struct device *dev, struct device_attribute *attr, char *buf) \
{ \
@@ -566,12 +598,7 @@
static ssize_t modalias_show(struct device *dev, struct device_attribute *attr, char *buf)
{
- struct parisc_device *padev = to_parisc_device(dev);
- struct parisc_device_id *id = &padev->id;
-
- return sprintf(buf, "parisc:t%02Xhv%04Xrev%02Xsv%08X\n",
- (u8)id->hw_type, (u16)id->hversion, (u8)id->hversion_rev,
- (u32)id->sversion);
+ return make_modalias(dev, buf);
}
static struct device_attribute parisc_device_attrs[] = {
@@ -587,6 +614,7 @@
struct bus_type parisc_bus_type = {
.name = "parisc",
.match = parisc_generic_match,
+ .uevent = parisc_uevent,
.dev_attrs = parisc_device_attrs,
.probe = parisc_driver_probe,
.remove = parisc_driver_remove,
diff --git a/arch/parisc/kernel/hpmc.S b/arch/parisc/kernel/hpmc.S
index 2cbf13b..5595a2f 100644
--- a/arch/parisc/kernel/hpmc.S
+++ b/arch/parisc/kernel/hpmc.S
@@ -80,6 +80,7 @@
.import intr_save, code
ENTRY(os_hpmc)
+.os_hpmc:
/*
* registers modified:
@@ -295,5 +296,10 @@
b .
nop
ENDPROC(os_hpmc)
-ENTRY(os_hpmc_end) /* this label used to compute os_hpmc checksum */
+.os_hpmc_end:
nop
+.data
+.align 4
+ .export os_hpmc_size
+os_hpmc_size:
+ .word .os_hpmc_end-.os_hpmc
diff --git a/arch/parisc/kernel/irq.c b/arch/parisc/kernel/irq.c
index 4cea935..ac2c822 100644
--- a/arch/parisc/kernel/irq.c
+++ b/arch/parisc/kernel/irq.c
@@ -298,7 +298,7 @@
irq_desc[irq].affinity = cpumask_of_cpu(cpu);
#endif
- return cpu_data[cpu].txn_addr;
+ return per_cpu(cpu_data, cpu).txn_addr;
}
@@ -309,8 +309,9 @@
next_cpu++; /* assign to "next" CPU we want this bugger on */
/* validate entry */
- while ((next_cpu < NR_CPUS) && (!cpu_data[next_cpu].txn_addr ||
- !cpu_online(next_cpu)))
+ while ((next_cpu < NR_CPUS) &&
+ (!per_cpu(cpu_data, next_cpu).txn_addr ||
+ !cpu_online(next_cpu)))
next_cpu++;
if (next_cpu >= NR_CPUS)
@@ -359,7 +360,7 @@
printk(KERN_DEBUG "redirecting irq %d from CPU %d to %d\n",
irq, smp_processor_id(), cpu);
gsc_writel(irq + CPU_IRQ_BASE,
- cpu_data[cpu].hpa);
+ per_cpu(cpu_data, cpu).hpa);
goto set_out;
}
#endif
@@ -421,5 +422,5 @@
void ack_bad_irq(unsigned int irq)
{
- printk("unexpected IRQ %d\n", irq);
+ printk(KERN_WARNING "unexpected IRQ %d\n", irq);
}
diff --git a/arch/parisc/kernel/pdc_cons.c b/arch/parisc/kernel/pdc_cons.c
index ccb6809..1ff366c 100644
--- a/arch/parisc/kernel/pdc_cons.c
+++ b/arch/parisc/kernel/pdc_cons.c
@@ -52,7 +52,7 @@
#include <linux/tty.h>
#include <asm/pdc.h> /* for iodc_call() proto and friends */
-static spinlock_t pdc_console_lock = SPIN_LOCK_UNLOCKED;
+static DEFINE_SPINLOCK(pdc_console_lock);
static void pdc_console_write(struct console *co, const char *s, unsigned count)
{
diff --git a/arch/parisc/kernel/perf.c b/arch/parisc/kernel/perf.c
index f696f57..75099ef 100644
--- a/arch/parisc/kernel/perf.c
+++ b/arch/parisc/kernel/perf.c
@@ -541,9 +541,9 @@
spin_lock_init(&perf_lock);
/* TODO: this only lets us access the first cpu.. what to do for SMP? */
- cpu_device = cpu_data[0].dev;
+ cpu_device = per_cpu(cpu_data, 0).dev;
printk("Performance monitoring counters enabled for %s\n",
- cpu_data[0].dev->name);
+ per_cpu(cpu_data, 0).dev->name);
return 0;
}
diff --git a/arch/parisc/kernel/processor.c b/arch/parisc/kernel/processor.c
index 370086f..ecb6093 100644
--- a/arch/parisc/kernel/processor.c
+++ b/arch/parisc/kernel/processor.c
@@ -3,7 +3,7 @@
* Initial setup-routines for HP 9000 based hardware.
*
* Copyright (C) 1991, 1992, 1995 Linus Torvalds
- * Modifications for PA-RISC (C) 1999 Helge Deller <deller@gmx.de>
+ * Modifications for PA-RISC (C) 1999-2008 Helge Deller <deller@gmx.de>
* Modifications copyright 1999 SuSE GmbH (Philipp Rumpf)
* Modifications copyright 2000 Martin K. Petersen <mkp@mkp.net>
* Modifications copyright 2000 Philipp Rumpf <prumpf@tux.org>
@@ -46,7 +46,7 @@
struct system_cpuinfo_parisc boot_cpu_data __read_mostly;
EXPORT_SYMBOL(boot_cpu_data);
-struct cpuinfo_parisc cpu_data[NR_CPUS] __read_mostly;
+DEFINE_PER_CPU(struct cpuinfo_parisc, cpu_data);
extern int update_cr16_clocksource(void); /* from time.c */
@@ -69,6 +69,23 @@
*/
/**
+ * init_cpu_profiler - enable/setup per cpu profiling hooks.
+ * @cpunum: The processor instance.
+ *
+ * FIXME: doesn't do much yet...
+ */
+static void __cpuinit
+init_percpu_prof(unsigned long cpunum)
+{
+ struct cpuinfo_parisc *p;
+
+ p = &per_cpu(cpu_data, cpunum);
+ p->prof_counter = 1;
+ p->prof_multiplier = 1;
+}
+
+
+/**
* processor_probe - Determine if processor driver should claim this device.
* @dev: The device which has been found.
*
@@ -147,7 +164,7 @@
}
#endif
- p = &cpu_data[cpuid];
+ p = &per_cpu(cpu_data, cpuid);
boot_cpu_data.cpu_count++;
/* initialize counters - CPU 0 gets it_value set in time_init() */
@@ -162,12 +179,9 @@
#ifdef CONFIG_SMP
/*
** FIXME: review if any other initialization is clobbered
- ** for boot_cpu by the above memset().
+ ** for boot_cpu by the above memset().
*/
-
- /* stolen from init_percpu_prof() */
- cpu_data[cpuid].prof_counter = 1;
- cpu_data[cpuid].prof_multiplier = 1;
+ init_percpu_prof(cpuid);
#endif
/*
@@ -261,19 +275,6 @@
}
-/**
- * init_cpu_profiler - enable/setup per cpu profiling hooks.
- * @cpunum: The processor instance.
- *
- * FIXME: doesn't do much yet...
- */
-static inline void __init
-init_percpu_prof(int cpunum)
-{
- cpu_data[cpunum].prof_counter = 1;
- cpu_data[cpunum].prof_multiplier = 1;
-}
-
/**
* init_per_cpu - Handle individual processor initializations.
@@ -293,7 +294,7 @@
*
* o Enable CPU profiling hooks.
*/
-int __init init_per_cpu(int cpunum)
+int __cpuinit init_per_cpu(int cpunum)
{
int ret;
struct pdc_coproc_cfg coproc_cfg;
@@ -307,8 +308,8 @@
/* FWIW, FP rev/model is a more accurate way to determine
** CPU type. CPU rev/model has some ambiguous cases.
*/
- cpu_data[cpunum].fp_rev = coproc_cfg.revision;
- cpu_data[cpunum].fp_model = coproc_cfg.model;
+ per_cpu(cpu_data, cpunum).fp_rev = coproc_cfg.revision;
+ per_cpu(cpu_data, cpunum).fp_model = coproc_cfg.model;
printk(KERN_INFO "FP[%d] enabled: Rev %ld Model %ld\n",
cpunum, coproc_cfg.revision, coproc_cfg.model);
@@ -344,16 +345,17 @@
int
show_cpuinfo (struct seq_file *m, void *v)
{
- int n;
+ unsigned long cpu;
- for(n=0; n<boot_cpu_data.cpu_count; n++) {
+ for_each_online_cpu(cpu) {
+ const struct cpuinfo_parisc *cpuinfo = &per_cpu(cpu_data, cpu);
#ifdef CONFIG_SMP
- if (0 == cpu_data[n].hpa)
+ if (0 == cpuinfo->hpa)
continue;
#endif
- seq_printf(m, "processor\t: %d\n"
+ seq_printf(m, "processor\t: %lu\n"
"cpu family\t: PA-RISC %s\n",
- n, boot_cpu_data.family_name);
+ cpu, boot_cpu_data.family_name);
seq_printf(m, "cpu\t\t: %s\n", boot_cpu_data.cpu_name );
@@ -365,8 +367,8 @@
seq_printf(m, "model\t\t: %s\n"
"model name\t: %s\n",
boot_cpu_data.pdc.sys_model_name,
- cpu_data[n].dev ?
- cpu_data[n].dev->name : "Unknown" );
+ cpuinfo->dev ?
+ cpuinfo->dev->name : "Unknown");
seq_printf(m, "hversion\t: 0x%08x\n"
"sversion\t: 0x%08x\n",
@@ -377,8 +379,8 @@
show_cache_info(m);
seq_printf(m, "bogomips\t: %lu.%02lu\n",
- cpu_data[n].loops_per_jiffy / (500000 / HZ),
- (cpu_data[n].loops_per_jiffy / (5000 / HZ)) % 100);
+ cpuinfo->loops_per_jiffy / (500000 / HZ),
+ (cpuinfo->loops_per_jiffy / (5000 / HZ)) % 100);
seq_printf(m, "software id\t: %ld\n\n",
boot_cpu_data.pdc.model.sw_id);
diff --git a/arch/parisc/kernel/setup.c b/arch/parisc/kernel/setup.c
index 7d27853..82131ca 100644
--- a/arch/parisc/kernel/setup.c
+++ b/arch/parisc/kernel/setup.c
@@ -58,11 +58,6 @@
EXPORT_SYMBOL(parisc_bus_is_phys);
#endif
-/* This sets the vmerge boundary and size, it's here because it has to
- * be available on all platforms (zero means no-virtual merging) */
-unsigned long parisc_vmerge_boundary = 0;
-unsigned long parisc_vmerge_max_size = 0;
-
void __init setup_cmdline(char **cmdline_p)
{
extern unsigned int boot_args[];
@@ -321,7 +316,7 @@
processor_init();
printk(KERN_INFO "CPU(s): %d x %s at %d.%06d MHz\n",
- boot_cpu_data.cpu_count,
+ num_present_cpus(),
boot_cpu_data.cpu_name,
boot_cpu_data.cpu_hz / 1000000,
boot_cpu_data.cpu_hz % 1000000 );
@@ -387,8 +382,8 @@
if (ret >= 0 && coproc_cfg.ccr_functional) {
mtctl(coproc_cfg.ccr_functional, 10);
- cpu_data[cpunum].fp_rev = coproc_cfg.revision;
- cpu_data[cpunum].fp_model = coproc_cfg.model;
+ per_cpu(cpu_data, cpunum).fp_rev = coproc_cfg.revision;
+ per_cpu(cpu_data, cpunum).fp_model = coproc_cfg.model;
asm volatile ("fstd %fr0,8(%sp)");
} else {
diff --git a/arch/parisc/kernel/smp.c b/arch/parisc/kernel/smp.c
index 80bc000..9995d7e 100644
--- a/arch/parisc/kernel/smp.c
+++ b/arch/parisc/kernel/smp.c
@@ -56,16 +56,17 @@
if (lvl >= smp_debug_lvl) \
printk(printargs);
#else
-#define smp_debug(lvl, ...)
+#define smp_debug(lvl, ...) do { } while(0)
#endif /* DEBUG_SMP */
DEFINE_SPINLOCK(smp_lock);
volatile struct task_struct *smp_init_current_idle_task;
-static volatile int cpu_now_booting __read_mostly = 0; /* track which CPU is booting */
+/* track which CPU is booting */
+static volatile int cpu_now_booting __cpuinitdata;
-static int parisc_max_cpus __read_mostly = 1;
+static int parisc_max_cpus __cpuinitdata = 1;
DEFINE_PER_CPU(spinlock_t, ipi_lock) = SPIN_LOCK_UNLOCKED;
@@ -123,7 +124,7 @@
ipi_interrupt(int irq, void *dev_id)
{
int this_cpu = smp_processor_id();
- struct cpuinfo_parisc *p = &cpu_data[this_cpu];
+ struct cpuinfo_parisc *p = &per_cpu(cpu_data, this_cpu);
unsigned long ops;
unsigned long flags;
@@ -202,13 +203,13 @@
static inline void
ipi_send(int cpu, enum ipi_message_type op)
{
- struct cpuinfo_parisc *p = &cpu_data[cpu];
+ struct cpuinfo_parisc *p = &per_cpu(cpu_data, cpu);
spinlock_t *lock = &per_cpu(ipi_lock, cpu);
unsigned long flags;
spin_lock_irqsave(lock, flags);
p->pending_ipi |= 1 << op;
- gsc_writel(IPI_IRQ - CPU_IRQ_BASE, cpu_data[cpu].hpa);
+ gsc_writel(IPI_IRQ - CPU_IRQ_BASE, p->hpa);
spin_unlock_irqrestore(lock, flags);
}
@@ -224,10 +225,7 @@
static inline void
send_IPI_single(int dest_cpu, enum ipi_message_type op)
{
- if (dest_cpu == NO_PROC_ID) {
- BUG();
- return;
- }
+ BUG_ON(dest_cpu == NO_PROC_ID);
ipi_send(dest_cpu, op);
}
@@ -309,8 +307,7 @@
/* Initialise the idle task for this CPU */
atomic_inc(&init_mm.mm_count);
current->active_mm = &init_mm;
- if(current->mm)
- BUG();
+ BUG_ON(current->mm);
enter_lazy_tlb(&init_mm, current);
init_IRQ(); /* make sure no IRQs are enabled or pending */
@@ -345,6 +342,7 @@
*/
int __cpuinit smp_boot_one_cpu(int cpuid)
{
+ const struct cpuinfo_parisc *p = &per_cpu(cpu_data, cpuid);
struct task_struct *idle;
long timeout;
@@ -376,7 +374,7 @@
smp_init_current_idle_task = idle ;
mb();
- printk("Releasing cpu %d now, hpa=%lx\n", cpuid, cpu_data[cpuid].hpa);
+ printk(KERN_INFO "Releasing cpu %d now, hpa=%lx\n", cpuid, p->hpa);
/*
** This gets PDC to release the CPU from a very tight loop.
@@ -387,7 +385,7 @@
** EIR{0}). MEM_RENDEZ is valid only when it is nonzero and the
** contents of memory are valid."
*/
- gsc_writel(TIMER_IRQ - CPU_IRQ_BASE, cpu_data[cpuid].hpa);
+ gsc_writel(TIMER_IRQ - CPU_IRQ_BASE, p->hpa);
mb();
/*
@@ -419,12 +417,12 @@
return 0;
}
-void __devinit smp_prepare_boot_cpu(void)
+void __init smp_prepare_boot_cpu(void)
{
- int bootstrap_processor=cpu_data[0].cpuid; /* CPU ID of BSP */
+ int bootstrap_processor = per_cpu(cpu_data, 0).cpuid;
/* Setup BSP mappings */
- printk("SMP: bootstrap CPU ID is %d\n",bootstrap_processor);
+ printk(KERN_INFO "SMP: bootstrap CPU ID is %d\n", bootstrap_processor);
cpu_set(bootstrap_processor, cpu_online_map);
cpu_set(bootstrap_processor, cpu_present_map);
diff --git a/arch/parisc/kernel/time.c b/arch/parisc/kernel/time.c
index 4d09203..9d46c43 100644
--- a/arch/parisc/kernel/time.c
+++ b/arch/parisc/kernel/time.c
@@ -60,7 +60,7 @@
unsigned long cycles_elapsed, ticks_elapsed;
unsigned long cycles_remainder;
unsigned int cpu = smp_processor_id();
- struct cpuinfo_parisc *cpuinfo = &cpu_data[cpu];
+ struct cpuinfo_parisc *cpuinfo = &per_cpu(cpu_data, cpu);
/* gcc can optimize for "read-only" case with a local clocktick */
unsigned long cpt = clocktick;
@@ -213,7 +213,7 @@
mtctl(next_tick, 16); /* kick off Interval Timer (CR16) */
- cpu_data[cpu].it_value = next_tick;
+ per_cpu(cpu_data, cpu).it_value = next_tick;
}
struct platform_device rtc_parisc_dev = {
diff --git a/arch/parisc/kernel/topology.c b/arch/parisc/kernel/topology.c
index d71cb01..f515938 100644
--- a/arch/parisc/kernel/topology.c
+++ b/arch/parisc/kernel/topology.c
@@ -22,14 +22,14 @@
#include <linux/cpu.h>
#include <linux/cache.h>
-static struct cpu cpu_devices[NR_CPUS] __read_mostly;
+static DEFINE_PER_CPU(struct cpu, cpu_devices);
static int __init topology_init(void)
{
int num;
for_each_present_cpu(num) {
- register_cpu(&cpu_devices[num], num);
+ register_cpu(&per_cpu(cpu_devices, num), num);
}
return 0;
}
diff --git a/arch/parisc/kernel/traps.c b/arch/parisc/kernel/traps.c
index 4c771cd..ba658d2 100644
--- a/arch/parisc/kernel/traps.c
+++ b/arch/parisc/kernel/traps.c
@@ -745,6 +745,10 @@
/* Fall Through */
case 27:
/* Data memory protection ID trap */
+ if (code == 27 && !user_mode(regs) &&
+ fixup_exception(regs))
+ return;
+
die_if_kernel("Protection id trap", regs, code);
si.si_code = SEGV_MAPERR;
si.si_signo = SIGSEGV;
@@ -821,8 +825,8 @@
int __init check_ivt(void *iva)
{
+ extern u32 os_hpmc_size;
extern const u32 os_hpmc[];
- extern const u32 os_hpmc_end[];
int i;
u32 check = 0;
@@ -839,8 +843,7 @@
*ivap++ = 0;
/* Compute Checksum for HPMC handler */
-
- length = os_hpmc_end - os_hpmc;
+ length = os_hpmc_size;
ivap[7] = length;
hpmcp = (u32 *)os_hpmc;
diff --git a/arch/parisc/kernel/unwind.c b/arch/parisc/kernel/unwind.c
index 6773c58..69dad5a 100644
--- a/arch/parisc/kernel/unwind.c
+++ b/arch/parisc/kernel/unwind.c
@@ -372,7 +372,7 @@
struct pt_regs *r = &t->thread.regs;
struct pt_regs *r2;
- r2 = kmalloc(sizeof(struct pt_regs), GFP_KERNEL);
+ r2 = kmalloc(sizeof(struct pt_regs), GFP_ATOMIC);
if (!r2)
return;
*r2 = *r;
diff --git a/arch/parisc/lib/iomap.c b/arch/parisc/lib/iomap.c
index 9abed07..5069e8b 100644
--- a/arch/parisc/lib/iomap.c
+++ b/arch/parisc/lib/iomap.c
@@ -261,7 +261,7 @@
iomem_write32r,
};
-const struct iomap_ops *iomap_ops[8] = {
+static const struct iomap_ops *iomap_ops[8] = {
[0] = &ioport_ops,
[7] = &iomem_ops
};
diff --git a/arch/parisc/lib/memcpy.c b/arch/parisc/lib/memcpy.c
index 2d68431..bbda909 100644
--- a/arch/parisc/lib/memcpy.c
+++ b/arch/parisc/lib/memcpy.c
@@ -275,7 +275,7 @@
/* Returns 0 for success, otherwise, returns number of bytes not transferred. */
-unsigned long pa_memcpy(void *dstp, const void *srcp, unsigned long len)
+static unsigned long pa_memcpy(void *dstp, const void *srcp, unsigned long len)
{
register unsigned long src, dst, t1, t2, t3;
register unsigned char *pcs, *pcd;
diff --git a/arch/parisc/mm/fault.c b/arch/parisc/mm/fault.c
index b2e3e9a..92c7fa4 100644
--- a/arch/parisc/mm/fault.c
+++ b/arch/parisc/mm/fault.c
@@ -139,13 +139,41 @@
}
#endif
+int fixup_exception(struct pt_regs *regs)
+{
+ const struct exception_table_entry *fix;
+
+ fix = search_exception_tables(regs->iaoq[0]);
+ if (fix) {
+ struct exception_data *d;
+ d = &__get_cpu_var(exception_data);
+ d->fault_ip = regs->iaoq[0];
+ d->fault_space = regs->isr;
+ d->fault_addr = regs->ior;
+
+ regs->iaoq[0] = ((fix->fixup) & ~3);
+ /*
+ * NOTE: In some cases the faulting instruction
+ * may be in the delay slot of a branch. We
+ * don't want to take the branch, so we don't
+ * increment iaoq[1], instead we set it to be
+ * iaoq[0]+4, and clear the B bit in the PSW
+ */
+ regs->iaoq[1] = regs->iaoq[0] + 4;
+ regs->gr[0] &= ~PSW_B; /* IPSW in gr[0] */
+
+ return 1;
+ }
+
+ return 0;
+}
+
void do_page_fault(struct pt_regs *regs, unsigned long code,
unsigned long address)
{
struct vm_area_struct *vma, *prev_vma;
struct task_struct *tsk = current;
struct mm_struct *mm = tsk->mm;
- const struct exception_table_entry *fix;
unsigned long acc_type;
int fault;
@@ -229,32 +257,8 @@
no_context:
- if (!user_mode(regs)) {
- fix = search_exception_tables(regs->iaoq[0]);
-
- if (fix) {
- struct exception_data *d;
-
- d = &__get_cpu_var(exception_data);
- d->fault_ip = regs->iaoq[0];
- d->fault_space = regs->isr;
- d->fault_addr = regs->ior;
-
- regs->iaoq[0] = ((fix->fixup) & ~3);
-
- /*
- * NOTE: In some cases the faulting instruction
- * may be in the delay slot of a branch. We
- * don't want to take the branch, so we don't
- * increment iaoq[1], instead we set it to be
- * iaoq[0]+4, and clear the B bit in the PSW
- */
-
- regs->iaoq[1] = regs->iaoq[0] + 4;
- regs->gr[0] &= ~PSW_B; /* IPSW in gr[0] */
-
- return;
- }
+ if (!user_mode(regs) && fixup_exception(regs)) {
+ return;
}
parisc_terminate("Bad Address (null pointer deref?)", regs, code, address);
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 79f25ce..84b8613 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -108,6 +108,8 @@
config PPC
bool
default y
+ select HAVE_FTRACE_MCOUNT_RECORD
+ select HAVE_DYNAMIC_FTRACE
select HAVE_FUNCTION_TRACER
select ARCH_WANT_OPTIONAL_GPIOLIB
select HAVE_IDE
@@ -326,7 +328,8 @@
config CRASH_DUMP
bool "Build a kdump crash kernel"
- depends on (PPC64 && RELOCATABLE) || 6xx
+ depends on PPC64 || 6xx
+ select RELOCATABLE if PPC64
help
Build a kernel suitable for use as a kdump capture kernel.
The same kernel binary can be used as production kernel and dump
diff --git a/arch/powerpc/boot/Makefile b/arch/powerpc/boot/Makefile
index ab6dda3..e84df33 100644
--- a/arch/powerpc/boot/Makefile
+++ b/arch/powerpc/boot/Makefile
@@ -356,7 +356,7 @@
@rm -f $@; ln $< $@
install: $(CONFIGURE) $(addprefix $(obj)/, $(image-y))
- sh -x $(srctree)/$(src)/install.sh "$(KERNELRELEASE)" vmlinux System.map "$(INSTALL_PATH)" $<
+ sh -x $(srctree)/$(src)/install.sh "$(KERNELRELEASE)" vmlinux System.map "$(INSTALL_PATH)" $^
# anything not in $(targets)
clean-files += $(image-) $(initrd-) cuImage.* dtbImage.* treeImage.* \
diff --git a/arch/powerpc/boot/dts/mpc836x_mds.dts b/arch/powerpc/boot/dts/mpc836x_mds.dts
index 14534d0..6e34f17 100644
--- a/arch/powerpc/boot/dts/mpc836x_mds.dts
+++ b/arch/powerpc/boot/dts/mpc836x_mds.dts
@@ -69,8 +69,18 @@
};
bcsr@1,0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
compatible = "fsl,mpc8360mds-bcsr";
reg = <1 0 0x8000>;
+ ranges = <0 1 0 0x8000>;
+
+ bcsr13: gpio-controller@d {
+ #gpio-cells = <2>;
+ compatible = "fsl,mpc8360mds-bcsr-gpio";
+ reg = <0xd 1>;
+ gpio-controller;
+ };
};
};
@@ -195,10 +205,21 @@
};
par_io@1400 {
+ #address-cells = <1>;
+ #size-cells = <1>;
reg = <0x1400 0x100>;
+ ranges = <0 0x1400 0x100>;
device_type = "par_io";
num-ports = <7>;
+ qe_pio_b: gpio-controller@18 {
+ #gpio-cells = <2>;
+ compatible = "fsl,mpc8360-qe-pario-bank",
+ "fsl,mpc8323-qe-pario-bank";
+ reg = <0x18 0x18>;
+ gpio-controller;
+ };
+
pio1: ucc_pin@01 {
pio-map = <
/* port pin dir open_drain assignment has_irq */
@@ -282,6 +303,15 @@
};
};
+ timer@440 {
+ compatible = "fsl,mpc8360-qe-gtm",
+ "fsl,qe-gtm", "fsl,gtm";
+ reg = <0x440 0x40>;
+ clock-frequency = <132000000>;
+ interrupts = <12 13 14 15>;
+ interrupt-parent = <&qeic>;
+ };
+
spi@4c0 {
cell-index = <0>;
compatible = "fsl,spi";
@@ -301,11 +331,20 @@
};
usb@6c0 {
- compatible = "qe_udc";
+ compatible = "fsl,mpc8360-qe-usb",
+ "fsl,mpc8323-qe-usb";
reg = <0x6c0 0x40 0x8b00 0x100>;
interrupts = <11>;
interrupt-parent = <&qeic>;
- mode = "slave";
+ fsl,fullspeed-clock = "clk21";
+ fsl,lowspeed-clock = "brg9";
+ gpios = <&qe_pio_b 2 0 /* USBOE */
+ &qe_pio_b 3 0 /* USBTP */
+ &qe_pio_b 8 0 /* USBTN */
+ &qe_pio_b 9 0 /* USBRP */
+ &qe_pio_b 11 0 /* USBRN */
+ &bcsr13 5 0 /* SPEED */
+ &bcsr13 4 1>; /* POWER */
};
enet0: ucc@2000 {
diff --git a/arch/powerpc/boot/dts/mpc836x_rdk.dts b/arch/powerpc/boot/dts/mpc836x_rdk.dts
index decadf3..37b7895 100644
--- a/arch/powerpc/boot/dts/mpc836x_rdk.dts
+++ b/arch/powerpc/boot/dts/mpc836x_rdk.dts
@@ -218,8 +218,23 @@
reg = <0x440 0x40>;
interrupts = <12 13 14 15>;
interrupt-parent = <&qeic>;
- /* filled by u-boot */
- clock-frequency = <0>;
+ clock-frequency = <166666666>;
+ };
+
+ usb@6c0 {
+ compatible = "fsl,mpc8360-qe-usb",
+ "fsl,mpc8323-qe-usb";
+ reg = <0x6c0 0x40 0x8b00 0x100>;
+ interrupts = <11>;
+ interrupt-parent = <&qeic>;
+ fsl,fullspeed-clock = "clk21";
+ gpios = <&qe_pio_b 2 0 /* USBOE */
+ &qe_pio_b 3 0 /* USBTP */
+ &qe_pio_b 8 0 /* USBTN */
+ &qe_pio_b 9 0 /* USBRP */
+ &qe_pio_b 11 0 /* USBRN */
+ &qe_pio_e 20 0 /* SPEED */
+ &qe_pio_e 21 1 /* POWER */>;
};
spi@4c0 {
diff --git a/arch/powerpc/boot/dts/mpc8641_hpcn.dts b/arch/powerpc/boot/dts/mpc8641_hpcn.dts
index 35d5e24..4481532 100644
--- a/arch/powerpc/boot/dts/mpc8641_hpcn.dts
+++ b/arch/powerpc/boot/dts/mpc8641_hpcn.dts
@@ -26,7 +26,13 @@
serial1 = &serial1;
pci0 = &pci0;
pci1 = &pci1;
- rapidio0 = &rapidio0;
+/*
+ * Only one of Rapid IO or PCI can be present due to HW limitations and
+ * due to the fact that the 2 now share address space in the new memory
+ * map. The most likely case is that we have PCI, so comment out the
+ * rapidio node. Leave it here for reference.
+ */
+ /* rapidio0 = &rapidio0; */
};
cpus {
@@ -62,18 +68,17 @@
reg = <0x00000000 0x40000000>; // 1G at 0x0
};
- localbus@f8005000 {
+ localbus@ffe05000 {
#address-cells = <2>;
#size-cells = <1>;
compatible = "fsl,mpc8641-localbus", "simple-bus";
- reg = <0xf8005000 0x1000>;
+ reg = <0xffe05000 0x1000>;
interrupts = <19 2>;
interrupt-parent = <&mpic>;
- ranges = <0 0 0xff800000 0x00800000
- 1 0 0xfe000000 0x01000000
- 2 0 0xf8200000 0x00100000
- 3 0 0xf8100000 0x00100000>;
+ ranges = <0 0 0xef800000 0x00800000
+ 2 0 0xffdf8000 0x00008000
+ 3 0 0xffdf0000 0x00008000>;
flash@0,0 {
compatible = "cfi-flash";
@@ -103,13 +108,13 @@
};
};
- soc8641@f8000000 {
+ soc8641@ffe00000 {
#address-cells = <1>;
#size-cells = <1>;
device_type = "soc";
compatible = "simple-bus";
- ranges = <0x00000000 0xf8000000 0x00100000>;
- reg = <0xf8000000 0x00001000>; // CCSRBAR
+ ranges = <0x00000000 0xffe00000 0x00100000>;
+ reg = <0xffe00000 0x00001000>; // CCSRBAR
bus-frequency = <0>;
i2c@3000 {
@@ -340,17 +345,17 @@
};
};
- pci0: pcie@f8008000 {
+ pci0: pcie@ffe08000 {
cell-index = <0>;
compatible = "fsl,mpc8641-pcie";
device_type = "pci";
#interrupt-cells = <1>;
#size-cells = <2>;
#address-cells = <3>;
- reg = <0xf8008000 0x1000>;
+ reg = <0xffe08000 0x1000>;
bus-range = <0x0 0xff>;
ranges = <0x02000000 0x0 0x80000000 0x80000000 0x0 0x20000000
- 0x01000000 0x0 0x00000000 0xe2000000 0x0 0x00100000>;
+ 0x01000000 0x0 0x00000000 0xffc00000 0x0 0x00010000>;
clock-frequency = <33333333>;
interrupt-parent = <&mpic>;
interrupts = <24 2>;
@@ -481,7 +486,7 @@
0x01000000 0x0 0x00000000
0x01000000 0x0 0x00000000
- 0x0 0x00100000>;
+ 0x0 0x00010000>;
uli1575@0 {
reg = <0 0 0 0 0>;
#size-cells = <2>;
@@ -491,7 +496,7 @@
0x0 0x20000000
0x01000000 0x0 0x00000000
0x01000000 0x0 0x00000000
- 0x0 0x00100000>;
+ 0x0 0x00010000>;
isa@1e {
device_type = "isa";
#interrupt-cells = <2>;
@@ -549,17 +554,17 @@
};
- pci1: pcie@f8009000 {
+ pci1: pcie@ffe09000 {
cell-index = <1>;
compatible = "fsl,mpc8641-pcie";
device_type = "pci";
#interrupt-cells = <1>;
#size-cells = <2>;
#address-cells = <3>;
- reg = <0xf8009000 0x1000>;
+ reg = <0xffe09000 0x1000>;
bus-range = <0 0xff>;
ranges = <0x02000000 0x0 0xa0000000 0xa0000000 0x0 0x20000000
- 0x01000000 0x0 0x00000000 0xe3000000 0x0 0x00100000>;
+ 0x01000000 0x0 0x00000000 0xffc10000 0x0 0x00010000>;
clock-frequency = <33333333>;
interrupt-parent = <&mpic>;
interrupts = <25 2>;
@@ -582,18 +587,21 @@
0x01000000 0x0 0x00000000
0x01000000 0x0 0x00000000
- 0x0 0x00100000>;
+ 0x0 0x00010000>;
};
};
- rapidio0: rapidio@f80c0000 {
+/*
+ rapidio0: rapidio@ffec0000 {
#address-cells = <2>;
#size-cells = <2>;
compatible = "fsl,rapidio-delta";
- reg = <0xf80c0000 0x20000>;
- ranges = <0 0 0xc0000000 0 0x20000000>;
+ reg = <0xffec0000 0x20000>;
+ ranges = <0 0 0x80000000 0 0x20000000>;
interrupt-parent = <&mpic>;
- /* err_irq bell_outb_irq bell_inb_irq
- msg1_tx_irq msg1_rx_irq msg2_tx_irq msg2_rx_irq */
+ // err_irq bell_outb_irq bell_inb_irq
+ // msg1_tx_irq msg1_rx_irq msg2_tx_irq msg2_rx_irq
interrupts = <48 2 49 2 50 2 53 2 54 2 55 2 56 2>;
};
+*/
+
};
diff --git a/arch/powerpc/boot/install.sh b/arch/powerpc/boot/install.sh
index b002bfd..51b2387 100644
--- a/arch/powerpc/boot/install.sh
+++ b/arch/powerpc/boot/install.sh
@@ -15,7 +15,7 @@
# $2 - kernel image file
# $3 - kernel map file
# $4 - default install path (blank if root directory)
-# $5 - kernel boot file, the zImage
+# $5 and more - kernel boot files; zImage*, uImage, cuImage.*, etc.
#
# User may have a custom install script
@@ -38,3 +38,15 @@
cat $2 > $4/$image_name
cp $3 $4/System.map
+
+# Copy all the bootable image files
+path=$4
+shift 4
+while [ $# -ne 0 ]; do
+ image_name=`basename $1`
+ if [ -f $path/$image_name ]; then
+ mv $path/$image_name $path/$image_name.old
+ fi
+ cat $1 > $path/$image_name
+ shift
+done;
diff --git a/arch/powerpc/configs/85xx/mpc8572_ds_defconfig b/arch/powerpc/configs/85xx/mpc8572_ds_defconfig
index 6355883..32aeb79 100644
--- a/arch/powerpc/configs/85xx/mpc8572_ds_defconfig
+++ b/arch/powerpc/configs/85xx/mpc8572_ds_defconfig
@@ -1,7 +1,7 @@
#
# Automatically generated make config: don't edit
-# Linux kernel version: 2.6.28-rc3
-# Sat Nov 8 12:40:13 2008
+# Linux kernel version: 2.6.28-rc8
+# Tue Dec 30 11:17:46 2008
#
# CONFIG_PPC64 is not set
@@ -21,7 +21,10 @@
CONFIG_FSL_EMB_PERFMON=y
# CONFIG_PHYS_64BIT is not set
CONFIG_SPE=y
+CONFIG_PPC_MMU_NOHASH=y
# CONFIG_PPC_MM_SLICES is not set
+CONFIG_SMP=y
+CONFIG_NR_CPUS=2
CONFIG_PPC32=y
CONFIG_WORD_SIZE=32
# CONFIG_ARCH_PHYS_ADDR_T_64BIT is not set
@@ -50,7 +53,7 @@
CONFIG_PPC_OF=y
CONFIG_OF=y
CONFIG_PPC_UDBG_16550=y
-# CONFIG_GENERIC_TBSYNC is not set
+CONFIG_GENERIC_TBSYNC=y
CONFIG_AUDIT_ARCH=y
CONFIG_GENERIC_BUG=y
CONFIG_DEFAULT_UIMAGE=y
@@ -62,7 +65,7 @@
# General setup
#
CONFIG_EXPERIMENTAL=y
-CONFIG_BROKEN_ON_SMP=y
+CONFIG_LOCK_KERNEL=y
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_LOCALVERSION=""
CONFIG_LOCALVERSION_AUTO=y
@@ -126,6 +129,7 @@
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
+CONFIG_USE_GENERIC_SMP_HELPERS=y
# CONFIG_HAVE_GENERIC_DMA_COHERENT is not set
CONFIG_SLABINFO=y
CONFIG_RT_MUTEXES=y
@@ -138,6 +142,7 @@
CONFIG_MODVERSIONS=y
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_KMOD=y
+CONFIG_STOP_MACHINE=y
CONFIG_BLOCK=y
CONFIG_LBD=y
# CONFIG_BLK_DEV_IO_TRACE is not set
@@ -197,6 +202,7 @@
# CONFIG_CPM2 is not set
CONFIG_FSL_ULI1575=y
# CONFIG_MPC8xxx_GPIO is not set
+# CONFIG_SIMPLE_GPIO is not set
#
# Kernel options
@@ -224,6 +230,7 @@
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
CONFIG_ARCH_HAS_WALK_MEMORY=y
CONFIG_ARCH_ENABLE_MEMORY_HOTREMOVE=y
+# CONFIG_IRQ_ALL_CPUS is not set
CONFIG_ARCH_FLATMEM_ENABLE=y
CONFIG_ARCH_POPULATES_NODE_MAP=y
CONFIG_SELECT_MEMORY_MODEL=y
@@ -241,6 +248,9 @@
CONFIG_BOUNCE=y
CONFIG_VIRT_TO_BUS=y
CONFIG_UNEVICTABLE_LRU=y
+CONFIG_PPC_4K_PAGES=y
+# CONFIG_PPC_16K_PAGES is not set
+# CONFIG_PPC_64K_PAGES is not set
CONFIG_FORCE_MAX_ZONEORDER=11
CONFIG_PROC_DEVICETREE=y
# CONFIG_CMDLINE_BOOL is not set
@@ -443,8 +453,10 @@
# CONFIG_EEPROM_93CX6 is not set
# CONFIG_SGI_IOC4 is not set
# CONFIG_TIFM_CORE is not set
+# CONFIG_ICS932S401 is not set
# CONFIG_ENCLOSURE_SERVICES is not set
# CONFIG_HP_ILO is not set
+# CONFIG_C2PORT is not set
CONFIG_HAVE_IDE=y
# CONFIG_IDE is not set
@@ -784,6 +796,7 @@
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256
+# CONFIG_HVC_UDBG is not set
# CONFIG_IPMI_HANDLER is not set
CONFIG_HW_RANDOM=y
CONFIG_NVRAM=y
@@ -869,11 +882,11 @@
# CONFIG_THERMAL is not set
# CONFIG_THERMAL_HWMON is not set
# CONFIG_WATCHDOG is not set
+CONFIG_SSB_POSSIBLE=y
#
# Sonics Silicon Backplane
#
-CONFIG_SSB_POSSIBLE=y
# CONFIG_SSB is not set
#
@@ -886,14 +899,7 @@
# CONFIG_PMIC_DA903X is not set
# CONFIG_MFD_WM8400 is not set
# CONFIG_MFD_WM8350_I2C is not set
-
-#
-# Voltage and Current regulators
-#
# CONFIG_REGULATOR is not set
-# CONFIG_REGULATOR_FIXED_VOLTAGE is not set
-# CONFIG_REGULATOR_VIRTUAL_CONSUMER is not set
-# CONFIG_REGULATOR_BQ24022 is not set
#
# Multimedia devices
@@ -1252,11 +1258,11 @@
# CONFIG_USB_TMC is not set
#
-# NOTE: USB_STORAGE enables SCSI, and 'SCSI disk support'
+# NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may also be needed;
#
#
-# may also be needed; see USB_STORAGE Help for more information
+# see USB_STORAGE Help for more information
#
CONFIG_USB_STORAGE=y
# CONFIG_USB_STORAGE_DEBUG is not set
@@ -1348,6 +1354,7 @@
# CONFIG_RTC_DRV_M41T80 is not set
# CONFIG_RTC_DRV_S35390A is not set
# CONFIG_RTC_DRV_FM3130 is not set
+# CONFIG_RTC_DRV_RX8581 is not set
#
# SPI RTC drivers
@@ -1624,6 +1631,7 @@
# CONFIG_SAMPLES is not set
CONFIG_HAVE_ARCH_KGDB=y
# CONFIG_KGDB is not set
+CONFIG_PRINT_STACK_DEPTH=64
# CONFIG_DEBUG_STACKOVERFLOW is not set
# CONFIG_DEBUG_STACK_USAGE is not set
# CONFIG_DEBUG_PAGEALLOC is not set
@@ -1649,11 +1657,16 @@
#
# CONFIG_CRYPTO_FIPS is not set
CONFIG_CRYPTO_ALGAPI=y
+CONFIG_CRYPTO_ALGAPI2=y
CONFIG_CRYPTO_AEAD=y
+CONFIG_CRYPTO_AEAD2=y
CONFIG_CRYPTO_BLKCIPHER=y
+CONFIG_CRYPTO_BLKCIPHER2=y
CONFIG_CRYPTO_HASH=y
-CONFIG_CRYPTO_RNG=y
+CONFIG_CRYPTO_HASH2=y
+CONFIG_CRYPTO_RNG2=y
CONFIG_CRYPTO_MANAGER=y
+CONFIG_CRYPTO_MANAGER2=y
# CONFIG_CRYPTO_GF128MUL is not set
# CONFIG_CRYPTO_NULL is not set
# CONFIG_CRYPTO_CRYPTD is not set
diff --git a/arch/powerpc/include/asm/cell-pmu.h b/arch/powerpc/include/asm/cell-pmu.h
index 8066eed..b4b7338a 100644
--- a/arch/powerpc/include/asm/cell-pmu.h
+++ b/arch/powerpc/include/asm/cell-pmu.h
@@ -37,9 +37,11 @@
#define CBE_PM_STOP_AT_MAX 0x40000000
#define CBE_PM_TRACE_MODE_GET(pm_control) (((pm_control) >> 28) & 0x3)
#define CBE_PM_TRACE_MODE_SET(mode) (((mode) & 0x3) << 28)
+#define CBE_PM_TRACE_BUF_OVFLW(bit) (((bit) & 0x1) << 17)
#define CBE_PM_COUNT_MODE_SET(count) (((count) & 0x3) << 18)
#define CBE_PM_FREEZE_ALL_CTRS 0x00100000
#define CBE_PM_ENABLE_EXT_TRACE 0x00008000
+#define CBE_PM_SPU_ADDR_TRACE_SET(msk) (((msk) & 0x3) << 9)
/* Macros for the trace_address register. */
#define CBE_PM_TRACE_BUF_FULL 0x00000800
diff --git a/arch/powerpc/include/asm/ioctls.h b/arch/powerpc/include/asm/ioctls.h
index 279a622..1842186 100644
--- a/arch/powerpc/include/asm/ioctls.h
+++ b/arch/powerpc/include/asm/ioctls.h
@@ -89,6 +89,8 @@
#define TIOCSBRK 0x5427 /* BSD compatibility */
#define TIOCCBRK 0x5428 /* BSD compatibility */
#define TIOCGSID 0x5429 /* Return the session ID of FD */
+#define TIOCGRS485 0x542e
+#define TIOCSRS485 0x542f
#define TIOCGPTN _IOR('T',0x30, unsigned int) /* Get Pty Number (of pty-mux device) */
#define TIOCSPTLCK _IOW('T',0x31, int) /* Lock/unlock Pty */
diff --git a/arch/powerpc/include/asm/kexec.h b/arch/powerpc/include/asm/kexec.h
index 6dbffc9..7e06b43 100644
--- a/arch/powerpc/include/asm/kexec.h
+++ b/arch/powerpc/include/asm/kexec.h
@@ -48,63 +48,8 @@
{
if (oldregs)
memcpy(newregs, oldregs, sizeof(*newregs));
-#ifdef __powerpc64__
- else {
- /* FIXME Merge this with xmon_save_regs ?? */
- unsigned long tmp1, tmp2;
- __asm__ __volatile__ (
- "std 0,0(%2)\n"
- "std 1,8(%2)\n"
- "std 2,16(%2)\n"
- "std 3,24(%2)\n"
- "std 4,32(%2)\n"
- "std 5,40(%2)\n"
- "std 6,48(%2)\n"
- "std 7,56(%2)\n"
- "std 8,64(%2)\n"
- "std 9,72(%2)\n"
- "std 10,80(%2)\n"
- "std 11,88(%2)\n"
- "std 12,96(%2)\n"
- "std 13,104(%2)\n"
- "std 14,112(%2)\n"
- "std 15,120(%2)\n"
- "std 16,128(%2)\n"
- "std 17,136(%2)\n"
- "std 18,144(%2)\n"
- "std 19,152(%2)\n"
- "std 20,160(%2)\n"
- "std 21,168(%2)\n"
- "std 22,176(%2)\n"
- "std 23,184(%2)\n"
- "std 24,192(%2)\n"
- "std 25,200(%2)\n"
- "std 26,208(%2)\n"
- "std 27,216(%2)\n"
- "std 28,224(%2)\n"
- "std 29,232(%2)\n"
- "std 30,240(%2)\n"
- "std 31,248(%2)\n"
- "mfmsr %0\n"
- "std %0, 264(%2)\n"
- "mfctr %0\n"
- "std %0, 280(%2)\n"
- "mflr %0\n"
- "std %0, 288(%2)\n"
- "bl 1f\n"
- "1: mflr %1\n"
- "std %1, 256(%2)\n"
- "mtlr %0\n"
- "mfxer %0\n"
- "std %0, 296(%2)\n"
- : "=&r" (tmp1), "=&r" (tmp2)
- : "b" (newregs)
- : "memory");
- }
-#else
else
ppc_save_regs(newregs);
-#endif /* __powerpc64__ */
}
extern void kexec_smp_wait(void); /* get and clear naca physid, wait for
diff --git a/arch/powerpc/include/asm/oprofile_impl.h b/arch/powerpc/include/asm/oprofile_impl.h
index 95035c6..639dc96 100644
--- a/arch/powerpc/include/asm/oprofile_impl.h
+++ b/arch/powerpc/include/asm/oprofile_impl.h
@@ -32,6 +32,12 @@
unsigned long mmcr0;
unsigned long mmcr1;
unsigned long mmcra;
+#ifdef CONFIG_OPROFILE_CELL
+ /* Register for oprofile user tool to check cell kernel profiling
+ * suport.
+ */
+ unsigned long cell_support;
+#endif
#endif
unsigned long enable_kernel;
unsigned long enable_user;
diff --git a/arch/powerpc/include/asm/ps3.h b/arch/powerpc/include/asm/ps3.h
index cff30c0..eead5c6 100644
--- a/arch/powerpc/include/asm/ps3.h
+++ b/arch/powerpc/include/asm/ps3.h
@@ -320,6 +320,7 @@
enum ps3_match_sub_id {
PS3_MATCH_SUB_ID_GPU_FB = 1,
+ PS3_MATCH_SUB_ID_GPU_RAMDISK = 2,
};
#define PS3_MODULE_ALIAS_EHCI "ps3:1:0"
@@ -332,6 +333,7 @@
#define PS3_MODULE_ALIAS_STOR_FLASH "ps3:8:0"
#define PS3_MODULE_ALIAS_SOUND "ps3:9:0"
#define PS3_MODULE_ALIAS_GPU_FB "ps3:10:1"
+#define PS3_MODULE_ALIAS_GPU_RAMDISK "ps3:10:2"
#define PS3_MODULE_ALIAS_LPM "ps3:11:0"
enum ps3_system_bus_device_type {
diff --git a/arch/powerpc/include/asm/qe.h b/arch/powerpc/include/asm/qe.h
index edee15d..a0a1531 100644
--- a/arch/powerpc/include/asm/qe.h
+++ b/arch/powerpc/include/asm/qe.h
@@ -17,6 +17,8 @@
#ifdef __KERNEL__
#include <linux/spinlock.h>
+#include <linux/errno.h>
+#include <linux/err.h>
#include <asm/cpm.h>
#include <asm/immap_qe.h>
@@ -84,7 +86,11 @@
extern spinlock_t cmxgcr_lock;
/* Export QE common operations */
+#ifdef CONFIG_QUICC_ENGINE
extern void __init qe_reset(void);
+#else
+static inline void qe_reset(void) {}
+#endif
/* QE PIO */
#define QE_PIO_PINS 32
@@ -101,16 +107,43 @@
#endif
};
-extern int par_io_init(struct device_node *np);
-extern int par_io_of_config(struct device_node *np);
#define QE_PIO_DIR_IN 2
#define QE_PIO_DIR_OUT 1
extern void __par_io_config_pin(struct qe_pio_regs __iomem *par_io, u8 pin,
int dir, int open_drain, int assignment,
int has_irq);
+#ifdef CONFIG_QUICC_ENGINE
+extern int par_io_init(struct device_node *np);
+extern int par_io_of_config(struct device_node *np);
extern int par_io_config_pin(u8 port, u8 pin, int dir, int open_drain,
int assignment, int has_irq);
extern int par_io_data_set(u8 port, u8 pin, u8 val);
+#else
+static inline int par_io_init(struct device_node *np) { return -ENOSYS; }
+static inline int par_io_of_config(struct device_node *np) { return -ENOSYS; }
+static inline int par_io_config_pin(u8 port, u8 pin, int dir, int open_drain,
+ int assignment, int has_irq) { return -ENOSYS; }
+static inline int par_io_data_set(u8 port, u8 pin, u8 val) { return -ENOSYS; }
+#endif /* CONFIG_QUICC_ENGINE */
+
+/*
+ * Pin multiplexing functions.
+ */
+struct qe_pin;
+#ifdef CONFIG_QE_GPIO
+extern struct qe_pin *qe_pin_request(struct device_node *np, int index);
+extern void qe_pin_free(struct qe_pin *qe_pin);
+extern void qe_pin_set_gpio(struct qe_pin *qe_pin);
+extern void qe_pin_set_dedicated(struct qe_pin *pin);
+#else
+static inline struct qe_pin *qe_pin_request(struct device_node *np, int index)
+{
+ return ERR_PTR(-ENOSYS);
+}
+static inline void qe_pin_free(struct qe_pin *qe_pin) {}
+static inline void qe_pin_set_gpio(struct qe_pin *qe_pin) {}
+static inline void qe_pin_set_dedicated(struct qe_pin *pin) {}
+#endif /* CONFIG_QE_GPIO */
/* QE internal API */
int qe_issue_cmd(u32 cmd, u32 device, u8 mcn_protocol, u32 cmd_input);
diff --git a/arch/powerpc/include/asm/qe_ic.h b/arch/powerpc/include/asm/qe_ic.h
index 56a7745..cf51966 100644
--- a/arch/powerpc/include/asm/qe_ic.h
+++ b/arch/powerpc/include/asm/qe_ic.h
@@ -17,6 +17,9 @@
#include <linux/irq.h>
+struct device_node;
+struct qe_ic;
+
#define NUM_OF_QE_IC_GROUPS 6
/* Flags when we init the QE IC */
@@ -54,17 +57,27 @@
QE_IC_GRP_RISCB /* QE interrupt controller RISC group B */
};
+#ifdef CONFIG_QUICC_ENGINE
void qe_ic_init(struct device_node *node, unsigned int flags,
void (*low_handler)(unsigned int irq, struct irq_desc *desc),
void (*high_handler)(unsigned int irq, struct irq_desc *desc));
+unsigned int qe_ic_get_low_irq(struct qe_ic *qe_ic);
+unsigned int qe_ic_get_high_irq(struct qe_ic *qe_ic);
+#else
+static inline void qe_ic_init(struct device_node *node, unsigned int flags,
+ void (*low_handler)(unsigned int irq, struct irq_desc *desc),
+ void (*high_handler)(unsigned int irq, struct irq_desc *desc))
+{}
+static inline unsigned int qe_ic_get_low_irq(struct qe_ic *qe_ic)
+{ return 0; }
+static inline unsigned int qe_ic_get_high_irq(struct qe_ic *qe_ic)
+{ return 0; }
+#endif /* CONFIG_QUICC_ENGINE */
+
void qe_ic_set_highest_priority(unsigned int virq, int high);
int qe_ic_set_priority(unsigned int virq, unsigned int priority);
int qe_ic_set_high_priority(unsigned int virq, unsigned int priority, int high);
-struct qe_ic;
-unsigned int qe_ic_get_low_irq(struct qe_ic *qe_ic);
-unsigned int qe_ic_get_high_irq(struct qe_ic *qe_ic);
-
static inline void qe_ic_cascade_low_ipic(unsigned int irq,
struct irq_desc *desc)
{
diff --git a/arch/powerpc/include/asm/spu.h b/arch/powerpc/include/asm/spu.h
index 8b2eb04..0ab8d869 100644
--- a/arch/powerpc/include/asm/spu.h
+++ b/arch/powerpc/include/asm/spu.h
@@ -128,7 +128,7 @@
int number;
unsigned int irqs[3];
u32 node;
- u64 flags;
+ unsigned long flags;
u64 class_0_pending;
u64 class_0_dar;
u64 class_1_dar;
diff --git a/arch/powerpc/kernel/Makefile b/arch/powerpc/kernel/Makefile
index 1308a86..8d1a419 100644
--- a/arch/powerpc/kernel/Makefile
+++ b/arch/powerpc/kernel/Makefile
@@ -29,7 +29,7 @@
obj-y := cputable.o ptrace.o syscalls.o \
irq.o align.o signal_32.o pmc.o vdso.o \
init_task.o process.o systbl.o idle.o \
- signal.o sysfs.o
+ signal.o sysfs.o cacheinfo.o
obj-y += vdso32/
obj-$(CONFIG_PPC64) += setup_64.o sys_ppc32.o \
signal_64.o ptrace32.o \
diff --git a/arch/powerpc/kernel/cacheinfo.c b/arch/powerpc/kernel/cacheinfo.c
new file mode 100644
index 0000000..b33f041
--- /dev/null
+++ b/arch/powerpc/kernel/cacheinfo.c
@@ -0,0 +1,837 @@
+/*
+ * Processor cache information made available to userspace via sysfs;
+ * intended to be compatible with x86 intel_cacheinfo implementation.
+ *
+ * Copyright 2008 IBM Corporation
+ * Author: Nathan Lynch
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ */
+
+#include <linux/cpu.h>
+#include <linux/cpumask.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/kobject.h>
+#include <linux/list.h>
+#include <linux/notifier.h>
+#include <linux/of.h>
+#include <linux/percpu.h>
+#include <asm/prom.h>
+
+#include "cacheinfo.h"
+
+/* per-cpu object for tracking:
+ * - a "cache" kobject for the top-level directory
+ * - a list of "index" objects representing the cpu's local cache hierarchy
+ */
+struct cache_dir {
+ struct kobject *kobj; /* bare (not embedded) kobject for cache
+ * directory */
+ struct cache_index_dir *index; /* list of index objects */
+};
+
+/* "index" object: each cpu's cache directory has an index
+ * subdirectory corresponding to a cache object associated with the
+ * cpu. This object's lifetime is managed via the embedded kobject.
+ */
+struct cache_index_dir {
+ struct kobject kobj;
+ struct cache_index_dir *next; /* next index in parent directory */
+ struct cache *cache;
+};
+
+/* Template for determining which OF properties to query for a given
+ * cache type */
+struct cache_type_info {
+ const char *name;
+ const char *size_prop;
+
+ /* Allow for both [di]-cache-line-size and
+ * [di]-cache-block-size properties. According to the PowerPC
+ * Processor binding, -line-size should be provided if it
+ * differs from the cache block size (that which is operated
+ * on by cache instructions), so we look for -line-size first.
+ * See cache_get_line_size(). */
+
+ const char *line_size_props[2];
+ const char *nr_sets_prop;
+};
+
+/* These are used to index the cache_type_info array. */
+#define CACHE_TYPE_UNIFIED 0
+#define CACHE_TYPE_INSTRUCTION 1
+#define CACHE_TYPE_DATA 2
+
+static const struct cache_type_info cache_type_info[] = {
+ {
+ /* PowerPC Processor binding says the [di]-cache-*
+ * must be equal on unified caches, so just use
+ * d-cache properties. */
+ .name = "Unified",
+ .size_prop = "d-cache-size",
+ .line_size_props = { "d-cache-line-size",
+ "d-cache-block-size", },
+ .nr_sets_prop = "d-cache-sets",
+ },
+ {
+ .name = "Instruction",
+ .size_prop = "i-cache-size",
+ .line_size_props = { "i-cache-line-size",
+ "i-cache-block-size", },
+ .nr_sets_prop = "i-cache-sets",
+ },
+ {
+ .name = "Data",
+ .size_prop = "d-cache-size",
+ .line_size_props = { "d-cache-line-size",
+ "d-cache-block-size", },
+ .nr_sets_prop = "d-cache-sets",
+ },
+};
+
+/* Cache object: each instance of this corresponds to a distinct cache
+ * in the system. There are separate objects for Harvard caches: one
+ * each for instruction and data, and each refers to the same OF node.
+ * The refcount of the OF node is elevated for the lifetime of the
+ * cache object. A cache object is released when its shared_cpu_map
+ * is cleared (see cache_cpu_clear).
+ *
+ * A cache object is on two lists: an unsorted global list
+ * (cache_list) of cache objects; and a singly-linked list
+ * representing the local cache hierarchy, which is ordered by level
+ * (e.g. L1d -> L1i -> L2 -> L3).
+ */
+struct cache {
+ struct device_node *ofnode; /* OF node for this cache, may be cpu */
+ struct cpumask shared_cpu_map; /* online CPUs using this cache */
+ int type; /* split cache disambiguation */
+ int level; /* level not explicit in device tree */
+ struct list_head list; /* global list of cache objects */
+ struct cache *next_local; /* next cache of >= level */
+};
+
+static DEFINE_PER_CPU(struct cache_dir *, cache_dir);
+
+/* traversal/modification of this list occurs only at cpu hotplug time;
+ * access is serialized by cpu hotplug locking
+ */
+static LIST_HEAD(cache_list);
+
+static struct cache_index_dir *kobj_to_cache_index_dir(struct kobject *k)
+{
+ return container_of(k, struct cache_index_dir, kobj);
+}
+
+static const char *cache_type_string(const struct cache *cache)
+{
+ return cache_type_info[cache->type].name;
+}
+
+static void __cpuinit cache_init(struct cache *cache, int type, int level, struct device_node *ofnode)
+{
+ cache->type = type;
+ cache->level = level;
+ cache->ofnode = of_node_get(ofnode);
+ INIT_LIST_HEAD(&cache->list);
+ list_add(&cache->list, &cache_list);
+}
+
+static struct cache *__cpuinit new_cache(int type, int level, struct device_node *ofnode)
+{
+ struct cache *cache;
+
+ cache = kzalloc(sizeof(*cache), GFP_KERNEL);
+ if (cache)
+ cache_init(cache, type, level, ofnode);
+
+ return cache;
+}
+
+static void release_cache_debugcheck(struct cache *cache)
+{
+ struct cache *iter;
+
+ list_for_each_entry(iter, &cache_list, list)
+ WARN_ONCE(iter->next_local == cache,
+ "cache for %s(%s) refers to cache for %s(%s)\n",
+ iter->ofnode->full_name,
+ cache_type_string(iter),
+ cache->ofnode->full_name,
+ cache_type_string(cache));
+}
+
+static void release_cache(struct cache *cache)
+{
+ if (!cache)
+ return;
+
+ pr_debug("freeing L%d %s cache for %s\n", cache->level,
+ cache_type_string(cache), cache->ofnode->full_name);
+
+ release_cache_debugcheck(cache);
+ list_del(&cache->list);
+ of_node_put(cache->ofnode);
+ kfree(cache);
+}
+
+static void cache_cpu_set(struct cache *cache, int cpu)
+{
+ struct cache *next = cache;
+
+ while (next) {
+ WARN_ONCE(cpumask_test_cpu(cpu, &next->shared_cpu_map),
+ "CPU %i already accounted in %s(%s)\n",
+ cpu, next->ofnode->full_name,
+ cache_type_string(next));
+ cpumask_set_cpu(cpu, &next->shared_cpu_map);
+ next = next->next_local;
+ }
+}
+
+static int cache_size(const struct cache *cache, unsigned int *ret)
+{
+ const char *propname;
+ const u32 *cache_size;
+
+ propname = cache_type_info[cache->type].size_prop;
+
+ cache_size = of_get_property(cache->ofnode, propname, NULL);
+ if (!cache_size)
+ return -ENODEV;
+
+ *ret = *cache_size;
+ return 0;
+}
+
+static int cache_size_kb(const struct cache *cache, unsigned int *ret)
+{
+ unsigned int size;
+
+ if (cache_size(cache, &size))
+ return -ENODEV;
+
+ *ret = size / 1024;
+ return 0;
+}
+
+/* not cache_line_size() because that's a macro in include/linux/cache.h */
+static int cache_get_line_size(const struct cache *cache, unsigned int *ret)
+{
+ const u32 *line_size;
+ int i, lim;
+
+ lim = ARRAY_SIZE(cache_type_info[cache->type].line_size_props);
+
+ for (i = 0; i < lim; i++) {
+ const char *propname;
+
+ propname = cache_type_info[cache->type].line_size_props[i];
+ line_size = of_get_property(cache->ofnode, propname, NULL);
+ if (line_size)
+ break;
+ }
+
+ if (!line_size)
+ return -ENODEV;
+
+ *ret = *line_size;
+ return 0;
+}
+
+static int cache_nr_sets(const struct cache *cache, unsigned int *ret)
+{
+ const char *propname;
+ const u32 *nr_sets;
+
+ propname = cache_type_info[cache->type].nr_sets_prop;
+
+ nr_sets = of_get_property(cache->ofnode, propname, NULL);
+ if (!nr_sets)
+ return -ENODEV;
+
+ *ret = *nr_sets;
+ return 0;
+}
+
+static int cache_associativity(const struct cache *cache, unsigned int *ret)
+{
+ unsigned int line_size;
+ unsigned int nr_sets;
+ unsigned int size;
+
+ if (cache_nr_sets(cache, &nr_sets))
+ goto err;
+
+ /* If the cache is fully associative, there is no need to
+ * check the other properties.
+ */
+ if (nr_sets == 1) {
+ *ret = 0;
+ return 0;
+ }
+
+ if (cache_get_line_size(cache, &line_size))
+ goto err;
+ if (cache_size(cache, &size))
+ goto err;
+
+ if (!(nr_sets > 0 && size > 0 && line_size > 0))
+ goto err;
+
+ *ret = (size / nr_sets) / line_size;
+ return 0;
+err:
+ return -ENODEV;
+}
+
+/* helper for dealing with split caches */
+static struct cache *cache_find_first_sibling(struct cache *cache)
+{
+ struct cache *iter;
+
+ if (cache->type == CACHE_TYPE_UNIFIED)
+ return cache;
+
+ list_for_each_entry(iter, &cache_list, list)
+ if (iter->ofnode == cache->ofnode && iter->next_local == cache)
+ return iter;
+
+ return cache;
+}
+
+/* return the first cache on a local list matching node */
+static struct cache *cache_lookup_by_node(const struct device_node *node)
+{
+ struct cache *cache = NULL;
+ struct cache *iter;
+
+ list_for_each_entry(iter, &cache_list, list) {
+ if (iter->ofnode != node)
+ continue;
+ cache = cache_find_first_sibling(iter);
+ break;
+ }
+
+ return cache;
+}
+
+static bool cache_node_is_unified(const struct device_node *np)
+{
+ return of_get_property(np, "cache-unified", NULL);
+}
+
+static struct cache *__cpuinit cache_do_one_devnode_unified(struct device_node *node, int level)
+{
+ struct cache *cache;
+
+ pr_debug("creating L%d ucache for %s\n", level, node->full_name);
+
+ cache = new_cache(CACHE_TYPE_UNIFIED, level, node);
+
+ return cache;
+}
+
+static struct cache *__cpuinit cache_do_one_devnode_split(struct device_node *node, int level)
+{
+ struct cache *dcache, *icache;
+
+ pr_debug("creating L%d dcache and icache for %s\n", level,
+ node->full_name);
+
+ dcache = new_cache(CACHE_TYPE_DATA, level, node);
+ icache = new_cache(CACHE_TYPE_INSTRUCTION, level, node);
+
+ if (!dcache || !icache)
+ goto err;
+
+ dcache->next_local = icache;
+
+ return dcache;
+err:
+ release_cache(dcache);
+ release_cache(icache);
+ return NULL;
+}
+
+static struct cache *__cpuinit cache_do_one_devnode(struct device_node *node, int level)
+{
+ struct cache *cache;
+
+ if (cache_node_is_unified(node))
+ cache = cache_do_one_devnode_unified(node, level);
+ else
+ cache = cache_do_one_devnode_split(node, level);
+
+ return cache;
+}
+
+static struct cache *__cpuinit cache_lookup_or_instantiate(struct device_node *node, int level)
+{
+ struct cache *cache;
+
+ cache = cache_lookup_by_node(node);
+
+ WARN_ONCE(cache && cache->level != level,
+ "cache level mismatch on lookup (got %d, expected %d)\n",
+ cache->level, level);
+
+ if (!cache)
+ cache = cache_do_one_devnode(node, level);
+
+ return cache;
+}
+
+static void __cpuinit link_cache_lists(struct cache *smaller, struct cache *bigger)
+{
+ while (smaller->next_local) {
+ if (smaller->next_local == bigger)
+ return; /* already linked */
+ smaller = smaller->next_local;
+ }
+
+ smaller->next_local = bigger;
+}
+
+static void __cpuinit do_subsidiary_caches_debugcheck(struct cache *cache)
+{
+ WARN_ON_ONCE(cache->level != 1);
+ WARN_ON_ONCE(strcmp(cache->ofnode->type, "cpu"));
+}
+
+static void __cpuinit do_subsidiary_caches(struct cache *cache)
+{
+ struct device_node *subcache_node;
+ int level = cache->level;
+
+ do_subsidiary_caches_debugcheck(cache);
+
+ while ((subcache_node = of_find_next_cache_node(cache->ofnode))) {
+ struct cache *subcache;
+
+ level++;
+ subcache = cache_lookup_or_instantiate(subcache_node, level);
+ of_node_put(subcache_node);
+ if (!subcache)
+ break;
+
+ link_cache_lists(cache, subcache);
+ cache = subcache;
+ }
+}
+
+static struct cache *__cpuinit cache_chain_instantiate(unsigned int cpu_id)
+{
+ struct device_node *cpu_node;
+ struct cache *cpu_cache = NULL;
+
+ pr_debug("creating cache object(s) for CPU %i\n", cpu_id);
+
+ cpu_node = of_get_cpu_node(cpu_id, NULL);
+ WARN_ONCE(!cpu_node, "no OF node found for CPU %i\n", cpu_id);
+ if (!cpu_node)
+ goto out;
+
+ cpu_cache = cache_lookup_or_instantiate(cpu_node, 1);
+ if (!cpu_cache)
+ goto out;
+
+ do_subsidiary_caches(cpu_cache);
+
+ cache_cpu_set(cpu_cache, cpu_id);
+out:
+ of_node_put(cpu_node);
+
+ return cpu_cache;
+}
+
+static struct cache_dir *__cpuinit cacheinfo_create_cache_dir(unsigned int cpu_id)
+{
+ struct cache_dir *cache_dir;
+ struct sys_device *sysdev;
+ struct kobject *kobj = NULL;
+
+ sysdev = get_cpu_sysdev(cpu_id);
+ WARN_ONCE(!sysdev, "no sysdev for CPU %i\n", cpu_id);
+ if (!sysdev)
+ goto err;
+
+ kobj = kobject_create_and_add("cache", &sysdev->kobj);
+ if (!kobj)
+ goto err;
+
+ cache_dir = kzalloc(sizeof(*cache_dir), GFP_KERNEL);
+ if (!cache_dir)
+ goto err;
+
+ cache_dir->kobj = kobj;
+
+ WARN_ON_ONCE(per_cpu(cache_dir, cpu_id) != NULL);
+
+ per_cpu(cache_dir, cpu_id) = cache_dir;
+
+ return cache_dir;
+err:
+ kobject_put(kobj);
+ return NULL;
+}
+
+static void cache_index_release(struct kobject *kobj)
+{
+ struct cache_index_dir *index;
+
+ index = kobj_to_cache_index_dir(kobj);
+
+ pr_debug("freeing index directory for L%d %s cache\n",
+ index->cache->level, cache_type_string(index->cache));
+
+ kfree(index);
+}
+
+static ssize_t cache_index_show(struct kobject *k, struct attribute *attr, char *buf)
+{
+ struct kobj_attribute *kobj_attr;
+
+ kobj_attr = container_of(attr, struct kobj_attribute, attr);
+
+ return kobj_attr->show(k, kobj_attr, buf);
+}
+
+static struct cache *index_kobj_to_cache(struct kobject *k)
+{
+ struct cache_index_dir *index;
+
+ index = kobj_to_cache_index_dir(k);
+
+ return index->cache;
+}
+
+static ssize_t size_show(struct kobject *k, struct kobj_attribute *attr, char *buf)
+{
+ unsigned int size_kb;
+ struct cache *cache;
+
+ cache = index_kobj_to_cache(k);
+
+ if (cache_size_kb(cache, &size_kb))
+ return -ENODEV;
+
+ return sprintf(buf, "%uK\n", size_kb);
+}
+
+static struct kobj_attribute cache_size_attr =
+ __ATTR(size, 0444, size_show, NULL);
+
+
+static ssize_t line_size_show(struct kobject *k, struct kobj_attribute *attr, char *buf)
+{
+ unsigned int line_size;
+ struct cache *cache;
+
+ cache = index_kobj_to_cache(k);
+
+ if (cache_get_line_size(cache, &line_size))
+ return -ENODEV;
+
+ return sprintf(buf, "%u\n", line_size);
+}
+
+static struct kobj_attribute cache_line_size_attr =
+ __ATTR(coherency_line_size, 0444, line_size_show, NULL);
+
+static ssize_t nr_sets_show(struct kobject *k, struct kobj_attribute *attr, char *buf)
+{
+ unsigned int nr_sets;
+ struct cache *cache;
+
+ cache = index_kobj_to_cache(k);
+
+ if (cache_nr_sets(cache, &nr_sets))
+ return -ENODEV;
+
+ return sprintf(buf, "%u\n", nr_sets);
+}
+
+static struct kobj_attribute cache_nr_sets_attr =
+ __ATTR(number_of_sets, 0444, nr_sets_show, NULL);
+
+static ssize_t associativity_show(struct kobject *k, struct kobj_attribute *attr, char *buf)
+{
+ unsigned int associativity;
+ struct cache *cache;
+
+ cache = index_kobj_to_cache(k);
+
+ if (cache_associativity(cache, &associativity))
+ return -ENODEV;
+
+ return sprintf(buf, "%u\n", associativity);
+}
+
+static struct kobj_attribute cache_assoc_attr =
+ __ATTR(ways_of_associativity, 0444, associativity_show, NULL);
+
+static ssize_t type_show(struct kobject *k, struct kobj_attribute *attr, char *buf)
+{
+ struct cache *cache;
+
+ cache = index_kobj_to_cache(k);
+
+ return sprintf(buf, "%s\n", cache_type_string(cache));
+}
+
+static struct kobj_attribute cache_type_attr =
+ __ATTR(type, 0444, type_show, NULL);
+
+static ssize_t level_show(struct kobject *k, struct kobj_attribute *attr, char *buf)
+{
+ struct cache_index_dir *index;
+ struct cache *cache;
+
+ index = kobj_to_cache_index_dir(k);
+ cache = index->cache;
+
+ return sprintf(buf, "%d\n", cache->level);
+}
+
+static struct kobj_attribute cache_level_attr =
+ __ATTR(level, 0444, level_show, NULL);
+
+static ssize_t shared_cpu_map_show(struct kobject *k, struct kobj_attribute *attr, char *buf)
+{
+ struct cache_index_dir *index;
+ struct cache *cache;
+ int len;
+ int n = 0;
+
+ index = kobj_to_cache_index_dir(k);
+ cache = index->cache;
+ len = PAGE_SIZE - 2;
+
+ if (len > 1) {
+ n = cpumask_scnprintf(buf, len, &cache->shared_cpu_map);
+ buf[n++] = '\n';
+ buf[n] = '\0';
+ }
+ return n;
+}
+
+static struct kobj_attribute cache_shared_cpu_map_attr =
+ __ATTR(shared_cpu_map, 0444, shared_cpu_map_show, NULL);
+
+/* Attributes which should always be created -- the kobject/sysfs core
+ * does this automatically via kobj_type->default_attrs. This is the
+ * minimum data required to uniquely identify a cache.
+ */
+static struct attribute *cache_index_default_attrs[] = {
+ &cache_type_attr.attr,
+ &cache_level_attr.attr,
+ &cache_shared_cpu_map_attr.attr,
+ NULL,
+};
+
+/* Attributes which should be created if the cache device node has the
+ * right properties -- see cacheinfo_create_index_opt_attrs
+ */
+static struct kobj_attribute *cache_index_opt_attrs[] = {
+ &cache_size_attr,
+ &cache_line_size_attr,
+ &cache_nr_sets_attr,
+ &cache_assoc_attr,
+};
+
+static struct sysfs_ops cache_index_ops = {
+ .show = cache_index_show,
+};
+
+static struct kobj_type cache_index_type = {
+ .release = cache_index_release,
+ .sysfs_ops = &cache_index_ops,
+ .default_attrs = cache_index_default_attrs,
+};
+
+static void __cpuinit cacheinfo_create_index_opt_attrs(struct cache_index_dir *dir)
+{
+ const char *cache_name;
+ const char *cache_type;
+ struct cache *cache;
+ char *buf;
+ int i;
+
+ buf = kmalloc(PAGE_SIZE, GFP_KERNEL);
+ if (!buf)
+ return;
+
+ cache = dir->cache;
+ cache_name = cache->ofnode->full_name;
+ cache_type = cache_type_string(cache);
+
+ /* We don't want to create an attribute that can't provide a
+ * meaningful value. Check the return value of each optional
+ * attribute's ->show method before registering the
+ * attribute.
+ */
+ for (i = 0; i < ARRAY_SIZE(cache_index_opt_attrs); i++) {
+ struct kobj_attribute *attr;
+ ssize_t rc;
+
+ attr = cache_index_opt_attrs[i];
+
+ rc = attr->show(&dir->kobj, attr, buf);
+ if (rc <= 0) {
+ pr_debug("not creating %s attribute for "
+ "%s(%s) (rc = %zd)\n",
+ attr->attr.name, cache_name,
+ cache_type, rc);
+ continue;
+ }
+ if (sysfs_create_file(&dir->kobj, &attr->attr))
+ pr_debug("could not create %s attribute for %s(%s)\n",
+ attr->attr.name, cache_name, cache_type);
+ }
+
+ kfree(buf);
+}
+
+static void __cpuinit cacheinfo_create_index_dir(struct cache *cache, int index, struct cache_dir *cache_dir)
+{
+ struct cache_index_dir *index_dir;
+ int rc;
+
+ index_dir = kzalloc(sizeof(*index_dir), GFP_KERNEL);
+ if (!index_dir)
+ goto err;
+
+ index_dir->cache = cache;
+
+ rc = kobject_init_and_add(&index_dir->kobj, &cache_index_type,
+ cache_dir->kobj, "index%d", index);
+ if (rc)
+ goto err;
+
+ index_dir->next = cache_dir->index;
+ cache_dir->index = index_dir;
+
+ cacheinfo_create_index_opt_attrs(index_dir);
+
+ return;
+err:
+ kfree(index_dir);
+}
+
+static void __cpuinit cacheinfo_sysfs_populate(unsigned int cpu_id, struct cache *cache_list)
+{
+ struct cache_dir *cache_dir;
+ struct cache *cache;
+ int index = 0;
+
+ cache_dir = cacheinfo_create_cache_dir(cpu_id);
+ if (!cache_dir)
+ return;
+
+ cache = cache_list;
+ while (cache) {
+ cacheinfo_create_index_dir(cache, index, cache_dir);
+ index++;
+ cache = cache->next_local;
+ }
+}
+
+void __cpuinit cacheinfo_cpu_online(unsigned int cpu_id)
+{
+ struct cache *cache;
+
+ cache = cache_chain_instantiate(cpu_id);
+ if (!cache)
+ return;
+
+ cacheinfo_sysfs_populate(cpu_id, cache);
+}
+
+#ifdef CONFIG_HOTPLUG_CPU /* functions needed for cpu offline */
+
+static struct cache *cache_lookup_by_cpu(unsigned int cpu_id)
+{
+ struct device_node *cpu_node;
+ struct cache *cache;
+
+ cpu_node = of_get_cpu_node(cpu_id, NULL);
+ WARN_ONCE(!cpu_node, "no OF node found for CPU %i\n", cpu_id);
+ if (!cpu_node)
+ return NULL;
+
+ cache = cache_lookup_by_node(cpu_node);
+ of_node_put(cpu_node);
+
+ return cache;
+}
+
+static void remove_index_dirs(struct cache_dir *cache_dir)
+{
+ struct cache_index_dir *index;
+
+ index = cache_dir->index;
+
+ while (index) {
+ struct cache_index_dir *next;
+
+ next = index->next;
+ kobject_put(&index->kobj);
+ index = next;
+ }
+}
+
+static void remove_cache_dir(struct cache_dir *cache_dir)
+{
+ remove_index_dirs(cache_dir);
+
+ kobject_put(cache_dir->kobj);
+
+ kfree(cache_dir);
+}
+
+static void cache_cpu_clear(struct cache *cache, int cpu)
+{
+ while (cache) {
+ struct cache *next = cache->next_local;
+
+ WARN_ONCE(!cpumask_test_cpu(cpu, &cache->shared_cpu_map),
+ "CPU %i not accounted in %s(%s)\n",
+ cpu, cache->ofnode->full_name,
+ cache_type_string(cache));
+
+ cpumask_clear_cpu(cpu, &cache->shared_cpu_map);
+
+ /* Release the cache object if all the cpus using it
+ * are offline */
+ if (cpumask_empty(&cache->shared_cpu_map))
+ release_cache(cache);
+
+ cache = next;
+ }
+}
+
+void cacheinfo_cpu_offline(unsigned int cpu_id)
+{
+ struct cache_dir *cache_dir;
+ struct cache *cache;
+
+ /* Prevent userspace from seeing inconsistent state - remove
+ * the sysfs hierarchy first */
+ cache_dir = per_cpu(cache_dir, cpu_id);
+
+ /* careful, sysfs population may have failed */
+ if (cache_dir)
+ remove_cache_dir(cache_dir);
+
+ per_cpu(cache_dir, cpu_id) = NULL;
+
+ /* clear the CPU's bit in its cache chain, possibly freeing
+ * cache objects */
+ cache = cache_lookup_by_cpu(cpu_id);
+ if (cache)
+ cache_cpu_clear(cache, cpu_id);
+}
+#endif /* CONFIG_HOTPLUG_CPU */
diff --git a/arch/powerpc/kernel/cacheinfo.h b/arch/powerpc/kernel/cacheinfo.h
new file mode 100644
index 0000000..a7b74d3
--- /dev/null
+++ b/arch/powerpc/kernel/cacheinfo.h
@@ -0,0 +1,8 @@
+#ifndef _PPC_CACHEINFO_H
+#define _PPC_CACHEINFO_H
+
+/* These are just hooks for sysfs.c to use. */
+extern void cacheinfo_cpu_online(unsigned int cpu_id);
+extern void cacheinfo_cpu_offline(unsigned int cpu_id);
+
+#endif /* _PPC_CACHEINFO_H */
diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
index 2538030..da5a385 100644
--- a/arch/powerpc/kernel/pci-common.c
+++ b/arch/powerpc/kernel/pci-common.c
@@ -16,7 +16,7 @@
* 2 of the License, or (at your option) any later version.
*/
-#undef DEBUG
+#define DEBUG
#include <linux/kernel.h>
#include <linux/pci.h>
@@ -1356,6 +1356,63 @@
}
}
+static void __init pcibios_reserve_legacy_regions(struct pci_bus *bus)
+{
+ struct pci_controller *hose = pci_bus_to_host(bus);
+ resource_size_t offset;
+ struct resource *res, *pres;
+ int i;
+
+ pr_debug("Reserving legacy ranges for domain %04x\n", pci_domain_nr(bus));
+
+ /* Check for IO */
+ if (!(hose->io_resource.flags & IORESOURCE_IO))
+ goto no_io;
+ offset = (unsigned long)hose->io_base_virt - _IO_BASE;
+ res = kzalloc(sizeof(struct resource), GFP_KERNEL);
+ BUG_ON(res == NULL);
+ res->name = "Legacy IO";
+ res->flags = IORESOURCE_IO;
+ res->start = offset;
+ res->end = (offset + 0xfff) & 0xfffffffful;
+ pr_debug("Candidate legacy IO: %pR\n", res);
+ if (request_resource(&hose->io_resource, res)) {
+ printk(KERN_DEBUG
+ "PCI %04x:%02x Cannot reserve Legacy IO %pR\n",
+ pci_domain_nr(bus), bus->number, res);
+ kfree(res);
+ }
+
+ no_io:
+ /* Check for memory */
+ offset = hose->pci_mem_offset;
+ pr_debug("hose mem offset: %016llx\n", (unsigned long long)offset);
+ for (i = 0; i < 3; i++) {
+ pres = &hose->mem_resources[i];
+ if (!(pres->flags & IORESOURCE_MEM))
+ continue;
+ pr_debug("hose mem res: %pR\n", pres);
+ if ((pres->start - offset) <= 0xa0000 &&
+ (pres->end - offset) >= 0xbffff)
+ break;
+ }
+ if (i >= 3)
+ return;
+ res = kzalloc(sizeof(struct resource), GFP_KERNEL);
+ BUG_ON(res == NULL);
+ res->name = "Legacy VGA memory";
+ res->flags = IORESOURCE_MEM;
+ res->start = 0xa0000 + offset;
+ res->end = 0xbffff + offset;
+ pr_debug("Candidate VGA memory: %pR\n", res);
+ if (request_resource(pres, res)) {
+ printk(KERN_DEBUG
+ "PCI %04x:%02x Cannot reserve VGA memory %pR\n",
+ pci_domain_nr(bus), bus->number, res);
+ kfree(res);
+ }
+}
+
void __init pcibios_resource_survey(void)
{
struct pci_bus *b;
@@ -1371,6 +1428,18 @@
pcibios_allocate_resources(1);
}
+ /* Before we start assigning unassigned resource, we try to reserve
+ * the low IO area and the VGA memory area if they intersect the
+ * bus available resources to avoid allocating things on top of them
+ */
+ if (!(ppc_pci_flags & PPC_PCI_PROBE_ONLY)) {
+ list_for_each_entry(b, &pci_root_buses, node)
+ pcibios_reserve_legacy_regions(b);
+ }
+
+ /* Now, if the platform didn't decide to blindly trust the firmware,
+ * we proceed to assigning things that were left unassigned
+ */
if (!(ppc_pci_flags & PPC_PCI_PROBE_ONLY)) {
pr_debug("PCI: Assigning unassigned resouces...\n");
pci_assign_unassigned_resources();
diff --git a/arch/powerpc/kernel/pci_64.c b/arch/powerpc/kernel/pci_64.c
index 39fadc6..586962f 100644
--- a/arch/powerpc/kernel/pci_64.c
+++ b/arch/powerpc/kernel/pci_64.c
@@ -560,9 +560,14 @@
* G5 machines... So when something asks for bus 0 io base
* (bus 0 is HT root), we return the AGP one instead.
*/
- if (machine_is_compatible("MacRISC4"))
- if (in_bus == 0)
+ if (in_bus == 0 && machine_is_compatible("MacRISC4")) {
+ struct device_node *agp;
+
+ agp = of_find_compatible_node(NULL, NULL, "u3-agp");
+ if (agp)
in_bus = 0xf0;
+ of_node_put(agp);
+ }
/* That syscall isn't quite compatible with PCI domains, but it's
* used on pre-domains setup. We return the first match
diff --git a/arch/powerpc/kernel/ppc_ksyms.c b/arch/powerpc/kernel/ppc_ksyms.c
index dcec132..c8b27bb 100644
--- a/arch/powerpc/kernel/ppc_ksyms.c
+++ b/arch/powerpc/kernel/ppc_ksyms.c
@@ -165,6 +165,7 @@
EXPORT_SYMBOL(irq_desc);
EXPORT_SYMBOL(tb_ticks_per_jiffy);
EXPORT_SYMBOL(cacheable_memcpy);
+EXPORT_SYMBOL(cacheable_memzero);
#endif
#ifdef CONFIG_PPC32
diff --git a/arch/powerpc/kernel/prom.c b/arch/powerpc/kernel/prom.c
index 6f73c73..c09cffa 100644
--- a/arch/powerpc/kernel/prom.c
+++ b/arch/powerpc/kernel/prom.c
@@ -824,11 +824,11 @@
#endif
#ifdef CONFIG_KEXEC
- lprop = (u64*)of_get_flat_dt_prop(node, "linux,crashkernel-base", NULL);
+ lprop = of_get_flat_dt_prop(node, "linux,crashkernel-base", NULL);
if (lprop)
crashk_res.start = *lprop;
- lprop = (u64*)of_get_flat_dt_prop(node, "linux,crashkernel-size", NULL);
+ lprop = of_get_flat_dt_prop(node, "linux,crashkernel-size", NULL);
if (lprop)
crashk_res.end = crashk_res.start + *lprop - 1;
#endif
@@ -893,12 +893,12 @@
u64 base, size, lmb_size;
unsigned int is_kexec_kdump = 0, rngs;
- ls = (cell_t *)of_get_flat_dt_prop(node, "ibm,lmb-size", &l);
+ ls = of_get_flat_dt_prop(node, "ibm,lmb-size", &l);
if (ls == NULL || l < dt_root_size_cells * sizeof(cell_t))
return 0;
lmb_size = dt_mem_next_cell(dt_root_size_cells, &ls);
- dm = (cell_t *)of_get_flat_dt_prop(node, "ibm,dynamic-memory", &l);
+ dm = of_get_flat_dt_prop(node, "ibm,dynamic-memory", &l);
if (dm == NULL || l < sizeof(cell_t))
return 0;
@@ -907,7 +907,7 @@
return 0;
/* check if this is a kexec/kdump kernel. */
- usm = (cell_t *)of_get_flat_dt_prop(node, "linux,drconf-usable-memory",
+ usm = of_get_flat_dt_prop(node, "linux,drconf-usable-memory",
&l);
if (usm != NULL)
is_kexec_kdump = 1;
@@ -981,9 +981,9 @@
} else if (strcmp(type, "memory") != 0)
return 0;
- reg = (cell_t *)of_get_flat_dt_prop(node, "linux,usable-memory", &l);
+ reg = of_get_flat_dt_prop(node, "linux,usable-memory", &l);
if (reg == NULL)
- reg = (cell_t *)of_get_flat_dt_prop(node, "reg", &l);
+ reg = of_get_flat_dt_prop(node, "reg", &l);
if (reg == NULL)
return 0;
diff --git a/arch/powerpc/kernel/prom_init.c b/arch/powerpc/kernel/prom_init.c
index 2445945..7f1b33d 100644
--- a/arch/powerpc/kernel/prom_init.c
+++ b/arch/powerpc/kernel/prom_init.c
@@ -1210,7 +1210,7 @@
/* Initialize the table to have a one-to-one mapping
* over the allocated size.
*/
- tce_entryp = (unsigned long *)base;
+ tce_entryp = (u64 *)base;
for (i = 0; i < (minsize >> 3) ;tce_entryp++, i++) {
tce_entry = (i << PAGE_SHIFT);
tce_entry |= 0x3;
diff --git a/arch/powerpc/kernel/sysfs.c b/arch/powerpc/kernel/sysfs.c
index 0c64f10..4a2ee08 100644
--- a/arch/powerpc/kernel/sysfs.c
+++ b/arch/powerpc/kernel/sysfs.c
@@ -18,6 +18,8 @@
#include <asm/machdep.h>
#include <asm/smp.h>
+#include "cacheinfo.h"
+
#ifdef CONFIG_PPC64
#include <asm/paca.h>
#include <asm/lppaca.h>
@@ -25,8 +27,6 @@
static DEFINE_PER_CPU(struct cpu, cpu_devices);
-static DEFINE_PER_CPU(struct kobject *, cache_toplevel);
-
/*
* SMT snooze delay stuff, 64-bit only for now
*/
@@ -343,283 +343,6 @@
#endif /* HAS_PPC_PMC_PA6T */
#endif /* HAS_PPC_PMC_CLASSIC */
-struct cache_desc {
- struct kobject kobj;
- struct cache_desc *next;
- const char *type; /* Instruction, Data, or Unified */
- u32 size; /* total cache size in KB */
- u32 line_size; /* in bytes */
- u32 nr_sets; /* number of sets */
- u32 level; /* e.g. 1, 2, 3... */
- u32 associativity; /* e.g. 8-way... 0 is fully associative */
-};
-
-DEFINE_PER_CPU(struct cache_desc *, cache_desc);
-
-static struct cache_desc *kobj_to_cache_desc(struct kobject *k)
-{
- return container_of(k, struct cache_desc, kobj);
-}
-
-static void cache_desc_release(struct kobject *k)
-{
- struct cache_desc *desc = kobj_to_cache_desc(k);
-
- pr_debug("%s: releasing %s\n", __func__, kobject_name(k));
-
- if (desc->next)
- kobject_put(&desc->next->kobj);
-
- kfree(kobj_to_cache_desc(k));
-}
-
-static ssize_t cache_desc_show(struct kobject *k, struct attribute *attr, char *buf)
-{
- struct kobj_attribute *kobj_attr;
-
- kobj_attr = container_of(attr, struct kobj_attribute, attr);
-
- return kobj_attr->show(k, kobj_attr, buf);
-}
-
-static struct sysfs_ops cache_desc_sysfs_ops = {
- .show = cache_desc_show,
-};
-
-static struct kobj_type cache_desc_type = {
- .release = cache_desc_release,
- .sysfs_ops = &cache_desc_sysfs_ops,
-};
-
-static ssize_t cache_size_show(struct kobject *k, struct kobj_attribute *attr, char *buf)
-{
- struct cache_desc *cache = kobj_to_cache_desc(k);
-
- return sprintf(buf, "%uK\n", cache->size);
-}
-
-static struct kobj_attribute cache_size_attr =
- __ATTR(size, 0444, cache_size_show, NULL);
-
-static ssize_t cache_line_size_show(struct kobject *k, struct kobj_attribute *attr, char *buf)
-{
- struct cache_desc *cache = kobj_to_cache_desc(k);
-
- return sprintf(buf, "%u\n", cache->line_size);
-}
-
-static struct kobj_attribute cache_line_size_attr =
- __ATTR(coherency_line_size, 0444, cache_line_size_show, NULL);
-
-static ssize_t cache_nr_sets_show(struct kobject *k, struct kobj_attribute *attr, char *buf)
-{
- struct cache_desc *cache = kobj_to_cache_desc(k);
-
- return sprintf(buf, "%u\n", cache->nr_sets);
-}
-
-static struct kobj_attribute cache_nr_sets_attr =
- __ATTR(number_of_sets, 0444, cache_nr_sets_show, NULL);
-
-static ssize_t cache_type_show(struct kobject *k, struct kobj_attribute *attr, char *buf)
-{
- struct cache_desc *cache = kobj_to_cache_desc(k);
-
- return sprintf(buf, "%s\n", cache->type);
-}
-
-static struct kobj_attribute cache_type_attr =
- __ATTR(type, 0444, cache_type_show, NULL);
-
-static ssize_t cache_level_show(struct kobject *k, struct kobj_attribute *attr, char *buf)
-{
- struct cache_desc *cache = kobj_to_cache_desc(k);
-
- return sprintf(buf, "%u\n", cache->level);
-}
-
-static struct kobj_attribute cache_level_attr =
- __ATTR(level, 0444, cache_level_show, NULL);
-
-static ssize_t cache_assoc_show(struct kobject *k, struct kobj_attribute *attr, char *buf)
-{
- struct cache_desc *cache = kobj_to_cache_desc(k);
-
- return sprintf(buf, "%u\n", cache->associativity);
-}
-
-static struct kobj_attribute cache_assoc_attr =
- __ATTR(ways_of_associativity, 0444, cache_assoc_show, NULL);
-
-struct cache_desc_info {
- const char *type;
- const char *size_prop;
- const char *line_size_prop;
- const char *nr_sets_prop;
-};
-
-/* PowerPC Processor binding says the [di]-cache-* must be equal on
- * unified caches, so just use d-cache properties. */
-static struct cache_desc_info ucache_info = {
- .type = "Unified",
- .size_prop = "d-cache-size",
- .line_size_prop = "d-cache-line-size",
- .nr_sets_prop = "d-cache-sets",
-};
-
-static struct cache_desc_info dcache_info = {
- .type = "Data",
- .size_prop = "d-cache-size",
- .line_size_prop = "d-cache-line-size",
- .nr_sets_prop = "d-cache-sets",
-};
-
-static struct cache_desc_info icache_info = {
- .type = "Instruction",
- .size_prop = "i-cache-size",
- .line_size_prop = "i-cache-line-size",
- .nr_sets_prop = "i-cache-sets",
-};
-
-static struct cache_desc * __cpuinit create_cache_desc(struct device_node *np, struct kobject *parent, int index, int level, struct cache_desc_info *info)
-{
- const u32 *cache_line_size;
- struct cache_desc *new;
- const u32 *cache_size;
- const u32 *nr_sets;
- int rc;
-
- new = kzalloc(sizeof(*new), GFP_KERNEL);
- if (!new)
- return NULL;
-
- rc = kobject_init_and_add(&new->kobj, &cache_desc_type, parent,
- "index%d", index);
- if (rc)
- goto err;
-
- /* type */
- new->type = info->type;
- rc = sysfs_create_file(&new->kobj, &cache_type_attr.attr);
- WARN_ON(rc);
-
- /* level */
- new->level = level;
- rc = sysfs_create_file(&new->kobj, &cache_level_attr.attr);
- WARN_ON(rc);
-
- /* size */
- cache_size = of_get_property(np, info->size_prop, NULL);
- if (cache_size) {
- new->size = *cache_size / 1024;
- rc = sysfs_create_file(&new->kobj,
- &cache_size_attr.attr);
- WARN_ON(rc);
- }
-
- /* coherency_line_size */
- cache_line_size = of_get_property(np, info->line_size_prop, NULL);
- if (cache_line_size) {
- new->line_size = *cache_line_size;
- rc = sysfs_create_file(&new->kobj,
- &cache_line_size_attr.attr);
- WARN_ON(rc);
- }
-
- /* number_of_sets */
- nr_sets = of_get_property(np, info->nr_sets_prop, NULL);
- if (nr_sets) {
- new->nr_sets = *nr_sets;
- rc = sysfs_create_file(&new->kobj,
- &cache_nr_sets_attr.attr);
- WARN_ON(rc);
- }
-
- /* ways_of_associativity */
- if (new->nr_sets == 1) {
- /* fully associative */
- new->associativity = 0;
- goto create_assoc;
- }
-
- if (new->nr_sets && new->size && new->line_size) {
- /* If we have values for all of these we can derive
- * the associativity. */
- new->associativity =
- ((new->size * 1024) / new->nr_sets) / new->line_size;
-create_assoc:
- rc = sysfs_create_file(&new->kobj,
- &cache_assoc_attr.attr);
- WARN_ON(rc);
- }
-
- return new;
-err:
- kfree(new);
- return NULL;
-}
-
-static bool cache_is_unified(struct device_node *np)
-{
- return of_get_property(np, "cache-unified", NULL);
-}
-
-static struct cache_desc * __cpuinit create_cache_index_info(struct device_node *np, struct kobject *parent, int index, int level)
-{
- struct device_node *next_cache;
- struct cache_desc *new, **end;
-
- pr_debug("%s(node = %s, index = %d)\n", __func__, np->full_name, index);
-
- if (cache_is_unified(np)) {
- new = create_cache_desc(np, parent, index, level,
- &ucache_info);
- } else {
- new = create_cache_desc(np, parent, index, level,
- &dcache_info);
- if (new) {
- index++;
- new->next = create_cache_desc(np, parent, index, level,
- &icache_info);
- }
- }
- if (!new)
- return NULL;
-
- end = &new->next;
- while (*end)
- end = &(*end)->next;
-
- next_cache = of_find_next_cache_node(np);
- if (!next_cache)
- goto out;
-
- *end = create_cache_index_info(next_cache, parent, ++index, ++level);
-
- of_node_put(next_cache);
-out:
- return new;
-}
-
-static void __cpuinit create_cache_info(struct sys_device *sysdev)
-{
- struct kobject *cache_toplevel;
- struct device_node *np = NULL;
- int cpu = sysdev->id;
-
- cache_toplevel = kobject_create_and_add("cache", &sysdev->kobj);
- if (!cache_toplevel)
- return;
- per_cpu(cache_toplevel, cpu) = cache_toplevel;
- np = of_get_cpu_node(cpu, NULL);
- if (np != NULL) {
- per_cpu(cache_desc, cpu) =
- create_cache_index_info(np, cache_toplevel, 0, 1);
- of_node_put(np);
- }
- return;
-}
-
static void __cpuinit register_cpu_online(unsigned int cpu)
{
struct cpu *c = &per_cpu(cpu_devices, cpu);
@@ -684,25 +407,10 @@
sysdev_create_file(s, &attr_dscr);
#endif /* CONFIG_PPC64 */
- create_cache_info(s);
+ cacheinfo_cpu_online(cpu);
}
#ifdef CONFIG_HOTPLUG_CPU
-static void remove_cache_info(struct sys_device *sysdev)
-{
- struct kobject *cache_toplevel;
- struct cache_desc *cache_desc;
- int cpu = sysdev->id;
-
- cache_desc = per_cpu(cache_desc, cpu);
- if (cache_desc != NULL)
- kobject_put(&cache_desc->kobj);
-
- cache_toplevel = per_cpu(cache_toplevel, cpu);
- if (cache_toplevel != NULL)
- kobject_put(cache_toplevel);
-}
-
static void unregister_cpu_online(unsigned int cpu)
{
struct cpu *c = &per_cpu(cpu_devices, cpu);
@@ -769,7 +477,7 @@
sysdev_remove_file(s, &attr_dscr);
#endif /* CONFIG_PPC64 */
- remove_cache_info(s);
+ cacheinfo_cpu_offline(cpu);
}
#endif /* CONFIG_HOTPLUG_CPU */
diff --git a/arch/powerpc/mm/mmu_decl.h b/arch/powerpc/mm/mmu_decl.h
index 4314b39..ad123bc 100644
--- a/arch/powerpc/mm/mmu_decl.h
+++ b/arch/powerpc/mm/mmu_decl.h
@@ -30,11 +30,11 @@
#if defined(CONFIG_40x) || defined(CONFIG_8xx)
static inline void _tlbil_all(void)
{
- asm volatile ("sync; tlbia; isync" : : : "memory")
+ asm volatile ("sync; tlbia; isync" : : : "memory");
}
static inline void _tlbil_pid(unsigned int pid)
{
- asm volatile ("sync; tlbia; isync" : : : "memory")
+ asm volatile ("sync; tlbia; isync" : : : "memory");
}
#else /* CONFIG_40x || CONFIG_8xx */
extern void _tlbil_all(void);
@@ -47,7 +47,7 @@
#ifdef CONFIG_8xx
static inline void _tlbil_va(unsigned long address, unsigned int pid)
{
- asm volatile ("tlbie %0; sync" : : "r" (address) : "memory")
+ asm volatile ("tlbie %0; sync" : : "r" (address) : "memory");
}
#else /* CONFIG_8xx */
extern void _tlbil_va(unsigned long address, unsigned int pid);
diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c
index cf81049..7393bd7 100644
--- a/arch/powerpc/mm/numa.c
+++ b/arch/powerpc/mm/numa.c
@@ -822,42 +822,50 @@
* required. nid is the preferred node and end is the physical address of
* the highest address in the node.
*
- * Returns the physical address of the memory.
+ * Returns the virtual address of the memory.
*/
-static void __init *careful_allocation(int nid, unsigned long size,
+static void __init *careful_zallocation(int nid, unsigned long size,
unsigned long align,
unsigned long end_pfn)
{
+ void *ret;
int new_nid;
- unsigned long ret = __lmb_alloc_base(size, align, end_pfn << PAGE_SHIFT);
+ unsigned long ret_paddr;
+
+ ret_paddr = __lmb_alloc_base(size, align, end_pfn << PAGE_SHIFT);
/* retry over all memory */
- if (!ret)
- ret = __lmb_alloc_base(size, align, lmb_end_of_DRAM());
+ if (!ret_paddr)
+ ret_paddr = __lmb_alloc_base(size, align, lmb_end_of_DRAM());
- if (!ret)
- panic("numa.c: cannot allocate %lu bytes on node %d",
+ if (!ret_paddr)
+ panic("numa.c: cannot allocate %lu bytes for node %d",
size, nid);
+ ret = __va(ret_paddr);
+
/*
- * If the memory came from a previously allocated node, we must
- * retry with the bootmem allocator.
+ * We initialize the nodes in numeric order: 0, 1, 2...
+ * and hand over control from the LMB allocator to the
+ * bootmem allocator. If this function is called for
+ * node 5, then we know that all nodes <5 are using the
+ * bootmem allocator instead of the LMB allocator.
+ *
+ * So, check the nid from which this allocation came
+ * and double check to see if we need to use bootmem
+ * instead of the LMB. We don't free the LMB memory
+ * since it would be useless.
*/
- new_nid = early_pfn_to_nid(ret >> PAGE_SHIFT);
+ new_nid = early_pfn_to_nid(ret_paddr >> PAGE_SHIFT);
if (new_nid < nid) {
- ret = (unsigned long)__alloc_bootmem_node(NODE_DATA(new_nid),
+ ret = __alloc_bootmem_node(NODE_DATA(new_nid),
size, align, 0);
- if (!ret)
- panic("numa.c: cannot allocate %lu bytes on node %d",
- size, new_nid);
-
- ret = __pa(ret);
-
- dbg("alloc_bootmem %lx %lx\n", ret, size);
+ dbg("alloc_bootmem %p %lx\n", ret, size);
}
- return (void *)ret;
+ memset(ret, 0, size);
+ return ret;
}
static struct notifier_block __cpuinitdata ppc64_numa_nb = {
@@ -952,7 +960,7 @@
for_each_online_node(nid) {
unsigned long start_pfn, end_pfn;
- unsigned long bootmem_paddr;
+ void *bootmem_vaddr;
unsigned long bootmap_pages;
get_pfn_range_for_nid(nid, &start_pfn, &end_pfn);
@@ -964,11 +972,9 @@
* previous nodes' bootmem to be initialized and have
* all reserved areas marked.
*/
- NODE_DATA(nid) = careful_allocation(nid,
+ NODE_DATA(nid) = careful_zallocation(nid,
sizeof(struct pglist_data),
SMP_CACHE_BYTES, end_pfn);
- NODE_DATA(nid) = __va(NODE_DATA(nid));
- memset(NODE_DATA(nid), 0, sizeof(struct pglist_data));
dbg("node %d\n", nid);
dbg("NODE_DATA() = %p\n", NODE_DATA(nid));
@@ -984,20 +990,20 @@
dbg("end_paddr = %lx\n", end_pfn << PAGE_SHIFT);
bootmap_pages = bootmem_bootmap_pages(end_pfn - start_pfn);
- bootmem_paddr = (unsigned long)careful_allocation(nid,
+ bootmem_vaddr = careful_zallocation(nid,
bootmap_pages << PAGE_SHIFT,
PAGE_SIZE, end_pfn);
- memset(__va(bootmem_paddr), 0, bootmap_pages << PAGE_SHIFT);
- dbg("bootmap_paddr = %lx\n", bootmem_paddr);
+ dbg("bootmap_vaddr = %p\n", bootmem_vaddr);
- init_bootmem_node(NODE_DATA(nid), bootmem_paddr >> PAGE_SHIFT,
+ init_bootmem_node(NODE_DATA(nid),
+ __pa(bootmem_vaddr) >> PAGE_SHIFT,
start_pfn, end_pfn);
free_bootmem_with_active_regions(nid, end_pfn);
/*
* Be very careful about moving this around. Future
- * calls to careful_allocation() depend on this getting
+ * calls to careful_zallocation() depend on this getting
* done correctly.
*/
mark_reserved_regions_for_nid(nid);
diff --git a/arch/powerpc/mm/pgtable_32.c b/arch/powerpc/mm/pgtable_32.c
index 38ff35f..22972cd 100644
--- a/arch/powerpc/mm/pgtable_32.c
+++ b/arch/powerpc/mm/pgtable_32.c
@@ -266,7 +266,8 @@
/* The PTE should never be already set nor present in the
* hash table
*/
- BUG_ON(pte_val(*pg) & (_PAGE_PRESENT | _PAGE_HASHPTE));
+ BUG_ON((pte_val(*pg) & (_PAGE_PRESENT | _PAGE_HASHPTE)) &&
+ flags);
set_pte_at(&init_mm, va, pg, pfn_pte(pa >> PAGE_SHIFT,
__pgprot(flags)));
}
diff --git a/arch/powerpc/mm/tlb_nohash.c b/arch/powerpc/mm/tlb_nohash.c
index 803a64c..39ac22b 100644
--- a/arch/powerpc/mm/tlb_nohash.c
+++ b/arch/powerpc/mm/tlb_nohash.c
@@ -189,8 +189,9 @@
smp_call_function(do_flush_tlb_mm_ipi, NULL, 1);
_tlbil_pid(0);
preempt_enable();
-#endif
+#else
_tlbil_pid(0);
+#endif
}
EXPORT_SYMBOL(flush_tlb_kernel_range);
diff --git a/arch/powerpc/oprofile/cell/pr_util.h b/arch/powerpc/oprofile/cell/pr_util.h
index 628009c..964b939 100644
--- a/arch/powerpc/oprofile/cell/pr_util.h
+++ b/arch/powerpc/oprofile/cell/pr_util.h
@@ -30,6 +30,10 @@
extern struct delayed_work spu_work;
extern int spu_prof_running;
+#define TRACE_ARRAY_SIZE 1024
+
+extern spinlock_t oprof_spu_smpl_arry_lck;
+
struct spu_overlay_info { /* map of sections within an SPU overlay */
unsigned int vma; /* SPU virtual memory address from elf */
unsigned int size; /* size of section from elf */
@@ -79,7 +83,7 @@
* the vma-to-fileoffset map.
*/
struct vma_to_fileoffset_map *create_vma_map(const struct spu *spu,
- u64 objectid);
+ unsigned long objectid);
unsigned int vma_map_lookup(struct vma_to_fileoffset_map *map,
unsigned int vma, const struct spu *aSpu,
int *grd_val);
@@ -89,10 +93,11 @@
* Entry point for SPU profiling.
* cycles_reset is the SPU_CYCLES count value specified by the user.
*/
-int start_spu_profiling(unsigned int cycles_reset);
+int start_spu_profiling_cycles(unsigned int cycles_reset);
+void start_spu_profiling_events(void);
-void stop_spu_profiling(void);
-
+void stop_spu_profiling_cycles(void);
+void stop_spu_profiling_events(void);
/* add the necessary profiling hooks */
int spu_sync_start(void);
diff --git a/arch/powerpc/oprofile/cell/spu_profiler.c b/arch/powerpc/oprofile/cell/spu_profiler.c
index 83faa95..9305dda 100644
--- a/arch/powerpc/oprofile/cell/spu_profiler.c
+++ b/arch/powerpc/oprofile/cell/spu_profiler.c
@@ -18,11 +18,21 @@
#include <asm/cell-pmu.h>
#include "pr_util.h"
-#define TRACE_ARRAY_SIZE 1024
#define SCALE_SHIFT 14
static u32 *samples;
+/* spu_prof_running is a flag used to indicate if spu profiling is enabled
+ * or not. It is set by the routines start_spu_profiling_cycles() and
+ * start_spu_profiling_events(). The flag is cleared by the routines
+ * stop_spu_profiling_cycles() and stop_spu_profiling_events(). These
+ * routines are called via global_start() and global_stop() which are called in
+ * op_powerpc_start() and op_powerpc_stop(). These routines are called once
+ * per system as a result of the user starting/stopping oprofile. Hence, only
+ * one CPU per user at a time will be changing the value of spu_prof_running.
+ * In general, OProfile does not protect against multiple users trying to run
+ * OProfile at a time.
+ */
int spu_prof_running;
static unsigned int profiling_interval;
@@ -31,8 +41,8 @@
#define SPU_PC_MASK 0xFFFF
-static DEFINE_SPINLOCK(sample_array_lock);
-unsigned long sample_array_lock_flags;
+DEFINE_SPINLOCK(oprof_spu_smpl_arry_lck);
+unsigned long oprof_spu_smpl_arry_lck_flags;
void set_spu_profiling_frequency(unsigned int freq_khz, unsigned int cycles_reset)
{
@@ -145,13 +155,13 @@
* sample array must be loaded and then processed for a given
* cpu. The sample array is not per cpu.
*/
- spin_lock_irqsave(&sample_array_lock,
- sample_array_lock_flags);
+ spin_lock_irqsave(&oprof_spu_smpl_arry_lck,
+ oprof_spu_smpl_arry_lck_flags);
num_samples = cell_spu_pc_collection(cpu);
if (num_samples == 0) {
- spin_unlock_irqrestore(&sample_array_lock,
- sample_array_lock_flags);
+ spin_unlock_irqrestore(&oprof_spu_smpl_arry_lck,
+ oprof_spu_smpl_arry_lck_flags);
continue;
}
@@ -162,8 +172,8 @@
num_samples);
}
- spin_unlock_irqrestore(&sample_array_lock,
- sample_array_lock_flags);
+ spin_unlock_irqrestore(&oprof_spu_smpl_arry_lck,
+ oprof_spu_smpl_arry_lck_flags);
}
smp_wmb(); /* insure spu event buffer updates are written */
@@ -182,13 +192,13 @@
static struct hrtimer timer;
/*
- * Entry point for SPU profiling.
+ * Entry point for SPU cycle profiling.
* NOTE: SPU profiling is done system-wide, not per-CPU.
*
* cycles_reset is the count value specified by the user when
* setting up OProfile to count SPU_CYCLES.
*/
-int start_spu_profiling(unsigned int cycles_reset)
+int start_spu_profiling_cycles(unsigned int cycles_reset)
{
ktime_t kt;
@@ -212,10 +222,30 @@
return 0;
}
-void stop_spu_profiling(void)
+/*
+ * Entry point for SPU event profiling.
+ * NOTE: SPU profiling is done system-wide, not per-CPU.
+ *
+ * cycles_reset is the count value specified by the user when
+ * setting up OProfile to count SPU_CYCLES.
+ */
+void start_spu_profiling_events(void)
+{
+ spu_prof_running = 1;
+ schedule_delayed_work(&spu_work, DEFAULT_TIMER_EXPIRE);
+
+ return;
+}
+
+void stop_spu_profiling_cycles(void)
{
spu_prof_running = 0;
hrtimer_cancel(&timer);
kfree(samples);
- pr_debug("SPU_PROF: stop_spu_profiling issued\n");
+ pr_debug("SPU_PROF: stop_spu_profiling_cycles issued\n");
+}
+
+void stop_spu_profiling_events(void)
+{
+ spu_prof_running = 0;
}
diff --git a/arch/powerpc/oprofile/common.c b/arch/powerpc/oprofile/common.c
index 17807ac..21f16ed 100644
--- a/arch/powerpc/oprofile/common.c
+++ b/arch/powerpc/oprofile/common.c
@@ -132,6 +132,28 @@
oprofilefs_create_ulong(sb, root, "mmcr0", &sys.mmcr0);
oprofilefs_create_ulong(sb, root, "mmcr1", &sys.mmcr1);
oprofilefs_create_ulong(sb, root, "mmcra", &sys.mmcra);
+#ifdef CONFIG_OPROFILE_CELL
+ /* create a file the user tool can check to see what level of profiling
+ * support exits with this kernel. Initialize bit mask to indicate
+ * what support the kernel has:
+ * bit 0 - Supports SPU event profiling in addition to PPU
+ * event and cycles; and SPU cycle profiling
+ * bits 1-31 - Currently unused.
+ *
+ * If the file does not exist, then the kernel only supports SPU
+ * cycle profiling, PPU event and cycle profiling.
+ */
+ oprofilefs_create_ulong(sb, root, "cell_support", &sys.cell_support);
+ sys.cell_support = 0x1; /* Note, the user OProfile tool must check
+ * that this bit is set before attempting to
+ * user SPU event profiling. Older kernels
+ * will not have this file, hence the user
+ * tool is not allowed to do SPU event
+ * profiling on older kernels. Older kernels
+ * will accept SPU events but collected data
+ * is garbage.
+ */
+#endif
#endif
for (i = 0; i < model->num_counters; ++i) {
diff --git a/arch/powerpc/oprofile/op_model_cell.c b/arch/powerpc/oprofile/op_model_cell.c
index 25a4ec2..ae06c62 100644
--- a/arch/powerpc/oprofile/op_model_cell.c
+++ b/arch/powerpc/oprofile/op_model_cell.c
@@ -40,14 +40,15 @@
#include "../platforms/cell/interrupt.h"
#include "cell/pr_util.h"
-static void cell_global_stop_spu(void);
+#define PPU_PROFILING 0
+#define SPU_PROFILING_CYCLES 1
+#define SPU_PROFILING_EVENTS 2
-/*
- * spu_cycle_reset is the number of cycles between samples.
- * This variable is used for SPU profiling and should ONLY be set
- * at the beginning of cell_reg_setup; otherwise, it's read-only.
- */
-static unsigned int spu_cycle_reset;
+#define SPU_EVENT_NUM_START 4100
+#define SPU_EVENT_NUM_STOP 4399
+#define SPU_PROFILE_EVENT_ADDR 4363 /* spu, address trace, decimal */
+#define SPU_PROFILE_EVENT_ADDR_MASK_A 0x146 /* sub unit set to zero */
+#define SPU_PROFILE_EVENT_ADDR_MASK_B 0x186 /* sub unit set to zero */
#define NUM_SPUS_PER_NODE 8
#define SPU_CYCLES_EVENT_NUM 2 /* event number for SPU_CYCLES */
@@ -66,6 +67,21 @@
#define MAX_SPU_COUNT 0xFFFFFF /* maximum 24 bit LFSR value */
+/* Minumum HW interval timer setting to send value to trace buffer is 10 cycle.
+ * To configure counter to send value every N cycles set counter to
+ * 2^32 - 1 - N.
+ */
+#define NUM_INTERVAL_CYC 0xFFFFFFFF - 10
+
+/*
+ * spu_cycle_reset is the number of cycles between samples.
+ * This variable is used for SPU profiling and should ONLY be set
+ * at the beginning of cell_reg_setup; otherwise, it's read-only.
+ */
+static unsigned int spu_cycle_reset;
+static unsigned int profiling_mode;
+static int spu_evnt_phys_spu_indx;
+
struct pmc_cntrl_data {
unsigned long vcntr;
unsigned long evnts;
@@ -105,6 +121,8 @@
u16 trace_mode;
u16 freeze;
u16 count_mode;
+ u16 spu_addr_trace;
+ u8 trace_buf_ovflw;
};
static struct {
@@ -122,7 +140,7 @@
#define GET_INPUT_CONTROL(x) ((x & 0x00000004) >> 2)
static DEFINE_PER_CPU(unsigned long[NR_PHYS_CTRS], pmc_values);
-
+static unsigned long spu_pm_cnt[MAX_NUMNODES * NUM_SPUS_PER_NODE];
static struct pmc_cntrl_data pmc_cntrl[NUM_THREADS][NR_PHYS_CTRS];
/*
@@ -152,6 +170,7 @@
static u32 virt_cntr_inter_mask;
static struct timer_list timer_virt_cntr;
+static struct timer_list timer_spu_event_swap;
/*
* pm_signal needs to be global since it is initialized in
@@ -165,7 +184,7 @@
static u32 reset_value[NR_PHYS_CTRS];
static int num_counters;
static int oprofile_running;
-static DEFINE_SPINLOCK(virt_cntr_lock);
+static DEFINE_SPINLOCK(cntr_lock);
static u32 ctr_enabled;
@@ -336,13 +355,13 @@
for (i = 0; i < NUM_DEBUG_BUS_WORDS; i++) {
if (bus_word & (1 << i)) {
pm_regs.debug_bus_control |=
- (bus_type << (30 - (2 * i)));
+ (bus_type << (30 - (2 * i)));
for (j = 0; j < NUM_INPUT_BUS_WORDS; j++) {
if (input_bus[j] == 0xff) {
input_bus[j] = i;
pm_regs.group_control |=
- (i << (30 - (2 * j)));
+ (i << (30 - (2 * j)));
break;
}
@@ -367,12 +386,16 @@
if (pm_regs.pm_cntrl.stop_at_max == 1)
val |= CBE_PM_STOP_AT_MAX;
- if (pm_regs.pm_cntrl.trace_mode == 1)
+ if (pm_regs.pm_cntrl.trace_mode != 0)
val |= CBE_PM_TRACE_MODE_SET(pm_regs.pm_cntrl.trace_mode);
+ if (pm_regs.pm_cntrl.trace_buf_ovflw == 1)
+ val |= CBE_PM_TRACE_BUF_OVFLW(pm_regs.pm_cntrl.trace_buf_ovflw);
if (pm_regs.pm_cntrl.freeze == 1)
val |= CBE_PM_FREEZE_ALL_CTRS;
+ val |= CBE_PM_SPU_ADDR_TRACE_SET(pm_regs.pm_cntrl.spu_addr_trace);
+
/*
* Routine set_count_mode must be called previously to set
* the count mode based on the user selection of user and kernel.
@@ -441,7 +464,7 @@
* not both playing with the counters on the same node.
*/
- spin_lock_irqsave(&virt_cntr_lock, flags);
+ spin_lock_irqsave(&cntr_lock, flags);
prev_hdw_thread = hdw_thread;
@@ -480,7 +503,7 @@
cbe_disable_pm_interrupts(cpu);
for (i = 0; i < num_counters; i++) {
per_cpu(pmc_values, cpu + prev_hdw_thread)[i]
- = cbe_read_ctr(cpu, i);
+ = cbe_read_ctr(cpu, i);
if (per_cpu(pmc_values, cpu + next_hdw_thread)[i]
== 0xFFFFFFFF)
@@ -527,7 +550,7 @@
cbe_enable_pm(cpu);
}
- spin_unlock_irqrestore(&virt_cntr_lock, flags);
+ spin_unlock_irqrestore(&cntr_lock, flags);
mod_timer(&timer_virt_cntr, jiffies + HZ / 10);
}
@@ -541,38 +564,146 @@
add_timer(&timer_virt_cntr);
}
-/* This function is called once for all cpus combined */
-static int cell_reg_setup(struct op_counter_config *ctr,
+static int cell_reg_setup_spu_cycles(struct op_counter_config *ctr,
struct op_system_config *sys, int num_ctrs)
{
- int i, j, cpu;
- spu_cycle_reset = 0;
-
- if (ctr[0].event == SPU_CYCLES_EVENT_NUM) {
- spu_cycle_reset = ctr[0].count;
-
- /*
- * Each node will need to make the rtas call to start
- * and stop SPU profiling. Get the token once and store it.
- */
- spu_rtas_token = rtas_token("ibm,cbe-spu-perftools");
-
- if (unlikely(spu_rtas_token == RTAS_UNKNOWN_SERVICE)) {
- printk(KERN_ERR
- "%s: rtas token ibm,cbe-spu-perftools unknown\n",
- __func__);
- return -EIO;
- }
- }
-
- pm_rtas_token = rtas_token("ibm,cbe-perftools");
+ spu_cycle_reset = ctr[0].count;
/*
- * For all events excetp PPU CYCLEs, each node will need to make
+ * Each node will need to make the rtas call to start
+ * and stop SPU profiling. Get the token once and store it.
+ */
+ spu_rtas_token = rtas_token("ibm,cbe-spu-perftools");
+
+ if (unlikely(spu_rtas_token == RTAS_UNKNOWN_SERVICE)) {
+ printk(KERN_ERR
+ "%s: rtas token ibm,cbe-spu-perftools unknown\n",
+ __func__);
+ return -EIO;
+ }
+ return 0;
+}
+
+/* Unfortunately, the hardware will only support event profiling
+ * on one SPU per node at a time. Therefore, we must time slice
+ * the profiling across all SPUs in the node. Note, we do this
+ * in parallel for each node. The following routine is called
+ * periodically based on kernel timer to switch which SPU is
+ * being monitored in a round robbin fashion.
+ */
+static void spu_evnt_swap(unsigned long data)
+{
+ int node;
+ int cur_phys_spu, nxt_phys_spu, cur_spu_evnt_phys_spu_indx;
+ unsigned long flags;
+ int cpu;
+ int ret;
+ u32 interrupt_mask;
+
+
+ /* enable interrupts on cntr 0 */
+ interrupt_mask = CBE_PM_CTR_OVERFLOW_INTR(0);
+
+ hdw_thread = 0;
+
+ /* Make sure spu event interrupt handler and spu event swap
+ * don't access the counters simultaneously.
+ */
+ spin_lock_irqsave(&cntr_lock, flags);
+
+ cur_spu_evnt_phys_spu_indx = spu_evnt_phys_spu_indx;
+
+ if (++(spu_evnt_phys_spu_indx) == NUM_SPUS_PER_NODE)
+ spu_evnt_phys_spu_indx = 0;
+
+ pm_signal[0].sub_unit = spu_evnt_phys_spu_indx;
+ pm_signal[1].sub_unit = spu_evnt_phys_spu_indx;
+ pm_signal[2].sub_unit = spu_evnt_phys_spu_indx;
+
+ /* switch the SPU being profiled on each node */
+ for_each_online_cpu(cpu) {
+ if (cbe_get_hw_thread_id(cpu))
+ continue;
+
+ node = cbe_cpu_to_node(cpu);
+ cur_phys_spu = (node * NUM_SPUS_PER_NODE)
+ + cur_spu_evnt_phys_spu_indx;
+ nxt_phys_spu = (node * NUM_SPUS_PER_NODE)
+ + spu_evnt_phys_spu_indx;
+
+ /*
+ * stop counters, save counter values, restore counts
+ * for previous physical SPU
+ */
+ cbe_disable_pm(cpu);
+ cbe_disable_pm_interrupts(cpu);
+
+ spu_pm_cnt[cur_phys_spu]
+ = cbe_read_ctr(cpu, 0);
+
+ /* restore previous count for the next spu to sample */
+ /* NOTE, hardware issue, counter will not start if the
+ * counter value is at max (0xFFFFFFFF).
+ */
+ if (spu_pm_cnt[nxt_phys_spu] >= 0xFFFFFFFF)
+ cbe_write_ctr(cpu, 0, 0xFFFFFFF0);
+ else
+ cbe_write_ctr(cpu, 0, spu_pm_cnt[nxt_phys_spu]);
+
+ pm_rtas_reset_signals(cbe_cpu_to_node(cpu));
+
+ /* setup the debug bus measure the one event and
+ * the two events to route the next SPU's PC on
+ * the debug bus
+ */
+ ret = pm_rtas_activate_signals(cbe_cpu_to_node(cpu), 3);
+ if (ret)
+ printk(KERN_ERR "%s: pm_rtas_activate_signals failed, "
+ "SPU event swap\n", __func__);
+
+ /* clear the trace buffer, don't want to take PC for
+ * previous SPU*/
+ cbe_write_pm(cpu, trace_address, 0);
+
+ enable_ctr(cpu, 0, pm_regs.pm07_cntrl);
+
+ /* Enable interrupts on the CPU thread that is starting */
+ cbe_enable_pm_interrupts(cpu, hdw_thread,
+ interrupt_mask);
+ cbe_enable_pm(cpu);
+ }
+
+ spin_unlock_irqrestore(&cntr_lock, flags);
+
+ /* swap approximately every 0.1 seconds */
+ mod_timer(&timer_spu_event_swap, jiffies + HZ / 25);
+}
+
+static void start_spu_event_swap(void)
+{
+ init_timer(&timer_spu_event_swap);
+ timer_spu_event_swap.function = spu_evnt_swap;
+ timer_spu_event_swap.data = 0UL;
+ timer_spu_event_swap.expires = jiffies + HZ / 25;
+ add_timer(&timer_spu_event_swap);
+}
+
+static int cell_reg_setup_spu_events(struct op_counter_config *ctr,
+ struct op_system_config *sys, int num_ctrs)
+{
+ int i;
+
+ /* routine is called once for all nodes */
+
+ spu_evnt_phys_spu_indx = 0;
+ /*
+ * For all events except PPU CYCLEs, each node will need to make
* the rtas cbe-perftools call to setup and reset the debug bus.
* Make the token lookup call once and store it in the global
* variable pm_rtas_token.
*/
+ pm_rtas_token = rtas_token("ibm,cbe-perftools");
+
if (unlikely(pm_rtas_token == RTAS_UNKNOWN_SERVICE)) {
printk(KERN_ERR
"%s: rtas token ibm,cbe-perftools unknown\n",
@@ -580,6 +711,58 @@
return -EIO;
}
+ /* setup the pm_control register settings,
+ * settings will be written per node by the
+ * cell_cpu_setup() function.
+ */
+ pm_regs.pm_cntrl.trace_buf_ovflw = 1;
+
+ /* Use the occurrence trace mode to have SPU PC saved
+ * to the trace buffer. Occurrence data in trace buffer
+ * is not used. Bit 2 must be set to store SPU addresses.
+ */
+ pm_regs.pm_cntrl.trace_mode = 2;
+
+ pm_regs.pm_cntrl.spu_addr_trace = 0x1; /* using debug bus
+ event 2 & 3 */
+
+ /* setup the debug bus event array with the SPU PC routing events.
+ * Note, pm_signal[0] will be filled in by set_pm_event() call below.
+ */
+ pm_signal[1].signal_group = SPU_PROFILE_EVENT_ADDR / 100;
+ pm_signal[1].bus_word = GET_BUS_WORD(SPU_PROFILE_EVENT_ADDR_MASK_A);
+ pm_signal[1].bit = SPU_PROFILE_EVENT_ADDR % 100;
+ pm_signal[1].sub_unit = spu_evnt_phys_spu_indx;
+
+ pm_signal[2].signal_group = SPU_PROFILE_EVENT_ADDR / 100;
+ pm_signal[2].bus_word = GET_BUS_WORD(SPU_PROFILE_EVENT_ADDR_MASK_B);
+ pm_signal[2].bit = SPU_PROFILE_EVENT_ADDR % 100;
+ pm_signal[2].sub_unit = spu_evnt_phys_spu_indx;
+
+ /* Set the user selected spu event to profile on,
+ * note, only one SPU profiling event is supported
+ */
+ num_counters = 1; /* Only support one SPU event at a time */
+ set_pm_event(0, ctr[0].event, ctr[0].unit_mask);
+
+ reset_value[0] = 0xFFFFFFFF - ctr[0].count;
+
+ /* global, used by cell_cpu_setup */
+ ctr_enabled |= 1;
+
+ /* Initialize the count for each SPU to the reset value */
+ for (i=0; i < MAX_NUMNODES * NUM_SPUS_PER_NODE; i++)
+ spu_pm_cnt[i] = reset_value[0];
+
+ return 0;
+}
+
+static int cell_reg_setup_ppu(struct op_counter_config *ctr,
+ struct op_system_config *sys, int num_ctrs)
+{
+ /* routine is called once for all nodes */
+ int i, j, cpu;
+
num_counters = num_ctrs;
if (unlikely(num_ctrs > NR_PHYS_CTRS)) {
@@ -589,14 +772,6 @@
__func__);
return -EIO;
}
- pm_regs.group_control = 0;
- pm_regs.debug_bus_control = 0;
-
- /* setup the pm_control register */
- memset(&pm_regs.pm_cntrl, 0, sizeof(struct pm_cntrl));
- pm_regs.pm_cntrl.stop_at_max = 1;
- pm_regs.pm_cntrl.trace_mode = 0;
- pm_regs.pm_cntrl.freeze = 1;
set_count_mode(sys->enable_kernel, sys->enable_user);
@@ -665,6 +840,63 @@
}
+/* This function is called once for all cpus combined */
+static int cell_reg_setup(struct op_counter_config *ctr,
+ struct op_system_config *sys, int num_ctrs)
+{
+ int ret=0;
+ spu_cycle_reset = 0;
+
+ /* initialize the spu_arr_trace value, will be reset if
+ * doing spu event profiling.
+ */
+ pm_regs.group_control = 0;
+ pm_regs.debug_bus_control = 0;
+ pm_regs.pm_cntrl.stop_at_max = 1;
+ pm_regs.pm_cntrl.trace_mode = 0;
+ pm_regs.pm_cntrl.freeze = 1;
+ pm_regs.pm_cntrl.trace_buf_ovflw = 0;
+ pm_regs.pm_cntrl.spu_addr_trace = 0;
+
+ /*
+ * For all events except PPU CYCLEs, each node will need to make
+ * the rtas cbe-perftools call to setup and reset the debug bus.
+ * Make the token lookup call once and store it in the global
+ * variable pm_rtas_token.
+ */
+ pm_rtas_token = rtas_token("ibm,cbe-perftools");
+
+ if (unlikely(pm_rtas_token == RTAS_UNKNOWN_SERVICE)) {
+ printk(KERN_ERR
+ "%s: rtas token ibm,cbe-perftools unknown\n",
+ __func__);
+ return -EIO;
+ }
+
+ if (ctr[0].event == SPU_CYCLES_EVENT_NUM) {
+ profiling_mode = SPU_PROFILING_CYCLES;
+ ret = cell_reg_setup_spu_cycles(ctr, sys, num_ctrs);
+ } else if ((ctr[0].event >= SPU_EVENT_NUM_START) &&
+ (ctr[0].event <= SPU_EVENT_NUM_STOP)) {
+ profiling_mode = SPU_PROFILING_EVENTS;
+ spu_cycle_reset = ctr[0].count;
+
+ /* for SPU event profiling, need to setup the
+ * pm_signal array with the events to route the
+ * SPU PC before making the FW call. Note, only
+ * one SPU event for profiling can be specified
+ * at a time.
+ */
+ cell_reg_setup_spu_events(ctr, sys, num_ctrs);
+ } else {
+ profiling_mode = PPU_PROFILING;
+ ret = cell_reg_setup_ppu(ctr, sys, num_ctrs);
+ }
+
+ return ret;
+}
+
+
/* This function is called once for each cpu */
static int cell_cpu_setup(struct op_counter_config *cntr)
@@ -672,8 +904,13 @@
u32 cpu = smp_processor_id();
u32 num_enabled = 0;
int i;
+ int ret;
- if (spu_cycle_reset)
+ /* Cycle based SPU profiling does not use the performance
+ * counters. The trace array is configured to collect
+ * the data.
+ */
+ if (profiling_mode == SPU_PROFILING_CYCLES)
return 0;
/* There is one performance monitor per processor chip (i.e. node),
@@ -686,7 +923,6 @@
cbe_disable_pm(cpu);
cbe_disable_pm_interrupts(cpu);
- cbe_write_pm(cpu, pm_interval, 0);
cbe_write_pm(cpu, pm_start_stop, 0);
cbe_write_pm(cpu, group_control, pm_regs.group_control);
cbe_write_pm(cpu, debug_bus_control, pm_regs.debug_bus_control);
@@ -703,7 +939,20 @@
* The pm_rtas_activate_signals will return -EIO if the FW
* call failed.
*/
- return pm_rtas_activate_signals(cbe_cpu_to_node(cpu), num_enabled);
+ if (profiling_mode == SPU_PROFILING_EVENTS) {
+ /* For SPU event profiling also need to setup the
+ * pm interval timer
+ */
+ ret = pm_rtas_activate_signals(cbe_cpu_to_node(cpu),
+ num_enabled+2);
+ /* store PC from debug bus to Trace buffer as often
+ * as possible (every 10 cycles)
+ */
+ cbe_write_pm(cpu, pm_interval, NUM_INTERVAL_CYC);
+ return ret;
+ } else
+ return pm_rtas_activate_signals(cbe_cpu_to_node(cpu),
+ num_enabled);
}
#define ENTRIES 303
@@ -885,7 +1134,122 @@
};
#endif
-static int cell_global_start_spu(struct op_counter_config *ctr)
+/*
+ * Note the generic OProfile stop calls do not support returning
+ * an error on stop. Hence, will not return an error if the FW
+ * calls fail on stop. Failure to reset the debug bus is not an issue.
+ * Failure to disable the SPU profiling is not an issue. The FW calls
+ * to enable the performance counters and debug bus will work even if
+ * the hardware was not cleanly reset.
+ */
+static void cell_global_stop_spu_cycles(void)
+{
+ int subfunc, rtn_value;
+ unsigned int lfsr_value;
+ int cpu;
+
+ oprofile_running = 0;
+ smp_wmb();
+
+#ifdef CONFIG_CPU_FREQ
+ cpufreq_unregister_notifier(&cpu_freq_notifier_block,
+ CPUFREQ_TRANSITION_NOTIFIER);
+#endif
+
+ for_each_online_cpu(cpu) {
+ if (cbe_get_hw_thread_id(cpu))
+ continue;
+
+ subfunc = 3; /*
+ * 2 - activate SPU tracing,
+ * 3 - deactivate
+ */
+ lfsr_value = 0x8f100000;
+
+ rtn_value = rtas_call(spu_rtas_token, 3, 1, NULL,
+ subfunc, cbe_cpu_to_node(cpu),
+ lfsr_value);
+
+ if (unlikely(rtn_value != 0)) {
+ printk(KERN_ERR
+ "%s: rtas call ibm,cbe-spu-perftools " \
+ "failed, return = %d\n",
+ __func__, rtn_value);
+ }
+
+ /* Deactivate the signals */
+ pm_rtas_reset_signals(cbe_cpu_to_node(cpu));
+ }
+
+ stop_spu_profiling_cycles();
+}
+
+static void cell_global_stop_spu_events(void)
+{
+ int cpu;
+ oprofile_running = 0;
+
+ stop_spu_profiling_events();
+ smp_wmb();
+
+ for_each_online_cpu(cpu) {
+ if (cbe_get_hw_thread_id(cpu))
+ continue;
+
+ cbe_sync_irq(cbe_cpu_to_node(cpu));
+ /* Stop the counters */
+ cbe_disable_pm(cpu);
+ cbe_write_pm07_control(cpu, 0, 0);
+
+ /* Deactivate the signals */
+ pm_rtas_reset_signals(cbe_cpu_to_node(cpu));
+
+ /* Deactivate interrupts */
+ cbe_disable_pm_interrupts(cpu);
+ }
+ del_timer_sync(&timer_spu_event_swap);
+}
+
+static void cell_global_stop_ppu(void)
+{
+ int cpu;
+
+ /*
+ * This routine will be called once for the system.
+ * There is one performance monitor per node, so we
+ * only need to perform this function once per node.
+ */
+ del_timer_sync(&timer_virt_cntr);
+ oprofile_running = 0;
+ smp_wmb();
+
+ for_each_online_cpu(cpu) {
+ if (cbe_get_hw_thread_id(cpu))
+ continue;
+
+ cbe_sync_irq(cbe_cpu_to_node(cpu));
+ /* Stop the counters */
+ cbe_disable_pm(cpu);
+
+ /* Deactivate the signals */
+ pm_rtas_reset_signals(cbe_cpu_to_node(cpu));
+
+ /* Deactivate interrupts */
+ cbe_disable_pm_interrupts(cpu);
+ }
+}
+
+static void cell_global_stop(void)
+{
+ if (profiling_mode == PPU_PROFILING)
+ cell_global_stop_ppu();
+ else if (profiling_mode == SPU_PROFILING_EVENTS)
+ cell_global_stop_spu_events();
+ else
+ cell_global_stop_spu_cycles();
+}
+
+static int cell_global_start_spu_cycles(struct op_counter_config *ctr)
{
int subfunc;
unsigned int lfsr_value;
@@ -951,18 +1315,18 @@
/* start profiling */
ret = rtas_call(spu_rtas_token, 3, 1, NULL, subfunc,
- cbe_cpu_to_node(cpu), lfsr_value);
+ cbe_cpu_to_node(cpu), lfsr_value);
if (unlikely(ret != 0)) {
printk(KERN_ERR
- "%s: rtas call ibm,cbe-spu-perftools failed, return = %d\n",
- __func__, ret);
+ "%s: rtas call ibm,cbe-spu-perftools failed, " \
+ "return = %d\n", __func__, ret);
rtas_error = -EIO;
goto out;
}
}
- rtas_error = start_spu_profiling(spu_cycle_reset);
+ rtas_error = start_spu_profiling_cycles(spu_cycle_reset);
if (rtas_error)
goto out_stop;
@@ -970,11 +1334,74 @@
return 0;
out_stop:
- cell_global_stop_spu(); /* clean up the PMU/debug bus */
+ cell_global_stop_spu_cycles(); /* clean up the PMU/debug bus */
out:
return rtas_error;
}
+static int cell_global_start_spu_events(struct op_counter_config *ctr)
+{
+ int cpu;
+ u32 interrupt_mask = 0;
+ int rtn = 0;
+
+ hdw_thread = 0;
+
+ /* spu event profiling, uses the performance counters to generate
+ * an interrupt. The hardware is setup to store the SPU program
+ * counter into the trace array. The occurrence mode is used to
+ * enable storing data to the trace buffer. The bits are set
+ * to send/store the SPU address in the trace buffer. The debug
+ * bus must be setup to route the SPU program counter onto the
+ * debug bus. The occurrence data in the trace buffer is not used.
+ */
+
+ /* This routine gets called once for the system.
+ * There is one performance monitor per node, so we
+ * only need to perform this function once per node.
+ */
+
+ for_each_online_cpu(cpu) {
+ if (cbe_get_hw_thread_id(cpu))
+ continue;
+
+ /*
+ * Setup SPU event-based profiling.
+ * Set perf_mon_control bit 0 to a zero before
+ * enabling spu collection hardware.
+ *
+ * Only support one SPU event on one SPU per node.
+ */
+ if (ctr_enabled & 1) {
+ cbe_write_ctr(cpu, 0, reset_value[0]);
+ enable_ctr(cpu, 0, pm_regs.pm07_cntrl);
+ interrupt_mask |=
+ CBE_PM_CTR_OVERFLOW_INTR(0);
+ } else {
+ /* Disable counter */
+ cbe_write_pm07_control(cpu, 0, 0);
+ }
+
+ cbe_get_and_clear_pm_interrupts(cpu);
+ cbe_enable_pm_interrupts(cpu, hdw_thread, interrupt_mask);
+ cbe_enable_pm(cpu);
+
+ /* clear the trace buffer */
+ cbe_write_pm(cpu, trace_address, 0);
+ }
+
+ /* Start the timer to time slice collecting the event profile
+ * on each of the SPUs. Note, can collect profile on one SPU
+ * per node at a time.
+ */
+ start_spu_event_swap();
+ start_spu_profiling_events();
+ oprofile_running = 1;
+ smp_wmb();
+
+ return rtn;
+}
+
static int cell_global_start_ppu(struct op_counter_config *ctr)
{
u32 cpu, i;
@@ -994,8 +1421,7 @@
if (ctr_enabled & (1 << i)) {
cbe_write_ctr(cpu, i, reset_value[i]);
enable_ctr(cpu, i, pm_regs.pm07_cntrl);
- interrupt_mask |=
- CBE_PM_CTR_OVERFLOW_INTR(i);
+ interrupt_mask |= CBE_PM_CTR_OVERFLOW_INTR(i);
} else {
/* Disable counter */
cbe_write_pm07_control(cpu, i, 0);
@@ -1024,99 +1450,162 @@
static int cell_global_start(struct op_counter_config *ctr)
{
- if (spu_cycle_reset)
- return cell_global_start_spu(ctr);
+ if (profiling_mode == SPU_PROFILING_CYCLES)
+ return cell_global_start_spu_cycles(ctr);
+ else if (profiling_mode == SPU_PROFILING_EVENTS)
+ return cell_global_start_spu_events(ctr);
else
return cell_global_start_ppu(ctr);
}
-/*
- * Note the generic OProfile stop calls do not support returning
- * an error on stop. Hence, will not return an error if the FW
- * calls fail on stop. Failure to reset the debug bus is not an issue.
- * Failure to disable the SPU profiling is not an issue. The FW calls
- * to enable the performance counters and debug bus will work even if
- * the hardware was not cleanly reset.
+
+/* The SPU interrupt handler
+ *
+ * SPU event profiling works as follows:
+ * The pm_signal[0] holds the one SPU event to be measured. It is routed on
+ * the debug bus using word 0 or 1. The value of pm_signal[1] and
+ * pm_signal[2] contain the necessary events to route the SPU program
+ * counter for the selected SPU onto the debug bus using words 2 and 3.
+ * The pm_interval register is setup to write the SPU PC value into the
+ * trace buffer at the maximum rate possible. The trace buffer is configured
+ * to store the PCs, wrapping when it is full. The performance counter is
+ * intialized to the max hardware count minus the number of events, N, between
+ * samples. Once the N events have occured, a HW counter overflow occurs
+ * causing the generation of a HW counter interrupt which also stops the
+ * writing of the SPU PC values to the trace buffer. Hence the last PC
+ * written to the trace buffer is the SPU PC that we want. Unfortunately,
+ * we have to read from the beginning of the trace buffer to get to the
+ * last value written. We just hope the PPU has nothing better to do then
+ * service this interrupt. The PC for the specific SPU being profiled is
+ * extracted from the trace buffer processed and stored. The trace buffer
+ * is cleared, interrupts are cleared, the counter is reset to max - N.
+ * A kernel timer is used to periodically call the routine spu_evnt_swap()
+ * to switch to the next physical SPU in the node to profile in round robbin
+ * order. This way data is collected for all SPUs on the node. It does mean
+ * that we need to use a relatively small value of N to ensure enough samples
+ * on each SPU are collected each SPU is being profiled 1/8 of the time.
+ * It may also be necessary to use a longer sample collection period.
*/
-static void cell_global_stop_spu(void)
+static void cell_handle_interrupt_spu(struct pt_regs *regs,
+ struct op_counter_config *ctr)
{
- int subfunc, rtn_value;
- unsigned int lfsr_value;
- int cpu;
+ u32 cpu, cpu_tmp;
+ u64 trace_entry;
+ u32 interrupt_mask;
+ u64 trace_buffer[2];
+ u64 last_trace_buffer;
+ u32 sample;
+ u32 trace_addr;
+ unsigned long sample_array_lock_flags;
+ int spu_num;
+ unsigned long flags;
- oprofile_running = 0;
+ /* Make sure spu event interrupt handler and spu event swap
+ * don't access the counters simultaneously.
+ */
+ cpu = smp_processor_id();
+ spin_lock_irqsave(&cntr_lock, flags);
-#ifdef CONFIG_CPU_FREQ
- cpufreq_unregister_notifier(&cpu_freq_notifier_block,
- CPUFREQ_TRANSITION_NOTIFIER);
-#endif
+ cpu_tmp = cpu;
+ cbe_disable_pm(cpu);
- for_each_online_cpu(cpu) {
- if (cbe_get_hw_thread_id(cpu))
- continue;
+ interrupt_mask = cbe_get_and_clear_pm_interrupts(cpu);
- subfunc = 3; /*
- * 2 - activate SPU tracing,
- * 3 - deactivate
- */
- lfsr_value = 0x8f100000;
+ sample = 0xABCDEF;
+ trace_entry = 0xfedcba;
+ last_trace_buffer = 0xdeadbeaf;
- rtn_value = rtas_call(spu_rtas_token, 3, 1, NULL,
- subfunc, cbe_cpu_to_node(cpu),
- lfsr_value);
+ if ((oprofile_running == 1) && (interrupt_mask != 0)) {
+ /* disable writes to trace buff */
+ cbe_write_pm(cpu, pm_interval, 0);
- if (unlikely(rtn_value != 0)) {
- printk(KERN_ERR
- "%s: rtas call ibm,cbe-spu-perftools failed, return = %d\n",
- __func__, rtn_value);
+ /* only have one perf cntr being used, cntr 0 */
+ if ((interrupt_mask & CBE_PM_CTR_OVERFLOW_INTR(0))
+ && ctr[0].enabled)
+ /* The SPU PC values will be read
+ * from the trace buffer, reset counter
+ */
+
+ cbe_write_ctr(cpu, 0, reset_value[0]);
+
+ trace_addr = cbe_read_pm(cpu, trace_address);
+
+ while (!(trace_addr & CBE_PM_TRACE_BUF_EMPTY)) {
+ /* There is data in the trace buffer to process
+ * Read the buffer until you get to the last
+ * entry. This is the value we want.
+ */
+
+ cbe_read_trace_buffer(cpu, trace_buffer);
+ trace_addr = cbe_read_pm(cpu, trace_address);
}
- /* Deactivate the signals */
- pm_rtas_reset_signals(cbe_cpu_to_node(cpu));
+ /* SPU Address 16 bit count format for 128 bit
+ * HW trace buffer is used for the SPU PC storage
+ * HDR bits 0:15
+ * SPU Addr 0 bits 16:31
+ * SPU Addr 1 bits 32:47
+ * unused bits 48:127
+ *
+ * HDR: bit4 = 1 SPU Address 0 valid
+ * HDR: bit5 = 1 SPU Address 1 valid
+ * - unfortunately, the valid bits don't seem to work
+ *
+ * Note trace_buffer[0] holds bits 0:63 of the HW
+ * trace buffer, trace_buffer[1] holds bits 64:127
+ */
+
+ trace_entry = trace_buffer[0]
+ & 0x00000000FFFF0000;
+
+ /* only top 16 of the 18 bit SPU PC address
+ * is stored in trace buffer, hence shift right
+ * by 16 -2 bits */
+ sample = trace_entry >> 14;
+ last_trace_buffer = trace_buffer[0];
+
+ spu_num = spu_evnt_phys_spu_indx
+ + (cbe_cpu_to_node(cpu) * NUM_SPUS_PER_NODE);
+
+ /* make sure only one process at a time is calling
+ * spu_sync_buffer()
+ */
+ spin_lock_irqsave(&oprof_spu_smpl_arry_lck,
+ sample_array_lock_flags);
+ spu_sync_buffer(spu_num, &sample, 1);
+ spin_unlock_irqrestore(&oprof_spu_smpl_arry_lck,
+ sample_array_lock_flags);
+
+ smp_wmb(); /* insure spu event buffer updates are written
+ * don't want events intermingled... */
+
+ /* The counters were frozen by the interrupt.
+ * Reenable the interrupt and restart the counters.
+ */
+ cbe_write_pm(cpu, pm_interval, NUM_INTERVAL_CYC);
+ cbe_enable_pm_interrupts(cpu, hdw_thread,
+ virt_cntr_inter_mask);
+
+ /* clear the trace buffer, re-enable writes to trace buff */
+ cbe_write_pm(cpu, trace_address, 0);
+ cbe_write_pm(cpu, pm_interval, NUM_INTERVAL_CYC);
+
+ /* The writes to the various performance counters only writes
+ * to a latch. The new values (interrupt setting bits, reset
+ * counter value etc.) are not copied to the actual registers
+ * until the performance monitor is enabled. In order to get
+ * this to work as desired, the permormance monitor needs to
+ * be disabled while writing to the latches. This is a
+ * HW design issue.
+ */
+ write_pm_cntrl(cpu);
+ cbe_enable_pm(cpu);
}
-
- stop_spu_profiling();
+ spin_unlock_irqrestore(&cntr_lock, flags);
}
-static void cell_global_stop_ppu(void)
-{
- int cpu;
-
- /*
- * This routine will be called once for the system.
- * There is one performance monitor per node, so we
- * only need to perform this function once per node.
- */
- del_timer_sync(&timer_virt_cntr);
- oprofile_running = 0;
- smp_wmb();
-
- for_each_online_cpu(cpu) {
- if (cbe_get_hw_thread_id(cpu))
- continue;
-
- cbe_sync_irq(cbe_cpu_to_node(cpu));
- /* Stop the counters */
- cbe_disable_pm(cpu);
-
- /* Deactivate the signals */
- pm_rtas_reset_signals(cbe_cpu_to_node(cpu));
-
- /* Deactivate interrupts */
- cbe_disable_pm_interrupts(cpu);
- }
-}
-
-static void cell_global_stop(void)
-{
- if (spu_cycle_reset)
- cell_global_stop_spu();
- else
- cell_global_stop_ppu();
-}
-
-static void cell_handle_interrupt(struct pt_regs *regs,
- struct op_counter_config *ctr)
+static void cell_handle_interrupt_ppu(struct pt_regs *regs,
+ struct op_counter_config *ctr)
{
u32 cpu;
u64 pc;
@@ -1132,7 +1621,7 @@
* routine are not running at the same time. See the
* cell_virtual_cntr() routine for additional comments.
*/
- spin_lock_irqsave(&virt_cntr_lock, flags);
+ spin_lock_irqsave(&cntr_lock, flags);
/*
* Need to disable and reenable the performance counters
@@ -1185,7 +1674,16 @@
*/
cbe_enable_pm(cpu);
}
- spin_unlock_irqrestore(&virt_cntr_lock, flags);
+ spin_unlock_irqrestore(&cntr_lock, flags);
+}
+
+static void cell_handle_interrupt(struct pt_regs *regs,
+ struct op_counter_config *ctr)
+{
+ if (profiling_mode == PPU_PROFILING)
+ cell_handle_interrupt_ppu(regs, ctr);
+ else
+ cell_handle_interrupt_spu(regs, ctr);
}
/*
@@ -1195,7 +1693,8 @@
*/
static int cell_sync_start(void)
{
- if (spu_cycle_reset)
+ if ((profiling_mode == SPU_PROFILING_CYCLES) ||
+ (profiling_mode == SPU_PROFILING_EVENTS))
return spu_sync_start();
else
return DO_GENERIC_SYNC;
@@ -1203,7 +1702,8 @@
static int cell_sync_stop(void)
{
- if (spu_cycle_reset)
+ if ((profiling_mode == SPU_PROFILING_CYCLES) ||
+ (profiling_mode == SPU_PROFILING_EVENTS))
return spu_sync_stop();
else
return 1;
diff --git a/arch/powerpc/platforms/52xx/mpc52xx_common.c b/arch/powerpc/platforms/52xx/mpc52xx_common.c
index ae7c34f..98367a0 100644
--- a/arch/powerpc/platforms/52xx/mpc52xx_common.c
+++ b/arch/powerpc/platforms/52xx/mpc52xx_common.c
@@ -42,7 +42,7 @@
* from interrupt context while node mapping (which calls ioremap())
* cannot be used at such point.
*/
-static spinlock_t mpc52xx_lock = SPIN_LOCK_UNLOCKED;
+static DEFINE_SPINLOCK(mpc52xx_lock);
static struct mpc52xx_gpt __iomem *mpc52xx_wdt;
static struct mpc52xx_cdm __iomem *mpc52xx_cdm;
diff --git a/arch/powerpc/platforms/83xx/mpc831x_rdb.c b/arch/powerpc/platforms/83xx/mpc831x_rdb.c
index a428f8d..5177bdd 100644
--- a/arch/powerpc/platforms/83xx/mpc831x_rdb.c
+++ b/arch/powerpc/platforms/83xx/mpc831x_rdb.c
@@ -42,7 +42,7 @@
mpc831x_usb_cfg();
}
-void __init mpc831x_rdb_init_IRQ(void)
+static void __init mpc831x_rdb_init_IRQ(void)
{
struct device_node *np;
diff --git a/arch/powerpc/platforms/83xx/mpc832x_mds.c b/arch/powerpc/platforms/83xx/mpc832x_mds.c
index ec43477..ec0b401b 100644
--- a/arch/powerpc/platforms/83xx/mpc832x_mds.c
+++ b/arch/powerpc/platforms/83xx/mpc832x_mds.c
@@ -49,8 +49,6 @@
#define DBG(fmt...)
#endif
-static u8 *bcsr_regs = NULL;
-
/* ************************************************************************
*
* Setup the architecture
@@ -59,13 +57,14 @@
static void __init mpc832x_sys_setup_arch(void)
{
struct device_node *np;
+ u8 __iomem *bcsr_regs = NULL;
if (ppc_md.progress)
ppc_md.progress("mpc832x_sys_setup_arch()", 0);
/* Map BCSR area */
np = of_find_node_by_name(NULL, "bcsr");
- if (np != 0) {
+ if (np) {
struct resource res;
of_address_to_resource(np, 0, &res);
@@ -93,9 +92,9 @@
!= NULL){
/* Reset the Ethernet PHYs */
#define BCSR8_FETH_RST 0x50
- bcsr_regs[8] &= ~BCSR8_FETH_RST;
+ clrbits8(&bcsr_regs[8], BCSR8_FETH_RST);
udelay(1000);
- bcsr_regs[8] |= BCSR8_FETH_RST;
+ setbits8(&bcsr_regs[8], BCSR8_FETH_RST);
iounmap(bcsr_regs);
of_node_put(np);
}
diff --git a/arch/powerpc/platforms/83xx/mpc832x_rdb.c b/arch/powerpc/platforms/83xx/mpc832x_rdb.c
index 0300268..2a1295f 100644
--- a/arch/powerpc/platforms/83xx/mpc832x_rdb.c
+++ b/arch/powerpc/platforms/83xx/mpc832x_rdb.c
@@ -38,6 +38,7 @@
#define DBG(fmt...)
#endif
+#ifdef CONFIG_QUICC_ENGINE
static void mpc83xx_spi_activate_cs(u8 cs, u8 polarity)
{
pr_debug("%s %d %d\n", __func__, cs, polarity);
@@ -77,8 +78,8 @@
mpc83xx_spi_activate_cs,
mpc83xx_spi_deactivate_cs);
}
-
machine_device_initcall(mpc832x_rdb, mpc832x_spi_init);
+#endif /* CONFIG_QUICC_ENGINE */
/* ************************************************************************
*
@@ -130,7 +131,7 @@
}
machine_device_initcall(mpc832x_rdb, mpc832x_declare_of_platform_devices);
-void __init mpc832x_rdb_init_IRQ(void)
+static void __init mpc832x_rdb_init_IRQ(void)
{
struct device_node *np;
diff --git a/arch/powerpc/platforms/83xx/mpc836x_mds.c b/arch/powerpc/platforms/83xx/mpc836x_mds.c
index 9d46e5b..09e9d6f 100644
--- a/arch/powerpc/platforms/83xx/mpc836x_mds.c
+++ b/arch/powerpc/platforms/83xx/mpc836x_mds.c
@@ -18,6 +18,7 @@
#include <linux/stddef.h>
#include <linux/kernel.h>
+#include <linux/compiler.h>
#include <linux/init.h>
#include <linux/errno.h>
#include <linux/reboot.h>
@@ -43,6 +44,7 @@
#include <asm/udbg.h>
#include <sysdev/fsl_soc.h>
#include <sysdev/fsl_pci.h>
+#include <sysdev/simple_gpio.h>
#include <asm/qe.h>
#include <asm/qe_ic.h>
@@ -55,8 +57,6 @@
#define DBG(fmt...)
#endif
-static u8 *bcsr_regs = NULL;
-
/* ************************************************************************
*
* Setup the architecture
@@ -65,13 +65,14 @@
static void __init mpc836x_mds_setup_arch(void)
{
struct device_node *np;
+ u8 __iomem *bcsr_regs = NULL;
if (ppc_md.progress)
ppc_md.progress("mpc836x_mds_setup_arch()", 0);
/* Map BCSR area */
np = of_find_node_by_name(NULL, "bcsr");
- if (np != 0) {
+ if (np) {
struct resource res;
of_address_to_resource(np, 0, &res);
@@ -93,6 +94,16 @@
for (np = NULL; (np = of_find_node_by_name(np, "ucc")) != NULL;)
par_io_of_config(np);
+#ifdef CONFIG_QE_USB
+ /* Must fixup Par IO before QE GPIO chips are registered. */
+ par_io_config_pin(1, 2, 1, 0, 3, 0); /* USBOE */
+ par_io_config_pin(1, 3, 1, 0, 3, 0); /* USBTP */
+ par_io_config_pin(1, 8, 1, 0, 1, 0); /* USBTN */
+ par_io_config_pin(1, 10, 2, 0, 3, 0); /* USBRXD */
+ par_io_config_pin(1, 9, 2, 1, 3, 0); /* USBRP */
+ par_io_config_pin(1, 11, 2, 1, 3, 0); /* USBRN */
+ par_io_config_pin(2, 20, 2, 0, 1, 0); /* CLK21 */
+#endif /* CONFIG_QE_USB */
}
if ((np = of_find_compatible_node(NULL, "network", "ucc_geth"))
@@ -151,6 +162,70 @@
}
machine_device_initcall(mpc836x_mds, mpc836x_declare_of_platform_devices);
+#ifdef CONFIG_QE_USB
+static int __init mpc836x_usb_cfg(void)
+{
+ u8 __iomem *bcsr;
+ struct device_node *np;
+ const char *mode;
+ int ret = 0;
+
+ np = of_find_compatible_node(NULL, NULL, "fsl,mpc8360mds-bcsr");
+ if (!np)
+ return -ENODEV;
+
+ bcsr = of_iomap(np, 0);
+ of_node_put(np);
+ if (!bcsr)
+ return -ENOMEM;
+
+ np = of_find_compatible_node(NULL, NULL, "fsl,mpc8323-qe-usb");
+ if (!np) {
+ ret = -ENODEV;
+ goto err;
+ }
+
+#define BCSR8_TSEC1M_MASK (0x3 << 6)
+#define BCSR8_TSEC1M_RGMII (0x0 << 6)
+#define BCSR8_TSEC2M_MASK (0x3 << 4)
+#define BCSR8_TSEC2M_RGMII (0x0 << 4)
+ /*
+ * Default is GMII (2), but we should set it to RGMII (0) if we use
+ * USB (Eth PHY is in RGMII mode anyway).
+ */
+ clrsetbits_8(&bcsr[8], BCSR8_TSEC1M_MASK | BCSR8_TSEC2M_MASK,
+ BCSR8_TSEC1M_RGMII | BCSR8_TSEC2M_RGMII);
+
+#define BCSR13_USBMASK 0x0f
+#define BCSR13_nUSBEN 0x08 /* 1 - Disable, 0 - Enable */
+#define BCSR13_USBSPEED 0x04 /* 1 - Full, 0 - Low */
+#define BCSR13_USBMODE 0x02 /* 1 - Host, 0 - Function */
+#define BCSR13_nUSBVCC 0x01 /* 1 - gets VBUS, 0 - supplies VBUS */
+
+ clrsetbits_8(&bcsr[13], BCSR13_USBMASK, BCSR13_USBSPEED);
+
+ mode = of_get_property(np, "mode", NULL);
+ if (mode && !strcmp(mode, "peripheral")) {
+ setbits8(&bcsr[13], BCSR13_nUSBVCC);
+ qe_usb_clock_set(QE_CLK21, 48000000);
+ } else {
+ setbits8(&bcsr[13], BCSR13_USBMODE);
+ /*
+ * The BCSR GPIOs are used to control power and
+ * speed of the USB transceiver. This is needed for
+ * the USB Host only.
+ */
+ simple_gpiochip_init("fsl,mpc8360mds-bcsr-gpio");
+ }
+
+ of_node_put(np);
+err:
+ iounmap(bcsr);
+ return ret;
+}
+machine_arch_initcall(mpc836x_mds, mpc836x_usb_cfg);
+#endif /* CONFIG_QE_USB */
+
static void __init mpc836x_mds_init_IRQ(void)
{
struct device_node *np;
diff --git a/arch/powerpc/platforms/83xx/mpc836x_rdk.c b/arch/powerpc/platforms/83xx/mpc836x_rdk.c
index a5273bb..b0090aa 100644
--- a/arch/powerpc/platforms/83xx/mpc836x_rdk.c
+++ b/arch/powerpc/platforms/83xx/mpc836x_rdk.c
@@ -51,8 +51,9 @@
for_each_compatible_node(np, "pci", "fsl,mpc8349-pci")
mpc83xx_add_bridge(np);
#endif
-
+#ifdef CONFIG_QUICC_ENGINE
qe_reset();
+#endif
}
static void __init mpc836x_rdk_init_IRQ(void)
@@ -71,13 +72,14 @@
*/
ipic_set_default_priority();
of_node_put(np);
-
+#ifdef CONFIG_QUICC_ENGINE
np = of_find_compatible_node(NULL, NULL, "fsl,qe-ic");
if (!np)
return;
qe_ic_init(np, 0, qe_ic_cascade_low_ipic, qe_ic_cascade_high_ipic);
of_node_put(np);
+#endif
}
/*
diff --git a/arch/powerpc/platforms/83xx/mpc837x_mds.c b/arch/powerpc/platforms/83xx/mpc837x_mds.c
index 8bb13c8..530ef99 100644
--- a/arch/powerpc/platforms/83xx/mpc837x_mds.c
+++ b/arch/powerpc/platforms/83xx/mpc837x_mds.c
@@ -26,7 +26,6 @@
#define BCSR12_USB_SER_MASK 0x8a
#define BCSR12_USB_SER_PIN 0x80
#define BCSR12_USB_SER_DEVICE 0x02
-extern int mpc837x_usb_cfg(void);
static int mpc837xmds_usb_cfg(void)
{
diff --git a/arch/powerpc/platforms/83xx/mpc837x_rdb.c b/arch/powerpc/platforms/83xx/mpc837x_rdb.c
index da030af..1d09654 100644
--- a/arch/powerpc/platforms/83xx/mpc837x_rdb.c
+++ b/arch/powerpc/platforms/83xx/mpc837x_rdb.c
@@ -21,8 +21,6 @@
#include "mpc83xx.h"
-extern int mpc837x_usb_cfg(void);
-
/* ************************************************************************
*
* Setup the architecture
diff --git a/arch/powerpc/platforms/83xx/mpc83xx.h b/arch/powerpc/platforms/83xx/mpc83xx.h
index 2a7cbab..83cfe51 100644
--- a/arch/powerpc/platforms/83xx/mpc83xx.h
+++ b/arch/powerpc/platforms/83xx/mpc83xx.h
@@ -61,6 +61,7 @@
extern void mpc83xx_restart(char *cmd);
extern long mpc83xx_time_init(void);
+extern int mpc837x_usb_cfg(void);
extern int mpc834x_usb_cfg(void);
extern int mpc831x_usb_cfg(void);
diff --git a/arch/powerpc/platforms/85xx/mpc85xx_ds.c b/arch/powerpc/platforms/85xx/mpc85xx_ds.c
index a8301c8..7326d90 100644
--- a/arch/powerpc/platforms/85xx/mpc85xx_ds.c
+++ b/arch/powerpc/platforms/85xx/mpc85xx_ds.c
@@ -148,6 +148,9 @@
/*
* Setup the architecture
*/
+#ifdef CONFIG_SMP
+extern void __init mpc85xx_smp_init(void);
+#endif
static void __init mpc85xx_ds_setup_arch(void)
{
#ifdef CONFIG_PCI
@@ -173,6 +176,10 @@
ppc_md.pci_exclude_device = mpc85xx_exclude_device;
#endif
+#ifdef CONFIG_SMP
+ mpc85xx_smp_init();
+#endif
+
printk("MPC85xx DS board from Freescale Semiconductor\n");
}
diff --git a/arch/powerpc/platforms/85xx/smp.c b/arch/powerpc/platforms/85xx/smp.c
index d652c71..79a0df1 100644
--- a/arch/powerpc/platforms/85xx/smp.c
+++ b/arch/powerpc/platforms/85xx/smp.c
@@ -58,6 +58,7 @@
if (cpu_rel_addr == NULL) {
printk(KERN_ERR "No cpu-release-addr for cpu %d\n", nr);
+ local_irq_restore(flags);
return;
}
diff --git a/arch/powerpc/platforms/Kconfig b/arch/powerpc/platforms/Kconfig
index 47e956c..47fe2be 100644
--- a/arch/powerpc/platforms/Kconfig
+++ b/arch/powerpc/platforms/Kconfig
@@ -312,4 +312,15 @@
Say Y here if you're going to use hardware that connects to the
MPC831x/834x/837x/8572/8610 GPIOs.
+config SIMPLE_GPIO
+ bool "Support for simple, memory-mapped GPIO controllers"
+ depends on PPC
+ select GENERIC_GPIO
+ select ARCH_REQUIRE_GPIOLIB
+ help
+ Say Y here to support simple, memory-mapped GPIO controllers.
+ These are usually BCSRs used to control board's switches, LEDs,
+ chip-selects, Ethernet/USB PHY's power and various other small
+ on-board peripherals.
+
endmenu
diff --git a/arch/powerpc/platforms/Kconfig.cputype b/arch/powerpc/platforms/Kconfig.cputype
index 3d0c776..e868b5c 100644
--- a/arch/powerpc/platforms/Kconfig.cputype
+++ b/arch/powerpc/platforms/Kconfig.cputype
@@ -231,7 +231,7 @@
If in doubt, say Y here.
config SMP
- depends on PPC_STD_MMU
+ depends on PPC_STD_MMU || FSL_BOOKE
bool "Symmetric multi-processing support"
---help---
This enables support for systems with more than one CPU. If you have
diff --git a/arch/powerpc/platforms/cell/beat_htab.c b/arch/powerpc/platforms/cell/beat_htab.c
index 2e67bd8..35b1ec4 100644
--- a/arch/powerpc/platforms/cell/beat_htab.c
+++ b/arch/powerpc/platforms/cell/beat_htab.c
@@ -44,8 +44,8 @@
static inline unsigned int beat_read_mask(unsigned hpte_group)
{
- unsigned long hpte_v[5];
unsigned long rmask = 0;
+ u64 hpte_v[5];
beat_read_htab_entries(0, hpte_group + 0, hpte_v);
if (!(hpte_v[0] & HPTE_V_BOLTED))
@@ -93,8 +93,7 @@
int psize, int ssize)
{
unsigned long lpar_rc;
- unsigned long slot;
- unsigned long hpte_v, hpte_r;
+ u64 hpte_v, hpte_r, slot;
/* same as iseries */
if (vflags & HPTE_V_SECONDARY)
@@ -153,8 +152,9 @@
static unsigned long beat_lpar_hpte_getword0(unsigned long slot)
{
- unsigned long dword0, dword[5];
+ unsigned long dword0;
unsigned long lpar_rc;
+ u64 dword[5];
lpar_rc = beat_read_htab_entries(0, slot & ~3UL, dword);
@@ -170,7 +170,7 @@
unsigned long size_bytes = 1UL << ppc64_pft_size;
unsigned long hpte_count = size_bytes >> 4;
int i;
- unsigned long dummy0, dummy1;
+ u64 dummy0, dummy1;
/* TODO: Use bulk call */
for (i = 0; i < hpte_count; i++)
@@ -189,7 +189,8 @@
int psize, int ssize, int local)
{
unsigned long lpar_rc;
- unsigned long dummy0, dummy1, want_v;
+ u64 dummy0, dummy1;
+ unsigned long want_v;
want_v = hpte_encode_v(va, psize, MMU_SEGSIZE_256M);
@@ -255,7 +256,8 @@
unsigned long ea,
int psize, int ssize)
{
- unsigned long lpar_rc, slot, vsid, va, dummy0, dummy1;
+ unsigned long lpar_rc, slot, vsid, va;
+ u64 dummy0, dummy1;
vsid = get_kernel_vsid(ea, MMU_SEGSIZE_256M);
va = (vsid << 28) | (ea & 0x0fffffff);
@@ -276,7 +278,7 @@
{
unsigned long want_v;
unsigned long lpar_rc;
- unsigned long dummy1, dummy2;
+ u64 dummy1, dummy2;
unsigned long flags;
DBG_LOW(" inval : slot=%lx, va=%016lx, psize: %d, local: %d\n",
@@ -315,8 +317,7 @@
int psize, int ssize)
{
unsigned long lpar_rc;
- unsigned long slot;
- unsigned long hpte_v, hpte_r;
+ u64 hpte_v, hpte_r, slot;
/* same as iseries */
if (vflags & HPTE_V_SECONDARY)
diff --git a/arch/powerpc/platforms/cell/beat_udbg.c b/arch/powerpc/platforms/cell/beat_udbg.c
index 6b418f6..350735b 100644
--- a/arch/powerpc/platforms/cell/beat_udbg.c
+++ b/arch/powerpc/platforms/cell/beat_udbg.c
@@ -40,8 +40,8 @@
}
/* Buffered chars getc */
-static long inbuflen;
-static long inbuf[2]; /* must be 2 longs */
+static u64 inbuflen;
+static u64 inbuf[2]; /* must be 2 u64s */
static int udbg_getc_poll_beat(void)
{
diff --git a/arch/powerpc/platforms/cell/cbe_cpufreq_pervasive.c b/arch/powerpc/platforms/cell/cbe_cpufreq_pervasive.c
index 70fa7ae..20472e4 100644
--- a/arch/powerpc/platforms/cell/cbe_cpufreq_pervasive.c
+++ b/arch/powerpc/platforms/cell/cbe_cpufreq_pervasive.c
@@ -54,7 +54,7 @@
{
struct cbe_pmd_regs __iomem *pmd_regs;
struct cbe_mic_tm_regs __iomem *mic_tm_regs;
- u64 flags;
+ unsigned long flags;
u64 value;
#ifdef DEBUG
long time;
diff --git a/arch/powerpc/platforms/cell/interrupt.c b/arch/powerpc/platforms/cell/interrupt.c
index 2d5bb22..28c04da 100644
--- a/arch/powerpc/platforms/cell/interrupt.c
+++ b/arch/powerpc/platforms/cell/interrupt.c
@@ -148,7 +148,7 @@
iic = &__get_cpu_var(iic);
*(unsigned long *) &pending =
- in_be64((unsigned long __iomem *) &iic->regs->pending_destr);
+ in_be64((u64 __iomem *) &iic->regs->pending_destr);
if (!(pending.flags & CBE_IIC_IRQ_VALID))
return NO_IRQ;
virq = irq_linear_revmap(iic_host, iic_pending_to_hwnum(pending));
diff --git a/arch/powerpc/platforms/cell/io-workarounds.c b/arch/powerpc/platforms/cell/io-workarounds.c
index b5f84e8..059cad6 100644
--- a/arch/powerpc/platforms/cell/io-workarounds.c
+++ b/arch/powerpc/platforms/cell/io-workarounds.c
@@ -130,14 +130,14 @@
};
-static void __iomem *iowa_ioremap(unsigned long addr, unsigned long size,
+static void __iomem *iowa_ioremap(phys_addr_t addr, unsigned long size,
unsigned long flags)
{
struct iowa_bus *bus;
void __iomem *res = __ioremap(addr, size, flags);
int busno;
- bus = iowa_pci_find(0, addr);
+ bus = iowa_pci_find(0, (unsigned long)addr);
if (bus != NULL) {
busno = bus - iowa_busses;
PCI_SET_ADDR_TOKEN(res, busno + 1);
diff --git a/arch/powerpc/platforms/cell/iommu.c b/arch/powerpc/platforms/cell/iommu.c
index 86db4dd..88d94b5 100644
--- a/arch/powerpc/platforms/cell/iommu.c
+++ b/arch/powerpc/platforms/cell/iommu.c
@@ -150,8 +150,8 @@
static void invalidate_tce_cache(struct cbe_iommu *iommu, unsigned long *pte,
long n_ptes)
{
- unsigned long __iomem *reg;
- unsigned long val;
+ u64 __iomem *reg;
+ u64 val;
long n;
reg = iommu->xlate_regs + IOC_IOPT_CacheInvd;
diff --git a/arch/powerpc/platforms/cell/spufs/spufs.h b/arch/powerpc/platforms/cell/spufs/spufs.h
index 15c62d3..3bf908e 100644
--- a/arch/powerpc/platforms/cell/spufs/spufs.h
+++ b/arch/powerpc/platforms/cell/spufs/spufs.h
@@ -314,7 +314,7 @@
* we need to call spu_release(ctx) before sleeping, and
* then spu_acquire(ctx) when awoken.
*
- * Returns with state_mutex re-acquired when successfull or
+ * Returns with state_mutex re-acquired when successful or
* with -ERESTARTSYS and the state_mutex dropped when interrupted.
*/
diff --git a/arch/powerpc/platforms/iseries/Kconfig b/arch/powerpc/platforms/iseries/Kconfig
index ed3753d..7ddd0a2 100644
--- a/arch/powerpc/platforms/iseries/Kconfig
+++ b/arch/powerpc/platforms/iseries/Kconfig
@@ -10,18 +10,21 @@
config VIODASD
tristate "iSeries Virtual I/O disk support"
depends on BLOCK
+ select VIOPATH
help
If you are running on an iSeries system and you want to use
virtual disks created and managed by OS/400, say Y.
config VIOCD
tristate "iSeries Virtual I/O CD support"
+ select VIOPATH
help
If you are running Linux on an IBM iSeries system and you want to
read a CD drive owned by OS/400, say Y here.
config VIOTAPE
tristate "iSeries Virtual Tape Support"
+ select VIOPATH
help
If you are running Linux on an iSeries system and you want Linux
to read and/or write a tape drive owned by OS/400, say Y here.
@@ -30,5 +33,3 @@
config VIOPATH
bool
- depends on VIODASD || VIOCD || VIOTAPE || ISERIES_VETH
- default y
diff --git a/arch/powerpc/platforms/iseries/setup.c b/arch/powerpc/platforms/iseries/setup.c
index 70b688c..24519b9 100644
--- a/arch/powerpc/platforms/iseries/setup.c
+++ b/arch/powerpc/platforms/iseries/setup.c
@@ -23,6 +23,7 @@
#include <linux/string.h>
#include <linux/seq_file.h>
#include <linux/kdev_t.h>
+#include <linux/kexec.h>
#include <linux/major.h>
#include <linux/root_dev.h>
#include <linux/kernel.h>
@@ -638,6 +639,13 @@
return 1;
}
+#ifdef CONFIG_KEXEC
+static int iseries_kexec_prepare(struct kimage *image)
+{
+ return -ENOSYS;
+}
+#endif
+
define_machine(iseries) {
.name = "iSeries",
.setup_arch = iSeries_setup_arch,
@@ -658,6 +666,9 @@
.probe = iseries_probe,
.ioremap = iseries_ioremap,
.iounmap = iseries_iounmap,
+#ifdef CONFIG_KEXEC
+ .machine_kexec_prepare = iseries_kexec_prepare,
+#endif
/* XXX Implement enable_pmcs for iSeries */
};
diff --git a/arch/powerpc/platforms/pasemi/cpufreq.c b/arch/powerpc/platforms/pasemi/cpufreq.c
index 58556b0..86db47c 100644
--- a/arch/powerpc/platforms/pasemi/cpufreq.c
+++ b/arch/powerpc/platforms/pasemi/cpufreq.c
@@ -112,7 +112,7 @@
static void set_astate(int cpu, unsigned int astate)
{
- u64 flags;
+ unsigned long flags;
/* Return if called before init has run */
if (unlikely(!sdcasr_mapbase))
diff --git a/arch/powerpc/platforms/pasemi/dma_lib.c b/arch/powerpc/platforms/pasemi/dma_lib.c
index 217af32..a6152d9 100644
--- a/arch/powerpc/platforms/pasemi/dma_lib.c
+++ b/arch/powerpc/platforms/pasemi/dma_lib.c
@@ -509,7 +509,7 @@
*/
int pasemi_dma_init(void)
{
- static spinlock_t init_lock = SPIN_LOCK_UNLOCKED;
+ static DEFINE_SPINLOCK(init_lock);
struct pci_dev *iob_pdev;
struct pci_dev *pdev;
struct resource res;
diff --git a/arch/powerpc/platforms/powermac/pci.c b/arch/powerpc/platforms/powermac/pci.c
index 54b7b76..04cdd32 100644
--- a/arch/powerpc/platforms/powermac/pci.c
+++ b/arch/powerpc/platforms/powermac/pci.c
@@ -661,6 +661,7 @@
pci_find_hose_for_OF_device(np);
if (!hose) {
printk(KERN_ERR "Can't find PCI hose for OHare2 !\n");
+ of_node_put(np);
return;
}
early_read_config_word(hose, bus, devfn, PCI_COMMAND, &cmd);
@@ -669,6 +670,7 @@
early_write_config_word(hose, bus, devfn, PCI_COMMAND, cmd);
}
has_second_ohare = 1;
+ of_node_put(np);
}
/*
diff --git a/arch/powerpc/platforms/powermac/time.c b/arch/powerpc/platforms/powermac/time.c
index 59eb840..1810e42 100644
--- a/arch/powerpc/platforms/powermac/time.c
+++ b/arch/powerpc/platforms/powermac/time.c
@@ -265,12 +265,15 @@
struct resource rsrc;
vias = of_find_node_by_name(NULL, "via-cuda");
- if (vias == 0)
+ if (vias == NULL)
vias = of_find_node_by_name(NULL, "via-pmu");
- if (vias == 0)
+ if (vias == NULL)
vias = of_find_node_by_name(NULL, "via");
- if (vias == 0 || of_address_to_resource(vias, 0, &rsrc))
+ if (vias == NULL || of_address_to_resource(vias, 0, &rsrc)) {
+ of_node_put(vias);
return 0;
+ }
+ of_node_put(vias);
via = ioremap(rsrc.start, rsrc.end - rsrc.start + 1);
if (via == NULL) {
printk(KERN_ERR "Failed to map VIA for timer calibration !\n");
@@ -297,7 +300,7 @@
ppc_tb_freq = (dstart - dend) * 100 / 6;
iounmap(via);
-
+
return 1;
}
#endif
diff --git a/arch/powerpc/platforms/ps3/device-init.c b/arch/powerpc/platforms/ps3/device-init.c
index dbc124e..ca71a12 100644
--- a/arch/powerpc/platforms/ps3/device-init.c
+++ b/arch/powerpc/platforms/ps3/device-init.c
@@ -518,6 +518,41 @@
return result;
}
+static int __init ps3_register_ramdisk_device(void)
+{
+ int result;
+ struct layout {
+ struct ps3_system_bus_device dev;
+ } *p;
+
+ pr_debug(" -> %s:%d\n", __func__, __LINE__);
+
+ p = kzalloc(sizeof(struct layout), GFP_KERNEL);
+
+ if (!p)
+ return -ENOMEM;
+
+ p->dev.match_id = PS3_MATCH_ID_GPU;
+ p->dev.match_sub_id = PS3_MATCH_SUB_ID_GPU_RAMDISK;
+ p->dev.dev_type = PS3_DEVICE_TYPE_IOC0;
+
+ result = ps3_system_bus_device_register(&p->dev);
+
+ if (result) {
+ pr_debug("%s:%d ps3_system_bus_device_register failed\n",
+ __func__, __LINE__);
+ goto fail_device_register;
+ }
+
+ pr_debug(" <- %s:%d\n", __func__, __LINE__);
+ return 0;
+
+fail_device_register:
+ kfree(p);
+ pr_debug(" <- %s:%d failed\n", __func__, __LINE__);
+ return result;
+}
+
/**
* ps3_setup_dynamic_device - Setup a dynamic device from the repository
*/
@@ -946,6 +981,8 @@
ps3_register_lpm_devices();
+ ps3_register_ramdisk_device();
+
pr_debug(" <- %s:%d\n", __func__, __LINE__);
return 0;
}
diff --git a/arch/powerpc/sysdev/Makefile b/arch/powerpc/sysdev/Makefile
index 5afce11..b33b28a 100644
--- a/arch/powerpc/sysdev/Makefile
+++ b/arch/powerpc/sysdev/Makefile
@@ -17,6 +17,7 @@
obj-$(CONFIG_FSL_LBC) += fsl_lbc.o
obj-$(CONFIG_FSL_GTM) += fsl_gtm.o
obj-$(CONFIG_MPC8xxx_GPIO) += mpc8xxx_gpio.o
+obj-$(CONFIG_SIMPLE_GPIO) += simple_gpio.o
obj-$(CONFIG_RAPIDIO) += fsl_rio.o
obj-$(CONFIG_TSI108_BRIDGE) += tsi108_pci.o tsi108_dev.o
obj-$(CONFIG_QUICC_ENGINE) += qe_lib/
diff --git a/arch/powerpc/sysdev/fsl_pci.c b/arch/powerpc/sysdev/fsl_pci.c
index d5f9ae0..f611d03 100644
--- a/arch/powerpc/sysdev/fsl_pci.c
+++ b/arch/powerpc/sysdev/fsl_pci.c
@@ -29,7 +29,8 @@
#if defined(CONFIG_PPC_85xx) || defined(CONFIG_PPC_86xx)
/* atmu setup for fsl pci/pcie controller */
-void __init setup_pci_atmu(struct pci_controller *hose, struct resource *rsrc)
+static void __init setup_pci_atmu(struct pci_controller *hose,
+ struct resource *rsrc)
{
struct ccsr_pci __iomem *pci;
int i;
@@ -86,7 +87,7 @@
out_be32(&pci->piw[2].piwar, PIWAR_2G);
}
-void __init setup_pci_cmd(struct pci_controller *hose)
+static void __init setup_pci_cmd(struct pci_controller *hose)
{
u16 cmd;
int cap_x;
@@ -130,7 +131,7 @@
return ;
}
-int __init fsl_pcie_check_link(struct pci_controller *hose)
+static int __init fsl_pcie_check_link(struct pci_controller *hose)
{
u32 val;
early_read_config_dword(hose, 0, 0, PCIE_LTSSM, &val);
diff --git a/arch/powerpc/sysdev/fsl_soc.h b/arch/powerpc/sysdev/fsl_soc.h
index 60f7f22..9c744e4 100644
--- a/arch/powerpc/sysdev/fsl_soc.h
+++ b/arch/powerpc/sysdev/fsl_soc.h
@@ -5,8 +5,13 @@
#include <asm/mmu.h>
extern phys_addr_t get_immrbase(void);
+#if defined(CONFIG_CPM2) || defined(CONFIG_QUICC_ENGINE) || defined(CONFIG_8xx)
extern u32 get_brgfreq(void);
extern u32 get_baudrate(void);
+#else
+static inline u32 get_brgfreq(void) { return -1; }
+static inline u32 get_baudrate(void) { return -1; }
+#endif
extern u32 fsl_get_sys_freq(void);
struct spi_board_info;
diff --git a/arch/powerpc/sysdev/qe_lib/Kconfig b/arch/powerpc/sysdev/qe_lib/Kconfig
index 76ffbc4..41ac3df 100644
--- a/arch/powerpc/sysdev/qe_lib/Kconfig
+++ b/arch/powerpc/sysdev/qe_lib/Kconfig
@@ -22,5 +22,6 @@
config QE_USB
bool
+ default y if USB_GADGET_FSL_QE
help
- QE USB Host Controller support
+ QE USB Controller support
diff --git a/arch/powerpc/sysdev/qe_lib/gpio.c b/arch/powerpc/sysdev/qe_lib/gpio.c
index 8e5a0bc..3485288 100644
--- a/arch/powerpc/sysdev/qe_lib/gpio.c
+++ b/arch/powerpc/sysdev/qe_lib/gpio.c
@@ -14,6 +14,7 @@
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/spinlock.h>
+#include <linux/err.h>
#include <linux/io.h>
#include <linux/of.h>
#include <linux/of_gpio.h>
@@ -24,8 +25,14 @@
struct of_mm_gpio_chip mm_gc;
spinlock_t lock;
+ unsigned long pin_flags[QE_PIO_PINS];
+#define QE_PIN_REQUESTED 0
+
/* shadowed data register to clear/set bits safely */
u32 cpdata;
+
+ /* saved_regs used to restore dedicated functions */
+ struct qe_pio_regs saved_regs;
};
static inline struct qe_gpio_chip *
@@ -40,6 +47,12 @@
struct qe_pio_regs __iomem *regs = mm_gc->regs;
qe_gc->cpdata = in_be32(®s->cpdata);
+ qe_gc->saved_regs.cpdata = qe_gc->cpdata;
+ qe_gc->saved_regs.cpdir1 = in_be32(®s->cpdir1);
+ qe_gc->saved_regs.cpdir2 = in_be32(®s->cpdir2);
+ qe_gc->saved_regs.cppar1 = in_be32(®s->cppar1);
+ qe_gc->saved_regs.cppar2 = in_be32(®s->cppar2);
+ qe_gc->saved_regs.cpodr = in_be32(®s->cpodr);
}
static int qe_gpio_get(struct gpio_chip *gc, unsigned int gpio)
@@ -103,6 +116,188 @@
return 0;
}
+struct qe_pin {
+ /*
+ * The qe_gpio_chip name is unfortunate, we should change that to
+ * something like qe_pio_controller. Someday.
+ */
+ struct qe_gpio_chip *controller;
+ int num;
+};
+
+/**
+ * qe_pin_request - Request a QE pin
+ * @np: device node to get a pin from
+ * @index: index of a pin in the device tree
+ * Context: non-atomic
+ *
+ * This function return qe_pin so that you could use it with the rest of
+ * the QE Pin Multiplexing API.
+ */
+struct qe_pin *qe_pin_request(struct device_node *np, int index)
+{
+ struct qe_pin *qe_pin;
+ struct device_node *gc;
+ struct of_gpio_chip *of_gc = NULL;
+ struct of_mm_gpio_chip *mm_gc;
+ struct qe_gpio_chip *qe_gc;
+ int err;
+ int size;
+ const void *gpio_spec;
+ const u32 *gpio_cells;
+ unsigned long flags;
+
+ qe_pin = kzalloc(sizeof(*qe_pin), GFP_KERNEL);
+ if (!qe_pin) {
+ pr_debug("%s: can't allocate memory\n", __func__);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ err = of_parse_phandles_with_args(np, "gpios", "#gpio-cells", index,
+ &gc, &gpio_spec);
+ if (err) {
+ pr_debug("%s: can't parse gpios property\n", __func__);
+ goto err0;
+ }
+
+ if (!of_device_is_compatible(gc, "fsl,mpc8323-qe-pario-bank")) {
+ pr_debug("%s: tried to get a non-qe pin\n", __func__);
+ err = -EINVAL;
+ goto err1;
+ }
+
+ of_gc = gc->data;
+ if (!of_gc) {
+ pr_debug("%s: gpio controller %s isn't registered\n",
+ np->full_name, gc->full_name);
+ err = -ENODEV;
+ goto err1;
+ }
+
+ gpio_cells = of_get_property(gc, "#gpio-cells", &size);
+ if (!gpio_cells || size != sizeof(*gpio_cells) ||
+ *gpio_cells != of_gc->gpio_cells) {
+ pr_debug("%s: wrong #gpio-cells for %s\n",
+ np->full_name, gc->full_name);
+ err = -EINVAL;
+ goto err1;
+ }
+
+ err = of_gc->xlate(of_gc, np, gpio_spec, NULL);
+ if (err < 0)
+ goto err1;
+
+ mm_gc = to_of_mm_gpio_chip(&of_gc->gc);
+ qe_gc = to_qe_gpio_chip(mm_gc);
+
+ spin_lock_irqsave(&qe_gc->lock, flags);
+
+ if (test_and_set_bit(QE_PIN_REQUESTED, &qe_gc->pin_flags[err]) == 0) {
+ qe_pin->controller = qe_gc;
+ qe_pin->num = err;
+ err = 0;
+ } else {
+ err = -EBUSY;
+ }
+
+ spin_unlock_irqrestore(&qe_gc->lock, flags);
+
+ if (!err)
+ return qe_pin;
+err1:
+ of_node_put(gc);
+err0:
+ kfree(qe_pin);
+ pr_debug("%s failed with status %d\n", __func__, err);
+ return ERR_PTR(err);
+}
+EXPORT_SYMBOL(qe_pin_request);
+
+/**
+ * qe_pin_free - Free a pin
+ * @qe_pin: pointer to the qe_pin structure
+ * Context: any
+ *
+ * This function frees the qe_pin structure and makes a pin available
+ * for further qe_pin_request() calls.
+ */
+void qe_pin_free(struct qe_pin *qe_pin)
+{
+ struct qe_gpio_chip *qe_gc = qe_pin->controller;
+ unsigned long flags;
+ const int pin = qe_pin->num;
+
+ spin_lock_irqsave(&qe_gc->lock, flags);
+ test_and_clear_bit(QE_PIN_REQUESTED, &qe_gc->pin_flags[pin]);
+ spin_unlock_irqrestore(&qe_gc->lock, flags);
+
+ kfree(qe_pin);
+}
+EXPORT_SYMBOL(qe_pin_free);
+
+/**
+ * qe_pin_set_dedicated - Revert a pin to a dedicated peripheral function mode
+ * @qe_pin: pointer to the qe_pin structure
+ * Context: any
+ *
+ * This function resets a pin to a dedicated peripheral function that
+ * has been set up by the firmware.
+ */
+void qe_pin_set_dedicated(struct qe_pin *qe_pin)
+{
+ struct qe_gpio_chip *qe_gc = qe_pin->controller;
+ struct qe_pio_regs __iomem *regs = qe_gc->mm_gc.regs;
+ struct qe_pio_regs *sregs = &qe_gc->saved_regs;
+ int pin = qe_pin->num;
+ u32 mask1 = 1 << (QE_PIO_PINS - (pin + 1));
+ u32 mask2 = 0x3 << (QE_PIO_PINS - (pin % (QE_PIO_PINS / 2) + 1) * 2);
+ bool second_reg = pin > (QE_PIO_PINS / 2) - 1;
+ unsigned long flags;
+
+ spin_lock_irqsave(&qe_gc->lock, flags);
+
+ if (second_reg) {
+ clrsetbits_be32(®s->cpdir2, mask2, sregs->cpdir2 & mask2);
+ clrsetbits_be32(®s->cppar2, mask2, sregs->cppar2 & mask2);
+ } else {
+ clrsetbits_be32(®s->cpdir1, mask2, sregs->cpdir1 & mask2);
+ clrsetbits_be32(®s->cppar1, mask2, sregs->cppar1 & mask2);
+ }
+
+ if (sregs->cpdata & mask1)
+ qe_gc->cpdata |= mask1;
+ else
+ qe_gc->cpdata &= ~mask1;
+
+ out_be32(®s->cpdata, qe_gc->cpdata);
+ clrsetbits_be32(®s->cpodr, mask1, sregs->cpodr & mask1);
+
+ spin_unlock_irqrestore(&qe_gc->lock, flags);
+}
+EXPORT_SYMBOL(qe_pin_set_dedicated);
+
+/**
+ * qe_pin_set_gpio - Set a pin to the GPIO mode
+ * @qe_pin: pointer to the qe_pin structure
+ * Context: any
+ *
+ * This function sets a pin to the GPIO mode.
+ */
+void qe_pin_set_gpio(struct qe_pin *qe_pin)
+{
+ struct qe_gpio_chip *qe_gc = qe_pin->controller;
+ struct qe_pio_regs __iomem *regs = qe_gc->mm_gc.regs;
+ unsigned long flags;
+
+ spin_lock_irqsave(&qe_gc->lock, flags);
+
+ /* Let's make it input by default, GPIO API is able to change that. */
+ __par_io_config_pin(regs, qe_pin->num, QE_PIO_DIR_IN, 0, 0, 0);
+
+ spin_unlock_irqrestore(&qe_gc->lock, flags);
+}
+EXPORT_SYMBOL(qe_pin_set_gpio);
+
static int __init qe_add_gpiochips(void)
{
struct device_node *np;
diff --git a/arch/powerpc/sysdev/simple_gpio.c b/arch/powerpc/sysdev/simple_gpio.c
new file mode 100644
index 0000000..43c4569
--- /dev/null
+++ b/arch/powerpc/sysdev/simple_gpio.c
@@ -0,0 +1,155 @@
+/*
+ * Simple Memory-Mapped GPIOs
+ *
+ * Copyright (c) MontaVista Software, Inc. 2008.
+ *
+ * Author: Anton Vorontsov <avorontsov@ru.mvista.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/ioport.h>
+#include <linux/io.h>
+#include <linux/of.h>
+#include <linux/of_gpio.h>
+#include <linux/gpio.h>
+#include <asm/prom.h>
+#include "simple_gpio.h"
+
+struct u8_gpio_chip {
+ struct of_mm_gpio_chip mm_gc;
+ spinlock_t lock;
+
+ /* shadowed data register to clear/set bits safely */
+ u8 data;
+};
+
+static struct u8_gpio_chip *to_u8_gpio_chip(struct of_mm_gpio_chip *mm_gc)
+{
+ return container_of(mm_gc, struct u8_gpio_chip, mm_gc);
+}
+
+static u8 u8_pin2mask(unsigned int pin)
+{
+ return 1 << (8 - 1 - pin);
+}
+
+static int u8_gpio_get(struct gpio_chip *gc, unsigned int gpio)
+{
+ struct of_mm_gpio_chip *mm_gc = to_of_mm_gpio_chip(gc);
+
+ return in_8(mm_gc->regs) & u8_pin2mask(gpio);
+}
+
+static void u8_gpio_set(struct gpio_chip *gc, unsigned int gpio, int val)
+{
+ struct of_mm_gpio_chip *mm_gc = to_of_mm_gpio_chip(gc);
+ struct u8_gpio_chip *u8_gc = to_u8_gpio_chip(mm_gc);
+ unsigned long flags;
+
+ spin_lock_irqsave(&u8_gc->lock, flags);
+
+ if (val)
+ u8_gc->data |= u8_pin2mask(gpio);
+ else
+ u8_gc->data &= ~u8_pin2mask(gpio);
+
+ out_8(mm_gc->regs, u8_gc->data);
+
+ spin_unlock_irqrestore(&u8_gc->lock, flags);
+}
+
+static int u8_gpio_dir_in(struct gpio_chip *gc, unsigned int gpio)
+{
+ return 0;
+}
+
+static int u8_gpio_dir_out(struct gpio_chip *gc, unsigned int gpio, int val)
+{
+ u8_gpio_set(gc, gpio, val);
+ return 0;
+}
+
+static void u8_gpio_save_regs(struct of_mm_gpio_chip *mm_gc)
+{
+ struct u8_gpio_chip *u8_gc = to_u8_gpio_chip(mm_gc);
+
+ u8_gc->data = in_8(mm_gc->regs);
+}
+
+static int __init u8_simple_gpiochip_add(struct device_node *np)
+{
+ int ret;
+ struct u8_gpio_chip *u8_gc;
+ struct of_mm_gpio_chip *mm_gc;
+ struct of_gpio_chip *of_gc;
+ struct gpio_chip *gc;
+
+ u8_gc = kzalloc(sizeof(*u8_gc), GFP_KERNEL);
+ if (!u8_gc)
+ return -ENOMEM;
+
+ spin_lock_init(&u8_gc->lock);
+
+ mm_gc = &u8_gc->mm_gc;
+ of_gc = &mm_gc->of_gc;
+ gc = &of_gc->gc;
+
+ mm_gc->save_regs = u8_gpio_save_regs;
+ of_gc->gpio_cells = 2;
+ gc->ngpio = 8;
+ gc->direction_input = u8_gpio_dir_in;
+ gc->direction_output = u8_gpio_dir_out;
+ gc->get = u8_gpio_get;
+ gc->set = u8_gpio_set;
+
+ ret = of_mm_gpiochip_add(np, mm_gc);
+ if (ret)
+ goto err;
+ return 0;
+err:
+ kfree(u8_gc);
+ return ret;
+}
+
+void __init simple_gpiochip_init(const char *compatible)
+{
+ struct device_node *np;
+
+ for_each_compatible_node(np, NULL, compatible) {
+ int ret;
+ struct resource r;
+
+ ret = of_address_to_resource(np, 0, &r);
+ if (ret)
+ goto err;
+
+ switch (resource_size(&r)) {
+ case 1:
+ ret = u8_simple_gpiochip_add(np);
+ if (ret)
+ goto err;
+ break;
+ default:
+ /*
+ * Whenever you need support for GPIO bank width > 1,
+ * please just turn u8_ code into huge macros, and
+ * construct needed uX_ code with it.
+ */
+ ret = -ENOSYS;
+ goto err;
+ }
+ continue;
+err:
+ pr_err("%s: registration failed, status %d\n",
+ np->full_name, ret);
+ }
+}
diff --git a/arch/powerpc/sysdev/simple_gpio.h b/arch/powerpc/sysdev/simple_gpio.h
new file mode 100644
index 0000000..3a7b0c5
--- /dev/null
+++ b/arch/powerpc/sysdev/simple_gpio.h
@@ -0,0 +1,12 @@
+#ifndef __SYSDEV_SIMPLE_GPIO_H
+#define __SYSDEV_SIMPLE_GPIO_H
+
+#include <linux/errno.h>
+
+#ifdef CONFIG_SIMPLE_GPIO
+extern void simple_gpiochip_init(const char *compatible);
+#else
+static inline void simple_gpiochip_init(const char *compatible) {}
+#endif /* CONFIG_SIMPLE_GPIO */
+
+#endif /* __SYSDEV_SIMPLE_GPIO_H */
diff --git a/arch/sparc/kernel/sun4m_smp.c b/arch/sparc/kernel/sun4m_smp.c
index 4f8d605..8040376 100644
--- a/arch/sparc/kernel/sun4m_smp.c
+++ b/arch/sparc/kernel/sun4m_smp.c
@@ -54,7 +54,8 @@
#define SMP_PRINTK(x)
#endif
-static inline unsigned long swap(volatile unsigned long *ptr, unsigned long val)
+static inline unsigned long
+swap_ulong(volatile unsigned long *ptr, unsigned long val)
{
__asm__ __volatile__("swap [%1], %0\n\t" :
"=&r" (val), "=&r" (ptr) :
@@ -90,7 +91,7 @@
* to call the scheduler code.
*/
/* Allow master to continue. */
- swap(&cpu_callin_map[cpuid], 1);
+ swap_ulong(&cpu_callin_map[cpuid], 1);
/* XXX: What's up with all the flushes? */
local_flush_cache_all();
diff --git a/arch/x86/include/asm/bitops.h b/arch/x86/include/asm/bitops.h
index 9fa9dcd..e02a359 100644
--- a/arch/x86/include/asm/bitops.h
+++ b/arch/x86/include/asm/bitops.h
@@ -300,7 +300,7 @@
return oldbit;
}
-static inline int constant_test_bit(int nr, const volatile unsigned long *addr)
+static inline int constant_test_bit(unsigned int nr, const volatile unsigned long *addr)
{
return ((1UL << (nr % BITS_PER_LONG)) &
(((unsigned long *)addr)[nr / BITS_PER_LONG])) != 0;
diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c
index 29dc0c8..d37593c 100644
--- a/arch/x86/kernel/acpi/boot.c
+++ b/arch/x86/kernel/acpi/boot.c
@@ -47,7 +47,7 @@
#endif
static int __initdata acpi_force = 0;
-
+u32 acpi_rsdt_forced;
#ifdef CONFIG_ACPI
int acpi_disabled = 0;
#else
@@ -1374,6 +1374,17 @@
"Invalid BIOS MADT, disabling ACPI\n");
disable_acpi();
}
+ } else {
+ /*
+ * ACPI found no MADT, and so ACPI wants UP PIC mode.
+ * In the event an MPS table was found, forget it.
+ * Boot with "acpi=off" to use MPS on such a system.
+ */
+ if (smp_found_config) {
+ printk(KERN_WARNING PREFIX
+ "No APIC-table, disabling MPS\n");
+ smp_found_config = 0;
+ }
}
/*
@@ -1809,6 +1820,10 @@
disable_acpi();
acpi_ht = 1;
}
+ /* acpi=rsdt use RSDT instead of XSDT */
+ else if (strcmp(arg, "rsdt") == 0) {
+ acpi_rsdt_forced = 1;
+ }
/* "acpi=noirq" disables ACPI interrupt routing */
else if (strcmp(arg, "noirq") == 0) {
acpi_noirq_set();
diff --git a/arch/x86/kernel/acpi/cstate.c b/arch/x86/kernel/acpi/cstate.c
index c2502eb..a4805b3 100644
--- a/arch/x86/kernel/acpi/cstate.c
+++ b/arch/x86/kernel/acpi/cstate.c
@@ -56,6 +56,7 @@
static short mwait_supported[ACPI_PROCESSOR_MAX_POWER];
#define MWAIT_SUBSTATE_MASK (0xf)
+#define MWAIT_CSTATE_MASK (0xf)
#define MWAIT_SUBSTATE_SIZE (4)
#define CPUID_MWAIT_LEAF (5)
@@ -98,7 +99,8 @@
cpuid(CPUID_MWAIT_LEAF, &eax, &ebx, &ecx, &edx);
/* Check whether this particular cx_type (in CST) is supported or not */
- cstate_type = (cx->address >> MWAIT_SUBSTATE_SIZE) + 1;
+ cstate_type = ((cx->address >> MWAIT_SUBSTATE_SIZE) &
+ MWAIT_CSTATE_MASK) + 1;
edx_part = edx >> (cstate_type * MWAIT_SUBSTATE_SIZE);
num_cstate_subtype = edx_part & MWAIT_SUBSTATE_MASK;
diff --git a/arch/x86/kernel/acpi/sleep.c b/arch/x86/kernel/acpi/sleep.c
index 806b4e9..707c1f6 100644
--- a/arch/x86/kernel/acpi/sleep.c
+++ b/arch/x86/kernel/acpi/sleep.c
@@ -159,6 +159,8 @@
#endif
if (strncmp(str, "old_ordering", 12) == 0)
acpi_old_suspend_ordering();
+ if (strncmp(str, "s4_nonvs", 8) == 0)
+ acpi_s4_no_nvs();
str = strchr(str, ',');
if (str != NULL)
str += strspn(str, ", \t");
diff --git a/arch/x86/kernel/cpu/cpufreq/longhaul.c b/arch/x86/kernel/cpu/cpufreq/longhaul.c
index b046185..a4cff5d 100644
--- a/arch/x86/kernel/cpu/cpufreq/longhaul.c
+++ b/arch/x86/kernel/cpu/cpufreq/longhaul.c
@@ -982,7 +982,7 @@
case 10:
printk(KERN_ERR PFX "Use acpi-cpufreq driver for VIA C7\n");
default:
- ;;
+ ;
}
return -ENODEV;
diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c
index 65a1394..e858268 100644
--- a/arch/x86/kernel/e820.c
+++ b/arch/x86/kernel/e820.c
@@ -665,6 +665,27 @@
}
#endif
+#ifdef CONFIG_HIBERNATION
+/**
+ * Mark ACPI NVS memory region, so that we can save/restore it during
+ * hibernation and the subsequent resume.
+ */
+static int __init e820_mark_nvs_memory(void)
+{
+ int i;
+
+ for (i = 0; i < e820.nr_map; i++) {
+ struct e820entry *ei = &e820.map[i];
+
+ if (ei->type == E820_NVS)
+ hibernate_nvs_register(ei->addr, ei->size);
+ }
+
+ return 0;
+}
+core_initcall(e820_mark_nvs_memory);
+#endif
+
/*
* Early reserved memory areas.
*/
diff --git a/arch/x86/kernel/early-quirks.c b/arch/x86/kernel/early-quirks.c
index 744aa7f..76b8cd9 100644
--- a/arch/x86/kernel/early-quirks.c
+++ b/arch/x86/kernel/early-quirks.c
@@ -201,6 +201,12 @@
void (*f)(int num, int slot, int func);
};
+/*
+ * Only works for devices on the root bus. If you add any devices
+ * not on bus 0 readd another loop level in early_quirks(). But
+ * be careful because at least the Nvidia quirk here relies on
+ * only matching on bus 0.
+ */
static struct chipset early_qrk[] __initdata = {
{ PCI_VENDOR_ID_NVIDIA, PCI_ANY_ID,
PCI_CLASS_BRIDGE_PCI, PCI_ANY_ID, QFLAG_APPLY_ONCE, nvidia_bugs },
@@ -267,17 +273,17 @@
void __init early_quirks(void)
{
- int num, slot, func;
+ int slot, func;
if (!early_pci_allowed())
return;
/* Poor man's PCI discovery */
- for (num = 0; num < 32; num++)
- for (slot = 0; slot < 32; slot++)
- for (func = 0; func < 8; func++) {
- /* Only probe function 0 on single fn devices */
- if (check_dev_quirk(num, slot, func))
- break;
- }
+ /* Only scan the root bus */
+ for (slot = 0; slot < 32; slot++)
+ for (func = 0; func < 8; func++) {
+ /* Only probe function 0 on single fn devices */
+ if (check_dev_quirk(0, slot, func))
+ break;
+ }
}
diff --git a/arch/x86/oprofile/op_model_amd.c b/arch/x86/oprofile/op_model_amd.c
index 98658f2..8fdf06e 100644
--- a/arch/x86/oprofile/op_model_amd.c
+++ b/arch/x86/oprofile/op_model_amd.c
@@ -2,7 +2,7 @@
* @file op_model_amd.c
* athlon / K7 / K8 / Family 10h model-specific MSR operations
*
- * @remark Copyright 2002-2008 OProfile authors
+ * @remark Copyright 2002-2009 OProfile authors
* @remark Read the file COPYING
*
* @author John Levon
@@ -10,7 +10,7 @@
* @author Graydon Hoare
* @author Robert Richter <robert.richter@amd.com>
* @author Barry Kasindorf
-*/
+ */
#include <linux/oprofile.h>
#include <linux/device.h>
@@ -60,53 +60,10 @@
#define IBS_OP_LOW_VALID_BIT (1ULL<<18) /* bit 18 */
#define IBS_OP_LOW_ENABLE (1ULL<<17) /* bit 17 */
-/* Codes used in cpu_buffer.c */
-/* This produces duplicate code, need to be fixed */
-#define IBS_FETCH_BEGIN 3
-#define IBS_OP_BEGIN 4
+#define IBS_FETCH_SIZE 6
+#define IBS_OP_SIZE 12
-/*
- * The function interface needs to be fixed, something like add
- * data. Should then be added to linux/oprofile.h.
- */
-extern void
-oprofile_add_ibs_sample(struct pt_regs * const regs,
- unsigned int * const ibs_sample, int ibs_code);
-
-struct ibs_fetch_sample {
- /* MSRC001_1031 IBS Fetch Linear Address Register */
- unsigned int ibs_fetch_lin_addr_low;
- unsigned int ibs_fetch_lin_addr_high;
- /* MSRC001_1030 IBS Fetch Control Register */
- unsigned int ibs_fetch_ctl_low;
- unsigned int ibs_fetch_ctl_high;
- /* MSRC001_1032 IBS Fetch Physical Address Register */
- unsigned int ibs_fetch_phys_addr_low;
- unsigned int ibs_fetch_phys_addr_high;
-};
-
-struct ibs_op_sample {
- /* MSRC001_1034 IBS Op Logical Address Register (IbsRIP) */
- unsigned int ibs_op_rip_low;
- unsigned int ibs_op_rip_high;
- /* MSRC001_1035 IBS Op Data Register */
- unsigned int ibs_op_data1_low;
- unsigned int ibs_op_data1_high;
- /* MSRC001_1036 IBS Op Data 2 Register */
- unsigned int ibs_op_data2_low;
- unsigned int ibs_op_data2_high;
- /* MSRC001_1037 IBS Op Data 3 Register */
- unsigned int ibs_op_data3_low;
- unsigned int ibs_op_data3_high;
- /* MSRC001_1038 IBS DC Linear Address Register (IbsDcLinAd) */
- unsigned int ibs_dc_linear_low;
- unsigned int ibs_dc_linear_high;
- /* MSRC001_1039 IBS DC Physical Address Register (IbsDcPhysAd) */
- unsigned int ibs_dc_phys_low;
- unsigned int ibs_dc_phys_high;
-};
-
-static int ibs_allowed; /* AMD Family10h and later */
+static int has_ibs; /* AMD Family10h and later */
struct op_ibs_config {
unsigned long op_enabled;
@@ -197,31 +154,29 @@
op_amd_handle_ibs(struct pt_regs * const regs,
struct op_msrs const * const msrs)
{
- unsigned int low, high;
- struct ibs_fetch_sample ibs_fetch;
- struct ibs_op_sample ibs_op;
+ u32 low, high;
+ u64 msr;
+ struct op_entry entry;
- if (!ibs_allowed)
+ if (!has_ibs)
return 1;
if (ibs_config.fetch_enabled) {
rdmsr(MSR_AMD64_IBSFETCHCTL, low, high);
if (high & IBS_FETCH_HIGH_VALID_BIT) {
- ibs_fetch.ibs_fetch_ctl_high = high;
- ibs_fetch.ibs_fetch_ctl_low = low;
- rdmsr(MSR_AMD64_IBSFETCHLINAD, low, high);
- ibs_fetch.ibs_fetch_lin_addr_high = high;
- ibs_fetch.ibs_fetch_lin_addr_low = low;
- rdmsr(MSR_AMD64_IBSFETCHPHYSAD, low, high);
- ibs_fetch.ibs_fetch_phys_addr_high = high;
- ibs_fetch.ibs_fetch_phys_addr_low = low;
-
- oprofile_add_ibs_sample(regs,
- (unsigned int *)&ibs_fetch,
- IBS_FETCH_BEGIN);
+ rdmsrl(MSR_AMD64_IBSFETCHLINAD, msr);
+ oprofile_write_reserve(&entry, regs, msr,
+ IBS_FETCH_CODE, IBS_FETCH_SIZE);
+ oprofile_add_data(&entry, (u32)msr);
+ oprofile_add_data(&entry, (u32)(msr >> 32));
+ oprofile_add_data(&entry, low);
+ oprofile_add_data(&entry, high);
+ rdmsrl(MSR_AMD64_IBSFETCHPHYSAD, msr);
+ oprofile_add_data(&entry, (u32)msr);
+ oprofile_add_data(&entry, (u32)(msr >> 32));
+ oprofile_write_commit(&entry);
/* reenable the IRQ */
- rdmsr(MSR_AMD64_IBSFETCHCTL, low, high);
high &= ~IBS_FETCH_HIGH_VALID_BIT;
high |= IBS_FETCH_HIGH_ENABLE;
low &= IBS_FETCH_LOW_MAX_CNT_MASK;
@@ -232,30 +187,29 @@
if (ibs_config.op_enabled) {
rdmsr(MSR_AMD64_IBSOPCTL, low, high);
if (low & IBS_OP_LOW_VALID_BIT) {
- rdmsr(MSR_AMD64_IBSOPRIP, low, high);
- ibs_op.ibs_op_rip_low = low;
- ibs_op.ibs_op_rip_high = high;
- rdmsr(MSR_AMD64_IBSOPDATA, low, high);
- ibs_op.ibs_op_data1_low = low;
- ibs_op.ibs_op_data1_high = high;
- rdmsr(MSR_AMD64_IBSOPDATA2, low, high);
- ibs_op.ibs_op_data2_low = low;
- ibs_op.ibs_op_data2_high = high;
- rdmsr(MSR_AMD64_IBSOPDATA3, low, high);
- ibs_op.ibs_op_data3_low = low;
- ibs_op.ibs_op_data3_high = high;
- rdmsr(MSR_AMD64_IBSDCLINAD, low, high);
- ibs_op.ibs_dc_linear_low = low;
- ibs_op.ibs_dc_linear_high = high;
- rdmsr(MSR_AMD64_IBSDCPHYSAD, low, high);
- ibs_op.ibs_dc_phys_low = low;
- ibs_op.ibs_dc_phys_high = high;
+ rdmsrl(MSR_AMD64_IBSOPRIP, msr);
+ oprofile_write_reserve(&entry, regs, msr,
+ IBS_OP_CODE, IBS_OP_SIZE);
+ oprofile_add_data(&entry, (u32)msr);
+ oprofile_add_data(&entry, (u32)(msr >> 32));
+ rdmsrl(MSR_AMD64_IBSOPDATA, msr);
+ oprofile_add_data(&entry, (u32)msr);
+ oprofile_add_data(&entry, (u32)(msr >> 32));
+ rdmsrl(MSR_AMD64_IBSOPDATA2, msr);
+ oprofile_add_data(&entry, (u32)msr);
+ oprofile_add_data(&entry, (u32)(msr >> 32));
+ rdmsrl(MSR_AMD64_IBSOPDATA3, msr);
+ oprofile_add_data(&entry, (u32)msr);
+ oprofile_add_data(&entry, (u32)(msr >> 32));
+ rdmsrl(MSR_AMD64_IBSDCLINAD, msr);
+ oprofile_add_data(&entry, (u32)msr);
+ oprofile_add_data(&entry, (u32)(msr >> 32));
+ rdmsrl(MSR_AMD64_IBSDCPHYSAD, msr);
+ oprofile_add_data(&entry, (u32)msr);
+ oprofile_add_data(&entry, (u32)(msr >> 32));
+ oprofile_write_commit(&entry);
/* reenable the IRQ */
- oprofile_add_ibs_sample(regs,
- (unsigned int *)&ibs_op,
- IBS_OP_BEGIN);
- rdmsr(MSR_AMD64_IBSOPCTL, low, high);
high = 0;
low &= ~IBS_OP_LOW_VALID_BIT;
low |= IBS_OP_LOW_ENABLE;
@@ -305,14 +259,14 @@
}
#ifdef CONFIG_OPROFILE_IBS
- if (ibs_allowed && ibs_config.fetch_enabled) {
+ if (has_ibs && ibs_config.fetch_enabled) {
low = (ibs_config.max_cnt_fetch >> 4) & 0xFFFF;
high = ((ibs_config.rand_en & 0x1) << 25) /* bit 57 */
+ IBS_FETCH_HIGH_ENABLE;
wrmsr(MSR_AMD64_IBSFETCHCTL, low, high);
}
- if (ibs_allowed && ibs_config.op_enabled) {
+ if (has_ibs && ibs_config.op_enabled) {
low = ((ibs_config.max_cnt_op >> 4) & 0xFFFF)
+ ((ibs_config.dispatched_ops & 0x1) << 19) /* bit 19 */
+ IBS_OP_LOW_ENABLE;
@@ -341,14 +295,14 @@
}
#ifdef CONFIG_OPROFILE_IBS
- if (ibs_allowed && ibs_config.fetch_enabled) {
+ if (has_ibs && ibs_config.fetch_enabled) {
/* clear max count and enable */
low = 0;
high = 0;
wrmsr(MSR_AMD64_IBSFETCHCTL, low, high);
}
- if (ibs_allowed && ibs_config.op_enabled) {
+ if (has_ibs && ibs_config.op_enabled) {
/* clear max count and enable */
low = 0;
high = 0;
@@ -409,6 +363,7 @@
| IBSCTL_LVTOFFSETVAL);
pci_read_config_dword(cpu_cfg, IBSCTL, &value);
if (value != (ibs_eilvt_off | IBSCTL_LVTOFFSETVAL)) {
+ pci_dev_put(cpu_cfg);
printk(KERN_DEBUG "Failed to setup IBS LVT offset, "
"IBSCTL = 0x%08x", value);
return 1;
@@ -436,20 +391,20 @@
/* uninitialize the APIC for the IBS interrupts if needed */
static void clear_ibs_nmi(void)
{
- if (ibs_allowed)
+ if (has_ibs)
on_each_cpu(apic_clear_ibs_nmi_per_cpu, NULL, 1);
}
/* initialize the APIC for the IBS interrupts if available */
static void ibs_init(void)
{
- ibs_allowed = boot_cpu_has(X86_FEATURE_IBS);
+ has_ibs = boot_cpu_has(X86_FEATURE_IBS);
- if (!ibs_allowed)
+ if (!has_ibs)
return;
if (init_ibs_nmi()) {
- ibs_allowed = 0;
+ has_ibs = 0;
return;
}
@@ -458,7 +413,7 @@
static void ibs_exit(void)
{
- if (!ibs_allowed)
+ if (!has_ibs)
return;
clear_ibs_nmi();
@@ -478,7 +433,7 @@
if (ret)
return ret;
- if (!ibs_allowed)
+ if (!has_ibs)
return ret;
/* model specific files */
diff --git a/block/Kconfig b/block/Kconfig
index ac0956f..0cbb3b8 100644
--- a/block/Kconfig
+++ b/block/Kconfig
@@ -36,6 +36,12 @@
This option also enables support for single files larger than
2TB.
+ The ext4 filesystem requires that this feature be enabled in
+ order to support filesystems that have the huge_file feature
+ enabled. Otherwise, it will refuse to mount any filesystems
+ that use the huge_file feature, which is enabled by default
+ by mke2fs.ext4. The GFS2 filesystem also requires this feature.
+
If unsure, say N.
config BLK_DEV_IO_TRACE
diff --git a/block/blk-map.c b/block/blk-map.c
index 2990447..f103729 100644
--- a/block/blk-map.c
+++ b/block/blk-map.c
@@ -42,7 +42,7 @@
static int __blk_rq_map_user(struct request_queue *q, struct request *rq,
struct rq_map_data *map_data, void __user *ubuf,
- unsigned int len, int null_mapped, gfp_t gfp_mask)
+ unsigned int len, gfp_t gfp_mask)
{
unsigned long uaddr;
struct bio *bio, *orig_bio;
@@ -63,7 +63,7 @@
if (IS_ERR(bio))
return PTR_ERR(bio);
- if (null_mapped)
+ if (map_data && map_data->null_mapped)
bio->bi_flags |= (1 << BIO_NULL_MAPPED);
orig_bio = bio;
@@ -114,17 +114,15 @@
{
unsigned long bytes_read = 0;
struct bio *bio = NULL;
- int ret, null_mapped = 0;
+ int ret;
if (len > (q->max_hw_sectors << 9))
return -EINVAL;
if (!len)
return -EINVAL;
- if (!ubuf) {
- if (!map_data || rq_data_dir(rq) != READ)
- return -EINVAL;
- null_mapped = 1;
- }
+
+ if (!ubuf && (!map_data || !map_data->null_mapped))
+ return -EINVAL;
while (bytes_read != len) {
unsigned long map_len, end, start;
@@ -143,13 +141,16 @@
map_len -= PAGE_SIZE;
ret = __blk_rq_map_user(q, rq, map_data, ubuf, map_len,
- null_mapped, gfp_mask);
+ gfp_mask);
if (ret < 0)
goto unmap_rq;
if (!bio)
bio = rq->bio;
bytes_read += ret;
ubuf += ret;
+
+ if (map_data)
+ map_data->offset += ret;
}
if (!bio_flagged(bio, BIO_USER_MAPPED))
diff --git a/crypto/async_tx/async_tx.c b/crypto/async_tx/async_tx.c
index dcbf1be..f21147f 100644
--- a/crypto/async_tx/async_tx.c
+++ b/crypto/async_tx/async_tx.c
@@ -28,351 +28,18 @@
#include <linux/async_tx.h>
#ifdef CONFIG_DMA_ENGINE
-static enum dma_state_client
-dma_channel_add_remove(struct dma_client *client,
- struct dma_chan *chan, enum dma_state state);
-
-static struct dma_client async_tx_dma = {
- .event_callback = dma_channel_add_remove,
- /* .cap_mask == 0 defaults to all channels */
-};
-
-/**
- * dma_cap_mask_all - enable iteration over all operation types
- */
-static dma_cap_mask_t dma_cap_mask_all;
-
-/**
- * chan_ref_percpu - tracks channel allocations per core/opertion
- */
-struct chan_ref_percpu {
- struct dma_chan_ref *ref;
-};
-
-static int channel_table_initialized;
-static struct chan_ref_percpu *channel_table[DMA_TX_TYPE_END];
-
-/**
- * async_tx_lock - protect modification of async_tx_master_list and serialize
- * rebalance operations
- */
-static spinlock_t async_tx_lock;
-
-static LIST_HEAD(async_tx_master_list);
-
-/* async_tx_issue_pending_all - start all transactions on all channels */
-void async_tx_issue_pending_all(void)
+static int __init async_tx_init(void)
{
- struct dma_chan_ref *ref;
-
- rcu_read_lock();
- list_for_each_entry_rcu(ref, &async_tx_master_list, node)
- ref->chan->device->device_issue_pending(ref->chan);
- rcu_read_unlock();
-}
-EXPORT_SYMBOL_GPL(async_tx_issue_pending_all);
-
-/* dma_wait_for_async_tx - spin wait for a transcation to complete
- * @tx: transaction to wait on
- */
-enum dma_status
-dma_wait_for_async_tx(struct dma_async_tx_descriptor *tx)
-{
- enum dma_status status;
- struct dma_async_tx_descriptor *iter;
- struct dma_async_tx_descriptor *parent;
-
- if (!tx)
- return DMA_SUCCESS;
-
- /* poll through the dependency chain, return when tx is complete */
- do {
- iter = tx;
-
- /* find the root of the unsubmitted dependency chain */
- do {
- parent = iter->parent;
- if (!parent)
- break;
- else
- iter = parent;
- } while (parent);
-
- /* there is a small window for ->parent == NULL and
- * ->cookie == -EBUSY
- */
- while (iter->cookie == -EBUSY)
- cpu_relax();
-
- status = dma_sync_wait(iter->chan, iter->cookie);
- } while (status == DMA_IN_PROGRESS || (iter != tx));
-
- return status;
-}
-EXPORT_SYMBOL_GPL(dma_wait_for_async_tx);
-
-/* async_tx_run_dependencies - helper routine for dma drivers to process
- * (start) dependent operations on their target channel
- * @tx: transaction with dependencies
- */
-void async_tx_run_dependencies(struct dma_async_tx_descriptor *tx)
-{
- struct dma_async_tx_descriptor *dep = tx->next;
- struct dma_async_tx_descriptor *dep_next;
- struct dma_chan *chan;
-
- if (!dep)
- return;
-
- chan = dep->chan;
-
- /* keep submitting up until a channel switch is detected
- * in that case we will be called again as a result of
- * processing the interrupt from async_tx_channel_switch
- */
- for (; dep; dep = dep_next) {
- spin_lock_bh(&dep->lock);
- dep->parent = NULL;
- dep_next = dep->next;
- if (dep_next && dep_next->chan == chan)
- dep->next = NULL; /* ->next will be submitted */
- else
- dep_next = NULL; /* submit current dep and terminate */
- spin_unlock_bh(&dep->lock);
-
- dep->tx_submit(dep);
- }
-
- chan->device->device_issue_pending(chan);
-}
-EXPORT_SYMBOL_GPL(async_tx_run_dependencies);
-
-static void
-free_dma_chan_ref(struct rcu_head *rcu)
-{
- struct dma_chan_ref *ref;
- ref = container_of(rcu, struct dma_chan_ref, rcu);
- kfree(ref);
-}
-
-static void
-init_dma_chan_ref(struct dma_chan_ref *ref, struct dma_chan *chan)
-{
- INIT_LIST_HEAD(&ref->node);
- INIT_RCU_HEAD(&ref->rcu);
- ref->chan = chan;
- atomic_set(&ref->count, 0);
-}
-
-/**
- * get_chan_ref_by_cap - returns the nth channel of the given capability
- * defaults to returning the channel with the desired capability and the
- * lowest reference count if the index can not be satisfied
- * @cap: capability to match
- * @index: nth channel desired, passing -1 has the effect of forcing the
- * default return value
- */
-static struct dma_chan_ref *
-get_chan_ref_by_cap(enum dma_transaction_type cap, int index)
-{
- struct dma_chan_ref *ret_ref = NULL, *min_ref = NULL, *ref;
-
- rcu_read_lock();
- list_for_each_entry_rcu(ref, &async_tx_master_list, node)
- if (dma_has_cap(cap, ref->chan->device->cap_mask)) {
- if (!min_ref)
- min_ref = ref;
- else if (atomic_read(&ref->count) <
- atomic_read(&min_ref->count))
- min_ref = ref;
-
- if (index-- == 0) {
- ret_ref = ref;
- break;
- }
- }
- rcu_read_unlock();
-
- if (!ret_ref)
- ret_ref = min_ref;
-
- if (ret_ref)
- atomic_inc(&ret_ref->count);
-
- return ret_ref;
-}
-
-/**
- * async_tx_rebalance - redistribute the available channels, optimize
- * for cpu isolation in the SMP case, and opertaion isolation in the
- * uniprocessor case
- */
-static void async_tx_rebalance(void)
-{
- int cpu, cap, cpu_idx = 0;
- unsigned long flags;
-
- if (!channel_table_initialized)
- return;
-
- spin_lock_irqsave(&async_tx_lock, flags);
-
- /* undo the last distribution */
- for_each_dma_cap_mask(cap, dma_cap_mask_all)
- for_each_possible_cpu(cpu) {
- struct dma_chan_ref *ref =
- per_cpu_ptr(channel_table[cap], cpu)->ref;
- if (ref) {
- atomic_set(&ref->count, 0);
- per_cpu_ptr(channel_table[cap], cpu)->ref =
- NULL;
- }
- }
-
- for_each_dma_cap_mask(cap, dma_cap_mask_all)
- for_each_online_cpu(cpu) {
- struct dma_chan_ref *new;
- if (NR_CPUS > 1)
- new = get_chan_ref_by_cap(cap, cpu_idx++);
- else
- new = get_chan_ref_by_cap(cap, -1);
-
- per_cpu_ptr(channel_table[cap], cpu)->ref = new;
- }
-
- spin_unlock_irqrestore(&async_tx_lock, flags);
-}
-
-static enum dma_state_client
-dma_channel_add_remove(struct dma_client *client,
- struct dma_chan *chan, enum dma_state state)
-{
- unsigned long found, flags;
- struct dma_chan_ref *master_ref, *ref;
- enum dma_state_client ack = DMA_DUP; /* default: take no action */
-
- switch (state) {
- case DMA_RESOURCE_AVAILABLE:
- found = 0;
- rcu_read_lock();
- list_for_each_entry_rcu(ref, &async_tx_master_list, node)
- if (ref->chan == chan) {
- found = 1;
- break;
- }
- rcu_read_unlock();
-
- pr_debug("async_tx: dma resource available [%s]\n",
- found ? "old" : "new");
-
- if (!found)
- ack = DMA_ACK;
- else
- break;
-
- /* add the channel to the generic management list */
- master_ref = kmalloc(sizeof(*master_ref), GFP_KERNEL);
- if (master_ref) {
- /* keep a reference until async_tx is unloaded */
- dma_chan_get(chan);
- init_dma_chan_ref(master_ref, chan);
- spin_lock_irqsave(&async_tx_lock, flags);
- list_add_tail_rcu(&master_ref->node,
- &async_tx_master_list);
- spin_unlock_irqrestore(&async_tx_lock,
- flags);
- } else {
- printk(KERN_WARNING "async_tx: unable to create"
- " new master entry in response to"
- " a DMA_RESOURCE_ADDED event"
- " (-ENOMEM)\n");
- return 0;
- }
-
- async_tx_rebalance();
- break;
- case DMA_RESOURCE_REMOVED:
- found = 0;
- spin_lock_irqsave(&async_tx_lock, flags);
- list_for_each_entry(ref, &async_tx_master_list, node)
- if (ref->chan == chan) {
- /* permit backing devices to go away */
- dma_chan_put(ref->chan);
- list_del_rcu(&ref->node);
- call_rcu(&ref->rcu, free_dma_chan_ref);
- found = 1;
- break;
- }
- spin_unlock_irqrestore(&async_tx_lock, flags);
-
- pr_debug("async_tx: dma resource removed [%s]\n",
- found ? "ours" : "not ours");
-
- if (found)
- ack = DMA_ACK;
- else
- break;
-
- async_tx_rebalance();
- break;
- case DMA_RESOURCE_SUSPEND:
- case DMA_RESOURCE_RESUME:
- printk(KERN_WARNING "async_tx: does not support dma channel"
- " suspend/resume\n");
- break;
- default:
- BUG();
- }
-
- return ack;
-}
-
-static int __init
-async_tx_init(void)
-{
- enum dma_transaction_type cap;
-
- spin_lock_init(&async_tx_lock);
- bitmap_fill(dma_cap_mask_all.bits, DMA_TX_TYPE_END);
-
- /* an interrupt will never be an explicit operation type.
- * clearing this bit prevents allocation to a slot in 'channel_table'
- */
- clear_bit(DMA_INTERRUPT, dma_cap_mask_all.bits);
-
- for_each_dma_cap_mask(cap, dma_cap_mask_all) {
- channel_table[cap] = alloc_percpu(struct chan_ref_percpu);
- if (!channel_table[cap])
- goto err;
- }
-
- channel_table_initialized = 1;
- dma_async_client_register(&async_tx_dma);
- dma_async_client_chan_request(&async_tx_dma);
+ dmaengine_get();
printk(KERN_INFO "async_tx: api initialized (async)\n");
return 0;
-err:
- printk(KERN_ERR "async_tx: initialization failure\n");
-
- while (--cap >= 0)
- free_percpu(channel_table[cap]);
-
- return 1;
}
static void __exit async_tx_exit(void)
{
- enum dma_transaction_type cap;
-
- channel_table_initialized = 0;
-
- for_each_dma_cap_mask(cap, dma_cap_mask_all)
- if (channel_table[cap])
- free_percpu(channel_table[cap]);
-
- dma_async_client_unregister(&async_tx_dma);
+ dmaengine_put();
}
/**
@@ -387,16 +54,9 @@
{
/* see if we can keep the chain on one channel */
if (depend_tx &&
- dma_has_cap(tx_type, depend_tx->chan->device->cap_mask))
+ dma_has_cap(tx_type, depend_tx->chan->device->cap_mask))
return depend_tx->chan;
- else if (likely(channel_table_initialized)) {
- struct dma_chan_ref *ref;
- int cpu = get_cpu();
- ref = per_cpu_ptr(channel_table[tx_type], cpu)->ref;
- put_cpu();
- return ref ? ref->chan : NULL;
- } else
- return NULL;
+ return dma_find_channel(tx_type);
}
EXPORT_SYMBOL_GPL(__async_tx_find_channel);
#else
diff --git a/drivers/Kconfig b/drivers/Kconfig
index 2f557f5..00cf955 100644
--- a/drivers/Kconfig
+++ b/drivers/Kconfig
@@ -107,4 +107,6 @@
source "drivers/xen/Kconfig"
source "drivers/staging/Kconfig"
+
+source "drivers/platform/Kconfig"
endmenu
diff --git a/drivers/Makefile b/drivers/Makefile
index e121b66..c1bf417 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -18,6 +18,9 @@
obj-$(CONFIG_XEN) += xen/
+# regulators early, since some subsystems rely on them to initialize
+obj-$(CONFIG_REGULATOR) += regulator/
+
# char/ comes before serial/ etc so that the VT console is the boot-time
# default.
obj-y += char/
@@ -101,5 +104,5 @@
obj-$(CONFIG_OF) += of/
obj-$(CONFIG_SSB) += ssb/
obj-$(CONFIG_VIRTIO) += virtio/
-obj-$(CONFIG_REGULATOR) += regulator/
obj-$(CONFIG_STAGING) += staging/
+obj-y += platform/
diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
index b0243fd..d7f9839 100644
--- a/drivers/acpi/Kconfig
+++ b/drivers/acpi/Kconfig
@@ -196,90 +196,6 @@
depends on (X86 || IA64)
default y if IA64_GENERIC || IA64_SGI_SN2
-config ACPI_WMI
- tristate "WMI (EXPERIMENTAL)"
- depends on X86
- depends on EXPERIMENTAL
- help
- This driver adds support for the ACPI-WMI (Windows Management
- Instrumentation) mapper device (PNP0C14) found on some systems.
-
- ACPI-WMI is a proprietary extension to ACPI to expose parts of the
- ACPI firmware to userspace - this is done through various vendor
- defined methods and data blocks in a PNP0C14 device, which are then
- made available for userspace to call.
-
- The implementation of this in Linux currently only exposes this to
- other kernel space drivers.
-
- This driver is a required dependency to build the firmware specific
- drivers needed on many machines, including Acer and HP laptops.
-
- It is safe to enable this driver even if your DSDT doesn't define
- any ACPI-WMI devices.
-
-config ACPI_ASUS
- tristate "ASUS/Medion Laptop Extras"
- depends on X86
- select BACKLIGHT_CLASS_DEVICE
- ---help---
- This driver provides support for extra features of ACPI-compatible
- ASUS laptops. As some of Medion laptops are made by ASUS, it may also
- support some Medion laptops (such as 9675 for example). It makes all
- the extra buttons generate standard ACPI events that go through
- /proc/acpi/events, and (on some models) adds support for changing the
- display brightness and output, switching the LCD backlight on and off,
- and most importantly, allows you to blink those fancy LEDs intended
- for reporting mail and wireless status.
-
- Note: display switching code is currently considered EXPERIMENTAL,
- toying with these values may even lock your machine.
-
- All settings are changed via /proc/acpi/asus directory entries. Owner
- and group for these entries can be set with asus_uid and asus_gid
- parameters.
-
- More information and a userspace daemon for handling the extra buttons
- at <http://sourceforge.net/projects/acpi4asus/>.
-
- If you have an ACPI-compatible ASUS laptop, say Y or M here. This
- driver is still under development, so if your laptop is unsupported or
- something works not quite as expected, please use the mailing list
- available on the above page (acpi4asus-user@lists.sourceforge.net).
-
- NOTE: This driver is deprecated and will probably be removed soon,
- use asus-laptop instead.
-
-config ACPI_TOSHIBA
- tristate "Toshiba Laptop Extras"
- depends on X86 && INPUT
- select INPUT_POLLDEV
- select NET
- select RFKILL
- select BACKLIGHT_CLASS_DEVICE
- ---help---
- This driver adds support for access to certain system settings
- on "legacy free" Toshiba laptops. These laptops can be recognized by
- their lack of a BIOS setup menu and APM support.
-
- On these machines, all system configuration is handled through the
- ACPI. This driver is required for access to controls not covered
- by the general ACPI drivers, such as LCD brightness, video output,
- etc.
-
- This driver differs from the non-ACPI Toshiba laptop driver (located
- under "Processor type and features") in several aspects.
- Configuration is accessed by reading and writing text files in the
- /proc tree instead of by program interface to /dev. Furthermore, no
- power management functions are exposed, as those are handled by the
- general ACPI drivers.
-
- More information about this driver is available at
- <http://memebeam.org/toys/ToshibaAcpiDriver>.
-
- If you have a legacy free Toshiba laptop (such as the Libretto L1
- series), say Y.
-
config ACPI_CUSTOM_DSDT_FILE
string "Custom DSDT Table file to include"
default ""
diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile
index 3c0c933..d80f4cc 100644
--- a/drivers/acpi/Makefile
+++ b/drivers/acpi/Makefile
@@ -2,15 +2,8 @@
# Makefile for the Linux ACPI interpreter
#
-export ACPI_CFLAGS
-
-ACPI_CFLAGS := -Os
-
-ifdef CONFIG_ACPI_DEBUG
- ACPI_CFLAGS += -DACPI_DEBUG_OUTPUT
-endif
-
-EXTRA_CFLAGS += $(ACPI_CFLAGS)
+ccflags-y := -Os
+ccflags-$(CONFIG_ACPI_DEBUG) += -DACPI_DEBUG_OUTPUT
#
# ACPI Boot-Time Table Parsing
@@ -22,9 +15,13 @@
# ACPI Core Subsystem (Interpreter)
#
obj-y += osl.o utils.o reboot.o\
- dispatcher/ events/ executer/ hardware/ \
- namespace/ parser/ resources/ tables/ \
- utilities/
+ acpica/
+
+# sleep related files
+obj-y += wakeup.o
+obj-y += main.o
+obj-$(CONFIG_ACPI_SLEEP) += proc.o
+
#
# ACPI Bus and Device Drivers
@@ -35,7 +32,6 @@
processor-objs += processor_perflib.o
endif
-obj-y += sleep/
obj-y += bus.o glue.o
obj-y += scan.o
# Keep EC driver first. Initialization of others depend on it.
@@ -59,9 +55,6 @@
obj-$(CONFIG_ACPI_SYSTEM) += system.o event.o
obj-$(CONFIG_ACPI_DEBUG) += debug.o
obj-$(CONFIG_ACPI_NUMA) += numa.o
-obj-$(CONFIG_ACPI_WMI) += wmi.o
-obj-$(CONFIG_ACPI_ASUS) += asus_acpi.o
-obj-$(CONFIG_ACPI_TOSHIBA) += toshiba_acpi.o
obj-$(CONFIG_ACPI_HOTPLUG_MEMORY) += acpi_memhotplug.o
obj-$(CONFIG_ACPI_PROCFS_POWER) += cm_sbs.o
obj-$(CONFIG_ACPI_SBS) += sbshc.o
diff --git a/drivers/acpi/acpica/Makefile b/drivers/acpi/acpica/Makefile
new file mode 100644
index 0000000..3f23298
--- /dev/null
+++ b/drivers/acpi/acpica/Makefile
@@ -0,0 +1,44 @@
+#
+# Makefile for ACPICA Core interpreter
+#
+
+ccflags-y := -Os
+ccflags-$(CONFIG_ACPI_DEBUG) += -DACPI_DEBUG_OUTPUT
+
+obj-y := dsfield.o dsmthdat.o dsopcode.o dswexec.o dswscope.o \
+ dsmethod.o dsobject.o dsutils.o dswload.o dswstate.o \
+ dsinit.o
+
+obj-y += evevent.o evregion.o evsci.o evxfevnt.o \
+ evmisc.o evrgnini.o evxface.o evxfregn.o \
+ evgpe.o evgpeblk.o
+
+obj-y += exconfig.o exfield.o exnames.o exoparg6.o exresolv.o exstorob.o\
+ exconvrt.o exfldio.o exoparg1.o exprep.o exresop.o exsystem.o\
+ excreate.o exmisc.o exoparg2.o exregion.o exstore.o exutils.o \
+ exdump.o exmutex.o exoparg3.o exresnte.o exstoren.o
+
+obj-y += hwacpi.o hwgpe.o hwregs.o hwsleep.o hwxface.o
+
+obj-$(ACPI_FUTURE_USAGE) += hwtimer.o
+
+obj-y += nsaccess.o nsload.o nssearch.o nsxfeval.o \
+ nsalloc.o nseval.o nsnames.o nsutils.o nsxfname.o \
+ nsdump.o nsinit.o nsobject.o nswalk.o nsxfobj.o \
+ nsparse.o nspredef.o
+
+obj-$(ACPI_FUTURE_USAGE) += nsdumpdv.o
+
+obj-y += psargs.o psparse.o psloop.o pstree.o pswalk.o \
+ psopcode.o psscope.o psutils.o psxface.o
+
+obj-y += rsaddr.o rscreate.o rsinfo.o rsio.o rslist.o rsmisc.o rsxface.o \
+ rscalc.o rsirq.o rsmemory.o rsutils.o
+
+obj-$(ACPI_FUTURE_USAGE) += rsdump.o
+
+obj-y += tbxface.o tbinstal.o tbutils.o tbfind.o tbfadt.o tbxfroot.o
+
+obj-y += utalloc.o utdebug.o uteval.o utinit.o utmisc.o utxface.o \
+ utcopy.o utdelete.o utglobal.o utmath.o utobject.o \
+ utstate.o utmutex.o utobject.o utresrc.o
diff --git a/drivers/acpi/acpica/accommon.h b/drivers/acpi/acpica/accommon.h
new file mode 100644
index 0000000..3b20786
--- /dev/null
+++ b/drivers/acpi/acpica/accommon.h
@@ -0,0 +1,63 @@
+/******************************************************************************
+ *
+ * Name: accommon.h - Common include files for generation of ACPICA source
+ *
+ *****************************************************************************/
+
+/*
+ * Copyright (C) 2000 - 2008, Intel Corp.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions, and the following disclaimer,
+ * without modification.
+ * 2. Redistributions in binary form must reproduce at minimum a disclaimer
+ * substantially similar to the "NO WARRANTY" disclaimer below
+ * ("Disclaimer") and any redistribution must be conditioned upon
+ * including a substantially similar Disclaimer requirement for further
+ * binary redistribution.
+ * 3. Neither the names of the above-listed copyright holders nor the names
+ * of any contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * Alternatively, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") version 2 as published by the Free
+ * Software Foundation.
+ *
+ * NO WARRANTY
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGES.
+ */
+
+#ifndef __ACCOMMON_H__
+#define __ACCOMMON_H__
+
+/*
+ * Common set of includes for all ACPICA source files.
+ * We put them here because we don't want to duplicate them
+ * in the the source code again and again.
+ *
+ * Note: The order of these include files is important.
+ */
+#include "acconfig.h" /* Global configuration constants */
+#include "acmacros.h" /* C macros */
+#include "aclocal.h" /* Internal data types */
+#include "acobject.h" /* ACPI internal object */
+#include "acstruct.h" /* Common structures */
+#include "acglobal.h" /* All global variables */
+#include "achware.h" /* Hardware defines and interfaces */
+#include "acutils.h" /* Utility interfaces */
+
+#endif /* __ACCOMMON_H__ */
diff --git a/include/acpi/acconfig.h b/drivers/acpi/acpica/acconfig.h
similarity index 98%
rename from include/acpi/acconfig.h
rename to drivers/acpi/acpica/acconfig.h
index 29feee2..e6777fb 100644
--- a/include/acpi/acconfig.h
+++ b/drivers/acpi/acpica/acconfig.h
@@ -61,10 +61,6 @@
*
*/
-/* Current ACPICA subsystem version in YYYYMMDD format */
-
-#define ACPI_CA_VERSION 0x20080926
-
/*
* OS name, used for the _OS object. The _OS object is essentially obsolete,
* but there is a large base of ASL/AML code in existing machines that check
@@ -119,6 +115,10 @@
#define ACPI_ROOT_TABLE_SIZE_INCREMENT 4
+/* Maximum number of While() loop iterations before forced abort */
+
+#define ACPI_MAX_LOOP_ITERATIONS 0xFFFF
+
/******************************************************************************
*
* ACPI Specification constants (Do not change unless the specification changes)
diff --git a/include/acpi/acdebug.h b/drivers/acpi/acpica/acdebug.h
similarity index 100%
rename from include/acpi/acdebug.h
rename to drivers/acpi/acpica/acdebug.h
diff --git a/include/acpi/acdispat.h b/drivers/acpi/acpica/acdispat.h
similarity index 100%
rename from include/acpi/acdispat.h
rename to drivers/acpi/acpica/acdispat.h
diff --git a/include/acpi/acevents.h b/drivers/acpi/acpica/acevents.h
similarity index 97%
rename from include/acpi/acevents.h
rename to drivers/acpi/acpica/acevents.h
index d5d099b..07e2013 100644
--- a/include/acpi/acevents.h
+++ b/drivers/acpi/acpica/acevents.h
@@ -93,11 +93,13 @@
*/
u8 acpi_ev_valid_gpe_event(struct acpi_gpe_event_info *gpe_event_info);
-acpi_status acpi_ev_walk_gpe_list(acpi_gpe_callback gpe_walk_callback);
+acpi_status
+acpi_ev_walk_gpe_list(acpi_gpe_callback gpe_walk_callback, void *context);
acpi_status
acpi_ev_delete_gpe_handlers(struct acpi_gpe_xrupt_info *gpe_xrupt_info,
- struct acpi_gpe_block_info *gpe_block);
+ struct acpi_gpe_block_info *gpe_block,
+ void *context);
acpi_status
acpi_ev_create_gpe_block(struct acpi_namespace_node *gpe_device,
diff --git a/include/acpi/acglobal.h b/drivers/acpi/acpica/acglobal.h
similarity index 97%
rename from include/acpi/acglobal.h
rename to drivers/acpi/acpica/acglobal.h
index 15dda46..ddb40f5 100644
--- a/include/acpi/acglobal.h
+++ b/drivers/acpi/acpica/acglobal.h
@@ -102,6 +102,12 @@
*/
ACPI_EXTERN u8 ACPI_INIT_GLOBAL(acpi_gbl_leave_wake_gpes_disabled, TRUE);
+/*
+ * Optionally use default values for the ACPI register widths. Set this to
+ * TRUE to use the defaults, if an FADT contains incorrect widths/lengths.
+ */
+ACPI_EXTERN u8 ACPI_INIT_GLOBAL(acpi_gbl_use_default_register_widths, TRUE);
+
/*****************************************************************************
*
* Debug support
@@ -140,7 +146,7 @@
*/
ACPI_EXTERN struct acpi_internal_rsdt acpi_gbl_root_table_list;
ACPI_EXTERN struct acpi_table_fadt acpi_gbl_FADT;
-extern u8 acpi_gbl_permanent_mmap;
+ACPI_EXTERN struct acpi_table_facs *acpi_gbl_FACS;
/* These addresses are calculated from FADT address values */
@@ -326,6 +332,7 @@
ACPI_EXTERN struct acpi_gpe_xrupt_info *acpi_gbl_gpe_xrupt_list_head;
ACPI_EXTERN struct acpi_gpe_block_info
*acpi_gbl_gpe_fadt_blocks[ACPI_MAX_GPE_BLOCKS];
+ACPI_EXTERN u32 acpi_current_gpe_count;
/*****************************************************************************
*
diff --git a/include/acpi/achware.h b/drivers/acpi/acpica/achware.h
similarity index 88%
rename from include/acpi/achware.h
rename to drivers/acpi/acpica/achware.h
index 97a72b1..58c69dc 100644
--- a/include/acpi/achware.h
+++ b/drivers/acpi/acpica/achware.h
@@ -44,11 +44,7 @@
#ifndef __ACHWARE_H__
#define __ACHWARE_H__
-/* PM Timer ticks per second (HZ) */
-
-#define PM_TIMER_FREQUENCY 3579545
-
-/* Values for the _SST reserved method */
+/* Values for the _SST predefined method */
#define ACPI_SST_INDICATOR_OFF 0
#define ACPI_SST_WORKING 1
@@ -56,8 +52,6 @@
#define ACPI_SST_SLEEPING 3
#define ACPI_SST_SLEEP_CONTEXT 4
-/* Prototypes */
-
/*
* hwacpi - high level functions
*/
@@ -75,13 +69,6 @@
acpi_status acpi_hw_register_write(u32 register_id, u32 value);
-acpi_status
-acpi_hw_low_level_read(u32 width,
- u32 * value, struct acpi_generic_address *reg);
-
-acpi_status
-acpi_hw_low_level_write(u32 width, u32 value, struct acpi_generic_address *reg);
-
acpi_status acpi_hw_clear_acpi_status(void);
/*
@@ -94,13 +81,13 @@
acpi_status
acpi_hw_disable_gpe_block(struct acpi_gpe_xrupt_info *gpe_xrupt_info,
- struct acpi_gpe_block_info *gpe_block);
+ struct acpi_gpe_block_info *gpe_block, void *context);
acpi_status acpi_hw_clear_gpe(struct acpi_gpe_event_info *gpe_event_info);
acpi_status
acpi_hw_clear_gpe_block(struct acpi_gpe_xrupt_info *gpe_xrupt_info,
- struct acpi_gpe_block_info *gpe_block);
+ struct acpi_gpe_block_info *gpe_block, void *context);
acpi_status
acpi_hw_get_gpe_status(struct acpi_gpe_event_info *gpe_event_info,
@@ -114,7 +101,8 @@
acpi_status
acpi_hw_enable_runtime_gpe_block(struct acpi_gpe_xrupt_info *gpe_xrupt_info,
- struct acpi_gpe_block_info *gpe_block);
+ struct acpi_gpe_block_info *gpe_block,
+ void *context);
#ifdef ACPI_FUTURE_USAGE
/*
diff --git a/include/acpi/acinterp.h b/drivers/acpi/acpica/acinterp.h
similarity index 100%
rename from include/acpi/acinterp.h
rename to drivers/acpi/acpica/acinterp.h
diff --git a/include/acpi/aclocal.h b/drivers/acpi/acpica/aclocal.h
similarity index 95%
rename from include/acpi/aclocal.h
rename to drivers/acpi/acpica/aclocal.h
index ecab527..492d027 100644
--- a/include/acpi/aclocal.h
+++ b/drivers/acpi/acpica/aclocal.h
@@ -46,8 +46,6 @@
/* acpisrc:struct_defs -- for acpisrc conversion */
-#define ACPI_WAIT_FOREVER 0xFFFF /* u16, as per ACPI spec */
-#define ACPI_DO_NOT_WAIT 0
#define ACPI_SERIALIZED 0xFF
typedef u32 acpi_mutex_handle;
@@ -120,11 +118,6 @@
#define ACPI_MAX_LOCK 1
#define ACPI_NUM_LOCK ACPI_MAX_LOCK+1
-/* Owner IDs are used to track namespace nodes for selective deletion */
-
-typedef u8 acpi_owner_id;
-#define ACPI_OWNER_ID_MAX 0xFF
-
/* This Thread ID means that the mutex is not in use (unlocked) */
#define ACPI_MUTEX_NOT_ACQUIRED (acpi_thread_id) 0
@@ -165,11 +158,6 @@
ACPI_IMODE_EXECUTE = 0x03
} acpi_interpreter_mode;
-union acpi_name_union {
- u32 integer;
- char ascii[4];
-};
-
/*
* The Namespace Node describes a named object that appears in the AML.
* descriptor_type is used to differentiate between internal descriptors.
@@ -216,26 +204,6 @@
#define ANOBJ_IS_BIT_OFFSET 0x40 /* i_aSL only: Reference is a bit offset */
#define ANOBJ_IS_REFERENCED 0x80 /* i_aSL only: Object was referenced */
-/*
- * ACPI Table Descriptor. One per ACPI table
- */
-struct acpi_table_desc {
- acpi_physical_address address;
- struct acpi_table_header *pointer;
- u32 length; /* Length fixed at 32 bits */
- union acpi_name_union signature;
- acpi_owner_id owner_id;
- u8 flags;
-};
-
-/* Flags for above */
-
-#define ACPI_TABLE_ORIGIN_UNKNOWN (0)
-#define ACPI_TABLE_ORIGIN_MAPPED (1)
-#define ACPI_TABLE_ORIGIN_ALLOCATED (2)
-#define ACPI_TABLE_ORIGIN_MASK (3)
-#define ACPI_TABLE_IS_LOADED (4)
-
/* One internal RSDT for table management */
struct acpi_internal_rsdt {
@@ -266,15 +234,6 @@
struct acpi_namespace_node *node;
};
-/*
- * Predefined Namespace items
- */
-struct acpi_predefined_names {
- char *name;
- u8 type;
- char *val;
-};
-
/* Object types used during package copies */
#define ACPI_COPY_TYPE_SIMPLE 0
@@ -487,10 +446,15 @@
struct acpi_gpe_block_info *gpe_block;
};
-typedef acpi_status(*acpi_gpe_callback) (struct acpi_gpe_xrupt_info *
- gpe_xrupt_info,
- struct acpi_gpe_block_info *
- gpe_block);
+struct acpi_gpe_device_info {
+ u32 index;
+ u32 next_block_base_index;
+ acpi_status status;
+ struct acpi_namespace_node *gpe_device;
+};
+
+typedef acpi_status(*acpi_gpe_callback) (struct acpi_gpe_xrupt_info *gpe_xrupt_info,
+ struct acpi_gpe_block_info *gpe_block, void *context);
/* Information about each particular fixed event */
@@ -566,6 +530,7 @@
union acpi_parse_object *predicate_op;
u8 *aml_predicate_start; /* Start of if/while predicate */
u8 *package_end; /* End of if/while block */
+ u32 loop_count; /* While() loop counter */
};
/*
@@ -671,6 +636,12 @@
union acpi_parse_object *arg; /* arguments and contained ops */
};
+#ifdef ACPI_DISASSEMBLER
+#define ACPI_DISASM_ONLY_MEMBERS(a) a;
+#else
+#define ACPI_DISASM_ONLY_MEMBERS(a)
+#endif
+
#define ACPI_PARSE_COMMON \
union acpi_parse_object *parent; /* Parent op */\
u8 descriptor_type; /* To differentiate various internal objs */\
@@ -790,9 +761,6 @@
*
****************************************************************************/
-#define PCI_ROOT_HID_STRING "PNP0A03"
-#define PCI_EXPRESS_ROOT_HID_STRING "PNP0A08"
-
struct acpi_bit_register_info {
u8 parent_register;
u8 bit_position;
@@ -1019,26 +987,4 @@
#define ACPI_MEM_LIST_MAX 1
#define ACPI_NUM_MEM_LISTS 2
-struct acpi_memory_list {
- char *list_name;
- void *list_head;
- u16 object_size;
- u16 max_depth;
- u16 current_depth;
- u16 link_offset;
-
-#ifdef ACPI_DBG_TRACK_ALLOCATIONS
-
- /* Statistics for debug memory tracking only */
-
- u32 total_allocated;
- u32 total_freed;
- u32 max_occupied;
- u32 total_size;
- u32 current_total_size;
- u32 requests;
- u32 hits;
-#endif
-};
-
#endif /* __ACLOCAL_H__ */
diff --git a/include/acpi/acmacros.h b/drivers/acpi/acpica/acmacros.h
similarity index 83%
rename from include/acpi/acmacros.h
rename to drivers/acpi/acpica/acmacros.h
index 1954c9d..9c127e8 100644
--- a/include/acpi/acmacros.h
+++ b/drivers/acpi/acpica/acmacros.h
@@ -45,23 +45,6 @@
#define __ACMACROS_H__
/*
- * Data manipulation macros
- */
-#define ACPI_LOWORD(l) ((u16)(u32)(l))
-#define ACPI_HIWORD(l) ((u16)((((u32)(l)) >> 16) & 0xFFFF))
-#define ACPI_LOBYTE(l) ((u8)(u16)(l))
-#define ACPI_HIBYTE(l) ((u8)((((u16)(l)) >> 8) & 0xFF))
-
-#define ACPI_SET_BIT(target,bit) ((target) |= (bit))
-#define ACPI_CLEAR_BIT(target,bit) ((target) &= ~(bit))
-#define ACPI_MIN(a,b) (((a)<(b))?(a):(b))
-#define ACPI_MAX(a,b) (((a)>(b))?(a):(b))
-
-/* Size calculation */
-
-#define ACPI_ARRAY_LENGTH(x) (sizeof(x) / sizeof((x)[0]))
-
-/*
* Extract data using a pointer. Any more than a byte and we
* get into potential aligment issues -- see the STORE macros below.
* Use with care.
@@ -76,39 +59,6 @@
#define ACPI_SET64(ptr) *ACPI_CAST_PTR (u64, ptr)
/*
- * Pointer manipulation
- */
-#define ACPI_CAST_PTR(t, p) ((t *) (acpi_uintptr_t) (p))
-#define ACPI_CAST_INDIRECT_PTR(t, p) ((t **) (acpi_uintptr_t) (p))
-#define ACPI_ADD_PTR(t, a, b) ACPI_CAST_PTR (t, (ACPI_CAST_PTR (u8, (a)) + (acpi_size)(b)))
-#define ACPI_PTR_DIFF(a, b) (acpi_size) (ACPI_CAST_PTR (u8, (a)) - ACPI_CAST_PTR (u8, (b)))
-
-/* Pointer/Integer type conversions */
-
-#define ACPI_TO_POINTER(i) ACPI_ADD_PTR (void, (void *) NULL, (acpi_size) i)
-#define ACPI_TO_INTEGER(p) ACPI_PTR_DIFF (p, (void *) NULL)
-#define ACPI_OFFSET(d, f) (acpi_size) ACPI_PTR_DIFF (&(((d *)0)->f), (void *) NULL)
-#define ACPI_PHYSADDR_TO_PTR(i) ACPI_TO_POINTER(i)
-#define ACPI_PTR_TO_PHYSADDR(i) ACPI_TO_INTEGER(i)
-
-#ifndef ACPI_MISALIGNMENT_NOT_SUPPORTED
-#define ACPI_COMPARE_NAME(a, b) (*ACPI_CAST_PTR (u32, (a)) == *ACPI_CAST_PTR (u32, (b)))
-#else
-#define ACPI_COMPARE_NAME(a, b) (!ACPI_STRNCMP (ACPI_CAST_PTR (char, (a)), ACPI_CAST_PTR (char, (b)), ACPI_NAME_SIZE))
-#endif
-
-/*
- * Full 64-bit integer must be available on both 32-bit and 64-bit platforms
- */
-struct acpi_integer_overlay {
- u32 lo_dword;
- u32 hi_dword;
-};
-
-#define ACPI_LODWORD(integer) (ACPI_CAST_PTR (struct acpi_integer_overlay, &integer)->lo_dword)
-#define ACPI_HIDWORD(integer) (ACPI_CAST_PTR (struct acpi_integer_overlay, &integer)->hi_dword)
-
-/*
* printf() format helpers
*/
@@ -209,7 +159,7 @@
/*
* The hardware does not support unaligned transfers. We must move the
* data one byte at a time. These macros work whether the source or
- * the destination (or both) is/are unaligned. (Little-endian move)
+ * the destination (or both) is/are unaligned. (Little-endian move)
*/
/* 16-bit source, 16/32/64 destination */
@@ -357,12 +307,6 @@
{(u32)(Pargs), (u32)(Iargs), (u32)(flags), obj_type, class, type}
#endif
-#ifdef ACPI_DISASSEMBLER
-#define ACPI_DISASM_ONLY_MEMBERS(a) a;
-#else
-#define ACPI_DISASM_ONLY_MEMBERS(a)
-#endif
-
#define ARG_TYPE_WIDTH 5
#define ARG_1(x) ((u32)(x))
#define ARG_2(x) ((u32)(x) << (1 * ARG_TYPE_WIDTH))
@@ -388,32 +332,16 @@
#define GET_CURRENT_ARG_TYPE(list) (list & ((u32) 0x1F))
#define INCREMENT_ARG_LIST(list) (list >>= ((u32) ARG_TYPE_WIDTH))
-#if defined (ACPI_DEBUG_OUTPUT) || !defined (ACPI_NO_ERROR_MESSAGES)
-/*
- * Module name is include in both debug and non-debug versions primarily for
- * error messages. The __FILE__ macro is not very useful for this, because it
- * often includes the entire pathname to the module
- */
-#define ACPI_MODULE_NAME(name) static const char ACPI_UNUSED_VAR _acpi_module_name[] = name;
-#else
-#define ACPI_MODULE_NAME(name)
-#endif
-
/*
* Ascii error messages can be configured out
*/
#ifndef ACPI_NO_ERROR_MESSAGES
-#define AE_INFO _acpi_module_name, __LINE__
/*
* Error reporting. Callers module and line number are inserted by AE_INFO,
* the plist contains a set of parens to allow variable-length lists.
* These macros are used for both the debug and non-debug versions of the code.
*/
-#define ACPI_INFO(plist) acpi_ut_info plist
-#define ACPI_WARNING(plist) acpi_ut_warning plist
-#define ACPI_EXCEPTION(plist) acpi_ut_exception plist
-#define ACPI_ERROR(plist) acpi_ut_error plist
#define ACPI_ERROR_NAMESPACE(s, e) acpi_ns_report_error (AE_INFO, s, e);
#define ACPI_ERROR_METHOD(s, n, p, e) acpi_ns_report_method_error (AE_INFO, s, n, p, e);
@@ -421,13 +349,9 @@
/* No error messages */
-#define ACPI_INFO(plist)
-#define ACPI_WARNING(plist)
-#define ACPI_EXCEPTION(plist)
-#define ACPI_ERROR(plist)
#define ACPI_ERROR_NAMESPACE(s, e)
#define ACPI_ERROR_METHOD(s, n, p, e)
-#endif
+#endif /* ACPI_NO_ERROR_MESSAGES */
/*
* Debug macros that are conditionally compiled
@@ -435,36 +359,8 @@
#ifdef ACPI_DEBUG_OUTPUT
/*
- * Common parameters used for debug output functions:
- * line number, function name, module(file) name, component ID
- */
-#define ACPI_DEBUG_PARAMETERS __LINE__, ACPI_GET_FUNCTION_NAME, _acpi_module_name, _COMPONENT
-
-/*
* Function entry tracing
*/
-
-/*
- * If ACPI_GET_FUNCTION_NAME was not defined in the compiler-dependent header,
- * define it now. This is the case where there the compiler does not support
- * a __func__ macro or equivalent.
- */
-#ifndef ACPI_GET_FUNCTION_NAME
-#define ACPI_GET_FUNCTION_NAME _acpi_function_name
-/*
- * The Name parameter should be the procedure name as a quoted string.
- * The function name is also used by the function exit macros below.
- * Note: (const char) is used to be compatible with the debug interfaces
- * and macros such as __func__.
- */
-#define ACPI_FUNCTION_NAME(name) static const char _acpi_function_name[] = #name;
-
-#else
-/* Compiler supports __func__ (or equivalent) -- Ignore this macro */
-
-#define ACPI_FUNCTION_NAME(name)
-#endif
-
#ifdef CONFIG_ACPI_DEBUG_FUNC_TRACE
#define ACPI_FUNCTION_TRACE(a) ACPI_FUNCTION_NAME(a) \
@@ -584,15 +480,6 @@
#define ACPI_DUMP_RESOURCE_LIST(a) acpi_rs_dump_resource_list(a)
#define ACPI_DUMP_BUFFER(a, b) acpi_ut_dump_buffer((u8 *) a, b, DB_BYTE_DISPLAY, _COMPONENT)
-/*
- * Master debug print macros
- * Print iff:
- * 1) Debug print for the current component is enabled
- * 2) Debug error level or trace level for the print statement is enabled
- */
-#define ACPI_DEBUG_PRINT(plist) acpi_ut_debug_print plist
-#define ACPI_DEBUG_PRINT_RAW(plist) acpi_ut_debug_print_raw plist
-
#else
/*
* This is the non-debug case -- make everything go away,
@@ -603,7 +490,6 @@
#define ACPI_DEBUG_DEFINE(a) do { } while(0)
#define ACPI_DEBUG_ONLY_MEMBERS(a) do { } while(0)
-#define ACPI_FUNCTION_NAME(a) do { } while(0)
#define ACPI_FUNCTION_TRACE(a) do { } while(0)
#define ACPI_FUNCTION_TRACE_PTR(a, b) do { } while(0)
#define ACPI_FUNCTION_TRACE_U32(a, b) do { } while(0)
@@ -619,8 +505,6 @@
#define ACPI_DUMP_PATHNAME(a, b, c, d) do { } while(0)
#define ACPI_DUMP_RESOURCE_LIST(a) do { } while(0)
#define ACPI_DUMP_BUFFER(a, b) do { } while(0)
-#define ACPI_DEBUG_PRINT(pl) do { } while(0)
-#define ACPI_DEBUG_PRINT_RAW(pl) do { } while(0)
#define return_VOID return
#define return_ACPI_STATUS(s) return(s)
@@ -629,7 +513,7 @@
#define return_UINT32(s) return(s)
#define return_PTR(s) return(s)
-#endif
+#endif /* ACPI_DEBUG_OUTPUT */
/*
* Some code only gets executed when the debugger is built in.
diff --git a/include/acpi/acnamesp.h b/drivers/acpi/acpica/acnamesp.h
similarity index 97%
rename from include/acpi/acnamesp.h
rename to drivers/acpi/acpica/acnamesp.h
index db4e6f6..46cb5b4 100644
--- a/include/acpi/acnamesp.h
+++ b/drivers/acpi/acpica/acnamesp.h
@@ -182,7 +182,9 @@
*/
acpi_status
acpi_ns_check_predefined_names(struct acpi_namespace_node *node,
- union acpi_operand_object *return_object);
+ u32 user_param_count,
+ acpi_status return_status,
+ union acpi_operand_object **return_object);
const union acpi_predefined_info *acpi_ns_check_for_predefined_name(struct
acpi_namespace_node
@@ -191,6 +193,7 @@
void
acpi_ns_check_parameter_count(char *pathname,
struct acpi_namespace_node *node,
+ u32 user_param_count,
const union acpi_predefined_info *info);
/*
diff --git a/include/acpi/acobject.h b/drivers/acpi/acpica/acobject.h
similarity index 100%
rename from include/acpi/acobject.h
rename to drivers/acpi/acpica/acobject.h
diff --git a/include/acpi/acopcode.h b/drivers/acpi/acpica/acopcode.h
similarity index 100%
rename from include/acpi/acopcode.h
rename to drivers/acpi/acpica/acopcode.h
diff --git a/include/acpi/acparser.h b/drivers/acpi/acpica/acparser.h
similarity index 100%
rename from include/acpi/acparser.h
rename to drivers/acpi/acpica/acparser.h
diff --git a/include/acpi/acpredef.h b/drivers/acpi/acpica/acpredef.h
similarity index 100%
rename from include/acpi/acpredef.h
rename to drivers/acpi/acpica/acpredef.h
diff --git a/include/acpi/acresrc.h b/drivers/acpi/acpica/acresrc.h
similarity index 100%
rename from include/acpi/acresrc.h
rename to drivers/acpi/acpica/acresrc.h
diff --git a/include/acpi/acstruct.h b/drivers/acpi/acpica/acstruct.h
similarity index 100%
rename from include/acpi/acstruct.h
rename to drivers/acpi/acpica/acstruct.h
diff --git a/include/acpi/actables.h b/drivers/acpi/acpica/actables.h
similarity index 98%
rename from include/acpi/actables.h
rename to drivers/acpi/acpica/actables.h
index 0cbe1b9..7ce6e33 100644
--- a/include/acpi/actables.h
+++ b/drivers/acpi/acpica/actables.h
@@ -94,6 +94,8 @@
/*
* tbutils - table manager utilities
*/
+acpi_status acpi_tb_initialize_facs(void);
+
u8 acpi_tb_tables_loaded(void);
void
diff --git a/include/acpi/acutils.h b/drivers/acpi/acpica/acutils.h
similarity index 93%
rename from include/acpi/acutils.h
rename to drivers/acpi/acpica/acutils.h
index d8307b2..80d8813 100644
--- a/include/acpi/acutils.h
+++ b/drivers/acpi/acpica/acutils.h
@@ -297,42 +297,6 @@
void acpi_ut_report_warning(char *module_name, u32 line_number);
-/* Error and message reporting interfaces */
-
-void ACPI_INTERNAL_VAR_XFACE
-acpi_ut_debug_print(u32 requested_debug_level,
- u32 line_number,
- const char *function_name,
- const char *module_name,
- u32 component_id,
- const char *format, ...) ACPI_PRINTF_LIKE(6);
-
-void ACPI_INTERNAL_VAR_XFACE
-acpi_ut_debug_print_raw(u32 requested_debug_level,
- u32 line_number,
- const char *function_name,
- const char *module_name,
- u32 component_id,
- const char *format, ...) ACPI_PRINTF_LIKE(6);
-
-void ACPI_INTERNAL_VAR_XFACE
-acpi_ut_error(const char *module_name,
- u32 line_number, const char *format, ...) ACPI_PRINTF_LIKE(3);
-
-void ACPI_INTERNAL_VAR_XFACE
-acpi_ut_exception(const char *module_name,
- u32 line_number,
- acpi_status status,
- const char *format, ...) ACPI_PRINTF_LIKE(4);
-
-void ACPI_INTERNAL_VAR_XFACE
-acpi_ut_warning(const char *module_name,
- u32 line_number, const char *format, ...) ACPI_PRINTF_LIKE(3);
-
-void ACPI_INTERNAL_VAR_XFACE
-acpi_ut_info(const char *module_name,
- u32 line_number, const char *format, ...) ACPI_PRINTF_LIKE(3);
-
/*
* utdelete - Object deletion and reference counts
*/
diff --git a/include/acpi/amlcode.h b/drivers/acpi/acpica/amlcode.h
similarity index 100%
rename from include/acpi/amlcode.h
rename to drivers/acpi/acpica/amlcode.h
diff --git a/include/acpi/amlresrc.h b/drivers/acpi/acpica/amlresrc.h
similarity index 100%
rename from include/acpi/amlresrc.h
rename to drivers/acpi/acpica/amlresrc.h
diff --git a/drivers/acpi/dispatcher/dsfield.c b/drivers/acpi/acpica/dsfield.c
similarity index 98%
rename from drivers/acpi/dispatcher/dsfield.c
rename to drivers/acpi/acpica/dsfield.c
index f988a5e..53e27bc 100644
--- a/drivers/acpi/dispatcher/dsfield.c
+++ b/drivers/acpi/acpica/dsfield.c
@@ -42,11 +42,12 @@
*/
#include <acpi/acpi.h>
-#include <acpi/amlcode.h>
-#include <acpi/acdispat.h>
-#include <acpi/acinterp.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acparser.h>
+#include "accommon.h"
+#include "amlcode.h"
+#include "acdispat.h"
+#include "acinterp.h"
+#include "acnamesp.h"
+#include "acparser.h"
#define _COMPONENT ACPI_DISPATCHER
ACPI_MODULE_NAME("dsfield")
diff --git a/drivers/acpi/dispatcher/dsinit.c b/drivers/acpi/acpica/dsinit.c
similarity index 98%
rename from drivers/acpi/dispatcher/dsinit.c
rename to drivers/acpi/acpica/dsinit.c
index 949f7c7..eb144b1 100644
--- a/drivers/acpi/dispatcher/dsinit.c
+++ b/drivers/acpi/acpica/dsinit.c
@@ -42,9 +42,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acdispat.h>
-#include <acpi/acnamesp.h>
-#include <acpi/actables.h>
+#include "accommon.h"
+#include "acdispat.h"
+#include "acnamesp.h"
+#include "actables.h"
#define _COMPONENT ACPI_DISPATCHER
ACPI_MODULE_NAME("dsinit")
diff --git a/drivers/acpi/dispatcher/dsmethod.c b/drivers/acpi/acpica/dsmethod.c
similarity index 98%
rename from drivers/acpi/dispatcher/dsmethod.c
rename to drivers/acpi/acpica/dsmethod.c
index 279a5a6..14b8b8e 100644
--- a/drivers/acpi/dispatcher/dsmethod.c
+++ b/drivers/acpi/acpica/dsmethod.c
@@ -42,11 +42,14 @@
*/
#include <acpi/acpi.h>
-#include <acpi/amlcode.h>
-#include <acpi/acdispat.h>
-#include <acpi/acinterp.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "amlcode.h"
+#include "acdispat.h"
+#include "acinterp.h"
+#include "acnamesp.h"
+#ifdef ACPI_DISASSEMBLER
#include <acpi/acdisasm.h>
+#endif
#define _COMPONENT ACPI_DISPATCHER
ACPI_MODULE_NAME("dsmethod")
@@ -412,6 +415,9 @@
if (obj_desc->method.method_flags & AML_METHOD_INTERNAL_ONLY) {
status = obj_desc->method.implementation(next_walk_state);
+ if (status == AE_OK) {
+ status = AE_CTRL_TERMINATE;
+ }
}
return_ACPI_STATUS(status);
diff --git a/drivers/acpi/dispatcher/dsmthdat.c b/drivers/acpi/acpica/dsmthdat.c
similarity index 99%
rename from drivers/acpi/dispatcher/dsmthdat.c
rename to drivers/acpi/acpica/dsmthdat.c
index d03f81b..da0f546 100644
--- a/drivers/acpi/dispatcher/dsmthdat.c
+++ b/drivers/acpi/acpica/dsmthdat.c
@@ -42,9 +42,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acdispat.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acinterp.h>
+#include "accommon.h"
+#include "acdispat.h"
+#include "acnamesp.h"
+#include "acinterp.h"
#define _COMPONENT ACPI_DISPATCHER
ACPI_MODULE_NAME("dsmthdat")
diff --git a/drivers/acpi/dispatcher/dsobject.c b/drivers/acpi/acpica/dsobject.c
similarity index 99%
rename from drivers/acpi/dispatcher/dsobject.c
rename to drivers/acpi/acpica/dsobject.c
index 4f08e59..15c628e 100644
--- a/drivers/acpi/dispatcher/dsobject.c
+++ b/drivers/acpi/acpica/dsobject.c
@@ -42,11 +42,12 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
-#include <acpi/amlcode.h>
-#include <acpi/acdispat.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acinterp.h>
+#include "accommon.h"
+#include "acparser.h"
+#include "amlcode.h"
+#include "acdispat.h"
+#include "acnamesp.h"
+#include "acinterp.h"
#define _COMPONENT ACPI_DISPATCHER
ACPI_MODULE_NAME("dsobject")
diff --git a/drivers/acpi/dispatcher/dsopcode.c b/drivers/acpi/acpica/dsopcode.c
similarity index 96%
rename from drivers/acpi/dispatcher/dsopcode.c
rename to drivers/acpi/acpica/dsopcode.c
index 69fae59..0c3b4dd 100644
--- a/drivers/acpi/dispatcher/dsopcode.c
+++ b/drivers/acpi/acpica/dsopcode.c
@@ -43,13 +43,14 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
-#include <acpi/amlcode.h>
-#include <acpi/acdispat.h>
-#include <acpi/acinterp.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acevents.h>
-#include <acpi/actables.h>
+#include "accommon.h"
+#include "acparser.h"
+#include "amlcode.h"
+#include "acdispat.h"
+#include "acinterp.h"
+#include "acnamesp.h"
+#include "acevents.h"
+#include "actables.h"
#define _COMPONENT ACPI_DISPATCHER
ACPI_MODULE_NAME("dsopcode")
@@ -1140,10 +1141,29 @@
op->common.aml_opcode, walk_state));
switch (op->common.aml_opcode) {
- case AML_IF_OP:
case AML_WHILE_OP:
/*
+ * If this is an additional iteration of a while loop, continue.
+ * There is no need to allocate a new control state.
+ */
+ if (walk_state->control_state) {
+ if (walk_state->control_state->control.aml_predicate_start
+ == (walk_state->parser_state.aml - 1)) {
+
+ /* Reset the state to start-of-loop */
+
+ walk_state->control_state->common.state =
+ ACPI_CONTROL_CONDITIONAL_EXECUTING;
+ break;
+ }
+ }
+
+ /*lint -fallthrough */
+
+ case AML_IF_OP:
+
+ /*
* IF/WHILE: Create a new control state to manage these
* constructs. We need to manage these as a stack, in order
* to handle nesting.
@@ -1243,13 +1263,36 @@
ACPI_DEBUG_PRINT((ACPI_DB_DISPATCH, "[WHILE_OP] Op=%p\n", op));
- if (walk_state->control_state->common.value) {
+ control_state = walk_state->control_state;
+ if (control_state->common.value) {
- /* Predicate was true, go back and evaluate it again! */
+ /* Predicate was true, the body of the loop was just executed */
+ /*
+ * This loop counter mechanism allows the interpreter to escape
+ * possibly infinite loops. This can occur in poorly written AML
+ * when the hardware does not respond within a while loop and the
+ * loop does not implement a timeout.
+ */
+ control_state->control.loop_count++;
+ if (control_state->control.loop_count >
+ ACPI_MAX_LOOP_ITERATIONS) {
+ status = AE_AML_INFINITE_LOOP;
+ break;
+ }
+
+ /*
+ * Go back and evaluate the predicate and maybe execute the loop
+ * another time
+ */
status = AE_CTRL_PENDING;
+ walk_state->aml_last_while =
+ control_state->control.aml_predicate_start;
+ break;
}
+ /* Predicate was false, terminate this while loop */
+
ACPI_DEBUG_PRINT((ACPI_DB_DISPATCH,
"[WHILE_OP] termination! Op=%p\n", op));
@@ -1257,9 +1300,6 @@
control_state =
acpi_ut_pop_generic_state(&walk_state->control_state);
-
- walk_state->aml_last_while =
- control_state->control.aml_predicate_start;
acpi_ut_delete_generic_state(control_state);
break;
diff --git a/drivers/acpi/dispatcher/dsutils.c b/drivers/acpi/acpica/dsutils.c
similarity index 99%
rename from drivers/acpi/dispatcher/dsutils.c
rename to drivers/acpi/acpica/dsutils.c
index b398982..dabc23a 100644
--- a/drivers/acpi/dispatcher/dsutils.c
+++ b/drivers/acpi/acpica/dsutils.c
@@ -42,12 +42,13 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
-#include <acpi/amlcode.h>
-#include <acpi/acdispat.h>
-#include <acpi/acinterp.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acdebug.h>
+#include "accommon.h"
+#include "acparser.h"
+#include "amlcode.h"
+#include "acdispat.h"
+#include "acinterp.h"
+#include "acnamesp.h"
+#include "acdebug.h"
#define _COMPONENT ACPI_DISPATCHER
ACPI_MODULE_NAME("dsutils")
diff --git a/drivers/acpi/dispatcher/dswexec.c b/drivers/acpi/acpica/dswexec.c
similarity index 98%
rename from drivers/acpi/dispatcher/dswexec.c
rename to drivers/acpi/acpica/dswexec.c
index 396fe12..350e665 100644
--- a/drivers/acpi/dispatcher/dswexec.c
+++ b/drivers/acpi/acpica/dswexec.c
@@ -43,12 +43,13 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
-#include <acpi/amlcode.h>
-#include <acpi/acdispat.h>
-#include <acpi/acinterp.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acdebug.h>
+#include "accommon.h"
+#include "acparser.h"
+#include "amlcode.h"
+#include "acdispat.h"
+#include "acinterp.h"
+#include "acnamesp.h"
+#include "acdebug.h"
#define _COMPONENT ACPI_DISPATCHER
ACPI_MODULE_NAME("dswexec")
diff --git a/drivers/acpi/dispatcher/dswload.c b/drivers/acpi/acpica/dswload.c
similarity index 99%
rename from drivers/acpi/dispatcher/dswload.c
rename to drivers/acpi/acpica/dswload.c
index dff7a3e..3023cea 100644
--- a/drivers/acpi/dispatcher/dswload.c
+++ b/drivers/acpi/acpica/dswload.c
@@ -42,12 +42,13 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
-#include <acpi/amlcode.h>
-#include <acpi/acdispat.h>
-#include <acpi/acinterp.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acevents.h>
+#include "accommon.h"
+#include "acparser.h"
+#include "amlcode.h"
+#include "acdispat.h"
+#include "acinterp.h"
+#include "acnamesp.h"
+#include "acevents.h"
#ifdef ACPI_ASL_COMPILER
#include <acpi/acdisasm.h>
diff --git a/drivers/acpi/dispatcher/dswscope.c b/drivers/acpi/acpica/dswscope.c
similarity index 98%
rename from drivers/acpi/dispatcher/dswscope.c
rename to drivers/acpi/acpica/dswscope.c
index 9e60732..908645e 100644
--- a/drivers/acpi/dispatcher/dswscope.c
+++ b/drivers/acpi/acpica/dswscope.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acdispat.h>
+#include "accommon.h"
+#include "acdispat.h"
#define _COMPONENT ACPI_DISPATCHER
ACPI_MODULE_NAME("dswscope")
diff --git a/drivers/acpi/dispatcher/dswstate.c b/drivers/acpi/acpica/dswstate.c
similarity index 99%
rename from drivers/acpi/dispatcher/dswstate.c
rename to drivers/acpi/acpica/dswstate.c
index b00d4af..40f92bf 100644
--- a/drivers/acpi/dispatcher/dswstate.c
+++ b/drivers/acpi/acpica/dswstate.c
@@ -42,9 +42,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
-#include <acpi/acdispat.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acparser.h"
+#include "acdispat.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_DISPATCHER
ACPI_MODULE_NAME("dswstate")
diff --git a/drivers/acpi/events/evevent.c b/drivers/acpi/acpica/evevent.c
similarity index 96%
rename from drivers/acpi/events/evevent.c
rename to drivers/acpi/acpica/evevent.c
index c56c5c6..803edd9 100644
--- a/drivers/acpi/events/evevent.c
+++ b/drivers/acpi/acpica/evevent.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acevents.h>
+#include "accommon.h"
+#include "acevents.h"
#define _COMPONENT ACPI_EVENTS
ACPI_MODULE_NAME("evevent")
@@ -72,8 +73,8 @@
/*
* Initialize the Fixed and General Purpose Events. This is done prior to
- * enabling SCIs to prevent interrupts from occurring before the handlers are
- * installed.
+ * enabling SCIs to prevent interrupts from occurring before the handlers
+ * are installed.
*/
status = acpi_ev_fixed_event_initialize();
if (ACPI_FAILURE(status)) {
@@ -192,8 +193,8 @@
acpi_status status;
/*
- * Initialize the structure that keeps track of fixed event handlers
- * and enable the fixed events.
+ * Initialize the structure that keeps track of fixed event handlers and
+ * enable the fixed events.
*/
for (i = 0; i < ACPI_NUM_FIXED_EVENTS; i++) {
acpi_gbl_fixed_event_handlers[i].handler = NULL;
@@ -237,7 +238,7 @@
/*
* Read the fixed feature status and enable registers, as all the cases
- * depend on their values. Ignore errors here.
+ * depend on their values. Ignore errors here.
*/
(void)acpi_hw_register_read(ACPI_REGISTER_PM1_STATUS, &fixed_status);
(void)acpi_hw_register_read(ACPI_REGISTER_PM1_ENABLE, &fixed_enable);
@@ -291,8 +292,8 @@
status_register_id, 1);
/*
- * Make sure we've got a handler. If not, report an error.
- * The event is disabled to prevent further interrupts.
+ * Make sure we've got a handler. If not, report an error. The event is
+ * disabled to prevent further interrupts.
*/
if (NULL == acpi_gbl_fixed_event_handlers[event].handler) {
(void)acpi_set_register(acpi_gbl_fixed_event_info[event].
diff --git a/drivers/acpi/events/evgpe.c b/drivers/acpi/acpica/evgpe.c
similarity index 94%
rename from drivers/acpi/events/evgpe.c
rename to drivers/acpi/acpica/evgpe.c
index f45c74f..f345ced 100644
--- a/drivers/acpi/events/evgpe.c
+++ b/drivers/acpi/acpica/evgpe.c
@@ -42,8 +42,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acevents.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acevents.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_EVENTS
ACPI_MODULE_NAME("evgpe")
@@ -125,7 +126,7 @@
(1 <<
(gpe_event_info->gpe_number - gpe_register_info->base_gpe_number));
- /* 1) Disable case. Simply clear all enable bits */
+ /* 1) Disable case. Simply clear all enable bits */
if (type == ACPI_GPE_DISABLE) {
ACPI_CLEAR_BIT(gpe_register_info->enable_for_wake,
@@ -134,7 +135,7 @@
return_ACPI_STATUS(AE_OK);
}
- /* 2) Enable case. Set/Clear the appropriate enable bits */
+ /* 2) Enable case. Set/Clear the appropriate enable bits */
switch (gpe_event_info->flags & ACPI_GPE_TYPE_MASK) {
case ACPI_GPE_TYPE_WAKE:
@@ -295,7 +296,7 @@
*
* FUNCTION: acpi_ev_get_gpe_event_info
*
- * PARAMETERS: gpe_device - Device node. NULL for GPE0/GPE1
+ * PARAMETERS: gpe_device - Device node. NULL for GPE0/GPE1
* gpe_number - Raw GPE number
*
* RETURN: A GPE event_info struct. NULL if not a valid GPE
@@ -372,7 +373,7 @@
*
* RETURN: INTERRUPT_HANDLED or INTERRUPT_NOT_HANDLED
*
- * DESCRIPTION: Detect if any GP events have occurred. This function is
+ * DESCRIPTION: Detect if any GP events have occurred. This function is
* executed at interrupt level.
*
******************************************************************************/
@@ -400,8 +401,8 @@
/*
* We need to obtain the GPE lock for both the data structs and registers
- * Note: Not necessary to obtain the hardware lock, since the GPE registers
- * are owned by the gpe_lock.
+ * Note: Not necessary to obtain the hardware lock, since the GPE
+ * registers are owned by the gpe_lock.
*/
flags = acpi_os_acquire_lock(acpi_gbl_gpe_lock);
@@ -410,9 +411,8 @@
gpe_block = gpe_xrupt_list->gpe_block_list_head;
while (gpe_block) {
/*
- * Read all of the 8-bit GPE status and enable registers
- * in this GPE block, saving all of them.
- * Find all currently active GP events.
+ * Read all of the 8-bit GPE status and enable registers in this GPE
+ * block, saving all of them. Find all currently active GP events.
*/
for (i = 0; i < gpe_block->register_count; i++) {
@@ -423,10 +423,8 @@
/* Read the Status Register */
status =
- acpi_hw_low_level_read(ACPI_GPE_REGISTER_WIDTH,
- &status_reg,
- &gpe_register_info->
- status_address);
+ acpi_read(&status_reg,
+ &gpe_register_info->status_address);
if (ACPI_FAILURE(status)) {
goto unlock_and_exit;
}
@@ -434,10 +432,8 @@
/* Read the Enable Register */
status =
- acpi_hw_low_level_read(ACPI_GPE_REGISTER_WIDTH,
- &enable_reg,
- &gpe_register_info->
- enable_address);
+ acpi_read(&enable_reg,
+ &gpe_register_info->enable_address);
if (ACPI_FAILURE(status)) {
goto unlock_and_exit;
}
@@ -527,8 +523,8 @@
(void)acpi_ev_enable_gpe(gpe_event_info, FALSE);
/*
- * Take a snapshot of the GPE info for this level - we copy the
- * info to prevent a race condition with remove_handler/remove_block.
+ * Take a snapshot of the GPE info for this level - we copy the info to
+ * prevent a race condition with remove_handler/remove_block.
*/
ACPI_MEMCPY(&local_gpe_event_info, gpe_event_info,
sizeof(struct acpi_gpe_event_info));
@@ -539,8 +535,8 @@
}
/*
- * Must check for control method type dispatch one more
- * time to avoid race with ev_gpe_install_handler
+ * Must check for control method type dispatch one more time to avoid a
+ * race with ev_gpe_install_handler
*/
if ((local_gpe_event_info.flags & ACPI_GPE_DISPATCH_MASK) ==
ACPI_GPE_DISPATCH_METHOD) {
@@ -584,8 +580,8 @@
if ((gpe_event_info->flags & ACPI_GPE_XRUPT_TYPE_MASK) ==
ACPI_GPE_LEVEL_TRIGGERED) {
/*
- * GPE is level-triggered, we clear the GPE status bit after
- * handling the event.
+ * GPE is level-triggered, we clear the GPE status bit after handling
+ * the event.
*/
status = acpi_hw_clear_gpe(gpe_event_info);
if (ACPI_FAILURE(status)) {
@@ -624,7 +620,7 @@
acpi_os_gpe_count(gpe_number);
/*
- * If edge-triggered, clear the GPE status bit now. Note that
+ * If edge-triggered, clear the GPE status bit now. Note that
* level-triggered events are cleared after the GPE is serviced.
*/
if ((gpe_event_info->flags & ACPI_GPE_XRUPT_TYPE_MASK) ==
@@ -650,7 +646,8 @@
/*
* Invoke the installed handler (at interrupt level)
- * Ignore return status for now. TBD: leave GPE disabled on error?
+ * Ignore return status for now.
+ * TBD: leave GPE disabled on error?
*/
(void)gpe_event_info->dispatch.handler->address(gpe_event_info->
dispatch.
@@ -708,7 +705,7 @@
gpe_number));
/*
- * Disable the GPE. The GPE will remain disabled until the ACPI
+ * Disable the GPE. The GPE will remain disabled until the ACPICA
* Core Subsystem is restarted, or a handler is installed.
*/
status = acpi_ev_disable_gpe(gpe_event_info);
diff --git a/drivers/acpi/events/evgpeblk.c b/drivers/acpi/acpica/evgpeblk.c
similarity index 93%
rename from drivers/acpi/events/evgpeblk.c
rename to drivers/acpi/acpica/evgpeblk.c
index 73c058e..484cc05 100644
--- a/drivers/acpi/events/evgpeblk.c
+++ b/drivers/acpi/acpica/evgpeblk.c
@@ -42,8 +42,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acevents.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acevents.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_EVENTS
ACPI_MODULE_NAME("evgpeblk")
@@ -124,6 +125,7 @@
* FUNCTION: acpi_ev_walk_gpe_list
*
* PARAMETERS: gpe_walk_callback - Routine called for each GPE block
+ * Context - Value passed to callback
*
* RETURN: Status
*
@@ -131,7 +133,8 @@
*
******************************************************************************/
-acpi_status acpi_ev_walk_gpe_list(acpi_gpe_callback gpe_walk_callback)
+acpi_status
+acpi_ev_walk_gpe_list(acpi_gpe_callback gpe_walk_callback, void *context)
{
struct acpi_gpe_block_info *gpe_block;
struct acpi_gpe_xrupt_info *gpe_xrupt_info;
@@ -154,8 +157,13 @@
/* One callback per GPE block */
- status = gpe_walk_callback(gpe_xrupt_info, gpe_block);
+ status =
+ gpe_walk_callback(gpe_xrupt_info, gpe_block,
+ context);
if (ACPI_FAILURE(status)) {
+ if (status == AE_CTRL_END) { /* Callback abort */
+ status = AE_OK;
+ }
goto unlock_and_exit;
}
@@ -186,7 +194,8 @@
acpi_status
acpi_ev_delete_gpe_handlers(struct acpi_gpe_xrupt_info *gpe_xrupt_info,
- struct acpi_gpe_block_info *gpe_block)
+ struct acpi_gpe_block_info *gpe_block,
+ void *context)
{
struct acpi_gpe_event_info *gpe_event_info;
u32 i;
@@ -309,17 +318,17 @@
(gpe_block->block_base_number +
(gpe_block->register_count * 8)))) {
/*
- * Not valid for this GPE block, just ignore it
- * However, it may be valid for a different GPE block, since GPE0 and GPE1
- * methods both appear under \_GPE.
+ * Not valid for this GPE block, just ignore it. However, it may be
+ * valid for a different GPE block, since GPE0 and GPE1 methods both
+ * appear under \_GPE.
*/
return_ACPI_STATUS(AE_OK);
}
/*
- * Now we can add this information to the gpe_event_info block
- * for use during dispatch of this GPE. Default type is RUNTIME, although
- * this may change when the _PRW methods are executed later.
+ * Now we can add this information to the gpe_event_info block for use
+ * during dispatch of this GPE. Default type is RUNTIME, although this may
+ * change when the _PRW methods are executed later.
*/
gpe_event_info =
&gpe_block->event_info[gpe_number - gpe_block->block_base_number];
@@ -394,8 +403,8 @@
gpe_block = gpe_info->gpe_block;
/*
- * The _PRW object must return a package, we are only interested
- * in the first element
+ * The _PRW object must return a package, we are only interested in the
+ * first element
*/
obj_desc = pkg_desc->package.elements[0];
@@ -434,7 +443,7 @@
/*
* Is this GPE within this block?
*
- * TRUE iff these conditions are true:
+ * TRUE if and only if these conditions are true:
* 1) The GPE devices match.
* 2) The GPE index(number) is within the range of the Gpe Block
* associated with the GPE device.
@@ -457,6 +466,7 @@
if (ACPI_FAILURE(status)) {
goto cleanup;
}
+
status =
acpi_ev_update_gpe_enable_masks(gpe_event_info,
ACPI_GPE_DISABLE);
@@ -476,9 +486,9 @@
* RETURN: A GPE interrupt block
*
* DESCRIPTION: Get or Create a GPE interrupt block. There is one interrupt
- * block per unique interrupt level used for GPEs.
- * Should be called only when the GPE lists are semaphore locked
- * and not subject to change.
+ * block per unique interrupt level used for GPEs. Should be
+ * called only when the GPE lists are semaphore locked and not
+ * subject to change.
*
******************************************************************************/
@@ -608,8 +618,9 @@
*
* FUNCTION: acpi_ev_install_gpe_block
*
- * PARAMETERS: gpe_block - New GPE block
- * interrupt_number - Xrupt to be associated with this GPE block
+ * PARAMETERS: gpe_block - New GPE block
+ * interrupt_number - Xrupt to be associated with this
+ * GPE block
*
* RETURN: Status
*
@@ -666,7 +677,7 @@
*
* FUNCTION: acpi_ev_delete_gpe_block
*
- * PARAMETERS: gpe_block - Existing GPE block
+ * PARAMETERS: gpe_block - Existing GPE block
*
* RETURN: Status
*
@@ -688,7 +699,8 @@
/* Disable all GPEs in this block */
- status = acpi_hw_disable_gpe_block(gpe_block->xrupt_block, gpe_block);
+ status =
+ acpi_hw_disable_gpe_block(gpe_block->xrupt_block, gpe_block, NULL);
if (!gpe_block->previous && !gpe_block->next) {
@@ -715,6 +727,9 @@
acpi_os_release_lock(acpi_gbl_gpe_lock, flags);
}
+ acpi_current_gpe_count -=
+ gpe_block->register_count * ACPI_GPE_REGISTER_WIDTH;
+
/* Free the gpe_block */
ACPI_FREE(gpe_block->register_info);
@@ -786,9 +801,9 @@
/*
* Initialize the GPE Register and Event structures. A goal of these
- * tables is to hide the fact that there are two separate GPE register sets
- * in a given GPE hardware block, the status registers occupy the first half,
- * and the enable registers occupy the second half.
+ * tables is to hide the fact that there are two separate GPE register
+ * sets in a given GPE hardware block, the status registers occupy the
+ * first half, and the enable registers occupy the second half.
*/
this_register = gpe_register_info;
this_event = gpe_event_info;
@@ -816,10 +831,8 @@
ACPI_GPE_REGISTER_WIDTH;
this_register->enable_address.bit_width =
ACPI_GPE_REGISTER_WIDTH;
- this_register->status_address.bit_offset =
- ACPI_GPE_REGISTER_WIDTH;
- this_register->enable_address.bit_offset =
- ACPI_GPE_REGISTER_WIDTH;
+ this_register->status_address.bit_offset = 0;
+ this_register->enable_address.bit_offset = 0;
/* Init the event_info for each GPE within this register */
@@ -832,18 +845,14 @@
/* Disable all GPEs within this register */
- status = acpi_hw_low_level_write(ACPI_GPE_REGISTER_WIDTH, 0x00,
- &this_register->
- enable_address);
+ status = acpi_write(0x00, &this_register->enable_address);
if (ACPI_FAILURE(status)) {
goto error_exit;
}
/* Clear any pending GPE events within this register */
- status = acpi_hw_low_level_write(ACPI_GPE_REGISTER_WIDTH, 0xFF,
- &this_register->
- status_address);
+ status = acpi_write(0xFF, &this_register->status_address);
if (ACPI_FAILURE(status)) {
goto error_exit;
}
@@ -956,6 +965,9 @@
gpe_device->name.ascii, gpe_block->register_count,
interrupt_number));
+ /* Update global count of currently available GPEs */
+
+ acpi_current_gpe_count += register_count * ACPI_GPE_REGISTER_WIDTH;
return_ACPI_STATUS(AE_OK);
}
@@ -1055,7 +1067,7 @@
/* Enable all valid runtime GPEs found above */
- status = acpi_hw_enable_runtime_gpe_block(NULL, gpe_block);
+ status = acpi_hw_enable_runtime_gpe_block(NULL, gpe_block, NULL);
if (ACPI_FAILURE(status)) {
ACPI_ERROR((AE_INFO, "Could not enable GPEs in GpeBlock %p",
gpe_block));
diff --git a/drivers/acpi/events/evmisc.c b/drivers/acpi/acpica/evmisc.c
similarity index 92%
rename from drivers/acpi/events/evmisc.c
rename to drivers/acpi/acpica/evmisc.c
index 1d5670b..5f89305 100644
--- a/drivers/acpi/events/evmisc.c
+++ b/drivers/acpi/acpica/evmisc.c
@@ -42,18 +42,15 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acevents.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acinterp.h>
+#include "accommon.h"
+#include "acevents.h"
+#include "acnamesp.h"
+#include "acinterp.h"
#define _COMPONENT ACPI_EVENTS
ACPI_MODULE_NAME("evmisc")
-/* Pointer to FACS needed for the Global Lock */
-static struct acpi_table_facs *facs = NULL;
-
/* Local prototypes */
-
static void ACPI_SYSTEM_XFACE acpi_ev_notify_dispatch(void *context);
static u32 acpi_ev_global_lock_handler(void *context);
@@ -152,7 +149,9 @@
break;
default:
+
/* All other types are not supported */
+
return (AE_TYPE);
}
}
@@ -193,9 +192,8 @@
acpi_ut_delete_generic_state(notify_info);
}
} else {
- /*
- * There is no notify handler (per-device or system) for this device.
- */
+ /* There is no notify handler (per-device or system) for this device */
+
ACPI_DEBUG_PRINT((ACPI_DB_INFO,
"No notify handler for Notify (%4.4s, %X) node %p\n",
acpi_ut_get_node_name(node), notify_value,
@@ -229,9 +227,8 @@
ACPI_FUNCTION_ENTRY();
/*
- * We will invoke a global notify handler if installed.
- * This is done _before_ we invoke the per-device handler attached
- * to the device.
+ * We will invoke a global notify handler if installed. This is done
+ * _before_ we invoke the per-device handler attached to the device.
*/
if (notify_info->notify.value <= ACPI_MAX_SYS_NOTIFY) {
@@ -299,7 +296,7 @@
* If we don't get it now, it will be marked pending and we will
* take another interrupt when it becomes free.
*/
- ACPI_ACQUIRE_GLOBAL_LOCK(facs, acquired);
+ ACPI_ACQUIRE_GLOBAL_LOCK(acpi_gbl_FACS, acquired);
if (acquired) {
/* Got the lock, now wake all threads waiting for it */
@@ -336,34 +333,27 @@
ACPI_FUNCTION_TRACE(ev_init_global_lock_handler);
- status = acpi_get_table_by_index(ACPI_TABLE_INDEX_FACS,
- ACPI_CAST_INDIRECT_PTR(struct
- acpi_table_header,
- &facs));
- if (ACPI_FAILURE(status)) {
- return_ACPI_STATUS(status);
- }
+ /* Attempt installation of the global lock handler */
- acpi_gbl_global_lock_present = TRUE;
status = acpi_install_fixed_event_handler(ACPI_EVENT_GLOBAL,
acpi_ev_global_lock_handler,
NULL);
/*
- * If the global lock does not exist on this platform, the attempt
- * to enable GBL_STATUS will fail (the GBL_ENABLE bit will not stick)
- * Map to AE_OK, but mark global lock as not present.
- * Any attempt to actually use the global lock will be flagged
- * with an error.
+ * If the global lock does not exist on this platform, the attempt to
+ * enable GBL_STATUS will fail (the GBL_ENABLE bit will not stick).
+ * Map to AE_OK, but mark global lock as not present. Any attempt to
+ * actually use the global lock will be flagged with an error.
*/
if (status == AE_NO_HARDWARE_RESPONSE) {
ACPI_ERROR((AE_INFO,
"No response from Global Lock hardware, disabling lock"));
acpi_gbl_global_lock_present = FALSE;
- status = AE_OK;
+ return_ACPI_STATUS(AE_OK);
}
+ acpi_gbl_global_lock_present = TRUE;
return_ACPI_STATUS(status);
}
@@ -462,8 +452,8 @@
}
/*
- * Make sure that a global lock actually exists. If not, just treat
- * the lock as a standard mutex.
+ * Make sure that a global lock actually exists. If not, just treat the
+ * lock as a standard mutex.
*/
if (!acpi_gbl_global_lock_present) {
acpi_gbl_global_lock_acquired = TRUE;
@@ -472,7 +462,7 @@
/* Attempt to acquire the actual hardware lock */
- ACPI_ACQUIRE_GLOBAL_LOCK(facs, acquired);
+ ACPI_ACQUIRE_GLOBAL_LOCK(acpi_gbl_FACS, acquired);
if (acquired) {
/* We got the lock */
@@ -536,7 +526,7 @@
/* Allow any thread to release the lock */
- ACPI_RELEASE_GLOBAL_LOCK(facs, pending);
+ ACPI_RELEASE_GLOBAL_LOCK(acpi_gbl_FACS, pending);
/*
* If the pending bit was set, we must write GBL_RLS to the control
@@ -582,8 +572,8 @@
if (acpi_gbl_events_initialized) {
/*
- * Disable all event-related functionality.
- * In all cases, on error, print a message but obviously we don't abort.
+ * Disable all event-related functionality. In all cases, on error,
+ * print a message but obviously we don't abort.
*/
/* Disable all fixed events */
@@ -599,7 +589,7 @@
/* Disable all GPEs in all GPE blocks */
- status = acpi_ev_walk_gpe_list(acpi_hw_disable_gpe_block);
+ status = acpi_ev_walk_gpe_list(acpi_hw_disable_gpe_block, NULL);
/* Remove SCI handler */
@@ -617,7 +607,7 @@
/* Deallocate all handler objects installed within GPE info structs */
- status = acpi_ev_walk_gpe_list(acpi_ev_delete_gpe_handlers);
+ status = acpi_ev_walk_gpe_list(acpi_ev_delete_gpe_handlers, NULL);
/* Return to original mode if necessary */
diff --git a/drivers/acpi/events/evregion.c b/drivers/acpi/acpica/evregion.c
similarity index 91%
rename from drivers/acpi/events/evregion.c
rename to drivers/acpi/acpica/evregion.c
index 236fbd1..665c088 100644
--- a/drivers/acpi/events/evregion.c
+++ b/drivers/acpi/acpica/evregion.c
@@ -42,22 +42,15 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acevents.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acinterp.h>
+#include "accommon.h"
+#include "acevents.h"
+#include "acnamesp.h"
+#include "acinterp.h"
#define _COMPONENT ACPI_EVENTS
ACPI_MODULE_NAME("evregion")
-#define ACPI_NUM_DEFAULT_SPACES 4
-static u8 acpi_gbl_default_address_spaces[ACPI_NUM_DEFAULT_SPACES] = {
- ACPI_ADR_SPACE_SYSTEM_MEMORY,
- ACPI_ADR_SPACE_SYSTEM_IO,
- ACPI_ADR_SPACE_PCI_CONFIG,
- ACPI_ADR_SPACE_DATA_TABLE
-};
/* Local prototypes */
-
static acpi_status
acpi_ev_reg_run(acpi_handle obj_handle,
u32 level, void *context, void **return_value);
@@ -66,6 +59,17 @@
acpi_ev_install_handler(acpi_handle obj_handle,
u32 level, void *context, void **return_value);
+/* These are the address spaces that will get default handlers */
+
+#define ACPI_NUM_DEFAULT_SPACES 4
+
+static u8 acpi_gbl_default_address_spaces[ACPI_NUM_DEFAULT_SPACES] = {
+ ACPI_ADR_SPACE_SYSTEM_MEMORY,
+ ACPI_ADR_SPACE_SYSTEM_IO,
+ ACPI_ADR_SPACE_PCI_CONFIG,
+ ACPI_ADR_SPACE_DATA_TABLE
+};
+
/*******************************************************************************
*
* FUNCTION: acpi_ev_install_region_handlers
@@ -91,18 +95,19 @@
}
/*
- * All address spaces (PCI Config, EC, SMBus) are scope dependent
- * and registration must occur for a specific device.
+ * All address spaces (PCI Config, EC, SMBus) are scope dependent and
+ * registration must occur for a specific device.
*
- * In the case of the system memory and IO address spaces there is currently
- * no device associated with the address space. For these we use the root.
+ * In the case of the system memory and IO address spaces there is
+ * currently no device associated with the address space. For these we
+ * use the root.
*
- * We install the default PCI config space handler at the root so
- * that this space is immediately available even though the we have
- * not enumerated all the PCI Root Buses yet. This is to conform
- * to the ACPI specification which states that the PCI config
- * space must be always available -- even though we are nowhere
- * near ready to find the PCI root buses at this point.
+ * We install the default PCI config space handler at the root so that
+ * this space is immediately available even though the we have not
+ * enumerated all the PCI Root Buses yet. This is to conform to the ACPI
+ * specification which states that the PCI config space must be always
+ * available -- even though we are nowhere near ready to find the PCI root
+ * buses at this point.
*
* NOTE: We ignore AE_ALREADY_EXISTS because this means that a handler
* has already been installed (via acpi_install_address_space_handler).
@@ -160,12 +165,11 @@
return_ACPI_STATUS(status);
}
- /*
- * Run the _REG methods for op_regions in each default address space
- */
- for (i = 0; i < ACPI_NUM_DEFAULT_SPACES; i++) {
+ /* Run the _REG methods for op_regions in each default address space */
- /* TBD: Make sure handler is the DEFAULT handler, otherwise
+ for (i = 0; i < ACPI_NUM_DEFAULT_SPACES; i++) {
+ /*
+ * TBD: Make sure handler is the DEFAULT handler, otherwise
* _REG will have already been run.
*/
status = acpi_ev_execute_reg_methods(acpi_gbl_root_node,
@@ -318,13 +322,13 @@
}
/*
- * It may be the case that the region has never been initialized
+ * It may be the case that the region has never been initialized.
* Some types of regions require special init code
*/
if (!(region_obj->region.flags & AOPOBJ_SETUP_COMPLETE)) {
- /*
- * This region has not been initialized yet, do it
- */
+
+ /* This region has not been initialized yet, do it */
+
region_setup = handler_desc->address_space.setup;
if (!region_setup) {
@@ -339,9 +343,9 @@
}
/*
- * We must exit the interpreter because the region
- * setup will potentially execute control methods
- * (e.g., _REG method for this region)
+ * We must exit the interpreter because the region setup will
+ * potentially execute control methods (for example, the _REG method
+ * for this region)
*/
acpi_ex_exit_interpreter();
@@ -364,9 +368,8 @@
return_ACPI_STATUS(status);
}
- /*
- * Region initialization may have been completed by region_setup
- */
+ /* Region initialization may have been completed by region_setup */
+
if (!(region_obj->region.flags & AOPOBJ_SETUP_COMPLETE)) {
region_obj->region.flags |= AOPOBJ_SETUP_COMPLETE;
@@ -521,8 +524,8 @@
}
/*
- * If the region has been activated, call the setup handler
- * with the deactivate notification
+ * If the region has been activated, call the setup handler with
+ * the deactivate notification
*/
if (region_obj->region.flags & AOPOBJ_SETUP_COMPLETE) {
region_setup = handler_obj->address_space.setup;
@@ -668,8 +671,8 @@
}
/*
- * We only care about regions.and objects
- * that are allowed to have address space handlers
+ * We only care about regions and objects that are allowed to have
+ * address space handlers
*/
if ((node->type != ACPI_TYPE_DEVICE) &&
(node->type != ACPI_TYPE_REGION) && (node != acpi_gbl_root_node)) {
@@ -710,9 +713,9 @@
/*
* Since the object we found it on was a device, then it
* means that someone has already installed a handler for
- * the branch of the namespace from this device on. Just
+ * the branch of the namespace from this device on. Just
* bail out telling the walk routine to not traverse this
- * branch. This preserves the scoping rule for handlers.
+ * branch. This preserves the scoping rule for handlers.
*/
return (AE_CTRL_DEPTH);
}
@@ -723,9 +726,8 @@
}
/*
- * As long as the device didn't have a handler for this
- * space we don't care about it. We just ignore it and
- * proceed.
+ * As long as the device didn't have a handler for this space we
+ * don't care about it. We just ignore it and proceed.
*/
return (AE_OK);
}
@@ -733,16 +735,14 @@
/* Object is a Region */
if (obj_desc->region.space_id != handler_obj->address_space.space_id) {
- /*
- * This region is for a different address space
- * -- just ignore it
- */
+
+ /* This region is for a different address space, just ignore it */
+
return (AE_OK);
}
/*
- * Now we have a region and it is for the handler's address
- * space type.
+ * Now we have a region and it is for the handler's address space type.
*
* First disconnect region for any previous handler (if any)
*/
@@ -786,9 +786,8 @@
ACPI_FUNCTION_TRACE(ev_install_space_handler);
/*
- * This registration is valid for only the types below
- * and the root. This is where the default handlers
- * get placed.
+ * This registration is valid for only the types below and the root. This
+ * is where the default handlers get placed.
*/
if ((node->type != ACPI_TYPE_DEVICE) &&
(node->type != ACPI_TYPE_PROCESSOR) &&
@@ -848,8 +847,8 @@
obj_desc = acpi_ns_get_attached_object(node);
if (obj_desc) {
/*
- * The attached device object already exists.
- * Make sure the handler is not already installed.
+ * The attached device object already exists. Make sure the handler
+ * is not already installed.
*/
handler_obj = obj_desc->device.handler;
@@ -864,8 +863,8 @@
handler) {
/*
* It is (relatively) OK to attempt to install the SAME
- * handler twice. This can easily happen
- * with PCI_Config space.
+ * handler twice. This can easily happen with the
+ * PCI_Config space.
*/
status = AE_SAME_HANDLER;
goto unlock_and_exit;
@@ -925,9 +924,8 @@
/*
* Install the handler
*
- * At this point there is no existing handler.
- * Just allocate the object for the handler and link it
- * into the list.
+ * At this point there is no existing handler. Just allocate the object
+ * for the handler and link it into the list.
*/
handler_obj =
acpi_ut_create_internal_object(ACPI_TYPE_LOCAL_ADDRESS_HANDLER);
@@ -1000,11 +998,10 @@
ACPI_FUNCTION_TRACE(ev_execute_reg_methods);
/*
- * Run all _REG methods for all Operation Regions for this
- * space ID. This is a separate walk in order to handle any
- * interdependencies between regions and _REG methods. (i.e. handlers
- * must be installed for all regions of this Space ID before we
- * can run any _REG methods)
+ * Run all _REG methods for all Operation Regions for this space ID. This
+ * is a separate walk in order to handle any interdependencies between
+ * regions and _REG methods. (i.e. handlers must be installed for all
+ * regions of this Space ID before we can run any _REG methods)
*/
status = acpi_ns_walk_namespace(ACPI_TYPE_ANY, node, ACPI_UINT32_MAX,
ACPI_NS_WALK_UNLOCK, acpi_ev_reg_run,
@@ -1042,8 +1039,8 @@
}
/*
- * We only care about regions.and objects
- * that are allowed to have address space handlers
+ * We only care about regions.and objects that are allowed to have address
+ * space handlers
*/
if ((node->type != ACPI_TYPE_REGION) && (node != acpi_gbl_root_node)) {
return (AE_OK);
@@ -1062,10 +1059,9 @@
/* Object is a Region */
if (obj_desc->region.space_id != space_id) {
- /*
- * This region is for a different address space
- * -- just ignore it
- */
+
+ /* This region is for a different address space, just ignore it */
+
return (AE_OK);
}
diff --git a/drivers/acpi/events/evrgnini.c b/drivers/acpi/acpica/evrgnini.c
similarity index 95%
rename from drivers/acpi/events/evrgnini.c
rename to drivers/acpi/acpica/evrgnini.c
index 6b94b38..f3f1fb4 100644
--- a/drivers/acpi/events/evrgnini.c
+++ b/drivers/acpi/acpica/evrgnini.c
@@ -42,8 +42,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acevents.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acevents.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_EVENTS
ACPI_MODULE_NAME("evrgnini")
@@ -233,9 +234,9 @@
if (ACPI_FAILURE(status)) {
if (status == AE_SAME_HANDLER) {
/*
- * It is OK if the handler is already installed on the root
- * bridge. Still need to return a context object for the
- * new PCI_Config operation region, however.
+ * It is OK if the handler is already installed on the
+ * root bridge. Still need to return a context object
+ * for the new PCI_Config operation region, however.
*/
status = AE_OK;
} else {
@@ -272,8 +273,8 @@
}
/*
- * For PCI_Config space access, we need the segment, bus,
- * device and function numbers. Acquire them here.
+ * For PCI_Config space access, we need the segment, bus, device and
+ * function numbers. Acquire them here.
*
* Find the parent device object. (This allows the operation region to be
* within a subscope under the device, such as a control method.)
@@ -289,16 +290,16 @@
}
/*
- * Get the PCI device and function numbers from the _ADR object
- * contained in the parent's scope.
+ * Get the PCI device and function numbers from the _ADR object contained
+ * in the parent's scope.
*/
status =
acpi_ut_evaluate_numeric_object(METHOD_NAME__ADR, pci_device_node,
&pci_value);
/*
- * The default is zero, and since the allocation above zeroed
- * the data, just do nothing on failure.
+ * The default is zero, and since the allocation above zeroed the data,
+ * just do nothing on failure.
*/
if (ACPI_SUCCESS(status)) {
pci_id->device = ACPI_HIWORD(ACPI_LODWORD(pci_value));
@@ -382,9 +383,8 @@
struct acpi_compatible_id_list *cid;
u32 i;
- /*
- * Get the _HID and check for a PCI Root Bridge
- */
+ /* Get the _HID and check for a PCI Root Bridge */
+
status = acpi_ut_execute_HID(node, &hid);
if (ACPI_FAILURE(status)) {
return (FALSE);
@@ -394,10 +394,8 @@
return (TRUE);
}
- /*
- * The _HID did not match.
- * Get the _CID and check for a PCI Root Bridge
- */
+ /* The _HID did not match. Get the _CID and check for a PCI Root Bridge */
+
status = acpi_ut_execute_CID(node, &cid);
if (ACPI_FAILURE(status)) {
return (FALSE);
@@ -516,9 +514,9 @@
* Get the appropriate address space handler for a newly
* created region.
*
- * This also performs address space specific initialization. For
+ * This also performs address space specific initialization. For
* example, PCI regions must have an _ADR object that contains
- * a PCI address in the scope of the definition. This address is
+ * a PCI address in the scope of the definition. This address is
* required to perform an access to PCI config space.
*
* MUTEX: Interpreter should be unlocked, because we may run the _REG
@@ -572,7 +570,7 @@
if (ACPI_SUCCESS(status)) {
/*
* The _REG method is optional and there can be only one per region
- * definition. This will be executed when the handler is attached
+ * definition. This will be executed when the handler is attached
* or removed
*/
region_obj2->extra.method_REG = method_node;
@@ -670,10 +668,8 @@
}
}
- /*
- * This node does not have the handler we need;
- * Pop up one level
- */
+ /* This node does not have the handler we need; Pop up one level */
+
node = acpi_ns_get_parent_node(node);
}
diff --git a/drivers/acpi/events/evsci.c b/drivers/acpi/acpica/evsci.c
similarity index 95%
rename from drivers/acpi/events/evsci.c
rename to drivers/acpi/acpica/evsci.c
index 2a8b778..567b356 100644
--- a/drivers/acpi/events/evsci.c
+++ b/drivers/acpi/acpica/evsci.c
@@ -43,7 +43,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acevents.h>
+#include "accommon.h"
+#include "acevents.h"
#define _COMPONENT ACPI_EVENTS
ACPI_MODULE_NAME("evsci")
@@ -115,10 +116,8 @@
* if this interrupt handler is installed, ACPI is enabled.
*/
- /*
- * GPEs:
- * Check for and dispatch any GPEs that have occurred
- */
+ /* GPEs: Check for and dispatch any GPEs that have occurred */
+
interrupt_handled |= acpi_ev_gpe_detect(gpe_xrupt_list);
return_UINT32(interrupt_handled);
@@ -158,11 +157,11 @@
* RETURN: E_OK if handler uninstalled OK, E_ERROR if handler was not
* installed to begin with
*
- * DESCRIPTION: Remove the SCI interrupt handler. No further SCIs will be
+ * DESCRIPTION: Remove the SCI interrupt handler. No further SCIs will be
* taken.
*
* Note: It doesn't seem important to disable all events or set the event
- * enable registers to their original values. The OS should disable
+ * enable registers to their original values. The OS should disable
* the SCI interrupt level when the handler is removed, so no more
* events will come in.
*
diff --git a/drivers/acpi/events/evxface.c b/drivers/acpi/acpica/evxface.c
similarity index 98%
rename from drivers/acpi/events/evxface.c
rename to drivers/acpi/acpica/evxface.c
index 94a6efe..3aca901 100644
--- a/drivers/acpi/events/evxface.c
+++ b/drivers/acpi/acpica/evxface.c
@@ -42,9 +42,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acevents.h>
-#include <acpi/acinterp.h>
+#include "accommon.h"
+#include "acnamesp.h"
+#include "acevents.h"
+#include "acinterp.h"
#define _COMPONENT ACPI_EVENTS
ACPI_MODULE_NAME("evxface")
@@ -267,7 +268,7 @@
/*
* Root Object:
* Registering a notify handler on the root object indicates that the
- * caller wishes to receive notifications for all objects. Note that
+ * caller wishes to receive notifications for all objects. Note that
* only one <external> global handler can be regsitered (per notify type).
*/
if (device == ACPI_ROOT_OBJECT) {
diff --git a/drivers/acpi/events/evxfevnt.c b/drivers/acpi/acpica/evxfevnt.c
similarity index 81%
rename from drivers/acpi/events/evxfevnt.c
rename to drivers/acpi/acpica/evxfevnt.c
index 41554f7..35485e4 100644
--- a/drivers/acpi/events/evxfevnt.c
+++ b/drivers/acpi/acpica/evxfevnt.c
@@ -42,13 +42,19 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acevents.h>
-#include <acpi/acnamesp.h>
-#include <acpi/actables.h>
+#include "accommon.h"
+#include "acevents.h"
+#include "acnamesp.h"
+#include "actables.h"
#define _COMPONENT ACPI_EVENTS
ACPI_MODULE_NAME("evxfevnt")
+/* Local prototypes */
+acpi_status
+acpi_ev_get_gpe_device(struct acpi_gpe_xrupt_info *gpe_xrupt_info,
+ struct acpi_gpe_block_info *gpe_block, void *context);
+
/*******************************************************************************
*
* FUNCTION: acpi_enable
@@ -60,6 +66,7 @@
* DESCRIPTION: Transfers the system into ACPI mode.
*
******************************************************************************/
+
acpi_status acpi_enable(void)
{
acpi_status status = AE_OK;
@@ -161,8 +168,8 @@
}
/*
- * Enable the requested fixed event (by writing a one to the
- * enable register bit)
+ * Enable the requested fixed event (by writing a one to the enable
+ * register bit)
*/
status =
acpi_set_register(acpi_gbl_fixed_event_info[event].
@@ -343,8 +350,8 @@
}
/*
- * Disable the requested fixed event (by writing a zero to the
- * enable register bit)
+ * Disable the requested fixed event (by writing a zero to the enable
+ * register bit)
*/
status =
acpi_set_register(acpi_gbl_fixed_event_info[event].
@@ -396,8 +403,8 @@
}
/*
- * Clear the requested fixed event (By writing a one to the
- * status register bit)
+ * Clear the requested fixed event (By writing a one to the status
+ * register bit)
*/
status =
acpi_set_register(acpi_gbl_fixed_event_info[event].
@@ -717,3 +724,148 @@
}
ACPI_EXPORT_SYMBOL(acpi_remove_gpe_block)
+
+/*******************************************************************************
+ *
+ * FUNCTION: acpi_get_gpe_device
+ *
+ * PARAMETERS: Index - System GPE index (0-current_gpe_count)
+ * gpe_device - Where the parent GPE Device is returned
+ *
+ * RETURN: Status
+ *
+ * DESCRIPTION: Obtain the GPE device associated with the input index. A NULL
+ * gpe device indicates that the gpe number is contained in one of
+ * the FADT-defined gpe blocks. Otherwise, the GPE block device.
+ *
+ ******************************************************************************/
+acpi_status
+acpi_get_gpe_device(u32 index, acpi_handle *gpe_device)
+{
+ struct acpi_gpe_device_info info;
+ acpi_status status;
+
+ ACPI_FUNCTION_TRACE(acpi_get_gpe_device);
+
+ if (!gpe_device) {
+ return_ACPI_STATUS(AE_BAD_PARAMETER);
+ }
+
+ if (index >= acpi_current_gpe_count) {
+ return_ACPI_STATUS(AE_NOT_EXIST);
+ }
+
+ /* Setup and walk the GPE list */
+
+ info.index = index;
+ info.status = AE_NOT_EXIST;
+ info.gpe_device = NULL;
+ info.next_block_base_index = 0;
+
+ status = acpi_ev_walk_gpe_list(acpi_ev_get_gpe_device, &info);
+ if (ACPI_FAILURE(status)) {
+ return_ACPI_STATUS(status);
+ }
+
+ *gpe_device = info.gpe_device;
+ return_ACPI_STATUS(info.status);
+}
+
+ACPI_EXPORT_SYMBOL(acpi_get_gpe_device)
+
+/*******************************************************************************
+ *
+ * FUNCTION: acpi_ev_get_gpe_device
+ *
+ * PARAMETERS: GPE_WALK_CALLBACK
+ *
+ * RETURN: Status
+ *
+ * DESCRIPTION: Matches the input GPE index (0-current_gpe_count) with a GPE
+ * block device. NULL if the GPE is one of the FADT-defined GPEs.
+ *
+ ******************************************************************************/
+acpi_status
+acpi_ev_get_gpe_device(struct acpi_gpe_xrupt_info *gpe_xrupt_info,
+ struct acpi_gpe_block_info *gpe_block, void *context)
+{
+ struct acpi_gpe_device_info *info = context;
+
+ /* Increment Index by the number of GPEs in this block */
+
+ info->next_block_base_index +=
+ (gpe_block->register_count * ACPI_GPE_REGISTER_WIDTH);
+
+ if (info->index < info->next_block_base_index) {
+ /*
+ * The GPE index is within this block, get the node. Leave the node
+ * NULL for the FADT-defined GPEs
+ */
+ if ((gpe_block->node)->type == ACPI_TYPE_DEVICE) {
+ info->gpe_device = gpe_block->node;
+ }
+
+ info->status = AE_OK;
+ return (AE_CTRL_END);
+ }
+
+ return (AE_OK);
+}
+
+/******************************************************************************
+ *
+ * FUNCTION: acpi_disable_all_gpes
+ *
+ * PARAMETERS: None
+ *
+ * RETURN: Status
+ *
+ * DESCRIPTION: Disable and clear all GPEs in all GPE blocks
+ *
+ ******************************************************************************/
+
+acpi_status acpi_disable_all_gpes(void)
+{
+ acpi_status status;
+
+ ACPI_FUNCTION_TRACE(acpi_disable_all_gpes);
+
+ status = acpi_ut_acquire_mutex(ACPI_MTX_EVENTS);
+ if (ACPI_FAILURE(status)) {
+ return_ACPI_STATUS(status);
+ }
+
+ status = acpi_hw_disable_all_gpes();
+ (void)acpi_ut_release_mutex(ACPI_MTX_EVENTS);
+
+ return_ACPI_STATUS(status);
+}
+
+/******************************************************************************
+ *
+ * FUNCTION: acpi_enable_all_runtime_gpes
+ *
+ * PARAMETERS: None
+ *
+ * RETURN: Status
+ *
+ * DESCRIPTION: Enable all "runtime" GPEs, in all GPE blocks
+ *
+ ******************************************************************************/
+
+acpi_status acpi_enable_all_runtime_gpes(void)
+{
+ acpi_status status;
+
+ ACPI_FUNCTION_TRACE(acpi_enable_all_runtime_gpes);
+
+ status = acpi_ut_acquire_mutex(ACPI_MTX_EVENTS);
+ if (ACPI_FAILURE(status)) {
+ return_ACPI_STATUS(status);
+ }
+
+ status = acpi_hw_enable_all_runtime_gpes();
+ (void)acpi_ut_release_mutex(ACPI_MTX_EVENTS);
+
+ return_ACPI_STATUS(status);
+}
diff --git a/drivers/acpi/events/evxfregn.c b/drivers/acpi/acpica/evxfregn.c
similarity index 98%
rename from drivers/acpi/events/evxfregn.c
rename to drivers/acpi/acpica/evxfregn.c
index e875080..479e7a3 100644
--- a/drivers/acpi/events/evxfregn.c
+++ b/drivers/acpi/acpica/evxfregn.c
@@ -43,8 +43,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acevents.h>
+#include "accommon.h"
+#include "acnamesp.h"
+#include "acevents.h"
#define _COMPONENT ACPI_EVENTS
ACPI_MODULE_NAME("evxfregn")
diff --git a/drivers/acpi/executer/exconfig.c b/drivers/acpi/acpica/exconfig.c
similarity index 98%
rename from drivers/acpi/executer/exconfig.c
rename to drivers/acpi/acpica/exconfig.c
index 74da6fa..932bbc2 100644
--- a/drivers/acpi/executer/exconfig.c
+++ b/drivers/acpi/acpica/exconfig.c
@@ -42,10 +42,11 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acinterp.h>
-#include <acpi/acnamesp.h>
-#include <acpi/actables.h>
-#include <acpi/acdispat.h>
+#include "accommon.h"
+#include "acinterp.h"
+#include "acnamesp.h"
+#include "actables.h"
+#include "acdispat.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exconfig")
diff --git a/drivers/acpi/executer/exconvrt.c b/drivers/acpi/acpica/exconvrt.c
similarity index 99%
rename from drivers/acpi/executer/exconvrt.c
rename to drivers/acpi/acpica/exconvrt.c
index 1d1f35a..0be1018 100644
--- a/drivers/acpi/executer/exconvrt.c
+++ b/drivers/acpi/acpica/exconvrt.c
@@ -42,8 +42,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acinterp.h>
-#include <acpi/amlcode.h>
+#include "accommon.h"
+#include "acinterp.h"
+#include "amlcode.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exconvrt")
diff --git a/drivers/acpi/executer/excreate.c b/drivers/acpi/acpica/excreate.c
similarity index 99%
rename from drivers/acpi/executer/excreate.c
rename to drivers/acpi/acpica/excreate.c
index ad09696..a57ad25 100644
--- a/drivers/acpi/executer/excreate.c
+++ b/drivers/acpi/acpica/excreate.c
@@ -42,9 +42,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acinterp.h>
-#include <acpi/amlcode.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acinterp.h"
+#include "amlcode.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("excreate")
diff --git a/drivers/acpi/executer/exdump.c b/drivers/acpi/acpica/exdump.c
similarity index 99%
rename from drivers/acpi/executer/exdump.c
rename to drivers/acpi/acpica/exdump.c
index d087a7d..aa31357 100644
--- a/drivers/acpi/executer/exdump.c
+++ b/drivers/acpi/acpica/exdump.c
@@ -42,9 +42,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acinterp.h>
-#include <acpi/amlcode.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acinterp.h"
+#include "amlcode.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exdump")
diff --git a/drivers/acpi/executer/exfield.c b/drivers/acpi/acpica/exfield.c
similarity index 99%
rename from drivers/acpi/executer/exfield.c
rename to drivers/acpi/acpica/exfield.c
index 3e440d8..a352d02 100644
--- a/drivers/acpi/executer/exfield.c
+++ b/drivers/acpi/acpica/exfield.c
@@ -42,8 +42,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acdispat.h>
-#include <acpi/acinterp.h>
+#include "accommon.h"
+#include "acdispat.h"
+#include "acinterp.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exfield")
diff --git a/drivers/acpi/executer/exfldio.c b/drivers/acpi/acpica/exfldio.c
similarity index 98%
rename from drivers/acpi/executer/exfldio.c
rename to drivers/acpi/acpica/exfldio.c
index 9ff9d1f..ef58ac4 100644
--- a/drivers/acpi/executer/exfldio.c
+++ b/drivers/acpi/acpica/exfldio.c
@@ -42,10 +42,11 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acinterp.h>
-#include <acpi/amlcode.h>
-#include <acpi/acevents.h>
-#include <acpi/acdispat.h>
+#include "accommon.h"
+#include "acinterp.h"
+#include "amlcode.h"
+#include "acevents.h"
+#include "acdispat.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exfldio")
@@ -498,14 +499,13 @@
return_ACPI_STATUS(status);
}
- ACPI_DEBUG_PRINT((ACPI_DB_BFIELD,
- "I/O to Data Register: ValuePtr %p\n",
- value));
-
if (read_write == ACPI_READ) {
/* Read the datum from the data_register */
+ ACPI_DEBUG_PRINT((ACPI_DB_BFIELD,
+ "Read from Data Register\n"));
+
status =
acpi_ex_extract_from_field(obj_desc->index_field.
data_obj, value,
@@ -513,6 +513,10 @@
} else {
/* Write the datum to the data_register */
+ ACPI_DEBUG_PRINT((ACPI_DB_BFIELD,
+ "Write to Data Register: Value %8.8X%8.8X\n",
+ ACPI_FORMAT_UINT64(*value)));
+
status =
acpi_ex_insert_into_field(obj_desc->index_field.
data_obj, value,
diff --git a/drivers/acpi/executer/exmisc.c b/drivers/acpi/acpica/exmisc.c
similarity index 99%
rename from drivers/acpi/executer/exmisc.c
rename to drivers/acpi/acpica/exmisc.c
index efb1913..6b0747a 100644
--- a/drivers/acpi/executer/exmisc.c
+++ b/drivers/acpi/acpica/exmisc.c
@@ -43,9 +43,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acinterp.h>
-#include <acpi/amlcode.h>
-#include <acpi/amlresrc.h>
+#include "accommon.h"
+#include "acinterp.h"
+#include "amlcode.h"
+#include "amlresrc.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exmisc")
diff --git a/drivers/acpi/executer/exmutex.c b/drivers/acpi/acpica/exmutex.c
similarity index 99%
rename from drivers/acpi/executer/exmutex.c
rename to drivers/acpi/acpica/exmutex.c
index a8bf3d7..d301c1f 100644
--- a/drivers/acpi/executer/exmutex.c
+++ b/drivers/acpi/acpica/exmutex.c
@@ -43,8 +43,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acinterp.h>
-#include <acpi/acevents.h>
+#include "accommon.h"
+#include "acinterp.h"
+#include "acevents.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exmutex")
diff --git a/drivers/acpi/executer/exnames.c b/drivers/acpi/acpica/exnames.c
similarity index 99%
rename from drivers/acpi/executer/exnames.c
rename to drivers/acpi/acpica/exnames.c
index 817e67b..ffdae12 100644
--- a/drivers/acpi/executer/exnames.c
+++ b/drivers/acpi/acpica/exnames.c
@@ -43,8 +43,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acinterp.h>
-#include <acpi/amlcode.h>
+#include "accommon.h"
+#include "acinterp.h"
+#include "amlcode.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exnames")
diff --git a/drivers/acpi/executer/exoparg1.c b/drivers/acpi/acpica/exoparg1.c
similarity index 99%
rename from drivers/acpi/executer/exoparg1.c
rename to drivers/acpi/acpica/exoparg1.c
index f622f9e..b530480 100644
--- a/drivers/acpi/executer/exoparg1.c
+++ b/drivers/acpi/acpica/exoparg1.c
@@ -43,11 +43,12 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
-#include <acpi/acdispat.h>
-#include <acpi/acinterp.h>
-#include <acpi/amlcode.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acparser.h"
+#include "acdispat.h"
+#include "acinterp.h"
+#include "amlcode.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exoparg1")
diff --git a/drivers/acpi/executer/exoparg2.c b/drivers/acpi/acpica/exoparg2.c
similarity index 98%
rename from drivers/acpi/executer/exoparg2.c
rename to drivers/acpi/acpica/exoparg2.c
index 368def5..0b4f513 100644
--- a/drivers/acpi/executer/exoparg2.c
+++ b/drivers/acpi/acpica/exoparg2.c
@@ -42,10 +42,11 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
-#include <acpi/acinterp.h>
-#include <acpi/acevents.h>
-#include <acpi/amlcode.h>
+#include "accommon.h"
+#include "acparser.h"
+#include "acinterp.h"
+#include "acevents.h"
+#include "amlcode.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exoparg2")
diff --git a/drivers/acpi/executer/exoparg3.c b/drivers/acpi/acpica/exoparg3.c
similarity index 98%
rename from drivers/acpi/executer/exoparg3.c
rename to drivers/acpi/acpica/exoparg3.c
index 9cb4197..c6520bb 100644
--- a/drivers/acpi/executer/exoparg3.c
+++ b/drivers/acpi/acpica/exoparg3.c
@@ -43,9 +43,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acinterp.h>
-#include <acpi/acparser.h>
-#include <acpi/amlcode.h>
+#include "accommon.h"
+#include "acinterp.h"
+#include "acparser.h"
+#include "amlcode.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exoparg3")
diff --git a/drivers/acpi/executer/exoparg6.c b/drivers/acpi/acpica/exoparg6.c
similarity index 98%
rename from drivers/acpi/executer/exoparg6.c
rename to drivers/acpi/acpica/exoparg6.c
index 67d4873..ae43f76 100644
--- a/drivers/acpi/executer/exoparg6.c
+++ b/drivers/acpi/acpica/exoparg6.c
@@ -43,9 +43,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acinterp.h>
-#include <acpi/acparser.h>
-#include <acpi/amlcode.h>
+#include "accommon.h"
+#include "acinterp.h"
+#include "acparser.h"
+#include "amlcode.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exoparg6")
diff --git a/drivers/acpi/executer/exprep.c b/drivers/acpi/acpica/exprep.c
similarity index 99%
rename from drivers/acpi/executer/exprep.c
rename to drivers/acpi/acpica/exprep.c
index a7dc87e..a226f74 100644
--- a/drivers/acpi/executer/exprep.c
+++ b/drivers/acpi/acpica/exprep.c
@@ -43,9 +43,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acinterp.h>
-#include <acpi/amlcode.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acinterp.h"
+#include "amlcode.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exprep")
diff --git a/drivers/acpi/executer/exregion.c b/drivers/acpi/acpica/exregion.c
similarity index 99%
rename from drivers/acpi/executer/exregion.c
rename to drivers/acpi/acpica/exregion.c
index 7a41c40..76ec8ff 100644
--- a/drivers/acpi/executer/exregion.c
+++ b/drivers/acpi/acpica/exregion.c
@@ -43,7 +43,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acinterp.h>
+#include "accommon.h"
+#include "acinterp.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exregion")
diff --git a/drivers/acpi/executer/exresnte.c b/drivers/acpi/acpica/exresnte.c
similarity index 98%
rename from drivers/acpi/executer/exresnte.c
rename to drivers/acpi/acpica/exresnte.c
index 423ad36..a063a74 100644
--- a/drivers/acpi/executer/exresnte.c
+++ b/drivers/acpi/acpica/exresnte.c
@@ -43,9 +43,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acdispat.h>
-#include <acpi/acinterp.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acdispat.h"
+#include "acinterp.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exresnte")
diff --git a/drivers/acpi/executer/exresolv.c b/drivers/acpi/acpica/exresolv.c
similarity index 98%
rename from drivers/acpi/executer/exresolv.c
rename to drivers/acpi/acpica/exresolv.c
index 60e8c47..f6105a6 100644
--- a/drivers/acpi/executer/exresolv.c
+++ b/drivers/acpi/acpica/exresolv.c
@@ -43,10 +43,11 @@
*/
#include <acpi/acpi.h>
-#include <acpi/amlcode.h>
-#include <acpi/acdispat.h>
-#include <acpi/acinterp.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "amlcode.h"
+#include "acdispat.h"
+#include "acinterp.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exresolv")
diff --git a/drivers/acpi/executer/exresop.c b/drivers/acpi/acpica/exresop.c
similarity index 99%
rename from drivers/acpi/executer/exresop.c
rename to drivers/acpi/acpica/exresop.c
index 0bb8259..3c38027 100644
--- a/drivers/acpi/executer/exresop.c
+++ b/drivers/acpi/acpica/exresop.c
@@ -43,10 +43,11 @@
*/
#include <acpi/acpi.h>
-#include <acpi/amlcode.h>
-#include <acpi/acparser.h>
-#include <acpi/acinterp.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "amlcode.h"
+#include "acparser.h"
+#include "acinterp.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exresop")
diff --git a/drivers/acpi/executer/exstore.c b/drivers/acpi/acpica/exstore.c
similarity index 99%
rename from drivers/acpi/executer/exstore.c
rename to drivers/acpi/acpica/exstore.c
index 1c118ba..e35e9b4 100644
--- a/drivers/acpi/executer/exstore.c
+++ b/drivers/acpi/acpica/exstore.c
@@ -43,10 +43,11 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acdispat.h>
-#include <acpi/acinterp.h>
-#include <acpi/amlcode.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acdispat.h"
+#include "acinterp.h"
+#include "amlcode.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exstore")
diff --git a/drivers/acpi/executer/exstoren.c b/drivers/acpi/acpica/exstoren.c
similarity index 98%
rename from drivers/acpi/executer/exstoren.c
rename to drivers/acpi/acpica/exstoren.c
index eef61a0..145d153 100644
--- a/drivers/acpi/executer/exstoren.c
+++ b/drivers/acpi/acpica/exstoren.c
@@ -44,8 +44,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acinterp.h>
-#include <acpi/amlcode.h>
+#include "accommon.h"
+#include "acinterp.h"
+#include "amlcode.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exstoren")
diff --git a/drivers/acpi/executer/exstorob.c b/drivers/acpi/acpica/exstorob.c
similarity index 99%
rename from drivers/acpi/executer/exstorob.c
rename to drivers/acpi/acpica/exstorob.c
index 9a75ff0..67340cc 100644
--- a/drivers/acpi/executer/exstorob.c
+++ b/drivers/acpi/acpica/exstorob.c
@@ -43,7 +43,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acinterp.h>
+#include "accommon.h"
+#include "acinterp.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exstorob")
diff --git a/drivers/acpi/executer/exsystem.c b/drivers/acpi/acpica/exsystem.c
similarity index 99%
rename from drivers/acpi/executer/exsystem.c
rename to drivers/acpi/acpica/exsystem.c
index 68990f1..3d00b93 100644
--- a/drivers/acpi/executer/exsystem.c
+++ b/drivers/acpi/acpica/exsystem.c
@@ -43,7 +43,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acinterp.h>
+#include "accommon.h"
+#include "acinterp.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exsystem")
diff --git a/drivers/acpi/executer/exutils.c b/drivers/acpi/acpica/exutils.c
similarity index 99%
rename from drivers/acpi/executer/exutils.c
rename to drivers/acpi/acpica/exutils.c
index 86c0388..32b85d6 100644
--- a/drivers/acpi/executer/exutils.c
+++ b/drivers/acpi/acpica/exutils.c
@@ -59,8 +59,9 @@
#define DEFINE_AML_GLOBALS
#include <acpi/acpi.h>
-#include <acpi/acinterp.h>
-#include <acpi/amlcode.h>
+#include "accommon.h"
+#include "acinterp.h"
+#include "amlcode.h"
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exutils")
diff --git a/drivers/acpi/hardware/hwacpi.c b/drivers/acpi/acpica/hwacpi.c
similarity index 99%
rename from drivers/acpi/hardware/hwacpi.c
rename to drivers/acpi/acpica/hwacpi.c
index 816894e..a9d4fea 100644
--- a/drivers/acpi/hardware/hwacpi.c
+++ b/drivers/acpi/acpica/hwacpi.c
@@ -43,6 +43,7 @@
*/
#include <acpi/acpi.h>
+#include "accommon.h"
#define _COMPONENT ACPI_HARDWARE
ACPI_MODULE_NAME("hwacpi")
diff --git a/drivers/acpi/hardware/hwgpe.c b/drivers/acpi/acpica/hwgpe.c
similarity index 86%
rename from drivers/acpi/hardware/hwgpe.c
rename to drivers/acpi/acpica/hwgpe.c
index 0b80db9..2013b66 100644
--- a/drivers/acpi/hardware/hwgpe.c
+++ b/drivers/acpi/acpica/hwgpe.c
@@ -43,7 +43,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acevents.h>
+#include "accommon.h"
+#include "acevents.h"
#define _COMPONENT ACPI_HARDWARE
ACPI_MODULE_NAME("hwgpe")
@@ -51,7 +52,8 @@
/* Local prototypes */
static acpi_status
acpi_hw_enable_wakeup_gpe_block(struct acpi_gpe_xrupt_info *gpe_xrupt_info,
- struct acpi_gpe_block_info *gpe_block);
+ struct acpi_gpe_block_info *gpe_block,
+ void *context);
/******************************************************************************
*
@@ -80,8 +82,7 @@
/* Get current value of the enable register that contains this GPE */
- status = acpi_hw_low_level_read(ACPI_GPE_REGISTER_WIDTH, &enable_mask,
- &gpe_register_info->enable_address);
+ status = acpi_read(&enable_mask, &gpe_register_info->enable_address);
if (ACPI_FAILURE(status)) {
return (status);
}
@@ -95,9 +96,7 @@
/* Write the updated enable mask */
- status = acpi_hw_low_level_write(ACPI_GPE_REGISTER_WIDTH, enable_mask,
- &gpe_register_info->enable_address);
-
+ status = acpi_write(enable_mask, &gpe_register_info->enable_address);
return (status);
}
@@ -132,8 +131,8 @@
/* Write the entire GPE (runtime) enable register */
- status = acpi_hw_low_level_write(8, gpe_register_info->enable_for_run,
- &gpe_register_info->enable_address);
+ status = acpi_write(gpe_register_info->enable_for_run,
+ &gpe_register_info->enable_address);
return (status);
}
@@ -166,9 +165,8 @@
* Write a one to the appropriate bit in the status register to
* clear this GPE.
*/
- status = acpi_hw_low_level_write(8, register_bit,
- &gpe_event_info->register_info->
- status_address);
+ status = acpi_write(register_bit,
+ &gpe_event_info->register_info->status_address);
return (status);
}
@@ -227,9 +225,7 @@
/* GPE currently active (status bit == 1)? */
- status =
- acpi_hw_low_level_read(8, &in_byte,
- &gpe_register_info->status_address);
+ status = acpi_read(&in_byte, &gpe_register_info->status_address);
if (ACPI_FAILURE(status)) {
goto unlock_and_exit;
}
@@ -260,8 +256,8 @@
******************************************************************************/
acpi_status
-acpi_hw_disable_gpe_block(struct acpi_gpe_xrupt_info * gpe_xrupt_info,
- struct acpi_gpe_block_info * gpe_block)
+acpi_hw_disable_gpe_block(struct acpi_gpe_xrupt_info *gpe_xrupt_info,
+ struct acpi_gpe_block_info *gpe_block, void *context)
{
u32 i;
acpi_status status;
@@ -272,9 +268,9 @@
/* Disable all GPEs in this register */
- status = acpi_hw_low_level_write(8, 0x00,
- &gpe_block->register_info[i].
- enable_address);
+ status =
+ acpi_write(0x00,
+ &gpe_block->register_info[i].enable_address);
if (ACPI_FAILURE(status)) {
return (status);
}
@@ -297,8 +293,8 @@
******************************************************************************/
acpi_status
-acpi_hw_clear_gpe_block(struct acpi_gpe_xrupt_info * gpe_xrupt_info,
- struct acpi_gpe_block_info * gpe_block)
+acpi_hw_clear_gpe_block(struct acpi_gpe_xrupt_info *gpe_xrupt_info,
+ struct acpi_gpe_block_info *gpe_block, void *context)
{
u32 i;
acpi_status status;
@@ -309,9 +305,9 @@
/* Clear status on all GPEs in this register */
- status = acpi_hw_low_level_write(8, 0xFF,
- &gpe_block->register_info[i].
- status_address);
+ status =
+ acpi_write(0xFF,
+ &gpe_block->register_info[i].status_address);
if (ACPI_FAILURE(status)) {
return (status);
}
@@ -335,8 +331,8 @@
******************************************************************************/
acpi_status
-acpi_hw_enable_runtime_gpe_block(struct acpi_gpe_xrupt_info * gpe_xrupt_info,
- struct acpi_gpe_block_info * gpe_block)
+acpi_hw_enable_runtime_gpe_block(struct acpi_gpe_xrupt_info *gpe_xrupt_info,
+ struct acpi_gpe_block_info *gpe_block, void *context)
{
u32 i;
acpi_status status;
@@ -352,12 +348,9 @@
/* Enable all "runtime" GPEs in this register */
- status =
- acpi_hw_low_level_write(8,
- gpe_block->register_info[i].
- enable_for_run,
- &gpe_block->register_info[i].
- enable_address);
+ status = acpi_write(gpe_block->register_info[i].enable_for_run,
+ &gpe_block->register_info[i].
+ enable_address);
if (ACPI_FAILURE(status)) {
return (status);
}
@@ -382,7 +375,8 @@
static acpi_status
acpi_hw_enable_wakeup_gpe_block(struct acpi_gpe_xrupt_info *gpe_xrupt_info,
- struct acpi_gpe_block_info *gpe_block)
+ struct acpi_gpe_block_info *gpe_block,
+ void *context)
{
u32 i;
acpi_status status;
@@ -396,11 +390,9 @@
/* Enable all "wake" GPEs in this register */
- status = acpi_hw_low_level_write(8,
- gpe_block->register_info[i].
- enable_for_wake,
- &gpe_block->register_info[i].
- enable_address);
+ status = acpi_write(gpe_block->register_info[i].enable_for_wake,
+ &gpe_block->register_info[i].
+ enable_address);
if (ACPI_FAILURE(status)) {
return (status);
}
@@ -427,8 +419,8 @@
ACPI_FUNCTION_TRACE(hw_disable_all_gpes);
- status = acpi_ev_walk_gpe_list(acpi_hw_disable_gpe_block);
- status = acpi_ev_walk_gpe_list(acpi_hw_clear_gpe_block);
+ status = acpi_ev_walk_gpe_list(acpi_hw_disable_gpe_block, NULL);
+ status = acpi_ev_walk_gpe_list(acpi_hw_clear_gpe_block, NULL);
return_ACPI_STATUS(status);
}
@@ -450,7 +442,7 @@
ACPI_FUNCTION_TRACE(hw_enable_all_runtime_gpes);
- status = acpi_ev_walk_gpe_list(acpi_hw_enable_runtime_gpe_block);
+ status = acpi_ev_walk_gpe_list(acpi_hw_enable_runtime_gpe_block, NULL);
return_ACPI_STATUS(status);
}
@@ -472,6 +464,6 @@
ACPI_FUNCTION_TRACE(hw_enable_all_wakeup_gpes);
- status = acpi_ev_walk_gpe_list(acpi_hw_enable_wakeup_gpe_block);
+ status = acpi_ev_walk_gpe_list(acpi_hw_enable_wakeup_gpe_block, NULL);
return_ACPI_STATUS(status);
}
diff --git a/drivers/acpi/acpica/hwregs.c b/drivers/acpi/acpica/hwregs.c
new file mode 100644
index 0000000..4dc43b0
--- /dev/null
+++ b/drivers/acpi/acpica/hwregs.c
@@ -0,0 +1,353 @@
+
+/*******************************************************************************
+ *
+ * Module Name: hwregs - Read/write access functions for the various ACPI
+ * control and status registers.
+ *
+ ******************************************************************************/
+
+/*
+ * Copyright (C) 2000 - 2008, Intel Corp.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions, and the following disclaimer,
+ * without modification.
+ * 2. Redistributions in binary form must reproduce at minimum a disclaimer
+ * substantially similar to the "NO WARRANTY" disclaimer below
+ * ("Disclaimer") and any redistribution must be conditioned upon
+ * including a substantially similar Disclaimer requirement for further
+ * binary redistribution.
+ * 3. Neither the names of the above-listed copyright holders nor the names
+ * of any contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * Alternatively, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") version 2 as published by the Free
+ * Software Foundation.
+ *
+ * NO WARRANTY
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGES.
+ */
+
+#include <acpi/acpi.h>
+#include "accommon.h"
+#include "acnamesp.h"
+#include "acevents.h"
+
+#define _COMPONENT ACPI_HARDWARE
+ACPI_MODULE_NAME("hwregs")
+
+/*******************************************************************************
+ *
+ * FUNCTION: acpi_hw_clear_acpi_status
+ *
+ * PARAMETERS: None
+ *
+ * RETURN: Status
+ *
+ * DESCRIPTION: Clears all fixed and general purpose status bits
+ * THIS FUNCTION MUST BE CALLED WITH INTERRUPTS DISABLED
+ *
+ ******************************************************************************/
+acpi_status acpi_hw_clear_acpi_status(void)
+{
+ acpi_status status;
+ acpi_cpu_flags lock_flags = 0;
+
+ ACPI_FUNCTION_TRACE(hw_clear_acpi_status);
+
+ ACPI_DEBUG_PRINT((ACPI_DB_IO, "About to write %04X to %04X\n",
+ ACPI_BITMASK_ALL_FIXED_STATUS,
+ (u16) acpi_gbl_FADT.xpm1a_event_block.address));
+
+ lock_flags = acpi_os_acquire_lock(acpi_gbl_hardware_lock);
+
+ status = acpi_hw_register_write(ACPI_REGISTER_PM1_STATUS,
+ ACPI_BITMASK_ALL_FIXED_STATUS);
+ if (ACPI_FAILURE(status)) {
+ goto unlock_and_exit;
+ }
+
+ /* Clear the fixed events */
+
+ if (acpi_gbl_FADT.xpm1b_event_block.address) {
+ status = acpi_write(ACPI_BITMASK_ALL_FIXED_STATUS,
+ &acpi_gbl_FADT.xpm1b_event_block);
+ if (ACPI_FAILURE(status)) {
+ goto unlock_and_exit;
+ }
+ }
+
+ /* Clear the GPE Bits in all GPE registers in all GPE blocks */
+
+ status = acpi_ev_walk_gpe_list(acpi_hw_clear_gpe_block, NULL);
+
+ unlock_and_exit:
+ acpi_os_release_lock(acpi_gbl_hardware_lock, lock_flags);
+ return_ACPI_STATUS(status);
+}
+
+/*******************************************************************************
+ *
+ * FUNCTION: acpi_hw_get_register_bit_mask
+ *
+ * PARAMETERS: register_id - Index of ACPI Register to access
+ *
+ * RETURN: The bitmask to be used when accessing the register
+ *
+ * DESCRIPTION: Map register_id into a register bitmask.
+ *
+ ******************************************************************************/
+
+struct acpi_bit_register_info *acpi_hw_get_bit_register_info(u32 register_id)
+{
+ ACPI_FUNCTION_ENTRY();
+
+ if (register_id > ACPI_BITREG_MAX) {
+ ACPI_ERROR((AE_INFO, "Invalid BitRegister ID: %X",
+ register_id));
+ return (NULL);
+ }
+
+ return (&acpi_gbl_bit_register_info[register_id]);
+}
+
+/******************************************************************************
+ *
+ * FUNCTION: acpi_hw_register_read
+ *
+ * PARAMETERS: register_id - ACPI Register ID
+ * return_value - Where the register value is returned
+ *
+ * RETURN: Status and the value read.
+ *
+ * DESCRIPTION: Read from the specified ACPI register
+ *
+ ******************************************************************************/
+acpi_status
+acpi_hw_register_read(u32 register_id, u32 * return_value)
+{
+ u32 value1 = 0;
+ u32 value2 = 0;
+ acpi_status status;
+
+ ACPI_FUNCTION_TRACE(hw_register_read);
+
+ switch (register_id) {
+ case ACPI_REGISTER_PM1_STATUS: /* 16-bit access */
+
+ status = acpi_read(&value1, &acpi_gbl_FADT.xpm1a_event_block);
+ if (ACPI_FAILURE(status)) {
+ goto exit;
+ }
+
+ /* PM1B is optional */
+
+ status = acpi_read(&value2, &acpi_gbl_FADT.xpm1b_event_block);
+ value1 |= value2;
+ break;
+
+ case ACPI_REGISTER_PM1_ENABLE: /* 16-bit access */
+
+ status = acpi_read(&value1, &acpi_gbl_xpm1a_enable);
+ if (ACPI_FAILURE(status)) {
+ goto exit;
+ }
+
+ /* PM1B is optional */
+
+ status = acpi_read(&value2, &acpi_gbl_xpm1b_enable);
+ value1 |= value2;
+ break;
+
+ case ACPI_REGISTER_PM1_CONTROL: /* 16-bit access */
+
+ status = acpi_read(&value1, &acpi_gbl_FADT.xpm1a_control_block);
+ if (ACPI_FAILURE(status)) {
+ goto exit;
+ }
+
+ status = acpi_read(&value2, &acpi_gbl_FADT.xpm1b_control_block);
+ value1 |= value2;
+ break;
+
+ case ACPI_REGISTER_PM2_CONTROL: /* 8-bit access */
+
+ status = acpi_read(&value1, &acpi_gbl_FADT.xpm2_control_block);
+ break;
+
+ case ACPI_REGISTER_PM_TIMER: /* 32-bit access */
+
+ status = acpi_read(&value1, &acpi_gbl_FADT.xpm_timer_block);
+ break;
+
+ case ACPI_REGISTER_SMI_COMMAND_BLOCK: /* 8-bit access */
+
+ status =
+ acpi_os_read_port(acpi_gbl_FADT.smi_command, &value1, 8);
+ break;
+
+ default:
+ ACPI_ERROR((AE_INFO, "Unknown Register ID: %X", register_id));
+ status = AE_BAD_PARAMETER;
+ break;
+ }
+
+ exit:
+
+ if (ACPI_SUCCESS(status)) {
+ *return_value = value1;
+ }
+
+ return_ACPI_STATUS(status);
+}
+
+/******************************************************************************
+ *
+ * FUNCTION: acpi_hw_register_write
+ *
+ * PARAMETERS: register_id - ACPI Register ID
+ * Value - The value to write
+ *
+ * RETURN: Status
+ *
+ * DESCRIPTION: Write to the specified ACPI register
+ *
+ * NOTE: In accordance with the ACPI specification, this function automatically
+ * preserves the value of the following bits, meaning that these bits cannot be
+ * changed via this interface:
+ *
+ * PM1_CONTROL[0] = SCI_EN
+ * PM1_CONTROL[9]
+ * PM1_STATUS[11]
+ *
+ * ACPI References:
+ * 1) Hardware Ignored Bits: When software writes to a register with ignored
+ * bit fields, it preserves the ignored bit fields
+ * 2) SCI_EN: OSPM always preserves this bit position
+ *
+ ******************************************************************************/
+
+acpi_status acpi_hw_register_write(u32 register_id, u32 value)
+{
+ acpi_status status;
+ u32 read_value;
+
+ ACPI_FUNCTION_TRACE(hw_register_write);
+
+ switch (register_id) {
+ case ACPI_REGISTER_PM1_STATUS: /* 16-bit access */
+
+ /* Perform a read first to preserve certain bits (per ACPI spec) */
+
+ status = acpi_hw_register_read(ACPI_REGISTER_PM1_STATUS,
+ &read_value);
+ if (ACPI_FAILURE(status)) {
+ goto exit;
+ }
+
+ /* Insert the bits to be preserved */
+
+ ACPI_INSERT_BITS(value, ACPI_PM1_STATUS_PRESERVED_BITS,
+ read_value);
+
+ /* Now we can write the data */
+
+ status = acpi_write(value, &acpi_gbl_FADT.xpm1a_event_block);
+ if (ACPI_FAILURE(status)) {
+ goto exit;
+ }
+
+ /* PM1B is optional */
+
+ status = acpi_write(value, &acpi_gbl_FADT.xpm1b_event_block);
+ break;
+
+ case ACPI_REGISTER_PM1_ENABLE: /* 16-bit access */
+
+ status = acpi_write(value, &acpi_gbl_xpm1a_enable);
+ if (ACPI_FAILURE(status)) {
+ goto exit;
+ }
+
+ /* PM1B is optional */
+
+ status = acpi_write(value, &acpi_gbl_xpm1b_enable);
+ break;
+
+ case ACPI_REGISTER_PM1_CONTROL: /* 16-bit access */
+
+ /*
+ * Perform a read first to preserve certain bits (per ACPI spec)
+ */
+ status = acpi_hw_register_read(ACPI_REGISTER_PM1_CONTROL,
+ &read_value);
+ if (ACPI_FAILURE(status)) {
+ goto exit;
+ }
+
+ /* Insert the bits to be preserved */
+
+ ACPI_INSERT_BITS(value, ACPI_PM1_CONTROL_PRESERVED_BITS,
+ read_value);
+
+ /* Now we can write the data */
+
+ status = acpi_write(value, &acpi_gbl_FADT.xpm1a_control_block);
+ if (ACPI_FAILURE(status)) {
+ goto exit;
+ }
+
+ status = acpi_write(value, &acpi_gbl_FADT.xpm1b_control_block);
+ break;
+
+ case ACPI_REGISTER_PM1A_CONTROL: /* 16-bit access */
+
+ status = acpi_write(value, &acpi_gbl_FADT.xpm1a_control_block);
+ break;
+
+ case ACPI_REGISTER_PM1B_CONTROL: /* 16-bit access */
+
+ status = acpi_write(value, &acpi_gbl_FADT.xpm1b_control_block);
+ break;
+
+ case ACPI_REGISTER_PM2_CONTROL: /* 8-bit access */
+
+ status = acpi_write(value, &acpi_gbl_FADT.xpm2_control_block);
+ break;
+
+ case ACPI_REGISTER_PM_TIMER: /* 32-bit access */
+
+ status = acpi_write(value, &acpi_gbl_FADT.xpm_timer_block);
+ break;
+
+ case ACPI_REGISTER_SMI_COMMAND_BLOCK: /* 8-bit access */
+
+ /* SMI_CMD is currently always in IO space */
+
+ status =
+ acpi_os_write_port(acpi_gbl_FADT.smi_command, value, 8);
+ break;
+
+ default:
+ status = AE_BAD_PARAMETER;
+ break;
+ }
+
+ exit:
+ return_ACPI_STATUS(status);
+}
diff --git a/drivers/acpi/hardware/hwsleep.c b/drivers/acpi/acpica/hwsleep.c
similarity index 90%
rename from drivers/acpi/hardware/hwsleep.c
rename to drivers/acpi/acpica/hwsleep.c
index 25dccdf..a2af2a4 100644
--- a/drivers/acpi/hardware/hwsleep.c
+++ b/drivers/acpi/acpica/hwsleep.c
@@ -43,7 +43,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/actables.h>
+#include "accommon.h"
+#include "actables.h"
#define _COMPONENT ACPI_HARDWARE
ACPI_MODULE_NAME("hwsleep")
@@ -52,31 +53,19 @@
*
* FUNCTION: acpi_set_firmware_waking_vector
*
- * PARAMETERS: physical_address - Physical address of ACPI real mode
+ * PARAMETERS: physical_address - 32-bit physical address of ACPI real mode
* entry point.
*
* RETURN: Status
*
- * DESCRIPTION: Access function for the firmware_waking_vector field in FACS
+ * DESCRIPTION: Sets the 32-bit firmware_waking_vector field of the FACS
*
******************************************************************************/
acpi_status
-acpi_set_firmware_waking_vector(acpi_physical_address physical_address)
+acpi_set_firmware_waking_vector(u32 physical_address)
{
- struct acpi_table_facs *facs;
- acpi_status status;
-
ACPI_FUNCTION_TRACE(acpi_set_firmware_waking_vector);
- /* Get the FACS */
-
- status = acpi_get_table_by_index(ACPI_TABLE_INDEX_FACS,
- ACPI_CAST_INDIRECT_PTR(struct
- acpi_table_header,
- &facs));
- if (ACPI_FAILURE(status)) {
- return_ACPI_STATUS(status);
- }
/*
* According to the ACPI specification 2.0c and later, the 64-bit
@@ -85,10 +74,16 @@
* Protected Mode. Some systems (for example HP dv5-1004nr) are known
* to fail to resume if the 64-bit vector is used.
*/
- if (facs->version >= 1)
- facs->xfirmware_waking_vector = 0;
- facs->firmware_waking_vector = (u32)physical_address;
+ /* Set the 32-bit vector */
+
+ acpi_gbl_FACS->firmware_waking_vector = physical_address;
+
+ /* Clear the 64-bit vector if it exists */
+
+ if ((acpi_gbl_FACS->length > 32) && (acpi_gbl_FACS->version >= 1)) {
+ acpi_gbl_FACS->xfirmware_waking_vector = 0;
+ }
return_ACPI_STATUS(AE_OK);
}
@@ -97,48 +92,39 @@
/*******************************************************************************
*
- * FUNCTION: acpi_get_firmware_waking_vector
+ * FUNCTION: acpi_set_firmware_waking_vector64
*
- * PARAMETERS: *physical_address - Where the contents of
- * the firmware_waking_vector field of
- * the FACS will be returned.
+ * PARAMETERS: physical_address - 64-bit physical address of ACPI protected
+ * mode entry point.
*
- * RETURN: Status, vector
+ * RETURN: Status
*
- * DESCRIPTION: Access function for the firmware_waking_vector field in FACS
+ * DESCRIPTION: Sets the 64-bit X_firmware_waking_vector field of the FACS, if
+ * it exists in the table.
*
******************************************************************************/
-#ifdef ACPI_FUTURE_USAGE
acpi_status
-acpi_get_firmware_waking_vector(acpi_physical_address * physical_address)
+acpi_set_firmware_waking_vector64(u64 physical_address)
{
- struct acpi_table_facs *facs;
- acpi_status status;
+ ACPI_FUNCTION_TRACE(acpi_set_firmware_waking_vector64);
- ACPI_FUNCTION_TRACE(acpi_get_firmware_waking_vector);
- if (!physical_address) {
- return_ACPI_STATUS(AE_BAD_PARAMETER);
+ /* Determine if the 64-bit vector actually exists */
+
+ if ((acpi_gbl_FACS->length <= 32) || (acpi_gbl_FACS->version < 1)) {
+ return_ACPI_STATUS(AE_NOT_EXIST);
}
- /* Get the FACS */
+ /* Clear 32-bit vector, set the 64-bit X_ vector */
- status = acpi_get_table_by_index(ACPI_TABLE_INDEX_FACS,
- ACPI_CAST_INDIRECT_PTR(struct
- acpi_table_header,
- &facs));
- if (ACPI_FAILURE(status)) {
- return_ACPI_STATUS(status);
- }
-
- /* Get the vector */
- *physical_address = (acpi_physical_address)facs->firmware_waking_vector;
+ acpi_gbl_FACS->firmware_waking_vector = 0;
+ acpi_gbl_FACS->xfirmware_waking_vector = physical_address;
return_ACPI_STATUS(AE_OK);
}
-ACPI_EXPORT_SYMBOL(acpi_get_firmware_waking_vector)
-#endif
+ACPI_EXPORT_SYMBOL(acpi_set_firmware_waking_vector64)
+
/*******************************************************************************
*
* FUNCTION: acpi_enter_sleep_state_prep
diff --git a/drivers/acpi/hardware/hwtimer.c b/drivers/acpi/acpica/hwtimer.c
similarity index 99%
rename from drivers/acpi/hardware/hwtimer.c
rename to drivers/acpi/acpica/hwtimer.c
index b53d575..b7f522c 100644
--- a/drivers/acpi/hardware/hwtimer.c
+++ b/drivers/acpi/acpica/hwtimer.c
@@ -43,6 +43,7 @@
*/
#include <acpi/acpi.h>
+#include "accommon.h"
#define _COMPONENT ACPI_HARDWARE
ACPI_MODULE_NAME("hwtimer")
diff --git a/drivers/acpi/hardware/hwregs.c b/drivers/acpi/acpica/hwxface.c
similarity index 62%
rename from drivers/acpi/hardware/hwregs.c
rename to drivers/acpi/acpica/hwxface.c
index ddf792a..ae597c0 100644
--- a/drivers/acpi/hardware/hwregs.c
+++ b/drivers/acpi/acpica/hwxface.c
@@ -1,10 +1,9 @@
-/*******************************************************************************
+/******************************************************************************
*
- * Module Name: hwregs - Read/write access functions for the various ACPI
- * control and status registers.
+ * Module Name: hwxface - Public ACPICA hardware interfaces
*
- ******************************************************************************/
+ *****************************************************************************/
/*
* Copyright (C) 2000 - 2008, Intel Corp.
@@ -44,63 +43,437 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acevents.h>
+#include "accommon.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_HARDWARE
-ACPI_MODULE_NAME("hwregs")
+ACPI_MODULE_NAME("hwxface")
-/*******************************************************************************
+/******************************************************************************
*
- * FUNCTION: acpi_hw_clear_acpi_status
+ * FUNCTION: acpi_reset
*
* PARAMETERS: None
*
- * RETURN: None
+ * RETURN: Status
*
- * DESCRIPTION: Clears all fixed and general purpose status bits
- * THIS FUNCTION MUST BE CALLED WITH INTERRUPTS DISABLED
+ * DESCRIPTION: Set reset register in memory or IO space. Note: Does not
+ * support reset register in PCI config space, this must be
+ * handled separately.
*
******************************************************************************/
-acpi_status acpi_hw_clear_acpi_status(void)
+acpi_status acpi_reset(void)
+{
+ struct acpi_generic_address *reset_reg;
+ acpi_status status;
+
+ ACPI_FUNCTION_TRACE(acpi_reset);
+
+ reset_reg = &acpi_gbl_FADT.reset_register;
+
+ /* Check if the reset register is supported */
+
+ if (!(acpi_gbl_FADT.flags & ACPI_FADT_RESET_REGISTER) ||
+ !reset_reg->address) {
+ return_ACPI_STATUS(AE_NOT_EXIST);
+ }
+
+ /* Write the reset value to the reset register */
+
+ status = acpi_write(acpi_gbl_FADT.reset_value, reset_reg);
+ return_ACPI_STATUS(status);
+}
+
+ACPI_EXPORT_SYMBOL(acpi_reset)
+
+/******************************************************************************
+ *
+ * FUNCTION: acpi_read
+ *
+ * PARAMETERS: Value - Where the value is returned
+ * Reg - GAS register structure
+ *
+ * RETURN: Status
+ *
+ * DESCRIPTION: Read from either memory or IO space.
+ *
+ ******************************************************************************/
+acpi_status acpi_read(u32 *value, struct acpi_generic_address *reg)
+{
+ u32 width;
+ u64 address;
+ acpi_status status;
+
+ ACPI_FUNCTION_NAME(acpi_read);
+
+ /*
+ * Must have a valid pointer to a GAS structure, and
+ * a non-zero address within. However, don't return an error
+ * because the PM1A/B code must not fail if B isn't present.
+ */
+ if (!reg) {
+ return (AE_OK);
+ }
+
+ /* Get a local copy of the address. Handles possible alignment issues */
+
+ ACPI_MOVE_64_TO_64(&address, ®->address);
+ if (!address) {
+ return (AE_OK);
+ }
+
+ /* Supported widths are 8/16/32 */
+
+ width = reg->bit_width;
+ if ((width != 8) && (width != 16) && (width != 32)) {
+ return (AE_SUPPORT);
+ }
+
+ /* Initialize entire 32-bit return value to zero */
+
+ *value = 0;
+
+ /*
+ * Two address spaces supported: Memory or IO.
+ * PCI_Config is not supported here because the GAS struct is insufficient
+ */
+ switch (reg->space_id) {
+ case ACPI_ADR_SPACE_SYSTEM_MEMORY:
+
+ status = acpi_os_read_memory((acpi_physical_address) address,
+ value, width);
+ break;
+
+ case ACPI_ADR_SPACE_SYSTEM_IO:
+
+ status =
+ acpi_os_read_port((acpi_io_address) address, value, width);
+ break;
+
+ default:
+ ACPI_ERROR((AE_INFO,
+ "Unsupported address space: %X", reg->space_id));
+ return (AE_BAD_PARAMETER);
+ }
+
+ ACPI_DEBUG_PRINT((ACPI_DB_IO,
+ "Read: %8.8X width %2d from %8.8X%8.8X (%s)\n",
+ *value, width, ACPI_FORMAT_UINT64(address),
+ acpi_ut_get_region_name(reg->space_id)));
+
+ return (status);
+}
+
+ACPI_EXPORT_SYMBOL(acpi_read)
+
+/******************************************************************************
+ *
+ * FUNCTION: acpi_write
+ *
+ * PARAMETERS: Value - To be written
+ * Reg - GAS register structure
+ *
+ * RETURN: Status
+ *
+ * DESCRIPTION: Write to either memory or IO space.
+ *
+ ******************************************************************************/
+acpi_status acpi_write(u32 value, struct acpi_generic_address *reg)
+{
+ u32 width;
+ u64 address;
+ acpi_status status;
+
+ ACPI_FUNCTION_NAME(acpi_write);
+
+ /*
+ * Must have a valid pointer to a GAS structure, and
+ * a non-zero address within. However, don't return an error
+ * because the PM1A/B code must not fail if B isn't present.
+ */
+ if (!reg) {
+ return (AE_OK);
+ }
+
+ /* Get a local copy of the address. Handles possible alignment issues */
+
+ ACPI_MOVE_64_TO_64(&address, ®->address);
+ if (!address) {
+ return (AE_OK);
+ }
+
+ /* Supported widths are 8/16/32 */
+
+ width = reg->bit_width;
+ if ((width != 8) && (width != 16) && (width != 32)) {
+ return (AE_SUPPORT);
+ }
+
+ /*
+ * Two address spaces supported: Memory or IO.
+ * PCI_Config is not supported here because the GAS struct is insufficient
+ */
+ switch (reg->space_id) {
+ case ACPI_ADR_SPACE_SYSTEM_MEMORY:
+
+ status = acpi_os_write_memory((acpi_physical_address) address,
+ value, width);
+ break;
+
+ case ACPI_ADR_SPACE_SYSTEM_IO:
+
+ status = acpi_os_write_port((acpi_io_address) address, value,
+ width);
+ break;
+
+ default:
+ ACPI_ERROR((AE_INFO,
+ "Unsupported address space: %X", reg->space_id));
+ return (AE_BAD_PARAMETER);
+ }
+
+ ACPI_DEBUG_PRINT((ACPI_DB_IO,
+ "Wrote: %8.8X width %2d to %8.8X%8.8X (%s)\n",
+ value, width, ACPI_FORMAT_UINT64(address),
+ acpi_ut_get_region_name(reg->space_id)));
+
+ return (status);
+}
+
+ACPI_EXPORT_SYMBOL(acpi_write)
+
+/*******************************************************************************
+ *
+ * FUNCTION: acpi_get_register_unlocked
+ *
+ * PARAMETERS: register_id - ID of ACPI bit_register to access
+ * return_value - Value that was read from the register
+ *
+ * RETURN: Status and the value read from specified Register. Value
+ * returned is normalized to bit0 (is shifted all the way right)
+ *
+ * DESCRIPTION: ACPI bit_register read function. Does not acquire the HW lock.
+ *
+ ******************************************************************************/
+acpi_status acpi_get_register_unlocked(u32 register_id, u32 *return_value)
+{
+ u32 register_value = 0;
+ struct acpi_bit_register_info *bit_reg_info;
+ acpi_status status;
+
+ ACPI_FUNCTION_TRACE(acpi_get_register_unlocked);
+
+ /* Get the info structure corresponding to the requested ACPI Register */
+
+ bit_reg_info = acpi_hw_get_bit_register_info(register_id);
+ if (!bit_reg_info) {
+ return_ACPI_STATUS(AE_BAD_PARAMETER);
+ }
+
+ /* Read from the register */
+
+ status = acpi_hw_register_read(bit_reg_info->parent_register,
+ ®ister_value);
+
+ if (ACPI_SUCCESS(status)) {
+
+ /* Normalize the value that was read */
+
+ register_value =
+ ((register_value & bit_reg_info->access_bit_mask)
+ >> bit_reg_info->bit_position);
+
+ *return_value = register_value;
+
+ ACPI_DEBUG_PRINT((ACPI_DB_IO, "Read value %8.8X register %X\n",
+ register_value,
+ bit_reg_info->parent_register));
+ }
+
+ return_ACPI_STATUS(status);
+}
+
+ACPI_EXPORT_SYMBOL(acpi_get_register_unlocked)
+
+/*******************************************************************************
+ *
+ * FUNCTION: acpi_get_register
+ *
+ * PARAMETERS: register_id - ID of ACPI bit_register to access
+ * return_value - Value that was read from the register
+ *
+ * RETURN: Status and the value read from specified Register. Value
+ * returned is normalized to bit0 (is shifted all the way right)
+ *
+ * DESCRIPTION: ACPI bit_register read function.
+ *
+ ******************************************************************************/
+acpi_status acpi_get_register(u32 register_id, u32 *return_value)
{
acpi_status status;
- acpi_cpu_flags lock_flags = 0;
+ acpi_cpu_flags flags;
- ACPI_FUNCTION_TRACE(hw_clear_acpi_status);
+ flags = acpi_os_acquire_lock(acpi_gbl_hardware_lock);
+ status = acpi_get_register_unlocked(register_id, return_value);
+ acpi_os_release_lock(acpi_gbl_hardware_lock, flags);
- ACPI_DEBUG_PRINT((ACPI_DB_IO, "About to write %04X to %04X\n",
- ACPI_BITMASK_ALL_FIXED_STATUS,
- (u16) acpi_gbl_FADT.xpm1a_event_block.address));
+ return (status);
+}
+
+ACPI_EXPORT_SYMBOL(acpi_get_register)
+
+/*******************************************************************************
+ *
+ * FUNCTION: acpi_set_register
+ *
+ * PARAMETERS: register_id - ID of ACPI bit_register to access
+ * Value - (only used on write) value to write to the
+ * Register, NOT pre-normalized to the bit pos
+ *
+ * RETURN: Status
+ *
+ * DESCRIPTION: ACPI Bit Register write function.
+ *
+ ******************************************************************************/
+acpi_status acpi_set_register(u32 register_id, u32 value)
+{
+ u32 register_value = 0;
+ struct acpi_bit_register_info *bit_reg_info;
+ acpi_status status;
+ acpi_cpu_flags lock_flags;
+
+ ACPI_FUNCTION_TRACE_U32(acpi_set_register, register_id);
+
+ /* Get the info structure corresponding to the requested ACPI Register */
+
+ bit_reg_info = acpi_hw_get_bit_register_info(register_id);
+ if (!bit_reg_info) {
+ ACPI_ERROR((AE_INFO, "Bad ACPI HW RegisterId: %X",
+ register_id));
+ return_ACPI_STATUS(AE_BAD_PARAMETER);
+ }
lock_flags = acpi_os_acquire_lock(acpi_gbl_hardware_lock);
- status = acpi_hw_register_write(ACPI_REGISTER_PM1_STATUS,
- ACPI_BITMASK_ALL_FIXED_STATUS);
+ /* Always do a register read first so we can insert the new bits */
+
+ status = acpi_hw_register_read(bit_reg_info->parent_register,
+ ®ister_value);
if (ACPI_FAILURE(status)) {
goto unlock_and_exit;
}
- /* Clear the fixed events */
+ /*
+ * Decode the Register ID
+ * Register ID = [Register block ID] | [bit ID]
+ *
+ * Check bit ID to fine locate Register offset.
+ * Check Mask to determine Register offset, and then read-write.
+ */
+ switch (bit_reg_info->parent_register) {
+ case ACPI_REGISTER_PM1_STATUS:
- if (acpi_gbl_FADT.xpm1b_event_block.address) {
- status =
- acpi_hw_low_level_write(16, ACPI_BITMASK_ALL_FIXED_STATUS,
- &acpi_gbl_FADT.xpm1b_event_block);
+ /*
+ * Status Registers are different from the rest. Clear by
+ * writing 1, and writing 0 has no effect. So, the only relevant
+ * information is the single bit we're interested in, all others should
+ * be written as 0 so they will be left unchanged.
+ */
+ value = ACPI_REGISTER_PREPARE_BITS(value,
+ bit_reg_info->bit_position,
+ bit_reg_info->
+ access_bit_mask);
+ if (value) {
+ status =
+ acpi_hw_register_write(ACPI_REGISTER_PM1_STATUS,
+ (u16) value);
+ register_value = 0;
+ }
+ break;
+
+ case ACPI_REGISTER_PM1_ENABLE:
+
+ ACPI_REGISTER_INSERT_VALUE(register_value,
+ bit_reg_info->bit_position,
+ bit_reg_info->access_bit_mask,
+ value);
+
+ status = acpi_hw_register_write(ACPI_REGISTER_PM1_ENABLE,
+ (u16) register_value);
+ break;
+
+ case ACPI_REGISTER_PM1_CONTROL:
+
+ /*
+ * Write the PM1 Control register.
+ * Note that at this level, the fact that there are actually TWO
+ * registers (A and B - and B may not exist) is abstracted.
+ */
+ ACPI_DEBUG_PRINT((ACPI_DB_IO, "PM1 control: Read %X\n",
+ register_value));
+
+ ACPI_REGISTER_INSERT_VALUE(register_value,
+ bit_reg_info->bit_position,
+ bit_reg_info->access_bit_mask,
+ value);
+
+ status = acpi_hw_register_write(ACPI_REGISTER_PM1_CONTROL,
+ (u16) register_value);
+ break;
+
+ case ACPI_REGISTER_PM2_CONTROL:
+
+ status = acpi_hw_register_read(ACPI_REGISTER_PM2_CONTROL,
+ ®ister_value);
if (ACPI_FAILURE(status)) {
goto unlock_and_exit;
}
+
+ ACPI_DEBUG_PRINT((ACPI_DB_IO,
+ "PM2 control: Read %X from %8.8X%8.8X\n",
+ register_value,
+ ACPI_FORMAT_UINT64(acpi_gbl_FADT.
+ xpm2_control_block.
+ address)));
+
+ ACPI_REGISTER_INSERT_VALUE(register_value,
+ bit_reg_info->bit_position,
+ bit_reg_info->access_bit_mask,
+ value);
+
+ ACPI_DEBUG_PRINT((ACPI_DB_IO,
+ "About to write %4.4X to %8.8X%8.8X\n",
+ register_value,
+ ACPI_FORMAT_UINT64(acpi_gbl_FADT.
+ xpm2_control_block.
+ address)));
+
+ status = acpi_hw_register_write(ACPI_REGISTER_PM2_CONTROL,
+ (u8) (register_value));
+ break;
+
+ default:
+ break;
}
- /* Clear the GPE Bits in all GPE registers in all GPE blocks */
-
- status = acpi_ev_walk_gpe_list(acpi_hw_clear_gpe_block);
-
unlock_and_exit:
+
acpi_os_release_lock(acpi_gbl_hardware_lock, lock_flags);
+
+ /* Normalize the value that was read */
+
+ ACPI_DEBUG_EXEC(register_value =
+ ((register_value & bit_reg_info->access_bit_mask) >>
+ bit_reg_info->bit_position));
+
+ ACPI_DEBUG_PRINT((ACPI_DB_IO,
+ "Set bits: %8.8X actual %8.8X register %X\n", value,
+ register_value, bit_reg_info->parent_register));
return_ACPI_STATUS(status);
}
+ACPI_EXPORT_SYMBOL(acpi_set_register)
+
/*******************************************************************************
*
* FUNCTION: acpi_get_sleep_type_data
@@ -115,9 +488,8 @@
* state.
*
******************************************************************************/
-
acpi_status
-acpi_get_sleep_type_data(u8 sleep_state, u8 * sleep_type_a, u8 * sleep_type_b)
+acpi_get_sleep_type_data(u8 sleep_state, u8 *sleep_type_a, u8 *sleep_type_b)
{
acpi_status status = AE_OK;
struct acpi_evaluate_info *info;
@@ -219,639 +591,3 @@
}
ACPI_EXPORT_SYMBOL(acpi_get_sleep_type_data)
-
-/*******************************************************************************
- *
- * FUNCTION: acpi_hw_get_register_bit_mask
- *
- * PARAMETERS: register_id - Index of ACPI Register to access
- *
- * RETURN: The bitmask to be used when accessing the register
- *
- * DESCRIPTION: Map register_id into a register bitmask.
- *
- ******************************************************************************/
-struct acpi_bit_register_info *acpi_hw_get_bit_register_info(u32 register_id)
-{
- ACPI_FUNCTION_ENTRY();
-
- if (register_id > ACPI_BITREG_MAX) {
- ACPI_ERROR((AE_INFO, "Invalid BitRegister ID: %X",
- register_id));
- return (NULL);
- }
-
- return (&acpi_gbl_bit_register_info[register_id]);
-}
-
-/*******************************************************************************
- *
- * FUNCTION: acpi_get_register
- *
- * PARAMETERS: register_id - ID of ACPI bit_register to access
- * return_value - Value that was read from the register
- *
- * RETURN: Status and the value read from specified Register. Value
- * returned is normalized to bit0 (is shifted all the way right)
- *
- * DESCRIPTION: ACPI bit_register read function.
- *
- ******************************************************************************/
-
-acpi_status acpi_get_register_unlocked(u32 register_id, u32 * return_value)
-{
- u32 register_value = 0;
- struct acpi_bit_register_info *bit_reg_info;
- acpi_status status;
-
- ACPI_FUNCTION_TRACE(acpi_get_register);
-
- /* Get the info structure corresponding to the requested ACPI Register */
-
- bit_reg_info = acpi_hw_get_bit_register_info(register_id);
- if (!bit_reg_info) {
- return_ACPI_STATUS(AE_BAD_PARAMETER);
- }
-
- /* Read from the register */
-
- status = acpi_hw_register_read(bit_reg_info->parent_register,
- ®ister_value);
-
- if (ACPI_SUCCESS(status)) {
-
- /* Normalize the value that was read */
-
- register_value =
- ((register_value & bit_reg_info->access_bit_mask)
- >> bit_reg_info->bit_position);
-
- *return_value = register_value;
-
- ACPI_DEBUG_PRINT((ACPI_DB_IO, "Read value %8.8X register %X\n",
- register_value,
- bit_reg_info->parent_register));
- }
-
- return_ACPI_STATUS(status);
-}
-
-acpi_status acpi_get_register(u32 register_id, u32 * return_value)
-{
- acpi_status status;
- acpi_cpu_flags flags;
- flags = acpi_os_acquire_lock(acpi_gbl_hardware_lock);
- status = acpi_get_register_unlocked(register_id, return_value);
- acpi_os_release_lock(acpi_gbl_hardware_lock, flags);
- return status;
-}
-
-ACPI_EXPORT_SYMBOL(acpi_get_register)
-
-/*******************************************************************************
- *
- * FUNCTION: acpi_set_register
- *
- * PARAMETERS: register_id - ID of ACPI bit_register to access
- * Value - (only used on write) value to write to the
- * Register, NOT pre-normalized to the bit pos
- *
- * RETURN: Status
- *
- * DESCRIPTION: ACPI Bit Register write function.
- *
- ******************************************************************************/
-acpi_status acpi_set_register(u32 register_id, u32 value)
-{
- u32 register_value = 0;
- struct acpi_bit_register_info *bit_reg_info;
- acpi_status status;
- acpi_cpu_flags lock_flags;
-
- ACPI_FUNCTION_TRACE_U32(acpi_set_register, register_id);
-
- /* Get the info structure corresponding to the requested ACPI Register */
-
- bit_reg_info = acpi_hw_get_bit_register_info(register_id);
- if (!bit_reg_info) {
- ACPI_ERROR((AE_INFO, "Bad ACPI HW RegisterId: %X",
- register_id));
- return_ACPI_STATUS(AE_BAD_PARAMETER);
- }
-
- lock_flags = acpi_os_acquire_lock(acpi_gbl_hardware_lock);
-
- /* Always do a register read first so we can insert the new bits */
-
- status = acpi_hw_register_read(bit_reg_info->parent_register,
- ®ister_value);
- if (ACPI_FAILURE(status)) {
- goto unlock_and_exit;
- }
-
- /*
- * Decode the Register ID
- * Register ID = [Register block ID] | [bit ID]
- *
- * Check bit ID to fine locate Register offset.
- * Check Mask to determine Register offset, and then read-write.
- */
- switch (bit_reg_info->parent_register) {
- case ACPI_REGISTER_PM1_STATUS:
-
- /*
- * Status Registers are different from the rest. Clear by
- * writing 1, and writing 0 has no effect. So, the only relevant
- * information is the single bit we're interested in, all others should
- * be written as 0 so they will be left unchanged.
- */
- value = ACPI_REGISTER_PREPARE_BITS(value,
- bit_reg_info->bit_position,
- bit_reg_info->
- access_bit_mask);
- if (value) {
- status = acpi_hw_register_write(ACPI_REGISTER_PM1_STATUS,
- (u16) value);
- register_value = 0;
- }
- break;
-
- case ACPI_REGISTER_PM1_ENABLE:
-
- ACPI_REGISTER_INSERT_VALUE(register_value,
- bit_reg_info->bit_position,
- bit_reg_info->access_bit_mask,
- value);
-
- status = acpi_hw_register_write(ACPI_REGISTER_PM1_ENABLE,
- (u16) register_value);
- break;
-
- case ACPI_REGISTER_PM1_CONTROL:
-
- /*
- * Write the PM1 Control register.
- * Note that at this level, the fact that there are actually TWO
- * registers (A and B - and B may not exist) is abstracted.
- */
- ACPI_DEBUG_PRINT((ACPI_DB_IO, "PM1 control: Read %X\n",
- register_value));
-
- ACPI_REGISTER_INSERT_VALUE(register_value,
- bit_reg_info->bit_position,
- bit_reg_info->access_bit_mask,
- value);
-
- status = acpi_hw_register_write(ACPI_REGISTER_PM1_CONTROL,
- (u16) register_value);
- break;
-
- case ACPI_REGISTER_PM2_CONTROL:
-
- status = acpi_hw_register_read(ACPI_REGISTER_PM2_CONTROL,
- ®ister_value);
- if (ACPI_FAILURE(status)) {
- goto unlock_and_exit;
- }
-
- ACPI_DEBUG_PRINT((ACPI_DB_IO,
- "PM2 control: Read %X from %8.8X%8.8X\n",
- register_value,
- ACPI_FORMAT_UINT64(acpi_gbl_FADT.
- xpm2_control_block.
- address)));
-
- ACPI_REGISTER_INSERT_VALUE(register_value,
- bit_reg_info->bit_position,
- bit_reg_info->access_bit_mask,
- value);
-
- ACPI_DEBUG_PRINT((ACPI_DB_IO,
- "About to write %4.4X to %8.8X%8.8X\n",
- register_value,
- ACPI_FORMAT_UINT64(acpi_gbl_FADT.
- xpm2_control_block.
- address)));
-
- status = acpi_hw_register_write(ACPI_REGISTER_PM2_CONTROL,
- (u8) (register_value));
- break;
-
- default:
- break;
- }
-
- unlock_and_exit:
-
- acpi_os_release_lock(acpi_gbl_hardware_lock, lock_flags);
-
- /* Normalize the value that was read */
-
- ACPI_DEBUG_EXEC(register_value =
- ((register_value & bit_reg_info->access_bit_mask) >>
- bit_reg_info->bit_position));
-
- ACPI_DEBUG_PRINT((ACPI_DB_IO,
- "Set bits: %8.8X actual %8.8X register %X\n", value,
- register_value, bit_reg_info->parent_register));
- return_ACPI_STATUS(status);
-}
-
-ACPI_EXPORT_SYMBOL(acpi_set_register)
-
-/******************************************************************************
- *
- * FUNCTION: acpi_hw_register_read
- *
- * PARAMETERS: register_id - ACPI Register ID
- * return_value - Where the register value is returned
- *
- * RETURN: Status and the value read.
- *
- * DESCRIPTION: Read from the specified ACPI register
- *
- ******************************************************************************/
-acpi_status
-acpi_hw_register_read(u32 register_id, u32 * return_value)
-{
- u32 value1 = 0;
- u32 value2 = 0;
- acpi_status status;
-
- ACPI_FUNCTION_TRACE(hw_register_read);
-
- switch (register_id) {
- case ACPI_REGISTER_PM1_STATUS: /* 16-bit access */
-
- status =
- acpi_hw_low_level_read(16, &value1,
- &acpi_gbl_FADT.xpm1a_event_block);
- if (ACPI_FAILURE(status)) {
- goto exit;
- }
-
- /* PM1B is optional */
-
- status =
- acpi_hw_low_level_read(16, &value2,
- &acpi_gbl_FADT.xpm1b_event_block);
- value1 |= value2;
- break;
-
- case ACPI_REGISTER_PM1_ENABLE: /* 16-bit access */
-
- status =
- acpi_hw_low_level_read(16, &value1, &acpi_gbl_xpm1a_enable);
- if (ACPI_FAILURE(status)) {
- goto exit;
- }
-
- /* PM1B is optional */
-
- status =
- acpi_hw_low_level_read(16, &value2, &acpi_gbl_xpm1b_enable);
- value1 |= value2;
- break;
-
- case ACPI_REGISTER_PM1_CONTROL: /* 16-bit access */
-
- status =
- acpi_hw_low_level_read(16, &value1,
- &acpi_gbl_FADT.xpm1a_control_block);
- if (ACPI_FAILURE(status)) {
- goto exit;
- }
-
- status =
- acpi_hw_low_level_read(16, &value2,
- &acpi_gbl_FADT.xpm1b_control_block);
- value1 |= value2;
- break;
-
- case ACPI_REGISTER_PM2_CONTROL: /* 8-bit access */
-
- status =
- acpi_hw_low_level_read(8, &value1,
- &acpi_gbl_FADT.xpm2_control_block);
- break;
-
- case ACPI_REGISTER_PM_TIMER: /* 32-bit access */
-
- status =
- acpi_hw_low_level_read(32, &value1,
- &acpi_gbl_FADT.xpm_timer_block);
- break;
-
- case ACPI_REGISTER_SMI_COMMAND_BLOCK: /* 8-bit access */
-
- status =
- acpi_os_read_port(acpi_gbl_FADT.smi_command, &value1, 8);
- break;
-
- default:
- ACPI_ERROR((AE_INFO, "Unknown Register ID: %X", register_id));
- status = AE_BAD_PARAMETER;
- break;
- }
-
- exit:
-
- if (ACPI_SUCCESS(status)) {
- *return_value = value1;
- }
-
- return_ACPI_STATUS(status);
-}
-
-/******************************************************************************
- *
- * FUNCTION: acpi_hw_register_write
- *
- * PARAMETERS: register_id - ACPI Register ID
- * Value - The value to write
- *
- * RETURN: Status
- *
- * DESCRIPTION: Write to the specified ACPI register
- *
- * NOTE: In accordance with the ACPI specification, this function automatically
- * preserves the value of the following bits, meaning that these bits cannot be
- * changed via this interface:
- *
- * PM1_CONTROL[0] = SCI_EN
- * PM1_CONTROL[9]
- * PM1_STATUS[11]
- *
- * ACPI References:
- * 1) Hardware Ignored Bits: When software writes to a register with ignored
- * bit fields, it preserves the ignored bit fields
- * 2) SCI_EN: OSPM always preserves this bit position
- *
- ******************************************************************************/
-
-acpi_status acpi_hw_register_write(u32 register_id, u32 value)
-{
- acpi_status status;
- u32 read_value;
-
- ACPI_FUNCTION_TRACE(hw_register_write);
-
- switch (register_id) {
- case ACPI_REGISTER_PM1_STATUS: /* 16-bit access */
-
- /* Perform a read first to preserve certain bits (per ACPI spec) */
-
- status = acpi_hw_register_read(ACPI_REGISTER_PM1_STATUS,
- &read_value);
- if (ACPI_FAILURE(status)) {
- goto exit;
- }
-
- /* Insert the bits to be preserved */
-
- ACPI_INSERT_BITS(value, ACPI_PM1_STATUS_PRESERVED_BITS,
- read_value);
-
- /* Now we can write the data */
-
- status =
- acpi_hw_low_level_write(16, value,
- &acpi_gbl_FADT.xpm1a_event_block);
- if (ACPI_FAILURE(status)) {
- goto exit;
- }
-
- /* PM1B is optional */
-
- status =
- acpi_hw_low_level_write(16, value,
- &acpi_gbl_FADT.xpm1b_event_block);
- break;
-
- case ACPI_REGISTER_PM1_ENABLE: /* 16-bit access */
-
- status =
- acpi_hw_low_level_write(16, value, &acpi_gbl_xpm1a_enable);
- if (ACPI_FAILURE(status)) {
- goto exit;
- }
-
- /* PM1B is optional */
-
- status =
- acpi_hw_low_level_write(16, value, &acpi_gbl_xpm1b_enable);
- break;
-
- case ACPI_REGISTER_PM1_CONTROL: /* 16-bit access */
-
- /*
- * Perform a read first to preserve certain bits (per ACPI spec)
- */
- status = acpi_hw_register_read(ACPI_REGISTER_PM1_CONTROL,
- &read_value);
- if (ACPI_FAILURE(status)) {
- goto exit;
- }
-
- /* Insert the bits to be preserved */
-
- ACPI_INSERT_BITS(value, ACPI_PM1_CONTROL_PRESERVED_BITS,
- read_value);
-
- /* Now we can write the data */
-
- status =
- acpi_hw_low_level_write(16, value,
- &acpi_gbl_FADT.xpm1a_control_block);
- if (ACPI_FAILURE(status)) {
- goto exit;
- }
-
- status =
- acpi_hw_low_level_write(16, value,
- &acpi_gbl_FADT.xpm1b_control_block);
- break;
-
- case ACPI_REGISTER_PM1A_CONTROL: /* 16-bit access */
-
- status =
- acpi_hw_low_level_write(16, value,
- &acpi_gbl_FADT.xpm1a_control_block);
- break;
-
- case ACPI_REGISTER_PM1B_CONTROL: /* 16-bit access */
-
- status =
- acpi_hw_low_level_write(16, value,
- &acpi_gbl_FADT.xpm1b_control_block);
- break;
-
- case ACPI_REGISTER_PM2_CONTROL: /* 8-bit access */
-
- status =
- acpi_hw_low_level_write(8, value,
- &acpi_gbl_FADT.xpm2_control_block);
- break;
-
- case ACPI_REGISTER_PM_TIMER: /* 32-bit access */
-
- status =
- acpi_hw_low_level_write(32, value,
- &acpi_gbl_FADT.xpm_timer_block);
- break;
-
- case ACPI_REGISTER_SMI_COMMAND_BLOCK: /* 8-bit access */
-
- /* SMI_CMD is currently always in IO space */
-
- status =
- acpi_os_write_port(acpi_gbl_FADT.smi_command, value, 8);
- break;
-
- default:
- status = AE_BAD_PARAMETER;
- break;
- }
-
- exit:
- return_ACPI_STATUS(status);
-}
-
-/******************************************************************************
- *
- * FUNCTION: acpi_hw_low_level_read
- *
- * PARAMETERS: Width - 8, 16, or 32
- * Value - Where the value is returned
- * Reg - GAS register structure
- *
- * RETURN: Status
- *
- * DESCRIPTION: Read from either memory or IO space.
- *
- ******************************************************************************/
-
-acpi_status
-acpi_hw_low_level_read(u32 width, u32 * value, struct acpi_generic_address *reg)
-{
- u64 address;
- acpi_status status;
-
- ACPI_FUNCTION_NAME(hw_low_level_read);
-
- /*
- * Must have a valid pointer to a GAS structure, and
- * a non-zero address within. However, don't return an error
- * because the PM1A/B code must not fail if B isn't present.
- */
- if (!reg) {
- return (AE_OK);
- }
-
- /* Get a local copy of the address. Handles possible alignment issues */
-
- ACPI_MOVE_64_TO_64(&address, ®->address);
- if (!address) {
- return (AE_OK);
- }
- *value = 0;
-
- /*
- * Two address spaces supported: Memory or IO.
- * PCI_Config is not supported here because the GAS struct is insufficient
- */
- switch (reg->space_id) {
- case ACPI_ADR_SPACE_SYSTEM_MEMORY:
-
- status = acpi_os_read_memory((acpi_physical_address) address,
- value, width);
- break;
-
- case ACPI_ADR_SPACE_SYSTEM_IO:
-
- status =
- acpi_os_read_port((acpi_io_address) address, value, width);
- break;
-
- default:
- ACPI_ERROR((AE_INFO,
- "Unsupported address space: %X", reg->space_id));
- return (AE_BAD_PARAMETER);
- }
-
- ACPI_DEBUG_PRINT((ACPI_DB_IO,
- "Read: %8.8X width %2d from %8.8X%8.8X (%s)\n",
- *value, width, ACPI_FORMAT_UINT64(address),
- acpi_ut_get_region_name(reg->space_id)));
-
- return (status);
-}
-
-/******************************************************************************
- *
- * FUNCTION: acpi_hw_low_level_write
- *
- * PARAMETERS: Width - 8, 16, or 32
- * Value - To be written
- * Reg - GAS register structure
- *
- * RETURN: Status
- *
- * DESCRIPTION: Write to either memory or IO space.
- *
- ******************************************************************************/
-
-acpi_status
-acpi_hw_low_level_write(u32 width, u32 value, struct acpi_generic_address * reg)
-{
- u64 address;
- acpi_status status;
-
- ACPI_FUNCTION_NAME(hw_low_level_write);
-
- /*
- * Must have a valid pointer to a GAS structure, and
- * a non-zero address within. However, don't return an error
- * because the PM1A/B code must not fail if B isn't present.
- */
- if (!reg) {
- return (AE_OK);
- }
-
- /* Get a local copy of the address. Handles possible alignment issues */
-
- ACPI_MOVE_64_TO_64(&address, ®->address);
- if (!address) {
- return (AE_OK);
- }
-
- /*
- * Two address spaces supported: Memory or IO.
- * PCI_Config is not supported here because the GAS struct is insufficient
- */
- switch (reg->space_id) {
- case ACPI_ADR_SPACE_SYSTEM_MEMORY:
-
- status = acpi_os_write_memory((acpi_physical_address) address,
- value, width);
- break;
-
- case ACPI_ADR_SPACE_SYSTEM_IO:
-
- status = acpi_os_write_port((acpi_io_address) address, value,
- width);
- break;
-
- default:
- ACPI_ERROR((AE_INFO,
- "Unsupported address space: %X", reg->space_id));
- return (AE_BAD_PARAMETER);
- }
-
- ACPI_DEBUG_PRINT((ACPI_DB_IO,
- "Wrote: %8.8X width %2d to %8.8X%8.8X (%s)\n",
- value, width, ACPI_FORMAT_UINT64(address),
- acpi_ut_get_region_name(reg->space_id)));
-
- return (status);
-}
diff --git a/drivers/acpi/namespace/nsaccess.c b/drivers/acpi/acpica/nsaccess.c
similarity index 98%
rename from drivers/acpi/namespace/nsaccess.c
rename to drivers/acpi/acpica/nsaccess.c
index c39a7f6..88303eb 100644
--- a/drivers/acpi/namespace/nsaccess.c
+++ b/drivers/acpi/acpica/nsaccess.c
@@ -42,9 +42,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/amlcode.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acdispat.h>
+#include "accommon.h"
+#include "amlcode.h"
+#include "acnamesp.h"
+#include "acdispat.h"
#define _COMPONENT ACPI_NAMESPACE
ACPI_MODULE_NAME("nsaccess")
@@ -165,12 +166,9 @@
obj_desc->method.method_flags =
AML_METHOD_INTERNAL_ONLY;
-
-#ifndef ACPI_DUMP_APP
obj_desc->method.implementation =
acpi_ut_osi_implementation;
#endif
-#endif
break;
case ACPI_TYPE_INTEGER:
@@ -521,11 +519,11 @@
}
/*
- * Search namespace for each segment of the name. Loop through and
+ * Search namespace for each segment of the name. Loop through and
* verify (or add to the namespace) each name segment.
*
* The object type is significant only at the last name
- * segment. (We don't care about the types along the path, only
+ * segment. (We don't care about the types along the path, only
* the type of the final target object.)
*/
this_search_type = ACPI_TYPE_ANY;
@@ -591,6 +589,10 @@
* segments).
*/
if (this_node->type == ACPI_TYPE_LOCAL_ALIAS) {
+ if (!this_node->object) {
+ return_ACPI_STATUS(AE_NOT_EXIST);
+ }
+
if (acpi_ns_opens_scope
(((struct acpi_namespace_node *)this_node->
object)->type)) {
diff --git a/drivers/acpi/namespace/nsalloc.c b/drivers/acpi/acpica/nsalloc.c
similarity index 99%
rename from drivers/acpi/namespace/nsalloc.c
rename to drivers/acpi/acpica/nsalloc.c
index 3a1740a..f976d84 100644
--- a/drivers/acpi/namespace/nsalloc.c
+++ b/drivers/acpi/acpica/nsalloc.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_NAMESPACE
ACPI_MODULE_NAME("nsalloc")
diff --git a/drivers/acpi/namespace/nsdump.c b/drivers/acpi/acpica/nsdump.c
similarity index 99%
rename from drivers/acpi/namespace/nsdump.c
rename to drivers/acpi/acpica/nsdump.c
index cc0ae39..0da33c8 100644
--- a/drivers/acpi/namespace/nsdump.c
+++ b/drivers/acpi/acpica/nsdump.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_NAMESPACE
ACPI_MODULE_NAME("nsdump")
diff --git a/drivers/acpi/namespace/nsdumpdv.c b/drivers/acpi/acpica/nsdumpdv.c
similarity index 98%
rename from drivers/acpi/namespace/nsdumpdv.c
rename to drivers/acpi/acpica/nsdumpdv.c
index 428f50f..41994fe 100644
--- a/drivers/acpi/namespace/nsdumpdv.c
+++ b/drivers/acpi/acpica/nsdumpdv.c
@@ -42,6 +42,7 @@
*/
#include <acpi/acpi.h>
+#include "accommon.h"
/* TBD: This entire module is apparently obsolete and should be removed */
@@ -49,7 +50,7 @@
ACPI_MODULE_NAME("nsdumpdv")
#ifdef ACPI_OBSOLETE_FUNCTIONS
#if defined(ACPI_DEBUG_OUTPUT) || defined(ACPI_DEBUGGER)
-#include <acpi/acnamesp.h>
+#include "acnamesp.h"
/*******************************************************************************
*
* FUNCTION: acpi_ns_dump_one_device
diff --git a/drivers/acpi/namespace/nseval.c b/drivers/acpi/acpica/nseval.c
similarity index 80%
rename from drivers/acpi/namespace/nseval.c
rename to drivers/acpi/acpica/nseval.c
index 4cdf03a..0f3d5f9 100644
--- a/drivers/acpi/namespace/nseval.c
+++ b/drivers/acpi/acpica/nseval.c
@@ -42,9 +42,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
-#include <acpi/acinterp.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acparser.h"
+#include "acinterp.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_NAMESPACE
ACPI_MODULE_NAME("nseval")
@@ -89,6 +90,7 @@
/* Initialize the return value to an invalid object */
info->return_object = NULL;
+ info->param_count = 0;
/*
* Get the actual namespace node for the target object. Handles these cases:
@@ -141,41 +143,17 @@
return_ACPI_STATUS(AE_NULL_OBJECT);
}
- /*
- * Calculate the number of arguments being passed to the method
- */
+ /* Count the number of arguments being passed to the method */
- info->param_count = 0;
if (info->parameters) {
- while (info->parameters[info->param_count])
+ while (info->parameters[info->param_count]) {
+ if (info->param_count > ACPI_METHOD_MAX_ARG) {
+ return_ACPI_STATUS(AE_LIMIT);
+ }
info->param_count++;
+ }
}
- /*
- * Warning if too few or too many arguments have been passed by the
- * caller. We don't want to abort here with an error because an
- * incorrect number of arguments may not cause the method to fail.
- * However, the method will fail if there are too few arguments passed
- * and the method attempts to use one of the missing ones.
- */
-
- if (info->param_count < info->obj_desc->method.param_count) {
- ACPI_WARNING((AE_INFO,
- "Insufficient arguments - "
- "method [%4.4s] needs %d, found %d",
- acpi_ut_get_node_name(info->resolved_node),
- info->obj_desc->method.param_count,
- info->param_count));
- } else if (info->param_count >
- info->obj_desc->method.param_count) {
- ACPI_WARNING((AE_INFO,
- "Excess arguments - "
- "method [%4.4s] needs %d, found %d",
- acpi_ut_get_node_name(info->
- resolved_node),
- info->obj_desc->method.param_count,
- info->param_count));
- }
ACPI_DUMP_PATHNAME(info->resolved_node, "Execute Method:",
ACPI_LV_INFO, _COMPONENT);
@@ -264,32 +242,13 @@
}
}
- /* Validation of return values for ACPI-predefined methods and objects */
-
- if ((status == AE_OK) || (status == AE_CTRL_RETURN_VALUE)) {
- /*
- * If this is the first evaluation, check the return value. This
- * ensures that any warnings will only be emitted during the very
- * first evaluation of the object.
- */
- if (!(node->flags & ANOBJ_EVALUATED)) {
- /*
- * Check for a predefined ACPI name. If found, validate the
- * returned object.
- *
- * Note: Ignore return status for now, emit warnings if there are
- * problems with the returned object. May change later to abort
- * the method on invalid return object.
- */
- (void)acpi_ns_check_predefined_names(node,
- info->
- return_object);
- }
-
- /* Mark the node as having been evaluated */
-
- node->flags |= ANOBJ_EVALUATED;
- }
+ /*
+ * Check input argument count against the ASL-defined count for a method.
+ * Also check predefined names: argument count and return value against
+ * the ACPI specification. Some incorrect return value types are repaired.
+ */
+ (void)acpi_ns_check_predefined_names(node, info->param_count,
+ status, &info->return_object);
/* Check if there is a return value that must be dealt with */
diff --git a/drivers/acpi/namespace/nsinit.c b/drivers/acpi/acpica/nsinit.c
similarity index 99%
rename from drivers/acpi/namespace/nsinit.c
rename to drivers/acpi/acpica/nsinit.c
index e4c5751..13501cb 100644
--- a/drivers/acpi/namespace/nsinit.c
+++ b/drivers/acpi/acpica/nsinit.c
@@ -42,9 +42,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acdispat.h>
-#include <acpi/acinterp.h>
+#include "accommon.h"
+#include "acnamesp.h"
+#include "acdispat.h"
+#include "acinterp.h"
#include <linux/nmi.h>
#define _COMPONENT ACPI_NAMESPACE
diff --git a/drivers/acpi/namespace/nsload.c b/drivers/acpi/acpica/nsload.c
similarity index 98%
rename from drivers/acpi/namespace/nsload.c
rename to drivers/acpi/acpica/nsload.c
index a4a412b..a0ba9e1 100644
--- a/drivers/acpi/namespace/nsload.c
+++ b/drivers/acpi/acpica/nsload.c
@@ -42,9 +42,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acdispat.h>
-#include <acpi/actables.h>
+#include "accommon.h"
+#include "acnamesp.h"
+#include "acdispat.h"
+#include "actables.h"
#define _COMPONENT ACPI_NAMESPACE
ACPI_MODULE_NAME("nsload")
diff --git a/drivers/acpi/namespace/nsnames.c b/drivers/acpi/acpica/nsnames.c
similarity index 98%
rename from drivers/acpi/namespace/nsnames.c
rename to drivers/acpi/acpica/nsnames.c
index 42a39a7..ae3dc10 100644
--- a/drivers/acpi/namespace/nsnames.c
+++ b/drivers/acpi/acpica/nsnames.c
@@ -42,8 +42,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/amlcode.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "amlcode.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_NAMESPACE
ACPI_MODULE_NAME("nsnames")
diff --git a/drivers/acpi/namespace/nsobject.c b/drivers/acpi/acpica/nsobject.c
similarity index 99%
rename from drivers/acpi/namespace/nsobject.c
rename to drivers/acpi/acpica/nsobject.c
index 15fe09e..08a97a5 100644
--- a/drivers/acpi/namespace/nsobject.c
+++ b/drivers/acpi/acpica/nsobject.c
@@ -43,7 +43,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_NAMESPACE
ACPI_MODULE_NAME("nsobject")
diff --git a/drivers/acpi/namespace/nsparse.c b/drivers/acpi/acpica/nsparse.c
similarity index 98%
rename from drivers/acpi/namespace/nsparse.c
rename to drivers/acpi/acpica/nsparse.c
index a82271a..b9e8d00 100644
--- a/drivers/acpi/namespace/nsparse.c
+++ b/drivers/acpi/acpica/nsparse.c
@@ -42,10 +42,11 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acparser.h>
-#include <acpi/acdispat.h>
-#include <acpi/actables.h>
+#include "accommon.h"
+#include "acnamesp.h"
+#include "acparser.h"
+#include "acdispat.h"
+#include "actables.h"
#define _COMPONENT ACPI_NAMESPACE
ACPI_MODULE_NAME("nsparse")
diff --git a/drivers/acpi/namespace/nspredef.c b/drivers/acpi/acpica/nspredef.c
similarity index 76%
rename from drivers/acpi/namespace/nspredef.c
rename to drivers/acpi/acpica/nspredef.c
index 0f17cf0..4527032 100644
--- a/drivers/acpi/namespace/nspredef.c
+++ b/drivers/acpi/acpica/nspredef.c
@@ -43,8 +43,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acpredef.h>
+#include "accommon.h"
+#include "acnamesp.h"
+#include "acpredef.h"
#define _COMPONENT ACPI_NAMESPACE
ACPI_MODULE_NAME("nspredef")
@@ -72,7 +73,7 @@
/* Local prototypes */
static acpi_status
acpi_ns_check_package(char *pathname,
- union acpi_operand_object *return_object,
+ union acpi_operand_object **return_object_ptr,
const union acpi_predefined_info *predefined);
static acpi_status
@@ -82,13 +83,18 @@
static acpi_status
acpi_ns_check_object_type(char *pathname,
- union acpi_operand_object *return_object,
+ union acpi_operand_object **return_object_ptr,
u32 expected_btypes, u32 package_index);
static acpi_status
acpi_ns_check_reference(char *pathname,
union acpi_operand_object *return_object);
+static acpi_status
+acpi_ns_repair_object(u32 expected_btypes,
+ u32 package_index,
+ union acpi_operand_object **return_object_ptr);
+
/*
* Names for the types that can be returned by the predefined objects.
* Used for warning messages. Must be in the same order as the ACPI_RTYPEs
@@ -108,8 +114,8 @@
* FUNCTION: acpi_ns_check_predefined_names
*
* PARAMETERS: Node - Namespace node for the method/object
- * return_object - Object returned from the evaluation of this
- * method/object
+ * return_object_ptr - Pointer to the object returned from the
+ * evaluation of a method or object
*
* RETURN: Status
*
@@ -119,8 +125,11 @@
acpi_status
acpi_ns_check_predefined_names(struct acpi_namespace_node *node,
- union acpi_operand_object *return_object)
+ u32 user_param_count,
+ acpi_status return_status,
+ union acpi_operand_object **return_object_ptr)
{
+ union acpi_operand_object *return_object = *return_object_ptr;
acpi_status status = AE_OK;
const union acpi_predefined_info *predefined;
char *pathname;
@@ -128,12 +137,6 @@
/* Match the name for this method/object against the predefined list */
predefined = acpi_ns_check_for_predefined_name(node);
- if (!predefined) {
-
- /* Name was not one of the predefined names */
-
- return (AE_OK);
- }
/* Get the full pathname to the object, for use in error messages */
@@ -143,10 +146,37 @@
}
/*
- * Check that the parameter count for this method is in accordance
- * with the ACPI specification.
+ * Check that the parameter count for this method matches the ASL
+ * definition. For predefined names, ensure that both the caller and
+ * the method itself are in accordance with the ACPI specification.
*/
- acpi_ns_check_parameter_count(pathname, node, predefined);
+ acpi_ns_check_parameter_count(pathname, node, user_param_count,
+ predefined);
+
+ /* If not a predefined name, we cannot validate the return object */
+
+ if (!predefined) {
+ goto exit;
+ }
+
+ /* If the method failed, we cannot validate the return object */
+
+ if ((return_status != AE_OK) && (return_status != AE_CTRL_RETURN_VALUE)) {
+ goto exit;
+ }
+
+ /*
+ * Only validate the return value on the first successful evaluation of
+ * the method. This ensures that any warnings will only be emitted during
+ * the very first evaluation of the method/object.
+ */
+ if (node->flags & ANOBJ_EVALUATED) {
+ goto exit;
+ }
+
+ /* Mark the node as having been successfully evaluated */
+
+ node->flags |= ANOBJ_EVALUATED;
/*
* If there is no return value, check if we require a return value for
@@ -171,7 +201,7 @@
* We have a return value, but if one wasn't expected, just exit, this is
* not a problem
*
- * For example, if "Implicit return value" is enabled, methods will
+ * For example, if the "Implicit Return" feature is enabled, methods will
* always return a value
*/
if (!predefined->info.expected_btypes) {
@@ -182,7 +212,7 @@
* Check that the type of the return object is what is expected for
* this predefined name
*/
- status = acpi_ns_check_object_type(pathname, return_object,
+ status = acpi_ns_check_object_type(pathname, return_object_ptr,
predefined->info.expected_btypes,
ACPI_NOT_PACKAGE);
if (ACPI_FAILURE(status)) {
@@ -193,11 +223,12 @@
if (ACPI_GET_OBJECT_TYPE(return_object) == ACPI_TYPE_PACKAGE) {
status =
- acpi_ns_check_package(pathname, return_object, predefined);
+ acpi_ns_check_package(pathname, return_object_ptr,
+ predefined);
}
exit:
- if (pathname) {
+ if (pathname != predefined->info.name) {
ACPI_FREE(pathname);
}
@@ -210,6 +241,7 @@
*
* PARAMETERS: Pathname - Full pathname to the node (for error msgs)
* Node - Namespace node for the method/object
+ * user_param_count - Number of args passed in by the caller
* Predefined - Pointer to entry in predefined name table
*
* RETURN: None
@@ -223,32 +255,76 @@
void
acpi_ns_check_parameter_count(char *pathname,
struct acpi_namespace_node *node,
+ u32 user_param_count,
const union acpi_predefined_info *predefined)
{
u32 param_count;
u32 required_params_current;
u32 required_params_old;
- /*
- * Check that the ASL-defined parameter count is what is expected for
- * this predefined name.
- *
- * Methods have 0-7 parameters. All other types have zero.
- */
+ /* Methods have 0-7 parameters. All other types have zero. */
+
param_count = 0;
if (node->type == ACPI_TYPE_METHOD) {
param_count = node->object->method.param_count;
}
- /* Validate parameter count - allow two different legal counts (_SCP) */
+ /* Argument count check for non-predefined methods/objects */
+
+ if (!predefined) {
+ /*
+ * Warning if too few or too many arguments have been passed by the
+ * caller. An incorrect number of arguments may not cause the method
+ * to fail. However, the method will fail if there are too few
+ * arguments and the method attempts to use one of the missing ones.
+ */
+ if (user_param_count < param_count) {
+ ACPI_WARNING((AE_INFO,
+ "%s: Insufficient arguments - needs %d, found %d",
+ pathname, param_count, user_param_count));
+ } else if (user_param_count > param_count) {
+ ACPI_WARNING((AE_INFO,
+ "%s: Excess arguments - needs %d, found %d",
+ pathname, param_count, user_param_count));
+ }
+ return;
+ }
+
+ /* Allow two different legal argument counts (_SCP, etc.) */
required_params_current = predefined->info.param_count & 0x0F;
required_params_old = predefined->info.param_count >> 4;
+ if (user_param_count != ACPI_UINT32_MAX) {
+
+ /* Validate the user-supplied parameter count */
+
+ if ((user_param_count != required_params_current) &&
+ (user_param_count != required_params_old)) {
+ ACPI_WARNING((AE_INFO,
+ "%s: Parameter count mismatch - caller passed %d, ACPI requires %d",
+ pathname, user_param_count,
+ required_params_current));
+ }
+ }
+
+ /*
+ * Only validate the argument count on the first successful evaluation of
+ * the method. This ensures that any warnings will only be emitted during
+ * the very first evaluation of the method/object.
+ */
+ if (node->flags & ANOBJ_EVALUATED) {
+ return;
+ }
+
+ /*
+ * Check that the ASL-defined parameter count is what is expected for
+ * this predefined name.
+ */
if ((param_count != required_params_current) &&
(param_count != required_params_old)) {
ACPI_WARNING((AE_INFO,
- "%s: Parameter count mismatch - ASL declared %d, expected %d",
+ "%s: Parameter count mismatch - ASL declared %d, ACPI requires %d",
pathname, param_count, required_params_current));
}
}
@@ -307,8 +383,8 @@
* FUNCTION: acpi_ns_check_package
*
* PARAMETERS: Pathname - Full pathname to the node (for error msgs)
- * return_object - Object returned from the evaluation of a
- * method or object
+ * return_object_ptr - Pointer to the object returned from the
+ * evaluation of a method or object
* Predefined - Pointer to entry in predefined name table
*
* RETURN: Status
@@ -320,9 +396,10 @@
static acpi_status
acpi_ns_check_package(char *pathname,
- union acpi_operand_object *return_object,
+ union acpi_operand_object **return_object_ptr,
const union acpi_predefined_info *predefined)
{
+ union acpi_operand_object *return_object = *return_object_ptr;
const union acpi_predefined_info *package;
union acpi_operand_object *sub_package;
union acpi_operand_object **elements;
@@ -408,7 +485,7 @@
* elements must be of the same type
*/
for (i = 0; i < count; i++) {
- status = acpi_ns_check_object_type(pathname, *elements,
+ status = acpi_ns_check_object_type(pathname, elements,
package->ret_info.
object_type1, i);
if (ACPI_FAILURE(status)) {
@@ -441,7 +518,7 @@
status =
acpi_ns_check_object_type(pathname,
- *elements,
+ elements,
package->
ret_info3.
object_type[i],
@@ -454,7 +531,7 @@
status =
acpi_ns_check_object_type(pathname,
- *elements,
+ elements,
package->
ret_info3.
tail_object_type,
@@ -471,7 +548,7 @@
/* First element is the (Integer) count of sub-packages to follow */
- status = acpi_ns_check_object_type(pathname, *elements,
+ status = acpi_ns_check_object_type(pathname, elements,
ACPI_RTYPE_INTEGER, 0);
if (ACPI_FAILURE(status)) {
return (status);
@@ -509,7 +586,7 @@
/* Each sub-object must be of type Package */
status =
- acpi_ns_check_object_type(pathname, sub_package,
+ acpi_ns_check_object_type(pathname, &sub_package,
ACPI_RTYPE_PACKAGE, i);
if (ACPI_FAILURE(status)) {
return (status);
@@ -567,12 +644,8 @@
for (j = 0; j < expected_count; j++) {
status =
acpi_ns_check_object_type(pathname,
- sub_elements
- [j],
- package->
- ret_info2.
- object_type
- [j], j);
+ &sub_elements[j],
+ package->ret_info2.object_type[j], j);
if (ACPI_FAILURE(status)) {
return (status);
}
@@ -611,7 +684,7 @@
status =
acpi_ns_check_object_type(pathname,
- *sub_elements,
+ sub_elements,
ACPI_RTYPE_INTEGER,
0);
if (ACPI_FAILURE(status)) {
@@ -708,7 +781,7 @@
* The second group can have a count of zero.
*/
for (i = 0; i < count1; i++) {
- status = acpi_ns_check_object_type(pathname, *this_element,
+ status = acpi_ns_check_object_type(pathname, this_element,
type1, i);
if (ACPI_FAILURE(status)) {
return (status);
@@ -717,7 +790,7 @@
}
for (i = 0; i < count2; i++) {
- status = acpi_ns_check_object_type(pathname, *this_element,
+ status = acpi_ns_check_object_type(pathname, this_element,
type2, (i + count1));
if (ACPI_FAILURE(status)) {
return (status);
@@ -733,8 +806,8 @@
* FUNCTION: acpi_ns_check_object_type
*
* PARAMETERS: Pathname - Full pathname to the node (for error msgs)
- * return_object - Object return from the execution of this
- * method/object
+ * return_object_ptr - Pointer to the object returned from the
+ * evaluation of a method or object
* expected_btypes - Bitmap of expected return type(s)
* package_index - Index of object within parent package (if
* applicable - ACPI_NOT_PACKAGE otherwise)
@@ -748,9 +821,10 @@
static acpi_status
acpi_ns_check_object_type(char *pathname,
- union acpi_operand_object *return_object,
+ union acpi_operand_object **return_object_ptr,
u32 expected_btypes, u32 package_index)
{
+ union acpi_operand_object *return_object = *return_object_ptr;
acpi_status status = AE_OK;
u32 return_btype;
char type_buffer[48]; /* Room for 5 types */
@@ -814,6 +888,14 @@
/* Is the object one of the expected types? */
if (!(return_btype & expected_btypes)) {
+
+ /* Type mismatch -- attempt repair of the returned object */
+
+ status = acpi_ns_repair_object(expected_btypes, package_index,
+ return_object_ptr);
+ if (ACPI_SUCCESS(status)) {
+ return (status);
+ }
goto type_error_exit;
}
@@ -898,3 +980,86 @@
return (AE_AML_OPERAND_TYPE);
}
+
+/*******************************************************************************
+ *
+ * FUNCTION: acpi_ns_repair_object
+ *
+ * PARAMETERS: Pathname - Full pathname to the node (for error msgs)
+ * package_index - Used to determine if target is in a package
+ * return_object_ptr - Pointer to the object returned from the
+ * evaluation of a method or object
+ *
+ * RETURN: Status. AE_OK if repair was successful.
+ *
+ * DESCRIPTION: Attempt to repair/convert a return object of a type that was
+ * not expected.
+ *
+ ******************************************************************************/
+
+static acpi_status
+acpi_ns_repair_object(u32 expected_btypes,
+ u32 package_index,
+ union acpi_operand_object **return_object_ptr)
+{
+ union acpi_operand_object *return_object = *return_object_ptr;
+ union acpi_operand_object *new_object;
+ acpi_size length;
+
+ switch (ACPI_GET_OBJECT_TYPE(return_object)) {
+ case ACPI_TYPE_BUFFER:
+
+ if (!(expected_btypes & ACPI_RTYPE_STRING)) {
+ return (AE_AML_OPERAND_TYPE);
+ }
+
+ /*
+ * Have a Buffer, expected a String, convert. Use a to_string
+ * conversion, no transform performed on the buffer data. The best
+ * example of this is the _BIF method, where the string data from
+ * the battery is often (incorrectly) returned as buffer object(s).
+ */
+ length = 0;
+ while ((length < return_object->buffer.length) &&
+ (return_object->buffer.pointer[length])) {
+ length++;
+ }
+
+ /* Allocate a new string object */
+
+ new_object = acpi_ut_create_string_object(length);
+ if (!new_object) {
+ return (AE_NO_MEMORY);
+ }
+
+ /*
+ * Copy the raw buffer data with no transform. String is already NULL
+ * terminated at Length+1.
+ */
+ ACPI_MEMCPY(new_object->string.pointer,
+ return_object->buffer.pointer, length);
+
+ /* Install the new return object */
+
+ acpi_ut_remove_reference(return_object);
+ *return_object_ptr = new_object;
+
+ /*
+ * If the object is a package element, we need to:
+ * 1. Decrement the reference count of the orignal object, it was
+ * incremented when building the package
+ * 2. Increment the reference count of the new object, it will be
+ * decremented when releasing the package
+ */
+ if (package_index != ACPI_NOT_PACKAGE) {
+ acpi_ut_remove_reference(return_object);
+ acpi_ut_add_reference(new_object);
+ }
+ return (AE_OK);
+
+ default:
+ break;
+ }
+
+ return (AE_AML_OPERAND_TYPE);
+}
diff --git a/drivers/acpi/namespace/nssearch.c b/drivers/acpi/acpica/nssearch.c
similarity index 99%
rename from drivers/acpi/namespace/nssearch.c
rename to drivers/acpi/acpica/nssearch.c
index a9a80bf..6fea13f 100644
--- a/drivers/acpi/namespace/nssearch.c
+++ b/drivers/acpi/acpica/nssearch.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_NAMESPACE
ACPI_MODULE_NAME("nssearch")
diff --git a/drivers/acpi/namespace/nsutils.c b/drivers/acpi/acpica/nsutils.c
similarity index 98%
rename from drivers/acpi/namespace/nsutils.c
rename to drivers/acpi/acpica/nsutils.c
index b0817e1..3e1149b 100644
--- a/drivers/acpi/namespace/nsutils.c
+++ b/drivers/acpi/acpica/nsutils.c
@@ -43,9 +43,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
-#include <acpi/amlcode.h>
-#include <acpi/actables.h>
+#include "accommon.h"
+#include "acnamesp.h"
+#include "amlcode.h"
+#include "actables.h"
#define _COMPONENT ACPI_NAMESPACE
ACPI_MODULE_NAME("nsutils")
@@ -314,9 +315,15 @@
*
* strlen() + 1 covers the first name_seg, which has no path separator
*/
- if (acpi_ns_valid_root_prefix(next_external_char[0])) {
+ if (acpi_ns_valid_root_prefix(*next_external_char)) {
info->fully_qualified = TRUE;
next_external_char++;
+
+ /* Skip redundant root_prefix, like \\_SB.PCI0.SBRG.EC0 */
+
+ while (acpi_ns_valid_root_prefix(*next_external_char)) {
+ next_external_char++;
+ }
} else {
/*
* Handle Carat prefixes
diff --git a/drivers/acpi/namespace/nswalk.c b/drivers/acpi/acpica/nswalk.c
similarity index 99%
rename from drivers/acpi/namespace/nswalk.c
rename to drivers/acpi/acpica/nswalk.c
index 3c905ce..200895f 100644
--- a/drivers/acpi/namespace/nswalk.c
+++ b/drivers/acpi/acpica/nswalk.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_NAMESPACE
ACPI_MODULE_NAME("nswalk")
diff --git a/drivers/acpi/namespace/nsxfeval.c b/drivers/acpi/acpica/nsxfeval.c
similarity index 99%
rename from drivers/acpi/namespace/nsxfeval.c
rename to drivers/acpi/acpica/nsxfeval.c
index a085cc3..22a7171 100644
--- a/drivers/acpi/namespace/nsxfeval.c
+++ b/drivers/acpi/acpica/nsxfeval.c
@@ -43,8 +43,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acinterp.h>
+#include "accommon.h"
+#include "acnamesp.h"
+#include "acinterp.h"
#define _COMPONENT ACPI_NAMESPACE
ACPI_MODULE_NAME("nsxfeval")
diff --git a/drivers/acpi/namespace/nsxfname.c b/drivers/acpi/acpica/nsxfname.c
similarity index 99%
rename from drivers/acpi/namespace/nsxfname.c
rename to drivers/acpi/acpica/nsxfname.c
index 5efa4e7..9589fea 100644
--- a/drivers/acpi/namespace/nsxfname.c
+++ b/drivers/acpi/acpica/nsxfname.c
@@ -43,7 +43,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_NAMESPACE
ACPI_MODULE_NAME("nsxfname")
diff --git a/drivers/acpi/namespace/nsxfobj.c b/drivers/acpi/acpica/nsxfobj.c
similarity index 99%
rename from drivers/acpi/namespace/nsxfobj.c
rename to drivers/acpi/acpica/nsxfobj.c
index 2b375ee..1c7efc1 100644
--- a/drivers/acpi/namespace/nsxfobj.c
+++ b/drivers/acpi/acpica/nsxfobj.c
@@ -43,7 +43,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_NAMESPACE
ACPI_MODULE_NAME("nsxfobj")
diff --git a/drivers/acpi/parser/psargs.c b/drivers/acpi/acpica/psargs.c
similarity index 99%
rename from drivers/acpi/parser/psargs.c
rename to drivers/acpi/acpica/psargs.c
index d830b29..b161f35 100644
--- a/drivers/acpi/parser/psargs.c
+++ b/drivers/acpi/acpica/psargs.c
@@ -42,10 +42,11 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
-#include <acpi/amlcode.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acdispat.h>
+#include "accommon.h"
+#include "acparser.h"
+#include "amlcode.h"
+#include "acnamesp.h"
+#include "acdispat.h"
#define _COMPONENT ACPI_PARSER
ACPI_MODULE_NAME("psargs")
diff --git a/drivers/acpi/parser/psloop.c b/drivers/acpi/acpica/psloop.c
similarity index 99%
rename from drivers/acpi/parser/psloop.c
rename to drivers/acpi/acpica/psloop.c
index 4647039..c5f6ce1 100644
--- a/drivers/acpi/parser/psloop.c
+++ b/drivers/acpi/acpica/psloop.c
@@ -50,9 +50,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
-#include <acpi/acdispat.h>
-#include <acpi/amlcode.h>
+#include "accommon.h"
+#include "acparser.h"
+#include "acdispat.h"
+#include "amlcode.h"
#define _COMPONENT ACPI_PARSER
ACPI_MODULE_NAME("psloop")
diff --git a/drivers/acpi/parser/psopcode.c b/drivers/acpi/acpica/psopcode.c
similarity index 99%
rename from drivers/acpi/parser/psopcode.c
rename to drivers/acpi/acpica/psopcode.c
index f425ab3..3bc3a60 100644
--- a/drivers/acpi/parser/psopcode.c
+++ b/drivers/acpi/acpica/psopcode.c
@@ -42,9 +42,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
-#include <acpi/acopcode.h>
-#include <acpi/amlcode.h>
+#include "accommon.h"
+#include "acparser.h"
+#include "acopcode.h"
+#include "amlcode.h"
#define _COMPONENT ACPI_PARSER
ACPI_MODULE_NAME("psopcode")
diff --git a/drivers/acpi/parser/psparse.c b/drivers/acpi/acpica/psparse.c
similarity index 97%
rename from drivers/acpi/parser/psparse.c
rename to drivers/acpi/acpica/psparse.c
index 68e932f..70838e9 100644
--- a/drivers/acpi/parser/psparse.c
+++ b/drivers/acpi/acpica/psparse.c
@@ -51,11 +51,12 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
-#include <acpi/acdispat.h>
-#include <acpi/amlcode.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acinterp.h>
+#include "accommon.h"
+#include "acparser.h"
+#include "acdispat.h"
+#include "amlcode.h"
+#include "acnamesp.h"
+#include "acinterp.h"
#define _COMPONENT ACPI_PARSER
ACPI_MODULE_NAME("psparse")
@@ -447,10 +448,22 @@
walk_state, walk_state->parser_state.aml,
walk_state->parser_state.aml_size));
+ if (!walk_state->parser_state.aml) {
+ return_ACPI_STATUS(AE_NULL_OBJECT);
+ }
+
/* Create and initialize a new thread state */
thread = acpi_ut_create_thread_state();
if (!thread) {
+ if (walk_state->method_desc) {
+
+ /* Executing a control method - additional cleanup */
+
+ acpi_ds_terminate_control_method(
+ walk_state->method_desc, walk_state);
+ }
+
acpi_ds_delete_walk_state(walk_state);
return_ACPI_STATUS(AE_NO_MEMORY);
}
diff --git a/drivers/acpi/parser/psscope.c b/drivers/acpi/acpica/psscope.c
similarity index 99%
rename from drivers/acpi/parser/psscope.c
rename to drivers/acpi/acpica/psscope.c
index ee50e67..2feca5c 100644
--- a/drivers/acpi/parser/psscope.c
+++ b/drivers/acpi/acpica/psscope.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
+#include "accommon.h"
+#include "acparser.h"
#define _COMPONENT ACPI_PARSER
ACPI_MODULE_NAME("psscope")
diff --git a/drivers/acpi/parser/pstree.c b/drivers/acpi/acpica/pstree.c
similarity index 98%
rename from drivers/acpi/parser/pstree.c
rename to drivers/acpi/acpica/pstree.c
index 1dd355d..4d33891 100644
--- a/drivers/acpi/parser/pstree.c
+++ b/drivers/acpi/acpica/pstree.c
@@ -42,8 +42,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
-#include <acpi/amlcode.h>
+#include "accommon.h"
+#include "acparser.h"
+#include "amlcode.h"
#define _COMPONENT ACPI_PARSER
ACPI_MODULE_NAME("pstree")
diff --git a/drivers/acpi/parser/psutils.c b/drivers/acpi/acpica/psutils.c
similarity index 98%
rename from drivers/acpi/parser/psutils.c
rename to drivers/acpi/acpica/psutils.c
index 7cf1f65..e636e07 100644
--- a/drivers/acpi/parser/psutils.c
+++ b/drivers/acpi/acpica/psutils.c
@@ -42,8 +42,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
-#include <acpi/amlcode.h>
+#include "accommon.h"
+#include "acparser.h"
+#include "amlcode.h"
#define _COMPONENT ACPI_PARSER
ACPI_MODULE_NAME("psutils")
diff --git a/drivers/acpi/parser/pswalk.c b/drivers/acpi/acpica/pswalk.c
similarity index 98%
rename from drivers/acpi/parser/pswalk.c
rename to drivers/acpi/acpica/pswalk.c
index 8b86ad5..78b8b79 100644
--- a/drivers/acpi/parser/pswalk.c
+++ b/drivers/acpi/acpica/pswalk.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
+#include "accommon.h"
+#include "acparser.h"
#define _COMPONENT ACPI_PARSER
ACPI_MODULE_NAME("pswalk")
diff --git a/drivers/acpi/parser/psxface.c b/drivers/acpi/acpica/psxface.c
similarity index 90%
rename from drivers/acpi/parser/psxface.c
rename to drivers/acpi/acpica/psxface.c
index 270469a..ff06032 100644
--- a/drivers/acpi/parser/psxface.c
+++ b/drivers/acpi/acpica/psxface.c
@@ -42,9 +42,11 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acparser.h>
-#include <acpi/acdispat.h>
-#include <acpi/acinterp.h>
+#include "accommon.h"
+#include "acparser.h"
+#include "acdispat.h"
+#include "acinterp.h"
+#include "amlcode.h"
#define _COMPONENT ACPI_PARSER
ACPI_MODULE_NAME("psxface")
@@ -278,6 +280,38 @@
goto cleanup;
}
+ /* Invoke an internal method if necessary */
+
+ if (info->obj_desc->method.method_flags & AML_METHOD_INTERNAL_ONLY) {
+ status = info->obj_desc->method.implementation(walk_state);
+ info->return_object = walk_state->return_desc;
+
+ /* Cleanup states */
+
+ acpi_ds_scope_stack_clear(walk_state);
+ acpi_ps_cleanup_scope(&walk_state->parser_state);
+ acpi_ds_terminate_control_method(walk_state->method_desc,
+ walk_state);
+ acpi_ds_delete_walk_state(walk_state);
+ goto cleanup;
+ }
+
+ /*
+ * Start method evaluation with an implicit return of zero.
+ * This is done for Windows compatibility.
+ */
+ if (acpi_gbl_enable_interpreter_slack) {
+ walk_state->implicit_return_obj =
+ acpi_ut_create_internal_object(ACPI_TYPE_INTEGER);
+ if (!walk_state->implicit_return_obj) {
+ status = AE_NO_MEMORY;
+ acpi_ds_delete_walk_state(walk_state);
+ goto cleanup;
+ }
+
+ walk_state->implicit_return_obj->integer.value = 0;
+ }
+
/* Parse the AML */
status = acpi_ps_parse_aml(walk_state);
diff --git a/drivers/acpi/resources/rsaddr.c b/drivers/acpi/acpica/rsaddr.c
similarity index 99%
rename from drivers/acpi/resources/rsaddr.c
rename to drivers/acpi/acpica/rsaddr.c
index 7f96332..1e437bf 100644
--- a/drivers/acpi/resources/rsaddr.c
+++ b/drivers/acpi/acpica/rsaddr.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acresrc.h>
+#include "accommon.h"
+#include "acresrc.h"
#define _COMPONENT ACPI_RESOURCES
ACPI_MODULE_NAME("rsaddr")
diff --git a/drivers/acpi/resources/rscalc.c b/drivers/acpi/acpica/rscalc.c
similarity index 99%
rename from drivers/acpi/resources/rscalc.c
rename to drivers/acpi/acpica/rscalc.c
index 8eaaecf..52865ee 100644
--- a/drivers/acpi/resources/rscalc.c
+++ b/drivers/acpi/acpica/rscalc.c
@@ -42,8 +42,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acresrc.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acresrc.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_RESOURCES
ACPI_MODULE_NAME("rscalc")
diff --git a/drivers/acpi/resources/rscreate.c b/drivers/acpi/acpica/rscreate.c
similarity index 99%
rename from drivers/acpi/resources/rscreate.c
rename to drivers/acpi/acpica/rscreate.c
index 08b8d73..61566b1 100644
--- a/drivers/acpi/resources/rscreate.c
+++ b/drivers/acpi/acpica/rscreate.c
@@ -42,8 +42,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acresrc.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acresrc.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_RESOURCES
ACPI_MODULE_NAME("rscreate")
diff --git a/drivers/acpi/resources/rsdump.c b/drivers/acpi/acpica/rsdump.c
similarity index 99%
rename from drivers/acpi/resources/rsdump.c
rename to drivers/acpi/acpica/rsdump.c
index 6bbbb7b..3f0ca5a 100644
--- a/drivers/acpi/resources/rsdump.c
+++ b/drivers/acpi/acpica/rsdump.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acresrc.h>
+#include "accommon.h"
+#include "acresrc.h"
#define _COMPONENT ACPI_RESOURCES
ACPI_MODULE_NAME("rsdump")
diff --git a/drivers/acpi/resources/rsinfo.c b/drivers/acpi/acpica/rsinfo.c
similarity index 99%
rename from drivers/acpi/resources/rsinfo.c
rename to drivers/acpi/acpica/rsinfo.c
index 3f0a1fe..77b25fd 100644
--- a/drivers/acpi/resources/rsinfo.c
+++ b/drivers/acpi/acpica/rsinfo.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acresrc.h>
+#include "accommon.h"
+#include "acresrc.h"
#define _COMPONENT ACPI_RESOURCES
ACPI_MODULE_NAME("rsinfo")
diff --git a/drivers/acpi/resources/rsio.c b/drivers/acpi/acpica/rsio.c
similarity index 99%
rename from drivers/acpi/resources/rsio.c
rename to drivers/acpi/acpica/rsio.c
index b66d42e..35a49aa 100644
--- a/drivers/acpi/resources/rsio.c
+++ b/drivers/acpi/acpica/rsio.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acresrc.h>
+#include "accommon.h"
+#include "acresrc.h"
#define _COMPONENT ACPI_RESOURCES
ACPI_MODULE_NAME("rsio")
diff --git a/drivers/acpi/resources/rsirq.c b/drivers/acpi/acpica/rsirq.c
similarity index 99%
rename from drivers/acpi/resources/rsirq.c
rename to drivers/acpi/acpica/rsirq.c
index a8805ef..2e02569 100644
--- a/drivers/acpi/resources/rsirq.c
+++ b/drivers/acpi/acpica/rsirq.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acresrc.h>
+#include "accommon.h"
+#include "acresrc.h"
#define _COMPONENT ACPI_RESOURCES
ACPI_MODULE_NAME("rsirq")
diff --git a/drivers/acpi/resources/rslist.c b/drivers/acpi/acpica/rslist.c
similarity index 99%
rename from drivers/acpi/resources/rslist.c
rename to drivers/acpi/acpica/rslist.c
index b78c7e7..1b1dbc6 100644
--- a/drivers/acpi/resources/rslist.c
+++ b/drivers/acpi/acpica/rslist.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acresrc.h>
+#include "accommon.h"
+#include "acresrc.h"
#define _COMPONENT ACPI_RESOURCES
ACPI_MODULE_NAME("rslist")
diff --git a/drivers/acpi/resources/rsmemory.c b/drivers/acpi/acpica/rsmemory.c
similarity index 99%
rename from drivers/acpi/resources/rsmemory.c
rename to drivers/acpi/acpica/rsmemory.c
index 63b21ab..ddc76ce 100644
--- a/drivers/acpi/resources/rsmemory.c
+++ b/drivers/acpi/acpica/rsmemory.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acresrc.h>
+#include "accommon.h"
+#include "acresrc.h"
#define _COMPONENT ACPI_RESOURCES
ACPI_MODULE_NAME("rsmemory")
diff --git a/drivers/acpi/resources/rsmisc.c b/drivers/acpi/acpica/rsmisc.c
similarity index 99%
rename from drivers/acpi/resources/rsmisc.c
rename to drivers/acpi/acpica/rsmisc.c
index 96a6c03..5bc49a5 100644
--- a/drivers/acpi/resources/rsmisc.c
+++ b/drivers/acpi/acpica/rsmisc.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acresrc.h>
+#include "accommon.h"
+#include "acresrc.h"
#define _COMPONENT ACPI_RESOURCES
ACPI_MODULE_NAME("rsmisc")
diff --git a/drivers/acpi/resources/rsutils.c b/drivers/acpi/acpica/rsutils.c
similarity index 99%
rename from drivers/acpi/resources/rsutils.c
rename to drivers/acpi/acpica/rsutils.c
index f7b3bcd..bc03d59 100644
--- a/drivers/acpi/resources/rsutils.c
+++ b/drivers/acpi/acpica/rsutils.c
@@ -42,8 +42,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acresrc.h>
+#include "accommon.h"
+#include "acnamesp.h"
+#include "acresrc.h"
#define _COMPONENT ACPI_RESOURCES
ACPI_MODULE_NAME("rsutils")
diff --git a/drivers/acpi/resources/rsxface.c b/drivers/acpi/acpica/rsxface.c
similarity index 99%
rename from drivers/acpi/resources/rsxface.c
rename to drivers/acpi/acpica/rsxface.c
index f59f4c4..69a2aa5 100644
--- a/drivers/acpi/resources/rsxface.c
+++ b/drivers/acpi/acpica/rsxface.c
@@ -42,8 +42,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acresrc.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acresrc.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_RESOURCES
ACPI_MODULE_NAME("rsxface")
diff --git a/drivers/acpi/tables/tbfadt.c b/drivers/acpi/acpica/tbfadt.c
similarity index 67%
rename from drivers/acpi/tables/tbfadt.c
rename to drivers/acpi/acpica/tbfadt.c
index 2817158..3636e4f 100644
--- a/drivers/acpi/tables/tbfadt.c
+++ b/drivers/acpi/acpica/tbfadt.c
@@ -42,15 +42,16 @@
*/
#include <acpi/acpi.h>
-#include <acpi/actables.h>
+#include "accommon.h"
+#include "actables.h"
#define _COMPONENT ACPI_TABLES
ACPI_MODULE_NAME("tbfadt")
/* Local prototypes */
-static void inline
+static inline void
acpi_tb_init_generic_address(struct acpi_generic_address *generic_address,
- u8 byte_width, u64 address);
+ u8 space_id, u8 byte_width, u64 address);
static void acpi_tb_convert_fadt(void);
@@ -60,9 +61,10 @@
typedef struct acpi_fadt_info {
char *name;
- u8 target;
- u8 source;
+ u8 address64;
+ u8 address32;
u8 length;
+ u8 default_length;
u8 type;
} acpi_fadt_info;
@@ -71,37 +73,61 @@
#define ACPI_FADT_SEPARATE_LENGTH 2
static struct acpi_fadt_info fadt_info_table[] = {
- {"Pm1aEventBlock", ACPI_FADT_OFFSET(xpm1a_event_block),
+ {"Pm1aEventBlock",
+ ACPI_FADT_OFFSET(xpm1a_event_block),
ACPI_FADT_OFFSET(pm1a_event_block),
- ACPI_FADT_OFFSET(pm1_event_length), ACPI_FADT_REQUIRED},
+ ACPI_FADT_OFFSET(pm1_event_length),
+ ACPI_PM1_REGISTER_WIDTH * 2, /* Enable + Status register */
+ ACPI_FADT_REQUIRED},
- {"Pm1bEventBlock", ACPI_FADT_OFFSET(xpm1b_event_block),
+ {"Pm1bEventBlock",
+ ACPI_FADT_OFFSET(xpm1b_event_block),
ACPI_FADT_OFFSET(pm1b_event_block),
- ACPI_FADT_OFFSET(pm1_event_length), 0},
+ ACPI_FADT_OFFSET(pm1_event_length),
+ ACPI_PM1_REGISTER_WIDTH * 2, /* Enable + Status register */
+ 0},
- {"Pm1aControlBlock", ACPI_FADT_OFFSET(xpm1a_control_block),
+ {"Pm1aControlBlock",
+ ACPI_FADT_OFFSET(xpm1a_control_block),
ACPI_FADT_OFFSET(pm1a_control_block),
- ACPI_FADT_OFFSET(pm1_control_length), ACPI_FADT_REQUIRED},
+ ACPI_FADT_OFFSET(pm1_control_length),
+ ACPI_PM1_REGISTER_WIDTH,
+ ACPI_FADT_REQUIRED},
- {"Pm1bControlBlock", ACPI_FADT_OFFSET(xpm1b_control_block),
+ {"Pm1bControlBlock",
+ ACPI_FADT_OFFSET(xpm1b_control_block),
ACPI_FADT_OFFSET(pm1b_control_block),
- ACPI_FADT_OFFSET(pm1_control_length), 0},
+ ACPI_FADT_OFFSET(pm1_control_length),
+ ACPI_PM1_REGISTER_WIDTH,
+ 0},
- {"Pm2ControlBlock", ACPI_FADT_OFFSET(xpm2_control_block),
+ {"Pm2ControlBlock",
+ ACPI_FADT_OFFSET(xpm2_control_block),
ACPI_FADT_OFFSET(pm2_control_block),
- ACPI_FADT_OFFSET(pm2_control_length), ACPI_FADT_SEPARATE_LENGTH},
+ ACPI_FADT_OFFSET(pm2_control_length),
+ ACPI_PM2_REGISTER_WIDTH,
+ ACPI_FADT_SEPARATE_LENGTH},
- {"PmTimerBlock", ACPI_FADT_OFFSET(xpm_timer_block),
+ {"PmTimerBlock",
+ ACPI_FADT_OFFSET(xpm_timer_block),
ACPI_FADT_OFFSET(pm_timer_block),
- ACPI_FADT_OFFSET(pm_timer_length), ACPI_FADT_REQUIRED},
+ ACPI_FADT_OFFSET(pm_timer_length),
+ ACPI_PM_TIMER_WIDTH,
+ ACPI_FADT_REQUIRED},
- {"Gpe0Block", ACPI_FADT_OFFSET(xgpe0_block),
+ {"Gpe0Block",
+ ACPI_FADT_OFFSET(xgpe0_block),
ACPI_FADT_OFFSET(gpe0_block),
- ACPI_FADT_OFFSET(gpe0_block_length), ACPI_FADT_SEPARATE_LENGTH},
+ ACPI_FADT_OFFSET(gpe0_block_length),
+ 0,
+ ACPI_FADT_SEPARATE_LENGTH},
- {"Gpe1Block", ACPI_FADT_OFFSET(xgpe1_block),
+ {"Gpe1Block",
+ ACPI_FADT_OFFSET(xgpe1_block),
ACPI_FADT_OFFSET(gpe1_block),
- ACPI_FADT_OFFSET(gpe1_block_length), ACPI_FADT_SEPARATE_LENGTH}
+ ACPI_FADT_OFFSET(gpe1_block_length),
+ 0,
+ ACPI_FADT_SEPARATE_LENGTH}
};
#define ACPI_FADT_INFO_ENTRIES (sizeof (fadt_info_table) / sizeof (struct acpi_fadt_info))
@@ -122,9 +148,9 @@
*
******************************************************************************/
-static void inline
+static inline void
acpi_tb_init_generic_address(struct acpi_generic_address *generic_address,
- u8 byte_width, u64 address)
+ u8 space_id, u8 byte_width, u64 address)
{
/*
@@ -135,10 +161,10 @@
/* All other fields are byte-wide */
- generic_address->space_id = ACPI_ADR_SPACE_SYSTEM_IO;
- generic_address->bit_width = byte_width << 3;
+ generic_address->space_id = space_id;
+ generic_address->bit_width = (u8)ACPI_MUL_8(byte_width);
generic_address->bit_offset = 0;
- generic_address->access_width = 0;
+ generic_address->access_width = 0; /* Access width ANY */
}
/*******************************************************************************
@@ -225,7 +251,8 @@
*/
if (length > sizeof(struct acpi_table_fadt)) {
ACPI_WARNING((AE_INFO,
- "FADT (revision %u) is longer than ACPI 2.0 version, truncating length 0x%X to 0x%zX",
+ "FADT (revision %u) is longer than ACPI 2.0 version, "
+ "truncating length 0x%X to 0x%zX",
table->revision, (unsigned)length,
sizeof(struct acpi_table_fadt)));
}
@@ -244,7 +271,6 @@
* 2) Validate some of the important values within the FADT
*/
acpi_tb_convert_fadt();
- acpi_tb_validate_fadt();
}
/*******************************************************************************
@@ -278,22 +304,36 @@
static void acpi_tb_convert_fadt(void)
{
- u8 pm1_register_length;
- struct acpi_generic_address *target;
+ u8 pm1_register_bit_width;
+ u8 pm1_register_byte_width;
+ struct acpi_generic_address *target64;
u32 i;
/* Update the local FADT table header length */
acpi_gbl_FADT.header.length = sizeof(struct acpi_table_fadt);
- /* Expand the 32-bit FACS and DSDT addresses to 64-bit as necessary */
-
+ /*
+ * Expand the 32-bit FACS and DSDT addresses to 64-bit as necessary.
+ * Later code will always use the X 64-bit field. Also, check for an
+ * address mismatch between the 32-bit and 64-bit address fields
+ * (FIRMWARE_CTRL/X_FIRMWARE_CTRL, DSDT/X_DSDT) which would indicate
+ * the presence of two FACS or two DSDT tables.
+ */
if (!acpi_gbl_FADT.Xfacs) {
acpi_gbl_FADT.Xfacs = (u64) acpi_gbl_FADT.facs;
+ } else if (acpi_gbl_FADT.facs &&
+ (acpi_gbl_FADT.Xfacs != (u64) acpi_gbl_FADT.facs)) {
+ ACPI_WARNING((AE_INFO,
+ "32/64 FACS address mismatch in FADT - two FACS tables!"));
}
if (!acpi_gbl_FADT.Xdsdt) {
acpi_gbl_FADT.Xdsdt = (u64) acpi_gbl_FADT.dsdt;
+ } else if (acpi_gbl_FADT.dsdt &&
+ (acpi_gbl_FADT.Xdsdt != (u64) acpi_gbl_FADT.dsdt)) {
+ ACPI_WARNING((AE_INFO,
+ "32/64 DSDT address mismatch in FADT - two DSDT tables!"));
}
/*
@@ -312,18 +352,23 @@
}
/*
- * Expand the ACPI 1.0 32-bit V1.0 addresses to the ACPI 2.0 64-bit "X"
- * generic address structures as necessary.
+ * Expand the ACPI 1.0 32-bit addresses to the ACPI 2.0 64-bit "X"
+ * generic address structures as necessary. Later code will always use
+ * the 64-bit address structures.
*/
for (i = 0; i < ACPI_FADT_INFO_ENTRIES; i++) {
- target =
+ target64 =
ACPI_ADD_PTR(struct acpi_generic_address, &acpi_gbl_FADT,
- fadt_info_table[i].target);
+ fadt_info_table[i].address64);
- /* Expand only if the X target is null */
+ /* Expand only if the 64-bit X target is null */
- if (!target->address) {
- acpi_tb_init_generic_address(target,
+ if (!target64->address) {
+
+ /* The space_id is always I/O for the 32-bit legacy address fields */
+
+ acpi_tb_init_generic_address(target64,
+ ACPI_ADR_SPACE_SYSTEM_IO,
*ACPI_ADD_PTR(u8,
&acpi_gbl_FADT,
fadt_info_table
@@ -332,10 +377,63 @@
&acpi_gbl_FADT,
fadt_info_table
[i].
- source));
+ address32));
}
}
+ /* Validate FADT values now, before we make any changes */
+
+ acpi_tb_validate_fadt();
+
+ /*
+ * Optionally check all register lengths against the default values and
+ * update them if they are incorrect.
+ */
+ if (acpi_gbl_use_default_register_widths) {
+ for (i = 0; i < ACPI_FADT_INFO_ENTRIES; i++) {
+ target64 =
+ ACPI_ADD_PTR(struct acpi_generic_address,
+ &acpi_gbl_FADT,
+ fadt_info_table[i].address64);
+
+ /*
+ * If a valid register (Address != 0) and the (default_length > 0)
+ * (Not a GPE register), then check the width against the default.
+ */
+ if ((target64->address) &&
+ (fadt_info_table[i].default_length > 0) &&
+ (fadt_info_table[i].default_length !=
+ target64->bit_width)) {
+ ACPI_WARNING((AE_INFO,
+ "Invalid length for %s: %d, using default %d",
+ fadt_info_table[i].name,
+ target64->bit_width,
+ fadt_info_table[i].
+ default_length));
+
+ /* Incorrect size, set width to the default */
+
+ target64->bit_width =
+ fadt_info_table[i].default_length;
+ }
+ }
+ }
+
+ /*
+ * Get the length of the individual PM1 registers (enable and status).
+ * Each register is defined to be (event block length / 2).
+ */
+ pm1_register_bit_width =
+ (u8)ACPI_DIV_2(acpi_gbl_FADT.xpm1a_event_block.bit_width);
+ pm1_register_byte_width = (u8)ACPI_DIV_8(pm1_register_bit_width);
+
+ /*
+ * Adjust the lengths of the PM1 Event Blocks so that they can be used to
+ * access the PM1 status register(s). Use (width / 2)
+ */
+ acpi_gbl_FADT.xpm1a_event_block.bit_width = pm1_register_bit_width;
+ acpi_gbl_FADT.xpm1b_event_block.bit_width = pm1_register_bit_width;
+
/*
* Calculate separate GAS structs for the PM1 Enable registers.
* These addresses do not appear (directly) in the FADT, so it is
@@ -356,14 +454,14 @@
" PM1_EVT_LEN (%u)\n",
acpi_gbl_FADT.xpm1a_event_block.bit_width,
acpi_gbl_FADT.pm1_event_length);
- pm1_register_length = (u8) ACPI_DIV_2(acpi_gbl_FADT.pm1_event_length);
/* The PM1A register block is required */
acpi_tb_init_generic_address(&acpi_gbl_xpm1a_enable,
- pm1_register_length,
+ acpi_gbl_FADT.xpm1a_event_block.space_id,
+ pm1_register_byte_width,
(acpi_gbl_FADT.xpm1a_event_block.address +
- pm1_register_length));
+ pm1_register_byte_width));
/* Don't forget to copy space_id of the GAS */
acpi_gbl_xpm1a_enable.space_id =
acpi_gbl_FADT.xpm1a_event_block.space_id;
@@ -379,9 +477,10 @@
acpi_gbl_FADT.xpm1b_event_block.bit_width,
acpi_gbl_FADT.pm1_event_length);
acpi_tb_init_generic_address(&acpi_gbl_xpm1b_enable,
- pm1_register_length,
+ acpi_gbl_FADT.xpm1b_event_block.space_id,
+ pm1_register_byte_width,
(acpi_gbl_FADT.xpm1b_event_block.
- address + pm1_register_length));
+ address + pm1_register_byte_width));
/* Don't forget to copy space_id of the GAS */
acpi_gbl_xpm1b_enable.space_id =
acpi_gbl_FADT.xpm1b_event_block.space_id;
@@ -411,26 +510,63 @@
static void acpi_tb_validate_fadt(void)
{
+ char *name;
u32 *address32;
struct acpi_generic_address *address64;
u8 length;
u32 i;
+ /*
+ * Check for FACS and DSDT address mismatches. An address mismatch between
+ * the 32-bit and 64-bit address fields (FIRMWARE_CTRL/X_FIRMWARE_CTRL and
+ * DSDT/X_DSDT) would indicate the presence of two FACS or two DSDT tables.
+ */
+ if (acpi_gbl_FADT.facs &&
+ (acpi_gbl_FADT.Xfacs != (u64) acpi_gbl_FADT.facs)) {
+ ACPI_WARNING((AE_INFO,
+ "32/64X FACS address mismatch in FADT - "
+ "two FACS tables! %8.8X/%8.8X%8.8X",
+ acpi_gbl_FADT.facs,
+ ACPI_FORMAT_UINT64(acpi_gbl_FADT.Xfacs)));
+ }
+
+ if (acpi_gbl_FADT.dsdt &&
+ (acpi_gbl_FADT.Xdsdt != (u64) acpi_gbl_FADT.dsdt)) {
+ ACPI_WARNING((AE_INFO,
+ "32/64X DSDT address mismatch in FADT - "
+ "two DSDT tables! %8.8X/%8.8X%8.8X",
+ acpi_gbl_FADT.dsdt,
+ ACPI_FORMAT_UINT64(acpi_gbl_FADT.Xdsdt)));
+ }
+
/* Examine all of the 64-bit extended address fields (X fields) */
for (i = 0; i < ACPI_FADT_INFO_ENTRIES; i++) {
-
- /* Generate pointers to the 32-bit and 64-bit addresses and get the length */
-
- address64 =
- ACPI_ADD_PTR(struct acpi_generic_address, &acpi_gbl_FADT,
- fadt_info_table[i].target);
+ /*
+ * Generate pointers to the 32-bit and 64-bit addresses, get the
+ * register length (width), and the register name
+ */
+ address64 = ACPI_ADD_PTR(struct acpi_generic_address,
+ &acpi_gbl_FADT,
+ fadt_info_table[i].address64);
address32 =
ACPI_ADD_PTR(u32, &acpi_gbl_FADT,
- fadt_info_table[i].source);
+ fadt_info_table[i].address32);
length =
*ACPI_ADD_PTR(u8, &acpi_gbl_FADT,
fadt_info_table[i].length);
+ name = fadt_info_table[i].name;
+
+ /*
+ * For each extended field, check for length mismatch between the
+ * legacy length field and the corresponding 64-bit X length field.
+ */
+ if (address64 && (address64->bit_width != ACPI_MUL_8(length))) {
+ ACPI_WARNING((AE_INFO,
+ "32/64X length mismatch in %s: %d/%d",
+ name, ACPI_MUL_8(length),
+ address64->bit_width));
+ }
if (fadt_info_table[i].type & ACPI_FADT_REQUIRED) {
/*
@@ -439,8 +575,8 @@
*/
if (!address64->address || !length) {
ACPI_ERROR((AE_INFO,
- "Required field \"%s\" has zero address and/or length: %8.8X%8.8X/%X",
- fadt_info_table[i].name,
+ "Required field %s has zero address and/or length: %8.8X%8.8X/%X",
+ name,
ACPI_FORMAT_UINT64(address64->
address),
length));
@@ -453,8 +589,8 @@
if ((address64->address && !length)
|| (!address64->address && length)) {
ACPI_WARNING((AE_INFO,
- "Optional field \"%s\" has zero address or length: %8.8X%8.8X/%X",
- fadt_info_table[i].name,
+ "Optional field %s has zero address or length: %8.8X%8.8X/%X",
+ name,
ACPI_FORMAT_UINT64(address64->
address),
length));
@@ -466,8 +602,8 @@
if (address64->address && *address32 &&
(address64->address != (u64) * address32)) {
ACPI_ERROR((AE_INFO,
- "32/64X address mismatch in \"%s\": [%8.8X] [%8.8X%8.8X], using 64X",
- fadt_info_table[i].name, *address32,
+ "32/64X address mismatch in %s: %8.8X/%8.8X%8.8X, using 64X",
+ name, *address32,
ACPI_FORMAT_UINT64(address64->address)));
}
}
diff --git a/drivers/acpi/tables/tbfind.c b/drivers/acpi/acpica/tbfind.c
similarity index 98%
rename from drivers/acpi/tables/tbfind.c
rename to drivers/acpi/acpica/tbfind.c
index 531584d..1054dfd 100644
--- a/drivers/acpi/tables/tbfind.c
+++ b/drivers/acpi/acpica/tbfind.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/actables.h>
+#include "accommon.h"
+#include "actables.h"
#define _COMPONENT ACPI_TABLES
ACPI_MODULE_NAME("tbfind")
diff --git a/drivers/acpi/tables/tbinstal.c b/drivers/acpi/acpica/tbinstal.c
similarity index 99%
rename from drivers/acpi/tables/tbinstal.c
rename to drivers/acpi/acpica/tbinstal.c
index 18747ce..37374b2 100644
--- a/drivers/acpi/tables/tbinstal.c
+++ b/drivers/acpi/acpica/tbinstal.c
@@ -42,8 +42,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
-#include <acpi/actables.h>
+#include "accommon.h"
+#include "acnamesp.h"
+#include "actables.h"
#define _COMPONENT ACPI_TABLES
ACPI_MODULE_NAME("tbinstal")
diff --git a/drivers/acpi/tables/tbutils.c b/drivers/acpi/acpica/tbutils.c
similarity index 95%
rename from drivers/acpi/tables/tbutils.c
rename to drivers/acpi/acpica/tbutils.c
index 0cc92ef..9684cc8 100644
--- a/drivers/acpi/tables/tbutils.c
+++ b/drivers/acpi/acpica/tbutils.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/actables.h>
+#include "accommon.h"
+#include "actables.h"
#define _COMPONENT ACPI_TABLES
ACPI_MODULE_NAME("tbutils")
@@ -113,6 +114,30 @@
/*******************************************************************************
*
+ * FUNCTION: acpi_tb_initialize_facs
+ *
+ * PARAMETERS: None
+ *
+ * RETURN: Status
+ *
+ * DESCRIPTION: Create a permanent mapping for the FADT and save it in a global
+ * for accessing the Global Lock and Firmware Waking Vector
+ *
+ ******************************************************************************/
+
+acpi_status acpi_tb_initialize_facs(void)
+{
+ acpi_status status;
+
+ status = acpi_get_table_by_index(ACPI_TABLE_INDEX_FACS,
+ ACPI_CAST_INDIRECT_PTR(struct
+ acpi_table_header,
+ &acpi_gbl_FACS));
+ return status;
+}
+
+/*******************************************************************************
+ *
* FUNCTION: acpi_tb_tables_loaded
*
* PARAMETERS: None
@@ -420,7 +445,8 @@
/* Differentiate between RSDT and XSDT root tables */
- if (rsdp->revision > 1 && rsdp->xsdt_physical_address) {
+ if (rsdp->revision > 1 && rsdp->xsdt_physical_address
+ && !acpi_rsdt_forced) {
/*
* Root table is an XSDT (64-bit physical addresses). We must use the
* XSDT if the revision is > 1 and the XSDT pointer is present, as per
diff --git a/drivers/acpi/tables/tbxface.c b/drivers/acpi/acpica/tbxface.c
similarity index 99%
rename from drivers/acpi/tables/tbxface.c
rename to drivers/acpi/acpica/tbxface.c
index fd7770a..c3e841f 100644
--- a/drivers/acpi/tables/tbxface.c
+++ b/drivers/acpi/acpica/tbxface.c
@@ -43,8 +43,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
-#include <acpi/actables.h>
+#include "accommon.h"
+#include "acnamesp.h"
+#include "actables.h"
#define _COMPONENT ACPI_TABLES
ACPI_MODULE_NAME("tbxface")
diff --git a/drivers/acpi/tables/tbxfroot.c b/drivers/acpi/acpica/tbxfroot.c
similarity index 99%
rename from drivers/acpi/tables/tbxfroot.c
rename to drivers/acpi/acpica/tbxfroot.c
index 2d157e0..b7fc8dd 100644
--- a/drivers/acpi/tables/tbxfroot.c
+++ b/drivers/acpi/acpica/tbxfroot.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/actables.h>
+#include "accommon.h"
+#include "actables.h"
#define _COMPONENT ACPI_TABLES
ACPI_MODULE_NAME("tbxfroot")
diff --git a/drivers/acpi/utilities/utalloc.c b/drivers/acpi/acpica/utalloc.c
similarity index 99%
rename from drivers/acpi/utilities/utalloc.c
rename to drivers/acpi/acpica/utalloc.c
index 241c535..7580f6b 100644
--- a/drivers/acpi/utilities/utalloc.c
+++ b/drivers/acpi/acpica/utalloc.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acdebug.h>
+#include "accommon.h"
+#include "acdebug.h"
#define _COMPONENT ACPI_UTILITIES
ACPI_MODULE_NAME("utalloc")
diff --git a/drivers/acpi/utilities/utcopy.c b/drivers/acpi/acpica/utcopy.c
similarity index 99%
rename from drivers/acpi/utilities/utcopy.c
rename to drivers/acpi/acpica/utcopy.c
index 5b2f7c2..b0dcfd3 100644
--- a/drivers/acpi/utilities/utcopy.c
+++ b/drivers/acpi/acpica/utcopy.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_UTILITIES
diff --git a/drivers/acpi/utilities/utdebug.c b/drivers/acpi/acpica/utdebug.c
similarity index 89%
rename from drivers/acpi/utilities/utdebug.c
rename to drivers/acpi/acpica/utdebug.c
index fd66ecb..38821f5 100644
--- a/drivers/acpi/utilities/utdebug.c
+++ b/drivers/acpi/acpica/utdebug.c
@@ -42,6 +42,7 @@
*/
#include <acpi/acpi.h>
+#include "accommon.h"
#define _COMPONENT ACPI_UTILITIES
ACPI_MODULE_NAME("utdebug")
@@ -136,7 +137,7 @@
/*******************************************************************************
*
- * FUNCTION: acpi_ut_debug_print
+ * FUNCTION: acpi_debug_print
*
* PARAMETERS: requested_debug_level - Requested debug print level
* line_number - Caller's line number (for error output)
@@ -154,11 +155,11 @@
******************************************************************************/
void ACPI_INTERNAL_VAR_XFACE
-acpi_ut_debug_print(u32 requested_debug_level,
- u32 line_number,
- const char *function_name,
- const char *module_name,
- u32 component_id, const char *format, ...)
+acpi_debug_print(u32 requested_debug_level,
+ u32 line_number,
+ const char *function_name,
+ const char *module_name,
+ u32 component_id, const char *format, ...)
{
acpi_thread_id thread_id;
va_list args;
@@ -205,11 +206,11 @@
va_end(args);
}
-ACPI_EXPORT_SYMBOL(acpi_ut_debug_print)
+ACPI_EXPORT_SYMBOL(acpi_debug_print)
/*******************************************************************************
*
- * FUNCTION: acpi_ut_debug_print_raw
+ * FUNCTION: acpi_debug_print_raw
*
* PARAMETERS: requested_debug_level - Requested debug print level
* line_number - Caller's line number
@@ -226,11 +227,11 @@
*
******************************************************************************/
void ACPI_INTERNAL_VAR_XFACE
-acpi_ut_debug_print_raw(u32 requested_debug_level,
- u32 line_number,
- const char *function_name,
- const char *module_name,
- u32 component_id, const char *format, ...)
+acpi_debug_print_raw(u32 requested_debug_level,
+ u32 line_number,
+ const char *function_name,
+ const char *module_name,
+ u32 component_id, const char *format, ...)
{
va_list args;
@@ -244,7 +245,7 @@
va_end(args);
}
-ACPI_EXPORT_SYMBOL(acpi_ut_debug_print_raw)
+ACPI_EXPORT_SYMBOL(acpi_debug_print_raw)
/*******************************************************************************
*
@@ -270,9 +271,9 @@
acpi_gbl_nesting_level++;
acpi_ut_track_stack_ptr();
- acpi_ut_debug_print(ACPI_LV_FUNCTIONS,
- line_number, function_name, module_name,
- component_id, "%s\n", acpi_gbl_fn_entry_str);
+ acpi_debug_print(ACPI_LV_FUNCTIONS,
+ line_number, function_name, module_name, component_id,
+ "%s\n", acpi_gbl_fn_entry_str);
}
ACPI_EXPORT_SYMBOL(acpi_ut_trace)
@@ -301,10 +302,9 @@
acpi_gbl_nesting_level++;
acpi_ut_track_stack_ptr();
- acpi_ut_debug_print(ACPI_LV_FUNCTIONS,
- line_number, function_name, module_name,
- component_id, "%s %p\n", acpi_gbl_fn_entry_str,
- pointer);
+ acpi_debug_print(ACPI_LV_FUNCTIONS,
+ line_number, function_name, module_name, component_id,
+ "%s %p\n", acpi_gbl_fn_entry_str, pointer);
}
/*******************************************************************************
@@ -333,10 +333,9 @@
acpi_gbl_nesting_level++;
acpi_ut_track_stack_ptr();
- acpi_ut_debug_print(ACPI_LV_FUNCTIONS,
- line_number, function_name, module_name,
- component_id, "%s %s\n", acpi_gbl_fn_entry_str,
- string);
+ acpi_debug_print(ACPI_LV_FUNCTIONS,
+ line_number, function_name, module_name, component_id,
+ "%s %s\n", acpi_gbl_fn_entry_str, string);
}
/*******************************************************************************
@@ -365,10 +364,9 @@
acpi_gbl_nesting_level++;
acpi_ut_track_stack_ptr();
- acpi_ut_debug_print(ACPI_LV_FUNCTIONS,
- line_number, function_name, module_name,
- component_id, "%s %08X\n", acpi_gbl_fn_entry_str,
- integer);
+ acpi_debug_print(ACPI_LV_FUNCTIONS,
+ line_number, function_name, module_name, component_id,
+ "%s %08X\n", acpi_gbl_fn_entry_str, integer);
}
/*******************************************************************************
@@ -393,9 +391,9 @@
const char *module_name, u32 component_id)
{
- acpi_ut_debug_print(ACPI_LV_FUNCTIONS,
- line_number, function_name, module_name,
- component_id, "%s\n", acpi_gbl_fn_exit_str);
+ acpi_debug_print(ACPI_LV_FUNCTIONS,
+ line_number, function_name, module_name, component_id,
+ "%s\n", acpi_gbl_fn_exit_str);
acpi_gbl_nesting_level--;
}
@@ -426,17 +424,16 @@
{
if (ACPI_SUCCESS(status)) {
- acpi_ut_debug_print(ACPI_LV_FUNCTIONS,
- line_number, function_name, module_name,
- component_id, "%s %s\n",
- acpi_gbl_fn_exit_str,
- acpi_format_exception(status));
+ acpi_debug_print(ACPI_LV_FUNCTIONS,
+ line_number, function_name, module_name,
+ component_id, "%s %s\n", acpi_gbl_fn_exit_str,
+ acpi_format_exception(status));
} else {
- acpi_ut_debug_print(ACPI_LV_FUNCTIONS,
- line_number, function_name, module_name,
- component_id, "%s ****Exception****: %s\n",
- acpi_gbl_fn_exit_str,
- acpi_format_exception(status));
+ acpi_debug_print(ACPI_LV_FUNCTIONS,
+ line_number, function_name, module_name,
+ component_id, "%s ****Exception****: %s\n",
+ acpi_gbl_fn_exit_str,
+ acpi_format_exception(status));
}
acpi_gbl_nesting_level--;
@@ -467,10 +464,10 @@
u32 component_id, acpi_integer value)
{
- acpi_ut_debug_print(ACPI_LV_FUNCTIONS,
- line_number, function_name, module_name,
- component_id, "%s %8.8X%8.8X\n",
- acpi_gbl_fn_exit_str, ACPI_FORMAT_UINT64(value));
+ acpi_debug_print(ACPI_LV_FUNCTIONS,
+ line_number, function_name, module_name, component_id,
+ "%s %8.8X%8.8X\n", acpi_gbl_fn_exit_str,
+ ACPI_FORMAT_UINT64(value));
acpi_gbl_nesting_level--;
}
@@ -499,9 +496,9 @@
const char *module_name, u32 component_id, u8 *ptr)
{
- acpi_ut_debug_print(ACPI_LV_FUNCTIONS,
- line_number, function_name, module_name,
- component_id, "%s %p\n", acpi_gbl_fn_exit_str, ptr);
+ acpi_debug_print(ACPI_LV_FUNCTIONS,
+ line_number, function_name, module_name, component_id,
+ "%s %p\n", acpi_gbl_fn_exit_str, ptr);
acpi_gbl_nesting_level--;
}
diff --git a/drivers/acpi/utilities/utdelete.c b/drivers/acpi/acpica/utdelete.c
similarity index 99%
rename from drivers/acpi/utilities/utdelete.c
rename to drivers/acpi/acpica/utdelete.c
index d197c6b..a0be9e3 100644
--- a/drivers/acpi/utilities/utdelete.c
+++ b/drivers/acpi/acpica/utdelete.c
@@ -42,9 +42,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acinterp.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acevents.h>
+#include "accommon.h"
+#include "acinterp.h"
+#include "acnamesp.h"
+#include "acevents.h"
#define _COMPONENT ACPI_UTILITIES
ACPI_MODULE_NAME("utdelete")
diff --git a/drivers/acpi/utilities/uteval.c b/drivers/acpi/acpica/uteval.c
similarity index 98%
rename from drivers/acpi/utilities/uteval.c
rename to drivers/acpi/acpica/uteval.c
index 352747e..da9450b 100644
--- a/drivers/acpi/utilities/uteval.c
+++ b/drivers/acpi/acpica/uteval.c
@@ -42,8 +42,9 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acinterp.h>
+#include "accommon.h"
+#include "acnamesp.h"
+#include "acinterp.h"
#define _COMPONENT ACPI_UTILITIES
ACPI_MODULE_NAME("uteval")
@@ -129,7 +130,7 @@
/* The interface is supported */
- return_ACPI_STATUS(AE_CTRL_TERMINATE);
+ return_ACPI_STATUS(AE_OK);
}
}
@@ -143,13 +144,13 @@
/* The interface is supported */
- return_ACPI_STATUS(AE_CTRL_TERMINATE);
+ return_ACPI_STATUS(AE_OK);
}
/* The interface is not supported */
return_desc->integer.value = 0;
- return_ACPI_STATUS(AE_CTRL_TERMINATE);
+ return_ACPI_STATUS(AE_OK);
}
/*******************************************************************************
diff --git a/drivers/acpi/utilities/utglobal.c b/drivers/acpi/acpica/utglobal.c
similarity index 98%
rename from drivers/acpi/utilities/utglobal.c
rename to drivers/acpi/acpica/utglobal.c
index 17ed5ac..a3ab9d9 100644
--- a/drivers/acpi/utilities/utglobal.c
+++ b/drivers/acpi/acpica/utglobal.c
@@ -44,11 +44,11 @@
#define DEFINE_ACPI_GLOBALS
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acnamesp.h"
-ACPI_EXPORT_SYMBOL(acpi_gbl_FADT)
#define _COMPONENT ACPI_UTILITIES
- ACPI_MODULE_NAME("utglobal")
+ACPI_MODULE_NAME("utglobal")
/*******************************************************************************
*
@@ -352,7 +352,7 @@
"PCI_Config",
"EmbeddedControl",
"SMBus",
- "CMOS",
+ "SystemCMOS",
"PCIBARTarget",
"DataTable"
};
@@ -756,6 +756,7 @@
acpi_gbl_gpe_xrupt_list_head = NULL;
acpi_gbl_gpe_fadt_blocks[0] = NULL;
acpi_gbl_gpe_fadt_blocks[1] = NULL;
+ acpi_current_gpe_count = 0;
/* Global handlers */
@@ -771,6 +772,7 @@
acpi_gbl_global_lock_mutex = NULL;
acpi_gbl_global_lock_acquired = FALSE;
acpi_gbl_global_lock_handle = 0;
+ acpi_gbl_global_lock_present = FALSE;
/* Miscellaneous variables */
@@ -815,5 +817,7 @@
return_ACPI_STATUS(AE_OK);
}
+ACPI_EXPORT_SYMBOL(acpi_gbl_FADT)
ACPI_EXPORT_SYMBOL(acpi_dbg_level)
ACPI_EXPORT_SYMBOL(acpi_dbg_layer)
+ACPI_EXPORT_SYMBOL(acpi_current_gpe_count)
diff --git a/drivers/acpi/utilities/utinit.c b/drivers/acpi/acpica/utinit.c
similarity index 97%
rename from drivers/acpi/utilities/utinit.c
rename to drivers/acpi/acpica/utinit.c
index cae515f..a54ca84 100644
--- a/drivers/acpi/utilities/utinit.c
+++ b/drivers/acpi/acpica/utinit.c
@@ -42,9 +42,10 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acevents.h>
-#include <acpi/actables.h>
+#include "accommon.h"
+#include "acnamesp.h"
+#include "acevents.h"
+#include "actables.h"
#define _COMPONENT ACPI_UTILITIES
ACPI_MODULE_NAME("utinit")
diff --git a/drivers/acpi/utilities/utmath.c b/drivers/acpi/acpica/utmath.c
similarity index 99%
rename from drivers/acpi/utilities/utmath.c
rename to drivers/acpi/acpica/utmath.c
index c927324..c9f682d 100644
--- a/drivers/acpi/utilities/utmath.c
+++ b/drivers/acpi/acpica/utmath.c
@@ -42,6 +42,7 @@
*/
#include <acpi/acpi.h>
+#include "accommon.h"
#define _COMPONENT ACPI_UTILITIES
ACPI_MODULE_NAME("utmath")
diff --git a/drivers/acpi/utilities/utmisc.c b/drivers/acpi/acpica/utmisc.c
similarity index 97%
rename from drivers/acpi/utilities/utmisc.c
rename to drivers/acpi/acpica/utmisc.c
index 9089a15..c1f7f4e 100644
--- a/drivers/acpi/utilities/utmisc.c
+++ b/drivers/acpi/acpica/utmisc.c
@@ -44,7 +44,8 @@
#include <linux/module.h>
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_UTILITIES
ACPI_MODULE_NAME("utmisc")
@@ -1016,7 +1017,7 @@
/*******************************************************************************
*
- * FUNCTION: acpi_ut_error, acpi_ut_warning, acpi_ut_info
+ * FUNCTION: acpi_error, acpi_exception, acpi_warning, acpi_info
*
* PARAMETERS: module_name - Caller's module name (for error output)
* line_number - Caller's line number (for error output)
@@ -1029,7 +1030,7 @@
******************************************************************************/
void ACPI_INTERNAL_VAR_XFACE
-acpi_ut_error(const char *module_name, u32 line_number, const char *format, ...)
+acpi_error(const char *module_name, u32 line_number, const char *format, ...)
{
va_list args;
@@ -1042,8 +1043,8 @@
}
void ACPI_INTERNAL_VAR_XFACE
-acpi_ut_exception(const char *module_name,
- u32 line_number, acpi_status status, const char *format, ...)
+acpi_exception(const char *module_name,
+ u32 line_number, acpi_status status, const char *format, ...)
{
va_list args;
@@ -1056,11 +1057,8 @@
va_end(args);
}
-EXPORT_SYMBOL(acpi_ut_exception);
-
void ACPI_INTERNAL_VAR_XFACE
-acpi_ut_warning(const char *module_name,
- u32 line_number, const char *format, ...)
+acpi_warning(const char *module_name, u32 line_number, const char *format, ...)
{
va_list args;
@@ -1073,7 +1071,7 @@
}
void ACPI_INTERNAL_VAR_XFACE
-acpi_ut_info(const char *module_name, u32 line_number, const char *format, ...)
+acpi_info(const char *module_name, u32 line_number, const char *format, ...)
{
va_list args;
@@ -1088,3 +1086,8 @@
acpi_os_printf("\n");
va_end(args);
}
+
+ACPI_EXPORT_SYMBOL(acpi_error)
+ACPI_EXPORT_SYMBOL(acpi_exception)
+ACPI_EXPORT_SYMBOL(acpi_warning)
+ACPI_EXPORT_SYMBOL(acpi_info)
diff --git a/drivers/acpi/utilities/utmutex.c b/drivers/acpi/acpica/utmutex.c
similarity index 99%
rename from drivers/acpi/utilities/utmutex.c
rename to drivers/acpi/acpica/utmutex.c
index 7331dde..14eb52c 100644
--- a/drivers/acpi/utilities/utmutex.c
+++ b/drivers/acpi/acpica/utmutex.c
@@ -42,6 +42,7 @@
*/
#include <acpi/acpi.h>
+#include "accommon.h"
#define _COMPONENT ACPI_UTILITIES
ACPI_MODULE_NAME("utmutex")
diff --git a/drivers/acpi/utilities/utobject.c b/drivers/acpi/acpica/utobject.c
similarity index 99%
rename from drivers/acpi/utilities/utobject.c
rename to drivers/acpi/acpica/utobject.c
index 4bef3cf..fd5ea75 100644
--- a/drivers/acpi/utilities/utobject.c
+++ b/drivers/acpi/acpica/utobject.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
+#include "accommon.h"
+#include "acnamesp.h"
#define _COMPONENT ACPI_UTILITIES
ACPI_MODULE_NAME("utobject")
diff --git a/drivers/acpi/utilities/utresrc.c b/drivers/acpi/acpica/utresrc.c
similarity index 99%
rename from drivers/acpi/utilities/utresrc.c
rename to drivers/acpi/acpica/utresrc.c
index c3e3e13..91b7c00 100644
--- a/drivers/acpi/utilities/utresrc.c
+++ b/drivers/acpi/acpica/utresrc.c
@@ -42,7 +42,8 @@
*/
#include <acpi/acpi.h>
-#include <acpi/amlresrc.h>
+#include "accommon.h"
+#include "amlresrc.h"
#define _COMPONENT ACPI_UTILITIES
ACPI_MODULE_NAME("utresrc")
diff --git a/drivers/acpi/utilities/utstate.c b/drivers/acpi/acpica/utstate.c
similarity index 99%
rename from drivers/acpi/utilities/utstate.c
rename to drivers/acpi/acpica/utstate.c
index 63a6d3d..0440c95 100644
--- a/drivers/acpi/utilities/utstate.c
+++ b/drivers/acpi/acpica/utstate.c
@@ -42,6 +42,7 @@
*/
#include <acpi/acpi.h>
+#include "accommon.h"
#define _COMPONENT ACPI_UTILITIES
ACPI_MODULE_NAME("utstate")
diff --git a/drivers/acpi/utilities/utxface.c b/drivers/acpi/acpica/utxface.c
similarity index 96%
rename from drivers/acpi/utilities/utxface.c
rename to drivers/acpi/acpica/utxface.c
index c198a4d..078a227 100644
--- a/drivers/acpi/utilities/utxface.c
+++ b/drivers/acpi/acpica/utxface.c
@@ -42,9 +42,11 @@
*/
#include <acpi/acpi.h>
-#include <acpi/acevents.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acdebug.h>
+#include "accommon.h"
+#include "acevents.h"
+#include "acnamesp.h"
+#include "acdebug.h"
+#include "actables.h"
#define _COMPONENT ACPI_UTILITIES
ACPI_MODULE_NAME("utxface")
@@ -148,6 +150,16 @@
}
/*
+ * Obtain a permanent mapping for the FACS. This is required for the
+ * Global Lock and the Firmware Waking Vector
+ */
+ status = acpi_tb_initialize_facs();
+ if (ACPI_FAILURE(status)) {
+ ACPI_WARNING((AE_INFO, "Could not map the FACS table"));
+ return_ACPI_STATUS(status);
+ }
+
+ /*
* Install the default op_region handlers. These are installed unless
* other handlers have already been installed via the
* install_address_space_handler interface.
diff --git a/drivers/acpi/battery.c b/drivers/acpi/battery.c
index 1423b0c..65132f9 100644
--- a/drivers/acpi/battery.c
+++ b/drivers/acpi/battery.c
@@ -471,7 +471,7 @@
static int acpi_battery_update(struct acpi_battery *battery)
{
- int result;
+ int result, old_present = acpi_battery_present(battery);
result = acpi_battery_get_status(battery);
if (result)
return result;
@@ -482,7 +482,8 @@
return 0;
}
#endif
- if (!battery->update_time) {
+ if (!battery->update_time ||
+ old_present != acpi_battery_present(battery)) {
result = acpi_battery_get_info(battery);
if (result)
return result;
diff --git a/drivers/acpi/cm_sbs.c b/drivers/acpi/cm_sbs.c
index 307963b..332fe4b 100644
--- a/drivers/acpi/cm_sbs.c
+++ b/drivers/acpi/cm_sbs.c
@@ -27,9 +27,6 @@
#include <linux/seq_file.h>
#include <acpi/acpi_bus.h>
#include <acpi/acpi_drivers.h>
-#include <acpi/acmacros.h>
-#include <acpi/actypes.h>
-#include <acpi/acutils.h>
ACPI_MODULE_NAME("cm_sbs");
#define ACPI_AC_CLASS "ac_adapter"
diff --git a/drivers/acpi/debug.c b/drivers/acpi/debug.c
index c483968..20223cb 100644
--- a/drivers/acpi/debug.c
+++ b/drivers/acpi/debug.c
@@ -9,7 +9,6 @@
#include <linux/moduleparam.h>
#include <asm/uaccess.h>
#include <acpi/acpi_drivers.h>
-#include <acpi/acglobal.h>
#define _COMPONENT ACPI_SYSTEM_COMPONENT
ACPI_MODULE_NAME("debug");
diff --git a/drivers/acpi/dispatcher/Makefile b/drivers/acpi/dispatcher/Makefile
deleted file mode 100644
index eb7e602..0000000
--- a/drivers/acpi/dispatcher/Makefile
+++ /dev/null
@@ -1,9 +0,0 @@
-#
-# Makefile for all Linux ACPI interpreter subdirectories
-#
-
-obj-y := dsfield.o dsmthdat.o dsopcode.o dswexec.o dswscope.o \
- dsmethod.o dsobject.o dsutils.o dswload.o dswstate.o \
- dsinit.o
-
-EXTRA_CFLAGS += $(ACPI_CFLAGS)
diff --git a/drivers/acpi/ec.c b/drivers/acpi/ec.c
index 30f3ef2..8dfcbb8 100644
--- a/drivers/acpi/ec.c
+++ b/drivers/acpi/ec.c
@@ -42,7 +42,6 @@
#include <asm/io.h>
#include <acpi/acpi_bus.h>
#include <acpi/acpi_drivers.h>
-#include <acpi/actypes.h>
#define ACPI_EC_CLASS "embedded_controller"
#define ACPI_EC_DEVICE_NAME "Embedded Controller"
@@ -370,7 +369,7 @@
* Note: samsung nv5000 doesn't work with ec burst mode.
* http://bugzilla.kernel.org/show_bug.cgi?id=4980
*/
-int acpi_ec_burst_enable(struct acpi_ec *ec)
+static int acpi_ec_burst_enable(struct acpi_ec *ec)
{
u8 d;
struct transaction t = {.command = ACPI_EC_BURST_ENABLE,
@@ -380,7 +379,7 @@
return acpi_ec_transaction(ec, &t, 0);
}
-int acpi_ec_burst_disable(struct acpi_ec *ec)
+static int acpi_ec_burst_disable(struct acpi_ec *ec)
{
struct transaction t = {.command = ACPI_EC_BURST_DISABLE,
.wdata = NULL, .rdata = NULL,
@@ -756,10 +755,15 @@
acpi_ec_register_query_methods(acpi_handle handle, u32 level,
void *context, void **return_value)
{
- struct acpi_namespace_node *node = handle;
+ char node_name[5];
+ struct acpi_buffer buffer = { sizeof(node_name), node_name };
struct acpi_ec *ec = context;
int value = 0;
- if (sscanf(node->name.ascii, "_Q%x", &value) == 1) {
+ acpi_status status;
+
+ status = acpi_get_name(handle, ACPI_SINGLE_NAME, &buffer);
+
+ if (ACPI_SUCCESS(status) && sscanf(node_name, "_Q%x", &value) == 1) {
acpi_ec_add_query_handler(ec, value, handle, NULL, NULL);
}
return AE_OK;
@@ -978,9 +982,9 @@
int __init acpi_ec_ecdt_probe(void)
{
- int ret;
acpi_status status;
struct acpi_table_ecdt *ecdt_ptr;
+ acpi_handle dummy;
boot_ec = make_acpi_ec();
if (!boot_ec)
@@ -1006,30 +1010,31 @@
boot_ec->gpe = ecdt_ptr->gpe;
boot_ec->handle = ACPI_ROOT_OBJECT;
acpi_get_handle(ACPI_ROOT_OBJECT, ecdt_ptr->id, &boot_ec->handle);
- } else {
- /* This workaround is needed only on some broken machines,
- * which require early EC, but fail to provide ECDT */
- acpi_handle x;
- printk(KERN_DEBUG PREFIX "Look up EC in DSDT\n");
- status = acpi_get_devices(ec_device_ids[0].id, ec_parse_device,
- boot_ec, NULL);
- /* Check that acpi_get_devices actually find something */
- if (ACPI_FAILURE(status) || !boot_ec->handle)
- goto error;
- /* We really need to limit this workaround, the only ASUS,
- * which needs it, has fake EC._INI method, so use it as flag.
- * Keep boot_ec struct as it will be needed soon.
- */
- if (ACPI_FAILURE(acpi_get_handle(boot_ec->handle, "_INI", &x)))
- return -ENODEV;
+ /* Add some basic check against completely broken table */
+ if (boot_ec->data_addr != boot_ec->command_addr)
+ goto install;
+ /* fall through */
}
-
- ret = ec_install_handlers(boot_ec);
- if (!ret) {
+ /* This workaround is needed only on some broken machines,
+ * which require early EC, but fail to provide ECDT */
+ printk(KERN_DEBUG PREFIX "Look up EC in DSDT\n");
+ status = acpi_get_devices(ec_device_ids[0].id, ec_parse_device,
+ boot_ec, NULL);
+ /* Check that acpi_get_devices actually find something */
+ if (ACPI_FAILURE(status) || !boot_ec->handle)
+ goto error;
+ /* We really need to limit this workaround, the only ASUS,
+ * which needs it, has fake EC._INI method, so use it as flag.
+ * Keep boot_ec struct as it will be needed soon.
+ */
+ if (ACPI_FAILURE(acpi_get_handle(boot_ec->handle, "_INI", &dummy)))
+ return -ENODEV;
+install:
+ if (!ec_install_handlers(boot_ec)) {
first_ec = boot_ec;
return 0;
}
- error:
+error:
kfree(boot_ec);
boot_ec = NULL;
return -ENODEV;
diff --git a/drivers/acpi/events/Makefile b/drivers/acpi/events/Makefile
deleted file mode 100644
index d29f2ee..0000000
--- a/drivers/acpi/events/Makefile
+++ /dev/null
@@ -1,9 +0,0 @@
-#
-# Makefile for all Linux ACPI interpreter subdirectories
-#
-
-obj-y := evevent.o evregion.o evsci.o evxfevnt.o \
- evmisc.o evrgnini.o evxface.o evxfregn.o \
- evgpe.o evgpeblk.o
-
-EXTRA_CFLAGS += $(ACPI_CFLAGS)
diff --git a/drivers/acpi/executer/Makefile b/drivers/acpi/executer/Makefile
deleted file mode 100644
index e09998a..0000000
--- a/drivers/acpi/executer/Makefile
+++ /dev/null
@@ -1,10 +0,0 @@
-#
-# Makefile for all Linux ACPI interpreter subdirectories
-#
-
-obj-y := exconfig.o exfield.o exnames.o exoparg6.o exresolv.o exstorob.o\
- exconvrt.o exfldio.o exoparg1.o exprep.o exresop.o exsystem.o\
- excreate.o exmisc.o exoparg2.o exregion.o exstore.o exutils.o \
- exdump.o exmutex.o exoparg3.o exresnte.o exstoren.o
-
-EXTRA_CFLAGS += $(ACPI_CFLAGS)
diff --git a/drivers/acpi/hardware/Makefile b/drivers/acpi/hardware/Makefile
deleted file mode 100644
index 438ad37..0000000
--- a/drivers/acpi/hardware/Makefile
+++ /dev/null
@@ -1,9 +0,0 @@
-#
-# Makefile for all Linux ACPI interpreter subdirectories
-#
-
-obj-y := hwacpi.o hwgpe.o hwregs.o hwsleep.o
-
-obj-$(ACPI_FUTURE_USAGE) += hwtimer.o
-
-EXTRA_CFLAGS += $(ACPI_CFLAGS)
diff --git a/drivers/acpi/sleep/main.c b/drivers/acpi/main.c
similarity index 92%
rename from drivers/acpi/sleep/main.c
rename to drivers/acpi/main.c
index 28a691cc..7e3c609 100644
--- a/drivers/acpi/sleep/main.c
+++ b/drivers/acpi/main.c
@@ -101,13 +101,26 @@
* cases.
*/
static bool set_sci_en_on_resume;
+/*
+ * The ACPI specification wants us to save NVS memory regions during hibernation
+ * and to restore them during the subsequent resume. However, it is not certain
+ * if this mechanism is going to work on all machines, so we allow the user to
+ * disable this mechanism using the 'acpi_sleep=s4_nonvs' kernel command line
+ * option.
+ */
+static bool s4_no_nvs;
+
+void __init acpi_s4_no_nvs(void)
+{
+ s4_no_nvs = true;
+}
/**
* acpi_pm_disable_gpes - Disable the GPEs.
*/
static int acpi_pm_disable_gpes(void)
{
- acpi_hw_disable_all_gpes();
+ acpi_disable_all_gpes();
return 0;
}
@@ -135,7 +148,7 @@
int error = __acpi_pm_prepare();
if (!error)
- acpi_hw_disable_all_gpes();
+ acpi_disable_all_gpes();
return error;
}
@@ -267,7 +280,7 @@
* (like wakeup GPE) haven't handler, this can avoid such GPE misfire.
* acpi_leave_sleep_state will reenable specific GPEs later
*/
- acpi_hw_disable_all_gpes();
+ acpi_disable_all_gpes();
local_irq_restore(flags);
printk(KERN_DEBUG "Back to C!\n");
@@ -394,9 +407,25 @@
static int acpi_hibernation_begin(void)
{
- acpi_target_sleep_state = ACPI_STATE_S4;
- acpi_sleep_tts_switch(acpi_target_sleep_state);
- return 0;
+ int error;
+
+ error = s4_no_nvs ? 0 : hibernate_nvs_alloc();
+ if (!error) {
+ acpi_target_sleep_state = ACPI_STATE_S4;
+ acpi_sleep_tts_switch(acpi_target_sleep_state);
+ }
+
+ return error;
+}
+
+static int acpi_hibernation_pre_snapshot(void)
+{
+ int error = acpi_pm_prepare();
+
+ if (!error)
+ hibernate_nvs_save();
+
+ return error;
}
static int acpi_hibernation_enter(void)
@@ -417,6 +446,12 @@
return ACPI_SUCCESS(status) ? 0 : -EFAULT;
}
+static void acpi_hibernation_finish(void)
+{
+ hibernate_nvs_free();
+ acpi_pm_finish();
+}
+
static void acpi_hibernation_leave(void)
{
/*
@@ -432,18 +467,20 @@
"cannot resume!\n");
panic("ACPI S4 hardware signature mismatch");
}
+ /* Restore the NVS memory area */
+ hibernate_nvs_restore();
}
static void acpi_pm_enable_gpes(void)
{
- acpi_hw_enable_all_runtime_gpes();
+ acpi_enable_all_runtime_gpes();
}
static struct platform_hibernation_ops acpi_hibernation_ops = {
.begin = acpi_hibernation_begin,
.end = acpi_pm_end,
- .pre_snapshot = acpi_pm_prepare,
- .finish = acpi_pm_finish,
+ .pre_snapshot = acpi_hibernation_pre_snapshot,
+ .finish = acpi_hibernation_finish,
.prepare = acpi_pm_prepare,
.enter = acpi_hibernation_enter,
.leave = acpi_hibernation_leave,
@@ -469,8 +506,22 @@
error = acpi_sleep_prepare(ACPI_STATE_S4);
+ if (!error) {
+ if (!s4_no_nvs)
+ error = hibernate_nvs_alloc();
+ if (!error)
+ acpi_target_sleep_state = ACPI_STATE_S4;
+ }
+ return error;
+}
+
+static int acpi_hibernation_pre_snapshot_old(void)
+{
+ int error = acpi_pm_disable_gpes();
+
if (!error)
- acpi_target_sleep_state = ACPI_STATE_S4;
+ hibernate_nvs_save();
+
return error;
}
@@ -481,8 +532,8 @@
static struct platform_hibernation_ops acpi_hibernation_ops_old = {
.begin = acpi_hibernation_begin_old,
.end = acpi_pm_end,
- .pre_snapshot = acpi_pm_disable_gpes,
- .finish = acpi_pm_finish,
+ .pre_snapshot = acpi_hibernation_pre_snapshot_old,
+ .finish = acpi_hibernation_finish,
.prepare = acpi_pm_disable_gpes,
.enter = acpi_hibernation_enter,
.leave = acpi_hibernation_leave,
@@ -622,7 +673,7 @@
{
/* Prepare to power off the system */
acpi_sleep_prepare(ACPI_STATE_S5);
- acpi_hw_disable_all_gpes();
+ acpi_disable_all_gpes();
}
static void acpi_power_off(void)
@@ -671,7 +722,7 @@
sleep_states[ACPI_STATE_S4] = 1;
printk(" S4");
if (!nosigcheck) {
- acpi_get_table_by_index(ACPI_TABLE_INDEX_FACS,
+ acpi_get_table(ACPI_SIG_FACS, 1,
(struct acpi_table_header **)&facs);
if (facs)
s4_hardware_signature =
diff --git a/drivers/acpi/namespace/Makefile b/drivers/acpi/namespace/Makefile
deleted file mode 100644
index 371a2da..0000000
--- a/drivers/acpi/namespace/Makefile
+++ /dev/null
@@ -1,12 +0,0 @@
-#
-# Makefile for all Linux ACPI interpreter subdirectories
-#
-
-obj-y := nsaccess.o nsload.o nssearch.o nsxfeval.o \
- nsalloc.o nseval.o nsnames.o nsutils.o nsxfname.o \
- nsdump.o nsinit.o nsobject.o nswalk.o nsxfobj.o \
- nsparse.o nspredef.o
-
-obj-$(ACPI_FUTURE_USAGE) += nsdumpdv.o
-
-EXTRA_CFLAGS += $(ACPI_CFLAGS)
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 25ceae9..c5e292a 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -29,7 +29,6 @@
#include <linux/errno.h>
#include <linux/acpi.h>
#include <acpi/acpi_bus.h>
-#include <acpi/acmacros.h>
#define ACPI_NUMA 0x80000000
#define _COMPONENT ACPI_NUMA
diff --git a/drivers/acpi/osl.c b/drivers/acpi/osl.c
index c811142..6729a49 100644
--- a/drivers/acpi/osl.c
+++ b/drivers/acpi/osl.c
@@ -726,7 +726,7 @@
dpc = kmalloc(sizeof(struct acpi_os_dpc), GFP_ATOMIC);
if (!dpc)
- return_ACPI_STATUS(AE_NO_MEMORY);
+ return AE_NO_MEMORY;
dpc->function = function;
dpc->context = context;
@@ -747,7 +747,7 @@
status = AE_ERROR;
kfree(dpc);
}
- return_ACPI_STATUS(status);
+ return status;
}
acpi_status acpi_os_execute(acpi_execute_type type,
diff --git a/drivers/acpi/parser/Makefile b/drivers/acpi/parser/Makefile
deleted file mode 100644
index db24ee0..0000000
--- a/drivers/acpi/parser/Makefile
+++ /dev/null
@@ -1,8 +0,0 @@
-#
-# Makefile for all Linux ACPI interpreter subdirectories
-#
-
-obj-y := psargs.o psparse.o psloop.o pstree.o pswalk.o \
- psopcode.o psscope.o psutils.o psxface.o
-
-EXTRA_CFLAGS += $(ACPI_CFLAGS)
diff --git a/drivers/acpi/pci_bind.c b/drivers/acpi/pci_bind.c
index 4b252ea..95650f8 100644
--- a/drivers/acpi/pci_bind.c
+++ b/drivers/acpi/pci_bind.c
@@ -99,7 +99,7 @@
*/
ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Device %s has PCI address %02x:%02x:%02x.%02x\n",
+ "Device %s has PCI address %04x:%02x:%02x.%d\n",
acpi_device_bid(device), id->segment, id->bus,
id->device, id->function));
@@ -111,12 +111,11 @@
int acpi_pci_bind(struct acpi_device *device)
{
int result = 0;
- acpi_status status = AE_OK;
- struct acpi_pci_data *data = NULL;
- struct acpi_pci_data *pdata = NULL;
- char *pathname = NULL;
- struct acpi_buffer buffer = { 0, NULL };
- acpi_handle handle = NULL;
+ acpi_status status;
+ struct acpi_pci_data *data;
+ struct acpi_pci_data *pdata;
+ struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL };
+ acpi_handle handle;
struct pci_dev *dev;
struct pci_bus *bus;
@@ -124,21 +123,18 @@
if (!device || !device->parent)
return -EINVAL;
- pathname = kzalloc(ACPI_PATHNAME_MAX, GFP_KERNEL);
- if (!pathname)
- return -ENOMEM;
- buffer.length = ACPI_PATHNAME_MAX;
- buffer.pointer = pathname;
-
data = kzalloc(sizeof(struct acpi_pci_data), GFP_KERNEL);
- if (!data) {
- kfree(pathname);
+ if (!data)
return -ENOMEM;
+
+ status = acpi_get_name(device->handle, ACPI_FULL_PATHNAME, &buffer);
+ if (ACPI_FAILURE(status)) {
+ kfree(data);
+ return -ENODEV;
}
- acpi_get_name(device->handle, ACPI_FULL_PATHNAME, &buffer);
ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Binding PCI device [%s]...\n",
- pathname));
+ (char *)buffer.pointer));
/*
* Segment & Bus
@@ -166,7 +162,7 @@
data->id.device = device->pnp.bus_address >> 16;
data->id.function = device->pnp.bus_address & 0xFFFF;
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "...to %02x:%02x:%02x.%02x\n",
+ ACPI_DEBUG_PRINT((ACPI_DB_INFO, "...to %04x:%02x:%02x.%d\n",
data->id.segment, data->id.bus, data->id.device,
data->id.function));
@@ -196,7 +192,7 @@
}
if (!data->dev) {
ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Device %02x:%02x:%02x.%02x not present in PCI namespace\n",
+ "Device %04x:%02x:%02x.%d not present in PCI namespace\n",
data->id.segment, data->id.bus,
data->id.device, data->id.function));
result = -ENODEV;
@@ -204,7 +200,7 @@
}
if (!data->dev->bus) {
printk(KERN_ERR PREFIX
- "Device %02x:%02x:%02x.%02x has invalid 'bus' field\n",
+ "Device %04x:%02x:%02x.%d has invalid 'bus' field\n",
data->id.segment, data->id.bus,
data->id.device, data->id.function);
result = -ENODEV;
@@ -219,7 +215,7 @@
*/
if (data->dev->subordinate) {
ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Device %02x:%02x:%02x.%02x is a PCI bridge\n",
+ "Device %04x:%02x:%02x.%d is a PCI bridge\n",
data->id.segment, data->id.bus,
data->id.device, data->id.function));
data->bus = data->dev->subordinate;
@@ -262,7 +258,7 @@
}
end:
- kfree(pathname);
+ kfree(buffer.pointer);
if (result)
kfree(data);
@@ -272,25 +268,21 @@
static int acpi_pci_unbind(struct acpi_device *device)
{
int result = 0;
- acpi_status status = AE_OK;
- struct acpi_pci_data *data = NULL;
- char *pathname = NULL;
- struct acpi_buffer buffer = { 0, NULL };
+ acpi_status status;
+ struct acpi_pci_data *data;
+ struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL };
if (!device || !device->parent)
return -EINVAL;
- pathname = kzalloc(ACPI_PATHNAME_MAX, GFP_KERNEL);
- if (!pathname)
- return -ENOMEM;
+ status = acpi_get_name(device->handle, ACPI_FULL_PATHNAME, &buffer);
+ if (ACPI_FAILURE(status))
+ return -ENODEV;
- buffer.length = ACPI_PATHNAME_MAX;
- buffer.pointer = pathname;
- acpi_get_name(device->handle, ACPI_FULL_PATHNAME, &buffer);
ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Unbinding PCI device [%s]...\n",
- pathname));
- kfree(pathname);
+ (char *) buffer.pointer));
+ kfree(buffer.pointer);
status =
acpi_get_data(device->handle, acpi_pci_data_handler,
@@ -322,50 +314,44 @@
struct acpi_pci_id *id, struct pci_bus *bus)
{
int result = 0;
- acpi_status status = AE_OK;
+ acpi_status status;
struct acpi_pci_data *data = NULL;
- char *pathname = NULL;
- struct acpi_buffer buffer = { 0, NULL };
-
- pathname = kzalloc(ACPI_PATHNAME_MAX, GFP_KERNEL);
- if (!pathname)
- return -ENOMEM;
-
- buffer.length = ACPI_PATHNAME_MAX;
- buffer.pointer = pathname;
+ struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL };
if (!device || !id || !bus) {
- kfree(pathname);
return -EINVAL;
}
data = kzalloc(sizeof(struct acpi_pci_data), GFP_KERNEL);
- if (!data) {
- kfree(pathname);
+ if (!data)
return -ENOMEM;
- }
data->id = *id;
data->bus = bus;
device->ops.bind = acpi_pci_bind;
device->ops.unbind = acpi_pci_unbind;
- acpi_get_name(device->handle, ACPI_FULL_PATHNAME, &buffer);
+ status = acpi_get_name(device->handle, ACPI_FULL_PATHNAME, &buffer);
+ if (ACPI_FAILURE(status)) {
+ kfree (data);
+ return -ENODEV;
+ }
ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Binding PCI root bridge [%s] to "
- "%02x:%02x\n", pathname, id->segment, id->bus));
+ "%04x:%02x\n", (char *)buffer.pointer,
+ id->segment, id->bus));
status = acpi_attach_data(device->handle, acpi_pci_data_handler, data);
if (ACPI_FAILURE(status)) {
ACPI_EXCEPTION((AE_INFO, status,
"Unable to attach ACPI-PCI context to device %s",
- pathname));
+ (char *)buffer.pointer));
result = -ENODEV;
goto end;
}
end:
- kfree(pathname);
+ kfree(buffer.pointer);
if (result != 0)
kfree(data);
diff --git a/drivers/acpi/pci_irq.c b/drivers/acpi/pci_irq.c
index bf79d83..891bdf6 100644
--- a/drivers/acpi/pci_irq.c
+++ b/drivers/acpi/pci_irq.c
@@ -4,6 +4,8 @@
* Copyright (C) 2001, 2002 Andy Grover <andrew.grover@intel.com>
* Copyright (C) 2001, 2002 Paul Diefenbaugh <paul.s.diefenbaugh@intel.com>
* Copyright (C) 2002 Dominik Brodowski <devel@brodo.de>
+ * (c) Copyright 2008 Hewlett-Packard Development Company, L.P.
+ * Bjorn Helgaas <bjorn.helgaas@hp.com>
*
* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*
@@ -41,29 +43,36 @@
#define _COMPONENT ACPI_PCI_COMPONENT
ACPI_MODULE_NAME("pci_irq");
-static struct acpi_prt_list acpi_prt;
+struct acpi_prt_entry {
+ struct list_head list;
+ struct acpi_pci_id id;
+ u8 pin;
+ acpi_handle link;
+ u32 index; /* GSI, or link _CRS index */
+};
+
+static LIST_HEAD(acpi_prt_list);
static DEFINE_SPINLOCK(acpi_prt_lock);
+static inline char pin_name(int pin)
+{
+ return 'A' + pin - 1;
+}
+
/* --------------------------------------------------------------------------
PCI IRQ Routing Table (PRT) Support
-------------------------------------------------------------------------- */
-static struct acpi_prt_entry *acpi_pci_irq_find_prt_entry(int segment,
- int bus,
- int device, int pin)
+static struct acpi_prt_entry *acpi_pci_irq_find_prt_entry(struct pci_dev *dev,
+ int pin)
{
- struct acpi_prt_entry *entry = NULL;
+ struct acpi_prt_entry *entry;
+ int segment = pci_domain_nr(dev->bus);
+ int bus = dev->bus->number;
+ int device = PCI_SLOT(dev->devfn);
- if (!acpi_prt.count)
- return NULL;
-
- /*
- * Parse through all PRT entries looking for a match on the specified
- * PCI device's segment, bus, device, and pin (don't care about func).
- *
- */
spin_lock(&acpi_prt_lock);
- list_for_each_entry(entry, &acpi_prt.entries, node) {
+ list_for_each_entry(entry, &acpi_prt_list, list) {
if ((segment == entry->id.segment)
&& (bus == entry->id.bus)
&& (device == entry->id.device)
@@ -72,7 +81,6 @@
return entry;
}
}
-
spin_unlock(&acpi_prt_lock);
return NULL;
}
@@ -124,25 +132,27 @@
char *actual_source;
};
+#define PCI_INTX_PIN(c) (c - 'A' + 1)
+
/*
* These systems have incorrect _PRT entries. The BIOS claims the PCI
* interrupt at the listed segment/bus/device/pin is connected to the first
* link device, but it is actually connected to the second.
*/
static struct prt_quirk prt_quirks[] = {
- { medion_md9580, 0, 0, 9, 'A',
+ { medion_md9580, 0, 0, 9, PCI_INTX_PIN('A'),
"\\_SB_.PCI0.ISA_.LNKA",
"\\_SB_.PCI0.ISA_.LNKB"},
- { dell_optiplex, 0, 0, 0xd, 'A',
+ { dell_optiplex, 0, 0, 0xd, PCI_INTX_PIN('A'),
"\\_SB_.LNKB",
"\\_SB_.LNKA"},
- { hp_t5710, 0, 0, 1, 'A',
+ { hp_t5710, 0, 0, 1, PCI_INTX_PIN('A'),
"\\_SB_.PCI0.LNK1",
"\\_SB_.PCI0.LNK3"},
};
-static void
-do_prt_fixups(struct acpi_prt_entry *entry, struct acpi_pci_routing_table *prt)
+static void do_prt_fixups(struct acpi_prt_entry *entry,
+ struct acpi_pci_routing_table *prt)
{
int i;
struct prt_quirk *quirk;
@@ -158,42 +168,43 @@
entry->id.segment == quirk->segment &&
entry->id.bus == quirk->bus &&
entry->id.device == quirk->device &&
- entry->pin + 'A' == quirk->pin &&
+ entry->pin == quirk->pin &&
!strcmp(prt->source, quirk->source) &&
strlen(prt->source) >= strlen(quirk->actual_source)) {
printk(KERN_WARNING PREFIX "firmware reports "
"%04x:%02x:%02x PCI INT %c connected to %s; "
"changing to %s\n",
entry->id.segment, entry->id.bus,
- entry->id.device, 'A' + entry->pin,
+ entry->id.device, pin_name(entry->pin),
prt->source, quirk->actual_source);
strcpy(prt->source, quirk->actual_source);
}
}
}
-static int
-acpi_pci_irq_add_entry(acpi_handle handle,
- int segment, int bus, struct acpi_pci_routing_table *prt)
+static int acpi_pci_irq_add_entry(acpi_handle handle, int segment, int bus,
+ struct acpi_pci_routing_table *prt)
{
- struct acpi_prt_entry *entry = NULL;
-
-
- if (!prt)
- return -EINVAL;
+ struct acpi_prt_entry *entry;
entry = kzalloc(sizeof(struct acpi_prt_entry), GFP_KERNEL);
if (!entry)
return -ENOMEM;
+ /*
+ * Note that the _PRT uses 0=INTA, 1=INTB, etc, while PCI uses
+ * 1=INTA, 2=INTB. We use the PCI encoding throughout, so convert
+ * it here.
+ */
entry->id.segment = segment;
entry->id.bus = bus;
entry->id.device = (prt->address >> 16) & 0xFFFF;
- entry->id.function = prt->address & 0xFFFF;
- entry->pin = prt->pin;
+ entry->pin = prt->pin + 1;
do_prt_fixups(entry, prt);
+ entry->index = prt->source_index;
+
/*
* Type 1: Dynamic
* ---------------
@@ -207,10 +218,9 @@
* (e.g. exists somewhere 'below' this _PRT entry in the ACPI
* namespace).
*/
- if (prt->source[0]) {
- acpi_get_handle(handle, prt->source, &entry->link.handle);
- entry->link.index = prt->source_index;
- }
+ if (prt->source[0])
+ acpi_get_handle(handle, prt->source, &entry->link);
+
/*
* Type 2: Static
* --------------
@@ -218,84 +228,38 @@
* the IRQ value, which is hardwired to specific interrupt inputs on
* the interrupt controller.
*/
- else
- entry->link.index = prt->source_index;
ACPI_DEBUG_PRINT_RAW((ACPI_DB_INFO,
- " %02X:%02X:%02X[%c] -> %s[%d]\n",
+ " %04x:%02x:%02x[%c] -> %s[%d]\n",
entry->id.segment, entry->id.bus,
- entry->id.device, ('A' + entry->pin), prt->source,
- entry->link.index));
+ entry->id.device, pin_name(entry->pin),
+ prt->source, entry->index));
spin_lock(&acpi_prt_lock);
- list_add_tail(&entry->node, &acpi_prt.entries);
- acpi_prt.count++;
+ list_add_tail(&entry->list, &acpi_prt_list);
spin_unlock(&acpi_prt_lock);
return 0;
}
-static void
-acpi_pci_irq_del_entry(int segment, int bus, struct acpi_prt_entry *entry)
-{
- if (segment == entry->id.segment && bus == entry->id.bus) {
- acpi_prt.count--;
- list_del(&entry->node);
- kfree(entry);
- }
-}
-
int acpi_pci_irq_add_prt(acpi_handle handle, int segment, int bus)
{
- acpi_status status = AE_OK;
- char *pathname = NULL;
- struct acpi_buffer buffer = { 0, NULL };
- struct acpi_pci_routing_table *prt = NULL;
- struct acpi_pci_routing_table *entry = NULL;
- static int first_time = 1;
+ acpi_status status;
+ struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL };
+ struct acpi_pci_routing_table *entry;
-
- pathname = kzalloc(ACPI_PATHNAME_MAX, GFP_KERNEL);
- if (!pathname)
- return -ENOMEM;
-
- if (first_time) {
- acpi_prt.count = 0;
- INIT_LIST_HEAD(&acpi_prt.entries);
- first_time = 0;
- }
-
- /*
- * NOTE: We're given a 'handle' to the _PRT object's parent device
- * (either a PCI root bridge or PCI-PCI bridge).
- */
-
- buffer.length = ACPI_PATHNAME_MAX;
- buffer.pointer = pathname;
- acpi_get_name(handle, ACPI_FULL_PATHNAME, &buffer);
+ /* 'handle' is the _PRT's parent (root bridge or PCI-PCI bridge) */
+ status = acpi_get_name(handle, ACPI_FULL_PATHNAME, &buffer);
+ if (ACPI_FAILURE(status))
+ return -ENODEV;
printk(KERN_DEBUG "ACPI: PCI Interrupt Routing Table [%s._PRT]\n",
- pathname);
+ (char *) buffer.pointer);
- /*
- * Evaluate this _PRT and add its entries to our global list (acpi_prt).
- */
+ kfree(buffer.pointer);
- buffer.length = 0;
+ buffer.length = ACPI_ALLOCATE_BUFFER;
buffer.pointer = NULL;
- kfree(pathname);
- status = acpi_get_irq_routing_table(handle, &buffer);
- if (status != AE_BUFFER_OVERFLOW) {
- ACPI_EXCEPTION((AE_INFO, status, "Evaluating _PRT [%s]",
- acpi_format_exception(status)));
- return -ENODEV;
- }
-
- prt = kzalloc(buffer.length, GFP_KERNEL);
- if (!prt) {
- return -ENOMEM;
- }
- buffer.pointer = prt;
status = acpi_get_irq_routing_table(handle, &buffer);
if (ACPI_FAILURE(status)) {
@@ -305,36 +269,30 @@
return -ENODEV;
}
- entry = prt;
-
+ entry = buffer.pointer;
while (entry && (entry->length > 0)) {
acpi_pci_irq_add_entry(handle, segment, bus, entry);
entry = (struct acpi_pci_routing_table *)
((unsigned long)entry + entry->length);
}
- kfree(prt);
-
+ kfree(buffer.pointer);
return 0;
}
void acpi_pci_irq_del_prt(int segment, int bus)
{
- struct list_head *node = NULL, *n = NULL;
- struct acpi_prt_entry *entry = NULL;
-
- if (!acpi_prt.count) {
- return;
- }
+ struct acpi_prt_entry *entry, *tmp;
printk(KERN_DEBUG
- "ACPI: Delete PCI Interrupt Routing Table for %x:%x\n", segment,
- bus);
+ "ACPI: Delete PCI Interrupt Routing Table for %04x:%02x\n",
+ segment, bus);
spin_lock(&acpi_prt_lock);
- list_for_each_safe(node, n, &acpi_prt.entries) {
- entry = list_entry(node, struct acpi_prt_entry, node);
-
- acpi_pci_irq_del_entry(segment, bus, entry);
+ list_for_each_entry_safe(entry, tmp, &acpi_prt_list, list) {
+ if (segment == entry->id.segment && bus == entry->id.bus) {
+ list_del(&entry->list);
+ kfree(entry);
+ }
}
spin_unlock(&acpi_prt_lock);
}
@@ -342,162 +300,26 @@
/* --------------------------------------------------------------------------
PCI Interrupt Routing Support
-------------------------------------------------------------------------- */
-typedef int (*irq_lookup_func) (struct acpi_prt_entry *, int *, int *, char **);
-
-static int
-acpi_pci_allocate_irq(struct acpi_prt_entry *entry,
- int *triggering, int *polarity, char **link)
+static struct acpi_prt_entry *acpi_pci_irq_lookup(struct pci_dev *dev, int pin)
{
- int irq;
+ struct acpi_prt_entry *entry;
+ struct pci_dev *bridge;
+ u8 bridge_pin, orig_pin = pin;
-
- if (entry->link.handle) {
- irq = acpi_pci_link_allocate_irq(entry->link.handle,
- entry->link.index, triggering,
- polarity, link);
- if (irq < 0) {
- printk(KERN_WARNING PREFIX
- "Invalid IRQ link routing entry\n");
- return -1;
- }
- } else {
- irq = entry->link.index;
- *triggering = ACPI_LEVEL_SENSITIVE;
- *polarity = ACPI_ACTIVE_LOW;
+ entry = acpi_pci_irq_find_prt_entry(dev, pin);
+ if (entry) {
+ ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Found %s[%c] _PRT entry\n",
+ pci_name(dev), pin_name(pin)));
+ return entry;
}
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Found IRQ %d\n", irq));
- return irq;
-}
-
-static int
-acpi_pci_free_irq(struct acpi_prt_entry *entry,
- int *triggering, int *polarity, char **link)
-{
- int irq;
-
- if (entry->link.handle) {
- irq = acpi_pci_link_free_irq(entry->link.handle);
- } else {
- irq = entry->link.index;
- }
- return irq;
-}
-
-#ifdef CONFIG_X86_IO_APIC
-extern int noioapicquirk;
-
-static int bridge_has_boot_interrupt_variant(struct pci_bus *bus)
-{
- struct pci_bus *bus_it;
-
- for (bus_it = bus ; bus_it ; bus_it = bus_it->parent) {
- if (!bus_it->self)
- return 0;
-
- printk(KERN_INFO "vendor=%04x device=%04x\n", bus_it->self->vendor,
- bus_it->self->device);
-
- if (bus_it->self->irq_reroute_variant)
- return bus_it->self->irq_reroute_variant;
- }
- return 0;
-}
-#endif /* CONFIG_X86_IO_APIC */
-
-/*
- * acpi_pci_irq_lookup
- * success: return IRQ >= 0
- * failure: return -1
- */
-static int
-acpi_pci_irq_lookup(struct pci_bus *bus,
- int device,
- int pin,
- int *triggering,
- int *polarity, char **link, irq_lookup_func func)
-{
- struct acpi_prt_entry *entry = NULL;
- int segment = pci_domain_nr(bus);
- int bus_nr = bus->number;
- int ret;
-
-
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Searching for PRT entry for %02x:%02x:%02x[%c]\n",
- segment, bus_nr, device, ('A' + pin)));
-
- entry = acpi_pci_irq_find_prt_entry(segment, bus_nr, device, pin);
- if (!entry) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "PRT entry not found\n"));
- return -1;
- }
-
- ret = func(entry, triggering, polarity, link);
-
-#ifdef CONFIG_X86_IO_APIC
- /*
- * Some chipsets (e.g. intel 6700PXH) generate a legacy INTx when the
- * IRQ entry in the chipset's IO-APIC is masked (as, e.g. the RT kernel
- * does during interrupt handling). When this INTx generation cannot be
- * disabled, we reroute these interrupts to their legacy equivalent to
- * get rid of spurious interrupts.
- */
- if (!noioapicquirk) {
- switch (bridge_has_boot_interrupt_variant(bus)) {
- case 0:
- /* no rerouting necessary */
- break;
-
- case INTEL_IRQ_REROUTE_VARIANT:
- /*
- * Remap according to INTx routing table in 6700PXH
- * specs, intel order number 302628-002, section
- * 2.15.2. Other chipsets (80332, ...) have the same
- * mapping and are handled here as well.
- */
- printk(KERN_INFO "pci irq %d -> rerouted to legacy "
- "irq %d\n", ret, (ret % 4) + 16);
- ret = (ret % 4) + 16;
- break;
-
- default:
- printk(KERN_INFO "not rerouting irq %d to legacy irq: "
- "unknown mapping\n", ret);
- break;
- }
- }
-#endif /* CONFIG_X86_IO_APIC */
-
- return ret;
-}
-
-/*
- * acpi_pci_irq_derive
- * success: return IRQ >= 0
- * failure: return < 0
- */
-static int
-acpi_pci_irq_derive(struct pci_dev *dev,
- int pin,
- int *triggering,
- int *polarity, char **link, irq_lookup_func func)
-{
- struct pci_dev *bridge = dev;
- int irq = -1;
- u8 bridge_pin = 0, orig_pin = pin;
-
-
- if (!dev)
- return -EINVAL;
-
/*
* Attempt to derive an IRQ for this device from a parent bridge's
* PCI interrupt routing entry (eg. yenta bridge and add-in card bridge).
*/
- while (irq < 0 && bridge->bus->self) {
- pin = (pin + PCI_SLOT(bridge->devfn)) % 4;
- bridge = bridge->bus->self;
+ bridge = dev->bus->self;
+ while (bridge) {
+ pin = (((pin - 1) + PCI_SLOT(dev->devfn)) % 4) + 1;
if ((bridge->class >> 8) == PCI_CLASS_BRIDGE_CARDBUS) {
/* PC card has the same IRQ as its cardbridge */
@@ -506,50 +328,40 @@
ACPI_DEBUG_PRINT((ACPI_DB_INFO,
"No interrupt pin configured for device %s\n",
pci_name(bridge)));
- return -1;
+ return NULL;
}
- /* Pin is from 0 to 3 */
- bridge_pin--;
pin = bridge_pin;
}
- irq = acpi_pci_irq_lookup(bridge->bus, PCI_SLOT(bridge->devfn),
- pin, triggering, polarity,
- link, func);
+ entry = acpi_pci_irq_find_prt_entry(bridge, pin);
+ if (entry) {
+ ACPI_DEBUG_PRINT((ACPI_DB_INFO,
+ "Derived GSI for %s INT %c from %s\n",
+ pci_name(dev), pin_name(orig_pin),
+ pci_name(bridge)));
+ return entry;
+ }
+
+ dev = bridge;
+ bridge = dev->bus->self;
}
- if (irq < 0) {
- dev_warn(&dev->dev, "can't derive routing for PCI INT %c\n",
- 'A' + orig_pin);
- return -1;
- }
-
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Derive IRQ %d for device %s from %s\n",
- irq, pci_name(dev), pci_name(bridge)));
-
- return irq;
+ dev_warn(&dev->dev, "can't derive routing for PCI INT %c\n",
+ pin_name(orig_pin));
+ return NULL;
}
-/*
- * acpi_pci_irq_enable
- * success: return 0
- * failure: return < 0
- */
-
int acpi_pci_irq_enable(struct pci_dev *dev)
{
- int irq = 0;
- u8 pin = 0;
+ struct acpi_prt_entry *entry;
+ int gsi;
+ u8 pin;
int triggering = ACPI_LEVEL_SENSITIVE;
int polarity = ACPI_ACTIVE_LOW;
char *link = NULL;
char link_desc[16];
int rc;
-
- if (!dev)
- return -EINVAL;
-
pin = dev->pin;
if (!pin) {
ACPI_DEBUG_PRINT((ACPI_DB_INFO,
@@ -557,31 +369,9 @@
pci_name(dev)));
return 0;
}
- pin--;
- if (!dev->bus) {
- dev_err(&dev->dev, "invalid (NULL) 'bus' field\n");
- return -ENODEV;
- }
-
- /*
- * First we check the PCI IRQ routing table (PRT) for an IRQ. PRT
- * values override any BIOS-assigned IRQs set during boot.
- */
- irq = acpi_pci_irq_lookup(dev->bus, PCI_SLOT(dev->devfn), pin,
- &triggering, &polarity, &link,
- acpi_pci_allocate_irq);
-
- /*
- * If no PRT entry was found, we'll try to derive an IRQ from the
- * device's parent bridge.
- */
- if (irq < 0)
- irq = acpi_pci_irq_derive(dev, pin, &triggering,
- &polarity, &link,
- acpi_pci_allocate_irq);
-
- if (irq < 0) {
+ entry = acpi_pci_irq_lookup(dev, pin);
+ if (!entry) {
/*
* IDE legacy mode controller IRQs are magic. Why do compat
* extensions always make such a nasty mess.
@@ -590,12 +380,24 @@
(dev->class & 0x05) == 0)
return 0;
}
+
+ if (entry) {
+ if (entry->link)
+ gsi = acpi_pci_link_allocate_irq(entry->link,
+ entry->index,
+ &triggering, &polarity,
+ &link);
+ else
+ gsi = entry->index;
+ } else
+ gsi = -1;
+
/*
* No IRQ known to the ACPI subsystem - maybe the BIOS /
* driver reported one, then use it. Exit in any case.
*/
- if (irq < 0) {
- dev_warn(&dev->dev, "PCI INT %c: no GSI", 'A' + pin);
+ if (gsi < 0) {
+ dev_warn(&dev->dev, "PCI INT %c: no GSI", pin_name(pin));
/* Interrupt Line values above 0xF are forbidden */
if (dev->irq > 0 && (dev->irq <= 0xF)) {
printk(" - using IRQ %d\n", dev->irq);
@@ -608,10 +410,10 @@
}
}
- rc = acpi_register_gsi(irq, triggering, polarity);
+ rc = acpi_register_gsi(gsi, triggering, polarity);
if (rc < 0) {
dev_warn(&dev->dev, "PCI INT %c: failed to register GSI\n",
- 'A' + pin);
+ pin_name(pin));
return rc;
}
dev->irq = rc;
@@ -622,7 +424,7 @@
link_desc[0] = '\0';
dev_info(&dev->dev, "PCI INT %c%s -> GSI %u (%s, %s) -> IRQ %d\n",
- 'A' + pin, link_desc, irq,
+ pin_name(pin), link_desc, gsi,
(triggering == ACPI_LEVEL_SENSITIVE) ? "level" : "edge",
(polarity == ACPI_ACTIVE_LOW) ? "low" : "high", dev->irq);
@@ -636,42 +438,28 @@
void acpi_pci_irq_disable(struct pci_dev *dev)
{
- int gsi = 0;
- u8 pin = 0;
- int triggering = ACPI_LEVEL_SENSITIVE;
- int polarity = ACPI_ACTIVE_LOW;
-
-
- if (!dev || !dev->bus)
- return;
+ struct acpi_prt_entry *entry;
+ int gsi;
+ u8 pin;
pin = dev->pin;
if (!pin)
return;
- pin--;
- /*
- * First we check the PCI IRQ routing table (PRT) for an IRQ.
- */
- gsi = acpi_pci_irq_lookup(dev->bus, PCI_SLOT(dev->devfn), pin,
- &triggering, &polarity, NULL,
- acpi_pci_free_irq);
- /*
- * If no PRT entry was found, we'll try to derive an IRQ from the
- * device's parent bridge.
- */
- if (gsi < 0)
- gsi = acpi_pci_irq_derive(dev, pin,
- &triggering, &polarity, NULL,
- acpi_pci_free_irq);
- if (gsi < 0)
+ entry = acpi_pci_irq_lookup(dev, pin);
+ if (!entry)
return;
+ if (entry->link)
+ gsi = acpi_pci_link_free_irq(entry->link);
+ else
+ gsi = entry->index;
+
/*
* TBD: It might be worth clearing dev->irq by magic constant
* (e.g. PCI_UNDEFINED_IRQ).
*/
- dev_info(&dev->dev, "PCI INT %c disabled\n", 'A' + pin);
+ dev_info(&dev->dev, "PCI INT %c disabled\n", pin_name(pin));
acpi_unregister_gsi(gsi);
}
diff --git a/drivers/acpi/pci_link.c b/drivers/acpi/pci_link.c
index e52ad91..1c6e73c 100644
--- a/drivers/acpi/pci_link.c
+++ b/drivers/acpi/pci_link.c
@@ -796,10 +796,6 @@
struct list_head *node = NULL;
struct acpi_pci_link *link = NULL;
-
- /* Make sure SCI is enabled again (Apple firmware bug?) */
- acpi_set_register(ACPI_BITREG_SCI_ENABLE, 1);
-
list_for_each(node, &acpi_link.entries) {
link = list_entry(node, struct acpi_pci_link, node);
if (!link) {
@@ -912,7 +908,7 @@
__setup("acpi_irq_nobalance", acpi_irq_nobalance_set);
-int __init acpi_irq_balance_set(char *str)
+static int __init acpi_irq_balance_set(char *str)
{
acpi_irq_balance = 1;
return 1;
diff --git a/drivers/acpi/power.c b/drivers/acpi/power.c
index bb7d50d..c926e7d 100644
--- a/drivers/acpi/power.c
+++ b/drivers/acpi/power.c
@@ -139,6 +139,8 @@
{
acpi_status status = AE_OK;
unsigned long long sta = 0;
+ char node_name[5];
+ struct acpi_buffer buffer = { sizeof(node_name), node_name };
if (!handle || !state)
@@ -151,8 +153,10 @@
*state = (sta & 0x01)?ACPI_POWER_RESOURCE_STATE_ON:
ACPI_POWER_RESOURCE_STATE_OFF;
+ acpi_get_name(handle, ACPI_SINGLE_NAME, &buffer);
+
ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Resource [%s] is %s\n",
- acpi_ut_get_node_name(handle),
+ node_name,
*state ? "on" : "off"));
return 0;
diff --git a/drivers/acpi/sleep/proc.c b/drivers/acpi/proc.c
similarity index 92%
rename from drivers/acpi/sleep/proc.c
rename to drivers/acpi/proc.c
index 4dbc227..428c911d 100644
--- a/drivers/acpi/sleep/proc.c
+++ b/drivers/acpi/proc.c
@@ -28,8 +28,6 @@
{
int i;
- ACPI_FUNCTION_TRACE("acpi_system_sleep_seq_show");
-
for (i = 0; i <= ACPI_STATE_S5; i++) {
if (sleep_states[i]) {
seq_printf(seq, "S%d ", i);
@@ -86,49 +84,44 @@
#ifdef HAVE_ACPI_LEGACY_ALARM
+static u32 cmos_bcd_read(int offset, int rtc_control);
+
static int acpi_system_alarm_seq_show(struct seq_file *seq, void *offset)
{
u32 sec, min, hr;
u32 day, mo, yr, cent = 0;
+ u32 today = 0;
unsigned char rtc_control = 0;
unsigned long flags;
- ACPI_FUNCTION_TRACE("acpi_system_alarm_seq_show");
-
spin_lock_irqsave(&rtc_lock, flags);
- sec = CMOS_READ(RTC_SECONDS_ALARM);
- min = CMOS_READ(RTC_MINUTES_ALARM);
- hr = CMOS_READ(RTC_HOURS_ALARM);
rtc_control = CMOS_READ(RTC_CONTROL);
+ sec = cmos_bcd_read(RTC_SECONDS_ALARM, rtc_control);
+ min = cmos_bcd_read(RTC_MINUTES_ALARM, rtc_control);
+ hr = cmos_bcd_read(RTC_HOURS_ALARM, rtc_control);
/* If we ever get an FACP with proper values... */
- if (acpi_gbl_FADT.day_alarm)
+ if (acpi_gbl_FADT.day_alarm) {
/* ACPI spec: only low 6 its should be cared */
day = CMOS_READ(acpi_gbl_FADT.day_alarm) & 0x3F;
- else
- day = CMOS_READ(RTC_DAY_OF_MONTH);
+ if (!(rtc_control & RTC_DM_BINARY) || RTC_ALWAYS_BCD)
+ day = bcd2bin(day);
+ } else
+ day = cmos_bcd_read(RTC_DAY_OF_MONTH, rtc_control);
if (acpi_gbl_FADT.month_alarm)
- mo = CMOS_READ(acpi_gbl_FADT.month_alarm);
- else
- mo = CMOS_READ(RTC_MONTH);
+ mo = cmos_bcd_read(acpi_gbl_FADT.month_alarm, rtc_control);
+ else {
+ mo = cmos_bcd_read(RTC_MONTH, rtc_control);
+ today = cmos_bcd_read(RTC_DAY_OF_MONTH, rtc_control);
+ }
if (acpi_gbl_FADT.century)
- cent = CMOS_READ(acpi_gbl_FADT.century);
+ cent = cmos_bcd_read(acpi_gbl_FADT.century, rtc_control);
- yr = CMOS_READ(RTC_YEAR);
+ yr = cmos_bcd_read(RTC_YEAR, rtc_control);
spin_unlock_irqrestore(&rtc_lock, flags);
- if (!(rtc_control & RTC_DM_BINARY) || RTC_ALWAYS_BCD) {
- sec = bcd2bin(sec);
- min = bcd2bin(min);
- hr = bcd2bin(hr);
- day = bcd2bin(day);
- mo = bcd2bin(mo);
- yr = bcd2bin(yr);
- cent = bcd2bin(cent);
- }
-
/* we're trusting the FADT (see above) */
if (!acpi_gbl_FADT.century)
/* If we're not trusting the FADT, we should at least make it
@@ -153,6 +146,20 @@
else
yr += cent * 100;
+ /*
+ * Show correct dates for alarms up to a month into the future.
+ * This solves issues for nearly all situations with the common
+ * 30-day alarm clocks in PC hardware.
+ */
+ if (day < today) {
+ if (mo < 12) {
+ mo += 1;
+ } else {
+ mo = 1;
+ yr += 1;
+ }
+ }
+
seq_printf(seq, "%4.4u-", yr);
(mo > 12) ? seq_puts(seq, "**-") : seq_printf(seq, "%2.2u-", mo);
(day > 31) ? seq_puts(seq, "** ") : seq_printf(seq, "%2.2u ", day);
@@ -227,13 +234,11 @@
int adjust = 0;
unsigned char rtc_control = 0;
- ACPI_FUNCTION_TRACE("acpi_system_write_alarm");
-
if (count > sizeof(alarm_string) - 1)
- return_VALUE(-EINVAL);
+ return -EINVAL;
if (copy_from_user(alarm_string, buffer, count))
- return_VALUE(-EFAULT);
+ return -EFAULT;
alarm_string[count] = '\0';
@@ -334,7 +339,7 @@
result = 0;
end:
- return_VALUE(result ? result : count);
+ return result ? result : count;
}
#endif /* HAVE_ACPI_LEGACY_ALARM */
diff --git a/drivers/acpi/reboot.c b/drivers/acpi/reboot.c
index a6b662c..93f9114 100644
--- a/drivers/acpi/reboot.c
+++ b/drivers/acpi/reboot.c
@@ -42,7 +42,7 @@
case ACPI_ADR_SPACE_SYSTEM_MEMORY:
case ACPI_ADR_SPACE_SYSTEM_IO:
printk(KERN_DEBUG "ACPI MEMORY or I/O RESET_REG.\n");
- acpi_hw_low_level_write(8, reset_value, rr);
+ acpi_reset();
break;
}
/* Wait ten seconds */
diff --git a/drivers/acpi/resources/Makefile b/drivers/acpi/resources/Makefile
deleted file mode 100644
index 8de4f69..0000000
--- a/drivers/acpi/resources/Makefile
+++ /dev/null
@@ -1,10 +0,0 @@
-#
-# Makefile for all Linux ACPI interpreter subdirectories
-#
-
-obj-y := rsaddr.o rscreate.o rsinfo.o rsio.o rslist.o rsmisc.o rsxface.o \
- rscalc.o rsirq.o rsmemory.o rsutils.o
-
-obj-$(ACPI_FUTURE_USAGE) += rsdump.o
-
-EXTRA_CFLAGS += $(ACPI_CFLAGS)
diff --git a/drivers/acpi/sbshc.c b/drivers/acpi/sbshc.c
index e53e590..0619734 100644
--- a/drivers/acpi/sbshc.c
+++ b/drivers/acpi/sbshc.c
@@ -10,7 +10,6 @@
#include <acpi/acpi_bus.h>
#include <acpi/acpi_drivers.h>
-#include <acpi/actypes.h>
#include <linux/wait.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
diff --git a/drivers/acpi/scan.c b/drivers/acpi/scan.c
index 39b7233..c54d7b6 100644
--- a/drivers/acpi/scan.c
+++ b/drivers/acpi/scan.c
@@ -10,7 +10,6 @@
#include <linux/kthread.h>
#include <acpi/acpi_drivers.h>
-#include <acpi/acinterp.h> /* for acpi_ex_eisa_id_to_string() */
#define _COMPONENT ACPI_BUS_COMPONENT
ACPI_MODULE_NAME("scan");
diff --git a/drivers/acpi/sleep/sleep.h b/drivers/acpi/sleep.h
similarity index 100%
rename from drivers/acpi/sleep/sleep.h
rename to drivers/acpi/sleep.h
diff --git a/drivers/acpi/sleep/Makefile b/drivers/acpi/sleep/Makefile
deleted file mode 100644
index f1fb888..0000000
--- a/drivers/acpi/sleep/Makefile
+++ /dev/null
@@ -1,5 +0,0 @@
-obj-y := wakeup.o
-obj-y += main.o
-obj-$(CONFIG_ACPI_SLEEP) += proc.o
-
-EXTRA_CFLAGS += $(ACPI_CFLAGS)
diff --git a/drivers/acpi/system.c b/drivers/acpi/system.c
index 6e4107f..391d035 100644
--- a/drivers/acpi/system.c
+++ b/drivers/acpi/system.c
@@ -192,65 +192,6 @@
};
static struct kobj_attribute *counter_attrs;
-static int count_num_gpes(void)
-{
- int count = 0;
- struct acpi_gpe_xrupt_info *gpe_xrupt_info;
- struct acpi_gpe_block_info *gpe_block;
- acpi_cpu_flags flags;
-
- flags = acpi_os_acquire_lock(acpi_gbl_gpe_lock);
-
- gpe_xrupt_info = acpi_gbl_gpe_xrupt_list_head;
- while (gpe_xrupt_info) {
- gpe_block = gpe_xrupt_info->gpe_block_list_head;
- while (gpe_block) {
- count += gpe_block->register_count *
- ACPI_GPE_REGISTER_WIDTH;
- gpe_block = gpe_block->next;
- }
- gpe_xrupt_info = gpe_xrupt_info->next;
- }
- acpi_os_release_lock(acpi_gbl_gpe_lock, flags);
-
- return count;
-}
-
-static int get_gpe_device(int index, acpi_handle *handle)
-{
- struct acpi_gpe_xrupt_info *gpe_xrupt_info;
- struct acpi_gpe_block_info *gpe_block;
- acpi_cpu_flags flags;
- struct acpi_namespace_node *node;
-
- flags = acpi_os_acquire_lock(acpi_gbl_gpe_lock);
-
- gpe_xrupt_info = acpi_gbl_gpe_xrupt_list_head;
- while (gpe_xrupt_info) {
- gpe_block = gpe_xrupt_info->gpe_block_list_head;
- node = gpe_block->node;
- while (gpe_block) {
- index -= gpe_block->register_count *
- ACPI_GPE_REGISTER_WIDTH;
- if (index < 0) {
- acpi_os_release_lock(acpi_gbl_gpe_lock, flags);
- /* return NULL if it's FADT GPE */
- if (node->type != ACPI_TYPE_DEVICE)
- *handle = NULL;
- else
- *handle = node;
- return 0;
- }
- node = gpe_block->node;
- gpe_block = gpe_block->next;
- }
- gpe_xrupt_info = gpe_xrupt_info->next;
- }
- acpi_os_release_lock(acpi_gbl_gpe_lock, flags);
-
- return -ENODEV;
-}
-
static void delete_gpe_attr_array(void)
{
struct event_counter *tmp = all_counters;
@@ -309,7 +250,7 @@
goto end;
if (index < num_gpes) {
- result = get_gpe_device(index, handle);
+ result = acpi_get_gpe_device(index, handle);
if (result) {
ACPI_EXCEPTION((AE_INFO, AE_NOT_FOUND,
"Invalid GPE 0x%x\n", index));
@@ -436,7 +377,7 @@
if (all_counters)
return;
- num_gpes = count_num_gpes();
+ num_gpes = acpi_current_gpe_count;
num_counters = num_gpes + ACPI_NUM_FIXED_EVENTS + NUM_COUNTERS_EXTRA;
all_attrs = kzalloc(sizeof(struct attribute *) * (num_counters + 1),
diff --git a/drivers/acpi/tables/Makefile b/drivers/acpi/tables/Makefile
deleted file mode 100644
index 7385efa..0000000
--- a/drivers/acpi/tables/Makefile
+++ /dev/null
@@ -1,7 +0,0 @@
-#
-# Makefile for all Linux ACPI interpreter subdirectories
-#
-
-obj-y := tbxface.o tbinstal.o tbutils.o tbfind.o tbfadt.o tbxfroot.o
-
-EXTRA_CFLAGS += $(ACPI_CFLAGS)
diff --git a/drivers/acpi/utilities/Makefile b/drivers/acpi/utilities/Makefile
deleted file mode 100644
index 88eff14..0000000
--- a/drivers/acpi/utilities/Makefile
+++ /dev/null
@@ -1,9 +0,0 @@
-#
-# Makefile for all Linux ACPI interpreter subdirectories
-#
-
-obj-y := utalloc.o utdebug.o uteval.o utinit.o utmisc.o utxface.o \
- utcopy.o utdelete.o utglobal.o utmath.o utobject.o \
- utstate.o utmutex.o utobject.o utcache.o utresrc.o
-
-EXTRA_CFLAGS += $(ACPI_CFLAGS)
diff --git a/drivers/acpi/utilities/utcache.c b/drivers/acpi/utilities/utcache.c
deleted file mode 100644
index 245fa80..0000000
--- a/drivers/acpi/utilities/utcache.c
+++ /dev/null
@@ -1,314 +0,0 @@
-/******************************************************************************
- *
- * Module Name: utcache - local cache allocation routines
- *
- *****************************************************************************/
-
-/*
- * Copyright (C) 2000 - 2008, Intel Corp.
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- * notice, this list of conditions, and the following disclaimer,
- * without modification.
- * 2. Redistributions in binary form must reproduce at minimum a disclaimer
- * substantially similar to the "NO WARRANTY" disclaimer below
- * ("Disclaimer") and any redistribution must be conditioned upon
- * including a substantially similar Disclaimer requirement for further
- * binary redistribution.
- * 3. Neither the names of the above-listed copyright holders nor the names
- * of any contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
- *
- * Alternatively, this software may be distributed under the terms of the
- * GNU General Public License ("GPL") version 2 as published by the Free
- * Software Foundation.
- *
- * NO WARRANTY
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
- * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
- * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGES.
- */
-
-#include <acpi/acpi.h>
-
-#define _COMPONENT ACPI_UTILITIES
-ACPI_MODULE_NAME("utcache")
-#ifdef ACPI_USE_LOCAL_CACHE
-/*******************************************************************************
- *
- * FUNCTION: acpi_os_create_cache
- *
- * PARAMETERS: cache_name - Ascii name for the cache
- * object_size - Size of each cached object
- * max_depth - Maximum depth of the cache (in objects)
- * return_cache - Where the new cache object is returned
- *
- * RETURN: Status
- *
- * DESCRIPTION: Create a cache object
- *
- ******************************************************************************/
-acpi_status
-acpi_os_create_cache(char *cache_name,
- u16 object_size,
- u16 max_depth, struct acpi_memory_list ** return_cache)
-{
- struct acpi_memory_list *cache;
-
- ACPI_FUNCTION_ENTRY();
-
- if (!cache_name || !return_cache || (object_size < 16)) {
- return (AE_BAD_PARAMETER);
- }
-
- /* Create the cache object */
-
- cache = acpi_os_allocate(sizeof(struct acpi_memory_list));
- if (!cache) {
- return (AE_NO_MEMORY);
- }
-
- /* Populate the cache object and return it */
-
- ACPI_MEMSET(cache, 0, sizeof(struct acpi_memory_list));
- cache->link_offset = 8;
- cache->list_name = cache_name;
- cache->object_size = object_size;
- cache->max_depth = max_depth;
-
- *return_cache = cache;
- return (AE_OK);
-}
-
-/*******************************************************************************
- *
- * FUNCTION: acpi_os_purge_cache
- *
- * PARAMETERS: Cache - Handle to cache object
- *
- * RETURN: Status
- *
- * DESCRIPTION: Free all objects within the requested cache.
- *
- ******************************************************************************/
-
-acpi_status acpi_os_purge_cache(struct acpi_memory_list * cache)
-{
- char *next;
-
- ACPI_FUNCTION_ENTRY();
-
- if (!cache) {
- return (AE_BAD_PARAMETER);
- }
-
- /* Walk the list of objects in this cache */
-
- while (cache->list_head) {
-
- /* Delete and unlink one cached state object */
-
- next = *(ACPI_CAST_INDIRECT_PTR(char,
- &(((char *)cache->
- list_head)[cache->
- link_offset])));
- ACPI_FREE(cache->list_head);
-
- cache->list_head = next;
- cache->current_depth--;
- }
-
- return (AE_OK);
-}
-
-/*******************************************************************************
- *
- * FUNCTION: acpi_os_delete_cache
- *
- * PARAMETERS: Cache - Handle to cache object
- *
- * RETURN: Status
- *
- * DESCRIPTION: Free all objects within the requested cache and delete the
- * cache object.
- *
- ******************************************************************************/
-
-acpi_status acpi_os_delete_cache(struct acpi_memory_list * cache)
-{
- acpi_status status;
-
- ACPI_FUNCTION_ENTRY();
-
- /* Purge all objects in the cache */
-
- status = acpi_os_purge_cache(cache);
- if (ACPI_FAILURE(status)) {
- return (status);
- }
-
- /* Now we can delete the cache object */
-
- ACPI_FREE(cache);
- return (AE_OK);
-}
-
-/*******************************************************************************
- *
- * FUNCTION: acpi_os_release_object
- *
- * PARAMETERS: Cache - Handle to cache object
- * Object - The object to be released
- *
- * RETURN: None
- *
- * DESCRIPTION: Release an object to the specified cache. If cache is full,
- * the object is deleted.
- *
- ******************************************************************************/
-
-acpi_status
-acpi_os_release_object(struct acpi_memory_list * cache, void *object)
-{
- acpi_status status;
-
- ACPI_FUNCTION_ENTRY();
-
- if (!cache || !object) {
- return (AE_BAD_PARAMETER);
- }
-
- /* If cache is full, just free this object */
-
- if (cache->current_depth >= cache->max_depth) {
- ACPI_FREE(object);
- ACPI_MEM_TRACKING(cache->total_freed++);
- }
-
- /* Otherwise put this object back into the cache */
-
- else {
- status = acpi_ut_acquire_mutex(ACPI_MTX_CACHES);
- if (ACPI_FAILURE(status)) {
- return (status);
- }
-
- /* Mark the object as cached */
-
- ACPI_MEMSET(object, 0xCA, cache->object_size);
- ACPI_SET_DESCRIPTOR_TYPE(object, ACPI_DESC_TYPE_CACHED);
-
- /* Put the object at the head of the cache list */
-
- *(ACPI_CAST_INDIRECT_PTR(char,
- &(((char *)object)[cache->
- link_offset]))) =
- cache->list_head;
- cache->list_head = object;
- cache->current_depth++;
-
- (void)acpi_ut_release_mutex(ACPI_MTX_CACHES);
- }
-
- return (AE_OK);
-}
-
-/*******************************************************************************
- *
- * FUNCTION: acpi_os_acquire_object
- *
- * PARAMETERS: Cache - Handle to cache object
- *
- * RETURN: the acquired object. NULL on error
- *
- * DESCRIPTION: Get an object from the specified cache. If cache is empty,
- * the object is allocated.
- *
- ******************************************************************************/
-
-void *acpi_os_acquire_object(struct acpi_memory_list *cache)
-{
- acpi_status status;
- void *object;
-
- ACPI_FUNCTION_NAME(os_acquire_object);
-
- if (!cache) {
- return (NULL);
- }
-
- status = acpi_ut_acquire_mutex(ACPI_MTX_CACHES);
- if (ACPI_FAILURE(status)) {
- return (NULL);
- }
-
- ACPI_MEM_TRACKING(cache->requests++);
-
- /* Check the cache first */
-
- if (cache->list_head) {
-
- /* There is an object available, use it */
-
- object = cache->list_head;
- cache->list_head = *(ACPI_CAST_INDIRECT_PTR(char,
- &(((char *)
- object)[cache->
- link_offset])));
-
- cache->current_depth--;
-
- ACPI_MEM_TRACKING(cache->hits++);
- ACPI_DEBUG_PRINT((ACPI_DB_EXEC,
- "Object %p from %s cache\n", object,
- cache->list_name));
-
- status = acpi_ut_release_mutex(ACPI_MTX_CACHES);
- if (ACPI_FAILURE(status)) {
- return (NULL);
- }
-
- /* Clear (zero) the previously used Object */
-
- ACPI_MEMSET(object, 0, cache->object_size);
- } else {
- /* The cache is empty, create a new object */
-
- ACPI_MEM_TRACKING(cache->total_allocated++);
-
-#ifdef ACPI_DBG_TRACK_ALLOCATIONS
- if ((cache->total_allocated - cache->total_freed) >
- cache->max_occupied) {
- cache->max_occupied =
- cache->total_allocated - cache->total_freed;
- }
-#endif
-
- /* Avoid deadlock with ACPI_ALLOCATE_ZEROED */
-
- status = acpi_ut_release_mutex(ACPI_MTX_CACHES);
- if (ACPI_FAILURE(status)) {
- return (NULL);
- }
-
- object = ACPI_ALLOCATE_ZEROED(cache->object_size);
- if (!object) {
- return (NULL);
- }
- }
-
- return (object);
-}
-#endif /* ACPI_USE_LOCAL_CACHE */
diff --git a/drivers/acpi/video.c b/drivers/acpi/video.c
index baa4419..f261737 100644
--- a/drivers/acpi/video.c
+++ b/drivers/acpi/video.c
@@ -36,6 +36,7 @@
#include <linux/backlight.h>
#include <linux/thermal.h>
#include <linux/video_output.h>
+#include <linux/sort.h>
#include <asm/uaccess.h>
#include <acpi/acpi_bus.h>
@@ -481,6 +482,7 @@
int status = AE_OK;
union acpi_object arg0 = { ACPI_TYPE_INTEGER };
struct acpi_object_list args = { 1, &arg0 };
+ int state;
arg0.integer.value = level;
@@ -489,6 +491,10 @@
status = acpi_evaluate_object(device->dev->handle, "_BCM",
&args, NULL);
device->brightness->curr = level;
+ for (state = 2; state < device->brightness->count; state++)
+ if (level == device->brightness->levels[state])
+ device->backlight->props.brightness = state - 2;
+
return status;
}
@@ -626,6 +632,16 @@
}
/*
+ * Simple comparison function used to sort backlight levels.
+ */
+
+static int
+acpi_video_cmp_level(const void *a, const void *b)
+{
+ return *(int *)a - *(int *)b;
+}
+
+/*
* Arg:
* device : video output device (LCD, CRT, ..)
*
@@ -676,6 +692,10 @@
count++;
}
+ /* don't sort the first two brightness levels */
+ sort(&br->levels[2], count - 2, sizeof(br->levels[2]),
+ acpi_video_cmp_level, NULL);
+
if (count < 2)
goto out_free_levels;
diff --git a/drivers/acpi/video_detect.c b/drivers/acpi/video_detect.c
index f022eb6..50e3d2d 100644
--- a/drivers/acpi/video_detect.c
+++ b/drivers/acpi/video_detect.c
@@ -234,7 +234,7 @@
* To force that backlight or display output switching is processed by vendor
* specific acpi drivers or video.ko driver.
*/
-int __init acpi_backlight(char *str)
+static int __init acpi_backlight(char *str)
{
if (str == NULL || *str == '\0')
return 1;
@@ -250,7 +250,7 @@
}
__setup("acpi_backlight=", acpi_backlight);
-int __init acpi_display_output(char *str)
+static int __init acpi_display_output(char *str)
{
if (str == NULL || *str == '\0')
return 1;
diff --git a/drivers/acpi/sleep/wakeup.c b/drivers/acpi/wakeup.c
similarity index 95%
rename from drivers/acpi/sleep/wakeup.c
rename to drivers/acpi/wakeup.c
index dea4c23..2d34806 100644
--- a/drivers/acpi/sleep/wakeup.c
+++ b/drivers/acpi/wakeup.c
@@ -8,7 +8,6 @@
#include <acpi/acpi_drivers.h>
#include <linux/kernel.h>
#include <linux/types.h>
-#include <acpi/acevents.h>
#include "sleep.h"
#define _COMPONENT ACPI_SYSTEM_COMPONENT
@@ -28,8 +27,6 @@
{
struct list_head *node, *next;
- ACPI_FUNCTION_TRACE("acpi_enable_wakeup_device_prep");
-
spin_lock(&acpi_device_lock);
list_for_each_safe(node, next, &acpi_wakeup_device_list) {
struct acpi_device *dev = container_of(node,
@@ -61,7 +58,6 @@
* Caution: this routine must be invoked when interrupt is disabled
* Refer ACPI2.0: P212
*/
- ACPI_FUNCTION_TRACE("acpi_enable_wakeup_device");
spin_lock(&acpi_device_lock);
list_for_each_safe(node, next, &acpi_wakeup_device_list) {
struct acpi_device *dev =
@@ -103,8 +99,6 @@
{
struct list_head *node, *next;
- ACPI_FUNCTION_TRACE("acpi_disable_wakeup_device");
-
spin_lock(&acpi_device_lock);
list_for_each_safe(node, next, &acpi_wakeup_device_list) {
struct acpi_device *dev =
diff --git a/drivers/amba/bus.c b/drivers/amba/bus.c
index 6b94fb7..00c46e0 100644
--- a/drivers/amba/bus.c
+++ b/drivers/amba/bus.c
@@ -12,9 +12,10 @@
#include <linux/device.h>
#include <linux/string.h>
#include <linux/slab.h>
+#include <linux/io.h>
#include <linux/amba/bus.h>
-#include <asm/io.h>
+#include <asm/irq.h>
#include <asm/sizes.h>
#define to_amba_device(d) container_of(d, struct amba_device, dev)
diff --git a/drivers/ata/ahci.c b/drivers/ata/ahci.c
index 656448c..7f701cb 100644
--- a/drivers/ata/ahci.c
+++ b/drivers/ata/ahci.c
@@ -105,7 +105,7 @@
board_ahci_ign_iferr = 2,
board_ahci_sb600 = 3,
board_ahci_mv = 4,
- board_ahci_sb700 = 5,
+ board_ahci_sb700 = 5, /* for SB700 and SB800 */
board_ahci_mcp65 = 6,
board_ahci_nopmp = 7,
@@ -439,7 +439,7 @@
.udma_mask = ATA_UDMA6,
.port_ops = &ahci_ops,
},
- /* board_ahci_sb700 */
+ /* board_ahci_sb700, for SB700 and SB800 */
{
AHCI_HFLAGS (AHCI_HFLAG_IGN_SERR_INTERNAL),
.flags = AHCI_FLAG_COMMON,
@@ -2446,6 +2446,8 @@
speed_s = "1.5";
else if (speed == 2)
speed_s = "3";
+ else if (speed == 3)
+ speed_s = "6";
else
speed_s = "?";
@@ -2610,6 +2612,10 @@
(pdev->revision == 0xa1 || pdev->revision == 0xa2))
hpriv->flags |= AHCI_HFLAG_NO_MSI;
+ /* SB800 does NOT need the workaround to ignore SERR_INTERNAL */
+ if (board_id == board_ahci_sb700 && pdev->revision >= 0x40)
+ hpriv->flags &= ~AHCI_HFLAG_IGN_SERR_INTERNAL;
+
if ((hpriv->flags & AHCI_HFLAG_NO_MSI) || pci_enable_msi(pdev))
pci_intx(pdev, 1);
diff --git a/drivers/ata/ata_piix.c b/drivers/ata/ata_piix.c
index 5fdf167..887d8f4 100644
--- a/drivers/ata/ata_piix.c
+++ b/drivers/ata/ata_piix.c
@@ -154,11 +154,13 @@
struct piix_host_priv {
const int *map;
+ u32 saved_iocfg;
void __iomem *sidpr;
};
static int piix_init_one(struct pci_dev *pdev,
const struct pci_device_id *ent);
+static void piix_remove_one(struct pci_dev *pdev);
static int piix_pata_prereset(struct ata_link *link, unsigned long deadline);
static void piix_set_piomode(struct ata_port *ap, struct ata_device *adev);
static void piix_set_dmamode(struct ata_port *ap, struct ata_device *adev);
@@ -296,7 +298,7 @@
.name = DRV_NAME,
.id_table = piix_pci_tbl,
.probe = piix_init_one,
- .remove = ata_pci_remove_one,
+ .remove = piix_remove_one,
#ifdef CONFIG_PM
.suspend = piix_pci_device_suspend,
.resume = piix_pci_device_resume,
@@ -308,7 +310,7 @@
};
static struct ata_port_operations piix_pata_ops = {
- .inherits = &ata_bmdma_port_ops,
+ .inherits = &ata_bmdma32_port_ops,
.cable_detect = ata_cable_40wire,
.set_piomode = piix_set_piomode,
.set_dmamode = piix_set_dmamode,
@@ -610,8 +612,9 @@
static int ich_pata_cable_detect(struct ata_port *ap)
{
struct pci_dev *pdev = to_pci_dev(ap->host->dev);
+ struct piix_host_priv *hpriv = ap->host->private_data;
const struct ich_laptop *lap = &ich_laptop[0];
- u8 tmp, mask;
+ u8 mask;
/* Check for specials - Acer Aspire 5602WLMi */
while (lap->device) {
@@ -625,8 +628,7 @@
/* check BIOS cable detect results */
mask = ap->port_no == 0 ? PIIX_80C_PRI : PIIX_80C_SEC;
- pci_read_config_byte(pdev, PIIX_IOCFG, &tmp);
- if ((tmp & mask) == 0)
+ if ((hpriv->saved_iocfg & mask) == 0)
return ATA_CBL_PATA40;
return ATA_CBL_PATA80;
}
@@ -1350,7 +1352,7 @@
return 0;
}
-static void piix_iocfg_bit18_quirk(struct pci_dev *pdev)
+static void piix_iocfg_bit18_quirk(struct ata_host *host)
{
static const struct dmi_system_id sysids[] = {
{
@@ -1367,7 +1369,8 @@
{ } /* terminate list */
};
- u32 iocfg;
+ struct pci_dev *pdev = to_pci_dev(host->dev);
+ struct piix_host_priv *hpriv = host->private_data;
if (!dmi_check_system(sysids))
return;
@@ -1376,12 +1379,11 @@
* seem to use it to disable a channel. Clear the bit on the
* affected systems.
*/
- pci_read_config_dword(pdev, PIIX_IOCFG, &iocfg);
- if (iocfg & (1 << 18)) {
+ if (hpriv->saved_iocfg & (1 << 18)) {
dev_printk(KERN_INFO, &pdev->dev,
"applying IOCFG bit18 quirk\n");
- iocfg &= ~(1 << 18);
- pci_write_config_dword(pdev, PIIX_IOCFG, iocfg);
+ pci_write_config_dword(pdev, PIIX_IOCFG,
+ hpriv->saved_iocfg & ~(1 << 18));
}
}
@@ -1430,6 +1432,17 @@
if (rc)
return rc;
+ hpriv = devm_kzalloc(dev, sizeof(*hpriv), GFP_KERNEL);
+ if (!hpriv)
+ return -ENOMEM;
+
+ /* Save IOCFG, this will be used for cable detection, quirk
+ * detection and restoration on detach. This is necessary
+ * because some ACPI implementations mess up cable related
+ * bits on _STM. Reported on kernel bz#11879.
+ */
+ pci_read_config_dword(pdev, PIIX_IOCFG, &hpriv->saved_iocfg);
+
/* ICH6R may be driven by either ata_piix or ahci driver
* regardless of BIOS configuration. Make sure AHCI mode is
* off.
@@ -1441,10 +1454,6 @@
}
/* SATA map init can change port_info, do it before prepping host */
- hpriv = devm_kzalloc(dev, sizeof(*hpriv), GFP_KERNEL);
- if (!hpriv)
- return -ENOMEM;
-
if (port_flags & ATA_FLAG_SATA)
hpriv->map = piix_init_sata_map(pdev, port_info,
piix_map_db_table[ent->driver_data]);
@@ -1463,7 +1472,7 @@
}
/* apply IOCFG bit18 quirk */
- piix_iocfg_bit18_quirk(pdev);
+ piix_iocfg_bit18_quirk(host);
/* On ICH5, some BIOSen disable the interrupt using the
* PCI_COMMAND_INTX_DISABLE bit added in PCI 2.3.
@@ -1488,6 +1497,16 @@
return ata_pci_sff_activate_host(host, ata_sff_interrupt, &piix_sht);
}
+static void piix_remove_one(struct pci_dev *pdev)
+{
+ struct ata_host *host = dev_get_drvdata(&pdev->dev);
+ struct piix_host_priv *hpriv = host->private_data;
+
+ pci_write_config_dword(pdev, PIIX_IOCFG, hpriv->saved_iocfg);
+
+ ata_pci_remove_one(pdev);
+}
+
static int __init piix_init(void)
{
int rc;
diff --git a/drivers/ata/libata-acpi.c b/drivers/ata/libata-acpi.c
index ef02e488..6273d98 100644
--- a/drivers/ata/libata-acpi.c
+++ b/drivers/ata/libata-acpi.c
@@ -19,12 +19,6 @@
#include "libata.h"
#include <acpi/acpi_bus.h>
-#include <acpi/acnames.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acparser.h>
-#include <acpi/acexcep.h>
-#include <acpi/acmacros.h>
-#include <acpi/actypes.h>
enum {
ATA_ACPI_FILTER_SETXFER = 1 << 0,
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index f178a45..c507a9a 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -1007,6 +1007,7 @@
static const char * const spd_str[] = {
"1.5 Gbps",
"3.0 Gbps",
+ "6.0 Gbps",
};
if (spd == 0 || (spd - 1) >= ARRAY_SIZE(spd_str))
@@ -2000,6 +2001,10 @@
as the caller should know this */
if (adev->link->ap->flags & ATA_FLAG_NO_IORDY)
return 0;
+ /* CF spec. r4.1 Table 22 says no iordy on PIO5 and PIO6. */
+ if (ata_id_is_cfa(adev->id)
+ && (adev->pio_mode == XFER_PIO_5 || adev->pio_mode == XFER_PIO_6))
+ return 0;
/* PIO3 and higher it is mandatory */
if (adev->pio_mode > XFER_PIO_2)
return 1;
@@ -4551,7 +4556,7 @@
struct scatterlist *sg = qc->sg;
int dir = qc->dma_dir;
- WARN_ON(sg == NULL);
+ WARN_ON_ONCE(sg == NULL);
VPRINTK("unmapping %u sg elements\n", qc->n_elem);
@@ -4771,7 +4776,7 @@
struct ata_port *ap = qc->ap;
unsigned int tag;
- WARN_ON(qc == NULL); /* ata_qc_from_tag _might_ return NULL */
+ WARN_ON_ONCE(qc == NULL); /* ata_qc_from_tag _might_ return NULL */
qc->flags = 0;
tag = qc->tag;
@@ -4786,8 +4791,8 @@
struct ata_port *ap = qc->ap;
struct ata_link *link = qc->dev->link;
- WARN_ON(qc == NULL); /* ata_qc_from_tag _might_ return NULL */
- WARN_ON(!(qc->flags & ATA_QCFLAG_ACTIVE));
+ WARN_ON_ONCE(qc == NULL); /* ata_qc_from_tag _might_ return NULL */
+ WARN_ON_ONCE(!(qc->flags & ATA_QCFLAG_ACTIVE));
if (likely(qc->flags & ATA_QCFLAG_DMAMAP))
ata_sg_clean(qc);
@@ -4873,7 +4878,7 @@
struct ata_device *dev = qc->dev;
struct ata_eh_info *ehi = &dev->link->eh_info;
- WARN_ON(ap->pflags & ATA_PFLAG_FROZEN);
+ WARN_ON_ONCE(ap->pflags & ATA_PFLAG_FROZEN);
if (unlikely(qc->err_mask))
qc->flags |= ATA_QCFLAG_FAILED;
@@ -4995,16 +5000,16 @@
* check is skipped for old EH because it reuses active qc to
* request ATAPI sense.
*/
- WARN_ON(ap->ops->error_handler && ata_tag_valid(link->active_tag));
+ WARN_ON_ONCE(ap->ops->error_handler && ata_tag_valid(link->active_tag));
if (ata_is_ncq(prot)) {
- WARN_ON(link->sactive & (1 << qc->tag));
+ WARN_ON_ONCE(link->sactive & (1 << qc->tag));
if (!link->sactive)
ap->nr_active_links++;
link->sactive |= 1 << qc->tag;
} else {
- WARN_ON(link->sactive);
+ WARN_ON_ONCE(link->sactive);
ap->nr_active_links++;
link->active_tag = qc->tag;
diff --git a/drivers/ata/libata-sff.c b/drivers/ata/libata-sff.c
index 9033d16..0eae9b4 100644
--- a/drivers/ata/libata-sff.c
+++ b/drivers/ata/libata-sff.c
@@ -66,6 +66,7 @@
.port_start = ata_sff_port_start,
};
+EXPORT_SYMBOL_GPL(ata_sff_port_ops);
const struct ata_port_operations ata_bmdma_port_ops = {
.inherits = &ata_sff_port_ops,
@@ -77,6 +78,14 @@
.bmdma_stop = ata_bmdma_stop,
.bmdma_status = ata_bmdma_status,
};
+EXPORT_SYMBOL_GPL(ata_bmdma_port_ops);
+
+const struct ata_port_operations ata_bmdma32_port_ops = {
+ .inherits = &ata_bmdma_port_ops,
+
+ .sff_data_xfer = ata_sff_data_xfer32,
+};
+EXPORT_SYMBOL_GPL(ata_bmdma32_port_ops);
/**
* ata_fill_sg - Fill PCI IDE PRD table
@@ -166,8 +175,9 @@
blen = len & 0xffff;
ap->prd[pi].addr = cpu_to_le32(addr);
if (blen == 0) {
- /* Some PATA chipsets like the CS5530 can't
- cope with 0x0000 meaning 64K as the spec says */
+ /* Some PATA chipsets like the CS5530 can't
+ cope with 0x0000 meaning 64K as the spec
+ says */
ap->prd[pi].flags_len = cpu_to_le32(0x8000);
blen = 0x8000;
ap->prd[++pi].addr = cpu_to_le32(addr + 0x8000);
@@ -200,6 +210,7 @@
ata_fill_sg(qc);
}
+EXPORT_SYMBOL_GPL(ata_sff_qc_prep);
/**
* ata_sff_dumb_qc_prep - Prepare taskfile for submission
@@ -217,6 +228,7 @@
ata_fill_sg_dumb(qc);
}
+EXPORT_SYMBOL_GPL(ata_sff_dumb_qc_prep);
/**
* ata_sff_check_status - Read device status reg & clear interrupt
@@ -233,6 +245,7 @@
{
return ioread8(ap->ioaddr.status_addr);
}
+EXPORT_SYMBOL_GPL(ata_sff_check_status);
/**
* ata_sff_altstatus - Read device alternate status reg
@@ -275,7 +288,7 @@
status = ata_sff_altstatus(ap);
/* Not us: We are busy */
if (status & ATA_BUSY)
- return status;
+ return status;
}
/* Clear INTRQ latch */
status = ap->ops->sff_check_status(ap);
@@ -319,6 +332,7 @@
ata_sff_sync(ap);
ndelay(400);
}
+EXPORT_SYMBOL_GPL(ata_sff_pause);
/**
* ata_sff_dma_pause - Pause before commencing DMA
@@ -327,7 +341,7 @@
* Perform I/O fencing and ensure sufficient cycle delays occur
* for the HDMA1:0 transition
*/
-
+
void ata_sff_dma_pause(struct ata_port *ap)
{
if (ap->ops->sff_check_altstatus || ap->ioaddr.altstatus_addr) {
@@ -341,6 +355,7 @@
corruption. */
BUG();
}
+EXPORT_SYMBOL_GPL(ata_sff_dma_pause);
/**
* ata_sff_busy_sleep - sleep until BSY clears, or timeout
@@ -396,6 +411,7 @@
return 0;
}
+EXPORT_SYMBOL_GPL(ata_sff_busy_sleep);
static int ata_sff_check_ready(struct ata_link *link)
{
@@ -422,6 +438,7 @@
{
return ata_wait_ready(link, deadline, ata_sff_check_ready);
}
+EXPORT_SYMBOL_GPL(ata_sff_wait_ready);
/**
* ata_sff_dev_select - Select device 0/1 on ATA bus
@@ -449,6 +466,7 @@
iowrite8(tmp, ap->ioaddr.device_addr);
ata_sff_pause(ap); /* needed; also flushes, for mmio */
}
+EXPORT_SYMBOL_GPL(ata_sff_dev_select);
/**
* ata_dev_select - Select device 0/1 on ATA bus
@@ -513,6 +531,7 @@
return tmp;
}
+EXPORT_SYMBOL_GPL(ata_sff_irq_on);
/**
* ata_sff_irq_clear - Clear PCI IDE BMDMA interrupt.
@@ -534,6 +553,7 @@
iowrite8(ioread8(mmio + ATA_DMA_STATUS), mmio + ATA_DMA_STATUS);
}
+EXPORT_SYMBOL_GPL(ata_sff_irq_clear);
/**
* ata_sff_tf_load - send taskfile registers to host controller
@@ -558,7 +578,7 @@
}
if (is_addr && (tf->flags & ATA_TFLAG_LBA48)) {
- WARN_ON(!ioaddr->ctl_addr);
+ WARN_ON_ONCE(!ioaddr->ctl_addr);
iowrite8(tf->hob_feature, ioaddr->feature_addr);
iowrite8(tf->hob_nsect, ioaddr->nsect_addr);
iowrite8(tf->hob_lbal, ioaddr->lbal_addr);
@@ -593,6 +613,7 @@
ata_wait_idle(ap);
}
+EXPORT_SYMBOL_GPL(ata_sff_tf_load);
/**
* ata_sff_tf_read - input device's ATA taskfile shadow registers
@@ -630,9 +651,10 @@
iowrite8(tf->ctl, ioaddr->ctl_addr);
ap->last_ctl = tf->ctl;
} else
- WARN_ON(1);
+ WARN_ON_ONCE(1);
}
}
+EXPORT_SYMBOL_GPL(ata_sff_tf_read);
/**
* ata_sff_exec_command - issue ATA command to host controller
@@ -652,6 +674,7 @@
iowrite8(tf->command, ap->ioaddr.command_addr);
ata_sff_pause(ap);
}
+EXPORT_SYMBOL_GPL(ata_sff_exec_command);
/**
* ata_tf_to_host - issue ATA taskfile to host controller
@@ -717,6 +740,53 @@
return words << 1;
}
+EXPORT_SYMBOL_GPL(ata_sff_data_xfer);
+
+/**
+ * ata_sff_data_xfer32 - Transfer data by PIO
+ * @dev: device to target
+ * @buf: data buffer
+ * @buflen: buffer length
+ * @rw: read/write
+ *
+ * Transfer data from/to the device data register by PIO using 32bit
+ * I/O operations.
+ *
+ * LOCKING:
+ * Inherited from caller.
+ *
+ * RETURNS:
+ * Bytes consumed.
+ */
+
+unsigned int ata_sff_data_xfer32(struct ata_device *dev, unsigned char *buf,
+ unsigned int buflen, int rw)
+{
+ struct ata_port *ap = dev->link->ap;
+ void __iomem *data_addr = ap->ioaddr.data_addr;
+ unsigned int words = buflen >> 2;
+ int slop = buflen & 3;
+
+ /* Transfer multiple of 4 bytes */
+ if (rw == READ)
+ ioread32_rep(data_addr, buf, words);
+ else
+ iowrite32_rep(data_addr, buf, words);
+
+ if (unlikely(slop)) {
+ __le32 pad;
+ if (rw == READ) {
+ pad = cpu_to_le32(ioread32(ap->ioaddr.data_addr));
+ memcpy(buf + buflen - slop, &pad, slop);
+ } else {
+ memcpy(&pad, buf + buflen - slop, slop);
+ iowrite32(le32_to_cpu(pad), ap->ioaddr.data_addr);
+ }
+ words++;
+ }
+ return words << 2;
+}
+EXPORT_SYMBOL_GPL(ata_sff_data_xfer32);
/**
* ata_sff_data_xfer_noirq - Transfer data by PIO
@@ -746,6 +816,7 @@
return consumed;
}
+EXPORT_SYMBOL_GPL(ata_sff_data_xfer_noirq);
/**
* ata_pio_sector - Transfer a sector of data.
@@ -820,7 +891,7 @@
/* READ/WRITE MULTIPLE */
unsigned int nsect;
- WARN_ON(qc->dev->multi_count == 0);
+ WARN_ON_ONCE(qc->dev->multi_count == 0);
nsect = min((qc->nbytes - qc->curbytes) / qc->sect_size,
qc->dev->multi_count);
@@ -847,7 +918,7 @@
{
/* send SCSI cdb */
DPRINTK("send cdb\n");
- WARN_ON(qc->dev->cdb_len < 12);
+ WARN_ON_ONCE(qc->dev->cdb_len < 12);
ap->ops->sff_data_xfer(qc->dev, qc->cdb, qc->dev->cdb_len, 1);
ata_sff_sync(ap);
@@ -922,13 +993,15 @@
buf = kmap_atomic(page, KM_IRQ0);
/* do the actual data transfer */
- consumed = ap->ops->sff_data_xfer(dev, buf + offset, count, rw);
+ consumed = ap->ops->sff_data_xfer(dev, buf + offset,
+ count, rw);
kunmap_atomic(buf, KM_IRQ0);
local_irq_restore(flags);
} else {
buf = page_address(page);
- consumed = ap->ops->sff_data_xfer(dev, buf + offset, count, rw);
+ consumed = ap->ops->sff_data_xfer(dev, buf + offset,
+ count, rw);
}
bytes -= min(bytes, consumed);
@@ -941,7 +1014,7 @@
}
/* consumed can be larger than count only for the last transfer */
- WARN_ON(qc->cursg && count != consumed);
+ WARN_ON_ONCE(qc->cursg && count != consumed);
if (bytes)
goto next_sg;
@@ -1013,18 +1086,19 @@
* RETURNS:
* 1 if ok in workqueue, 0 otherwise.
*/
-static inline int ata_hsm_ok_in_wq(struct ata_port *ap, struct ata_queued_cmd *qc)
+static inline int ata_hsm_ok_in_wq(struct ata_port *ap,
+ struct ata_queued_cmd *qc)
{
if (qc->tf.flags & ATA_TFLAG_POLLING)
return 1;
if (ap->hsm_task_state == HSM_ST_FIRST) {
if (qc->tf.protocol == ATA_PROT_PIO &&
- (qc->tf.flags & ATA_TFLAG_WRITE))
+ (qc->tf.flags & ATA_TFLAG_WRITE))
return 1;
if (ata_is_atapi(qc->tf.protocol) &&
- !(qc->dev->flags & ATA_DFLAG_CDB_INTR))
+ !(qc->dev->flags & ATA_DFLAG_CDB_INTR))
return 1;
}
@@ -1098,13 +1172,13 @@
unsigned long flags = 0;
int poll_next;
- WARN_ON((qc->flags & ATA_QCFLAG_ACTIVE) == 0);
+ WARN_ON_ONCE((qc->flags & ATA_QCFLAG_ACTIVE) == 0);
/* Make sure ata_sff_qc_issue() does not throw things
* like DMA polling into the workqueue. Notice that
* in_wq is not equivalent to (qc->tf.flags & ATA_TFLAG_POLLING).
*/
- WARN_ON(in_wq != ata_hsm_ok_in_wq(ap, qc));
+ WARN_ON_ONCE(in_wq != ata_hsm_ok_in_wq(ap, qc));
fsm_start:
DPRINTK("ata%u: protocol %d task_state %d (dev_stat 0x%X)\n",
@@ -1313,7 +1387,7 @@
DPRINTK("ata%u: dev %u command complete, drv_stat 0x%x\n",
ap->print_id, qc->dev->devno, status);
- WARN_ON(qc->err_mask & (AC_ERR_DEV | AC_ERR_HSM));
+ WARN_ON_ONCE(qc->err_mask & (AC_ERR_DEV | AC_ERR_HSM));
ap->hsm_task_state = HSM_ST_IDLE;
@@ -1338,6 +1412,7 @@
return poll_next;
}
+EXPORT_SYMBOL_GPL(ata_sff_hsm_move);
void ata_pio_task(struct work_struct *work)
{
@@ -1348,7 +1423,7 @@
int poll_next;
fsm_start:
- WARN_ON(ap->hsm_task_state == HSM_ST_IDLE);
+ WARN_ON_ONCE(ap->hsm_task_state == HSM_ST_IDLE);
/*
* This is purely heuristic. This is a fast path.
@@ -1437,7 +1512,7 @@
break;
case ATA_PROT_DMA:
- WARN_ON(qc->tf.flags & ATA_TFLAG_POLLING);
+ WARN_ON_ONCE(qc->tf.flags & ATA_TFLAG_POLLING);
ap->ops->sff_tf_load(ap, &qc->tf); /* load tf registers */
ap->ops->bmdma_setup(qc); /* set up bmdma */
@@ -1489,7 +1564,7 @@
break;
case ATAPI_PROT_DMA:
- WARN_ON(qc->tf.flags & ATA_TFLAG_POLLING);
+ WARN_ON_ONCE(qc->tf.flags & ATA_TFLAG_POLLING);
ap->ops->sff_tf_load(ap, &qc->tf); /* load tf registers */
ap->ops->bmdma_setup(qc); /* set up bmdma */
@@ -1501,12 +1576,13 @@
break;
default:
- WARN_ON(1);
+ WARN_ON_ONCE(1);
return AC_ERR_SYSTEM;
}
return 0;
}
+EXPORT_SYMBOL_GPL(ata_sff_qc_issue);
/**
* ata_sff_qc_fill_rtf - fill result TF using ->sff_tf_read
@@ -1526,6 +1602,7 @@
qc->ap->ops->sff_tf_read(qc->ap, &qc->result_tf);
return true;
}
+EXPORT_SYMBOL_GPL(ata_sff_qc_fill_rtf);
/**
* ata_sff_host_intr - Handle host interrupt for given (port, task)
@@ -1623,6 +1700,7 @@
#endif
return 0; /* irq not handled */
}
+EXPORT_SYMBOL_GPL(ata_sff_host_intr);
/**
* ata_sff_interrupt - Default ATA host interrupt handler
@@ -1667,6 +1745,7 @@
return IRQ_RETVAL(handled);
}
+EXPORT_SYMBOL_GPL(ata_sff_interrupt);
/**
* ata_sff_freeze - Freeze SFF controller port
@@ -1695,6 +1774,7 @@
ap->ops->sff_irq_clear(ap);
}
+EXPORT_SYMBOL_GPL(ata_sff_freeze);
/**
* ata_sff_thaw - Thaw SFF controller port
@@ -1712,6 +1792,7 @@
ap->ops->sff_irq_clear(ap);
ap->ops->sff_irq_on(ap);
}
+EXPORT_SYMBOL_GPL(ata_sff_thaw);
/**
* ata_sff_prereset - prepare SFF link for reset
@@ -1753,6 +1834,7 @@
return 0;
}
+EXPORT_SYMBOL_GPL(ata_sff_prereset);
/**
* ata_devchk - PATA device presence detection
@@ -1865,6 +1947,7 @@
return class;
}
+EXPORT_SYMBOL_GPL(ata_sff_dev_classify);
/**
* ata_sff_wait_after_reset - wait for devices to become ready after reset
@@ -1941,6 +2024,7 @@
return ret;
}
+EXPORT_SYMBOL_GPL(ata_sff_wait_after_reset);
static int ata_bus_softreset(struct ata_port *ap, unsigned int devmask,
unsigned long deadline)
@@ -2013,6 +2097,7 @@
DPRINTK("EXIT, classes[0]=%u [1]=%u\n", classes[0], classes[1]);
return 0;
}
+EXPORT_SYMBOL_GPL(ata_sff_softreset);
/**
* sata_sff_hardreset - reset host port via SATA phy reset
@@ -2045,6 +2130,7 @@
DPRINTK("EXIT, class=%u\n", *class);
return rc;
}
+EXPORT_SYMBOL_GPL(sata_sff_hardreset);
/**
* ata_sff_postreset - SFF postreset callback
@@ -2080,6 +2166,7 @@
if (ap->ioaddr.ctl_addr)
iowrite8(ap->ctl, ap->ioaddr.ctl_addr);
}
+EXPORT_SYMBOL_GPL(ata_sff_postreset);
/**
* ata_sff_error_handler - Stock error handler for BMDMA controller
@@ -2152,6 +2239,7 @@
ata_do_eh(ap, ap->ops->prereset, softreset, hardreset,
ap->ops->postreset);
}
+EXPORT_SYMBOL_GPL(ata_sff_error_handler);
/**
* ata_sff_post_internal_cmd - Stock post_internal_cmd for SFF controller
@@ -2174,6 +2262,7 @@
spin_unlock_irqrestore(ap->lock, flags);
}
+EXPORT_SYMBOL_GPL(ata_sff_post_internal_cmd);
/**
* ata_sff_port_start - Set port up for dma.
@@ -2194,6 +2283,7 @@
return ata_port_start(ap);
return 0;
}
+EXPORT_SYMBOL_GPL(ata_sff_port_start);
/**
* ata_sff_std_ports - initialize ioaddr with standard port offsets.
@@ -2219,6 +2309,7 @@
ioaddr->status_addr = ioaddr->cmd_addr + ATA_REG_STATUS;
ioaddr->command_addr = ioaddr->cmd_addr + ATA_REG_CMD;
}
+EXPORT_SYMBOL_GPL(ata_sff_std_ports);
unsigned long ata_bmdma_mode_filter(struct ata_device *adev,
unsigned long xfer_mask)
@@ -2230,6 +2321,7 @@
xfer_mask &= ~(ATA_MASK_MWDMA | ATA_MASK_UDMA);
return xfer_mask;
}
+EXPORT_SYMBOL_GPL(ata_bmdma_mode_filter);
/**
* ata_bmdma_setup - Set up PCI IDE BMDMA transaction
@@ -2258,6 +2350,7 @@
/* issue r/w command */
ap->ops->sff_exec_command(ap, &qc->tf);
}
+EXPORT_SYMBOL_GPL(ata_bmdma_setup);
/**
* ata_bmdma_start - Start a PCI IDE BMDMA transaction
@@ -2290,6 +2383,7 @@
* unneccessarily delayed for MMIO
*/
}
+EXPORT_SYMBOL_GPL(ata_bmdma_start);
/**
* ata_bmdma_stop - Stop PCI IDE BMDMA transfer
@@ -2314,6 +2408,7 @@
/* one-PIO-cycle guaranteed wait, per spec, for HDMA1:0 transition */
ata_sff_dma_pause(ap);
}
+EXPORT_SYMBOL_GPL(ata_bmdma_stop);
/**
* ata_bmdma_status - Read PCI IDE BMDMA status
@@ -2330,6 +2425,7 @@
{
return ioread8(ap->ioaddr.bmdma_addr + ATA_DMA_STATUS);
}
+EXPORT_SYMBOL_GPL(ata_bmdma_status);
/**
* ata_bus_reset - reset host port and associated ATA channel
@@ -2422,6 +2518,7 @@
DPRINTK("EXIT\n");
}
+EXPORT_SYMBOL_GPL(ata_bus_reset);
#ifdef CONFIG_PCI
@@ -2449,6 +2546,7 @@
return -EOPNOTSUPP;
return 0;
}
+EXPORT_SYMBOL_GPL(ata_pci_bmdma_clear_simplex);
/**
* ata_pci_bmdma_init - acquire PCI BMDMA resources and init ATA host
@@ -2501,11 +2599,12 @@
host->flags |= ATA_HOST_SIMPLEX;
ata_port_desc(ap, "bmdma 0x%llx",
- (unsigned long long)pci_resource_start(pdev, 4) + 8 * i);
+ (unsigned long long)pci_resource_start(pdev, 4) + 8 * i);
}
return 0;
}
+EXPORT_SYMBOL_GPL(ata_pci_bmdma_init);
static int ata_resources_present(struct pci_dev *pdev, int port)
{
@@ -2513,7 +2612,7 @@
/* Check the PCI resources for this channel are enabled */
port = port * 2;
- for (i = 0; i < 2; i ++) {
+ for (i = 0; i < 2; i++) {
if (pci_resource_start(pdev, port + i) == 0 ||
pci_resource_len(pdev, port + i) == 0)
return 0;
@@ -2598,6 +2697,7 @@
return 0;
}
+EXPORT_SYMBOL_GPL(ata_pci_sff_init_host);
/**
* ata_pci_sff_prepare_host - helper to prepare native PCI ATA host
@@ -2615,7 +2715,7 @@
* 0 on success, -errno otherwise.
*/
int ata_pci_sff_prepare_host(struct pci_dev *pdev,
- const struct ata_port_info * const * ppi,
+ const struct ata_port_info * const *ppi,
struct ata_host **r_host)
{
struct ata_host *host;
@@ -2645,17 +2745,18 @@
*r_host = host;
return 0;
- err_bmdma:
+err_bmdma:
/* This is necessary because PCI and iomap resources are
* merged and releasing the top group won't release the
* acquired resources if some of those have been acquired
* before entering this function.
*/
pcim_iounmap_regions(pdev, 0xf);
- err_out:
+err_out:
devres_release_group(&pdev->dev, NULL);
return rc;
}
+EXPORT_SYMBOL_GPL(ata_pci_sff_prepare_host);
/**
* ata_pci_sff_activate_host - start SFF host, request IRQ and register it
@@ -2741,7 +2842,7 @@
}
rc = ata_host_register(host, sht);
- out:
+out:
if (rc == 0)
devres_remove_group(dev, NULL);
else
@@ -2749,6 +2850,7 @@
return rc;
}
+EXPORT_SYMBOL_GPL(ata_pci_sff_activate_host);
/**
* ata_pci_sff_init_one - Initialize/register PCI IDE host controller
@@ -2776,7 +2878,7 @@
* Zero on success, negative on errno-based value on error.
*/
int ata_pci_sff_init_one(struct pci_dev *pdev,
- const struct ata_port_info * const * ppi,
+ const struct ata_port_info * const *ppi,
struct scsi_host_template *sht, void *host_priv)
{
struct device *dev = &pdev->dev;
@@ -2815,7 +2917,7 @@
pci_set_master(pdev);
rc = ata_pci_sff_activate_host(host, ata_sff_interrupt, sht);
- out:
+out:
if (rc == 0)
devres_remove_group(&pdev->dev, NULL);
else
@@ -2823,54 +2925,7 @@
return rc;
}
-
-#endif /* CONFIG_PCI */
-
-EXPORT_SYMBOL_GPL(ata_sff_port_ops);
-EXPORT_SYMBOL_GPL(ata_bmdma_port_ops);
-EXPORT_SYMBOL_GPL(ata_sff_qc_prep);
-EXPORT_SYMBOL_GPL(ata_sff_dumb_qc_prep);
-EXPORT_SYMBOL_GPL(ata_sff_dev_select);
-EXPORT_SYMBOL_GPL(ata_sff_check_status);
-EXPORT_SYMBOL_GPL(ata_sff_dma_pause);
-EXPORT_SYMBOL_GPL(ata_sff_pause);
-EXPORT_SYMBOL_GPL(ata_sff_busy_sleep);
-EXPORT_SYMBOL_GPL(ata_sff_wait_ready);
-EXPORT_SYMBOL_GPL(ata_sff_tf_load);
-EXPORT_SYMBOL_GPL(ata_sff_tf_read);
-EXPORT_SYMBOL_GPL(ata_sff_exec_command);
-EXPORT_SYMBOL_GPL(ata_sff_data_xfer);
-EXPORT_SYMBOL_GPL(ata_sff_data_xfer_noirq);
-EXPORT_SYMBOL_GPL(ata_sff_irq_on);
-EXPORT_SYMBOL_GPL(ata_sff_irq_clear);
-EXPORT_SYMBOL_GPL(ata_sff_hsm_move);
-EXPORT_SYMBOL_GPL(ata_sff_qc_issue);
-EXPORT_SYMBOL_GPL(ata_sff_qc_fill_rtf);
-EXPORT_SYMBOL_GPL(ata_sff_host_intr);
-EXPORT_SYMBOL_GPL(ata_sff_interrupt);
-EXPORT_SYMBOL_GPL(ata_sff_freeze);
-EXPORT_SYMBOL_GPL(ata_sff_thaw);
-EXPORT_SYMBOL_GPL(ata_sff_prereset);
-EXPORT_SYMBOL_GPL(ata_sff_dev_classify);
-EXPORT_SYMBOL_GPL(ata_sff_wait_after_reset);
-EXPORT_SYMBOL_GPL(ata_sff_softreset);
-EXPORT_SYMBOL_GPL(sata_sff_hardreset);
-EXPORT_SYMBOL_GPL(ata_sff_postreset);
-EXPORT_SYMBOL_GPL(ata_sff_error_handler);
-EXPORT_SYMBOL_GPL(ata_sff_post_internal_cmd);
-EXPORT_SYMBOL_GPL(ata_sff_port_start);
-EXPORT_SYMBOL_GPL(ata_sff_std_ports);
-EXPORT_SYMBOL_GPL(ata_bmdma_mode_filter);
-EXPORT_SYMBOL_GPL(ata_bmdma_setup);
-EXPORT_SYMBOL_GPL(ata_bmdma_start);
-EXPORT_SYMBOL_GPL(ata_bmdma_stop);
-EXPORT_SYMBOL_GPL(ata_bmdma_status);
-EXPORT_SYMBOL_GPL(ata_bus_reset);
-#ifdef CONFIG_PCI
-EXPORT_SYMBOL_GPL(ata_pci_bmdma_clear_simplex);
-EXPORT_SYMBOL_GPL(ata_pci_bmdma_init);
-EXPORT_SYMBOL_GPL(ata_pci_sff_init_host);
-EXPORT_SYMBOL_GPL(ata_pci_sff_prepare_host);
-EXPORT_SYMBOL_GPL(ata_pci_sff_activate_host);
EXPORT_SYMBOL_GPL(ata_pci_sff_init_one);
+
#endif /* CONFIG_PCI */
+
diff --git a/drivers/ata/pata_acpi.c b/drivers/ata/pata_acpi.c
index e2e332d..8b77a98 100644
--- a/drivers/ata/pata_acpi.c
+++ b/drivers/ata/pata_acpi.c
@@ -13,12 +13,6 @@
#include <linux/device.h>
#include <scsi/scsi_host.h>
#include <acpi/acpi_bus.h>
-#include <acpi/acnames.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acparser.h>
-#include <acpi/acexcep.h>
-#include <acpi/acmacros.h>
-#include <acpi/actypes.h>
#include <linux/libata.h>
#include <linux/ata.h>
diff --git a/drivers/ata/pata_ali.c b/drivers/ata/pata_ali.c
index 73c466e..a7999c1 100644
--- a/drivers/ata/pata_ali.c
+++ b/drivers/ata/pata_ali.c
@@ -19,7 +19,9 @@
*
* TODO/CHECK
* Cannot have ATAPI on both master & slave for rev < c2 (???) but
- * otherwise should do atapi DMA.
+ * otherwise should do atapi DMA (For now for old we do PIO only for
+ * ATAPI)
+ * Review Sunblade workaround.
*/
#include <linux/kernel.h>
@@ -33,12 +35,14 @@
#include <linux/dmi.h>
#define DRV_NAME "pata_ali"
-#define DRV_VERSION "0.7.5"
+#define DRV_VERSION "0.7.8"
static int ali_atapi_dma = 0;
module_param_named(atapi_dma, ali_atapi_dma, int, 0644);
MODULE_PARM_DESC(atapi_dma, "Enable ATAPI DMA (0=disable, 1=enable)");
+static struct pci_dev *isa_bridge;
+
/*
* Cable special cases
*/
@@ -147,8 +151,7 @@
pci_read_config_byte(pdev, pio_fifo, &fifo);
fifo &= ~(0x0F << shift);
- if (on)
- fifo |= (on << shift);
+ fifo |= (on << shift);
pci_write_config_byte(pdev, pio_fifo, fifo);
}
@@ -337,6 +340,23 @@
return 0;
}
+static void ali_c2_c3_postreset(struct ata_link *link, unsigned int *classes)
+{
+ u8 r;
+ int port_bit = 4 << link->ap->port_no;
+
+ /* If our bridge is an ALI 1533 then do the extra work */
+ if (isa_bridge) {
+ /* Tristate and re-enable the bus signals */
+ pci_read_config_byte(isa_bridge, 0x58, &r);
+ r &= ~port_bit;
+ pci_write_config_byte(isa_bridge, 0x58, r);
+ r |= port_bit;
+ pci_write_config_byte(isa_bridge, 0x58, r);
+ }
+ ata_sff_postreset(link, classes);
+}
+
static struct scsi_host_template ali_sht = {
ATA_BMDMA_SHT(DRV_NAME),
};
@@ -349,10 +369,11 @@
.inherits = &ata_sff_port_ops,
.cable_detect = ata_cable_40wire,
.set_piomode = ali_set_piomode,
+ .sff_data_xfer = ata_sff_data_xfer32,
};
static const struct ata_port_operations ali_dma_base_ops = {
- .inherits = &ata_bmdma_port_ops,
+ .inherits = &ata_bmdma32_port_ops,
.set_piomode = ali_set_piomode,
.set_dmamode = ali_set_dmamode,
};
@@ -377,6 +398,17 @@
.check_atapi_dma = ali_check_atapi_dma,
.cable_detect = ali_c2_cable_detect,
.dev_config = ali_lock_sectors,
+ .postreset = ali_c2_c3_postreset,
+};
+
+/*
+ * Port operations for DMA capable ALi with cable detect
+ */
+static struct ata_port_operations ali_c4_port_ops = {
+ .inherits = &ali_dma_base_ops,
+ .check_atapi_dma = ali_check_atapi_dma,
+ .cable_detect = ali_c2_cable_detect,
+ .dev_config = ali_lock_sectors,
};
/*
@@ -401,27 +433,40 @@
static void ali_init_chipset(struct pci_dev *pdev)
{
u8 tmp;
- struct pci_dev *north, *isa_bridge;
+ struct pci_dev *north;
/*
* The chipset revision selects the driver operations and
* mode data.
*/
- if (pdev->revision >= 0x20 && pdev->revision < 0xC2) {
- /* 1543-E/F, 1543C-C, 1543C-D, 1543C-E */
+ if (pdev->revision <= 0x20) {
+ pci_read_config_byte(pdev, 0x53, &tmp);
+ tmp |= 0x03;
+ pci_write_config_byte(pdev, 0x53, tmp);
+ } else {
+ pci_read_config_byte(pdev, 0x4a, &tmp);
+ pci_write_config_byte(pdev, 0x4a, tmp | 0x20);
pci_read_config_byte(pdev, 0x4B, &tmp);
- /* Clear CD-ROM DMA write bit */
- tmp &= 0x7F;
- pci_write_config_byte(pdev, 0x4B, tmp);
- } else if (pdev->revision >= 0xC2) {
- /* Enable cable detection logic */
- pci_read_config_byte(pdev, 0x4B, &tmp);
- pci_write_config_byte(pdev, 0x4B, tmp | 0x08);
+ if (pdev->revision < 0xC2)
+ /* 1543-E/F, 1543C-C, 1543C-D, 1543C-E */
+ /* Clear CD-ROM DMA write bit */
+ tmp &= 0x7F;
+ /* Cable and UDMA */
+ pci_write_config_byte(pdev, 0x4B, tmp | 0x09);
+ /*
+ * CD_ROM DMA on (0x53 bit 0). Enable this even if we want
+ * to use PIO. 0x53 bit 1 (rev 20 only) - enable FIFO control
+ * via 0x54/55.
+ */
+ pci_read_config_byte(pdev, 0x53, &tmp);
+ if (pdev->revision >= 0xc7)
+ tmp |= 0x03;
+ else
+ tmp |= 0x01; /* CD_ROM enable for DMA */
+ pci_write_config_byte(pdev, 0x53, tmp);
}
north = pci_get_bus_and_slot(0, PCI_DEVFN(0,0));
- isa_bridge = pci_get_device(PCI_VENDOR_ID_AL, PCI_DEVICE_ID_AL_M1533, NULL);
-
if (north && north->vendor == PCI_VENDOR_ID_AL && isa_bridge) {
/* Configure the ALi bridge logic. For non ALi rely on BIOS.
Set the south bridge enable bit */
@@ -431,22 +476,6 @@
else if (pdev->revision > 0xC2 && pdev->revision < 0xC5)
pci_write_config_byte(isa_bridge, 0x79, tmp | 0x02);
}
- if (pdev->revision >= 0x20) {
- /*
- * CD_ROM DMA on (0x53 bit 0). Enable this even if we want
- * to use PIO. 0x53 bit 1 (rev 20 only) - enable FIFO control
- * via 0x54/55.
- */
- pci_read_config_byte(pdev, 0x53, &tmp);
- if (pdev->revision <= 0x20)
- tmp &= ~0x02;
- if (pdev->revision >= 0xc7)
- tmp |= 0x03;
- else
- tmp |= 0x01; /* CD_ROM enable for DMA */
- pci_write_config_byte(pdev, 0x53, tmp);
- }
- pci_dev_put(isa_bridge);
pci_dev_put(north);
ata_pci_bmdma_clear_simplex(pdev);
}
@@ -503,7 +532,7 @@
.pio_mask = 0x1f,
.mwdma_mask = 0x07,
.udma_mask = ATA_UDMA5,
- .port_ops = &ali_c2_port_ops
+ .port_ops = &ali_c4_port_ops
};
/* Revision 0xC5 is UDMA133 with LBA48 DMA */
static const struct ata_port_info info_c5 = {
@@ -516,7 +545,6 @@
const struct ata_port_info *ppi[] = { NULL, NULL };
u8 tmp;
- struct pci_dev *isa_bridge;
int rc;
rc = pcim_enable_device(pdev);
@@ -543,14 +571,12 @@
ali_init_chipset(pdev);
- isa_bridge = pci_get_device(PCI_VENDOR_ID_AL, PCI_DEVICE_ID_AL_M1533, NULL);
if (isa_bridge && pdev->revision >= 0x20 && pdev->revision < 0xC2) {
/* Are we paired with a UDMA capable chip */
pci_read_config_byte(isa_bridge, 0x5E, &tmp);
if ((tmp & 0x1E) == 0x12)
ppi[0] = &info_20_udma;
}
- pci_dev_put(isa_bridge);
return ata_pci_sff_init_one(pdev, ppi, &ali_sht, NULL);
}
@@ -590,13 +616,20 @@
static int __init ali_init(void)
{
- return pci_register_driver(&ali_pci_driver);
+ int ret;
+ isa_bridge = pci_get_device(PCI_VENDOR_ID_AL, PCI_DEVICE_ID_AL_M1533, NULL);
+
+ ret = pci_register_driver(&ali_pci_driver);
+ if (ret < 0)
+ pci_dev_put(isa_bridge);
+ return ret;
}
static void __exit ali_exit(void)
{
pci_unregister_driver(&ali_pci_driver);
+ pci_dev_put(isa_bridge);
}
diff --git a/drivers/ata/pata_amd.c b/drivers/ata/pata_amd.c
index 0ec9c7d..63719ab 100644
--- a/drivers/ata/pata_amd.c
+++ b/drivers/ata/pata_amd.c
@@ -24,7 +24,7 @@
#include <linux/libata.h>
#define DRV_NAME "pata_amd"
-#define DRV_VERSION "0.3.10"
+#define DRV_VERSION "0.3.11"
/**
* timing_setup - shared timing computation and load
@@ -345,7 +345,7 @@
};
static const struct ata_port_operations amd_base_port_ops = {
- .inherits = &ata_bmdma_port_ops,
+ .inherits = &ata_bmdma32_port_ops,
.prereset = amd_pre_reset,
};
diff --git a/drivers/ata/pata_hpt366.c b/drivers/ata/pata_hpt366.c
index e0c4f05..65c28e5 100644
--- a/drivers/ata/pata_hpt366.c
+++ b/drivers/ata/pata_hpt366.c
@@ -30,7 +30,7 @@
#define DRV_VERSION "0.6.2"
struct hpt_clock {
- u8 xfer_speed;
+ u8 xfer_mode;
u32 timing;
};
@@ -189,28 +189,6 @@
return ata_bmdma_mode_filter(adev, mask);
}
-/**
- * hpt36x_find_mode - reset the hpt36x bus
- * @ap: ATA port
- * @speed: transfer mode
- *
- * Return the 32bit register programming information for this channel
- * that matches the speed provided.
- */
-
-static u32 hpt36x_find_mode(struct ata_port *ap, int speed)
-{
- struct hpt_clock *clocks = ap->host->private_data;
-
- while(clocks->xfer_speed) {
- if (clocks->xfer_speed == speed)
- return clocks->timing;
- clocks++;
- }
- BUG();
- return 0xffffffffU; /* silence compiler warning */
-}
-
static int hpt36x_cable_detect(struct ata_port *ap)
{
struct pci_dev *pdev = to_pci_dev(ap->host->dev);
@@ -226,6 +204,49 @@
return ATA_CBL_PATA80;
}
+static void hpt366_set_mode(struct ata_port *ap, struct ata_device *adev,
+ u8 mode)
+{
+ struct hpt_clock *clocks = ap->host->private_data;
+ struct pci_dev *pdev = to_pci_dev(ap->host->dev);
+ u32 addr1 = 0x40 + 4 * (adev->devno + 2 * ap->port_no);
+ u32 addr2 = 0x51 + 4 * ap->port_no;
+ u32 mask, reg;
+ u8 fast;
+
+ /* Fast interrupt prediction disable, hold off interrupt disable */
+ pci_read_config_byte(pdev, addr2, &fast);
+ if (fast & 0x80) {
+ fast &= ~0x80;
+ pci_write_config_byte(pdev, addr2, fast);
+ }
+
+ /* determine timing mask and find matching clock entry */
+ if (mode < XFER_MW_DMA_0)
+ mask = 0xc1f8ffff;
+ else if (mode < XFER_UDMA_0)
+ mask = 0x303800ff;
+ else
+ mask = 0x30070000;
+
+ while (clocks->xfer_mode) {
+ if (clocks->xfer_mode == mode)
+ break;
+ clocks++;
+ }
+ if (!clocks->xfer_mode)
+ BUG();
+
+ /*
+ * Combine new mode bits with old config bits and disable
+ * on-chip PIO FIFO/buffer (and PIO MST mode as well) to avoid
+ * problems handling I/O errors later.
+ */
+ pci_read_config_dword(pdev, addr1, ®);
+ reg = ((reg & ~mask) | (clocks->timing & mask)) & ~0xc0000000;
+ pci_write_config_dword(pdev, addr1, reg);
+}
+
/**
* hpt366_set_piomode - PIO setup
* @ap: ATA interface
@@ -236,28 +257,7 @@
static void hpt366_set_piomode(struct ata_port *ap, struct ata_device *adev)
{
- struct pci_dev *pdev = to_pci_dev(ap->host->dev);
- u32 addr1, addr2;
- u32 reg;
- u32 mode;
- u8 fast;
-
- addr1 = 0x40 + 4 * (adev->devno + 2 * ap->port_no);
- addr2 = 0x51 + 4 * ap->port_no;
-
- /* Fast interrupt prediction disable, hold off interrupt disable */
- pci_read_config_byte(pdev, addr2, &fast);
- if (fast & 0x80) {
- fast &= ~0x80;
- pci_write_config_byte(pdev, addr2, fast);
- }
-
- pci_read_config_dword(pdev, addr1, ®);
- mode = hpt36x_find_mode(ap, adev->pio_mode);
- mode &= ~0x8000000; /* No FIFO in PIO */
- mode &= ~0x30070000; /* Leave config bits alone */
- reg &= 0x30070000; /* Strip timing bits */
- pci_write_config_dword(pdev, addr1, reg | mode);
+ hpt366_set_mode(ap, adev, adev->pio_mode);
}
/**
@@ -271,28 +271,7 @@
static void hpt366_set_dmamode(struct ata_port *ap, struct ata_device *adev)
{
- struct pci_dev *pdev = to_pci_dev(ap->host->dev);
- u32 addr1, addr2;
- u32 reg;
- u32 mode;
- u8 fast;
-
- addr1 = 0x40 + 4 * (adev->devno + 2 * ap->port_no);
- addr2 = 0x51 + 4 * ap->port_no;
-
- /* Fast interrupt prediction disable, hold off interrupt disable */
- pci_read_config_byte(pdev, addr2, &fast);
- if (fast & 0x80) {
- fast &= ~0x80;
- pci_write_config_byte(pdev, addr2, fast);
- }
-
- pci_read_config_dword(pdev, addr1, ®);
- mode = hpt36x_find_mode(ap, adev->dma_mode);
- mode |= 0x8000000; /* FIFO in MWDMA or UDMA */
- mode &= ~0xC0000000; /* Leave config bits alone */
- reg &= 0xC0000000; /* Strip timing bits */
- pci_write_config_dword(pdev, addr1, reg | mode);
+ hpt366_set_mode(ap, adev, adev->dma_mode);
}
static struct scsi_host_template hpt36x_sht = {
diff --git a/drivers/ata/pata_hpt3x3.c b/drivers/ata/pata_hpt3x3.c
index f11a320..f19cc64 100644
--- a/drivers/ata/pata_hpt3x3.c
+++ b/drivers/ata/pata_hpt3x3.c
@@ -23,7 +23,7 @@
#include <linux/libata.h>
#define DRV_NAME "pata_hpt3x3"
-#define DRV_VERSION "0.5.3"
+#define DRV_VERSION "0.6.1"
/**
* hpt3x3_set_piomode - PIO setup
@@ -80,14 +80,48 @@
r2 &= ~(0x11 << dn); /* Clear MWDMA and UDMA bits */
if (adev->dma_mode >= XFER_UDMA_0)
- r2 |= (0x10 << dn); /* Ultra mode */
+ r2 |= (0x01 << dn); /* Ultra mode */
else
- r2 |= (0x01 << dn); /* MWDMA */
+ r2 |= (0x10 << dn); /* MWDMA */
pci_write_config_dword(pdev, 0x44, r1);
pci_write_config_dword(pdev, 0x48, r2);
}
-#endif /* CONFIG_PATA_HPT3X3_DMA */
+
+/**
+ * hpt3x3_freeze - DMA workaround
+ * @ap: port to freeze
+ *
+ * When freezing an HPT3x3 we must stop any pending DMA before
+ * writing to the control register or the chip will hang
+ */
+
+static void hpt3x3_freeze(struct ata_port *ap)
+{
+ void __iomem *mmio = ap->ioaddr.bmdma_addr;
+
+ iowrite8(ioread8(mmio + ATA_DMA_CMD) & ~ ATA_DMA_START,
+ mmio + ATA_DMA_CMD);
+ ata_sff_dma_pause(ap);
+ ata_sff_freeze(ap);
+}
+
+/**
+ * hpt3x3_bmdma_setup - DMA workaround
+ * @qc: Queued command
+ *
+ * When issuing BMDMA we must clean up the error/active bits in
+ * software on this device
+ */
+
+static void hpt3x3_bmdma_setup(struct ata_queued_cmd *qc)
+{
+ struct ata_port *ap = qc->ap;
+ u8 r = ioread8(ap->ioaddr.bmdma_addr + ATA_DMA_STATUS);
+ r |= ATA_DMA_INTR | ATA_DMA_ERR;
+ iowrite8(r, ap->ioaddr.bmdma_addr + ATA_DMA_STATUS);
+ return ata_bmdma_setup(qc);
+}
/**
* hpt3x3_atapi_dma - ATAPI DMA check
@@ -101,18 +135,23 @@
return 1;
}
+#endif /* CONFIG_PATA_HPT3X3_DMA */
+
static struct scsi_host_template hpt3x3_sht = {
ATA_BMDMA_SHT(DRV_NAME),
};
static struct ata_port_operations hpt3x3_port_ops = {
.inherits = &ata_bmdma_port_ops,
- .check_atapi_dma= hpt3x3_atapi_dma,
.cable_detect = ata_cable_40wire,
.set_piomode = hpt3x3_set_piomode,
#if defined(CONFIG_PATA_HPT3X3_DMA)
.set_dmamode = hpt3x3_set_dmamode,
+ .bmdma_setup = hpt3x3_bmdma_setup,
+ .check_atapi_dma= hpt3x3_atapi_dma,
+ .freeze = hpt3x3_freeze,
#endif
+
};
/**
diff --git a/drivers/ata/pata_mpiix.c b/drivers/ata/pata_mpiix.c
index 7c8faa4..aa576ca 100644
--- a/drivers/ata/pata_mpiix.c
+++ b/drivers/ata/pata_mpiix.c
@@ -35,7 +35,7 @@
#include <linux/libata.h>
#define DRV_NAME "pata_mpiix"
-#define DRV_VERSION "0.7.6"
+#define DRV_VERSION "0.7.7"
enum {
IDETIM = 0x6C, /* IDE control register */
@@ -146,6 +146,7 @@
.cable_detect = ata_cable_40wire,
.set_piomode = mpiix_set_piomode,
.prereset = mpiix_pre_reset,
+ .sff_data_xfer = ata_sff_data_xfer32,
};
static int mpiix_init_one(struct pci_dev *dev, const struct pci_device_id *id)
diff --git a/drivers/ata/pata_platform.c b/drivers/ata/pata_platform.c
index 6afa07a..d8d743a 100644
--- a/drivers/ata/pata_platform.c
+++ b/drivers/ata/pata_platform.c
@@ -186,7 +186,7 @@
* A platform bus ATA device has been unplugged. Perform the needed
* cleanup. Also called on module unload for any active devices.
*/
-int __devexit __pata_platform_remove(struct device *dev)
+int __pata_platform_remove(struct device *dev)
{
struct ata_host *host = dev_get_drvdata(dev);
diff --git a/drivers/ata/pata_sil680.c b/drivers/ata/pata_sil680.c
index 83580a5..9e764e5 100644
--- a/drivers/ata/pata_sil680.c
+++ b/drivers/ata/pata_sil680.c
@@ -32,7 +32,7 @@
#include <linux/libata.h>
#define DRV_NAME "pata_sil680"
-#define DRV_VERSION "0.4.8"
+#define DRV_VERSION "0.4.9"
#define SIL680_MMIO_BAR 5
@@ -195,7 +195,7 @@
};
static struct ata_port_operations sil680_port_ops = {
- .inherits = &ata_bmdma_port_ops,
+ .inherits = &ata_bmdma32_port_ops,
.cable_detect = sil680_cable_detect,
.set_piomode = sil680_set_piomode,
.set_dmamode = sil680_set_dmamode,
diff --git a/drivers/ata/sata_sil24.c b/drivers/ata/sata_sil24.c
index ccee930..2590c22 100644
--- a/drivers/ata/sata_sil24.c
+++ b/drivers/ata/sata_sil24.c
@@ -51,13 +51,6 @@
__le32 flags;
};
-/*
- * Port multiplier
- */
-struct sil24_port_multiplier {
- __le32 diag;
- __le32 sactive;
-};
enum {
SIL24_HOST_BAR = 0,
diff --git a/drivers/atm/iphase.c b/drivers/atm/iphase.c
index 088885e..e1c7611 100644
--- a/drivers/atm/iphase.c
+++ b/drivers/atm/iphase.c
@@ -64,7 +64,7 @@
#include <linux/jiffies.h>
#include "iphase.h"
#include "suni.h"
-#define swap(x) (((x & 0xff) << 8) | ((x & 0xff00) >> 8))
+#define swap_byte_order(x) (((x & 0xff) << 8) | ((x & 0xff00) >> 8))
#define PRIV(dev) ((struct suni_priv *) dev->phy_data)
@@ -1306,7 +1306,7 @@
// get real pkt length pwang_test
trailer = (struct cpcs_trailer*)((u_char *)skb->data +
skb->len - sizeof(*trailer));
- length = swap(trailer->length);
+ length = swap_byte_order(trailer->length);
if ((length > iadev->rx_buf_sz) || (length >
(skb->len - sizeof(struct cpcs_trailer))))
{
@@ -2995,7 +2995,7 @@
skb->len, PCI_DMA_TODEVICE);
wr_ptr->local_pkt_addr = (buf_desc_ptr->buf_start_hi << 16) |
buf_desc_ptr->buf_start_lo;
- /* wr_ptr->bytes = swap(total_len); didn't seem to affect ?? */
+ /* wr_ptr->bytes = swap_byte_order(total_len); didn't seem to affect?? */
wr_ptr->bytes = skb->len;
/* hw bug - DLEs of 0x2d, 0x2e, 0x2f cause DMA lockup */
diff --git a/drivers/char/Kconfig b/drivers/char/Kconfig
index 35914b6..f5be808 100644
--- a/drivers/char/Kconfig
+++ b/drivers/char/Kconfig
@@ -616,6 +616,7 @@
default y
select HVC_DRIVER
select HVC_IRQ
+ select VIOPATH
help
iSeries machines support a hypervisor virtual console.
diff --git a/drivers/char/hvc_beat.c b/drivers/char/hvc_beat.c
index 91cdb35..0afc8b8 100644
--- a/drivers/char/hvc_beat.c
+++ b/drivers/char/hvc_beat.c
@@ -44,7 +44,7 @@
static unsigned char q[sizeof(unsigned long) * 2]
__attribute__((aligned(sizeof(unsigned long))));
static int qlen = 0;
- unsigned long got;
+ u64 got;
again:
if (qlen) {
@@ -63,7 +63,7 @@
}
}
if (beat_get_term_char(vtermno, &got,
- ((unsigned long *)q), ((unsigned long *)q) + 1) == 0) {
+ ((u64 *)q), ((u64 *)q) + 1) == 0) {
qlen = got;
goto again;
}
diff --git a/drivers/char/pty.c b/drivers/char/pty.c
index 112a6ba..146c976 100644
--- a/drivers/char/pty.c
+++ b/drivers/char/pty.c
@@ -32,7 +32,7 @@
/* These are global because they are accessed in tty_io.c */
#ifdef CONFIG_UNIX98_PTYS
-struct tty_driver *ptm_driver;
+static struct tty_driver *ptm_driver;
static struct tty_driver *pts_driver;
#endif
diff --git a/drivers/char/rtc.c b/drivers/char/rtc.c
index 20d6efb..e0d0f8b 100644
--- a/drivers/char/rtc.c
+++ b/drivers/char/rtc.c
@@ -48,9 +48,10 @@
* CONFIG_HPET_EMULATE_RTC
* 1.12a Maciej W. Rozycki: Handle memory-mapped chips properly.
* 1.12ac Alan Cox: Allow read access to the day of week register
+ * 1.12b David John: Remove calls to the BKL.
*/
-#define RTC_VERSION "1.12ac"
+#define RTC_VERSION "1.12b"
/*
* Note that *all* calls to CMOS_READ and CMOS_WRITE are done with
@@ -73,7 +74,6 @@
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/spinlock.h>
-#include <linux/smp_lock.h>
#include <linux/sysctl.h>
#include <linux/wait.h>
#include <linux/bcd.h>
@@ -182,8 +182,8 @@
/*
* rtc_status is never changed by rtc_interrupt, and ioctl/open/close is
- * protected by the big kernel lock. However, ioctl can still disable the timer
- * in rtc_status and then with del_timer after the interrupt has read
+ * protected by the spin lock rtc_lock. However, ioctl can still disable the
+ * timer in rtc_status and then with del_timer after the interrupt has read
* rtc_status but before mod_timer is called, which would then reenable the
* timer (but you would need to have an awful timing before you'd trip on it)
*/
@@ -720,9 +720,7 @@
static long rtc_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
{
long ret;
- lock_kernel();
ret = rtc_do_ioctl(cmd, arg, 0);
- unlock_kernel();
return ret;
}
@@ -731,12 +729,8 @@
* Also clear the previous interrupt data on an open, and clean
* up things on a close.
*/
-
-/* We use rtc_lock to protect against concurrent opens. So the BKL is not
- * needed here. Or anywhere else in this driver. */
static int rtc_open(struct inode *inode, struct file *file)
{
- lock_kernel();
spin_lock_irq(&rtc_lock);
if (rtc_status & RTC_IS_OPEN)
@@ -746,12 +740,10 @@
rtc_irq_data = 0;
spin_unlock_irq(&rtc_lock);
- unlock_kernel();
return 0;
out_busy:
spin_unlock_irq(&rtc_lock);
- unlock_kernel();
return -EBUSY;
}
@@ -800,7 +792,6 @@
}
#ifdef RTC_IRQ
-/* Called without the kernel lock - fine */
static unsigned int rtc_poll(struct file *file, poll_table *wait)
{
unsigned long l;
diff --git a/drivers/char/tpm/tpm_bios.c b/drivers/char/tpm/tpm_bios.c
index 68f052b..ed306eb 100644
--- a/drivers/char/tpm/tpm_bios.c
+++ b/drivers/char/tpm/tpm_bios.c
@@ -23,8 +23,6 @@
#include <linux/security.h>
#include <linux/module.h>
#include <acpi/acpi.h>
-#include <acpi/actypes.h>
-#include <acpi/actbl.h>
#include "tpm.h"
#define TCG_EVENT_NAME_LEN_MAX 255
diff --git a/drivers/char/tpm/tpm_nsc.c b/drivers/char/tpm/tpm_nsc.c
index ab18c1e..70efba2 100644
--- a/drivers/char/tpm/tpm_nsc.c
+++ b/drivers/char/tpm/tpm_nsc.c
@@ -273,12 +273,23 @@
}
}
-static struct device_driver nsc_drv = {
- .name = "tpm_nsc",
- .bus = &platform_bus_type,
- .owner = THIS_MODULE,
- .suspend = tpm_pm_suspend,
- .resume = tpm_pm_resume,
+static int tpm_nsc_suspend(struct platform_device *dev, pm_message_t msg)
+{
+ return tpm_pm_suspend(&dev->dev, msg);
+}
+
+static int tpm_nsc_resume(struct platform_device *dev)
+{
+ return tpm_pm_resume(&dev->dev);
+}
+
+static struct platform_driver nsc_drv = {
+ .suspend = tpm_nsc_suspend,
+ .resume = tpm_nsc_resume,
+ .driver = {
+ .name = "tpm_nsc",
+ .owner = THIS_MODULE,
+ },
};
static int __init init_nsc(void)
@@ -297,7 +308,7 @@
return -ENODEV;
}
- err = driver_register(&nsc_drv);
+ err = platform_driver_register(&nsc_drv);
if (err)
return err;
@@ -308,17 +319,15 @@
/* enable the DPM module */
tpm_write_index(nscAddrBase, NSC_LDC_INDEX, 0x01);
- pdev = kzalloc(sizeof(struct platform_device), GFP_KERNEL);
+ pdev = platform_device_alloc("tpm_nscl0", -1);
if (!pdev) {
rc = -ENOMEM;
goto err_unreg_drv;
}
- pdev->name = "tpm_nscl0";
- pdev->id = -1;
pdev->num_resources = 0;
+ pdev->dev.driver = &nsc_drv.driver;
pdev->dev.release = tpm_nsc_remove;
- pdev->dev.driver = &nsc_drv;
if ((rc = platform_device_register(pdev)) < 0)
goto err_free_dev;
@@ -377,7 +386,7 @@
err_free_dev:
kfree(pdev);
err_unreg_drv:
- driver_unregister(&nsc_drv);
+ platform_driver_unregister(&nsc_drv);
return rc;
}
@@ -390,7 +399,7 @@
pdev = NULL;
}
- driver_unregister(&nsc_drv);
+ platform_driver_unregister(&nsc_drv);
}
module_init(init_nsc);
diff --git a/drivers/char/vt.c b/drivers/char/vt.c
index 8001421..7900bd6 100644
--- a/drivers/char/vt.c
+++ b/drivers/char/vt.c
@@ -969,8 +969,7 @@
* Takes the console sem and the called methods then take the tty
* termios_mutex and the tty ctrl_lock in that order.
*/
-
-int vt_resize(struct tty_struct *tty, struct winsize *ws)
+static int vt_resize(struct tty_struct *tty, struct winsize *ws)
{
struct vc_data *vc = tty->driver_data;
int ret;
diff --git a/drivers/cpuidle/governors/menu.c b/drivers/cpuidle/governors/menu.c
index 8d7cf3f..f1df59f 100644
--- a/drivers/cpuidle/governors/menu.c
+++ b/drivers/cpuidle/governors/menu.c
@@ -15,12 +15,14 @@
#include <linux/tick.h>
#define BREAK_FUZZ 4 /* 4 us */
+#define PRED_HISTORY_PCT 50
struct menu_device {
int last_state_idx;
unsigned int expected_us;
unsigned int predicted_us;
+ unsigned int current_predicted_us;
unsigned int last_measured_us;
unsigned int elapsed_us;
};
@@ -47,6 +49,12 @@
data->expected_us =
(u32) ktime_to_ns(tick_nohz_get_sleep_length()) / 1000;
+ /* Recalculate predicted_us based on prediction_history_pct */
+ data->predicted_us *= PRED_HISTORY_PCT;
+ data->predicted_us += (100 - PRED_HISTORY_PCT) *
+ data->current_predicted_us;
+ data->predicted_us /= 100;
+
/* find the deepest idle state that satisfies our constraints */
for (i = CPUIDLE_DRIVER_STATE_START + 1; i < dev->state_count; i++) {
struct cpuidle_state *s = &dev->states[i];
@@ -97,7 +105,7 @@
measured_us = -1;
/* Predict time until next break event */
- data->predicted_us = max(measured_us, data->last_measured_us);
+ data->current_predicted_us = max(measured_us, data->last_measured_us);
if (last_idle_us + BREAK_FUZZ <
data->expected_us - target->exit_latency) {
diff --git a/drivers/dca/dca-core.c b/drivers/dca/dca-core.c
index d883e1b..5543384 100644
--- a/drivers/dca/dca-core.c
+++ b/drivers/dca/dca-core.c
@@ -270,6 +270,6 @@
dca_sysfs_exit();
}
-subsys_initcall(dca_init);
+arch_initcall(dca_init);
module_exit(dca_exit);
diff --git a/drivers/dma/Kconfig b/drivers/dma/Kconfig
index 904e575..e34b064 100644
--- a/drivers/dma/Kconfig
+++ b/drivers/dma/Kconfig
@@ -33,7 +33,6 @@
config INTEL_IOP_ADMA
tristate "Intel IOP ADMA support"
depends on ARCH_IOP32X || ARCH_IOP33X || ARCH_IOP13XX
- select ASYNC_CORE
select DMA_ENGINE
help
Enable support for the Intel(R) IOP Series RAID engines.
@@ -59,7 +58,6 @@
config MV_XOR
bool "Marvell XOR engine support"
depends on PLAT_ORION
- select ASYNC_CORE
select DMA_ENGINE
---help---
Enable support for the Marvell XOR engine.
diff --git a/drivers/dma/dmaengine.c b/drivers/dma/dmaengine.c
index 6579965..403dbe7 100644
--- a/drivers/dma/dmaengine.c
+++ b/drivers/dma/dmaengine.c
@@ -31,32 +31,18 @@
*
* LOCKING:
*
- * The subsystem keeps two global lists, dma_device_list and dma_client_list.
- * Both of these are protected by a mutex, dma_list_mutex.
+ * The subsystem keeps a global list of dma_device structs it is protected by a
+ * mutex, dma_list_mutex.
+ *
+ * A subsystem can get access to a channel by calling dmaengine_get() followed
+ * by dma_find_channel(), or if it has need for an exclusive channel it can call
+ * dma_request_channel(). Once a channel is allocated a reference is taken
+ * against its corresponding driver to disable removal.
*
* Each device has a channels list, which runs unlocked but is never modified
* once the device is registered, it's just setup by the driver.
*
- * Each client is responsible for keeping track of the channels it uses. See
- * the definition of dma_event_callback in dmaengine.h.
- *
- * Each device has a kref, which is initialized to 1 when the device is
- * registered. A kref_get is done for each device registered. When the
- * device is released, the corresponding kref_put is done in the release
- * method. Every time one of the device's channels is allocated to a client,
- * a kref_get occurs. When the channel is freed, the corresponding kref_put
- * happens. The device's release function does a completion, so
- * unregister_device does a remove event, device_unregister, a kref_put
- * for the first reference, then waits on the completion for all other
- * references to finish.
- *
- * Each channel has an open-coded implementation of Rusty Russell's "bigref,"
- * with a kref and a per_cpu local_t. A dma_chan_get is called when a client
- * signals that it wants to use a channel, and dma_chan_put is called when
- * a channel is removed or a client using it is unregistered. A client can
- * take extra references per outstanding transaction, as is the case with
- * the NET DMA client. The release function does a kref_put on the device.
- * -ChrisL, DanW
+ * See Documentation/dmaengine.txt for more details
*/
#include <linux/init.h>
@@ -70,54 +56,85 @@
#include <linux/rcupdate.h>
#include <linux/mutex.h>
#include <linux/jiffies.h>
+#include <linux/rculist.h>
+#include <linux/idr.h>
static DEFINE_MUTEX(dma_list_mutex);
static LIST_HEAD(dma_device_list);
-static LIST_HEAD(dma_client_list);
+static long dmaengine_ref_count;
+static struct idr dma_idr;
/* --- sysfs implementation --- */
+/**
+ * dev_to_dma_chan - convert a device pointer to the its sysfs container object
+ * @dev - device node
+ *
+ * Must be called under dma_list_mutex
+ */
+static struct dma_chan *dev_to_dma_chan(struct device *dev)
+{
+ struct dma_chan_dev *chan_dev;
+
+ chan_dev = container_of(dev, typeof(*chan_dev), device);
+ return chan_dev->chan;
+}
+
static ssize_t show_memcpy_count(struct device *dev, struct device_attribute *attr, char *buf)
{
- struct dma_chan *chan = to_dma_chan(dev);
+ struct dma_chan *chan;
unsigned long count = 0;
int i;
+ int err;
- for_each_possible_cpu(i)
- count += per_cpu_ptr(chan->local, i)->memcpy_count;
+ mutex_lock(&dma_list_mutex);
+ chan = dev_to_dma_chan(dev);
+ if (chan) {
+ for_each_possible_cpu(i)
+ count += per_cpu_ptr(chan->local, i)->memcpy_count;
+ err = sprintf(buf, "%lu\n", count);
+ } else
+ err = -ENODEV;
+ mutex_unlock(&dma_list_mutex);
- return sprintf(buf, "%lu\n", count);
+ return err;
}
static ssize_t show_bytes_transferred(struct device *dev, struct device_attribute *attr,
char *buf)
{
- struct dma_chan *chan = to_dma_chan(dev);
+ struct dma_chan *chan;
unsigned long count = 0;
int i;
+ int err;
- for_each_possible_cpu(i)
- count += per_cpu_ptr(chan->local, i)->bytes_transferred;
+ mutex_lock(&dma_list_mutex);
+ chan = dev_to_dma_chan(dev);
+ if (chan) {
+ for_each_possible_cpu(i)
+ count += per_cpu_ptr(chan->local, i)->bytes_transferred;
+ err = sprintf(buf, "%lu\n", count);
+ } else
+ err = -ENODEV;
+ mutex_unlock(&dma_list_mutex);
- return sprintf(buf, "%lu\n", count);
+ return err;
}
static ssize_t show_in_use(struct device *dev, struct device_attribute *attr, char *buf)
{
- struct dma_chan *chan = to_dma_chan(dev);
- int in_use = 0;
+ struct dma_chan *chan;
+ int err;
- if (unlikely(chan->slow_ref) &&
- atomic_read(&chan->refcount.refcount) > 1)
- in_use = 1;
- else {
- if (local_read(&(per_cpu_ptr(chan->local,
- get_cpu())->refcount)) > 0)
- in_use = 1;
- put_cpu();
- }
+ mutex_lock(&dma_list_mutex);
+ chan = dev_to_dma_chan(dev);
+ if (chan)
+ err = sprintf(buf, "%d\n", chan->client_count);
+ else
+ err = -ENODEV;
+ mutex_unlock(&dma_list_mutex);
- return sprintf(buf, "%d\n", in_use);
+ return err;
}
static struct device_attribute dma_attrs[] = {
@@ -127,78 +144,112 @@
__ATTR_NULL
};
-static void dma_async_device_cleanup(struct kref *kref);
-
-static void dma_dev_release(struct device *dev)
+static void chan_dev_release(struct device *dev)
{
- struct dma_chan *chan = to_dma_chan(dev);
- kref_put(&chan->device->refcount, dma_async_device_cleanup);
+ struct dma_chan_dev *chan_dev;
+
+ chan_dev = container_of(dev, typeof(*chan_dev), device);
+ if (atomic_dec_and_test(chan_dev->idr_ref)) {
+ mutex_lock(&dma_list_mutex);
+ idr_remove(&dma_idr, chan_dev->dev_id);
+ mutex_unlock(&dma_list_mutex);
+ kfree(chan_dev->idr_ref);
+ }
+ kfree(chan_dev);
}
static struct class dma_devclass = {
.name = "dma",
.dev_attrs = dma_attrs,
- .dev_release = dma_dev_release,
+ .dev_release = chan_dev_release,
};
/* --- client and device registration --- */
-#define dma_chan_satisfies_mask(chan, mask) \
- __dma_chan_satisfies_mask((chan), &(mask))
+#define dma_device_satisfies_mask(device, mask) \
+ __dma_device_satisfies_mask((device), &(mask))
static int
-__dma_chan_satisfies_mask(struct dma_chan *chan, dma_cap_mask_t *want)
+__dma_device_satisfies_mask(struct dma_device *device, dma_cap_mask_t *want)
{
dma_cap_mask_t has;
- bitmap_and(has.bits, want->bits, chan->device->cap_mask.bits,
+ bitmap_and(has.bits, want->bits, device->cap_mask.bits,
DMA_TX_TYPE_END);
return bitmap_equal(want->bits, has.bits, DMA_TX_TYPE_END);
}
-/**
- * dma_client_chan_alloc - try to allocate channels to a client
- * @client: &dma_client
- *
- * Called with dma_list_mutex held.
- */
-static void dma_client_chan_alloc(struct dma_client *client)
+static struct module *dma_chan_to_owner(struct dma_chan *chan)
{
- struct dma_device *device;
- struct dma_chan *chan;
- int desc; /* allocated descriptor count */
- enum dma_state_client ack;
+ return chan->device->dev->driver->owner;
+}
- /* Find a channel */
- list_for_each_entry(device, &dma_device_list, global_node) {
- /* Does the client require a specific DMA controller? */
- if (client->slave && client->slave->dma_dev
- && client->slave->dma_dev != device->dev)
- continue;
+/**
+ * balance_ref_count - catch up the channel reference count
+ * @chan - channel to balance ->client_count versus dmaengine_ref_count
+ *
+ * balance_ref_count must be called under dma_list_mutex
+ */
+static void balance_ref_count(struct dma_chan *chan)
+{
+ struct module *owner = dma_chan_to_owner(chan);
- list_for_each_entry(chan, &device->channels, device_node) {
- if (!dma_chan_satisfies_mask(chan, client->cap_mask))
- continue;
-
- desc = chan->device->device_alloc_chan_resources(
- chan, client);
- if (desc >= 0) {
- ack = client->event_callback(client,
- chan,
- DMA_RESOURCE_AVAILABLE);
-
- /* we are done once this client rejects
- * an available resource
- */
- if (ack == DMA_ACK) {
- dma_chan_get(chan);
- chan->client_count++;
- } else if (ack == DMA_NAK)
- return;
- }
- }
+ while (chan->client_count < dmaengine_ref_count) {
+ __module_get(owner);
+ chan->client_count++;
}
}
+/**
+ * dma_chan_get - try to grab a dma channel's parent driver module
+ * @chan - channel to grab
+ *
+ * Must be called under dma_list_mutex
+ */
+static int dma_chan_get(struct dma_chan *chan)
+{
+ int err = -ENODEV;
+ struct module *owner = dma_chan_to_owner(chan);
+
+ if (chan->client_count) {
+ __module_get(owner);
+ err = 0;
+ } else if (try_module_get(owner))
+ err = 0;
+
+ if (err == 0)
+ chan->client_count++;
+
+ /* allocate upon first client reference */
+ if (chan->client_count == 1 && err == 0) {
+ int desc_cnt = chan->device->device_alloc_chan_resources(chan);
+
+ if (desc_cnt < 0) {
+ err = desc_cnt;
+ chan->client_count = 0;
+ module_put(owner);
+ } else if (!dma_has_cap(DMA_PRIVATE, chan->device->cap_mask))
+ balance_ref_count(chan);
+ }
+
+ return err;
+}
+
+/**
+ * dma_chan_put - drop a reference to a dma channel's parent driver module
+ * @chan - channel to release
+ *
+ * Must be called under dma_list_mutex
+ */
+static void dma_chan_put(struct dma_chan *chan)
+{
+ if (!chan->client_count)
+ return; /* this channel failed alloc_chan_resources */
+ chan->client_count--;
+ module_put(dma_chan_to_owner(chan));
+ if (chan->client_count == 0)
+ chan->device->device_free_chan_resources(chan);
+}
+
enum dma_status dma_sync_wait(struct dma_chan *chan, dma_cookie_t cookie)
{
enum dma_status status;
@@ -218,138 +269,342 @@
EXPORT_SYMBOL(dma_sync_wait);
/**
- * dma_chan_cleanup - release a DMA channel's resources
- * @kref: kernel reference structure that contains the DMA channel device
+ * dma_cap_mask_all - enable iteration over all operation types
*/
-void dma_chan_cleanup(struct kref *kref)
-{
- struct dma_chan *chan = container_of(kref, struct dma_chan, refcount);
- chan->device->device_free_chan_resources(chan);
- kref_put(&chan->device->refcount, dma_async_device_cleanup);
-}
-EXPORT_SYMBOL(dma_chan_cleanup);
-
-static void dma_chan_free_rcu(struct rcu_head *rcu)
-{
- struct dma_chan *chan = container_of(rcu, struct dma_chan, rcu);
- int bias = 0x7FFFFFFF;
- int i;
- for_each_possible_cpu(i)
- bias -= local_read(&per_cpu_ptr(chan->local, i)->refcount);
- atomic_sub(bias, &chan->refcount.refcount);
- kref_put(&chan->refcount, dma_chan_cleanup);
-}
-
-static void dma_chan_release(struct dma_chan *chan)
-{
- atomic_add(0x7FFFFFFF, &chan->refcount.refcount);
- chan->slow_ref = 1;
- call_rcu(&chan->rcu, dma_chan_free_rcu);
-}
+static dma_cap_mask_t dma_cap_mask_all;
/**
- * dma_chans_notify_available - broadcast available channels to the clients
+ * dma_chan_tbl_ent - tracks channel allocations per core/operation
+ * @chan - associated channel for this entry
*/
-static void dma_clients_notify_available(void)
-{
- struct dma_client *client;
-
- mutex_lock(&dma_list_mutex);
-
- list_for_each_entry(client, &dma_client_list, global_node)
- dma_client_chan_alloc(client);
-
- mutex_unlock(&dma_list_mutex);
-}
+struct dma_chan_tbl_ent {
+ struct dma_chan *chan;
+};
/**
- * dma_chans_notify_available - tell the clients that a channel is going away
- * @chan: channel on its way out
+ * channel_table - percpu lookup table for memory-to-memory offload providers
*/
-static void dma_clients_notify_removed(struct dma_chan *chan)
+static struct dma_chan_tbl_ent *channel_table[DMA_TX_TYPE_END];
+
+static int __init dma_channel_table_init(void)
{
- struct dma_client *client;
- enum dma_state_client ack;
+ enum dma_transaction_type cap;
+ int err = 0;
- mutex_lock(&dma_list_mutex);
+ bitmap_fill(dma_cap_mask_all.bits, DMA_TX_TYPE_END);
- list_for_each_entry(client, &dma_client_list, global_node) {
- ack = client->event_callback(client, chan,
- DMA_RESOURCE_REMOVED);
+ /* 'interrupt', 'private', and 'slave' are channel capabilities,
+ * but are not associated with an operation so they do not need
+ * an entry in the channel_table
+ */
+ clear_bit(DMA_INTERRUPT, dma_cap_mask_all.bits);
+ clear_bit(DMA_PRIVATE, dma_cap_mask_all.bits);
+ clear_bit(DMA_SLAVE, dma_cap_mask_all.bits);
- /* client was holding resources for this channel so
- * free it
- */
- if (ack == DMA_ACK) {
- dma_chan_put(chan);
- chan->client_count--;
+ for_each_dma_cap_mask(cap, dma_cap_mask_all) {
+ channel_table[cap] = alloc_percpu(struct dma_chan_tbl_ent);
+ if (!channel_table[cap]) {
+ err = -ENOMEM;
+ break;
}
}
- mutex_unlock(&dma_list_mutex);
+ if (err) {
+ pr_err("dmaengine: initialization failure\n");
+ for_each_dma_cap_mask(cap, dma_cap_mask_all)
+ if (channel_table[cap])
+ free_percpu(channel_table[cap]);
+ }
+
+ return err;
}
+arch_initcall(dma_channel_table_init);
/**
- * dma_async_client_register - register a &dma_client
- * @client: ptr to a client structure with valid 'event_callback' and 'cap_mask'
+ * dma_find_channel - find a channel to carry out the operation
+ * @tx_type: transaction type
*/
-void dma_async_client_register(struct dma_client *client)
+struct dma_chan *dma_find_channel(enum dma_transaction_type tx_type)
{
- /* validate client data */
- BUG_ON(dma_has_cap(DMA_SLAVE, client->cap_mask) &&
- !client->slave);
+ struct dma_chan *chan;
+ int cpu;
- mutex_lock(&dma_list_mutex);
- list_add_tail(&client->global_node, &dma_client_list);
- mutex_unlock(&dma_list_mutex);
+ WARN_ONCE(dmaengine_ref_count == 0,
+ "client called %s without a reference", __func__);
+
+ cpu = get_cpu();
+ chan = per_cpu_ptr(channel_table[tx_type], cpu)->chan;
+ put_cpu();
+
+ return chan;
}
-EXPORT_SYMBOL(dma_async_client_register);
+EXPORT_SYMBOL(dma_find_channel);
/**
- * dma_async_client_unregister - unregister a client and free the &dma_client
- * @client: &dma_client to free
- *
- * Force frees any allocated DMA channels, frees the &dma_client memory
+ * dma_issue_pending_all - flush all pending operations across all channels
*/
-void dma_async_client_unregister(struct dma_client *client)
+void dma_issue_pending_all(void)
{
struct dma_device *device;
struct dma_chan *chan;
- enum dma_state_client ack;
- if (!client)
- return;
+ WARN_ONCE(dmaengine_ref_count == 0,
+ "client called %s without a reference", __func__);
- mutex_lock(&dma_list_mutex);
- /* free all channels the client is holding */
- list_for_each_entry(device, &dma_device_list, global_node)
- list_for_each_entry(chan, &device->channels, device_node) {
- ack = client->event_callback(client, chan,
- DMA_RESOURCE_REMOVED);
-
- if (ack == DMA_ACK) {
- dma_chan_put(chan);
- chan->client_count--;
- }
- }
-
- list_del(&client->global_node);
- mutex_unlock(&dma_list_mutex);
+ rcu_read_lock();
+ list_for_each_entry_rcu(device, &dma_device_list, global_node) {
+ if (dma_has_cap(DMA_PRIVATE, device->cap_mask))
+ continue;
+ list_for_each_entry(chan, &device->channels, device_node)
+ if (chan->client_count)
+ device->device_issue_pending(chan);
+ }
+ rcu_read_unlock();
}
-EXPORT_SYMBOL(dma_async_client_unregister);
+EXPORT_SYMBOL(dma_issue_pending_all);
/**
- * dma_async_client_chan_request - send all available channels to the
- * client that satisfy the capability mask
- * @client - requester
+ * nth_chan - returns the nth channel of the given capability
+ * @cap: capability to match
+ * @n: nth channel desired
+ *
+ * Defaults to returning the channel with the desired capability and the
+ * lowest reference count when 'n' cannot be satisfied. Must be called
+ * under dma_list_mutex.
*/
-void dma_async_client_chan_request(struct dma_client *client)
+static struct dma_chan *nth_chan(enum dma_transaction_type cap, int n)
+{
+ struct dma_device *device;
+ struct dma_chan *chan;
+ struct dma_chan *ret = NULL;
+ struct dma_chan *min = NULL;
+
+ list_for_each_entry(device, &dma_device_list, global_node) {
+ if (!dma_has_cap(cap, device->cap_mask) ||
+ dma_has_cap(DMA_PRIVATE, device->cap_mask))
+ continue;
+ list_for_each_entry(chan, &device->channels, device_node) {
+ if (!chan->client_count)
+ continue;
+ if (!min)
+ min = chan;
+ else if (chan->table_count < min->table_count)
+ min = chan;
+
+ if (n-- == 0) {
+ ret = chan;
+ break; /* done */
+ }
+ }
+ if (ret)
+ break; /* done */
+ }
+
+ if (!ret)
+ ret = min;
+
+ if (ret)
+ ret->table_count++;
+
+ return ret;
+}
+
+/**
+ * dma_channel_rebalance - redistribute the available channels
+ *
+ * Optimize for cpu isolation (each cpu gets a dedicated channel for an
+ * operation type) in the SMP case, and operation isolation (avoid
+ * multi-tasking channels) in the non-SMP case. Must be called under
+ * dma_list_mutex.
+ */
+static void dma_channel_rebalance(void)
+{
+ struct dma_chan *chan;
+ struct dma_device *device;
+ int cpu;
+ int cap;
+ int n;
+
+ /* undo the last distribution */
+ for_each_dma_cap_mask(cap, dma_cap_mask_all)
+ for_each_possible_cpu(cpu)
+ per_cpu_ptr(channel_table[cap], cpu)->chan = NULL;
+
+ list_for_each_entry(device, &dma_device_list, global_node) {
+ if (dma_has_cap(DMA_PRIVATE, device->cap_mask))
+ continue;
+ list_for_each_entry(chan, &device->channels, device_node)
+ chan->table_count = 0;
+ }
+
+ /* don't populate the channel_table if no clients are available */
+ if (!dmaengine_ref_count)
+ return;
+
+ /* redistribute available channels */
+ n = 0;
+ for_each_dma_cap_mask(cap, dma_cap_mask_all)
+ for_each_online_cpu(cpu) {
+ if (num_possible_cpus() > 1)
+ chan = nth_chan(cap, n++);
+ else
+ chan = nth_chan(cap, -1);
+
+ per_cpu_ptr(channel_table[cap], cpu)->chan = chan;
+ }
+}
+
+static struct dma_chan *private_candidate(dma_cap_mask_t *mask, struct dma_device *dev,
+ dma_filter_fn fn, void *fn_param)
+{
+ struct dma_chan *chan;
+
+ if (!__dma_device_satisfies_mask(dev, mask)) {
+ pr_debug("%s: wrong capabilities\n", __func__);
+ return NULL;
+ }
+ /* devices with multiple channels need special handling as we need to
+ * ensure that all channels are either private or public.
+ */
+ if (dev->chancnt > 1 && !dma_has_cap(DMA_PRIVATE, dev->cap_mask))
+ list_for_each_entry(chan, &dev->channels, device_node) {
+ /* some channels are already publicly allocated */
+ if (chan->client_count)
+ return NULL;
+ }
+
+ list_for_each_entry(chan, &dev->channels, device_node) {
+ if (chan->client_count) {
+ pr_debug("%s: %s busy\n",
+ __func__, dma_chan_name(chan));
+ continue;
+ }
+ if (fn && !fn(chan, fn_param)) {
+ pr_debug("%s: %s filter said false\n",
+ __func__, dma_chan_name(chan));
+ continue;
+ }
+ return chan;
+ }
+
+ return NULL;
+}
+
+/**
+ * dma_request_channel - try to allocate an exclusive channel
+ * @mask: capabilities that the channel must satisfy
+ * @fn: optional callback to disposition available channels
+ * @fn_param: opaque parameter to pass to dma_filter_fn
+ */
+struct dma_chan *__dma_request_channel(dma_cap_mask_t *mask, dma_filter_fn fn, void *fn_param)
+{
+ struct dma_device *device, *_d;
+ struct dma_chan *chan = NULL;
+ int err;
+
+ /* Find a channel */
+ mutex_lock(&dma_list_mutex);
+ list_for_each_entry_safe(device, _d, &dma_device_list, global_node) {
+ chan = private_candidate(mask, device, fn, fn_param);
+ if (chan) {
+ /* Found a suitable channel, try to grab, prep, and
+ * return it. We first set DMA_PRIVATE to disable
+ * balance_ref_count as this channel will not be
+ * published in the general-purpose allocator
+ */
+ dma_cap_set(DMA_PRIVATE, device->cap_mask);
+ err = dma_chan_get(chan);
+
+ if (err == -ENODEV) {
+ pr_debug("%s: %s module removed\n", __func__,
+ dma_chan_name(chan));
+ list_del_rcu(&device->global_node);
+ } else if (err)
+ pr_err("dmaengine: failed to get %s: (%d)\n",
+ dma_chan_name(chan), err);
+ else
+ break;
+ chan = NULL;
+ }
+ }
+ mutex_unlock(&dma_list_mutex);
+
+ pr_debug("%s: %s (%s)\n", __func__, chan ? "success" : "fail",
+ chan ? dma_chan_name(chan) : NULL);
+
+ return chan;
+}
+EXPORT_SYMBOL_GPL(__dma_request_channel);
+
+void dma_release_channel(struct dma_chan *chan)
{
mutex_lock(&dma_list_mutex);
- dma_client_chan_alloc(client);
+ WARN_ONCE(chan->client_count != 1,
+ "chan reference count %d != 1\n", chan->client_count);
+ dma_chan_put(chan);
mutex_unlock(&dma_list_mutex);
}
-EXPORT_SYMBOL(dma_async_client_chan_request);
+EXPORT_SYMBOL_GPL(dma_release_channel);
+
+/**
+ * dmaengine_get - register interest in dma_channels
+ */
+void dmaengine_get(void)
+{
+ struct dma_device *device, *_d;
+ struct dma_chan *chan;
+ int err;
+
+ mutex_lock(&dma_list_mutex);
+ dmaengine_ref_count++;
+
+ /* try to grab channels */
+ list_for_each_entry_safe(device, _d, &dma_device_list, global_node) {
+ if (dma_has_cap(DMA_PRIVATE, device->cap_mask))
+ continue;
+ list_for_each_entry(chan, &device->channels, device_node) {
+ err = dma_chan_get(chan);
+ if (err == -ENODEV) {
+ /* module removed before we could use it */
+ list_del_rcu(&device->global_node);
+ break;
+ } else if (err)
+ pr_err("dmaengine: failed to get %s: (%d)\n",
+ dma_chan_name(chan), err);
+ }
+ }
+
+ /* if this is the first reference and there were channels
+ * waiting we need to rebalance to get those channels
+ * incorporated into the channel table
+ */
+ if (dmaengine_ref_count == 1)
+ dma_channel_rebalance();
+ mutex_unlock(&dma_list_mutex);
+}
+EXPORT_SYMBOL(dmaengine_get);
+
+/**
+ * dmaengine_put - let dma drivers be removed when ref_count == 0
+ */
+void dmaengine_put(void)
+{
+ struct dma_device *device;
+ struct dma_chan *chan;
+
+ mutex_lock(&dma_list_mutex);
+ dmaengine_ref_count--;
+ BUG_ON(dmaengine_ref_count < 0);
+ /* drop channel references */
+ list_for_each_entry(device, &dma_device_list, global_node) {
+ if (dma_has_cap(DMA_PRIVATE, device->cap_mask))
+ continue;
+ list_for_each_entry(chan, &device->channels, device_node)
+ dma_chan_put(chan);
+ }
+ mutex_unlock(&dma_list_mutex);
+}
+EXPORT_SYMBOL(dmaengine_put);
/**
* dma_async_device_register - registers DMA devices found
@@ -357,9 +612,9 @@
*/
int dma_async_device_register(struct dma_device *device)
{
- static int id;
int chancnt = 0, rc;
struct dma_chan* chan;
+ atomic_t *idr_ref;
if (!device)
return -ENODEV;
@@ -386,57 +641,83 @@
BUG_ON(!device->device_issue_pending);
BUG_ON(!device->dev);
- init_completion(&device->done);
- kref_init(&device->refcount);
-
+ idr_ref = kmalloc(sizeof(*idr_ref), GFP_KERNEL);
+ if (!idr_ref)
+ return -ENOMEM;
+ atomic_set(idr_ref, 0);
+ idr_retry:
+ if (!idr_pre_get(&dma_idr, GFP_KERNEL))
+ return -ENOMEM;
mutex_lock(&dma_list_mutex);
- device->dev_id = id++;
+ rc = idr_get_new(&dma_idr, NULL, &device->dev_id);
mutex_unlock(&dma_list_mutex);
+ if (rc == -EAGAIN)
+ goto idr_retry;
+ else if (rc != 0)
+ return rc;
/* represent channels in sysfs. Probably want devs too */
list_for_each_entry(chan, &device->channels, device_node) {
chan->local = alloc_percpu(typeof(*chan->local));
if (chan->local == NULL)
continue;
+ chan->dev = kzalloc(sizeof(*chan->dev), GFP_KERNEL);
+ if (chan->dev == NULL) {
+ free_percpu(chan->local);
+ continue;
+ }
chan->chan_id = chancnt++;
- chan->dev.class = &dma_devclass;
- chan->dev.parent = device->dev;
- dev_set_name(&chan->dev, "dma%dchan%d",
+ chan->dev->device.class = &dma_devclass;
+ chan->dev->device.parent = device->dev;
+ chan->dev->chan = chan;
+ chan->dev->idr_ref = idr_ref;
+ chan->dev->dev_id = device->dev_id;
+ atomic_inc(idr_ref);
+ dev_set_name(&chan->dev->device, "dma%dchan%d",
device->dev_id, chan->chan_id);
- rc = device_register(&chan->dev);
+ rc = device_register(&chan->dev->device);
if (rc) {
- chancnt--;
free_percpu(chan->local);
chan->local = NULL;
goto err_out;
}
-
- /* One for the channel, one of the class device */
- kref_get(&device->refcount);
- kref_get(&device->refcount);
- kref_init(&chan->refcount);
chan->client_count = 0;
- chan->slow_ref = 0;
- INIT_RCU_HEAD(&chan->rcu);
}
+ device->chancnt = chancnt;
mutex_lock(&dma_list_mutex);
- list_add_tail(&device->global_node, &dma_device_list);
+ /* take references on public channels */
+ if (dmaengine_ref_count && !dma_has_cap(DMA_PRIVATE, device->cap_mask))
+ list_for_each_entry(chan, &device->channels, device_node) {
+ /* if clients are already waiting for channels we need
+ * to take references on their behalf
+ */
+ if (dma_chan_get(chan) == -ENODEV) {
+ /* note we can only get here for the first
+ * channel as the remaining channels are
+ * guaranteed to get a reference
+ */
+ rc = -ENODEV;
+ mutex_unlock(&dma_list_mutex);
+ goto err_out;
+ }
+ }
+ list_add_tail_rcu(&device->global_node, &dma_device_list);
+ dma_channel_rebalance();
mutex_unlock(&dma_list_mutex);
- dma_clients_notify_available();
-
return 0;
err_out:
list_for_each_entry(chan, &device->channels, device_node) {
if (chan->local == NULL)
continue;
- kref_put(&device->refcount, dma_async_device_cleanup);
- device_unregister(&chan->dev);
- chancnt--;
+ mutex_lock(&dma_list_mutex);
+ chan->dev->chan = NULL;
+ mutex_unlock(&dma_list_mutex);
+ device_unregister(&chan->dev->device);
free_percpu(chan->local);
}
return rc;
@@ -444,37 +725,30 @@
EXPORT_SYMBOL(dma_async_device_register);
/**
- * dma_async_device_cleanup - function called when all references are released
- * @kref: kernel reference object
- */
-static void dma_async_device_cleanup(struct kref *kref)
-{
- struct dma_device *device;
-
- device = container_of(kref, struct dma_device, refcount);
- complete(&device->done);
-}
-
-/**
- * dma_async_device_unregister - unregisters DMA devices
+ * dma_async_device_unregister - unregister a DMA device
* @device: &dma_device
+ *
+ * This routine is called by dma driver exit routines, dmaengine holds module
+ * references to prevent it being called while channels are in use.
*/
void dma_async_device_unregister(struct dma_device *device)
{
struct dma_chan *chan;
mutex_lock(&dma_list_mutex);
- list_del(&device->global_node);
+ list_del_rcu(&device->global_node);
+ dma_channel_rebalance();
mutex_unlock(&dma_list_mutex);
list_for_each_entry(chan, &device->channels, device_node) {
- dma_clients_notify_removed(chan);
- device_unregister(&chan->dev);
- dma_chan_release(chan);
+ WARN_ONCE(chan->client_count,
+ "%s called while %d clients hold a reference\n",
+ __func__, chan->client_count);
+ mutex_lock(&dma_list_mutex);
+ chan->dev->chan = NULL;
+ mutex_unlock(&dma_list_mutex);
+ device_unregister(&chan->dev->device);
}
-
- kref_put(&device->refcount, dma_async_device_cleanup);
- wait_for_completion(&device->done);
}
EXPORT_SYMBOL(dma_async_device_unregister);
@@ -626,10 +900,96 @@
}
EXPORT_SYMBOL(dma_async_tx_descriptor_init);
+/* dma_wait_for_async_tx - spin wait for a transaction to complete
+ * @tx: in-flight transaction to wait on
+ *
+ * This routine assumes that tx was obtained from a call to async_memcpy,
+ * async_xor, async_memset, etc which ensures that tx is "in-flight" (prepped
+ * and submitted). Walking the parent chain is only meant to cover for DMA
+ * drivers that do not implement the DMA_INTERRUPT capability and may race with
+ * the driver's descriptor cleanup routine.
+ */
+enum dma_status
+dma_wait_for_async_tx(struct dma_async_tx_descriptor *tx)
+{
+ enum dma_status status;
+ struct dma_async_tx_descriptor *iter;
+ struct dma_async_tx_descriptor *parent;
+
+ if (!tx)
+ return DMA_SUCCESS;
+
+ WARN_ONCE(tx->parent, "%s: speculatively walking dependency chain for"
+ " %s\n", __func__, dma_chan_name(tx->chan));
+
+ /* poll through the dependency chain, return when tx is complete */
+ do {
+ iter = tx;
+
+ /* find the root of the unsubmitted dependency chain */
+ do {
+ parent = iter->parent;
+ if (!parent)
+ break;
+ else
+ iter = parent;
+ } while (parent);
+
+ /* there is a small window for ->parent == NULL and
+ * ->cookie == -EBUSY
+ */
+ while (iter->cookie == -EBUSY)
+ cpu_relax();
+
+ status = dma_sync_wait(iter->chan, iter->cookie);
+ } while (status == DMA_IN_PROGRESS || (iter != tx));
+
+ return status;
+}
+EXPORT_SYMBOL_GPL(dma_wait_for_async_tx);
+
+/* dma_run_dependencies - helper routine for dma drivers to process
+ * (start) dependent operations on their target channel
+ * @tx: transaction with dependencies
+ */
+void dma_run_dependencies(struct dma_async_tx_descriptor *tx)
+{
+ struct dma_async_tx_descriptor *dep = tx->next;
+ struct dma_async_tx_descriptor *dep_next;
+ struct dma_chan *chan;
+
+ if (!dep)
+ return;
+
+ chan = dep->chan;
+
+ /* keep submitting up until a channel switch is detected
+ * in that case we will be called again as a result of
+ * processing the interrupt from async_tx_channel_switch
+ */
+ for (; dep; dep = dep_next) {
+ spin_lock_bh(&dep->lock);
+ dep->parent = NULL;
+ dep_next = dep->next;
+ if (dep_next && dep_next->chan == chan)
+ dep->next = NULL; /* ->next will be submitted */
+ else
+ dep_next = NULL; /* submit current dep and terminate */
+ spin_unlock_bh(&dep->lock);
+
+ dep->tx_submit(dep);
+ }
+
+ chan->device->device_issue_pending(chan);
+}
+EXPORT_SYMBOL_GPL(dma_run_dependencies);
+
static int __init dma_bus_init(void)
{
+ idr_init(&dma_idr);
mutex_init(&dma_list_mutex);
return class_register(&dma_devclass);
}
-subsys_initcall(dma_bus_init);
+arch_initcall(dma_bus_init);
+
diff --git a/drivers/dma/dmatest.c b/drivers/dma/dmatest.c
index ed9636b..3603f1e 100644
--- a/drivers/dma/dmatest.c
+++ b/drivers/dma/dmatest.c
@@ -35,7 +35,7 @@
static unsigned int max_channels;
module_param(max_channels, uint, S_IRUGO);
-MODULE_PARM_DESC(nr_channels,
+MODULE_PARM_DESC(max_channels,
"Maximum number of channels to use (default: all)");
/*
@@ -71,7 +71,7 @@
/*
* These are protected by dma_list_mutex since they're only used by
- * the DMA client event callback
+ * the DMA filter function callback
*/
static LIST_HEAD(dmatest_channels);
static unsigned int nr_channels;
@@ -80,7 +80,7 @@
{
if (test_channel[0] == '\0')
return true;
- return strcmp(dev_name(&chan->dev), test_channel) == 0;
+ return strcmp(dma_chan_name(chan), test_channel) == 0;
}
static bool dmatest_match_device(struct dma_device *device)
@@ -215,7 +215,6 @@
smp_rmb();
chan = thread->chan;
- dma_chan_get(chan);
while (!kthread_should_stop()) {
total_tests++;
@@ -293,7 +292,6 @@
}
ret = 0;
- dma_chan_put(chan);
kfree(thread->dstbuf);
err_dstbuf:
kfree(thread->srcbuf);
@@ -319,21 +317,16 @@
kfree(dtc);
}
-static enum dma_state_client dmatest_add_channel(struct dma_chan *chan)
+static int dmatest_add_channel(struct dma_chan *chan)
{
struct dmatest_chan *dtc;
struct dmatest_thread *thread;
unsigned int i;
- /* Have we already been told about this channel? */
- list_for_each_entry(dtc, &dmatest_channels, node)
- if (dtc->chan == chan)
- return DMA_DUP;
-
dtc = kmalloc(sizeof(struct dmatest_chan), GFP_KERNEL);
if (!dtc) {
- pr_warning("dmatest: No memory for %s\n", dev_name(&chan->dev));
- return DMA_NAK;
+ pr_warning("dmatest: No memory for %s\n", dma_chan_name(chan));
+ return -ENOMEM;
}
dtc->chan = chan;
@@ -343,16 +336,16 @@
thread = kzalloc(sizeof(struct dmatest_thread), GFP_KERNEL);
if (!thread) {
pr_warning("dmatest: No memory for %s-test%u\n",
- dev_name(&chan->dev), i);
+ dma_chan_name(chan), i);
break;
}
thread->chan = dtc->chan;
smp_wmb();
thread->task = kthread_run(dmatest_func, thread, "%s-test%u",
- dev_name(&chan->dev), i);
+ dma_chan_name(chan), i);
if (IS_ERR(thread->task)) {
pr_warning("dmatest: Failed to run thread %s-test%u\n",
- dev_name(&chan->dev), i);
+ dma_chan_name(chan), i);
kfree(thread);
break;
}
@@ -362,86 +355,62 @@
list_add_tail(&thread->node, &dtc->threads);
}
- pr_info("dmatest: Started %u threads using %s\n", i, dev_name(&chan->dev));
+ pr_info("dmatest: Started %u threads using %s\n", i, dma_chan_name(chan));
list_add_tail(&dtc->node, &dmatest_channels);
nr_channels++;
- return DMA_ACK;
+ return 0;
}
-static enum dma_state_client dmatest_remove_channel(struct dma_chan *chan)
+static bool filter(struct dma_chan *chan, void *param)
{
- struct dmatest_chan *dtc, *_dtc;
-
- list_for_each_entry_safe(dtc, _dtc, &dmatest_channels, node) {
- if (dtc->chan == chan) {
- list_del(&dtc->node);
- dmatest_cleanup_channel(dtc);
- pr_debug("dmatest: lost channel %s\n",
- dev_name(&chan->dev));
- return DMA_ACK;
- }
- }
-
- return DMA_DUP;
+ if (!dmatest_match_channel(chan) || !dmatest_match_device(chan->device))
+ return false;
+ else
+ return true;
}
-/*
- * Start testing threads as new channels are assigned to us, and kill
- * them when the channels go away.
- *
- * When we unregister the client, all channels are removed so this
- * will also take care of cleaning things up when the module is
- * unloaded.
- */
-static enum dma_state_client
-dmatest_event(struct dma_client *client, struct dma_chan *chan,
- enum dma_state state)
-{
- enum dma_state_client ack = DMA_NAK;
-
- switch (state) {
- case DMA_RESOURCE_AVAILABLE:
- if (!dmatest_match_channel(chan)
- || !dmatest_match_device(chan->device))
- ack = DMA_DUP;
- else if (max_channels && nr_channels >= max_channels)
- ack = DMA_NAK;
- else
- ack = dmatest_add_channel(chan);
- break;
-
- case DMA_RESOURCE_REMOVED:
- ack = dmatest_remove_channel(chan);
- break;
-
- default:
- pr_info("dmatest: Unhandled event %u (%s)\n",
- state, dev_name(&chan->dev));
- break;
- }
-
- return ack;
-}
-
-static struct dma_client dmatest_client = {
- .event_callback = dmatest_event,
-};
-
static int __init dmatest_init(void)
{
- dma_cap_set(DMA_MEMCPY, dmatest_client.cap_mask);
- dma_async_client_register(&dmatest_client);
- dma_async_client_chan_request(&dmatest_client);
+ dma_cap_mask_t mask;
+ struct dma_chan *chan;
+ int err = 0;
- return 0;
+ dma_cap_zero(mask);
+ dma_cap_set(DMA_MEMCPY, mask);
+ for (;;) {
+ chan = dma_request_channel(mask, filter, NULL);
+ if (chan) {
+ err = dmatest_add_channel(chan);
+ if (err == 0)
+ continue;
+ else {
+ dma_release_channel(chan);
+ break; /* add_channel failed, punt */
+ }
+ } else
+ break; /* no more channels available */
+ if (max_channels && nr_channels >= max_channels)
+ break; /* we have all we need */
+ }
+
+ return err;
}
-module_init(dmatest_init);
+/* when compiled-in wait for drivers to load first */
+late_initcall(dmatest_init);
static void __exit dmatest_exit(void)
{
- dma_async_client_unregister(&dmatest_client);
+ struct dmatest_chan *dtc, *_dtc;
+
+ list_for_each_entry_safe(dtc, _dtc, &dmatest_channels, node) {
+ list_del(&dtc->node);
+ dmatest_cleanup_channel(dtc);
+ pr_debug("dmatest: dropped channel %s\n",
+ dma_chan_name(dtc->chan));
+ dma_release_channel(dtc->chan);
+ }
}
module_exit(dmatest_exit);
diff --git a/drivers/dma/dw_dmac.c b/drivers/dma/dw_dmac.c
index 0778d99..6b702cc 100644
--- a/drivers/dma/dw_dmac.c
+++ b/drivers/dma/dw_dmac.c
@@ -70,6 +70,15 @@
* the controller, though.
*/
+static struct device *chan2dev(struct dma_chan *chan)
+{
+ return &chan->dev->device;
+}
+static struct device *chan2parent(struct dma_chan *chan)
+{
+ return chan->dev->device.parent;
+}
+
static struct dw_desc *dwc_first_active(struct dw_dma_chan *dwc)
{
return list_entry(dwc->active_list.next, struct dw_desc, desc_node);
@@ -93,12 +102,12 @@
ret = desc;
break;
}
- dev_dbg(&dwc->chan.dev, "desc %p not ACKed\n", desc);
+ dev_dbg(chan2dev(&dwc->chan), "desc %p not ACKed\n", desc);
i++;
}
spin_unlock_bh(&dwc->lock);
- dev_vdbg(&dwc->chan.dev, "scanned %u descriptors on freelist\n", i);
+ dev_vdbg(chan2dev(&dwc->chan), "scanned %u descriptors on freelist\n", i);
return ret;
}
@@ -108,10 +117,10 @@
struct dw_desc *child;
list_for_each_entry(child, &desc->txd.tx_list, desc_node)
- dma_sync_single_for_cpu(dwc->chan.dev.parent,
+ dma_sync_single_for_cpu(chan2parent(&dwc->chan),
child->txd.phys, sizeof(child->lli),
DMA_TO_DEVICE);
- dma_sync_single_for_cpu(dwc->chan.dev.parent,
+ dma_sync_single_for_cpu(chan2parent(&dwc->chan),
desc->txd.phys, sizeof(desc->lli),
DMA_TO_DEVICE);
}
@@ -129,11 +138,11 @@
spin_lock_bh(&dwc->lock);
list_for_each_entry(child, &desc->txd.tx_list, desc_node)
- dev_vdbg(&dwc->chan.dev,
+ dev_vdbg(chan2dev(&dwc->chan),
"moving child desc %p to freelist\n",
child);
list_splice_init(&desc->txd.tx_list, &dwc->free_list);
- dev_vdbg(&dwc->chan.dev, "moving desc %p to freelist\n", desc);
+ dev_vdbg(chan2dev(&dwc->chan), "moving desc %p to freelist\n", desc);
list_add(&desc->desc_node, &dwc->free_list);
spin_unlock_bh(&dwc->lock);
}
@@ -163,9 +172,9 @@
/* ASSERT: channel is idle */
if (dma_readl(dw, CH_EN) & dwc->mask) {
- dev_err(&dwc->chan.dev,
+ dev_err(chan2dev(&dwc->chan),
"BUG: Attempted to start non-idle channel\n");
- dev_err(&dwc->chan.dev,
+ dev_err(chan2dev(&dwc->chan),
" SAR: 0x%x DAR: 0x%x LLP: 0x%x CTL: 0x%x:%08x\n",
channel_readl(dwc, SAR),
channel_readl(dwc, DAR),
@@ -193,7 +202,7 @@
void *param;
struct dma_async_tx_descriptor *txd = &desc->txd;
- dev_vdbg(&dwc->chan.dev, "descriptor %u complete\n", txd->cookie);
+ dev_vdbg(chan2dev(&dwc->chan), "descriptor %u complete\n", txd->cookie);
dwc->completed = txd->cookie;
callback = txd->callback;
@@ -208,11 +217,11 @@
* mapped before they were submitted...
*/
if (!(txd->flags & DMA_COMPL_SKIP_DEST_UNMAP))
- dma_unmap_page(dwc->chan.dev.parent, desc->lli.dar, desc->len,
- DMA_FROM_DEVICE);
+ dma_unmap_page(chan2parent(&dwc->chan), desc->lli.dar,
+ desc->len, DMA_FROM_DEVICE);
if (!(txd->flags & DMA_COMPL_SKIP_SRC_UNMAP))
- dma_unmap_page(dwc->chan.dev.parent, desc->lli.sar, desc->len,
- DMA_TO_DEVICE);
+ dma_unmap_page(chan2parent(&dwc->chan), desc->lli.sar,
+ desc->len, DMA_TO_DEVICE);
/*
* The API requires that no submissions are done from a
@@ -228,7 +237,7 @@
LIST_HEAD(list);
if (dma_readl(dw, CH_EN) & dwc->mask) {
- dev_err(&dwc->chan.dev,
+ dev_err(chan2dev(&dwc->chan),
"BUG: XFER bit set, but channel not idle!\n");
/* Try to continue after resetting the channel... */
@@ -273,7 +282,7 @@
return;
}
- dev_vdbg(&dwc->chan.dev, "scan_descriptors: llp=0x%x\n", llp);
+ dev_vdbg(chan2dev(&dwc->chan), "scan_descriptors: llp=0x%x\n", llp);
list_for_each_entry_safe(desc, _desc, &dwc->active_list, desc_node) {
if (desc->lli.llp == llp)
@@ -292,7 +301,7 @@
dwc_descriptor_complete(dwc, desc);
}
- dev_err(&dwc->chan.dev,
+ dev_err(chan2dev(&dwc->chan),
"BUG: All descriptors done, but channel not idle!\n");
/* Try to continue after resetting the channel... */
@@ -308,7 +317,7 @@
static void dwc_dump_lli(struct dw_dma_chan *dwc, struct dw_lli *lli)
{
- dev_printk(KERN_CRIT, &dwc->chan.dev,
+ dev_printk(KERN_CRIT, chan2dev(&dwc->chan),
" desc: s0x%x d0x%x l0x%x c0x%x:%x\n",
lli->sar, lli->dar, lli->llp,
lli->ctlhi, lli->ctllo);
@@ -342,9 +351,9 @@
* controller flagged an error instead of scribbling over
* random memory locations.
*/
- dev_printk(KERN_CRIT, &dwc->chan.dev,
+ dev_printk(KERN_CRIT, chan2dev(&dwc->chan),
"Bad descriptor submitted for DMA!\n");
- dev_printk(KERN_CRIT, &dwc->chan.dev,
+ dev_printk(KERN_CRIT, chan2dev(&dwc->chan),
" cookie: %d\n", bad_desc->txd.cookie);
dwc_dump_lli(dwc, &bad_desc->lli);
list_for_each_entry(child, &bad_desc->txd.tx_list, desc_node)
@@ -442,12 +451,12 @@
* for DMA. But this is hard to do in a race-free manner.
*/
if (list_empty(&dwc->active_list)) {
- dev_vdbg(&tx->chan->dev, "tx_submit: started %u\n",
+ dev_vdbg(chan2dev(tx->chan), "tx_submit: started %u\n",
desc->txd.cookie);
dwc_dostart(dwc, desc);
list_add_tail(&desc->desc_node, &dwc->active_list);
} else {
- dev_vdbg(&tx->chan->dev, "tx_submit: queued %u\n",
+ dev_vdbg(chan2dev(tx->chan), "tx_submit: queued %u\n",
desc->txd.cookie);
list_add_tail(&desc->desc_node, &dwc->queue);
@@ -472,11 +481,11 @@
unsigned int dst_width;
u32 ctllo;
- dev_vdbg(&chan->dev, "prep_dma_memcpy d0x%x s0x%x l0x%zx f0x%lx\n",
+ dev_vdbg(chan2dev(chan), "prep_dma_memcpy d0x%x s0x%x l0x%zx f0x%lx\n",
dest, src, len, flags);
if (unlikely(!len)) {
- dev_dbg(&chan->dev, "prep_dma_memcpy: length is zero!\n");
+ dev_dbg(chan2dev(chan), "prep_dma_memcpy: length is zero!\n");
return NULL;
}
@@ -516,7 +525,7 @@
first = desc;
} else {
prev->lli.llp = desc->txd.phys;
- dma_sync_single_for_device(chan->dev.parent,
+ dma_sync_single_for_device(chan2parent(chan),
prev->txd.phys, sizeof(prev->lli),
DMA_TO_DEVICE);
list_add_tail(&desc->desc_node,
@@ -531,7 +540,7 @@
prev->lli.ctllo |= DWC_CTLL_INT_EN;
prev->lli.llp = 0;
- dma_sync_single_for_device(chan->dev.parent,
+ dma_sync_single_for_device(chan2parent(chan),
prev->txd.phys, sizeof(prev->lli),
DMA_TO_DEVICE);
@@ -562,15 +571,15 @@
struct scatterlist *sg;
size_t total_len = 0;
- dev_vdbg(&chan->dev, "prep_dma_slave\n");
+ dev_vdbg(chan2dev(chan), "prep_dma_slave\n");
if (unlikely(!dws || !sg_len))
return NULL;
- reg_width = dws->slave.reg_width;
+ reg_width = dws->reg_width;
prev = first = NULL;
- sg_len = dma_map_sg(chan->dev.parent, sgl, sg_len, direction);
+ sg_len = dma_map_sg(chan2parent(chan), sgl, sg_len, direction);
switch (direction) {
case DMA_TO_DEVICE:
@@ -579,7 +588,7 @@
| DWC_CTLL_DST_FIX
| DWC_CTLL_SRC_INC
| DWC_CTLL_FC_M2P);
- reg = dws->slave.tx_reg;
+ reg = dws->tx_reg;
for_each_sg(sgl, sg, sg_len, i) {
struct dw_desc *desc;
u32 len;
@@ -587,7 +596,7 @@
desc = dwc_desc_get(dwc);
if (!desc) {
- dev_err(&chan->dev,
+ dev_err(chan2dev(chan),
"not enough descriptors available\n");
goto err_desc_get;
}
@@ -607,7 +616,7 @@
first = desc;
} else {
prev->lli.llp = desc->txd.phys;
- dma_sync_single_for_device(chan->dev.parent,
+ dma_sync_single_for_device(chan2parent(chan),
prev->txd.phys,
sizeof(prev->lli),
DMA_TO_DEVICE);
@@ -625,7 +634,7 @@
| DWC_CTLL_SRC_FIX
| DWC_CTLL_FC_P2M);
- reg = dws->slave.rx_reg;
+ reg = dws->rx_reg;
for_each_sg(sgl, sg, sg_len, i) {
struct dw_desc *desc;
u32 len;
@@ -633,7 +642,7 @@
desc = dwc_desc_get(dwc);
if (!desc) {
- dev_err(&chan->dev,
+ dev_err(chan2dev(chan),
"not enough descriptors available\n");
goto err_desc_get;
}
@@ -653,7 +662,7 @@
first = desc;
} else {
prev->lli.llp = desc->txd.phys;
- dma_sync_single_for_device(chan->dev.parent,
+ dma_sync_single_for_device(chan2parent(chan),
prev->txd.phys,
sizeof(prev->lli),
DMA_TO_DEVICE);
@@ -673,7 +682,7 @@
prev->lli.ctllo |= DWC_CTLL_INT_EN;
prev->lli.llp = 0;
- dma_sync_single_for_device(chan->dev.parent,
+ dma_sync_single_for_device(chan2parent(chan),
prev->txd.phys, sizeof(prev->lli),
DMA_TO_DEVICE);
@@ -758,29 +767,21 @@
spin_unlock_bh(&dwc->lock);
}
-static int dwc_alloc_chan_resources(struct dma_chan *chan,
- struct dma_client *client)
+static int dwc_alloc_chan_resources(struct dma_chan *chan)
{
struct dw_dma_chan *dwc = to_dw_dma_chan(chan);
struct dw_dma *dw = to_dw_dma(chan->device);
struct dw_desc *desc;
- struct dma_slave *slave;
struct dw_dma_slave *dws;
int i;
u32 cfghi;
u32 cfglo;
- dev_vdbg(&chan->dev, "alloc_chan_resources\n");
-
- /* Channels doing slave DMA can only handle one client. */
- if (dwc->dws || client->slave) {
- if (chan->client_count)
- return -EBUSY;
- }
+ dev_vdbg(chan2dev(chan), "alloc_chan_resources\n");
/* ASSERT: channel is idle */
if (dma_readl(dw, CH_EN) & dwc->mask) {
- dev_dbg(&chan->dev, "DMA channel not idle?\n");
+ dev_dbg(chan2dev(chan), "DMA channel not idle?\n");
return -EIO;
}
@@ -789,23 +790,17 @@
cfghi = DWC_CFGH_FIFO_MODE;
cfglo = 0;
- slave = client->slave;
- if (slave) {
+ dws = dwc->dws;
+ if (dws) {
/*
* We need controller-specific data to set up slave
* transfers.
*/
- BUG_ON(!slave->dma_dev || slave->dma_dev != dw->dma.dev);
+ BUG_ON(!dws->dma_dev || dws->dma_dev != dw->dma.dev);
- dws = container_of(slave, struct dw_dma_slave, slave);
-
- dwc->dws = dws;
cfghi = dws->cfg_hi;
cfglo = dws->cfg_lo;
- } else {
- dwc->dws = NULL;
}
-
channel_writel(dwc, CFG_LO, cfglo);
channel_writel(dwc, CFG_HI, cfghi);
@@ -822,7 +817,7 @@
desc = kzalloc(sizeof(struct dw_desc), GFP_KERNEL);
if (!desc) {
- dev_info(&chan->dev,
+ dev_info(chan2dev(chan),
"only allocated %d descriptors\n", i);
spin_lock_bh(&dwc->lock);
break;
@@ -832,7 +827,7 @@
desc->txd.tx_submit = dwc_tx_submit;
desc->txd.flags = DMA_CTRL_ACK;
INIT_LIST_HEAD(&desc->txd.tx_list);
- desc->txd.phys = dma_map_single(chan->dev.parent, &desc->lli,
+ desc->txd.phys = dma_map_single(chan2parent(chan), &desc->lli,
sizeof(desc->lli), DMA_TO_DEVICE);
dwc_desc_put(dwc, desc);
@@ -847,7 +842,7 @@
spin_unlock_bh(&dwc->lock);
- dev_dbg(&chan->dev,
+ dev_dbg(chan2dev(chan),
"alloc_chan_resources allocated %d descriptors\n", i);
return i;
@@ -860,7 +855,7 @@
struct dw_desc *desc, *_desc;
LIST_HEAD(list);
- dev_dbg(&chan->dev, "free_chan_resources (descs allocated=%u)\n",
+ dev_dbg(chan2dev(chan), "free_chan_resources (descs allocated=%u)\n",
dwc->descs_allocated);
/* ASSERT: channel is idle */
@@ -881,13 +876,13 @@
spin_unlock_bh(&dwc->lock);
list_for_each_entry_safe(desc, _desc, &list, desc_node) {
- dev_vdbg(&chan->dev, " freeing descriptor %p\n", desc);
- dma_unmap_single(chan->dev.parent, desc->txd.phys,
+ dev_vdbg(chan2dev(chan), " freeing descriptor %p\n", desc);
+ dma_unmap_single(chan2parent(chan), desc->txd.phys,
sizeof(desc->lli), DMA_TO_DEVICE);
kfree(desc);
}
- dev_vdbg(&chan->dev, "free_chan_resources done\n");
+ dev_vdbg(chan2dev(chan), "free_chan_resources done\n");
}
/*----------------------------------------------------------------------*/
diff --git a/drivers/dma/fsldma.c b/drivers/dma/fsldma.c
index 0b95dcc..ca70a21 100644
--- a/drivers/dma/fsldma.c
+++ b/drivers/dma/fsldma.c
@@ -366,8 +366,7 @@
*
* Return - The number of descriptors allocated.
*/
-static int fsl_dma_alloc_chan_resources(struct dma_chan *chan,
- struct dma_client *client)
+static int fsl_dma_alloc_chan_resources(struct dma_chan *chan)
{
struct fsl_dma_chan *fsl_chan = to_fsl_chan(chan);
@@ -823,7 +822,7 @@
*/
WARN_ON(fdev->feature != new_fsl_chan->feature);
- new_fsl_chan->dev = &new_fsl_chan->common.dev;
+ new_fsl_chan->dev = &new_fsl_chan->common.dev->device;
new_fsl_chan->reg_base = ioremap(new_fsl_chan->reg.start,
new_fsl_chan->reg.end - new_fsl_chan->reg.start + 1);
diff --git a/drivers/dma/ioat.c b/drivers/dma/ioat.c
index 9b16a3a..4105d65 100644
--- a/drivers/dma/ioat.c
+++ b/drivers/dma/ioat.c
@@ -75,60 +75,10 @@
module_param(ioat_dca_enabled, int, 0644);
MODULE_PARM_DESC(ioat_dca_enabled, "control support of dca service (default: 1)");
-static int ioat_setup_functionality(struct pci_dev *pdev, void __iomem *iobase)
-{
- struct ioat_device *device = pci_get_drvdata(pdev);
- u8 version;
- int err = 0;
-
- version = readb(iobase + IOAT_VER_OFFSET);
- switch (version) {
- case IOAT_VER_1_2:
- device->dma = ioat_dma_probe(pdev, iobase);
- if (device->dma && ioat_dca_enabled)
- device->dca = ioat_dca_init(pdev, iobase);
- break;
- case IOAT_VER_2_0:
- device->dma = ioat_dma_probe(pdev, iobase);
- if (device->dma && ioat_dca_enabled)
- device->dca = ioat2_dca_init(pdev, iobase);
- break;
- case IOAT_VER_3_0:
- device->dma = ioat_dma_probe(pdev, iobase);
- if (device->dma && ioat_dca_enabled)
- device->dca = ioat3_dca_init(pdev, iobase);
- break;
- default:
- err = -ENODEV;
- break;
- }
- if (!device->dma)
- err = -ENODEV;
- return err;
-}
-
-static void ioat_shutdown_functionality(struct pci_dev *pdev)
-{
- struct ioat_device *device = pci_get_drvdata(pdev);
-
- dev_err(&pdev->dev, "Removing dma and dca services\n");
- if (device->dca) {
- unregister_dca_provider(device->dca);
- free_dca_provider(device->dca);
- device->dca = NULL;
- }
-
- if (device->dma) {
- ioat_dma_remove(device->dma);
- device->dma = NULL;
- }
-}
-
static struct pci_driver ioat_pci_driver = {
.name = "ioatdma",
.id_table = ioat_pci_tbl,
.probe = ioat_probe,
- .shutdown = ioat_shutdown_functionality,
.remove = __devexit_p(ioat_remove),
};
@@ -179,7 +129,29 @@
pci_set_master(pdev);
- err = ioat_setup_functionality(pdev, iobase);
+ switch (readb(iobase + IOAT_VER_OFFSET)) {
+ case IOAT_VER_1_2:
+ device->dma = ioat_dma_probe(pdev, iobase);
+ if (device->dma && ioat_dca_enabled)
+ device->dca = ioat_dca_init(pdev, iobase);
+ break;
+ case IOAT_VER_2_0:
+ device->dma = ioat_dma_probe(pdev, iobase);
+ if (device->dma && ioat_dca_enabled)
+ device->dca = ioat2_dca_init(pdev, iobase);
+ break;
+ case IOAT_VER_3_0:
+ device->dma = ioat_dma_probe(pdev, iobase);
+ if (device->dma && ioat_dca_enabled)
+ device->dca = ioat3_dca_init(pdev, iobase);
+ break;
+ default:
+ err = -ENODEV;
+ break;
+ }
+ if (!device->dma)
+ err = -ENODEV;
+
if (err)
goto err_version;
@@ -198,17 +170,21 @@
return err;
}
-/*
- * It is unsafe to remove this module: if removed while a requested
- * dma is outstanding, esp. from tcp, it is possible to hang while
- * waiting for something that will never finish. However, if you're
- * feeling lucky, this usually works just fine.
- */
static void __devexit ioat_remove(struct pci_dev *pdev)
{
struct ioat_device *device = pci_get_drvdata(pdev);
- ioat_shutdown_functionality(pdev);
+ dev_err(&pdev->dev, "Removing dma and dca services\n");
+ if (device->dca) {
+ unregister_dca_provider(device->dca);
+ free_dca_provider(device->dca);
+ device->dca = NULL;
+ }
+
+ if (device->dma) {
+ ioat_dma_remove(device->dma);
+ device->dma = NULL;
+ }
kfree(device);
}
diff --git a/drivers/dma/ioat_dma.c b/drivers/dma/ioat_dma.c
index 6607fdd..b3759c4 100644
--- a/drivers/dma/ioat_dma.c
+++ b/drivers/dma/ioat_dma.c
@@ -734,8 +734,7 @@
* ioat_dma_alloc_chan_resources - returns the number of allocated descriptors
* @chan: the channel to be filled out
*/
-static int ioat_dma_alloc_chan_resources(struct dma_chan *chan,
- struct dma_client *client)
+static int ioat_dma_alloc_chan_resources(struct dma_chan *chan)
{
struct ioat_dma_chan *ioat_chan = to_ioat_chan(chan);
struct ioat_desc_sw *desc;
@@ -1341,12 +1340,11 @@
*/
#define IOAT_TEST_SIZE 2000
-DECLARE_COMPLETION(test_completion);
static void ioat_dma_test_callback(void *dma_async_param)
{
- printk(KERN_ERR "ioatdma: ioat_dma_test_callback(%p)\n",
- dma_async_param);
- complete(&test_completion);
+ struct completion *cmp = dma_async_param;
+
+ complete(cmp);
}
/**
@@ -1363,6 +1361,7 @@
dma_addr_t dma_dest, dma_src;
dma_cookie_t cookie;
int err = 0;
+ struct completion cmp;
src = kzalloc(sizeof(u8) * IOAT_TEST_SIZE, GFP_KERNEL);
if (!src)
@@ -1381,7 +1380,7 @@
dma_chan = container_of(device->common.channels.next,
struct dma_chan,
device_node);
- if (device->common.device_alloc_chan_resources(dma_chan, NULL) < 1) {
+ if (device->common.device_alloc_chan_resources(dma_chan) < 1) {
dev_err(&device->pdev->dev,
"selftest cannot allocate chan resource\n");
err = -ENODEV;
@@ -1402,8 +1401,9 @@
}
async_tx_ack(tx);
+ init_completion(&cmp);
tx->callback = ioat_dma_test_callback;
- tx->callback_param = (void *)0x8086;
+ tx->callback_param = &cmp;
cookie = tx->tx_submit(tx);
if (cookie < 0) {
dev_err(&device->pdev->dev,
@@ -1413,7 +1413,7 @@
}
device->common.device_issue_pending(dma_chan);
- wait_for_completion_timeout(&test_completion, msecs_to_jiffies(3000));
+ wait_for_completion_timeout(&cmp, msecs_to_jiffies(3000));
if (device->common.device_is_tx_complete(dma_chan, cookie, NULL, NULL)
!= DMA_SUCCESS) {
diff --git a/drivers/dma/iop-adma.c b/drivers/dma/iop-adma.c
index 6be3172..ea5440d 100644
--- a/drivers/dma/iop-adma.c
+++ b/drivers/dma/iop-adma.c
@@ -24,7 +24,6 @@
#include <linux/init.h>
#include <linux/module.h>
-#include <linux/async_tx.h>
#include <linux/delay.h>
#include <linux/dma-mapping.h>
#include <linux/spinlock.h>
@@ -116,7 +115,7 @@
}
/* run dependent operations */
- async_tx_run_dependencies(&desc->async_tx);
+ dma_run_dependencies(&desc->async_tx);
return cookie;
}
@@ -270,8 +269,6 @@
break;
}
- BUG_ON(!seen_current);
-
if (cookie > 0) {
iop_chan->completed_cookie = cookie;
pr_debug("\tcompleted cookie %d\n", cookie);
@@ -471,8 +468,7 @@
* greater than 2x the number slots needed to satisfy a device->max_xor
* request.
* */
-static int iop_adma_alloc_chan_resources(struct dma_chan *chan,
- struct dma_client *client)
+static int iop_adma_alloc_chan_resources(struct dma_chan *chan)
{
char *hw_desc;
int idx;
@@ -866,7 +862,7 @@
dma_chan = container_of(device->common.channels.next,
struct dma_chan,
device_node);
- if (iop_adma_alloc_chan_resources(dma_chan, NULL) < 1) {
+ if (iop_adma_alloc_chan_resources(dma_chan) < 1) {
err = -ENODEV;
goto out;
}
@@ -964,7 +960,7 @@
dma_chan = container_of(device->common.channels.next,
struct dma_chan,
device_node);
- if (iop_adma_alloc_chan_resources(dma_chan, NULL) < 1) {
+ if (iop_adma_alloc_chan_resources(dma_chan) < 1) {
err = -ENODEV;
goto out;
}
@@ -1115,26 +1111,13 @@
struct iop_adma_device *device = platform_get_drvdata(dev);
struct dma_chan *chan, *_chan;
struct iop_adma_chan *iop_chan;
- int i;
struct iop_adma_platform_data *plat_data = dev->dev.platform_data;
dma_async_device_unregister(&device->common);
- for (i = 0; i < 3; i++) {
- unsigned int irq;
- irq = platform_get_irq(dev, i);
- free_irq(irq, device);
- }
-
dma_free_coherent(&dev->dev, plat_data->pool_size,
device->dma_desc_pool_virt, device->dma_desc_pool);
- do {
- struct resource *res;
- res = platform_get_resource(dev, IORESOURCE_MEM, 0);
- release_mem_region(res->start, res->end - res->start);
- } while (0);
-
list_for_each_entry_safe(chan, _chan, &device->common.channels,
device_node) {
iop_chan = to_iop_adma_chan(chan);
@@ -1255,7 +1238,6 @@
spin_lock_init(&iop_chan->lock);
INIT_LIST_HEAD(&iop_chan->chain);
INIT_LIST_HEAD(&iop_chan->all_slots);
- INIT_RCU_HEAD(&iop_chan->common.rcu);
iop_chan->common.device = dma_dev;
list_add_tail(&iop_chan->common.device_node, &dma_dev->channels);
@@ -1431,16 +1413,12 @@
return platform_driver_register(&iop_adma_driver);
}
-/* it's currently unsafe to unload this module */
-#if 0
static void __exit iop_adma_exit (void)
{
platform_driver_unregister(&iop_adma_driver);
return;
}
module_exit(iop_adma_exit);
-#endif
-
module_init(iop_adma_init);
MODULE_AUTHOR("Intel Corporation");
diff --git a/drivers/dma/mv_xor.c b/drivers/dma/mv_xor.c
index bcda174..d35cbd1 100644
--- a/drivers/dma/mv_xor.c
+++ b/drivers/dma/mv_xor.c
@@ -18,7 +18,6 @@
#include <linux/init.h>
#include <linux/module.h>
-#include <linux/async_tx.h>
#include <linux/delay.h>
#include <linux/dma-mapping.h>
#include <linux/spinlock.h>
@@ -340,7 +339,7 @@
}
/* run dependent operations */
- async_tx_run_dependencies(&desc->async_tx);
+ dma_run_dependencies(&desc->async_tx);
return cookie;
}
@@ -607,8 +606,7 @@
}
/* returns the number of allocated descriptors */
-static int mv_xor_alloc_chan_resources(struct dma_chan *chan,
- struct dma_client *client)
+static int mv_xor_alloc_chan_resources(struct dma_chan *chan)
{
char *hw_desc;
int idx;
@@ -958,7 +956,7 @@
dma_chan = container_of(device->common.channels.next,
struct dma_chan,
device_node);
- if (mv_xor_alloc_chan_resources(dma_chan, NULL) < 1) {
+ if (mv_xor_alloc_chan_resources(dma_chan) < 1) {
err = -ENODEV;
goto out;
}
@@ -1053,7 +1051,7 @@
dma_chan = container_of(device->common.channels.next,
struct dma_chan,
device_node);
- if (mv_xor_alloc_chan_resources(dma_chan, NULL) < 1) {
+ if (mv_xor_alloc_chan_resources(dma_chan) < 1) {
err = -ENODEV;
goto out;
}
@@ -1221,7 +1219,6 @@
INIT_LIST_HEAD(&mv_chan->chain);
INIT_LIST_HEAD(&mv_chan->completed_slots);
INIT_LIST_HEAD(&mv_chan->all_slots);
- INIT_RCU_HEAD(&mv_chan->common.rcu);
mv_chan->common.device = dma_dev;
list_add_tail(&mv_chan->common.device_node, &dma_dev->channels);
diff --git a/drivers/firmware/dcdbas.c b/drivers/firmware/dcdbas.c
index 50a071f..777fba4 100644
--- a/drivers/firmware/dcdbas.c
+++ b/drivers/firmware/dcdbas.c
@@ -238,11 +238,11 @@
}
/**
- * smi_request: generate SMI request
+ * dcdbas_smi_request: generate SMI request
*
* Called with smi_data_lock.
*/
-static int smi_request(struct smi_cmd *smi_cmd)
+int dcdbas_smi_request(struct smi_cmd *smi_cmd)
{
cpumask_t old_mask;
int ret = 0;
@@ -309,14 +309,14 @@
switch (val) {
case 2:
/* Raw SMI */
- ret = smi_request(smi_cmd);
+ ret = dcdbas_smi_request(smi_cmd);
if (!ret)
ret = count;
break;
case 1:
/* Calling Interface SMI */
smi_cmd->ebx = (u32) virt_to_phys(smi_cmd->command_buffer);
- ret = smi_request(smi_cmd);
+ ret = dcdbas_smi_request(smi_cmd);
if (!ret)
ret = count;
break;
@@ -333,6 +333,7 @@
mutex_unlock(&smi_data_lock);
return ret;
}
+EXPORT_SYMBOL(dcdbas_smi_request);
/**
* host_control_smi: generate host control SMI
diff --git a/drivers/firmware/dcdbas.h b/drivers/firmware/dcdbas.h
index 87bc341..ca3cb0a 100644
--- a/drivers/firmware/dcdbas.h
+++ b/drivers/firmware/dcdbas.h
@@ -101,5 +101,7 @@
} __attribute__ ((packed)) parameters;
} __attribute__ ((packed));
+int dcdbas_smi_request(struct smi_cmd *smi_cmd);
+
#endif /* _DCDBAS_H_ */
diff --git a/drivers/firmware/memmap.c b/drivers/firmware/memmap.c
index 3bf8ee1..261b9aa 100644
--- a/drivers/firmware/memmap.c
+++ b/drivers/firmware/memmap.c
@@ -56,9 +56,9 @@
ssize_t (*show)(struct firmware_map_entry *entry, char *buf);
};
-struct memmap_attribute memmap_start_attr = __ATTR_RO(start);
-struct memmap_attribute memmap_end_attr = __ATTR_RO(end);
-struct memmap_attribute memmap_type_attr = __ATTR_RO(type);
+static struct memmap_attribute memmap_start_attr = __ATTR_RO(start);
+static struct memmap_attribute memmap_end_attr = __ATTR_RO(end);
+static struct memmap_attribute memmap_type_attr = __ATTR_RO(type);
/*
* These are default attributes that are added for every memmap entry.
diff --git a/drivers/ide/ide-acpi.c b/drivers/ide/ide-acpi.c
index 2f9e941..d8f295b 100644
--- a/drivers/ide/ide-acpi.c
+++ b/drivers/ide/ide-acpi.c
@@ -18,12 +18,6 @@
#include <linux/dmi.h>
#include <acpi/acpi_bus.h>
-#include <acpi/acnames.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acparser.h>
-#include <acpi/acexcep.h>
-#include <acpi/acmacros.h>
-#include <acpi/actypes.h>
#define REGS_PER_GTF 7
struct taskfile_array {
diff --git a/drivers/ieee1394/eth1394.c b/drivers/ieee1394/eth1394.c
index a074bfd..1a919df 100644
--- a/drivers/ieee1394/eth1394.c
+++ b/drivers/ieee1394/eth1394.c
@@ -245,12 +245,6 @@
return 0;
}
-/* Return statistics to the caller */
-static struct net_device_stats *ether1394_stats(struct net_device *dev)
-{
- return &(((struct eth1394_priv *)netdev_priv(dev))->stats);
-}
-
/* FIXME: What to do if we timeout? I think a host reset is probably in order,
* so that's what we do. Should we increment the stat counters too? */
static void ether1394_tx_timeout(struct net_device *dev)
@@ -516,16 +510,19 @@
.parse = ether1394_header_parse,
};
+static const struct net_device_ops ether1394_netdev_ops = {
+ .ndo_open = ether1394_open,
+ .ndo_stop = ether1394_stop,
+ .ndo_start_xmit = ether1394_tx,
+ .ndo_tx_timeout = ether1394_tx_timeout,
+ .ndo_change_mtu = ether1394_change_mtu,
+};
+
static void ether1394_init_dev(struct net_device *dev)
{
- dev->open = ether1394_open;
- dev->stop = ether1394_stop;
- dev->hard_start_xmit = ether1394_tx;
- dev->get_stats = ether1394_stats;
- dev->tx_timeout = ether1394_tx_timeout;
- dev->change_mtu = ether1394_change_mtu;
dev->header_ops = ðer1394_header_ops;
+ dev->netdev_ops = ðer1394_netdev_ops;
SET_ETHTOOL_OPS(dev, ðtool_ops);
@@ -1075,7 +1072,7 @@
HPSB_PRINT(KERN_ERR, "ether1394 rx: sender nodeid "
"lookup failure: " NODE_BUS_FMT,
NODE_BUS_ARGS(priv->host, srcid));
- priv->stats.rx_dropped++;
+ dev->stats.rx_dropped++;
return -1;
}
ud = node->ud;
@@ -1098,7 +1095,7 @@
skb = dev_alloc_skb(len + dev->hard_header_len + 15);
if (unlikely(!skb)) {
ETH1394_PRINT_G(KERN_ERR, "Out of memory\n");
- priv->stats.rx_dropped++;
+ dev->stats.rx_dropped++;
return -1;
}
skb_reserve(skb, (dev->hard_header_len + 15) & ~15);
@@ -1217,15 +1214,15 @@
spin_lock_irqsave(&priv->lock, flags);
if (!skb->protocol) {
- priv->stats.rx_errors++;
- priv->stats.rx_dropped++;
+ dev->stats.rx_errors++;
+ dev->stats.rx_dropped++;
dev_kfree_skb_any(skb);
} else if (netif_rx(skb) == NET_RX_DROP) {
- priv->stats.rx_errors++;
- priv->stats.rx_dropped++;
+ dev->stats.rx_errors++;
+ dev->stats.rx_dropped++;
} else {
- priv->stats.rx_packets++;
- priv->stats.rx_bytes += skb->len;
+ dev->stats.rx_packets++;
+ dev->stats.rx_bytes += skb->len;
}
spin_unlock_irqrestore(&priv->lock, flags);
@@ -1234,8 +1231,6 @@
if (netif_queue_stopped(dev))
netif_wake_queue(dev);
- dev->last_rx = jiffies;
-
return 0;
}
@@ -1509,17 +1504,18 @@
static void ether1394_dg_complete(struct packet_task *ptask, int fail)
{
struct sk_buff *skb = ptask->skb;
- struct eth1394_priv *priv = netdev_priv(skb->dev);
+ struct net_device *dev = skb->dev;
+ struct eth1394_priv *priv = netdev_priv(dev);
unsigned long flags;
/* Statistics */
spin_lock_irqsave(&priv->lock, flags);
if (fail) {
- priv->stats.tx_dropped++;
- priv->stats.tx_errors++;
+ dev->stats.tx_dropped++;
+ dev->stats.tx_errors++;
} else {
- priv->stats.tx_bytes += skb->len;
- priv->stats.tx_packets++;
+ dev->stats.tx_bytes += skb->len;
+ dev->stats.tx_packets++;
}
spin_unlock_irqrestore(&priv->lock, flags);
@@ -1696,8 +1692,8 @@
dev_kfree_skb(skb);
spin_lock_irqsave(&priv->lock, flags);
- priv->stats.tx_dropped++;
- priv->stats.tx_errors++;
+ dev->stats.tx_dropped++;
+ dev->stats.tx_errors++;
spin_unlock_irqrestore(&priv->lock, flags);
/*
diff --git a/drivers/ieee1394/eth1394.h b/drivers/ieee1394/eth1394.h
index e1b5ea8..d53bac4 100644
--- a/drivers/ieee1394/eth1394.h
+++ b/drivers/ieee1394/eth1394.h
@@ -54,7 +54,6 @@
/* Private structure for our ethernet driver */
struct eth1394_priv {
- struct net_device_stats stats; /* Device stats */
struct hpsb_host *host; /* The card for this dev */
u16 bc_maxpayload; /* Max broadcast payload */
u8 bc_sspd; /* Max broadcast speed */
diff --git a/drivers/infiniband/hw/nes/nes_cm.c b/drivers/infiniband/hw/nes/nes_cm.c
index a812db2..6ba57e9 100644
--- a/drivers/infiniband/hw/nes/nes_cm.c
+++ b/drivers/infiniband/hw/nes/nes_cm.c
@@ -2705,7 +2705,7 @@
sizeof(struct ietf_mpa_frame));
- /* notify OF layer that accept event was successfull */
+ /* notify OF layer that accept event was successful */
cm_id->add_ref(cm_id);
cm_event.event = IW_CM_EVENT_ESTABLISHED;
diff --git a/drivers/input/mouse/pxa930_trkball.c b/drivers/input/mouse/pxa930_trkball.c
index a0f45c4..d297acc 100644
--- a/drivers/input/mouse/pxa930_trkball.c
+++ b/drivers/input/mouse/pxa930_trkball.c
@@ -186,7 +186,7 @@
error = request_irq(irq, pxa930_trkball_interrupt, IRQF_DISABLED,
pdev->name, trkball);
if (error) {
- dev_err(&pdev->dev, "failed to request irq: %d\n", ret);
+ dev_err(&pdev->dev, "failed to request irq: %d\n", error);
goto failed_free_io;
}
@@ -227,7 +227,7 @@
iounmap(trkball->mmio_base);
failed:
kfree(trkball);
- return ret;
+ return error;
}
static int __devexit pxa930_trkball_remove(struct platform_device *pdev)
diff --git a/drivers/isdn/hardware/eicon/debuglib.h b/drivers/isdn/hardware/eicon/debuglib.h
index 016410c..8ea5877 100644
--- a/drivers/isdn/hardware/eicon/debuglib.h
+++ b/drivers/isdn/hardware/eicon/debuglib.h
@@ -235,7 +235,7 @@
typedef void ( * DbgEv) (unsigned short, unsigned long, va_list) ;
typedef void ( * DbgIrq) (unsigned short, int, char *, va_list) ;
typedef struct _DbgHandle_
-{ char Registered ; /* driver successfull registered */
+{ char Registered ; /* driver successfully registered */
#define DBG_HANDLE_REG_NEW 0x01 /* this (new) structure */
#define DBG_HANDLE_REG_OLD 0x7f /* old structure (see below) */
char Version; /* version of this structure */
diff --git a/drivers/isdn/hardware/eicon/os_4bri.c b/drivers/isdn/hardware/eicon/os_4bri.c
index 7b4ec3f..c964b8d 100644
--- a/drivers/isdn/hardware/eicon/os_4bri.c
+++ b/drivers/isdn/hardware/eicon/os_4bri.c
@@ -997,7 +997,7 @@
diva_xdi_display_adapter_features(IoAdapter->ANum);
for (i = 0; i < IoAdapter->tasks; i++) {
- DBG_LOG(("A(%d) %s adapter successfull started",
+ DBG_LOG(("A(%d) %s adapter successfully started",
IoAdapter->QuadroList->QuadroAdapter[i]->ANum,
(IoAdapter->tasks == 1) ? "BRI 2.0" : "4BRI"))
diva_xdi_didd_register_adapter(IoAdapter->QuadroList->QuadroAdapter[i]->ANum);
diff --git a/drivers/isdn/hardware/eicon/os_bri.c b/drivers/isdn/hardware/eicon/os_bri.c
index f31bba5..08f0199 100644
--- a/drivers/isdn/hardware/eicon/os_bri.c
+++ b/drivers/isdn/hardware/eicon/os_bri.c
@@ -736,7 +736,7 @@
IoAdapter->Properties.Features = (word) features;
diva_xdi_display_adapter_features(IoAdapter->ANum);
- DBG_LOG(("A(%d) BRI adapter successfull started", IoAdapter->ANum))
+ DBG_LOG(("A(%d) BRI adapter successfully started", IoAdapter->ANum))
/*
Register with DIDD
*/
diff --git a/drivers/isdn/hardware/eicon/os_pri.c b/drivers/isdn/hardware/eicon/os_pri.c
index 9033565..5d65405 100644
--- a/drivers/isdn/hardware/eicon/os_pri.c
+++ b/drivers/isdn/hardware/eicon/os_pri.c
@@ -513,7 +513,7 @@
diva_xdi_display_adapter_features(IoAdapter->ANum);
- DBG_LOG(("A(%d) PRI adapter successfull started", IoAdapter->ANum))
+ DBG_LOG(("A(%d) PRI adapter successfully started", IoAdapter->ANum))
/*
Register with DIDD
*/
diff --git a/drivers/isdn/hysdn/hysdn_net.c b/drivers/isdn/hysdn/hysdn_net.c
index 7ee5bd9..579974c 100644
--- a/drivers/isdn/hysdn/hysdn_net.c
+++ b/drivers/isdn/hysdn/hysdn_net.c
@@ -38,16 +38,12 @@
/* inside the definition. */
/****************************************************************************/
struct net_local {
- struct net_device netdev; /* the network device */
- struct net_device_stats stats;
- /* additional vars may be added here */
- char dev_name[9]; /* our own device name */
-
/* Tx control lock. This protects the transmit buffer ring
* state along with the "tx full" state of the driver. This
* means all netif_queue flow control actions are protected
* by this lock as well.
*/
+ struct net_device *dev;
spinlock_t lock;
struct sk_buff *skbs[MAX_SKB_BUFFERS]; /* pointers to tx-skbs */
int in_idx, out_idx; /* indexes to buffer ring */
@@ -55,15 +51,6 @@
}; /* net_local */
-/*****************************************************/
-/* Get the current statistics for this card. */
-/* This may be called with the card open or closed ! */
-/*****************************************************/
-static struct net_device_stats *
-net_get_stats(struct net_device *dev)
-{
- return (&((struct net_local *) dev)->stats);
-} /* net_device_stats */
/*********************************************************************/
/* Open/initialize the board. This is called (in the current kernel) */
@@ -182,8 +169,8 @@
if (!lp->sk_count)
return; /* error condition */
- lp->stats.tx_packets++;
- lp->stats.tx_bytes += lp->skbs[lp->out_idx]->len;
+ lp->dev->stats.tx_packets++;
+ lp->dev->stats.tx_bytes += lp->skbs[lp->out_idx]->len;
dev_kfree_skb(lp->skbs[lp->out_idx++]); /* free skb */
if (lp->out_idx >= MAX_SKB_BUFFERS)
@@ -200,29 +187,30 @@
hysdn_rx_netpkt(hysdn_card * card, unsigned char *buf, unsigned short len)
{
struct net_local *lp = card->netif;
+ struct net_device *dev = lp->dev;
struct sk_buff *skb;
if (!lp)
return; /* non existing device */
- lp->stats.rx_bytes += len;
+ dev->stats.rx_bytes += len;
skb = dev_alloc_skb(len);
if (skb == NULL) {
printk(KERN_NOTICE "%s: Memory squeeze, dropping packet.\n",
- lp->netdev.name);
- lp->stats.rx_dropped++;
+ dev->name);
+ dev->stats.rx_dropped++;
return;
}
/* copy the data */
memcpy(skb_put(skb, len), buf, len);
/* determine the used protocol */
- skb->protocol = eth_type_trans(skb, &lp->netdev);
+ skb->protocol = eth_type_trans(skb, dev);
+
+ dev->stats.rx_packets++; /* adjust packet count */
netif_rx(skb);
- lp->stats.rx_packets++; /* adjust packet count */
-
} /* hysdn_rx_netpkt */
/*****************************************************/
@@ -242,24 +230,15 @@
return (lp->skbs[lp->out_idx]); /* next packet to send */
} /* hysdn_tx_netget */
+static const struct net_device_ops hysdn_netdev_ops = {
+ .ndo_open = net_open,
+ .ndo_stop = net_close,
+ .ndo_start_xmit = net_send_packet,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
-/*******************************************/
-/* init function called by register device */
-/*******************************************/
-static int
-net_init(struct net_device *dev)
-{
- /* setup the function table */
- dev->open = net_open;
- dev->stop = net_close;
- dev->hard_start_xmit = net_send_packet;
- dev->get_stats = net_get_stats;
-
- /* Fill in the fields of the device structure with ethernet values. */
- ether_setup(dev);
-
- return (0); /* success */
-} /* net_init */
/*****************************************************************************/
/* hysdn_net_create creates a new net device for the given card. If a device */
@@ -271,28 +250,34 @@
{
struct net_device *dev;
int i;
+ struct net_local *lp;
+
if(!card) {
printk(KERN_WARNING "No card-pt in hysdn_net_create!\n");
return (-ENOMEM);
}
hysdn_net_release(card); /* release an existing net device */
- if ((dev = kzalloc(sizeof(struct net_local), GFP_KERNEL)) == NULL) {
+
+ dev = alloc_etherdev(sizeof(struct net_local));
+ if (!dev) {
printk(KERN_WARNING "HYSDN: unable to allocate mem\n");
return (-ENOMEM);
}
+ lp = netdev_priv(dev);
+ lp->dev = dev;
+
+ dev->netdev_ops = &hysdn_netdev_ops;
spin_lock_init(&((struct net_local *) dev)->lock);
/* initialise necessary or informing fields */
dev->base_addr = card->iobase; /* IO address */
dev->irq = card->irq; /* irq */
- dev->init = net_init; /* the init function of the device */
- if(dev->name) {
- strcpy(dev->name, ((struct net_local *) dev)->dev_name);
- }
+
+ dev->netdev_ops = &hysdn_netdev_ops;
if ((i = register_netdev(dev))) {
printk(KERN_WARNING "HYSDN: unable to create network device\n");
- kfree(dev);
+ free_netdev(dev);
return (i);
}
dev->ml_priv = card; /* remember pointer to own data structure */
@@ -316,7 +301,7 @@
return (0); /* non existing */
card->netif = NULL; /* clear out pointer */
- dev->stop(dev); /* close the device */
+ net_close(dev);
flush_tx_buffers((struct net_local *) dev); /* empty buffers */
diff --git a/drivers/isdn/i4l/isdn_net.c b/drivers/isdn/i4l/isdn_net.c
index 023ea11..7c5f970 100644
--- a/drivers/isdn/i4l/isdn_net.c
+++ b/drivers/isdn/i4l/isdn_net.c
@@ -1485,6 +1485,24 @@
return (rc);
}
+
+static int isdn_net_ioctl(struct net_device *dev,
+ struct ifreq *ifr, int cmd)
+{
+ isdn_net_local *lp = (isdn_net_local *) netdev_priv(dev);
+
+ switch (lp->p_encap) {
+#ifdef CONFIG_ISDN_PPP
+ case ISDN_NET_ENCAP_SYNCPPP:
+ return isdn_ppp_dev_ioctl(dev, ifr, cmd);
+#endif
+ case ISDN_NET_ENCAP_CISCOHDLCK:
+ return isdn_ciscohdlck_dev_ioctl(dev, ifr, cmd);
+ default:
+ return -EINVAL;
+ }
+}
+
/* called via cisco_timer.function */
static void
isdn_net_ciscohdlck_slarp_send_keepalive(unsigned long data)
@@ -1998,23 +2016,6 @@
ushort max_hlhdr_len = 0;
int drvidx;
- ether_setup(ndev);
- ndev->header_ops = NULL;
-
- /* Setup the generic properties */
- ndev->mtu = 1500;
- ndev->flags = IFF_NOARP|IFF_POINTOPOINT;
- ndev->type = ARPHRD_ETHER;
- ndev->addr_len = ETH_ALEN;
- ndev->validate_addr = NULL;
-
- /* for clients with MPPP maybe higher values better */
- ndev->tx_queue_len = 30;
-
- /* The ISDN-specific entries in the device structure. */
- ndev->open = &isdn_net_open;
- ndev->hard_start_xmit = &isdn_net_start_xmit;
-
/*
* up till binding we ask the protocol layer to reserve as much
* as we might need for HL layer
@@ -2026,9 +2027,6 @@
max_hlhdr_len = dev->drv[drvidx]->interface->hl_hdrlen;
ndev->hard_header_len = ETH_HLEN + max_hlhdr_len;
- ndev->stop = &isdn_net_close;
- ndev->get_stats = &isdn_net_get_stats;
- ndev->do_ioctl = NULL;
return 0;
}
@@ -2508,6 +2506,19 @@
return (isdn_net_force_dial_lp(p->local));
}
+/* The ISDN-specific entries in the device structure. */
+static const struct net_device_ops isdn_netdev_ops = {
+ .ndo_init = isdn_net_init,
+ .ndo_open = isdn_net_open,
+ .ndo_stop = isdn_net_close,
+ .ndo_do_ioctl = isdn_net_ioctl,
+
+ .ndo_validate_addr = NULL,
+ .ndo_start_xmit = isdn_net_start_xmit,
+ .ndo_get_stats = isdn_net_get_stats,
+ .ndo_tx_timeout = isdn_net_tx_timeout,
+};
+
/*
* Helper for alloc_netdev()
*/
@@ -2515,7 +2526,20 @@
{
isdn_net_local *lp = netdev_priv(dev);
+ ether_setup(dev);
+
dev->flags = IFF_NOARP | IFF_POINTOPOINT;
+ /* Setup the generic properties */
+ dev->mtu = 1500;
+ dev->flags = IFF_NOARP|IFF_POINTOPOINT;
+ dev->type = ARPHRD_ETHER;
+ dev->addr_len = ETH_ALEN;
+ dev->header_ops = NULL;
+ dev->netdev_ops = &isdn_netdev_ops;
+
+ /* for clients with MPPP maybe higher values better */
+ dev->tx_queue_len = 30;
+
lp->p_encap = ISDN_NET_ENCAP_RAWIP;
lp->magic = ISDN_NET_MAGIC;
lp->last = lp;
@@ -2570,7 +2594,7 @@
return NULL;
}
netdev->local = netdev_priv(netdev->dev);
- netdev->dev->init = isdn_net_init;
+
if (master) {
/* Device shall be a slave */
struct net_device *p = MASTER_TO_SLAVE(master);
@@ -2588,7 +2612,6 @@
/*
* Watchdog timer (currently) for master only.
*/
- netdev->dev->tx_timeout = isdn_net_tx_timeout;
netdev->dev->watchdog_timeo = ISDN_NET_TX_TIMEOUT;
if (register_netdev(netdev->dev) != 0) {
printk(KERN_WARNING "isdn_net: Could not register net-device\n");
@@ -2704,7 +2727,6 @@
#else
p->dev->type = ARPHRD_PPP; /* change ARP type */
p->dev->addr_len = 0;
- p->dev->do_ioctl = isdn_ppp_dev_ioctl;
#endif
break;
case ISDN_NET_ENCAP_X25IFACE:
@@ -2718,7 +2740,6 @@
#endif
break;
case ISDN_NET_ENCAP_CISCOHDLCK:
- p->dev->do_ioctl = isdn_ciscohdlck_dev_ioctl;
break;
default:
if( cfg->p_encap >= 0 &&
diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c
index ab7c8e4..7199437 100644
--- a/drivers/md/bitmap.c
+++ b/drivers/md/bitmap.c
@@ -215,7 +215,6 @@
/* choose a good rdev and read the page from there */
mdk_rdev_t *rdev;
- struct list_head *tmp;
sector_t target;
if (!page)
@@ -223,7 +222,7 @@
if (!page)
return ERR_PTR(-ENOMEM);
- rdev_for_each(rdev, tmp, mddev) {
+ list_for_each_entry(rdev, &mddev->disks, same_set) {
if (! test_bit(In_sync, &rdev->flags)
|| test_bit(Faulty, &rdev->flags))
continue;
@@ -964,9 +963,11 @@
*/
page = bitmap->sb_page;
offset = sizeof(bitmap_super_t);
- read_sb_page(bitmap->mddev, bitmap->offset,
- page,
- index, count);
+ if (!file)
+ read_sb_page(bitmap->mddev,
+ bitmap->offset,
+ page,
+ index, count);
} else if (file) {
page = read_page(file, index, bitmap, count);
offset = 0;
diff --git a/drivers/md/faulty.c b/drivers/md/faulty.c
index f26c1f9..86d9adf 100644
--- a/drivers/md/faulty.c
+++ b/drivers/md/faulty.c
@@ -283,7 +283,6 @@
static int run(mddev_t *mddev)
{
mdk_rdev_t *rdev;
- struct list_head *tmp;
int i;
conf_t *conf = kmalloc(sizeof(*conf), GFP_KERNEL);
@@ -296,7 +295,7 @@
}
conf->nfaults = 0;
- rdev_for_each(rdev, tmp, mddev)
+ list_for_each_entry(rdev, &mddev->disks, same_set)
conf->rdev = rdev;
mddev->array_sectors = mddev->size * 2;
diff --git a/drivers/md/linear.c b/drivers/md/linear.c
index 3b90c5c..1e3aea9 100644
--- a/drivers/md/linear.c
+++ b/drivers/md/linear.c
@@ -105,7 +105,6 @@
int i, nb_zone, cnt;
sector_t min_sectors;
sector_t curr_sector;
- struct list_head *tmp;
conf = kzalloc (sizeof (*conf) + raid_disks*sizeof(dev_info_t),
GFP_KERNEL);
@@ -115,7 +114,7 @@
cnt = 0;
conf->array_sectors = 0;
- rdev_for_each(rdev, tmp, mddev) {
+ list_for_each_entry(rdev, &mddev->disks, same_set) {
int j = rdev->raid_disk;
dev_info_t *disk = conf->disks + j;
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 1b1d326..41e2509 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -214,20 +214,33 @@
return mddev;
}
+static void mddev_delayed_delete(struct work_struct *ws)
+{
+ mddev_t *mddev = container_of(ws, mddev_t, del_work);
+ kobject_del(&mddev->kobj);
+ kobject_put(&mddev->kobj);
+}
+
static void mddev_put(mddev_t *mddev)
{
if (!atomic_dec_and_lock(&mddev->active, &all_mddevs_lock))
return;
- if (!mddev->raid_disks && list_empty(&mddev->disks)) {
+ if (!mddev->raid_disks && list_empty(&mddev->disks) &&
+ !mddev->hold_active) {
list_del(&mddev->all_mddevs);
- spin_unlock(&all_mddevs_lock);
- blk_cleanup_queue(mddev->queue);
- if (mddev->sysfs_state)
- sysfs_put(mddev->sysfs_state);
- mddev->sysfs_state = NULL;
- kobject_put(&mddev->kobj);
- } else
- spin_unlock(&all_mddevs_lock);
+ if (mddev->gendisk) {
+ /* we did a probe so need to clean up.
+ * Call schedule_work inside the spinlock
+ * so that flush_scheduled_work() after
+ * mddev_find will succeed in waiting for the
+ * work to be done.
+ */
+ INIT_WORK(&mddev->del_work, mddev_delayed_delete);
+ schedule_work(&mddev->del_work);
+ } else
+ kfree(mddev);
+ }
+ spin_unlock(&all_mddevs_lock);
}
static mddev_t * mddev_find(dev_t unit)
@@ -236,15 +249,50 @@
retry:
spin_lock(&all_mddevs_lock);
- list_for_each_entry(mddev, &all_mddevs, all_mddevs)
- if (mddev->unit == unit) {
- mddev_get(mddev);
- spin_unlock(&all_mddevs_lock);
- kfree(new);
- return mddev;
- }
- if (new) {
+ if (unit) {
+ list_for_each_entry(mddev, &all_mddevs, all_mddevs)
+ if (mddev->unit == unit) {
+ mddev_get(mddev);
+ spin_unlock(&all_mddevs_lock);
+ kfree(new);
+ return mddev;
+ }
+
+ if (new) {
+ list_add(&new->all_mddevs, &all_mddevs);
+ spin_unlock(&all_mddevs_lock);
+ new->hold_active = UNTIL_IOCTL;
+ return new;
+ }
+ } else if (new) {
+ /* find an unused unit number */
+ static int next_minor = 512;
+ int start = next_minor;
+ int is_free = 0;
+ int dev = 0;
+ while (!is_free) {
+ dev = MKDEV(MD_MAJOR, next_minor);
+ next_minor++;
+ if (next_minor > MINORMASK)
+ next_minor = 0;
+ if (next_minor == start) {
+ /* Oh dear, all in use. */
+ spin_unlock(&all_mddevs_lock);
+ kfree(new);
+ return NULL;
+ }
+
+ is_free = 1;
+ list_for_each_entry(mddev, &all_mddevs, all_mddevs)
+ if (mddev->unit == dev) {
+ is_free = 0;
+ break;
+ }
+ }
+ new->unit = dev;
+ new->md_minor = MINOR(dev);
+ new->hold_active = UNTIL_STOP;
list_add(&new->all_mddevs, &all_mddevs);
spin_unlock(&all_mddevs_lock);
return new;
@@ -275,16 +323,6 @@
new->resync_max = MaxSector;
new->level = LEVEL_NONE;
- new->queue = blk_alloc_queue(GFP_KERNEL);
- if (!new->queue) {
- kfree(new);
- return NULL;
- }
- /* Can be unlocked because the queue is new: no concurrency */
- queue_flag_set_unlocked(QUEUE_FLAG_CLUSTER, new->queue);
-
- blk_queue_make_request(new->queue, md_fail_request);
-
goto retry;
}
@@ -307,25 +345,23 @@
static mdk_rdev_t * find_rdev_nr(mddev_t *mddev, int nr)
{
- mdk_rdev_t * rdev;
- struct list_head *tmp;
+ mdk_rdev_t *rdev;
- rdev_for_each(rdev, tmp, mddev) {
+ list_for_each_entry(rdev, &mddev->disks, same_set)
if (rdev->desc_nr == nr)
return rdev;
- }
+
return NULL;
}
static mdk_rdev_t * find_rdev(mddev_t * mddev, dev_t dev)
{
- struct list_head *tmp;
mdk_rdev_t *rdev;
- rdev_for_each(rdev, tmp, mddev) {
+ list_for_each_entry(rdev, &mddev->disks, same_set)
if (rdev->bdev->bd_dev == dev)
return rdev;
- }
+
return NULL;
}
@@ -861,7 +897,6 @@
static void super_90_sync(mddev_t *mddev, mdk_rdev_t *rdev)
{
mdp_super_t *sb;
- struct list_head *tmp;
mdk_rdev_t *rdev2;
int next_spare = mddev->raid_disks;
@@ -933,7 +968,7 @@
sb->state |= (1<<MD_SB_BITMAP_PRESENT);
sb->disks[0].state = (1<<MD_DISK_REMOVED);
- rdev_for_each(rdev2, tmp, mddev) {
+ list_for_each_entry(rdev2, &mddev->disks, same_set) {
mdp_disk_t *d;
int desc_nr;
if (rdev2->raid_disk >= 0 && test_bit(In_sync, &rdev2->flags)
@@ -1259,7 +1294,6 @@
static void super_1_sync(mddev_t *mddev, mdk_rdev_t *rdev)
{
struct mdp_superblock_1 *sb;
- struct list_head *tmp;
mdk_rdev_t *rdev2;
int max_dev, i;
/* make rdev->sb match mddev and rdev data. */
@@ -1307,7 +1341,7 @@
}
max_dev = 0;
- rdev_for_each(rdev2, tmp, mddev)
+ list_for_each_entry(rdev2, &mddev->disks, same_set)
if (rdev2->desc_nr+1 > max_dev)
max_dev = rdev2->desc_nr+1;
@@ -1316,7 +1350,7 @@
for (i=0; i<max_dev;i++)
sb->dev_roles[i] = cpu_to_le16(0xfffe);
- rdev_for_each(rdev2, tmp, mddev) {
+ list_for_each_entry(rdev2, &mddev->disks, same_set) {
i = rdev2->desc_nr;
if (test_bit(Faulty, &rdev2->flags))
sb->dev_roles[i] = cpu_to_le16(0xfffe);
@@ -1466,6 +1500,9 @@
list_add_rcu(&rdev->same_set, &mddev->disks);
bd_claim_by_disk(rdev->bdev, rdev->bdev->bd_holder, mddev->gendisk);
+
+ /* May as well allow recovery to be retried once */
+ mddev->recovery_disabled = 0;
return 0;
fail:
@@ -1571,8 +1608,7 @@
static void export_array(mddev_t *mddev)
{
- struct list_head *tmp;
- mdk_rdev_t *rdev;
+ mdk_rdev_t *rdev, *tmp;
rdev_for_each(rdev, tmp, mddev) {
if (!rdev->mddev) {
@@ -1593,7 +1629,7 @@
desc->major,desc->minor,desc->raid_disk,desc->state);
}
-static void print_sb(mdp_super_t *sb)
+static void print_sb_90(mdp_super_t *sb)
{
int i;
@@ -1624,10 +1660,57 @@
}
printk(KERN_INFO "md: THIS: ");
print_desc(&sb->this_disk);
-
}
-static void print_rdev(mdk_rdev_t *rdev)
+static void print_sb_1(struct mdp_superblock_1 *sb)
+{
+ __u8 *uuid;
+
+ uuid = sb->set_uuid;
+ printk(KERN_INFO "md: SB: (V:%u) (F:0x%08x) Array-ID:<%02x%02x%02x%02x"
+ ":%02x%02x:%02x%02x:%02x%02x:%02x%02x%02x%02x%02x%02x>\n"
+ KERN_INFO "md: Name: \"%s\" CT:%llu\n",
+ le32_to_cpu(sb->major_version),
+ le32_to_cpu(sb->feature_map),
+ uuid[0], uuid[1], uuid[2], uuid[3],
+ uuid[4], uuid[5], uuid[6], uuid[7],
+ uuid[8], uuid[9], uuid[10], uuid[11],
+ uuid[12], uuid[13], uuid[14], uuid[15],
+ sb->set_name,
+ (unsigned long long)le64_to_cpu(sb->ctime)
+ & MD_SUPERBLOCK_1_TIME_SEC_MASK);
+
+ uuid = sb->device_uuid;
+ printk(KERN_INFO "md: L%u SZ%llu RD:%u LO:%u CS:%u DO:%llu DS:%llu SO:%llu"
+ " RO:%llu\n"
+ KERN_INFO "md: Dev:%08x UUID: %02x%02x%02x%02x:%02x%02x:%02x%02x:%02x%02x"
+ ":%02x%02x%02x%02x%02x%02x\n"
+ KERN_INFO "md: (F:0x%08x) UT:%llu Events:%llu ResyncOffset:%llu CSUM:0x%08x\n"
+ KERN_INFO "md: (MaxDev:%u) \n",
+ le32_to_cpu(sb->level),
+ (unsigned long long)le64_to_cpu(sb->size),
+ le32_to_cpu(sb->raid_disks),
+ le32_to_cpu(sb->layout),
+ le32_to_cpu(sb->chunksize),
+ (unsigned long long)le64_to_cpu(sb->data_offset),
+ (unsigned long long)le64_to_cpu(sb->data_size),
+ (unsigned long long)le64_to_cpu(sb->super_offset),
+ (unsigned long long)le64_to_cpu(sb->recovery_offset),
+ le32_to_cpu(sb->dev_number),
+ uuid[0], uuid[1], uuid[2], uuid[3],
+ uuid[4], uuid[5], uuid[6], uuid[7],
+ uuid[8], uuid[9], uuid[10], uuid[11],
+ uuid[12], uuid[13], uuid[14], uuid[15],
+ sb->devflags,
+ (unsigned long long)le64_to_cpu(sb->utime) & MD_SUPERBLOCK_1_TIME_SEC_MASK,
+ (unsigned long long)le64_to_cpu(sb->events),
+ (unsigned long long)le64_to_cpu(sb->resync_offset),
+ le32_to_cpu(sb->sb_csum),
+ le32_to_cpu(sb->max_dev)
+ );
+}
+
+static void print_rdev(mdk_rdev_t *rdev, int major_version)
{
char b[BDEVNAME_SIZE];
printk(KERN_INFO "md: rdev %s, SZ:%08llu F:%d S:%d DN:%u\n",
@@ -1635,15 +1718,22 @@
test_bit(Faulty, &rdev->flags), test_bit(In_sync, &rdev->flags),
rdev->desc_nr);
if (rdev->sb_loaded) {
- printk(KERN_INFO "md: rdev superblock:\n");
- print_sb((mdp_super_t*)page_address(rdev->sb_page));
+ printk(KERN_INFO "md: rdev superblock (MJ:%d):\n", major_version);
+ switch (major_version) {
+ case 0:
+ print_sb_90((mdp_super_t*)page_address(rdev->sb_page));
+ break;
+ case 1:
+ print_sb_1((struct mdp_superblock_1 *)page_address(rdev->sb_page));
+ break;
+ }
} else
printk(KERN_INFO "md: no rdev superblock!\n");
}
static void md_print_devices(void)
{
- struct list_head *tmp, *tmp2;
+ struct list_head *tmp;
mdk_rdev_t *rdev;
mddev_t *mddev;
char b[BDEVNAME_SIZE];
@@ -1658,12 +1748,12 @@
bitmap_print_sb(mddev->bitmap);
else
printk("%s: ", mdname(mddev));
- rdev_for_each(rdev, tmp2, mddev)
+ list_for_each_entry(rdev, &mddev->disks, same_set)
printk("<%s>", bdevname(rdev->bdev,b));
printk("\n");
- rdev_for_each(rdev, tmp2, mddev)
- print_rdev(rdev);
+ list_for_each_entry(rdev, &mddev->disks, same_set)
+ print_rdev(rdev, mddev->major_version);
}
printk("md: **********************************\n");
printk("\n");
@@ -1679,9 +1769,8 @@
* with the rest of the array)
*/
mdk_rdev_t *rdev;
- struct list_head *tmp;
- rdev_for_each(rdev, tmp, mddev) {
+ list_for_each_entry(rdev, &mddev->disks, same_set) {
if (rdev->sb_events == mddev->events ||
(nospares &&
rdev->raid_disk < 0 &&
@@ -1699,7 +1788,6 @@
static void md_update_sb(mddev_t * mddev, int force_change)
{
- struct list_head *tmp;
mdk_rdev_t *rdev;
int sync_req;
int nospares = 0;
@@ -1790,7 +1878,7 @@
mdname(mddev),mddev->in_sync);
bitmap_update_sb(mddev->bitmap);
- rdev_for_each(rdev, tmp, mddev) {
+ list_for_each_entry(rdev, &mddev->disks, same_set) {
char b[BDEVNAME_SIZE];
dprintk(KERN_INFO "md: ");
if (rdev->sb_loaded != 1)
@@ -1999,7 +2087,6 @@
md_wakeup_thread(rdev->mddev->thread);
} else if (rdev->mddev->pers) {
mdk_rdev_t *rdev2;
- struct list_head *tmp;
/* Activating a spare .. or possibly reactivating
* if we every get bitmaps working here.
*/
@@ -2010,7 +2097,7 @@
if (rdev->mddev->pers->hot_add_disk == NULL)
return -EINVAL;
- rdev_for_each(rdev2, tmp, rdev->mddev)
+ list_for_each_entry(rdev2, &rdev->mddev->disks, same_set)
if (rdev2->raid_disk == slot)
return -EEXIST;
@@ -2125,14 +2212,14 @@
*/
mddev_t *mddev;
int overlap = 0;
- struct list_head *tmp, *tmp2;
+ struct list_head *tmp;
mddev_unlock(my_mddev);
for_each_mddev(mddev, tmp) {
mdk_rdev_t *rdev2;
mddev_lock(mddev);
- rdev_for_each(rdev2, tmp2, mddev)
+ list_for_each_entry(rdev2, &mddev->disks, same_set)
if (test_bit(AllReserved, &rdev2->flags) ||
(rdev->bdev == rdev2->bdev &&
rdev != rdev2 &&
@@ -2328,8 +2415,7 @@
static void analyze_sbs(mddev_t * mddev)
{
int i;
- struct list_head *tmp;
- mdk_rdev_t *rdev, *freshest;
+ mdk_rdev_t *rdev, *freshest, *tmp;
char b[BDEVNAME_SIZE];
freshest = NULL;
@@ -3046,7 +3132,7 @@
}
set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
md_wakeup_thread(mddev->thread);
- sysfs_notify(&mddev->kobj, NULL, "sync_action");
+ sysfs_notify_dirent(mddev->sysfs_action);
return len;
}
@@ -3404,6 +3490,8 @@
if (!capable(CAP_SYS_ADMIN))
return -EACCES;
rv = mddev_lock(mddev);
+ if (mddev->hold_active == UNTIL_IOCTL)
+ mddev->hold_active = 0;
if (!rv) {
rv = entry->store(mddev, page, length);
mddev_unlock(mddev);
@@ -3414,6 +3502,17 @@
static void md_free(struct kobject *ko)
{
mddev_t *mddev = container_of(ko, mddev_t, kobj);
+
+ if (mddev->sysfs_state)
+ sysfs_put(mddev->sysfs_state);
+
+ if (mddev->gendisk) {
+ del_gendisk(mddev->gendisk);
+ put_disk(mddev->gendisk);
+ }
+ if (mddev->queue)
+ blk_cleanup_queue(mddev->queue);
+
kfree(mddev);
}
@@ -3429,34 +3528,74 @@
int mdp_major = 0;
-static struct kobject *md_probe(dev_t dev, int *part, void *data)
+static int md_alloc(dev_t dev, char *name)
{
static DEFINE_MUTEX(disks_mutex);
mddev_t *mddev = mddev_find(dev);
struct gendisk *disk;
- int partitioned = (MAJOR(dev) != MD_MAJOR);
- int shift = partitioned ? MdpMinorShift : 0;
- int unit = MINOR(dev) >> shift;
+ int partitioned;
+ int shift;
+ int unit;
int error;
if (!mddev)
- return NULL;
+ return -ENODEV;
+
+ partitioned = (MAJOR(mddev->unit) != MD_MAJOR);
+ shift = partitioned ? MdpMinorShift : 0;
+ unit = MINOR(mddev->unit) >> shift;
+
+ /* wait for any previous instance if this device
+ * to be completed removed (mddev_delayed_delete).
+ */
+ flush_scheduled_work();
mutex_lock(&disks_mutex);
if (mddev->gendisk) {
mutex_unlock(&disks_mutex);
mddev_put(mddev);
- return NULL;
+ return -EEXIST;
}
+
+ if (name) {
+ /* Need to ensure that 'name' is not a duplicate.
+ */
+ mddev_t *mddev2;
+ spin_lock(&all_mddevs_lock);
+
+ list_for_each_entry(mddev2, &all_mddevs, all_mddevs)
+ if (mddev2->gendisk &&
+ strcmp(mddev2->gendisk->disk_name, name) == 0) {
+ spin_unlock(&all_mddevs_lock);
+ return -EEXIST;
+ }
+ spin_unlock(&all_mddevs_lock);
+ }
+
+ mddev->queue = blk_alloc_queue(GFP_KERNEL);
+ if (!mddev->queue) {
+ mutex_unlock(&disks_mutex);
+ mddev_put(mddev);
+ return -ENOMEM;
+ }
+ /* Can be unlocked because the queue is new: no concurrency */
+ queue_flag_set_unlocked(QUEUE_FLAG_CLUSTER, mddev->queue);
+
+ blk_queue_make_request(mddev->queue, md_fail_request);
+
disk = alloc_disk(1 << shift);
if (!disk) {
mutex_unlock(&disks_mutex);
+ blk_cleanup_queue(mddev->queue);
+ mddev->queue = NULL;
mddev_put(mddev);
- return NULL;
+ return -ENOMEM;
}
- disk->major = MAJOR(dev);
+ disk->major = MAJOR(mddev->unit);
disk->first_minor = unit << shift;
- if (partitioned)
+ if (name)
+ strcpy(disk->disk_name, name);
+ else if (partitioned)
sprintf(disk->disk_name, "md_d%d", unit);
else
sprintf(disk->disk_name, "md%d", unit);
@@ -3464,7 +3603,7 @@
disk->private_data = mddev;
disk->queue = mddev->queue;
/* Allow extended partitions. This makes the
- * 'mdp' device redundant, but we can really
+ * 'mdp' device redundant, but we can't really
* remove it now.
*/
disk->flags |= GENHD_FL_EXT_DEVT;
@@ -3480,9 +3619,35 @@
kobject_uevent(&mddev->kobj, KOBJ_ADD);
mddev->sysfs_state = sysfs_get_dirent(mddev->kobj.sd, "array_state");
}
+ mddev_put(mddev);
+ return 0;
+}
+
+static struct kobject *md_probe(dev_t dev, int *part, void *data)
+{
+ md_alloc(dev, NULL);
return NULL;
}
+static int add_named_array(const char *val, struct kernel_param *kp)
+{
+ /* val must be "md_*" where * is not all digits.
+ * We allocate an array with a large free minor number, and
+ * set the name to val. val must not already be an active name.
+ */
+ int len = strlen(val);
+ char buf[DISK_NAME_LEN];
+
+ while (len && val[len-1] == '\n')
+ len--;
+ if (len >= DISK_NAME_LEN)
+ return -E2BIG;
+ strlcpy(buf, val, len+1);
+ if (strncmp(buf, "md_", 3) != 0)
+ return -EINVAL;
+ return md_alloc(0, buf);
+}
+
static void md_safemode_timeout(unsigned long data)
{
mddev_t *mddev = (mddev_t *) data;
@@ -3501,7 +3666,6 @@
{
int err;
int chunk_size;
- struct list_head *tmp;
mdk_rdev_t *rdev;
struct gendisk *disk;
struct mdk_personality *pers;
@@ -3540,7 +3704,7 @@
}
/* devices must have minimum size of one chunk */
- rdev_for_each(rdev, tmp, mddev) {
+ list_for_each_entry(rdev, &mddev->disks, same_set) {
if (test_bit(Faulty, &rdev->flags))
continue;
if (rdev->size < chunk_size / 1024) {
@@ -3565,7 +3729,7 @@
* the only valid external interface is through the md
* device.
*/
- rdev_for_each(rdev, tmp, mddev) {
+ list_for_each_entry(rdev, &mddev->disks, same_set) {
if (test_bit(Faulty, &rdev->flags))
continue;
sync_blockdev(rdev->bdev);
@@ -3630,10 +3794,10 @@
*/
char b[BDEVNAME_SIZE], b2[BDEVNAME_SIZE];
mdk_rdev_t *rdev2;
- struct list_head *tmp2;
int warned = 0;
- rdev_for_each(rdev, tmp, mddev) {
- rdev_for_each(rdev2, tmp2, mddev) {
+
+ list_for_each_entry(rdev, &mddev->disks, same_set)
+ list_for_each_entry(rdev2, &mddev->disks, same_set) {
if (rdev < rdev2 &&
rdev->bdev->bd_contains ==
rdev2->bdev->bd_contains) {
@@ -3647,7 +3811,7 @@
warned = 1;
}
}
- }
+
if (warned)
printk(KERN_WARNING
"True protection against single-disk"
@@ -3684,6 +3848,7 @@
printk(KERN_WARNING
"md: cannot register extra attributes for %s\n",
mdname(mddev));
+ mddev->sysfs_action = sysfs_get_dirent(mddev->kobj.sd, "sync_action");
} else if (mddev->ro == 2) /* auto-readonly not meaningful */
mddev->ro = 0;
@@ -3694,7 +3859,7 @@
mddev->safemode_delay = (200 * HZ)/1000 +1; /* 200 msec delay */
mddev->in_sync = 1;
- rdev_for_each(rdev, tmp, mddev)
+ list_for_each_entry(rdev, &mddev->disks, same_set)
if (rdev->raid_disk >= 0) {
char nm[20];
sprintf(nm, "rd%d", rdev->raid_disk);
@@ -3725,9 +3890,8 @@
* it will remove the drives and not do the right thing
*/
if (mddev->degraded && !mddev->sync_thread) {
- struct list_head *rtmp;
int spares = 0;
- rdev_for_each(rdev, rtmp, mddev)
+ list_for_each_entry(rdev, &mddev->disks, same_set)
if (rdev->raid_disk >= 0 &&
!test_bit(In_sync, &rdev->flags) &&
!test_bit(Faulty, &rdev->flags))
@@ -3754,7 +3918,8 @@
mddev->changed = 1;
md_new_event(mddev);
sysfs_notify_dirent(mddev->sysfs_state);
- sysfs_notify(&mddev->kobj, NULL, "sync_action");
+ if (mddev->sysfs_action)
+ sysfs_notify_dirent(mddev->sysfs_action);
sysfs_notify(&mddev->kobj, NULL, "degraded");
kobject_uevent(&disk_to_dev(mddev->gendisk)->kobj, KOBJ_CHANGE);
return 0;
@@ -3854,9 +4019,12 @@
mddev->queue->merge_bvec_fn = NULL;
mddev->queue->unplug_fn = NULL;
mddev->queue->backing_dev_info.congested_fn = NULL;
- if (mddev->pers->sync_request)
+ if (mddev->pers->sync_request) {
sysfs_remove_group(&mddev->kobj, &md_redundancy_group);
-
+ if (mddev->sysfs_action)
+ sysfs_put(mddev->sysfs_action);
+ mddev->sysfs_action = NULL;
+ }
module_put(mddev->pers->owner);
mddev->pers = NULL;
/* tell userspace to handle 'inactive' */
@@ -3883,7 +4051,6 @@
*/
if (mode == 0) {
mdk_rdev_t *rdev;
- struct list_head *tmp;
printk(KERN_INFO "md: %s stopped.\n", mdname(mddev));
@@ -3895,7 +4062,7 @@
}
mddev->bitmap_offset = 0;
- rdev_for_each(rdev, tmp, mddev)
+ list_for_each_entry(rdev, &mddev->disks, same_set)
if (rdev->raid_disk >= 0) {
char nm[20];
sprintf(nm, "rd%d", rdev->raid_disk);
@@ -3941,6 +4108,8 @@
mddev->barriers_work = 0;
mddev->safemode = 0;
kobject_uevent(&disk_to_dev(mddev->gendisk)->kobj, KOBJ_CHANGE);
+ if (mddev->hold_active == UNTIL_STOP)
+ mddev->hold_active = 0;
} else if (mddev->pers)
printk(KERN_INFO "md: %s switched to read-only mode.\n",
@@ -3956,7 +4125,6 @@
static void autorun_array(mddev_t *mddev)
{
mdk_rdev_t *rdev;
- struct list_head *tmp;
int err;
if (list_empty(&mddev->disks))
@@ -3964,7 +4132,7 @@
printk(KERN_INFO "md: running: ");
- rdev_for_each(rdev, tmp, mddev) {
+ list_for_each_entry(rdev, &mddev->disks, same_set) {
char b[BDEVNAME_SIZE];
printk("<%s>", bdevname(rdev->bdev,b));
}
@@ -3991,8 +4159,7 @@
*/
static void autorun_devices(int part)
{
- struct list_head *tmp;
- mdk_rdev_t *rdev0, *rdev;
+ mdk_rdev_t *rdev0, *rdev, *tmp;
mddev_t *mddev;
char b[BDEVNAME_SIZE];
@@ -4007,7 +4174,7 @@
printk(KERN_INFO "md: considering %s ...\n",
bdevname(rdev0->bdev,b));
INIT_LIST_HEAD(&candidates);
- rdev_for_each_list(rdev, tmp, pending_raid_disks)
+ rdev_for_each_list(rdev, tmp, &pending_raid_disks)
if (super_90_load(rdev, rdev0, 0) >= 0) {
printk(KERN_INFO "md: adding %s ...\n",
bdevname(rdev->bdev,b));
@@ -4053,7 +4220,7 @@
} else {
printk(KERN_INFO "md: created %s\n", mdname(mddev));
mddev->persistent = 1;
- rdev_for_each_list(rdev, tmp, candidates) {
+ rdev_for_each_list(rdev, tmp, &candidates) {
list_del_init(&rdev->same_set);
if (bind_rdev_to_array(rdev, mddev))
export_rdev(rdev);
@@ -4064,7 +4231,7 @@
/* on success, candidates will be empty, on error
* it won't...
*/
- rdev_for_each_list(rdev, tmp, candidates) {
+ rdev_for_each_list(rdev, tmp, &candidates) {
list_del_init(&rdev->same_set);
export_rdev(rdev);
}
@@ -4093,10 +4260,9 @@
mdu_array_info_t info;
int nr,working,active,failed,spare;
mdk_rdev_t *rdev;
- struct list_head *tmp;
nr=working=active=failed=spare=0;
- rdev_for_each(rdev, tmp, mddev) {
+ list_for_each_entry(rdev, &mddev->disks, same_set) {
nr++;
if (test_bit(Faulty, &rdev->flags))
failed++;
@@ -4614,9 +4780,8 @@
static int update_size(mddev_t *mddev, sector_t num_sectors)
{
- mdk_rdev_t * rdev;
+ mdk_rdev_t *rdev;
int rv;
- struct list_head *tmp;
int fit = (num_sectors == 0);
if (mddev->pers->resize == NULL)
@@ -4638,7 +4803,7 @@
* grow, and re-add.
*/
return -EBUSY;
- rdev_for_each(rdev, tmp, mddev) {
+ list_for_each_entry(rdev, &mddev->disks, same_set) {
sector_t avail;
avail = rdev->size * 2;
@@ -5000,6 +5165,9 @@
done_unlock:
abort_unlock:
+ if (mddev->hold_active == UNTIL_IOCTL &&
+ err != -EINVAL)
+ mddev->hold_active = 0;
mddev_unlock(mddev);
return err;
@@ -5016,14 +5184,25 @@
* Succeed if we can lock the mddev, which confirms that
* it isn't being stopped right now.
*/
- mddev_t *mddev = bdev->bd_disk->private_data;
+ mddev_t *mddev = mddev_find(bdev->bd_dev);
int err;
+ if (mddev->gendisk != bdev->bd_disk) {
+ /* we are racing with mddev_put which is discarding this
+ * bd_disk.
+ */
+ mddev_put(mddev);
+ /* Wait until bdev->bd_disk is definitely gone */
+ flush_scheduled_work();
+ /* Then retry the open from the top */
+ return -ERESTARTSYS;
+ }
+ BUG_ON(mddev != bdev->bd_disk->private_data);
+
if ((err = mutex_lock_interruptible_nested(&mddev->reconfig_mutex, 1)))
goto out;
err = 0;
- mddev_get(mddev);
atomic_inc(&mddev->openers);
mddev_unlock(mddev);
@@ -5187,11 +5366,10 @@
{
int i = 0;
mdk_rdev_t *rdev;
- struct list_head *tmp;
seq_printf(seq, "unused devices: ");
- rdev_for_each_list(rdev, tmp, pending_raid_disks) {
+ list_for_each_entry(rdev, &pending_raid_disks, same_set) {
char b[BDEVNAME_SIZE];
i++;
seq_printf(seq, "%s ",
@@ -5350,7 +5528,6 @@
{
mddev_t *mddev = v;
sector_t size;
- struct list_head *tmp2;
mdk_rdev_t *rdev;
struct mdstat_info *mi = seq->private;
struct bitmap *bitmap;
@@ -5387,7 +5564,7 @@
}
size = 0;
- rdev_for_each(rdev, tmp2, mddev) {
+ list_for_each_entry(rdev, &mddev->disks, same_set) {
char b[BDEVNAME_SIZE];
seq_printf(seq, " %s[%d]",
bdevname(rdev->bdev,b), rdev->desc_nr);
@@ -5694,7 +5871,6 @@
struct list_head *tmp;
sector_t last_check;
int skipped = 0;
- struct list_head *rtmp;
mdk_rdev_t *rdev;
char *desc;
@@ -5799,7 +5975,7 @@
/* recovery follows the physical size of devices */
max_sectors = mddev->size << 1;
j = MaxSector;
- rdev_for_each(rdev, rtmp, mddev)
+ list_for_each_entry(rdev, &mddev->disks, same_set)
if (rdev->raid_disk >= 0 &&
!test_bit(Faulty, &rdev->flags) &&
!test_bit(In_sync, &rdev->flags) &&
@@ -5949,7 +6125,7 @@
} else {
if (!test_bit(MD_RECOVERY_INTR, &mddev->recovery))
mddev->curr_resync = MaxSector;
- rdev_for_each(rdev, rtmp, mddev)
+ list_for_each_entry(rdev, &mddev->disks, same_set)
if (rdev->raid_disk >= 0 &&
!test_bit(Faulty, &rdev->flags) &&
!test_bit(In_sync, &rdev->flags) &&
@@ -5985,10 +6161,9 @@
static int remove_and_add_spares(mddev_t *mddev)
{
mdk_rdev_t *rdev;
- struct list_head *rtmp;
int spares = 0;
- rdev_for_each(rdev, rtmp, mddev)
+ list_for_each_entry(rdev, &mddev->disks, same_set)
if (rdev->raid_disk >= 0 &&
!test_bit(Blocked, &rdev->flags) &&
(test_bit(Faulty, &rdev->flags) ||
@@ -6003,8 +6178,8 @@
}
}
- if (mddev->degraded && ! mddev->ro) {
- rdev_for_each(rdev, rtmp, mddev) {
+ if (mddev->degraded && ! mddev->ro && !mddev->recovery_disabled) {
+ list_for_each_entry(rdev, &mddev->disks, same_set) {
if (rdev->raid_disk >= 0 &&
!test_bit(In_sync, &rdev->flags) &&
!test_bit(Blocked, &rdev->flags))
@@ -6056,7 +6231,6 @@
void md_check_recovery(mddev_t *mddev)
{
mdk_rdev_t *rdev;
- struct list_head *rtmp;
if (mddev->bitmap)
@@ -6120,7 +6294,7 @@
if (mddev->flags)
md_update_sb(mddev, 0);
- rdev_for_each(rdev, rtmp, mddev)
+ list_for_each_entry(rdev, &mddev->disks, same_set)
if (test_and_clear_bit(StateChanged, &rdev->flags))
sysfs_notify_dirent(rdev->sysfs_state);
@@ -6149,13 +6323,13 @@
* information must be scrapped
*/
if (!mddev->degraded)
- rdev_for_each(rdev, rtmp, mddev)
+ list_for_each_entry(rdev, &mddev->disks, same_set)
rdev->saved_raid_disk = -1;
mddev->recovery = 0;
/* flag recovery needed just to double check */
set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
- sysfs_notify(&mddev->kobj, NULL, "sync_action");
+ sysfs_notify_dirent(mddev->sysfs_action);
md_new_event(mddev);
goto unlock;
}
@@ -6216,7 +6390,7 @@
mddev->recovery = 0;
} else
md_wakeup_thread(mddev->sync_thread);
- sysfs_notify(&mddev->kobj, NULL, "sync_action");
+ sysfs_notify_dirent(mddev->sysfs_action);
md_new_event(mddev);
}
unlock:
@@ -6224,7 +6398,8 @@
clear_bit(MD_RECOVERY_RUNNING, &mddev->recovery);
if (test_and_clear_bit(MD_RECOVERY_RECOVER,
&mddev->recovery))
- sysfs_notify(&mddev->kobj, NULL, "sync_action");
+ if (mddev->sysfs_action)
+ sysfs_notify_dirent(mddev->sysfs_action);
}
mddev_unlock(mddev);
}
@@ -6386,14 +6561,8 @@
unregister_sysctl_table(raid_table_header);
remove_proc_entry("mdstat", NULL);
for_each_mddev(mddev, tmp) {
- struct gendisk *disk = mddev->gendisk;
- if (!disk)
- continue;
export_array(mddev);
- del_gendisk(disk);
- put_disk(disk);
- mddev->gendisk = NULL;
- mddev_put(mddev);
+ mddev->hold_active = 0;
}
}
@@ -6418,6 +6587,7 @@
module_param_call(start_ro, set_ro, get_ro, NULL, S_IRUSR|S_IWUSR);
module_param(start_dirty_degraded, int, S_IRUGO|S_IWUSR);
+module_param_call(new_array, add_named_array, NULL, NULL, S_IWUSR);
EXPORT_SYMBOL(register_md_personality);
EXPORT_SYMBOL(unregister_md_personality);
diff --git a/drivers/md/multipath.c b/drivers/md/multipath.c
index d4ac47d..f6d08f2 100644
--- a/drivers/md/multipath.c
+++ b/drivers/md/multipath.c
@@ -408,7 +408,6 @@
int disk_idx;
struct multipath_info *disk;
mdk_rdev_t *rdev;
- struct list_head *tmp;
if (mddev->level != LEVEL_MULTIPATH) {
printk("multipath: %s: raid level not set to multipath IO (%d)\n",
@@ -441,7 +440,7 @@
}
conf->working_disks = 0;
- rdev_for_each(rdev, tmp, mddev) {
+ list_for_each_entry(rdev, &mddev->disks, same_set) {
disk_idx = rdev->raid_disk;
if (disk_idx < 0 ||
disk_idx >= mddev->raid_disks)
diff --git a/drivers/md/raid0.c b/drivers/md/raid0.c
index 8ac6488..c605ba8 100644
--- a/drivers/md/raid0.c
+++ b/drivers/md/raid0.c
@@ -53,11 +53,10 @@
static int create_strip_zones (mddev_t *mddev)
{
int i, c, j;
- sector_t current_offset, curr_zone_offset;
+ sector_t current_start, curr_zone_start;
sector_t min_spacing;
raid0_conf_t *conf = mddev_to_conf(mddev);
mdk_rdev_t *smallest, *rdev1, *rdev2, *rdev;
- struct list_head *tmp1, *tmp2;
struct strip_zone *zone;
int cnt;
char b[BDEVNAME_SIZE];
@@ -67,19 +66,19 @@
*/
conf->nr_strip_zones = 0;
- rdev_for_each(rdev1, tmp1, mddev) {
- printk("raid0: looking at %s\n",
+ list_for_each_entry(rdev1, &mddev->disks, same_set) {
+ printk(KERN_INFO "raid0: looking at %s\n",
bdevname(rdev1->bdev,b));
c = 0;
- rdev_for_each(rdev2, tmp2, mddev) {
- printk("raid0: comparing %s(%llu)",
+ list_for_each_entry(rdev2, &mddev->disks, same_set) {
+ printk(KERN_INFO "raid0: comparing %s(%llu)",
bdevname(rdev1->bdev,b),
(unsigned long long)rdev1->size);
- printk(" with %s(%llu)\n",
+ printk(KERN_INFO " with %s(%llu)\n",
bdevname(rdev2->bdev,b),
(unsigned long long)rdev2->size);
if (rdev2 == rdev1) {
- printk("raid0: END\n");
+ printk(KERN_INFO "raid0: END\n");
break;
}
if (rdev2->size == rdev1->size)
@@ -88,19 +87,20 @@
* Not unique, don't count it as a new
* group
*/
- printk("raid0: EQUAL\n");
+ printk(KERN_INFO "raid0: EQUAL\n");
c = 1;
break;
}
- printk("raid0: NOT EQUAL\n");
+ printk(KERN_INFO "raid0: NOT EQUAL\n");
}
if (!c) {
- printk("raid0: ==> UNIQUE\n");
+ printk(KERN_INFO "raid0: ==> UNIQUE\n");
conf->nr_strip_zones++;
- printk("raid0: %d zones\n", conf->nr_strip_zones);
+ printk(KERN_INFO "raid0: %d zones\n",
+ conf->nr_strip_zones);
}
}
- printk("raid0: FINAL %d zones\n", conf->nr_strip_zones);
+ printk(KERN_INFO "raid0: FINAL %d zones\n", conf->nr_strip_zones);
conf->strip_zone = kzalloc(sizeof(struct strip_zone)*
conf->nr_strip_zones, GFP_KERNEL);
@@ -119,16 +119,17 @@
cnt = 0;
smallest = NULL;
zone->dev = conf->devlist;
- rdev_for_each(rdev1, tmp1, mddev) {
+ list_for_each_entry(rdev1, &mddev->disks, same_set) {
int j = rdev1->raid_disk;
if (j < 0 || j >= mddev->raid_disks) {
- printk("raid0: bad disk number %d - aborting!\n", j);
+ printk(KERN_ERR "raid0: bad disk number %d - "
+ "aborting!\n", j);
goto abort;
}
if (zone->dev[j]) {
- printk("raid0: multiple devices for %d - aborting!\n",
- j);
+ printk(KERN_ERR "raid0: multiple devices for %d - "
+ "aborting!\n", j);
goto abort;
}
zone->dev[j] = rdev1;
@@ -149,16 +150,16 @@
cnt++;
}
if (cnt != mddev->raid_disks) {
- printk("raid0: too few disks (%d of %d) - aborting!\n",
- cnt, mddev->raid_disks);
+ printk(KERN_ERR "raid0: too few disks (%d of %d) - "
+ "aborting!\n", cnt, mddev->raid_disks);
goto abort;
}
zone->nb_dev = cnt;
- zone->size = smallest->size * cnt;
- zone->zone_offset = 0;
+ zone->sectors = smallest->size * cnt * 2;
+ zone->zone_start = 0;
- current_offset = smallest->size;
- curr_zone_offset = zone->size;
+ current_start = smallest->size * 2;
+ curr_zone_start = zone->sectors;
/* now do the other zones */
for (i = 1; i < conf->nr_strip_zones; i++)
@@ -166,40 +167,41 @@
zone = conf->strip_zone + i;
zone->dev = conf->strip_zone[i-1].dev + mddev->raid_disks;
- printk("raid0: zone %d\n", i);
- zone->dev_offset = current_offset;
+ printk(KERN_INFO "raid0: zone %d\n", i);
+ zone->dev_start = current_start;
smallest = NULL;
c = 0;
for (j=0; j<cnt; j++) {
char b[BDEVNAME_SIZE];
rdev = conf->strip_zone[0].dev[j];
- printk("raid0: checking %s ...", bdevname(rdev->bdev,b));
- if (rdev->size > current_offset)
- {
- printk(" contained as device %d\n", c);
+ printk(KERN_INFO "raid0: checking %s ...",
+ bdevname(rdev->bdev, b));
+ if (rdev->size > current_start / 2) {
+ printk(KERN_INFO " contained as device %d\n",
+ c);
zone->dev[c] = rdev;
c++;
if (!smallest || (rdev->size <smallest->size)) {
smallest = rdev;
- printk(" (%llu) is smallest!.\n",
+ printk(KERN_INFO " (%llu) is smallest!.\n",
(unsigned long long)rdev->size);
}
} else
- printk(" nope.\n");
+ printk(KERN_INFO " nope.\n");
}
zone->nb_dev = c;
- zone->size = (smallest->size - current_offset) * c;
- printk("raid0: zone->nb_dev: %d, size: %llu\n",
- zone->nb_dev, (unsigned long long)zone->size);
+ zone->sectors = (smallest->size * 2 - current_start) * c;
+ printk(KERN_INFO "raid0: zone->nb_dev: %d, sectors: %llu\n",
+ zone->nb_dev, (unsigned long long)zone->sectors);
- zone->zone_offset = curr_zone_offset;
- curr_zone_offset += zone->size;
+ zone->zone_start = curr_zone_start;
+ curr_zone_start += zone->sectors;
- current_offset = smallest->size;
- printk("raid0: current zone offset: %llu\n",
- (unsigned long long)current_offset);
+ current_start = smallest->size * 2;
+ printk(KERN_INFO "raid0: current zone start: %llu\n",
+ (unsigned long long)current_start);
}
/* Now find appropriate hash spacing.
@@ -210,16 +212,16 @@
* strip though as it's size has no bearing on the efficacy of the hash
* table.
*/
- conf->hash_spacing = curr_zone_offset;
- min_spacing = curr_zone_offset;
+ conf->spacing = curr_zone_start;
+ min_spacing = curr_zone_start;
sector_div(min_spacing, PAGE_SIZE/sizeof(struct strip_zone*));
for (i=0; i < conf->nr_strip_zones-1; i++) {
- sector_t sz = 0;
- for (j=i; j<conf->nr_strip_zones-1 &&
- sz < min_spacing ; j++)
- sz += conf->strip_zone[j].size;
- if (sz >= min_spacing && sz < conf->hash_spacing)
- conf->hash_spacing = sz;
+ sector_t s = 0;
+ for (j = i; j < conf->nr_strip_zones - 1 &&
+ s < min_spacing; j++)
+ s += conf->strip_zone[j].sectors;
+ if (s >= min_spacing && s < conf->spacing)
+ conf->spacing = s;
}
mddev->queue->unplug_fn = raid0_unplug;
@@ -227,7 +229,7 @@
mddev->queue->backing_dev_info.congested_fn = raid0_congested;
mddev->queue->backing_dev_info.congested_data = mddev;
- printk("raid0: done.\n");
+ printk(KERN_INFO "raid0: done.\n");
return 0;
abort:
return 1;
@@ -262,10 +264,9 @@
static int raid0_run (mddev_t *mddev)
{
unsigned cur=0, i=0, nb_zone;
- s64 size;
+ s64 sectors;
raid0_conf_t *conf;
mdk_rdev_t *rdev;
- struct list_head *tmp;
if (mddev->chunk_size == 0) {
printk(KERN_ERR "md/raid0: non-zero chunk size required.\n");
@@ -291,54 +292,54 @@
/* calculate array device size */
mddev->array_sectors = 0;
- rdev_for_each(rdev, tmp, mddev)
+ list_for_each_entry(rdev, &mddev->disks, same_set)
mddev->array_sectors += rdev->size * 2;
- printk("raid0 : md_size is %llu blocks.\n",
- (unsigned long long)mddev->array_sectors / 2);
- printk("raid0 : conf->hash_spacing is %llu blocks.\n",
- (unsigned long long)conf->hash_spacing);
+ printk(KERN_INFO "raid0 : md_size is %llu sectors.\n",
+ (unsigned long long)mddev->array_sectors);
+ printk(KERN_INFO "raid0 : conf->spacing is %llu sectors.\n",
+ (unsigned long long)conf->spacing);
{
- sector_t s = mddev->array_sectors / 2;
- sector_t space = conf->hash_spacing;
+ sector_t s = mddev->array_sectors;
+ sector_t space = conf->spacing;
int round;
- conf->preshift = 0;
+ conf->sector_shift = 0;
if (sizeof(sector_t) > sizeof(u32)) {
/*shift down space and s so that sector_div will work */
while (space > (sector_t) (~(u32)0)) {
s >>= 1;
space >>= 1;
s += 1; /* force round-up */
- conf->preshift++;
+ conf->sector_shift++;
}
}
round = sector_div(s, (u32)space) ? 1 : 0;
nb_zone = s + round;
}
- printk("raid0 : nb_zone is %d.\n", nb_zone);
+ printk(KERN_INFO "raid0 : nb_zone is %d.\n", nb_zone);
- printk("raid0 : Allocating %Zd bytes for hash.\n",
+ printk(KERN_INFO "raid0 : Allocating %zu bytes for hash.\n",
nb_zone*sizeof(struct strip_zone*));
conf->hash_table = kmalloc (sizeof (struct strip_zone *)*nb_zone, GFP_KERNEL);
if (!conf->hash_table)
goto out_free_conf;
- size = conf->strip_zone[cur].size;
+ sectors = conf->strip_zone[cur].sectors;
conf->hash_table[0] = conf->strip_zone + cur;
for (i=1; i< nb_zone; i++) {
- while (size <= conf->hash_spacing) {
+ while (sectors <= conf->spacing) {
cur++;
- size += conf->strip_zone[cur].size;
+ sectors += conf->strip_zone[cur].sectors;
}
- size -= conf->hash_spacing;
+ sectors -= conf->spacing;
conf->hash_table[i] = conf->strip_zone + cur;
}
- if (conf->preshift) {
- conf->hash_spacing >>= conf->preshift;
- /* round hash_spacing up so when we divide by it, we
+ if (conf->sector_shift) {
+ conf->spacing >>= conf->sector_shift;
+ /* round spacing up so when we divide by it, we
* err on the side of too-low, which is safest
*/
- conf->hash_spacing++;
+ conf->spacing++;
}
/* calculate the max read-ahead size.
@@ -387,12 +388,12 @@
static int raid0_make_request (struct request_queue *q, struct bio *bio)
{
mddev_t *mddev = q->queuedata;
- unsigned int sect_in_chunk, chunksize_bits, chunk_size, chunk_sects;
+ unsigned int sect_in_chunk, chunksect_bits, chunk_sects;
raid0_conf_t *conf = mddev_to_conf(mddev);
struct strip_zone *zone;
mdk_rdev_t *tmp_dev;
sector_t chunk;
- sector_t block, rsect;
+ sector_t sector, rsect;
const int rw = bio_data_dir(bio);
int cpu;
@@ -407,11 +408,9 @@
bio_sectors(bio));
part_stat_unlock();
- chunk_size = mddev->chunk_size >> 10;
chunk_sects = mddev->chunk_size >> 9;
- chunksize_bits = ffz(~chunk_size);
- block = bio->bi_sector >> 1;
-
+ chunksect_bits = ffz(~chunk_sects);
+ sector = bio->bi_sector;
if (unlikely(chunk_sects < (bio->bi_sector & (chunk_sects - 1)) + (bio->bi_size >> 9))) {
struct bio_pair *bp;
@@ -434,28 +433,27 @@
{
- sector_t x = block >> conf->preshift;
- sector_div(x, (u32)conf->hash_spacing);
+ sector_t x = sector >> conf->sector_shift;
+ sector_div(x, (u32)conf->spacing);
zone = conf->hash_table[x];
}
-
- while (block >= (zone->zone_offset + zone->size))
+
+ while (sector >= zone->zone_start + zone->sectors)
zone++;
-
- sect_in_chunk = bio->bi_sector & ((chunk_size<<1) -1);
+
+ sect_in_chunk = bio->bi_sector & (chunk_sects - 1);
{
- sector_t x = (block - zone->zone_offset) >> chunksize_bits;
+ sector_t x = (sector - zone->zone_start) >> chunksect_bits;
sector_div(x, zone->nb_dev);
chunk = x;
- x = block >> chunksize_bits;
+ x = sector >> chunksect_bits;
tmp_dev = zone->dev[sector_div(x, zone->nb_dev)];
}
- rsect = (((chunk << chunksize_bits) + zone->dev_offset)<<1)
- + sect_in_chunk;
+ rsect = (chunk << chunksect_bits) + zone->dev_start + sect_in_chunk;
bio->bi_bdev = tmp_dev->bdev;
bio->bi_sector = rsect + tmp_dev->data_offset;
@@ -467,7 +465,7 @@
bad_map:
printk("raid0_make_request bug: can't convert block across chunks"
- " or bigger than %dk %llu %d\n", chunk_size,
+ " or bigger than %dk %llu %d\n", chunk_sects / 2,
(unsigned long long)bio->bi_sector, bio->bi_size >> 10);
bio_io_error(bio);
@@ -492,10 +490,10 @@
seq_printf(seq, "%s/", bdevname(
conf->strip_zone[j].dev[k]->bdev,b));
- seq_printf(seq, "] zo=%d do=%d s=%d\n",
- conf->strip_zone[j].zone_offset,
- conf->strip_zone[j].dev_offset,
- conf->strip_zone[j].size);
+ seq_printf(seq, "] zs=%d ds=%d s=%d\n",
+ conf->strip_zone[j].zone_start,
+ conf->strip_zone[j].dev_start,
+ conf->strip_zone[j].sectors);
}
#endif
seq_printf(seq, " %dk chunks", mddev->chunk_size/1024);
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 9c788e2..7b4f5f7 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -1016,12 +1016,16 @@
* else mark the drive as failed
*/
if (test_bit(In_sync, &rdev->flags)
- && (conf->raid_disks - mddev->degraded) == 1)
+ && (conf->raid_disks - mddev->degraded) == 1) {
/*
* Don't fail the drive, act as though we were just a
- * normal single drive
+ * normal single drive.
+ * However don't try a recovery from this drive as
+ * it is very likely to fail.
*/
+ mddev->recovery_disabled = 1;
return;
+ }
if (test_and_clear_bit(In_sync, &rdev->flags)) {
unsigned long flags;
spin_lock_irqsave(&conf->device_lock, flags);
@@ -1919,7 +1923,6 @@
int i, j, disk_idx;
mirror_info_t *disk;
mdk_rdev_t *rdev;
- struct list_head *tmp;
if (mddev->level != 1) {
printk("raid1: %s: raid level not set to mirroring (%d)\n",
@@ -1964,7 +1967,7 @@
spin_lock_init(&conf->device_lock);
mddev->queue->queue_lock = &conf->device_lock;
- rdev_for_each(rdev, tmp, mddev) {
+ list_for_each_entry(rdev, &mddev->disks, same_set) {
disk_idx = rdev->raid_disk;
if (disk_idx >= mddev->raid_disks
|| disk_idx < 0)
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index 970a96e..6736d6d 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -2025,7 +2025,6 @@
int i, disk_idx;
mirror_info_t *disk;
mdk_rdev_t *rdev;
- struct list_head *tmp;
int nc, fc, fo;
sector_t stride, size;
@@ -2108,7 +2107,7 @@
spin_lock_init(&conf->device_lock);
mddev->queue->queue_lock = &conf->device_lock;
- rdev_for_each(rdev, tmp, mddev) {
+ list_for_each_entry(rdev, &mddev->disks, same_set) {
disk_idx = rdev->raid_disk;
if (disk_idx >= mddev->raid_disks
|| disk_idx < 0)
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index a36a743..a5ba080 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -3998,7 +3998,6 @@
int raid_disk, memory;
mdk_rdev_t *rdev;
struct disk_info *disk;
- struct list_head *tmp;
int working_disks = 0;
if (mddev->level != 5 && mddev->level != 4 && mddev->level != 6) {
@@ -4108,7 +4107,7 @@
pr_debug("raid5: run(%s) called.\n", mdname(mddev));
- rdev_for_each(rdev, tmp, mddev) {
+ list_for_each_entry(rdev, &mddev->disks, same_set) {
raid_disk = rdev->raid_disk;
if (raid_disk >= conf->raid_disks
|| raid_disk < 0)
@@ -4533,7 +4532,6 @@
{
raid5_conf_t *conf = mddev_to_conf(mddev);
mdk_rdev_t *rdev;
- struct list_head *rtmp;
int spares = 0;
int added_devices = 0;
unsigned long flags;
@@ -4541,7 +4539,7 @@
if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery))
return -EBUSY;
- rdev_for_each(rdev, rtmp, mddev)
+ list_for_each_entry(rdev, &mddev->disks, same_set)
if (rdev->raid_disk < 0 &&
!test_bit(Faulty, &rdev->flags))
spares++;
@@ -4563,7 +4561,7 @@
/* Add some new drives, as many as will fit.
* We know there are enough to make the newly sized array work.
*/
- rdev_for_each(rdev, rtmp, mddev)
+ list_for_each_entry(rdev, &mddev->disks, same_set)
if (rdev->raid_disk < 0 &&
!test_bit(Faulty, &rdev->flags)) {
if (raid5_add_disk(mddev, rdev) == 0) {
diff --git a/drivers/media/dvb/dvb-core/dvb_net.c b/drivers/media/dvb/dvb-core/dvb_net.c
index 03fd9dd..f6ba846 100644
--- a/drivers/media/dvb/dvb-core/dvb_net.c
+++ b/drivers/media/dvb/dvb-core/dvb_net.c
@@ -125,7 +125,6 @@
struct dvb_net_priv {
int in_use;
- struct net_device_stats stats;
u16 pid;
struct net_device *net;
struct dvb_net *host;
@@ -384,8 +383,8 @@
if (priv->ule_skb) {
dev_kfree_skb( priv->ule_skb );
/* Prepare for next SNDU. */
- priv->stats.rx_errors++;
- priv->stats.rx_frame_errors++;
+ dev->stats.rx_errors++;
+ dev->stats.rx_frame_errors++;
}
reset_ule(priv);
priv->need_pusi = 1;
@@ -438,8 +437,8 @@
dev_kfree_skb( priv->ule_skb );
/* Prepare for next SNDU. */
// reset_ule(priv); moved to below.
- priv->stats.rx_errors++;
- priv->stats.rx_frame_errors++;
+ dev->stats.rx_errors++;
+ dev->stats.rx_frame_errors++;
}
reset_ule(priv);
/* skip to next PUSI. */
@@ -460,8 +459,8 @@
/* Drop partly decoded SNDU, reset state, resync on PUSI. */
if (priv->ule_skb) {
dev_kfree_skb( priv->ule_skb );
- priv->stats.rx_errors++;
- priv->stats.rx_frame_errors++;
+ dev->stats.rx_errors++;
+ dev->stats.rx_frame_errors++;
}
reset_ule(priv);
priv->need_pusi = 1;
@@ -477,8 +476,8 @@
if (priv->ule_sndu_remain > 183) {
/* Current SNDU lacks more data than there could be available in the
* current TS cell. */
- priv->stats.rx_errors++;
- priv->stats.rx_length_errors++;
+ dev->stats.rx_errors++;
+ dev->stats.rx_length_errors++;
printk(KERN_WARNING "%lu: Expected %d more SNDU bytes, but "
"got PUSI (pf %d, ts_remain %d). Flushing incomplete payload.\n",
priv->ts_count, priv->ule_sndu_remain, ts[4], ts_remain);
@@ -520,8 +519,8 @@
if (priv->ule_sndu_len < 5) {
printk(KERN_WARNING "%lu: Invalid ULE SNDU length %u. "
"Resyncing.\n", priv->ts_count, priv->ule_sndu_len);
- priv->stats.rx_errors++;
- priv->stats.rx_length_errors++;
+ dev->stats.rx_errors++;
+ dev->stats.rx_length_errors++;
priv->ule_sndu_len = 0;
priv->need_pusi = 1;
new_ts = 1;
@@ -573,7 +572,7 @@
if (priv->ule_skb == NULL) {
printk(KERN_NOTICE "%s: Memory squeeze, dropping packet.\n",
dev->name);
- priv->stats.rx_dropped++;
+ dev->stats.rx_dropped++;
return;
}
@@ -637,8 +636,8 @@
ule_dump = 1;
#endif
- priv->stats.rx_errors++;
- priv->stats.rx_crc_errors++;
+ dev->stats.rx_errors++;
+ dev->stats.rx_crc_errors++;
dev_kfree_skb(priv->ule_skb);
} else {
/* CRC32 verified OK. */
@@ -744,8 +743,8 @@
* receive the packet anyhow. */
/* if (priv->ule_dbit && skb->pkt_type == PACKET_OTHERHOST)
priv->ule_skb->pkt_type = PACKET_HOST; */
- priv->stats.rx_packets++;
- priv->stats.rx_bytes += priv->ule_skb->len;
+ dev->stats.rx_packets++;
+ dev->stats.rx_bytes += priv->ule_skb->len;
netif_rx(priv->ule_skb);
}
sndu_done:
@@ -800,8 +799,7 @@
{
u8 *eth;
struct sk_buff *skb;
- struct net_device_stats *stats =
- &((struct dvb_net_priv *) netdev_priv(dev))->stats;
+ struct net_device_stats *stats = &dev->stats;
int snap = 0;
/* note: pkt_len includes a 32bit checksum */
@@ -1216,28 +1214,29 @@
return dvb_net_feed_stop(dev);
}
-static struct net_device_stats * dvb_net_get_stats(struct net_device *dev)
-{
- return &((struct dvb_net_priv *) netdev_priv(dev))->stats;
-}
-
static const struct header_ops dvb_header_ops = {
.create = eth_header,
.parse = eth_header_parse,
.rebuild = eth_rebuild_header,
};
+
+static const struct net_device_ops dvb_netdev_ops = {
+ .ndo_open = dvb_net_open,
+ .ndo_stop = dvb_net_stop,
+ .ndo_start_xmit = dvb_net_tx,
+ .ndo_set_multicast_list = dvb_net_set_multicast_list,
+ .ndo_set_mac_address = dvb_net_set_mac,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
static void dvb_net_setup(struct net_device *dev)
{
ether_setup(dev);
dev->header_ops = &dvb_header_ops;
- dev->open = dvb_net_open;
- dev->stop = dvb_net_stop;
- dev->hard_start_xmit = dvb_net_tx;
- dev->get_stats = dvb_net_get_stats;
- dev->set_multicast_list = dvb_net_set_multicast_list;
- dev->set_mac_address = dvb_net_set_mac;
+ dev->netdev_ops = &dvb_netdev_ops;
dev->mtu = 4096;
dev->mc_count = 0;
diff --git a/drivers/message/fusion/mptctl.c b/drivers/message/fusion/mptctl.c
index b89f476..c638171 100644
--- a/drivers/message/fusion/mptctl.c
+++ b/drivers/message/fusion/mptctl.c
@@ -308,10 +308,11 @@
{
int rc = 1;
- dctlprintk(ioctl->ioc, printk(MYIOC_s_DEBUG_FMT ": Timeout Expired! Host %d\n",
- ioctl->ioc->name, ioctl->ioc->id));
if (ioctl == NULL)
return;
+ dctlprintk(ioctl->ioc,
+ printk(MYIOC_s_DEBUG_FMT ": Timeout Expired! Host %d\n",
+ ioctl->ioc->name, ioctl->ioc->id));
ioctl->wait_done = 0;
if (ioctl->reset & MPTCTL_RESET_OK)
diff --git a/drivers/message/fusion/mptlan.c b/drivers/message/fusion/mptlan.c
index a13f6ee..c2804f2 100644
--- a/drivers/message/fusion/mptlan.c
+++ b/drivers/message/fusion/mptlan.c
@@ -106,7 +106,6 @@
u32 total_posted;
u32 total_received;
- struct net_device_stats stats; /* Per device statistics */
struct delayed_work post_buckets_task;
struct net_device *dev;
@@ -548,15 +547,6 @@
}
/*=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=*/
-static struct net_device_stats *
-mpt_lan_get_stats(struct net_device *dev)
-{
- struct mpt_lan_priv *priv = netdev_priv(dev);
-
- return (struct net_device_stats *) &priv->stats;
-}
-
-/*=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=*/
static int
mpt_lan_change_mtu(struct net_device *dev, int new_mtu)
{
@@ -594,8 +584,8 @@
ctx = GET_LAN_BUFFER_CONTEXT(tmsg);
sent = priv->SendCtl[ctx].skb;
- priv->stats.tx_packets++;
- priv->stats.tx_bytes += sent->len;
+ dev->stats.tx_packets++;
+ dev->stats.tx_bytes += sent->len;
dioprintk((KERN_INFO MYNAM ": %s/%s: @%s, skb %p sent.\n",
IOC_AND_NETDEV_NAMES_s_s(dev),
@@ -636,7 +626,7 @@
switch (le16_to_cpu(pSendRep->IOCStatus) & MPI_IOCSTATUS_MASK) {
case MPI_IOCSTATUS_SUCCESS:
- priv->stats.tx_packets += count;
+ dev->stats.tx_packets += count;
break;
case MPI_IOCSTATUS_LAN_CANCELED:
@@ -644,13 +634,13 @@
break;
case MPI_IOCSTATUS_INVALID_SGL:
- priv->stats.tx_errors += count;
+ dev->stats.tx_errors += count;
printk (KERN_ERR MYNAM ": %s/%s: ERROR - Invalid SGL sent to IOC!\n",
IOC_AND_NETDEV_NAMES_s_s(dev));
goto out;
default:
- priv->stats.tx_errors += count;
+ dev->stats.tx_errors += count;
break;
}
@@ -661,7 +651,7 @@
ctx = GET_LAN_BUFFER_CONTEXT(le32_to_cpu(*pContext));
sent = priv->SendCtl[ctx].skb;
- priv->stats.tx_bytes += sent->len;
+ dev->stats.tx_bytes += sent->len;
dioprintk((KERN_INFO MYNAM ": %s/%s: @%s, skb %p sent.\n",
IOC_AND_NETDEV_NAMES_s_s(dev),
@@ -842,8 +832,8 @@
"delivered to upper level.\n",
IOC_AND_NETDEV_NAMES_s_s(dev), skb->len));
- priv->stats.rx_bytes += skb->len;
- priv->stats.rx_packets++;
+ dev->stats.rx_bytes += skb->len;
+ dev->stats.rx_packets++;
skb->dev = dev;
netif_rx(skb);
@@ -1308,6 +1298,14 @@
post_buckets_task.work));
}
+static const struct net_device_ops mpt_netdev_ops = {
+ .ndo_open = mpt_lan_open,
+ .ndo_stop = mpt_lan_close,
+ .ndo_start_xmit = mpt_lan_sdu_send,
+ .ndo_change_mtu = mpt_lan_change_mtu,
+ .ndo_tx_timeout = mpt_lan_tx_timeout,
+};
+
/*=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=*/
static struct net_device *
mpt_register_lan_device (MPT_ADAPTER *mpt_dev, int pnum)
@@ -1372,15 +1370,7 @@
priv->tx_max_out = (tx_max_out_p <= MPT_TX_MAX_OUT_LIM) ?
tx_max_out_p : MPT_TX_MAX_OUT_LIM;
- dev->open = mpt_lan_open;
- dev->stop = mpt_lan_close;
- dev->get_stats = mpt_lan_get_stats;
- dev->set_multicast_list = NULL;
- dev->change_mtu = mpt_lan_change_mtu;
- dev->hard_start_xmit = mpt_lan_sdu_send;
-
-/* Not in 2.3.42. Need 2.3.45+ */
- dev->tx_timeout = mpt_lan_tx_timeout;
+ dev->netdev_ops = &mpt_netdev_ops;
dev->watchdog_timeo = MPT_LAN_TX_TIMEOUT;
dlprintk((KERN_INFO MYNAM ": Finished registering dev "
diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig
index fee7304..419c378 100644
--- a/drivers/misc/Kconfig
+++ b/drivers/misc/Kconfig
@@ -120,7 +120,7 @@
cards are supported via 'MMC/SD Card support: TI Flash Media MMC/SD
Interface support (MMC_TIFM_SD)'.
- To compile this driver as a module, choose M here: the module will
+ To compile this driver as a module, choose M here: the module will
be called tifm_core.
config TIFM_7XX1
@@ -133,100 +133,9 @@
To make actual use of the device, you will have to select some
flash card format drivers, as outlined in the TIFM_CORE Help.
- To compile this driver as a module, choose M here: the module will
+ To compile this driver as a module, choose M here: the module will
be called tifm_7xx1.
-config ACER_WMI
- tristate "Acer WMI Laptop Extras (EXPERIMENTAL)"
- depends on X86
- depends on EXPERIMENTAL
- depends on ACPI
- depends on LEDS_CLASS
- depends on NEW_LEDS
- depends on BACKLIGHT_CLASS_DEVICE
- depends on SERIO_I8042
- depends on RFKILL
- select ACPI_WMI
- ---help---
- This is a driver for newer Acer (and Wistron) laptops. It adds
- wireless radio and bluetooth control, and on some laptops,
- exposes the mail LED and LCD backlight.
-
- For more information about this driver see
- <file:Documentation/laptops/acer-wmi.txt>
-
- If you have an ACPI-WMI compatible Acer/ Wistron laptop, say Y or M
- here.
-
-config ASUS_LAPTOP
- tristate "Asus Laptop Extras (EXPERIMENTAL)"
- depends on X86
- depends on ACPI
- depends on EXPERIMENTAL && !ACPI_ASUS
- depends on LEDS_CLASS
- depends on NEW_LEDS
- depends on BACKLIGHT_CLASS_DEVICE
- ---help---
- This is the new Linux driver for Asus laptops. It may also support some
- MEDION, JVC or VICTOR laptops. It makes all the extra buttons generate
- standard ACPI events that go through /proc/acpi/events. It also adds
- support for video output switching, LCD backlight control, Bluetooth and
- Wlan control, and most importantly, allows you to blink those fancy LEDs.
-
- For more information and a userspace daemon for handling the extra
- buttons see <http://acpi4asus.sf.net/>.
-
- If you have an ACPI-compatible ASUS laptop, say Y or M here.
-
-config FUJITSU_LAPTOP
- tristate "Fujitsu Laptop Extras"
- depends on X86
- depends on ACPI
- depends on INPUT
- depends on BACKLIGHT_CLASS_DEVICE
- ---help---
- This is a driver for laptops built by Fujitsu:
-
- * P2xxx/P5xxx/S6xxx/S7xxx series Lifebooks
- * Possibly other Fujitsu laptop models
- * Tested with S6410 and S7020
-
- It adds support for LCD brightness control and some hotkeys.
-
- If you have a Fujitsu laptop, say Y or M here.
-
-config FUJITSU_LAPTOP_DEBUG
- bool "Verbose debug mode for Fujitsu Laptop Extras"
- depends on FUJITSU_LAPTOP
- default n
- ---help---
- Enables extra debug output from the fujitsu extras driver, at the
- expense of a slight increase in driver size.
-
- If you are not sure, say N here.
-
-config TC1100_WMI
- tristate "HP Compaq TC1100 Tablet WMI Extras (EXPERIMENTAL)"
- depends on X86 && !X86_64
- depends on EXPERIMENTAL
- depends on ACPI
- select ACPI_WMI
- ---help---
- This is a driver for the WMI extensions (wireless and bluetooth power
- control) of the HP Compaq TC1100 tablet.
-
-config HP_WMI
- tristate "HP WMI extras"
- depends on ACPI_WMI
- depends on INPUT
- depends on RFKILL
- help
- Say Y here if you want to support WMI-based hotkeys on HP laptops and
- to read data from WMI such as docking or ambient light sensor state.
-
- To compile this driver as a module, choose M here: the module will
- be called hp-wmi.
-
config ICS932S401
tristate "Integrated Circuits ICS932S401"
depends on I2C && EXPERIMENTAL
@@ -237,170 +146,6 @@
This driver can also be built as a module. If so, the module
will be called ics932s401.
-config MSI_LAPTOP
- tristate "MSI Laptop Extras"
- depends on X86
- depends on ACPI
- depends on BACKLIGHT_CLASS_DEVICE
- ---help---
- This is a driver for laptops built by MSI (MICRO-STAR
- INTERNATIONAL):
-
- MSI MegaBook S270 (MS-1013)
- Cytron/TCM/Medion/Tchibo MD96100/SAM2000
-
- It adds support for Bluetooth, WLAN and LCD brightness control.
-
- More information about this driver is available at
- <http://0pointer.de/lennart/tchibo.html>.
-
- If you have an MSI S270 laptop, say Y or M here.
-
-config PANASONIC_LAPTOP
- tristate "Panasonic Laptop Extras"
- depends on X86 && INPUT && ACPI
- depends on BACKLIGHT_CLASS_DEVICE
- ---help---
- This driver adds support for access to backlight control and hotkeys
- on Panasonic Let's Note laptops.
-
- If you have a Panasonic Let's note laptop (such as the R1(N variant),
- R2, R3, R5, T2, W2 and Y2 series), say Y.
-
-config COMPAL_LAPTOP
- tristate "Compal Laptop Extras"
- depends on X86
- depends on ACPI
- depends on BACKLIGHT_CLASS_DEVICE
- ---help---
- This is a driver for laptops built by Compal:
-
- Compal FL90/IFL90
- Compal FL91/IFL91
- Compal FL92/JFL92
- Compal FT00/IFT00
-
- It adds support for Bluetooth, WLAN and LCD brightness control.
-
- If you have an Compal FL9x/IFL9x/FT00 laptop, say Y or M here.
-
-config SONY_LAPTOP
- tristate "Sony Laptop Extras"
- depends on X86 && ACPI
- select BACKLIGHT_CLASS_DEVICE
- depends on INPUT
- ---help---
- This mini-driver drives the SNC and SPIC devices present in the ACPI
- BIOS of the Sony Vaio laptops.
-
- It gives access to some extra laptop functionalities like Bluetooth,
- screen brightness control, Fn keys and allows powering on/off some
- devices.
-
- Read <file:Documentation/laptops/sony-laptop.txt> for more information.
-
-config SONYPI_COMPAT
- bool "Sonypi compatibility"
- depends on SONY_LAPTOP
- ---help---
- Build the sonypi driver compatibility code into the sony-laptop driver.
-
-config THINKPAD_ACPI
- tristate "ThinkPad ACPI Laptop Extras"
- depends on X86 && ACPI
- select BACKLIGHT_LCD_SUPPORT
- select BACKLIGHT_CLASS_DEVICE
- select HWMON
- select NVRAM
- select INPUT
- select NEW_LEDS
- select LEDS_CLASS
- select NET
- select RFKILL
- ---help---
- This is a driver for the IBM and Lenovo ThinkPad laptops. It adds
- support for Fn-Fx key combinations, Bluetooth control, video
- output switching, ThinkLight control, UltraBay eject and more.
- For more information about this driver see
- <file:Documentation/laptops/thinkpad-acpi.txt> and
- <http://ibm-acpi.sf.net/> .
-
- This driver was formerly known as ibm-acpi.
-
- If you have an IBM or Lenovo ThinkPad laptop, say Y or M here.
-
-config THINKPAD_ACPI_DEBUG
- bool "Verbose debug mode"
- depends on THINKPAD_ACPI
- default n
- ---help---
- Enables extra debugging information, at the expense of a slightly
- increase in driver size.
-
- If you are not sure, say N here.
-
-config THINKPAD_ACPI_DOCK
- bool "Legacy Docking Station Support"
- depends on THINKPAD_ACPI
- depends on ACPI_DOCK=n
- default n
- ---help---
- Allows the thinkpad_acpi driver to handle docking station events.
- This support was made obsolete by the generic ACPI docking station
- support (CONFIG_ACPI_DOCK). It will allow locking and removing the
- laptop from the docking station, but will not properly connect PCI
- devices.
-
- If you are not sure, say N here.
-
-config THINKPAD_ACPI_BAY
- bool "Legacy Removable Bay Support"
- depends on THINKPAD_ACPI
- default y
- ---help---
- Allows the thinkpad_acpi driver to handle removable bays. It will
- electrically disable the device in the bay, and also generate
- notifications when the bay lever is ejected or inserted.
-
- If you are not sure, say Y here.
-
-config THINKPAD_ACPI_VIDEO
- bool "Video output control support"
- depends on THINKPAD_ACPI
- default y
- ---help---
- Allows the thinkpad_acpi driver to provide an interface to control
- the various video output ports.
-
- This feature often won't work well, depending on ThinkPad model,
- display state, video output devices in use, whether there is a X
- server running, phase of the moon, and the current mood of
- Schroedinger's cat. If you can use X.org's RandR to control
- your ThinkPad's video output ports instead of this feature,
- don't think twice: do it and say N here to save some memory.
-
- If you are not sure, say Y here.
-
-config THINKPAD_ACPI_HOTKEY_POLL
- bool "Support NVRAM polling for hot keys"
- depends on THINKPAD_ACPI
- default y
- ---help---
- Some thinkpad models benefit from NVRAM polling to detect a few of
- the hot key press events. If you know your ThinkPad model does not
- need to do NVRAM polling to support any of the hot keys you use,
- unselecting this option will save about 1kB of memory.
-
- ThinkPads T40 and newer, R52 and newer, and X31 and newer are
- unlikely to need NVRAM polling in their latest BIOS versions.
-
- NVRAM polling can detect at most the following keys: ThinkPad/Access
- IBM, Zoom, Switch Display (fn+F7), ThinkLight, Volume up/down/mute,
- Brightness up/down, Display Expand (fn+F8), Hibernate (fn+F12).
-
- If you are not sure, say Y here. The driver enables polling only if
- it is strictly necessary to do so.
-
config ATMEL_SSC
tristate "Device driver for Atmel SSC peripheral"
depends on AVR32 || ARCH_AT91
@@ -413,31 +158,6 @@
If unsure, say N.
-config INTEL_MENLOW
- tristate "Thermal Management driver for Intel menlow platform"
- depends on ACPI_THERMAL
- select THERMAL
- depends on X86
- ---help---
- ACPI thermal management enhancement driver on
- Intel Menlow platform.
-
- If unsure, say N.
-
-config EEEPC_LAPTOP
- tristate "Eee PC Hotkey Driver (EXPERIMENTAL)"
- depends on X86
- depends on ACPI
- depends on BACKLIGHT_CLASS_DEVICE
- depends on HWMON
- depends on EXPERIMENTAL
- depends on RFKILL
- ---help---
- This driver supports the Fn-Fx keys on Eee PC laptops.
- It also adds the ability to switch camera/wlan on/off.
-
- If you have an Eee PC laptop, say Y or M here.
-
config ENCLOSURE_SERVICES
tristate "Enclosure Services"
default n
@@ -498,6 +218,18 @@
This option enables addition debugging code for the SGI GRU driver. If
you are unsure, say N.
+config DELL_LAPTOP
+ tristate "Dell Laptop Extras (EXPERIMENTAL)"
+ depends on X86
+ depends on DCDBAS
+ depends on EXPERIMENTAL
+ depends on BACKLIGHT_CLASS_DEVICE
+ depends on RFKILL
+ default n
+ ---help---
+ This driver adds support for rfkill and backlight control to Dell
+ laptops.
+
source "drivers/misc/c2port/Kconfig"
endif # MISC_DEVICES
diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile
index 817f7f5..9cf8ae6 100644
--- a/drivers/misc/Makefile
+++ b/drivers/misc/Makefile
@@ -1,32 +1,20 @@
#
# Makefile for misc devices that really don't fit anywhere else.
#
-obj- := misc.o # Dummy rule to force built-in.o to be made
obj-$(CONFIG_IBM_ASM) += ibmasm/
obj-$(CONFIG_HDPU_FEATURES) += hdpuftrs/
-obj-$(CONFIG_ASUS_LAPTOP) += asus-laptop.o
-obj-$(CONFIG_EEEPC_LAPTOP) += eeepc-laptop.o
-obj-$(CONFIG_MSI_LAPTOP) += msi-laptop.o
-obj-$(CONFIG_COMPAL_LAPTOP) += compal-laptop.o
-obj-$(CONFIG_ACER_WMI) += acer-wmi.o
obj-$(CONFIG_ATMEL_PWM) += atmel_pwm.o
obj-$(CONFIG_ATMEL_SSC) += atmel-ssc.o
obj-$(CONFIG_ATMEL_TCLIB) += atmel_tclib.o
-obj-$(CONFIG_HP_WMI) += hp-wmi.o
obj-$(CONFIG_ICS932S401) += ics932s401.o
-obj-$(CONFIG_TC1100_WMI) += tc1100-wmi.o
obj-$(CONFIG_LKDTM) += lkdtm.o
obj-$(CONFIG_TIFM_CORE) += tifm_core.o
+obj-$(CONFIG_DELL_LAPTOP) += dell-laptop.o
obj-$(CONFIG_TIFM_7XX1) += tifm_7xx1.o
obj-$(CONFIG_PHANTOM) += phantom.o
obj-$(CONFIG_SGI_IOC4) += ioc4.o
-obj-$(CONFIG_SONY_LAPTOP) += sony-laptop.o
-obj-$(CONFIG_THINKPAD_ACPI) += thinkpad_acpi.o
-obj-$(CONFIG_FUJITSU_LAPTOP) += fujitsu-laptop.o
-obj-$(CONFIG_PANASONIC_LAPTOP) += panasonic-laptop.o
obj-$(CONFIG_EEPROM_93CX6) += eeprom_93cx6.o
-obj-$(CONFIG_INTEL_MENLOW) += intel_menlow.o
obj-$(CONFIG_ENCLOSURE_SERVICES) += enclosure.o
obj-$(CONFIG_KGDB_TESTS) += kgdbts.o
obj-$(CONFIG_SGI_XP) += sgi-xp/
diff --git a/drivers/misc/dell-laptop.c b/drivers/misc/dell-laptop.c
new file mode 100644
index 0000000..4d33a20
--- /dev/null
+++ b/drivers/misc/dell-laptop.c
@@ -0,0 +1,436 @@
+/*
+ * Driver for Dell laptop extras
+ *
+ * Copyright (c) Red Hat <mjg@redhat.com>
+ *
+ * Based on documentation in the libsmbios package, Copyright (C) 2005 Dell
+ * Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/platform_device.h>
+#include <linux/backlight.h>
+#include <linux/err.h>
+#include <linux/dmi.h>
+#include <linux/io.h>
+#include <linux/rfkill.h>
+#include <linux/power_supply.h>
+#include <linux/acpi.h>
+#include "../firmware/dcdbas.h"
+
+#define BRIGHTNESS_TOKEN 0x7d
+
+/* This structure will be modified by the firmware when we enter
+ * system management mode, hence the volatiles */
+
+struct calling_interface_buffer {
+ u16 class;
+ u16 select;
+ volatile u32 input[4];
+ volatile u32 output[4];
+} __packed;
+
+struct calling_interface_token {
+ u16 tokenID;
+ u16 location;
+ union {
+ u16 value;
+ u16 stringlength;
+ };
+};
+
+struct calling_interface_structure {
+ struct dmi_header header;
+ u16 cmdIOAddress;
+ u8 cmdIOCode;
+ u32 supportedCmds;
+ struct calling_interface_token tokens[];
+} __packed;
+
+static int da_command_address;
+static int da_command_code;
+static int da_num_tokens;
+static struct calling_interface_token *da_tokens;
+
+static struct backlight_device *dell_backlight_device;
+static struct rfkill *wifi_rfkill;
+static struct rfkill *bluetooth_rfkill;
+static struct rfkill *wwan_rfkill;
+
+static const struct dmi_system_id __initdata dell_device_table[] = {
+ {
+ .ident = "Dell laptop",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
+ DMI_MATCH(DMI_CHASSIS_TYPE, "8"),
+ },
+ },
+ { }
+};
+
+static void parse_da_table(const struct dmi_header *dm)
+{
+ /* Final token is a terminator, so we don't want to copy it */
+ int tokens = (dm->length-11)/sizeof(struct calling_interface_token)-1;
+ struct calling_interface_structure *table =
+ container_of(dm, struct calling_interface_structure, header);
+
+ /* 4 bytes of table header, plus 7 bytes of Dell header, plus at least
+ 6 bytes of entry */
+
+ if (dm->length < 17)
+ return;
+
+ da_command_address = table->cmdIOAddress;
+ da_command_code = table->cmdIOCode;
+
+ da_tokens = krealloc(da_tokens, (da_num_tokens + tokens) *
+ sizeof(struct calling_interface_token),
+ GFP_KERNEL);
+
+ if (!da_tokens)
+ return;
+
+ memcpy(da_tokens+da_num_tokens, table->tokens,
+ sizeof(struct calling_interface_token) * tokens);
+
+ da_num_tokens += tokens;
+}
+
+static void find_tokens(const struct dmi_header *dm)
+{
+ switch (dm->type) {
+ case 0xd4: /* Indexed IO */
+ break;
+ case 0xd5: /* Protected Area Type 1 */
+ break;
+ case 0xd6: /* Protected Area Type 2 */
+ break;
+ case 0xda: /* Calling interface */
+ parse_da_table(dm);
+ break;
+ }
+}
+
+static int find_token_location(int tokenid)
+{
+ int i;
+ for (i = 0; i < da_num_tokens; i++) {
+ if (da_tokens[i].tokenID == tokenid)
+ return da_tokens[i].location;
+ }
+
+ return -1;
+}
+
+static struct calling_interface_buffer *
+dell_send_request(struct calling_interface_buffer *buffer, int class,
+ int select)
+{
+ struct smi_cmd command;
+
+ command.magic = SMI_CMD_MAGIC;
+ command.command_address = da_command_address;
+ command.command_code = da_command_code;
+ command.ebx = virt_to_phys(buffer);
+ command.ecx = 0x42534931;
+
+ buffer->class = class;
+ buffer->select = select;
+
+ dcdbas_smi_request(&command);
+
+ return buffer;
+}
+
+/* Derived from information in DellWirelessCtl.cpp:
+ Class 17, select 11 is radio control. It returns an array of 32-bit values.
+
+ result[0]: return code
+ result[1]:
+ Bit 0: Hardware switch supported
+ Bit 1: Wifi locator supported
+ Bit 2: Wifi is supported
+ Bit 3: Bluetooth is supported
+ Bit 4: WWAN is supported
+ Bit 5: Wireless keyboard supported
+ Bits 6-7: Reserved
+ Bit 8: Wifi is installed
+ Bit 9: Bluetooth is installed
+ Bit 10: WWAN is installed
+ Bits 11-15: Reserved
+ Bit 16: Hardware switch is on
+ Bit 17: Wifi is blocked
+ Bit 18: Bluetooth is blocked
+ Bit 19: WWAN is blocked
+ Bits 20-31: Reserved
+ result[2]: NVRAM size in bytes
+ result[3]: NVRAM format version number
+*/
+
+static int dell_rfkill_set(int radio, enum rfkill_state state)
+{
+ struct calling_interface_buffer buffer;
+ int disable = (state == RFKILL_STATE_UNBLOCKED) ? 0 : 1;
+
+ memset(&buffer, 0, sizeof(struct calling_interface_buffer));
+ buffer.input[0] = (1 | (radio<<8) | (disable << 16));
+ dell_send_request(&buffer, 17, 11);
+
+ return 0;
+}
+
+static int dell_wifi_set(void *data, enum rfkill_state state)
+{
+ return dell_rfkill_set(1, state);
+}
+
+static int dell_bluetooth_set(void *data, enum rfkill_state state)
+{
+ return dell_rfkill_set(2, state);
+}
+
+static int dell_wwan_set(void *data, enum rfkill_state state)
+{
+ return dell_rfkill_set(3, state);
+}
+
+static int dell_rfkill_get(int bit, enum rfkill_state *state)
+{
+ struct calling_interface_buffer buffer;
+ int status;
+ int new_state = RFKILL_STATE_HARD_BLOCKED;
+
+ memset(&buffer, 0, sizeof(struct calling_interface_buffer));
+ dell_send_request(&buffer, 17, 11);
+ status = buffer.output[1];
+
+ if (status & (1<<16))
+ new_state = RFKILL_STATE_SOFT_BLOCKED;
+
+ if (status & (1<<bit))
+ *state = new_state;
+ else
+ *state = RFKILL_STATE_UNBLOCKED;
+
+ return 0;
+}
+
+static int dell_wifi_get(void *data, enum rfkill_state *state)
+{
+ return dell_rfkill_get(17, state);
+}
+
+static int dell_bluetooth_get(void *data, enum rfkill_state *state)
+{
+ return dell_rfkill_get(18, state);
+}
+
+static int dell_wwan_get(void *data, enum rfkill_state *state)
+{
+ return dell_rfkill_get(19, state);
+}
+
+static int dell_setup_rfkill(void)
+{
+ struct calling_interface_buffer buffer;
+ int status;
+ int ret;
+
+ memset(&buffer, 0, sizeof(struct calling_interface_buffer));
+ dell_send_request(&buffer, 17, 11);
+ status = buffer.output[1];
+
+ if ((status & (1<<2|1<<8)) == (1<<2|1<<8)) {
+ wifi_rfkill = rfkill_allocate(NULL, RFKILL_TYPE_WLAN);
+ if (!wifi_rfkill)
+ goto err_wifi;
+ wifi_rfkill->name = "dell-wifi";
+ wifi_rfkill->toggle_radio = dell_wifi_set;
+ wifi_rfkill->get_state = dell_wifi_get;
+ ret = rfkill_register(wifi_rfkill);
+ if (ret)
+ goto err_wifi;
+ }
+
+ if ((status & (1<<3|1<<9)) == (1<<3|1<<9)) {
+ bluetooth_rfkill = rfkill_allocate(NULL, RFKILL_TYPE_BLUETOOTH);
+ if (!bluetooth_rfkill)
+ goto err_bluetooth;
+ bluetooth_rfkill->name = "dell-bluetooth";
+ bluetooth_rfkill->toggle_radio = dell_bluetooth_set;
+ bluetooth_rfkill->get_state = dell_bluetooth_get;
+ ret = rfkill_register(bluetooth_rfkill);
+ if (ret)
+ goto err_bluetooth;
+ }
+
+ if ((status & (1<<4|1<<10)) == (1<<4|1<<10)) {
+ wwan_rfkill = rfkill_allocate(NULL, RFKILL_TYPE_WWAN);
+ if (!wwan_rfkill)
+ goto err_wwan;
+ wwan_rfkill->name = "dell-wwan";
+ wwan_rfkill->toggle_radio = dell_wwan_set;
+ wwan_rfkill->get_state = dell_wwan_get;
+ ret = rfkill_register(wwan_rfkill);
+ if (ret)
+ goto err_wwan;
+ }
+
+ return 0;
+err_wwan:
+ if (wwan_rfkill)
+ rfkill_free(wwan_rfkill);
+ if (bluetooth_rfkill) {
+ rfkill_unregister(bluetooth_rfkill);
+ bluetooth_rfkill = NULL;
+ }
+err_bluetooth:
+ if (bluetooth_rfkill)
+ rfkill_free(bluetooth_rfkill);
+ if (wifi_rfkill) {
+ rfkill_unregister(wifi_rfkill);
+ wifi_rfkill = NULL;
+ }
+err_wifi:
+ if (wifi_rfkill)
+ rfkill_free(wifi_rfkill);
+
+ return ret;
+}
+
+static int dell_send_intensity(struct backlight_device *bd)
+{
+ struct calling_interface_buffer buffer;
+
+ memset(&buffer, 0, sizeof(struct calling_interface_buffer));
+ buffer.input[0] = find_token_location(BRIGHTNESS_TOKEN);
+ buffer.input[1] = bd->props.brightness;
+
+ if (buffer.input[0] == -1)
+ return -ENODEV;
+
+ if (power_supply_is_system_supplied() > 0)
+ dell_send_request(&buffer, 1, 2);
+ else
+ dell_send_request(&buffer, 1, 1);
+
+ return 0;
+}
+
+static int dell_get_intensity(struct backlight_device *bd)
+{
+ struct calling_interface_buffer buffer;
+
+ memset(&buffer, 0, sizeof(struct calling_interface_buffer));
+ buffer.input[0] = find_token_location(BRIGHTNESS_TOKEN);
+
+ if (buffer.input[0] == -1)
+ return -ENODEV;
+
+ if (power_supply_is_system_supplied() > 0)
+ dell_send_request(&buffer, 0, 2);
+ else
+ dell_send_request(&buffer, 0, 1);
+
+ return buffer.output[1];
+}
+
+static struct backlight_ops dell_ops = {
+ .get_brightness = dell_get_intensity,
+ .update_status = dell_send_intensity,
+};
+
+static int __init dell_init(void)
+{
+ struct calling_interface_buffer buffer;
+ int max_intensity = 0;
+ int ret;
+
+ if (!dmi_check_system(dell_device_table))
+ return -ENODEV;
+
+ dmi_walk(find_tokens);
+
+ if (!da_tokens) {
+ printk(KERN_INFO "dell-laptop: Unable to find dmi tokens\n");
+ return -ENODEV;
+ }
+
+ ret = dell_setup_rfkill();
+
+ if (ret) {
+ printk(KERN_WARNING "dell-laptop: Unable to setup rfkill\n");
+ goto out;
+ }
+
+#ifdef CONFIG_ACPI
+ /* In the event of an ACPI backlight being available, don't
+ * register the platform controller.
+ */
+ if (acpi_video_backlight_support())
+ return 0;
+#endif
+
+ memset(&buffer, 0, sizeof(struct calling_interface_buffer));
+ buffer.input[0] = find_token_location(BRIGHTNESS_TOKEN);
+
+ if (buffer.input[0] != -1) {
+ dell_send_request(&buffer, 0, 2);
+ max_intensity = buffer.output[3];
+ }
+
+ if (max_intensity) {
+ dell_backlight_device = backlight_device_register(
+ "dell_backlight",
+ NULL, NULL,
+ &dell_ops);
+
+ if (IS_ERR(dell_backlight_device)) {
+ ret = PTR_ERR(dell_backlight_device);
+ dell_backlight_device = NULL;
+ goto out;
+ }
+
+ dell_backlight_device->props.max_brightness = max_intensity;
+ dell_backlight_device->props.brightness =
+ dell_get_intensity(dell_backlight_device);
+ backlight_update_status(dell_backlight_device);
+ }
+
+ return 0;
+out:
+ if (wifi_rfkill)
+ rfkill_unregister(wifi_rfkill);
+ if (bluetooth_rfkill)
+ rfkill_unregister(bluetooth_rfkill);
+ if (wwan_rfkill)
+ rfkill_unregister(wwan_rfkill);
+ kfree(da_tokens);
+ return ret;
+}
+
+static void __exit dell_exit(void)
+{
+ backlight_device_unregister(dell_backlight_device);
+ if (wifi_rfkill)
+ rfkill_unregister(wifi_rfkill);
+ if (bluetooth_rfkill)
+ rfkill_unregister(bluetooth_rfkill);
+ if (wwan_rfkill)
+ rfkill_unregister(wwan_rfkill);
+}
+
+module_init(dell_init);
+module_exit(dell_exit);
+
+MODULE_AUTHOR("Matthew Garrett <mjg@redhat.com>");
+MODULE_DESCRIPTION("Dell laptop driver");
+MODULE_LICENSE("GPL");
+MODULE_ALIAS("dmi:*svnDellInc.:*:ct8:*");
diff --git a/drivers/misc/enclosure.c b/drivers/misc/enclosure.c
index 0736cff..3cf61ec 100644
--- a/drivers/misc/enclosure.c
+++ b/drivers/misc/enclosure.c
@@ -119,7 +119,7 @@
edev->edev.class = &enclosure_class;
edev->edev.parent = get_device(dev);
edev->cb = cb;
- snprintf(edev->edev.bus_id, BUS_ID_SIZE, "%s", name);
+ dev_set_name(&edev->edev, name);
err = device_register(&edev->edev);
if (err)
goto err;
@@ -170,7 +170,7 @@
static void enclosure_link_name(struct enclosure_component *cdev, char *name)
{
strcpy(name, "enclosure_device:");
- strcat(name, cdev->cdev.bus_id);
+ strcat(name, dev_name(&cdev->cdev));
}
static void enclosure_remove_links(struct enclosure_component *cdev)
@@ -256,9 +256,9 @@
cdev = &ecomp->cdev;
cdev->parent = get_device(&edev->edev);
if (name)
- snprintf(cdev->bus_id, BUS_ID_SIZE, "%s", name);
+ dev_set_name(cdev, name);
else
- snprintf(cdev->bus_id, BUS_ID_SIZE, "%u", number);
+ dev_set_name(cdev, "%u", number);
cdev->release = enclosure_component_release;
cdev->groups = enclosure_groups;
diff --git a/drivers/misc/sgi-xp/xpnet.c b/drivers/misc/sgi-xp/xpnet.c
index 81152b3..7957f52 100644
--- a/drivers/misc/sgi-xp/xpnet.c
+++ b/drivers/misc/sgi-xp/xpnet.c
@@ -95,11 +95,6 @@
atomic_t use_count;
};
-/* driver specific structure pointed to by the device structure */
-struct xpnet_dev_private {
- struct net_device_stats stats;
-};
-
struct net_device *xpnet_device;
/*
@@ -153,7 +148,6 @@
struct sk_buff *skb;
void *dst;
enum xp_retval ret;
- struct xpnet_dev_private *priv = netdev_priv(xpnet_device);
if (!XPNET_VALID_MSG(msg)) {
/*
@@ -161,7 +155,7 @@
*/
xpc_received(partid, channel, (void *)msg);
- priv->stats.rx_errors++;
+ xpnet_device->stats.rx_errors++;
return;
}
@@ -176,7 +170,7 @@
xpc_received(partid, channel, (void *)msg);
- priv->stats.rx_errors++;
+ xpnet_device->stats.rx_errors++;
return;
}
@@ -226,7 +220,7 @@
xpc_received(partid, channel, (void *)msg);
- priv->stats.rx_errors++;
+ xpnet_device->stats.rx_errors++;
return;
}
@@ -247,8 +241,8 @@
skb_end_pointer(skb), skb->len);
xpnet_device->last_rx = jiffies;
- priv->stats.rx_packets++;
- priv->stats.rx_bytes += skb->len + ETH_HLEN;
+ xpnet_device->stats.rx_packets++;
+ xpnet_device->stats.rx_bytes += skb->len + ETH_HLEN;
netif_rx_ni(skb);
xpc_received(partid, channel, (void *)msg);
@@ -353,26 +347,6 @@
}
/*
- * Required for the net_device structure.
- */
-static int
-xpnet_dev_set_config(struct net_device *dev, struct ifmap *new_map)
-{
- return 0;
-}
-
-/*
- * Return statistics to the caller.
- */
-static struct net_device_stats *
-xpnet_dev_get_stats(struct net_device *dev)
-{
- struct xpnet_dev_private *priv = netdev_priv(dev);
-
- return &priv->stats;
-}
-
-/*
* Notification that the other end has received the message and
* DMA'd the skb information. At this point, they are done with
* our side. When all recipients are done processing, we
@@ -453,7 +427,6 @@
struct xpnet_pending_msg *queued_msg;
u64 start_addr, end_addr;
short dest_partid;
- struct xpnet_dev_private *priv = netdev_priv(dev);
u16 embedded_bytes = 0;
dev_dbg(xpnet, ">skb->head=0x%p skb->data=0x%p skb->tail=0x%p "
@@ -476,7 +449,7 @@
dev_warn(xpnet, "failed to kmalloc %ld bytes; dropping "
"packet\n", sizeof(struct xpnet_pending_msg));
- priv->stats.tx_errors++;
+ dev->stats.tx_errors++;
return -ENOMEM;
}
@@ -526,8 +499,8 @@
kfree(queued_msg);
}
- priv->stats.tx_packets++;
- priv->stats.tx_bytes += skb->len;
+ dev->stats.tx_packets++;
+ dev->stats.tx_bytes += skb->len;
return 0;
}
@@ -538,12 +511,19 @@
static void
xpnet_dev_tx_timeout(struct net_device *dev)
{
- struct xpnet_dev_private *priv = netdev_priv(dev);
-
- priv->stats.tx_errors++;
- return;
+ dev->stats.tx_errors++;
}
+static const struct net_device_ops xpnet_netdev_ops = {
+ .ndo_open = xpnet_dev_open,
+ .ndo_stop = xpnet_dev_stop,
+ .ndo_start_xmit = xpnet_dev_hard_start_xmit,
+ .ndo_change_mtu = xpnet_dev_change_mtu,
+ .ndo_tx_timeout = xpnet_dev_tx_timeout,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
static int __init
xpnet_init(void)
{
@@ -563,8 +543,7 @@
* use ether_setup() to init the majority of our device
* structure and then override the necessary pieces.
*/
- xpnet_device = alloc_netdev(sizeof(struct xpnet_dev_private),
- XPNET_DEVICE_NAME, ether_setup);
+ xpnet_device = alloc_netdev(0, XPNET_DEVICE_NAME, ether_setup);
if (xpnet_device == NULL) {
kfree(xpnet_broadcast_partitions);
return -ENOMEM;
@@ -573,13 +552,6 @@
netif_carrier_off(xpnet_device);
xpnet_device->mtu = XPNET_DEF_MTU;
- xpnet_device->change_mtu = xpnet_dev_change_mtu;
- xpnet_device->open = xpnet_dev_open;
- xpnet_device->get_stats = xpnet_dev_get_stats;
- xpnet_device->stop = xpnet_dev_stop;
- xpnet_device->hard_start_xmit = xpnet_dev_hard_start_xmit;
- xpnet_device->tx_timeout = xpnet_dev_tx_timeout;
- xpnet_device->set_config = xpnet_dev_set_config;
/*
* Multicast assumes the LSB of the first octet is set for multicast
diff --git a/drivers/mmc/host/atmel-mci.c b/drivers/mmc/host/atmel-mci.c
index 1e97916..76bfe16 100644
--- a/drivers/mmc/host/atmel-mci.c
+++ b/drivers/mmc/host/atmel-mci.c
@@ -55,7 +55,6 @@
struct atmel_mci_dma {
#ifdef CONFIG_MMC_ATMELMCI_DMA
- struct dma_client client;
struct dma_chan *chan;
struct dma_async_tx_descriptor *data_desc;
#endif
@@ -593,10 +592,8 @@
/* If we don't have a channel, we can't do DMA */
chan = host->dma.chan;
- if (chan) {
- dma_chan_get(chan);
+ if (chan)
host->data_chan = chan;
- }
if (!chan)
return -ENODEV;
@@ -1443,60 +1440,6 @@
return IRQ_HANDLED;
}
-#ifdef CONFIG_MMC_ATMELMCI_DMA
-
-static inline struct atmel_mci *
-dma_client_to_atmel_mci(struct dma_client *client)
-{
- return container_of(client, struct atmel_mci, dma.client);
-}
-
-static enum dma_state_client atmci_dma_event(struct dma_client *client,
- struct dma_chan *chan, enum dma_state state)
-{
- struct atmel_mci *host;
- enum dma_state_client ret = DMA_NAK;
-
- host = dma_client_to_atmel_mci(client);
-
- switch (state) {
- case DMA_RESOURCE_AVAILABLE:
- spin_lock_bh(&host->lock);
- if (!host->dma.chan) {
- host->dma.chan = chan;
- ret = DMA_ACK;
- }
- spin_unlock_bh(&host->lock);
-
- if (ret == DMA_ACK)
- dev_info(&host->pdev->dev,
- "Using %s for DMA transfers\n",
- chan->dev.bus_id);
- break;
-
- case DMA_RESOURCE_REMOVED:
- spin_lock_bh(&host->lock);
- if (host->dma.chan == chan) {
- host->dma.chan = NULL;
- ret = DMA_ACK;
- }
- spin_unlock_bh(&host->lock);
-
- if (ret == DMA_ACK)
- dev_info(&host->pdev->dev,
- "Lost %s, falling back to PIO\n",
- chan->dev.bus_id);
- break;
-
- default:
- break;
- }
-
-
- return ret;
-}
-#endif /* CONFIG_MMC_ATMELMCI_DMA */
-
static int __init atmci_init_slot(struct atmel_mci *host,
struct mci_slot_pdata *slot_data, unsigned int id,
u32 sdc_reg)
@@ -1600,6 +1543,18 @@
mmc_free_host(slot->mmc);
}
+#ifdef CONFIG_MMC_ATMELMCI_DMA
+static bool filter(struct dma_chan *chan, void *slave)
+{
+ struct dw_dma_slave *dws = slave;
+
+ if (dws->dma_dev == chan->device->dev)
+ return true;
+ else
+ return false;
+}
+#endif
+
static int __init atmci_probe(struct platform_device *pdev)
{
struct mci_platform_data *pdata;
@@ -1652,22 +1607,20 @@
goto err_request_irq;
#ifdef CONFIG_MMC_ATMELMCI_DMA
- if (pdata->dma_slave) {
- struct dma_slave *slave = pdata->dma_slave;
+ if (pdata->dma_slave.dma_dev) {
+ struct dw_dma_slave *dws = &pdata->dma_slave;
+ dma_cap_mask_t mask;
- slave->tx_reg = regs->start + MCI_TDR;
- slave->rx_reg = regs->start + MCI_RDR;
+ dws->tx_reg = regs->start + MCI_TDR;
+ dws->rx_reg = regs->start + MCI_RDR;
/* Try to grab a DMA channel */
- host->dma.client.event_callback = atmci_dma_event;
- dma_cap_set(DMA_SLAVE, host->dma.client.cap_mask);
- host->dma.client.slave = slave;
-
- dma_async_client_register(&host->dma.client);
- dma_async_client_chan_request(&host->dma.client);
- } else {
- dev_notice(&pdev->dev, "DMA not available, using PIO\n");
+ dma_cap_zero(mask);
+ dma_cap_set(DMA_SLAVE, mask);
+ host->dma.chan = dma_request_channel(mask, filter, dws);
}
+ if (!host->dma.chan)
+ dev_notice(&pdev->dev, "DMA not available, using PIO\n");
#endif /* CONFIG_MMC_ATMELMCI_DMA */
platform_set_drvdata(pdev, host);
@@ -1699,8 +1652,8 @@
err_init_slot:
#ifdef CONFIG_MMC_ATMELMCI_DMA
- if (pdata->dma_slave)
- dma_async_client_unregister(&host->dma.client);
+ if (host->dma.chan)
+ dma_release_channel(host->dma.chan);
#endif
free_irq(irq, host);
err_request_irq:
@@ -1731,8 +1684,8 @@
clk_disable(host->mck);
#ifdef CONFIG_MMC_ATMELMCI_DMA
- if (host->dma.client.slave)
- dma_async_client_unregister(&host->dma.client);
+ if (host->dma.chan)
+ dma_release_channel(host->dma.chan);
#endif
free_irq(platform_get_irq(pdev, 0), host);
@@ -1761,7 +1714,7 @@
platform_driver_unregister(&atmci_driver);
}
-module_init(atmci_init);
+late_initcall(atmci_init); /* try to load after dma driver when built-in */
module_exit(atmci_exit);
MODULE_DESCRIPTION("Atmel Multimedia Card Interface driver");
diff --git a/drivers/mtd/Kconfig b/drivers/mtd/Kconfig
index a90d50c..7d04fb9 100644
--- a/drivers/mtd/Kconfig
+++ b/drivers/mtd/Kconfig
@@ -45,6 +45,14 @@
devices. Partitioning on NFTL 'devices' is a different - that's the
'normal' form of partitioning used on a block device.
+config MTD_TESTS
+ tristate "MTD tests support"
+ depends on m
+ help
+ This option includes various MTD tests into compilation. The tests
+ should normally be compiled as kernel modules. The modules perform
+ various checks and verifications when loaded.
+
config MTD_REDBOOT_PARTS
tristate "RedBoot partition table parsing"
depends on MTD_PARTITIONS
@@ -316,6 +324,8 @@
source "drivers/mtd/onenand/Kconfig"
+source "drivers/mtd/lpddr/Kconfig"
+
source "drivers/mtd/ubi/Kconfig"
endif # MTD
diff --git a/drivers/mtd/Makefile b/drivers/mtd/Makefile
index 4b77335..4521b1e 100644
--- a/drivers/mtd/Makefile
+++ b/drivers/mtd/Makefile
@@ -29,6 +29,6 @@
nftl-objs := nftlcore.o nftlmount.o
inftl-objs := inftlcore.o inftlmount.o
-obj-y += chips/ maps/ devices/ nand/ onenand/
+obj-y += chips/ lpddr/ maps/ devices/ nand/ onenand/ tests/
obj-$(CONFIG_MTD_UBI) += ubi/
diff --git a/drivers/mtd/chips/cfi_cmdset_0001.c b/drivers/mtd/chips/cfi_cmdset_0001.c
index c93a8be..f5ab6fa 100644
--- a/drivers/mtd/chips/cfi_cmdset_0001.c
+++ b/drivers/mtd/chips/cfi_cmdset_0001.c
@@ -58,8 +58,8 @@
static int cfi_intelext_writev(struct mtd_info *, const struct kvec *, unsigned long, loff_t, size_t *);
static int cfi_intelext_erase_varsize(struct mtd_info *, struct erase_info *);
static void cfi_intelext_sync (struct mtd_info *);
-static int cfi_intelext_lock(struct mtd_info *mtd, loff_t ofs, size_t len);
-static int cfi_intelext_unlock(struct mtd_info *mtd, loff_t ofs, size_t len);
+static int cfi_intelext_lock(struct mtd_info *mtd, loff_t ofs, uint64_t len);
+static int cfi_intelext_unlock(struct mtd_info *mtd, loff_t ofs, uint64_t len);
#ifdef CONFIG_MTD_OTP
static int cfi_intelext_read_fact_prot_reg (struct mtd_info *, loff_t, size_t, size_t *, u_char *);
static int cfi_intelext_read_user_prot_reg (struct mtd_info *, loff_t, size_t, size_t *, u_char *);
@@ -558,8 +558,8 @@
}
for (i=0; i<mtd->numeraseregions;i++){
- printk(KERN_DEBUG "erase region %d: offset=0x%x,size=0x%x,blocks=%d\n",
- i,mtd->eraseregions[i].offset,
+ printk(KERN_DEBUG "erase region %d: offset=0x%llx,size=0x%x,blocks=%d\n",
+ i,(unsigned long long)mtd->eraseregions[i].offset,
mtd->eraseregions[i].erasesize,
mtd->eraseregions[i].numblocks);
}
@@ -2058,7 +2058,7 @@
return ret;
}
-static int cfi_intelext_lock(struct mtd_info *mtd, loff_t ofs, size_t len)
+static int cfi_intelext_lock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
{
int ret;
@@ -2082,7 +2082,7 @@
return ret;
}
-static int cfi_intelext_unlock(struct mtd_info *mtd, loff_t ofs, size_t len)
+static int cfi_intelext_unlock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
{
int ret;
diff --git a/drivers/mtd/chips/cfi_cmdset_0002.c b/drivers/mtd/chips/cfi_cmdset_0002.c
index d74ec46..94bb61e 100644
--- a/drivers/mtd/chips/cfi_cmdset_0002.c
+++ b/drivers/mtd/chips/cfi_cmdset_0002.c
@@ -71,8 +71,8 @@
static void put_chip(struct map_info *map, struct flchip *chip, unsigned long adr);
#include "fwh_lock.h"
-static int cfi_atmel_lock(struct mtd_info *mtd, loff_t ofs, size_t len);
-static int cfi_atmel_unlock(struct mtd_info *mtd, loff_t ofs, size_t len);
+static int cfi_atmel_lock(struct mtd_info *mtd, loff_t ofs, uint64_t len);
+static int cfi_atmel_unlock(struct mtd_info *mtd, loff_t ofs, uint64_t len);
static struct mtd_chip_driver cfi_amdstd_chipdrv = {
.probe = NULL, /* Not usable directly */
@@ -322,6 +322,14 @@
};
+static void cfi_fixup_major_minor(struct cfi_private *cfi,
+ struct cfi_pri_amdstd *extp)
+{
+ if (cfi->mfr == CFI_MFR_SAMSUNG && cfi->id == 0x257e &&
+ extp->MajorVersion == '0')
+ extp->MajorVersion = '1';
+}
+
struct mtd_info *cfi_cmdset_0002(struct map_info *map, int primary)
{
struct cfi_private *cfi = map->fldrv_priv;
@@ -363,6 +371,8 @@
return NULL;
}
+ cfi_fixup_major_minor(cfi, extp);
+
if (extp->MajorVersion != '1' ||
(extp->MinorVersion < '0' || extp->MinorVersion > '4')) {
printk(KERN_ERR " Unknown Amd/Fujitsu Extended Query "
@@ -1774,12 +1784,12 @@
return ret;
}
-static int cfi_atmel_lock(struct mtd_info *mtd, loff_t ofs, size_t len)
+static int cfi_atmel_lock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
{
return cfi_varsize_frob(mtd, do_atmel_lock, ofs, len, NULL);
}
-static int cfi_atmel_unlock(struct mtd_info *mtd, loff_t ofs, size_t len)
+static int cfi_atmel_unlock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
{
return cfi_varsize_frob(mtd, do_atmel_unlock, ofs, len, NULL);
}
diff --git a/drivers/mtd/chips/cfi_cmdset_0020.c b/drivers/mtd/chips/cfi_cmdset_0020.c
index d4714dd..6c740f3 100644
--- a/drivers/mtd/chips/cfi_cmdset_0020.c
+++ b/drivers/mtd/chips/cfi_cmdset_0020.c
@@ -42,8 +42,8 @@
unsigned long count, loff_t to, size_t *retlen);
static int cfi_staa_erase_varsize(struct mtd_info *, struct erase_info *);
static void cfi_staa_sync (struct mtd_info *);
-static int cfi_staa_lock(struct mtd_info *mtd, loff_t ofs, size_t len);
-static int cfi_staa_unlock(struct mtd_info *mtd, loff_t ofs, size_t len);
+static int cfi_staa_lock(struct mtd_info *mtd, loff_t ofs, uint64_t len);
+static int cfi_staa_unlock(struct mtd_info *mtd, loff_t ofs, uint64_t len);
static int cfi_staa_suspend (struct mtd_info *);
static void cfi_staa_resume (struct mtd_info *);
@@ -221,8 +221,8 @@
}
for (i=0; i<mtd->numeraseregions;i++){
- printk(KERN_DEBUG "%d: offset=0x%x,size=0x%x,blocks=%d\n",
- i,mtd->eraseregions[i].offset,
+ printk(KERN_DEBUG "%d: offset=0x%llx,size=0x%x,blocks=%d\n",
+ i, (unsigned long long)mtd->eraseregions[i].offset,
mtd->eraseregions[i].erasesize,
mtd->eraseregions[i].numblocks);
}
@@ -964,7 +964,7 @@
adr += regions[i].erasesize;
len -= regions[i].erasesize;
- if (adr % (1<< cfi->chipshift) == ((regions[i].offset + (regions[i].erasesize * regions[i].numblocks)) %( 1<< cfi->chipshift)))
+ if (adr % (1<< cfi->chipshift) == (((unsigned long)regions[i].offset + (regions[i].erasesize * regions[i].numblocks)) %( 1<< cfi->chipshift)))
i++;
if (adr >> cfi->chipshift) {
@@ -1135,7 +1135,7 @@
spin_unlock_bh(chip->mutex);
return 0;
}
-static int cfi_staa_lock(struct mtd_info *mtd, loff_t ofs, size_t len)
+static int cfi_staa_lock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
{
struct map_info *map = mtd->priv;
struct cfi_private *cfi = map->fldrv_priv;
@@ -1284,7 +1284,7 @@
spin_unlock_bh(chip->mutex);
return 0;
}
-static int cfi_staa_unlock(struct mtd_info *mtd, loff_t ofs, size_t len)
+static int cfi_staa_unlock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
{
struct map_info *map = mtd->priv;
struct cfi_private *cfi = map->fldrv_priv;
diff --git a/drivers/mtd/chips/fwh_lock.h b/drivers/mtd/chips/fwh_lock.h
index ab44f2b..57e0e4e 100644
--- a/drivers/mtd/chips/fwh_lock.h
+++ b/drivers/mtd/chips/fwh_lock.h
@@ -77,7 +77,7 @@
}
-static int fwh_lock_varsize(struct mtd_info *mtd, loff_t ofs, size_t len)
+static int fwh_lock_varsize(struct mtd_info *mtd, loff_t ofs, uint64_t len)
{
int ret;
@@ -88,7 +88,7 @@
}
-static int fwh_unlock_varsize(struct mtd_info *mtd, loff_t ofs, size_t len)
+static int fwh_unlock_varsize(struct mtd_info *mtd, loff_t ofs, uint64_t len)
{
int ret;
diff --git a/drivers/mtd/devices/Kconfig b/drivers/mtd/devices/Kconfig
index 6fde0a2..bc33200 100644
--- a/drivers/mtd/devices/Kconfig
+++ b/drivers/mtd/devices/Kconfig
@@ -120,6 +120,13 @@
doesn't have access to, memory beyond the mem=xxx limit, nvram,
memory on the video card, etc...
+config MTD_PS3VRAM
+ tristate "PS3 video RAM"
+ depends on FB_PS3
+ help
+ This driver allows you to use excess PS3 video RAM as volatile
+ storage or system swap.
+
config MTD_LART
tristate "28F160xx flash driver for LART"
depends on SA1100_LART
diff --git a/drivers/mtd/devices/Makefile b/drivers/mtd/devices/Makefile
index 0993d5c..e51521d 100644
--- a/drivers/mtd/devices/Makefile
+++ b/drivers/mtd/devices/Makefile
@@ -16,3 +16,4 @@
obj-$(CONFIG_MTD_BLOCK2MTD) += block2mtd.o
obj-$(CONFIG_MTD_DATAFLASH) += mtd_dataflash.o
obj-$(CONFIG_MTD_M25P80) += m25p80.o
+obj-$(CONFIG_MTD_PS3VRAM) += ps3vram.o
diff --git a/drivers/mtd/devices/lart.c b/drivers/mtd/devices/lart.c
index f4bda4c..578de1c 100644
--- a/drivers/mtd/devices/lart.c
+++ b/drivers/mtd/devices/lart.c
@@ -619,7 +619,7 @@
};
#endif
-int __init lart_flash_init (void)
+static int __init lart_flash_init (void)
{
int result;
memset (&mtd,0,sizeof (mtd));
@@ -690,7 +690,7 @@
return (result);
}
-void __exit lart_flash_exit (void)
+static void __exit lart_flash_exit (void)
{
#ifndef HAVE_PARTITIONS
del_mtd_device (&mtd);
@@ -705,5 +705,3 @@
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Abraham vd Merwe <abraham@2d3d.co.za>");
MODULE_DESCRIPTION("MTD driver for Intel 28F160F3 on LART board");
-
-
diff --git a/drivers/mtd/devices/m25p80.c b/drivers/mtd/devices/m25p80.c
index 5733f06..7c3fc76 100644
--- a/drivers/mtd/devices/m25p80.c
+++ b/drivers/mtd/devices/m25p80.c
@@ -20,6 +20,7 @@
#include <linux/device.h>
#include <linux/interrupt.h>
#include <linux/mutex.h>
+#include <linux/math64.h>
#include <linux/mtd/mtd.h>
#include <linux/mtd/partitions.h>
@@ -169,9 +170,9 @@
*/
static int erase_chip(struct m25p *flash)
{
- DEBUG(MTD_DEBUG_LEVEL3, "%s: %s %dKiB\n",
- dev_name(&flash->spi->dev), __func__,
- flash->mtd.size / 1024);
+ DEBUG(MTD_DEBUG_LEVEL3, "%s: %s %lldKiB\n",
+ dev_name(&flash->spi->dev), __func__,
+ (long long)(flash->mtd.size >> 10));
/* Wait until finished previous write command. */
if (wait_till_ready(flash))
@@ -232,18 +233,18 @@
{
struct m25p *flash = mtd_to_m25p(mtd);
u32 addr,len;
+ uint32_t rem;
- DEBUG(MTD_DEBUG_LEVEL2, "%s: %s %s 0x%08x, len %d\n",
- dev_name(&flash->spi->dev), __func__, "at",
- (u32)instr->addr, instr->len);
+ DEBUG(MTD_DEBUG_LEVEL2, "%s: %s %s 0x%llx, len %lld\n",
+ dev_name(&flash->spi->dev), __func__, "at",
+ (long long)instr->addr, (long long)instr->len);
/* sanity checks */
if (instr->addr + instr->len > flash->mtd.size)
return -EINVAL;
- if ((instr->addr % mtd->erasesize) != 0
- || (instr->len % mtd->erasesize) != 0) {
+ div_u64_rem(instr->len, mtd->erasesize, &rem);
+ if (rem)
return -EINVAL;
- }
addr = instr->addr;
len = instr->len;
@@ -677,24 +678,24 @@
flash->mtd.erasesize = info->sector_size;
}
- dev_info(&spi->dev, "%s (%d Kbytes)\n", info->name,
- flash->mtd.size / 1024);
+ dev_info(&spi->dev, "%s (%lld Kbytes)\n", info->name,
+ (long long)flash->mtd.size >> 10);
DEBUG(MTD_DEBUG_LEVEL2,
- "mtd .name = %s, .size = 0x%.8x (%uMiB) "
+ "mtd .name = %s, .size = 0x%llx (%lldMiB) "
".erasesize = 0x%.8x (%uKiB) .numeraseregions = %d\n",
flash->mtd.name,
- flash->mtd.size, flash->mtd.size / (1024*1024),
+ (long long)flash->mtd.size, (long long)(flash->mtd.size >> 20),
flash->mtd.erasesize, flash->mtd.erasesize / 1024,
flash->mtd.numeraseregions);
if (flash->mtd.numeraseregions)
for (i = 0; i < flash->mtd.numeraseregions; i++)
DEBUG(MTD_DEBUG_LEVEL2,
- "mtd.eraseregions[%d] = { .offset = 0x%.8x, "
+ "mtd.eraseregions[%d] = { .offset = 0x%llx, "
".erasesize = 0x%.8x (%uKiB), "
".numblocks = %d }\n",
- i, flash->mtd.eraseregions[i].offset,
+ i, (long long)flash->mtd.eraseregions[i].offset,
flash->mtd.eraseregions[i].erasesize,
flash->mtd.eraseregions[i].erasesize / 1024,
flash->mtd.eraseregions[i].numblocks);
@@ -722,12 +723,12 @@
if (nr_parts > 0) {
for (i = 0; i < nr_parts; i++) {
DEBUG(MTD_DEBUG_LEVEL2, "partitions[%d] = "
- "{.name = %s, .offset = 0x%.8x, "
- ".size = 0x%.8x (%uKiB) }\n",
+ "{.name = %s, .offset = 0x%llx, "
+ ".size = 0x%llx (%lldKiB) }\n",
i, parts[i].name,
- parts[i].offset,
- parts[i].size,
- parts[i].size / 1024);
+ (long long)parts[i].offset,
+ (long long)parts[i].size,
+ (long long)(parts[i].size >> 10));
}
flash->partitioned = 1;
return add_mtd_partitions(&flash->mtd, parts, nr_parts);
diff --git a/drivers/mtd/devices/mtd_dataflash.c b/drivers/mtd/devices/mtd_dataflash.c
index 65126cd..d44f741 100644
--- a/drivers/mtd/devices/mtd_dataflash.c
+++ b/drivers/mtd/devices/mtd_dataflash.c
@@ -16,6 +16,7 @@
#include <linux/device.h>
#include <linux/mutex.h>
#include <linux/err.h>
+#include <linux/math64.h>
#include <linux/spi/spi.h>
#include <linux/spi/flash.h>
@@ -152,15 +153,20 @@
struct spi_message msg;
unsigned blocksize = priv->page_size << 3;
uint8_t *command;
+ uint32_t rem;
- DEBUG(MTD_DEBUG_LEVEL2, "%s: erase addr=0x%x len 0x%x\n",
- dev_name(&spi->dev),
- instr->addr, instr->len);
+ DEBUG(MTD_DEBUG_LEVEL2, "%s: erase addr=0x%llx len 0x%llx\n",
+ dev_name(&spi->dev), (long long)instr->addr,
+ (long long)instr->len);
/* Sanity checks */
- if ((instr->addr + instr->len) > mtd->size
- || (instr->len % priv->page_size) != 0
- || (instr->addr % priv->page_size) != 0)
+ if (instr->addr + instr->len > mtd->size)
+ return -EINVAL;
+ div_u64_rem(instr->len, priv->page_size, &rem);
+ if (rem)
+ return -EINVAL;
+ div_u64_rem(instr->addr, priv->page_size, &rem);
+ if (rem)
return -EINVAL;
spi_message_init(&msg);
@@ -178,7 +184,7 @@
/* Calculate flash page address; use block erase (for speed) if
* we're at a block boundary and need to erase the whole block.
*/
- pageaddr = instr->addr / priv->page_size;
+ pageaddr = div_u64(instr->len, priv->page_size);
do_block = (pageaddr & 0x7) == 0 && instr->len >= blocksize;
pageaddr = pageaddr << priv->page_offset;
@@ -667,8 +673,8 @@
if (revision >= 'c')
otp_tag = otp_setup(device, revision);
- dev_info(&spi->dev, "%s (%d KBytes) pagesize %d bytes%s\n",
- name, DIV_ROUND_UP(device->size, 1024),
+ dev_info(&spi->dev, "%s (%lld KBytes) pagesize %d bytes%s\n",
+ name, (long long)((device->size + 1023) >> 10),
pagesize, otp_tag);
dev_set_drvdata(&spi->dev, priv);
diff --git a/drivers/mtd/devices/ps3vram.c b/drivers/mtd/devices/ps3vram.c
new file mode 100644
index 0000000..d21e9be
--- /dev/null
+++ b/drivers/mtd/devices/ps3vram.c
@@ -0,0 +1,768 @@
+/**
+ * ps3vram - Use extra PS3 video ram as MTD block device.
+ *
+ * Copyright (c) 2007-2008 Jim Paris <jim@jtan.com>
+ * Added support RSX DMA Vivien Chappelier <vivien.chappelier@free.fr>
+ */
+
+#include <linux/io.h>
+#include <linux/mm.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/slab.h>
+#include <linux/version.h>
+#include <linux/gfp.h>
+#include <linux/delay.h>
+#include <linux/mtd/mtd.h>
+
+#include <asm/lv1call.h>
+#include <asm/ps3.h>
+
+#define DEVICE_NAME "ps3vram"
+
+#define XDR_BUF_SIZE (2 * 1024 * 1024) /* XDR buffer (must be 1MiB aligned) */
+#define XDR_IOIF 0x0c000000
+
+#define FIFO_BASE XDR_IOIF
+#define FIFO_SIZE (64 * 1024)
+
+#define DMA_PAGE_SIZE (4 * 1024)
+
+#define CACHE_PAGE_SIZE (256 * 1024)
+#define CACHE_PAGE_COUNT ((XDR_BUF_SIZE - FIFO_SIZE) / CACHE_PAGE_SIZE)
+
+#define CACHE_OFFSET CACHE_PAGE_SIZE
+#define FIFO_OFFSET 0
+
+#define CTRL_PUT 0x10
+#define CTRL_GET 0x11
+#define CTRL_TOP 0x15
+
+#define UPLOAD_SUBCH 1
+#define DOWNLOAD_SUBCH 2
+
+#define NV_MEMORY_TO_MEMORY_FORMAT_OFFSET_IN 0x0000030c
+#define NV_MEMORY_TO_MEMORY_FORMAT_NOTIFY 0x00000104
+
+#define L1GPU_CONTEXT_ATTRIBUTE_FB_BLIT 0x601
+
+struct mtd_info ps3vram_mtd;
+
+#define CACHE_PAGE_PRESENT 1
+#define CACHE_PAGE_DIRTY 2
+
+struct ps3vram_tag {
+ unsigned int address;
+ unsigned int flags;
+};
+
+struct ps3vram_cache {
+ unsigned int page_count;
+ unsigned int page_size;
+ struct ps3vram_tag *tags;
+};
+
+struct ps3vram_priv {
+ u64 memory_handle;
+ u64 context_handle;
+ u32 *ctrl;
+ u32 *reports;
+ u8 __iomem *ddr_base;
+ u8 *xdr_buf;
+
+ u32 *fifo_base;
+ u32 *fifo_ptr;
+
+ struct device *dev;
+ struct ps3vram_cache cache;
+
+ /* Used to serialize cache/DMA operations */
+ struct mutex lock;
+};
+
+#define DMA_NOTIFIER_HANDLE_BASE 0x66604200 /* first DMA notifier handle */
+#define DMA_NOTIFIER_OFFSET_BASE 0x1000 /* first DMA notifier offset */
+#define DMA_NOTIFIER_SIZE 0x40
+#define NOTIFIER 7 /* notifier used for completion report */
+
+/* A trailing '-' means to subtract off ps3fb_videomemory.size */
+char *size = "256M-";
+module_param(size, charp, 0);
+MODULE_PARM_DESC(size, "memory size");
+
+static u32 *ps3vram_get_notifier(u32 *reports, int notifier)
+{
+ return (void *) reports +
+ DMA_NOTIFIER_OFFSET_BASE +
+ DMA_NOTIFIER_SIZE * notifier;
+}
+
+static void ps3vram_notifier_reset(struct mtd_info *mtd)
+{
+ int i;
+
+ struct ps3vram_priv *priv = mtd->priv;
+ u32 *notify = ps3vram_get_notifier(priv->reports, NOTIFIER);
+ for (i = 0; i < 4; i++)
+ notify[i] = 0xffffffff;
+}
+
+static int ps3vram_notifier_wait(struct mtd_info *mtd, unsigned int timeout_ms)
+{
+ struct ps3vram_priv *priv = mtd->priv;
+ u32 *notify = ps3vram_get_notifier(priv->reports, NOTIFIER);
+ unsigned long timeout = jiffies + msecs_to_jiffies(timeout_ms);
+
+ do {
+ if (!notify[3])
+ return 0;
+ msleep(1);
+ } while (time_before(jiffies, timeout));
+
+ return -ETIMEDOUT;
+}
+
+static void ps3vram_init_ring(struct mtd_info *mtd)
+{
+ struct ps3vram_priv *priv = mtd->priv;
+
+ priv->ctrl[CTRL_PUT] = FIFO_BASE + FIFO_OFFSET;
+ priv->ctrl[CTRL_GET] = FIFO_BASE + FIFO_OFFSET;
+}
+
+static int ps3vram_wait_ring(struct mtd_info *mtd, unsigned int timeout_ms)
+{
+ struct ps3vram_priv *priv = mtd->priv;
+ unsigned long timeout = jiffies + msecs_to_jiffies(timeout_ms);
+
+ do {
+ if (priv->ctrl[CTRL_PUT] == priv->ctrl[CTRL_GET])
+ return 0;
+ msleep(1);
+ } while (time_before(jiffies, timeout));
+
+ dev_dbg(priv->dev, "%s:%d: FIFO timeout (%08x/%08x/%08x)\n", __func__,
+ __LINE__, priv->ctrl[CTRL_PUT], priv->ctrl[CTRL_GET],
+ priv->ctrl[CTRL_TOP]);
+
+ return -ETIMEDOUT;
+}
+
+static void ps3vram_out_ring(struct ps3vram_priv *priv, u32 data)
+{
+ *(priv->fifo_ptr)++ = data;
+}
+
+static void ps3vram_begin_ring(struct ps3vram_priv *priv, u32 chan,
+ u32 tag, u32 size)
+{
+ ps3vram_out_ring(priv, (size << 18) | (chan << 13) | tag);
+}
+
+static void ps3vram_rewind_ring(struct mtd_info *mtd)
+{
+ struct ps3vram_priv *priv = mtd->priv;
+ u64 status;
+
+ ps3vram_out_ring(priv, 0x20000000 | (FIFO_BASE + FIFO_OFFSET));
+
+ priv->ctrl[CTRL_PUT] = FIFO_BASE + FIFO_OFFSET;
+
+ /* asking the HV for a blit will kick the fifo */
+ status = lv1_gpu_context_attribute(priv->context_handle,
+ L1GPU_CONTEXT_ATTRIBUTE_FB_BLIT,
+ 0, 0, 0, 0);
+ if (status)
+ dev_err(priv->dev, "%s:%d: lv1_gpu_context_attribute failed\n",
+ __func__, __LINE__);
+
+ priv->fifo_ptr = priv->fifo_base;
+}
+
+static void ps3vram_fire_ring(struct mtd_info *mtd)
+{
+ struct ps3vram_priv *priv = mtd->priv;
+ u64 status;
+
+ mutex_lock(&ps3_gpu_mutex);
+
+ priv->ctrl[CTRL_PUT] = FIFO_BASE + FIFO_OFFSET +
+ (priv->fifo_ptr - priv->fifo_base) * sizeof(u32);
+
+ /* asking the HV for a blit will kick the fifo */
+ status = lv1_gpu_context_attribute(priv->context_handle,
+ L1GPU_CONTEXT_ATTRIBUTE_FB_BLIT,
+ 0, 0, 0, 0);
+ if (status)
+ dev_err(priv->dev, "%s:%d: lv1_gpu_context_attribute failed\n",
+ __func__, __LINE__);
+
+ if ((priv->fifo_ptr - priv->fifo_base) * sizeof(u32) >
+ FIFO_SIZE - 1024) {
+ dev_dbg(priv->dev, "%s:%d: fifo full, rewinding\n", __func__,
+ __LINE__);
+ ps3vram_wait_ring(mtd, 200);
+ ps3vram_rewind_ring(mtd);
+ }
+
+ mutex_unlock(&ps3_gpu_mutex);
+}
+
+static void ps3vram_bind(struct mtd_info *mtd)
+{
+ struct ps3vram_priv *priv = mtd->priv;
+
+ ps3vram_begin_ring(priv, UPLOAD_SUBCH, 0, 1);
+ ps3vram_out_ring(priv, 0x31337303);
+ ps3vram_begin_ring(priv, UPLOAD_SUBCH, 0x180, 3);
+ ps3vram_out_ring(priv, DMA_NOTIFIER_HANDLE_BASE + NOTIFIER);
+ ps3vram_out_ring(priv, 0xfeed0001); /* DMA system RAM instance */
+ ps3vram_out_ring(priv, 0xfeed0000); /* DMA video RAM instance */
+
+ ps3vram_begin_ring(priv, DOWNLOAD_SUBCH, 0, 1);
+ ps3vram_out_ring(priv, 0x3137c0de);
+ ps3vram_begin_ring(priv, DOWNLOAD_SUBCH, 0x180, 3);
+ ps3vram_out_ring(priv, DMA_NOTIFIER_HANDLE_BASE + NOTIFIER);
+ ps3vram_out_ring(priv, 0xfeed0000); /* DMA video RAM instance */
+ ps3vram_out_ring(priv, 0xfeed0001); /* DMA system RAM instance */
+
+ ps3vram_fire_ring(mtd);
+}
+
+static int ps3vram_upload(struct mtd_info *mtd, unsigned int src_offset,
+ unsigned int dst_offset, int len, int count)
+{
+ struct ps3vram_priv *priv = mtd->priv;
+
+ ps3vram_begin_ring(priv, UPLOAD_SUBCH,
+ NV_MEMORY_TO_MEMORY_FORMAT_OFFSET_IN, 8);
+ ps3vram_out_ring(priv, XDR_IOIF + src_offset);
+ ps3vram_out_ring(priv, dst_offset);
+ ps3vram_out_ring(priv, len);
+ ps3vram_out_ring(priv, len);
+ ps3vram_out_ring(priv, len);
+ ps3vram_out_ring(priv, count);
+ ps3vram_out_ring(priv, (1 << 8) | 1);
+ ps3vram_out_ring(priv, 0);
+
+ ps3vram_notifier_reset(mtd);
+ ps3vram_begin_ring(priv, UPLOAD_SUBCH,
+ NV_MEMORY_TO_MEMORY_FORMAT_NOTIFY, 1);
+ ps3vram_out_ring(priv, 0);
+ ps3vram_begin_ring(priv, UPLOAD_SUBCH, 0x100, 1);
+ ps3vram_out_ring(priv, 0);
+ ps3vram_fire_ring(mtd);
+ if (ps3vram_notifier_wait(mtd, 200) < 0) {
+ dev_dbg(priv->dev, "%s:%d: notifier timeout\n", __func__,
+ __LINE__);
+ return -1;
+ }
+
+ return 0;
+}
+
+static int ps3vram_download(struct mtd_info *mtd, unsigned int src_offset,
+ unsigned int dst_offset, int len, int count)
+{
+ struct ps3vram_priv *priv = mtd->priv;
+
+ ps3vram_begin_ring(priv, DOWNLOAD_SUBCH,
+ NV_MEMORY_TO_MEMORY_FORMAT_OFFSET_IN, 8);
+ ps3vram_out_ring(priv, src_offset);
+ ps3vram_out_ring(priv, XDR_IOIF + dst_offset);
+ ps3vram_out_ring(priv, len);
+ ps3vram_out_ring(priv, len);
+ ps3vram_out_ring(priv, len);
+ ps3vram_out_ring(priv, count);
+ ps3vram_out_ring(priv, (1 << 8) | 1);
+ ps3vram_out_ring(priv, 0);
+
+ ps3vram_notifier_reset(mtd);
+ ps3vram_begin_ring(priv, DOWNLOAD_SUBCH,
+ NV_MEMORY_TO_MEMORY_FORMAT_NOTIFY, 1);
+ ps3vram_out_ring(priv, 0);
+ ps3vram_begin_ring(priv, DOWNLOAD_SUBCH, 0x100, 1);
+ ps3vram_out_ring(priv, 0);
+ ps3vram_fire_ring(mtd);
+ if (ps3vram_notifier_wait(mtd, 200) < 0) {
+ dev_dbg(priv->dev, "%s:%d: notifier timeout\n", __func__,
+ __LINE__);
+ return -1;
+ }
+
+ return 0;
+}
+
+static void ps3vram_cache_evict(struct mtd_info *mtd, int entry)
+{
+ struct ps3vram_priv *priv = mtd->priv;
+ struct ps3vram_cache *cache = &priv->cache;
+
+ if (cache->tags[entry].flags & CACHE_PAGE_DIRTY) {
+ dev_dbg(priv->dev, "%s:%d: flushing %d : 0x%08x\n", __func__,
+ __LINE__, entry, cache->tags[entry].address);
+ if (ps3vram_upload(mtd,
+ CACHE_OFFSET + entry * cache->page_size,
+ cache->tags[entry].address,
+ DMA_PAGE_SIZE,
+ cache->page_size / DMA_PAGE_SIZE) < 0) {
+ dev_dbg(priv->dev, "%s:%d: failed to upload from "
+ "0x%x to 0x%x size 0x%x\n", __func__, __LINE__,
+ entry * cache->page_size,
+ cache->tags[entry].address, cache->page_size);
+ }
+ cache->tags[entry].flags &= ~CACHE_PAGE_DIRTY;
+ }
+}
+
+static void ps3vram_cache_load(struct mtd_info *mtd, int entry,
+ unsigned int address)
+{
+ struct ps3vram_priv *priv = mtd->priv;
+ struct ps3vram_cache *cache = &priv->cache;
+
+ dev_dbg(priv->dev, "%s:%d: fetching %d : 0x%08x\n", __func__, __LINE__,
+ entry, address);
+ if (ps3vram_download(mtd,
+ address,
+ CACHE_OFFSET + entry * cache->page_size,
+ DMA_PAGE_SIZE,
+ cache->page_size / DMA_PAGE_SIZE) < 0) {
+ dev_err(priv->dev, "%s:%d: failed to download from "
+ "0x%x to 0x%x size 0x%x\n", __func__, __LINE__, address,
+ entry * cache->page_size, cache->page_size);
+ }
+
+ cache->tags[entry].address = address;
+ cache->tags[entry].flags |= CACHE_PAGE_PRESENT;
+}
+
+
+static void ps3vram_cache_flush(struct mtd_info *mtd)
+{
+ struct ps3vram_priv *priv = mtd->priv;
+ struct ps3vram_cache *cache = &priv->cache;
+ int i;
+
+ dev_dbg(priv->dev, "%s:%d: FLUSH\n", __func__, __LINE__);
+ for (i = 0; i < cache->page_count; i++) {
+ ps3vram_cache_evict(mtd, i);
+ cache->tags[i].flags = 0;
+ }
+}
+
+static unsigned int ps3vram_cache_match(struct mtd_info *mtd, loff_t address)
+{
+ struct ps3vram_priv *priv = mtd->priv;
+ struct ps3vram_cache *cache = &priv->cache;
+ unsigned int base;
+ unsigned int offset;
+ int i;
+ static int counter;
+
+ offset = (unsigned int) (address & (cache->page_size - 1));
+ base = (unsigned int) (address - offset);
+
+ /* fully associative check */
+ for (i = 0; i < cache->page_count; i++) {
+ if ((cache->tags[i].flags & CACHE_PAGE_PRESENT) &&
+ cache->tags[i].address == base) {
+ dev_dbg(priv->dev, "%s:%d: found entry %d : 0x%08x\n",
+ __func__, __LINE__, i, cache->tags[i].address);
+ return i;
+ }
+ }
+
+ /* choose a random entry */
+ i = (jiffies + (counter++)) % cache->page_count;
+ dev_dbg(priv->dev, "%s:%d: using entry %d\n", __func__, __LINE__, i);
+
+ ps3vram_cache_evict(mtd, i);
+ ps3vram_cache_load(mtd, i, base);
+
+ return i;
+}
+
+static int ps3vram_cache_init(struct mtd_info *mtd)
+{
+ struct ps3vram_priv *priv = mtd->priv;
+
+ priv->cache.page_count = CACHE_PAGE_COUNT;
+ priv->cache.page_size = CACHE_PAGE_SIZE;
+ priv->cache.tags = kzalloc(sizeof(struct ps3vram_tag) *
+ CACHE_PAGE_COUNT, GFP_KERNEL);
+ if (priv->cache.tags == NULL) {
+ dev_err(priv->dev, "%s:%d: could not allocate cache tags\n",
+ __func__, __LINE__);
+ return -ENOMEM;
+ }
+
+ dev_info(priv->dev, "created ram cache: %d entries, %d KiB each\n",
+ CACHE_PAGE_COUNT, CACHE_PAGE_SIZE / 1024);
+
+ return 0;
+}
+
+static void ps3vram_cache_cleanup(struct mtd_info *mtd)
+{
+ struct ps3vram_priv *priv = mtd->priv;
+
+ ps3vram_cache_flush(mtd);
+ kfree(priv->cache.tags);
+}
+
+static int ps3vram_erase(struct mtd_info *mtd, struct erase_info *instr)
+{
+ struct ps3vram_priv *priv = mtd->priv;
+
+ if (instr->addr + instr->len > mtd->size)
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ ps3vram_cache_flush(mtd);
+
+ /* Set bytes to 0xFF */
+ memset_io(priv->ddr_base + instr->addr, 0xFF, instr->len);
+
+ mutex_unlock(&priv->lock);
+
+ instr->state = MTD_ERASE_DONE;
+ mtd_erase_callback(instr);
+
+ return 0;
+}
+
+static int ps3vram_read(struct mtd_info *mtd, loff_t from, size_t len,
+ size_t *retlen, u_char *buf)
+{
+ struct ps3vram_priv *priv = mtd->priv;
+ unsigned int cached, count;
+
+ dev_dbg(priv->dev, "%s:%d: from=0x%08x len=0x%zx\n", __func__, __LINE__,
+ (unsigned int)from, len);
+
+ if (from >= mtd->size)
+ return -EINVAL;
+
+ if (len > mtd->size - from)
+ len = mtd->size - from;
+
+ /* Copy from vram to buf */
+ count = len;
+ while (count) {
+ unsigned int offset, avail;
+ unsigned int entry;
+
+ offset = (unsigned int) (from & (priv->cache.page_size - 1));
+ avail = priv->cache.page_size - offset;
+
+ mutex_lock(&priv->lock);
+
+ entry = ps3vram_cache_match(mtd, from);
+ cached = CACHE_OFFSET + entry * priv->cache.page_size + offset;
+
+ dev_dbg(priv->dev, "%s:%d: from=%08x cached=%08x offset=%08x "
+ "avail=%08x count=%08x\n", __func__, __LINE__,
+ (unsigned int)from, cached, offset, avail, count);
+
+ if (avail > count)
+ avail = count;
+ memcpy(buf, priv->xdr_buf + cached, avail);
+
+ mutex_unlock(&priv->lock);
+
+ buf += avail;
+ count -= avail;
+ from += avail;
+ }
+
+ *retlen = len;
+ return 0;
+}
+
+static int ps3vram_write(struct mtd_info *mtd, loff_t to, size_t len,
+ size_t *retlen, const u_char *buf)
+{
+ struct ps3vram_priv *priv = mtd->priv;
+ unsigned int cached, count;
+
+ if (to >= mtd->size)
+ return -EINVAL;
+
+ if (len > mtd->size - to)
+ len = mtd->size - to;
+
+ /* Copy from buf to vram */
+ count = len;
+ while (count) {
+ unsigned int offset, avail;
+ unsigned int entry;
+
+ offset = (unsigned int) (to & (priv->cache.page_size - 1));
+ avail = priv->cache.page_size - offset;
+
+ mutex_lock(&priv->lock);
+
+ entry = ps3vram_cache_match(mtd, to);
+ cached = CACHE_OFFSET + entry * priv->cache.page_size + offset;
+
+ dev_dbg(priv->dev, "%s:%d: to=%08x cached=%08x offset=%08x "
+ "avail=%08x count=%08x\n", __func__, __LINE__,
+ (unsigned int)to, cached, offset, avail, count);
+
+ if (avail > count)
+ avail = count;
+ memcpy(priv->xdr_buf + cached, buf, avail);
+
+ priv->cache.tags[entry].flags |= CACHE_PAGE_DIRTY;
+
+ mutex_unlock(&priv->lock);
+
+ buf += avail;
+ count -= avail;
+ to += avail;
+ }
+
+ *retlen = len;
+ return 0;
+}
+
+static int __devinit ps3vram_probe(struct ps3_system_bus_device *dev)
+{
+ struct ps3vram_priv *priv;
+ int status;
+ u64 ddr_lpar;
+ u64 ctrl_lpar;
+ u64 info_lpar;
+ u64 reports_lpar;
+ u64 ddr_size;
+ u64 reports_size;
+ int ret = -ENOMEM;
+ char *rest;
+
+ ret = -EIO;
+ ps3vram_mtd.priv = kzalloc(sizeof(struct ps3vram_priv), GFP_KERNEL);
+ if (!ps3vram_mtd.priv)
+ goto out;
+ priv = ps3vram_mtd.priv;
+
+ mutex_init(&priv->lock);
+ priv->dev = &dev->core;
+
+ /* Allocate XDR buffer (1MiB aligned) */
+ priv->xdr_buf = (void *)__get_free_pages(GFP_KERNEL,
+ get_order(XDR_BUF_SIZE));
+ if (priv->xdr_buf == NULL) {
+ dev_dbg(&dev->core, "%s:%d: could not allocate XDR buffer\n",
+ __func__, __LINE__);
+ ret = -ENOMEM;
+ goto out_free_priv;
+ }
+
+ /* Put FIFO at begginning of XDR buffer */
+ priv->fifo_base = (u32 *) (priv->xdr_buf + FIFO_OFFSET);
+ priv->fifo_ptr = priv->fifo_base;
+
+ /* XXX: Need to open GPU, in case ps3fb or snd_ps3 aren't loaded */
+ if (ps3_open_hv_device(dev)) {
+ dev_err(&dev->core, "%s:%d: ps3_open_hv_device failed\n",
+ __func__, __LINE__);
+ ret = -EAGAIN;
+ goto out_close_gpu;
+ }
+
+ /* Request memory */
+ status = -1;
+ ddr_size = memparse(size, &rest);
+ if (*rest == '-')
+ ddr_size -= ps3fb_videomemory.size;
+ ddr_size = ALIGN(ddr_size, 1024*1024);
+ if (ddr_size <= 0) {
+ dev_err(&dev->core, "%s:%d: specified size is too small\n",
+ __func__, __LINE__);
+ ret = -EINVAL;
+ goto out_close_gpu;
+ }
+
+ while (ddr_size > 0) {
+ status = lv1_gpu_memory_allocate(ddr_size, 0, 0, 0, 0,
+ &priv->memory_handle,
+ &ddr_lpar);
+ if (!status)
+ break;
+ ddr_size -= 1024*1024;
+ }
+ if (status || ddr_size <= 0) {
+ dev_err(&dev->core, "%s:%d: lv1_gpu_memory_allocate failed\n",
+ __func__, __LINE__);
+ ret = -ENOMEM;
+ goto out_free_xdr_buf;
+ }
+
+ /* Request context */
+ status = lv1_gpu_context_allocate(priv->memory_handle,
+ 0,
+ &priv->context_handle,
+ &ctrl_lpar,
+ &info_lpar,
+ &reports_lpar,
+ &reports_size);
+ if (status) {
+ dev_err(&dev->core, "%s:%d: lv1_gpu_context_allocate failed\n",
+ __func__, __LINE__);
+ ret = -ENOMEM;
+ goto out_free_memory;
+ }
+
+ /* Map XDR buffer to RSX */
+ status = lv1_gpu_context_iomap(priv->context_handle, XDR_IOIF,
+ ps3_mm_phys_to_lpar(__pa(priv->xdr_buf)),
+ XDR_BUF_SIZE, 0);
+ if (status) {
+ dev_err(&dev->core, "%s:%d: lv1_gpu_context_iomap failed\n",
+ __func__, __LINE__);
+ ret = -ENOMEM;
+ goto out_free_context;
+ }
+
+ priv->ddr_base = ioremap_flags(ddr_lpar, ddr_size, _PAGE_NO_CACHE);
+
+ if (!priv->ddr_base) {
+ dev_err(&dev->core, "%s:%d: ioremap failed\n", __func__,
+ __LINE__);
+ ret = -ENOMEM;
+ goto out_free_context;
+ }
+
+ priv->ctrl = ioremap(ctrl_lpar, 64 * 1024);
+ if (!priv->ctrl) {
+ dev_err(&dev->core, "%s:%d: ioremap failed\n", __func__,
+ __LINE__);
+ ret = -ENOMEM;
+ goto out_unmap_vram;
+ }
+
+ priv->reports = ioremap(reports_lpar, reports_size);
+ if (!priv->reports) {
+ dev_err(&dev->core, "%s:%d: ioremap failed\n", __func__,
+ __LINE__);
+ ret = -ENOMEM;
+ goto out_unmap_ctrl;
+ }
+
+ mutex_lock(&ps3_gpu_mutex);
+ ps3vram_init_ring(&ps3vram_mtd);
+ mutex_unlock(&ps3_gpu_mutex);
+
+ ps3vram_mtd.name = "ps3vram";
+ ps3vram_mtd.size = ddr_size;
+ ps3vram_mtd.flags = MTD_CAP_RAM;
+ ps3vram_mtd.erase = ps3vram_erase;
+ ps3vram_mtd.point = NULL;
+ ps3vram_mtd.unpoint = NULL;
+ ps3vram_mtd.read = ps3vram_read;
+ ps3vram_mtd.write = ps3vram_write;
+ ps3vram_mtd.owner = THIS_MODULE;
+ ps3vram_mtd.type = MTD_RAM;
+ ps3vram_mtd.erasesize = CACHE_PAGE_SIZE;
+ ps3vram_mtd.writesize = 1;
+
+ ps3vram_bind(&ps3vram_mtd);
+
+ mutex_lock(&ps3_gpu_mutex);
+ ret = ps3vram_wait_ring(&ps3vram_mtd, 100);
+ mutex_unlock(&ps3_gpu_mutex);
+ if (ret < 0) {
+ dev_err(&dev->core, "%s:%d: failed to initialize channels\n",
+ __func__, __LINE__);
+ ret = -ETIMEDOUT;
+ goto out_unmap_reports;
+ }
+
+ ps3vram_cache_init(&ps3vram_mtd);
+
+ if (add_mtd_device(&ps3vram_mtd)) {
+ dev_err(&dev->core, "%s:%d: add_mtd_device failed\n",
+ __func__, __LINE__);
+ ret = -EAGAIN;
+ goto out_cache_cleanup;
+ }
+
+ dev_info(&dev->core, "reserved %u MiB of gpu memory\n",
+ (unsigned int)(ddr_size / 1024 / 1024));
+
+ return 0;
+
+out_cache_cleanup:
+ ps3vram_cache_cleanup(&ps3vram_mtd);
+out_unmap_reports:
+ iounmap(priv->reports);
+out_unmap_ctrl:
+ iounmap(priv->ctrl);
+out_unmap_vram:
+ iounmap(priv->ddr_base);
+out_free_context:
+ lv1_gpu_context_free(priv->context_handle);
+out_free_memory:
+ lv1_gpu_memory_free(priv->memory_handle);
+out_close_gpu:
+ ps3_close_hv_device(dev);
+out_free_xdr_buf:
+ free_pages((unsigned long) priv->xdr_buf, get_order(XDR_BUF_SIZE));
+out_free_priv:
+ kfree(ps3vram_mtd.priv);
+ ps3vram_mtd.priv = NULL;
+out:
+ return ret;
+}
+
+static int ps3vram_shutdown(struct ps3_system_bus_device *dev)
+{
+ struct ps3vram_priv *priv;
+
+ priv = ps3vram_mtd.priv;
+
+ del_mtd_device(&ps3vram_mtd);
+ ps3vram_cache_cleanup(&ps3vram_mtd);
+ iounmap(priv->reports);
+ iounmap(priv->ctrl);
+ iounmap(priv->ddr_base);
+ lv1_gpu_context_free(priv->context_handle);
+ lv1_gpu_memory_free(priv->memory_handle);
+ ps3_close_hv_device(dev);
+ free_pages((unsigned long) priv->xdr_buf, get_order(XDR_BUF_SIZE));
+ kfree(priv);
+ return 0;
+}
+
+static struct ps3_system_bus_driver ps3vram_driver = {
+ .match_id = PS3_MATCH_ID_GPU,
+ .match_sub_id = PS3_MATCH_SUB_ID_GPU_RAMDISK,
+ .core.name = DEVICE_NAME,
+ .core.owner = THIS_MODULE,
+ .probe = ps3vram_probe,
+ .remove = ps3vram_shutdown,
+ .shutdown = ps3vram_shutdown,
+};
+
+static int __init ps3vram_init(void)
+{
+ return ps3_system_bus_driver_register(&ps3vram_driver);
+}
+
+static void __exit ps3vram_exit(void)
+{
+ ps3_system_bus_driver_unregister(&ps3vram_driver);
+}
+
+module_init(ps3vram_init);
+module_exit(ps3vram_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Jim Paris <jim@jtan.com>");
+MODULE_DESCRIPTION("MTD driver for PS3 video RAM");
+MODULE_ALIAS(PS3_MODULE_ALIAS_GPU_RAMDISK);
diff --git a/drivers/mtd/ftl.c b/drivers/mtd/ftl.c
index 9bf581c..a790c06 100644
--- a/drivers/mtd/ftl.c
+++ b/drivers/mtd/ftl.c
@@ -109,25 +109,25 @@
/* Each memory region corresponds to a minor device */
typedef struct partition_t {
struct mtd_blktrans_dev mbd;
- u_int32_t state;
- u_int32_t *VirtualBlockMap;
- u_int32_t *VirtualPageMap;
- u_int32_t FreeTotal;
+ uint32_t state;
+ uint32_t *VirtualBlockMap;
+ uint32_t *VirtualPageMap;
+ uint32_t FreeTotal;
struct eun_info_t {
- u_int32_t Offset;
- u_int32_t EraseCount;
- u_int32_t Free;
- u_int32_t Deleted;
+ uint32_t Offset;
+ uint32_t EraseCount;
+ uint32_t Free;
+ uint32_t Deleted;
} *EUNInfo;
struct xfer_info_t {
- u_int32_t Offset;
- u_int32_t EraseCount;
- u_int16_t state;
+ uint32_t Offset;
+ uint32_t EraseCount;
+ uint16_t state;
} *XferInfo;
- u_int16_t bam_index;
- u_int32_t *bam_cache;
- u_int16_t DataUnits;
- u_int32_t BlocksPerUnit;
+ uint16_t bam_index;
+ uint32_t *bam_cache;
+ uint16_t DataUnits;
+ uint32_t BlocksPerUnit;
erase_unit_header_t header;
} partition_t;
@@ -199,8 +199,8 @@
static int build_maps(partition_t *part)
{
erase_unit_header_t header;
- u_int16_t xvalid, xtrans, i;
- u_int blocks, j;
+ uint16_t xvalid, xtrans, i;
+ unsigned blocks, j;
int hdr_ok, ret = -1;
ssize_t retval;
loff_t offset;
@@ -269,14 +269,14 @@
/* Set up virtual page map */
blocks = le32_to_cpu(header.FormattedSize) >> header.BlockSize;
- part->VirtualBlockMap = vmalloc(blocks * sizeof(u_int32_t));
+ part->VirtualBlockMap = vmalloc(blocks * sizeof(uint32_t));
if (!part->VirtualBlockMap)
goto out_XferInfo;
- memset(part->VirtualBlockMap, 0xff, blocks * sizeof(u_int32_t));
+ memset(part->VirtualBlockMap, 0xff, blocks * sizeof(uint32_t));
part->BlocksPerUnit = (1 << header.EraseUnitSize) >> header.BlockSize;
- part->bam_cache = kmalloc(part->BlocksPerUnit * sizeof(u_int32_t),
+ part->bam_cache = kmalloc(part->BlocksPerUnit * sizeof(uint32_t),
GFP_KERNEL);
if (!part->bam_cache)
goto out_VirtualBlockMap;
@@ -290,7 +290,7 @@
offset = part->EUNInfo[i].Offset + le32_to_cpu(header.BAMOffset);
ret = part->mbd.mtd->read(part->mbd.mtd, offset,
- part->BlocksPerUnit * sizeof(u_int32_t), &retval,
+ part->BlocksPerUnit * sizeof(uint32_t), &retval,
(unsigned char *)part->bam_cache);
if (ret)
@@ -332,7 +332,7 @@
======================================================================*/
static int erase_xfer(partition_t *part,
- u_int16_t xfernum)
+ uint16_t xfernum)
{
int ret;
struct xfer_info_t *xfer;
@@ -408,7 +408,7 @@
erase_unit_header_t header;
struct xfer_info_t *xfer;
int nbam, ret;
- u_int32_t ctl;
+ uint32_t ctl;
ssize_t retlen;
loff_t offset;
@@ -430,15 +430,15 @@
}
/* Write the BAM stub */
- nbam = (part->BlocksPerUnit * sizeof(u_int32_t) +
+ nbam = (part->BlocksPerUnit * sizeof(uint32_t) +
le32_to_cpu(part->header.BAMOffset) + SECTOR_SIZE - 1) / SECTOR_SIZE;
offset = xfer->Offset + le32_to_cpu(part->header.BAMOffset);
ctl = cpu_to_le32(BLOCK_CONTROL);
- for (i = 0; i < nbam; i++, offset += sizeof(u_int32_t)) {
+ for (i = 0; i < nbam; i++, offset += sizeof(uint32_t)) {
- ret = part->mbd.mtd->write(part->mbd.mtd, offset, sizeof(u_int32_t),
+ ret = part->mbd.mtd->write(part->mbd.mtd, offset, sizeof(uint32_t),
&retlen, (u_char *)&ctl);
if (ret)
@@ -461,18 +461,18 @@
======================================================================*/
-static int copy_erase_unit(partition_t *part, u_int16_t srcunit,
- u_int16_t xferunit)
+static int copy_erase_unit(partition_t *part, uint16_t srcunit,
+ uint16_t xferunit)
{
u_char buf[SECTOR_SIZE];
struct eun_info_t *eun;
struct xfer_info_t *xfer;
- u_int32_t src, dest, free, i;
- u_int16_t unit;
+ uint32_t src, dest, free, i;
+ uint16_t unit;
int ret;
ssize_t retlen;
loff_t offset;
- u_int16_t srcunitswap = cpu_to_le16(srcunit);
+ uint16_t srcunitswap = cpu_to_le16(srcunit);
eun = &part->EUNInfo[srcunit];
xfer = &part->XferInfo[xferunit];
@@ -486,7 +486,7 @@
offset = eun->Offset + le32_to_cpu(part->header.BAMOffset);
ret = part->mbd.mtd->read(part->mbd.mtd, offset,
- part->BlocksPerUnit * sizeof(u_int32_t),
+ part->BlocksPerUnit * sizeof(uint32_t),
&retlen, (u_char *) (part->bam_cache));
/* mark the cache bad, in case we get an error later */
@@ -503,7 +503,7 @@
offset = xfer->Offset + 20; /* Bad! */
unit = cpu_to_le16(0x7fff);
- ret = part->mbd.mtd->write(part->mbd.mtd, offset, sizeof(u_int16_t),
+ ret = part->mbd.mtd->write(part->mbd.mtd, offset, sizeof(uint16_t),
&retlen, (u_char *) &unit);
if (ret) {
@@ -560,7 +560,7 @@
/* All clear? Then update the LogicalEUN again */
- ret = part->mbd.mtd->write(part->mbd.mtd, xfer->Offset + 20, sizeof(u_int16_t),
+ ret = part->mbd.mtd->write(part->mbd.mtd, xfer->Offset + 20, sizeof(uint16_t),
&retlen, (u_char *)&srcunitswap);
if (ret) {
@@ -605,8 +605,8 @@
static int reclaim_block(partition_t *part)
{
- u_int16_t i, eun, xfer;
- u_int32_t best;
+ uint16_t i, eun, xfer;
+ uint32_t best;
int queued, ret;
DEBUG(0, "ftl_cs: reclaiming space...\n");
@@ -723,10 +723,10 @@
}
#endif
-static u_int32_t find_free(partition_t *part)
+static uint32_t find_free(partition_t *part)
{
- u_int16_t stop, eun;
- u_int32_t blk;
+ uint16_t stop, eun;
+ uint32_t blk;
size_t retlen;
int ret;
@@ -749,7 +749,7 @@
ret = part->mbd.mtd->read(part->mbd.mtd,
part->EUNInfo[eun].Offset + le32_to_cpu(part->header.BAMOffset),
- part->BlocksPerUnit * sizeof(u_int32_t),
+ part->BlocksPerUnit * sizeof(uint32_t),
&retlen, (u_char *) (part->bam_cache));
if (ret) {
@@ -786,7 +786,7 @@
static int ftl_read(partition_t *part, caddr_t buffer,
u_long sector, u_long nblocks)
{
- u_int32_t log_addr, bsize;
+ uint32_t log_addr, bsize;
u_long i;
int ret;
size_t offset, retlen;
@@ -829,14 +829,14 @@
======================================================================*/
-static int set_bam_entry(partition_t *part, u_int32_t log_addr,
- u_int32_t virt_addr)
+static int set_bam_entry(partition_t *part, uint32_t log_addr,
+ uint32_t virt_addr)
{
- u_int32_t bsize, blk, le_virt_addr;
+ uint32_t bsize, blk, le_virt_addr;
#ifdef PSYCHO_DEBUG
- u_int32_t old_addr;
+ uint32_t old_addr;
#endif
- u_int16_t eun;
+ uint16_t eun;
int ret;
size_t retlen, offset;
@@ -845,11 +845,11 @@
bsize = 1 << part->header.EraseUnitSize;
eun = log_addr / bsize;
blk = (log_addr % bsize) / SECTOR_SIZE;
- offset = (part->EUNInfo[eun].Offset + blk * sizeof(u_int32_t) +
+ offset = (part->EUNInfo[eun].Offset + blk * sizeof(uint32_t) +
le32_to_cpu(part->header.BAMOffset));
#ifdef PSYCHO_DEBUG
- ret = part->mbd.mtd->read(part->mbd.mtd, offset, sizeof(u_int32_t),
+ ret = part->mbd.mtd->read(part->mbd.mtd, offset, sizeof(uint32_t),
&retlen, (u_char *)&old_addr);
if (ret) {
printk(KERN_WARNING"ftl: Error reading old_addr in set_bam_entry: %d\n",ret);
@@ -886,7 +886,7 @@
#endif
part->bam_cache[blk] = le_virt_addr;
}
- ret = part->mbd.mtd->write(part->mbd.mtd, offset, sizeof(u_int32_t),
+ ret = part->mbd.mtd->write(part->mbd.mtd, offset, sizeof(uint32_t),
&retlen, (u_char *)&le_virt_addr);
if (ret) {
@@ -900,7 +900,7 @@
static int ftl_write(partition_t *part, caddr_t buffer,
u_long sector, u_long nblocks)
{
- u_int32_t bsize, log_addr, virt_addr, old_addr, blk;
+ uint32_t bsize, log_addr, virt_addr, old_addr, blk;
u_long i;
int ret;
size_t retlen, offset;
diff --git a/drivers/mtd/inftlcore.c b/drivers/mtd/inftlcore.c
index 50ce138..73f0522 100644
--- a/drivers/mtd/inftlcore.c
+++ b/drivers/mtd/inftlcore.c
@@ -50,7 +50,7 @@
struct INFTLrecord *inftl;
unsigned long temp;
- if (mtd->type != MTD_NANDFLASH)
+ if (mtd->type != MTD_NANDFLASH || mtd->size > UINT_MAX)
return;
/* OK, this is moderately ugly. But probably safe. Alternatives? */
if (memcmp(mtd->name, "DiskOnChip", 10))
diff --git a/drivers/mtd/inftlmount.c b/drivers/mtd/inftlmount.c
index 9113628..f751dd9 100644
--- a/drivers/mtd/inftlmount.c
+++ b/drivers/mtd/inftlmount.c
@@ -63,7 +63,7 @@
* otherwise.
*/
inftl->EraseSize = inftl->mbd.mtd->erasesize;
- inftl->nb_blocks = inftl->mbd.mtd->size / inftl->EraseSize;
+ inftl->nb_blocks = (u32)inftl->mbd.mtd->size / inftl->EraseSize;
inftl->MediaUnit = BLOCK_NIL;
@@ -187,7 +187,7 @@
mh->BlockMultiplierBits);
inftl->EraseSize = inftl->mbd.mtd->erasesize <<
mh->BlockMultiplierBits;
- inftl->nb_blocks = inftl->mbd.mtd->size / inftl->EraseSize;
+ inftl->nb_blocks = (u32)inftl->mbd.mtd->size / inftl->EraseSize;
block >>= mh->BlockMultiplierBits;
}
diff --git a/drivers/mtd/lpddr/Kconfig b/drivers/mtd/lpddr/Kconfig
new file mode 100644
index 0000000..acd4ea9
--- /dev/null
+++ b/drivers/mtd/lpddr/Kconfig
@@ -0,0 +1,22 @@
+# drivers/mtd/chips/Kconfig
+
+menu "LPDDR flash memory drivers"
+ depends on MTD!=n
+
+config MTD_LPDDR
+ tristate "Support for LPDDR flash chips"
+ select MTD_QINFO_PROBE
+ help
+ This option enables support of LPDDR (Low power double data rate)
+ flash chips. Synonymous with Mobile-DDR. It is a new standard for
+ DDR memories, intended for battery-operated systems.
+
+config MTD_QINFO_PROBE
+ tristate "Detect flash chips by QINFO probe"
+ help
+ Device Information for LPDDR chips is offered through the Overlay
+ Window QINFO interface, permits software to be used for entire
+ families of devices. This serves similar purpose of CFI on legacy
+ Flash products
+endmenu
+
diff --git a/drivers/mtd/lpddr/Makefile b/drivers/mtd/lpddr/Makefile
new file mode 100644
index 0000000..da48e46
--- /dev/null
+++ b/drivers/mtd/lpddr/Makefile
@@ -0,0 +1,6 @@
+#
+# linux/drivers/mtd/lpddr/Makefile
+#
+
+obj-$(CONFIG_MTD_QINFO_PROBE) += qinfo_probe.o
+obj-$(CONFIG_MTD_LPDDR) += lpddr_cmds.o
diff --git a/drivers/mtd/lpddr/lpddr_cmds.c b/drivers/mtd/lpddr/lpddr_cmds.c
new file mode 100644
index 0000000..e22ca49
--- /dev/null
+++ b/drivers/mtd/lpddr/lpddr_cmds.c
@@ -0,0 +1,796 @@
+/*
+ * LPDDR flash memory device operations. This module provides read, write,
+ * erase, lock/unlock support for LPDDR flash memories
+ * (C) 2008 Korolev Alexey <akorolev@infradead.org>
+ * (C) 2008 Vasiliy Leonenko <vasiliy.leonenko@gmail.com>
+ * Many thanks to Roman Borisov for intial enabling
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+ * 02110-1301, USA.
+ * TODO:
+ * Implement VPP management
+ * Implement XIP support
+ * Implement OTP support
+ */
+#include <linux/mtd/pfow.h>
+#include <linux/mtd/qinfo.h>
+
+static int lpddr_read(struct mtd_info *mtd, loff_t adr, size_t len,
+ size_t *retlen, u_char *buf);
+static int lpddr_write_buffers(struct mtd_info *mtd, loff_t to,
+ size_t len, size_t *retlen, const u_char *buf);
+static int lpddr_writev(struct mtd_info *mtd, const struct kvec *vecs,
+ unsigned long count, loff_t to, size_t *retlen);
+static int lpddr_erase(struct mtd_info *mtd, struct erase_info *instr);
+static int lpddr_lock(struct mtd_info *mtd, loff_t ofs, uint64_t len);
+static int lpddr_unlock(struct mtd_info *mtd, loff_t ofs, uint64_t len);
+static int lpddr_point(struct mtd_info *mtd, loff_t adr, size_t len,
+ size_t *retlen, void **mtdbuf, resource_size_t *phys);
+static void lpddr_unpoint(struct mtd_info *mtd, loff_t adr, size_t len);
+static int get_chip(struct map_info *map, struct flchip *chip, int mode);
+static int chip_ready(struct map_info *map, struct flchip *chip, int mode);
+static void put_chip(struct map_info *map, struct flchip *chip);
+
+struct mtd_info *lpddr_cmdset(struct map_info *map)
+{
+ struct lpddr_private *lpddr = map->fldrv_priv;
+ struct flchip_shared *shared;
+ struct flchip *chip;
+ struct mtd_info *mtd;
+ int numchips;
+ int i, j;
+
+ mtd = kzalloc(sizeof(*mtd), GFP_KERNEL);
+ if (!mtd) {
+ printk(KERN_ERR "Failed to allocate memory for MTD device\n");
+ return NULL;
+ }
+ mtd->priv = map;
+ mtd->type = MTD_NORFLASH;
+
+ /* Fill in the default mtd operations */
+ mtd->read = lpddr_read;
+ mtd->type = MTD_NORFLASH;
+ mtd->flags = MTD_CAP_NORFLASH;
+ mtd->flags &= ~MTD_BIT_WRITEABLE;
+ mtd->erase = lpddr_erase;
+ mtd->write = lpddr_write_buffers;
+ mtd->writev = lpddr_writev;
+ mtd->read_oob = NULL;
+ mtd->write_oob = NULL;
+ mtd->sync = NULL;
+ mtd->lock = lpddr_lock;
+ mtd->unlock = lpddr_unlock;
+ mtd->suspend = NULL;
+ mtd->resume = NULL;
+ if (map_is_linear(map)) {
+ mtd->point = lpddr_point;
+ mtd->unpoint = lpddr_unpoint;
+ }
+ mtd->block_isbad = NULL;
+ mtd->block_markbad = NULL;
+ mtd->size = 1 << lpddr->qinfo->DevSizeShift;
+ mtd->erasesize = 1 << lpddr->qinfo->UniformBlockSizeShift;
+ mtd->writesize = 1 << lpddr->qinfo->BufSizeShift;
+
+ shared = kmalloc(sizeof(struct flchip_shared) * lpddr->numchips,
+ GFP_KERNEL);
+ if (!shared) {
+ kfree(lpddr);
+ kfree(mtd);
+ return NULL;
+ }
+
+ chip = &lpddr->chips[0];
+ numchips = lpddr->numchips / lpddr->qinfo->HWPartsNum;
+ for (i = 0; i < numchips; i++) {
+ shared[i].writing = shared[i].erasing = NULL;
+ spin_lock_init(&shared[i].lock);
+ for (j = 0; j < lpddr->qinfo->HWPartsNum; j++) {
+ *chip = lpddr->chips[i];
+ chip->start += j << lpddr->chipshift;
+ chip->oldstate = chip->state = FL_READY;
+ chip->priv = &shared[i];
+ /* those should be reset too since
+ they create memory references. */
+ init_waitqueue_head(&chip->wq);
+ spin_lock_init(&chip->_spinlock);
+ chip->mutex = &chip->_spinlock;
+ chip++;
+ }
+ }
+
+ return mtd;
+}
+EXPORT_SYMBOL(lpddr_cmdset);
+
+static int wait_for_ready(struct map_info *map, struct flchip *chip,
+ unsigned int chip_op_time)
+{
+ unsigned int timeo, reset_timeo, sleep_time;
+ unsigned int dsr;
+ flstate_t chip_state = chip->state;
+ int ret = 0;
+
+ /* set our timeout to 8 times the expected delay */
+ timeo = chip_op_time * 8;
+ if (!timeo)
+ timeo = 500000;
+ reset_timeo = timeo;
+ sleep_time = chip_op_time / 2;
+
+ for (;;) {
+ dsr = CMDVAL(map_read(map, map->pfow_base + PFOW_DSR));
+ if (dsr & DSR_READY_STATUS)
+ break;
+ if (!timeo) {
+ printk(KERN_ERR "%s: Flash timeout error state %d \n",
+ map->name, chip_state);
+ ret = -ETIME;
+ break;
+ }
+
+ /* OK Still waiting. Drop the lock, wait a while and retry. */
+ spin_unlock(chip->mutex);
+ if (sleep_time >= 1000000/HZ) {
+ /*
+ * Half of the normal delay still remaining
+ * can be performed with a sleeping delay instead
+ * of busy waiting.
+ */
+ msleep(sleep_time/1000);
+ timeo -= sleep_time;
+ sleep_time = 1000000/HZ;
+ } else {
+ udelay(1);
+ cond_resched();
+ timeo--;
+ }
+ spin_lock(chip->mutex);
+
+ while (chip->state != chip_state) {
+ /* Someone's suspended the operation: sleep */
+ DECLARE_WAITQUEUE(wait, current);
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ add_wait_queue(&chip->wq, &wait);
+ spin_unlock(chip->mutex);
+ schedule();
+ remove_wait_queue(&chip->wq, &wait);
+ spin_lock(chip->mutex);
+ }
+ if (chip->erase_suspended || chip->write_suspended) {
+ /* Suspend has occured while sleep: reset timeout */
+ timeo = reset_timeo;
+ chip->erase_suspended = chip->write_suspended = 0;
+ }
+ }
+ /* check status for errors */
+ if (dsr & DSR_ERR) {
+ /* Clear DSR*/
+ map_write(map, CMD(~(DSR_ERR)), map->pfow_base + PFOW_DSR);
+ printk(KERN_WARNING"%s: Bad status on wait: 0x%x \n",
+ map->name, dsr);
+ print_drs_error(dsr);
+ ret = -EIO;
+ }
+ chip->state = FL_READY;
+ return ret;
+}
+
+static int get_chip(struct map_info *map, struct flchip *chip, int mode)
+{
+ int ret;
+ DECLARE_WAITQUEUE(wait, current);
+
+ retry:
+ if (chip->priv && (mode == FL_WRITING || mode == FL_ERASING)
+ && chip->state != FL_SYNCING) {
+ /*
+ * OK. We have possibility for contension on the write/erase
+ * operations which are global to the real chip and not per
+ * partition. So let's fight it over in the partition which
+ * currently has authority on the operation.
+ *
+ * The rules are as follows:
+ *
+ * - any write operation must own shared->writing.
+ *
+ * - any erase operation must own _both_ shared->writing and
+ * shared->erasing.
+ *
+ * - contension arbitration is handled in the owner's context.
+ *
+ * The 'shared' struct can be read and/or written only when
+ * its lock is taken.
+ */
+ struct flchip_shared *shared = chip->priv;
+ struct flchip *contender;
+ spin_lock(&shared->lock);
+ contender = shared->writing;
+ if (contender && contender != chip) {
+ /*
+ * The engine to perform desired operation on this
+ * partition is already in use by someone else.
+ * Let's fight over it in the context of the chip
+ * currently using it. If it is possible to suspend,
+ * that other partition will do just that, otherwise
+ * it'll happily send us to sleep. In any case, when
+ * get_chip returns success we're clear to go ahead.
+ */
+ ret = spin_trylock(contender->mutex);
+ spin_unlock(&shared->lock);
+ if (!ret)
+ goto retry;
+ spin_unlock(chip->mutex);
+ ret = chip_ready(map, contender, mode);
+ spin_lock(chip->mutex);
+
+ if (ret == -EAGAIN) {
+ spin_unlock(contender->mutex);
+ goto retry;
+ }
+ if (ret) {
+ spin_unlock(contender->mutex);
+ return ret;
+ }
+ spin_lock(&shared->lock);
+
+ /* We should not own chip if it is already in FL_SYNCING
+ * state. Put contender and retry. */
+ if (chip->state == FL_SYNCING) {
+ put_chip(map, contender);
+ spin_unlock(contender->mutex);
+ goto retry;
+ }
+ spin_unlock(contender->mutex);
+ }
+
+ /* Check if we have suspended erase on this chip.
+ Must sleep in such a case. */
+ if (mode == FL_ERASING && shared->erasing
+ && shared->erasing->oldstate == FL_ERASING) {
+ spin_unlock(&shared->lock);
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ add_wait_queue(&chip->wq, &wait);
+ spin_unlock(chip->mutex);
+ schedule();
+ remove_wait_queue(&chip->wq, &wait);
+ spin_lock(chip->mutex);
+ goto retry;
+ }
+
+ /* We now own it */
+ shared->writing = chip;
+ if (mode == FL_ERASING)
+ shared->erasing = chip;
+ spin_unlock(&shared->lock);
+ }
+
+ ret = chip_ready(map, chip, mode);
+ if (ret == -EAGAIN)
+ goto retry;
+
+ return ret;
+}
+
+static int chip_ready(struct map_info *map, struct flchip *chip, int mode)
+{
+ struct lpddr_private *lpddr = map->fldrv_priv;
+ int ret = 0;
+ DECLARE_WAITQUEUE(wait, current);
+
+ /* Prevent setting state FL_SYNCING for chip in suspended state. */
+ if (FL_SYNCING == mode && FL_READY != chip->oldstate)
+ goto sleep;
+
+ switch (chip->state) {
+ case FL_READY:
+ case FL_JEDEC_QUERY:
+ return 0;
+
+ case FL_ERASING:
+ if (!lpddr->qinfo->SuspEraseSupp ||
+ !(mode == FL_READY || mode == FL_POINT))
+ goto sleep;
+
+ map_write(map, CMD(LPDDR_SUSPEND),
+ map->pfow_base + PFOW_PROGRAM_ERASE_SUSPEND);
+ chip->oldstate = FL_ERASING;
+ chip->state = FL_ERASE_SUSPENDING;
+ ret = wait_for_ready(map, chip, 0);
+ if (ret) {
+ /* Oops. something got wrong. */
+ /* Resume and pretend we weren't here. */
+ map_write(map, CMD(LPDDR_RESUME),
+ map->pfow_base + PFOW_COMMAND_CODE);
+ map_write(map, CMD(LPDDR_START_EXECUTION),
+ map->pfow_base + PFOW_COMMAND_EXECUTE);
+ chip->state = FL_ERASING;
+ chip->oldstate = FL_READY;
+ printk(KERN_ERR "%s: suspend operation failed."
+ "State may be wrong \n", map->name);
+ return -EIO;
+ }
+ chip->erase_suspended = 1;
+ chip->state = FL_READY;
+ return 0;
+ /* Erase suspend */
+ case FL_POINT:
+ /* Only if there's no operation suspended... */
+ if (mode == FL_READY && chip->oldstate == FL_READY)
+ return 0;
+
+ default:
+sleep:
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ add_wait_queue(&chip->wq, &wait);
+ spin_unlock(chip->mutex);
+ schedule();
+ remove_wait_queue(&chip->wq, &wait);
+ spin_lock(chip->mutex);
+ return -EAGAIN;
+ }
+}
+
+static void put_chip(struct map_info *map, struct flchip *chip)
+{
+ if (chip->priv) {
+ struct flchip_shared *shared = chip->priv;
+ spin_lock(&shared->lock);
+ if (shared->writing == chip && chip->oldstate == FL_READY) {
+ /* We own the ability to write, but we're done */
+ shared->writing = shared->erasing;
+ if (shared->writing && shared->writing != chip) {
+ /* give back the ownership */
+ struct flchip *loaner = shared->writing;
+ spin_lock(loaner->mutex);
+ spin_unlock(&shared->lock);
+ spin_unlock(chip->mutex);
+ put_chip(map, loaner);
+ spin_lock(chip->mutex);
+ spin_unlock(loaner->mutex);
+ wake_up(&chip->wq);
+ return;
+ }
+ shared->erasing = NULL;
+ shared->writing = NULL;
+ } else if (shared->erasing == chip && shared->writing != chip) {
+ /*
+ * We own the ability to erase without the ability
+ * to write, which means the erase was suspended
+ * and some other partition is currently writing.
+ * Don't let the switch below mess things up since
+ * we don't have ownership to resume anything.
+ */
+ spin_unlock(&shared->lock);
+ wake_up(&chip->wq);
+ return;
+ }
+ spin_unlock(&shared->lock);
+ }
+
+ switch (chip->oldstate) {
+ case FL_ERASING:
+ chip->state = chip->oldstate;
+ map_write(map, CMD(LPDDR_RESUME),
+ map->pfow_base + PFOW_COMMAND_CODE);
+ map_write(map, CMD(LPDDR_START_EXECUTION),
+ map->pfow_base + PFOW_COMMAND_EXECUTE);
+ chip->oldstate = FL_READY;
+ chip->state = FL_ERASING;
+ break;
+ case FL_READY:
+ break;
+ default:
+ printk(KERN_ERR "%s: put_chip() called with oldstate %d!\n",
+ map->name, chip->oldstate);
+ }
+ wake_up(&chip->wq);
+}
+
+int do_write_buffer(struct map_info *map, struct flchip *chip,
+ unsigned long adr, const struct kvec **pvec,
+ unsigned long *pvec_seek, int len)
+{
+ struct lpddr_private *lpddr = map->fldrv_priv;
+ map_word datum;
+ int ret, wbufsize, word_gap, words;
+ const struct kvec *vec;
+ unsigned long vec_seek;
+ unsigned long prog_buf_ofs;
+
+ wbufsize = 1 << lpddr->qinfo->BufSizeShift;
+
+ spin_lock(chip->mutex);
+ ret = get_chip(map, chip, FL_WRITING);
+ if (ret) {
+ spin_unlock(chip->mutex);
+ return ret;
+ }
+ /* Figure out the number of words to write */
+ word_gap = (-adr & (map_bankwidth(map)-1));
+ words = (len - word_gap + map_bankwidth(map) - 1) / map_bankwidth(map);
+ if (!word_gap) {
+ words--;
+ } else {
+ word_gap = map_bankwidth(map) - word_gap;
+ adr -= word_gap;
+ datum = map_word_ff(map);
+ }
+ /* Write data */
+ /* Get the program buffer offset from PFOW register data first*/
+ prog_buf_ofs = map->pfow_base + CMDVAL(map_read(map,
+ map->pfow_base + PFOW_PROGRAM_BUFFER_OFFSET));
+ vec = *pvec;
+ vec_seek = *pvec_seek;
+ do {
+ int n = map_bankwidth(map) - word_gap;
+
+ if (n > vec->iov_len - vec_seek)
+ n = vec->iov_len - vec_seek;
+ if (n > len)
+ n = len;
+
+ if (!word_gap && (len < map_bankwidth(map)))
+ datum = map_word_ff(map);
+
+ datum = map_word_load_partial(map, datum,
+ vec->iov_base + vec_seek, word_gap, n);
+
+ len -= n;
+ word_gap += n;
+ if (!len || word_gap == map_bankwidth(map)) {
+ map_write(map, datum, prog_buf_ofs);
+ prog_buf_ofs += map_bankwidth(map);
+ word_gap = 0;
+ }
+
+ vec_seek += n;
+ if (vec_seek == vec->iov_len) {
+ vec++;
+ vec_seek = 0;
+ }
+ } while (len);
+ *pvec = vec;
+ *pvec_seek = vec_seek;
+
+ /* GO GO GO */
+ send_pfow_command(map, LPDDR_BUFF_PROGRAM, adr, wbufsize, NULL);
+ chip->state = FL_WRITING;
+ ret = wait_for_ready(map, chip, (1<<lpddr->qinfo->ProgBufferTime));
+ if (ret) {
+ printk(KERN_WARNING"%s Buffer program error: %d at %lx; \n",
+ map->name, ret, adr);
+ goto out;
+ }
+
+ out: put_chip(map, chip);
+ spin_unlock(chip->mutex);
+ return ret;
+}
+
+int do_erase_oneblock(struct mtd_info *mtd, loff_t adr)
+{
+ struct map_info *map = mtd->priv;
+ struct lpddr_private *lpddr = map->fldrv_priv;
+ int chipnum = adr >> lpddr->chipshift;
+ struct flchip *chip = &lpddr->chips[chipnum];
+ int ret;
+
+ spin_lock(chip->mutex);
+ ret = get_chip(map, chip, FL_ERASING);
+ if (ret) {
+ spin_unlock(chip->mutex);
+ return ret;
+ }
+ send_pfow_command(map, LPDDR_BLOCK_ERASE, adr, 0, NULL);
+ chip->state = FL_ERASING;
+ ret = wait_for_ready(map, chip, (1<<lpddr->qinfo->BlockEraseTime)*1000);
+ if (ret) {
+ printk(KERN_WARNING"%s Erase block error %d at : %llx\n",
+ map->name, ret, adr);
+ goto out;
+ }
+ out: put_chip(map, chip);
+ spin_unlock(chip->mutex);
+ return ret;
+}
+
+static int lpddr_read(struct mtd_info *mtd, loff_t adr, size_t len,
+ size_t *retlen, u_char *buf)
+{
+ struct map_info *map = mtd->priv;
+ struct lpddr_private *lpddr = map->fldrv_priv;
+ int chipnum = adr >> lpddr->chipshift;
+ struct flchip *chip = &lpddr->chips[chipnum];
+ int ret = 0;
+
+ spin_lock(chip->mutex);
+ ret = get_chip(map, chip, FL_READY);
+ if (ret) {
+ spin_unlock(chip->mutex);
+ return ret;
+ }
+
+ map_copy_from(map, buf, adr, len);
+ *retlen = len;
+
+ put_chip(map, chip);
+ spin_unlock(chip->mutex);
+ return ret;
+}
+
+static int lpddr_point(struct mtd_info *mtd, loff_t adr, size_t len,
+ size_t *retlen, void **mtdbuf, resource_size_t *phys)
+{
+ struct map_info *map = mtd->priv;
+ struct lpddr_private *lpddr = map->fldrv_priv;
+ int chipnum = adr >> lpddr->chipshift;
+ unsigned long ofs, last_end = 0;
+ struct flchip *chip = &lpddr->chips[chipnum];
+ int ret = 0;
+
+ if (!map->virt || (adr + len > mtd->size))
+ return -EINVAL;
+
+ /* ofs: offset within the first chip that the first read should start */
+ ofs = adr - (chipnum << lpddr->chipshift);
+
+ *mtdbuf = (void *)map->virt + chip->start + ofs;
+ *retlen = 0;
+
+ while (len) {
+ unsigned long thislen;
+
+ if (chipnum >= lpddr->numchips)
+ break;
+
+ /* We cannot point across chips that are virtually disjoint */
+ if (!last_end)
+ last_end = chip->start;
+ else if (chip->start != last_end)
+ break;
+
+ if ((len + ofs - 1) >> lpddr->chipshift)
+ thislen = (1<<lpddr->chipshift) - ofs;
+ else
+ thislen = len;
+ /* get the chip */
+ spin_lock(chip->mutex);
+ ret = get_chip(map, chip, FL_POINT);
+ spin_unlock(chip->mutex);
+ if (ret)
+ break;
+
+ chip->state = FL_POINT;
+ chip->ref_point_counter++;
+ *retlen += thislen;
+ len -= thislen;
+
+ ofs = 0;
+ last_end += 1 << lpddr->chipshift;
+ chipnum++;
+ chip = &lpddr->chips[chipnum];
+ }
+ return 0;
+}
+
+static void lpddr_unpoint (struct mtd_info *mtd, loff_t adr, size_t len)
+{
+ struct map_info *map = mtd->priv;
+ struct lpddr_private *lpddr = map->fldrv_priv;
+ int chipnum = adr >> lpddr->chipshift;
+ unsigned long ofs;
+
+ /* ofs: offset within the first chip that the first read should start */
+ ofs = adr - (chipnum << lpddr->chipshift);
+
+ while (len) {
+ unsigned long thislen;
+ struct flchip *chip;
+
+ chip = &lpddr->chips[chipnum];
+ if (chipnum >= lpddr->numchips)
+ break;
+
+ if ((len + ofs - 1) >> lpddr->chipshift)
+ thislen = (1<<lpddr->chipshift) - ofs;
+ else
+ thislen = len;
+
+ spin_lock(chip->mutex);
+ if (chip->state == FL_POINT) {
+ chip->ref_point_counter--;
+ if (chip->ref_point_counter == 0)
+ chip->state = FL_READY;
+ } else
+ printk(KERN_WARNING "%s: Warning: unpoint called on non"
+ "pointed region\n", map->name);
+
+ put_chip(map, chip);
+ spin_unlock(chip->mutex);
+
+ len -= thislen;
+ ofs = 0;
+ chipnum++;
+ }
+}
+
+static int lpddr_write_buffers(struct mtd_info *mtd, loff_t to, size_t len,
+ size_t *retlen, const u_char *buf)
+{
+ struct kvec vec;
+
+ vec.iov_base = (void *) buf;
+ vec.iov_len = len;
+
+ return lpddr_writev(mtd, &vec, 1, to, retlen);
+}
+
+
+static int lpddr_writev(struct mtd_info *mtd, const struct kvec *vecs,
+ unsigned long count, loff_t to, size_t *retlen)
+{
+ struct map_info *map = mtd->priv;
+ struct lpddr_private *lpddr = map->fldrv_priv;
+ int ret = 0;
+ int chipnum;
+ unsigned long ofs, vec_seek, i;
+ int wbufsize = 1 << lpddr->qinfo->BufSizeShift;
+
+ size_t len = 0;
+
+ for (i = 0; i < count; i++)
+ len += vecs[i].iov_len;
+
+ *retlen = 0;
+ if (!len)
+ return 0;
+
+ chipnum = to >> lpddr->chipshift;
+
+ ofs = to;
+ vec_seek = 0;
+
+ do {
+ /* We must not cross write block boundaries */
+ int size = wbufsize - (ofs & (wbufsize-1));
+
+ if (size > len)
+ size = len;
+
+ ret = do_write_buffer(map, &lpddr->chips[chipnum],
+ ofs, &vecs, &vec_seek, size);
+ if (ret)
+ return ret;
+
+ ofs += size;
+ (*retlen) += size;
+ len -= size;
+
+ /* Be nice and reschedule with the chip in a usable
+ * state for other processes */
+ cond_resched();
+
+ } while (len);
+
+ return 0;
+}
+
+static int lpddr_erase(struct mtd_info *mtd, struct erase_info *instr)
+{
+ unsigned long ofs, len;
+ int ret;
+ struct map_info *map = mtd->priv;
+ struct lpddr_private *lpddr = map->fldrv_priv;
+ int size = 1 << lpddr->qinfo->UniformBlockSizeShift;
+
+ ofs = instr->addr;
+ len = instr->len;
+
+ if (ofs > mtd->size || (len + ofs) > mtd->size)
+ return -EINVAL;
+
+ while (len > 0) {
+ ret = do_erase_oneblock(mtd, ofs);
+ if (ret)
+ return ret;
+ ofs += size;
+ len -= size;
+ }
+ instr->state = MTD_ERASE_DONE;
+ mtd_erase_callback(instr);
+
+ return 0;
+}
+
+#define DO_XXLOCK_LOCK 1
+#define DO_XXLOCK_UNLOCK 2
+int do_xxlock(struct mtd_info *mtd, loff_t adr, uint32_t len, int thunk)
+{
+ int ret = 0;
+ struct map_info *map = mtd->priv;
+ struct lpddr_private *lpddr = map->fldrv_priv;
+ int chipnum = adr >> lpddr->chipshift;
+ struct flchip *chip = &lpddr->chips[chipnum];
+
+ spin_lock(chip->mutex);
+ ret = get_chip(map, chip, FL_LOCKING);
+ if (ret) {
+ spin_unlock(chip->mutex);
+ return ret;
+ }
+
+ if (thunk == DO_XXLOCK_LOCK) {
+ send_pfow_command(map, LPDDR_LOCK_BLOCK, adr, adr + len, NULL);
+ chip->state = FL_LOCKING;
+ } else if (thunk == DO_XXLOCK_UNLOCK) {
+ send_pfow_command(map, LPDDR_UNLOCK_BLOCK, adr, adr + len, NULL);
+ chip->state = FL_UNLOCKING;
+ } else
+ BUG();
+
+ ret = wait_for_ready(map, chip, 1);
+ if (ret) {
+ printk(KERN_ERR "%s: block unlock error status %d \n",
+ map->name, ret);
+ goto out;
+ }
+out: put_chip(map, chip);
+ spin_unlock(chip->mutex);
+ return ret;
+}
+
+static int lpddr_lock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
+{
+ return do_xxlock(mtd, ofs, len, DO_XXLOCK_LOCK);
+}
+
+static int lpddr_unlock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
+{
+ return do_xxlock(mtd, ofs, len, DO_XXLOCK_UNLOCK);
+}
+
+int word_program(struct map_info *map, loff_t adr, uint32_t curval)
+{
+ int ret;
+ struct lpddr_private *lpddr = map->fldrv_priv;
+ int chipnum = adr >> lpddr->chipshift;
+ struct flchip *chip = &lpddr->chips[chipnum];
+
+ spin_lock(chip->mutex);
+ ret = get_chip(map, chip, FL_WRITING);
+ if (ret) {
+ spin_unlock(chip->mutex);
+ return ret;
+ }
+
+ send_pfow_command(map, LPDDR_WORD_PROGRAM, adr, 0x00, (map_word *)&curval);
+
+ ret = wait_for_ready(map, chip, (1<<lpddr->qinfo->SingleWordProgTime));
+ if (ret) {
+ printk(KERN_WARNING"%s word_program error at: %llx; val: %x\n",
+ map->name, adr, curval);
+ goto out;
+ }
+
+out: put_chip(map, chip);
+ spin_unlock(chip->mutex);
+ return ret;
+}
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Alexey Korolev <akorolev@infradead.org>");
+MODULE_DESCRIPTION("MTD driver for LPDDR flash chips");
diff --git a/drivers/mtd/lpddr/qinfo_probe.c b/drivers/mtd/lpddr/qinfo_probe.c
new file mode 100644
index 0000000..79bf40f
--- /dev/null
+++ b/drivers/mtd/lpddr/qinfo_probe.c
@@ -0,0 +1,255 @@
+/*
+ * Probing flash chips with QINFO records.
+ * (C) 2008 Korolev Alexey <akorolev@infradead.org>
+ * (C) 2008 Vasiliy Leonenko <vasiliy.leonenko@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+ * 02110-1301, USA.
+ */
+#include <linux/module.h>
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/errno.h>
+#include <linux/slab.h>
+#include <linux/interrupt.h>
+
+#include <linux/mtd/xip.h>
+#include <linux/mtd/map.h>
+#include <linux/mtd/pfow.h>
+#include <linux/mtd/qinfo.h>
+
+static int lpddr_chip_setup(struct map_info *map, struct lpddr_private *lpddr);
+struct mtd_info *lpddr_probe(struct map_info *map);
+static struct lpddr_private *lpddr_probe_chip(struct map_info *map);
+static int lpddr_pfow_present(struct map_info *map,
+ struct lpddr_private *lpddr);
+
+static struct qinfo_query_info qinfo_array[] = {
+ /* General device info */
+ {0, 0, "DevSizeShift", "Device size 2^n bytes"},
+ {0, 3, "BufSizeShift", "Program buffer size 2^n bytes"},
+ /* Erase block information */
+ {1, 1, "TotalBlocksNum", "Total number of blocks"},
+ {1, 2, "UniformBlockSizeShift", "Uniform block size 2^n bytes"},
+ /* Partition information */
+ {2, 1, "HWPartsNum", "Number of hardware partitions"},
+ /* Optional features */
+ {5, 1, "SuspEraseSupp", "Suspend erase supported"},
+ /* Operation typical time */
+ {10, 0, "SingleWordProgTime", "Single word program 2^n u-sec"},
+ {10, 1, "ProgBufferTime", "Program buffer write 2^n u-sec"},
+ {10, 2, "BlockEraseTime", "Block erase 2^n m-sec"},
+ {10, 3, "FullChipEraseTime", "Full chip erase 2^n m-sec"},
+};
+
+static long lpddr_get_qinforec_pos(struct map_info *map, char *id_str)
+{
+ int qinfo_lines = sizeof(qinfo_array)/sizeof(struct qinfo_query_info);
+ int i;
+ int bankwidth = map_bankwidth(map) * 8;
+ int major, minor;
+
+ for (i = 0; i < qinfo_lines; i++) {
+ if (strcmp(id_str, qinfo_array[i].id_str) == 0) {
+ major = qinfo_array[i].major & ((1 << bankwidth) - 1);
+ minor = qinfo_array[i].minor & ((1 << bankwidth) - 1);
+ return minor | (major << bankwidth);
+ }
+ }
+ printk(KERN_ERR"%s qinfo id string is wrong! \n", map->name);
+ BUG();
+ return -1;
+}
+
+static uint16_t lpddr_info_query(struct map_info *map, char *id_str)
+{
+ unsigned int dsr, val;
+ int bits_per_chip = map_bankwidth(map) * 8;
+ unsigned long adr = lpddr_get_qinforec_pos(map, id_str);
+ int attempts = 20;
+
+ /* Write a request for the PFOW record */
+ map_write(map, CMD(LPDDR_INFO_QUERY),
+ map->pfow_base + PFOW_COMMAND_CODE);
+ map_write(map, CMD(adr & ((1 << bits_per_chip) - 1)),
+ map->pfow_base + PFOW_COMMAND_ADDRESS_L);
+ map_write(map, CMD(adr >> bits_per_chip),
+ map->pfow_base + PFOW_COMMAND_ADDRESS_H);
+ map_write(map, CMD(LPDDR_START_EXECUTION),
+ map->pfow_base + PFOW_COMMAND_EXECUTE);
+
+ while ((attempts--) > 0) {
+ dsr = CMDVAL(map_read(map, map->pfow_base + PFOW_DSR));
+ if (dsr & DSR_READY_STATUS)
+ break;
+ udelay(10);
+ }
+
+ val = CMDVAL(map_read(map, map->pfow_base + PFOW_COMMAND_DATA));
+ return val;
+}
+
+static int lpddr_pfow_present(struct map_info *map, struct lpddr_private *lpddr)
+{
+ map_word pfow_val[4];
+
+ /* Check identification string */
+ pfow_val[0] = map_read(map, map->pfow_base + PFOW_QUERY_STRING_P);
+ pfow_val[1] = map_read(map, map->pfow_base + PFOW_QUERY_STRING_F);
+ pfow_val[2] = map_read(map, map->pfow_base + PFOW_QUERY_STRING_O);
+ pfow_val[3] = map_read(map, map->pfow_base + PFOW_QUERY_STRING_W);
+
+ if (!map_word_equal(map, CMD('P'), pfow_val[0]))
+ goto out;
+
+ if (!map_word_equal(map, CMD('F'), pfow_val[1]))
+ goto out;
+
+ if (!map_word_equal(map, CMD('O'), pfow_val[2]))
+ goto out;
+
+ if (!map_word_equal(map, CMD('W'), pfow_val[3]))
+ goto out;
+
+ return 1; /* "PFOW" is found */
+out:
+ printk(KERN_WARNING"%s: PFOW string at 0x%lx is not found \n",
+ map->name, map->pfow_base);
+ return 0;
+}
+
+static int lpddr_chip_setup(struct map_info *map, struct lpddr_private *lpddr)
+{
+
+ lpddr->qinfo = kmalloc(sizeof(struct qinfo_chip), GFP_KERNEL);
+ if (!lpddr->qinfo) {
+ printk(KERN_WARNING "%s: no memory for LPDDR qinfo structure\n",
+ map->name);
+ return 0;
+ }
+ memset(lpddr->qinfo, 0, sizeof(struct qinfo_chip));
+
+ /* Get the ManuID */
+ lpddr->ManufactId = CMDVAL(map_read(map, map->pfow_base + PFOW_MANUFACTURER_ID));
+ /* Get the DeviceID */
+ lpddr->DevId = CMDVAL(map_read(map, map->pfow_base + PFOW_DEVICE_ID));
+ /* read parameters from chip qinfo table */
+ lpddr->qinfo->DevSizeShift = lpddr_info_query(map, "DevSizeShift");
+ lpddr->qinfo->TotalBlocksNum = lpddr_info_query(map, "TotalBlocksNum");
+ lpddr->qinfo->BufSizeShift = lpddr_info_query(map, "BufSizeShift");
+ lpddr->qinfo->HWPartsNum = lpddr_info_query(map, "HWPartsNum");
+ lpddr->qinfo->UniformBlockSizeShift =
+ lpddr_info_query(map, "UniformBlockSizeShift");
+ lpddr->qinfo->SuspEraseSupp = lpddr_info_query(map, "SuspEraseSupp");
+ lpddr->qinfo->SingleWordProgTime =
+ lpddr_info_query(map, "SingleWordProgTime");
+ lpddr->qinfo->ProgBufferTime = lpddr_info_query(map, "ProgBufferTime");
+ lpddr->qinfo->BlockEraseTime = lpddr_info_query(map, "BlockEraseTime");
+ return 1;
+}
+static struct lpddr_private *lpddr_probe_chip(struct map_info *map)
+{
+ struct lpddr_private lpddr;
+ struct lpddr_private *retlpddr;
+ int numvirtchips;
+
+
+ if ((map->pfow_base + 0x1000) >= map->size) {
+ printk(KERN_NOTICE"%s Probe at base (0x%08lx) past the end of"
+ "the map(0x%08lx)\n", map->name,
+ (unsigned long)map->pfow_base, map->size - 1);
+ return NULL;
+ }
+ memset(&lpddr, 0, sizeof(struct lpddr_private));
+ if (!lpddr_pfow_present(map, &lpddr))
+ return NULL;
+
+ if (!lpddr_chip_setup(map, &lpddr))
+ return NULL;
+
+ /* Ok so we found a chip */
+ lpddr.chipshift = lpddr.qinfo->DevSizeShift;
+ lpddr.numchips = 1;
+
+ numvirtchips = lpddr.numchips * lpddr.qinfo->HWPartsNum;
+ retlpddr = kmalloc(sizeof(struct lpddr_private) +
+ numvirtchips * sizeof(struct flchip), GFP_KERNEL);
+ if (!retlpddr)
+ return NULL;
+
+ memset(retlpddr, 0, sizeof(struct lpddr_private) +
+ numvirtchips * sizeof(struct flchip));
+ memcpy(retlpddr, &lpddr, sizeof(struct lpddr_private));
+
+ retlpddr->numchips = numvirtchips;
+ retlpddr->chipshift = retlpddr->qinfo->DevSizeShift -
+ __ffs(retlpddr->qinfo->HWPartsNum);
+
+ return retlpddr;
+}
+
+struct mtd_info *lpddr_probe(struct map_info *map)
+{
+ struct mtd_info *mtd = NULL;
+ struct lpddr_private *lpddr;
+
+ /* First probe the map to see if we havecan open PFOW here */
+ lpddr = lpddr_probe_chip(map);
+ if (!lpddr)
+ return NULL;
+
+ map->fldrv_priv = lpddr;
+ mtd = lpddr_cmdset(map);
+ if (mtd) {
+ if (mtd->size > map->size) {
+ printk(KERN_WARNING "Reducing visibility of %ldKiB chip"
+ "to %ldKiB\n", (unsigned long)mtd->size >> 10,
+ (unsigned long)map->size >> 10);
+ mtd->size = map->size;
+ }
+ return mtd;
+ }
+
+ kfree(lpddr->qinfo);
+ kfree(lpddr);
+ map->fldrv_priv = NULL;
+ return NULL;
+}
+
+static struct mtd_chip_driver lpddr_chipdrv = {
+ .probe = lpddr_probe,
+ .name = "qinfo_probe",
+ .module = THIS_MODULE
+};
+
+static int __init lpddr_probe_init(void)
+{
+ register_mtd_chip_driver(&lpddr_chipdrv);
+ return 0;
+}
+
+static void __exit lpddr_probe_exit(void)
+{
+ unregister_mtd_chip_driver(&lpddr_chipdrv);
+}
+
+module_init(lpddr_probe_init);
+module_exit(lpddr_probe_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Vasiliy Leonenko <vasiliy.leonenko@gmail.com>");
+MODULE_DESCRIPTION("Driver to probe qinfo flash chips");
+
diff --git a/drivers/mtd/maps/Kconfig b/drivers/mtd/maps/Kconfig
index 5ea1693..0225cbb 100644
--- a/drivers/mtd/maps/Kconfig
+++ b/drivers/mtd/maps/Kconfig
@@ -10,8 +10,8 @@
paged mappings of flash chips.
config MTD_PHYSMAP
- tristate "CFI Flash device in physical memory map"
- depends on MTD_CFI || MTD_JEDECPROBE || MTD_ROM
+ tristate "Flash device in physical memory map"
+ depends on MTD_CFI || MTD_JEDECPROBE || MTD_ROM || MTD_LPDDR
help
This provides a 'mapping' driver which allows the NOR Flash and
ROM driver code to communicate with chips which are mapped
@@ -23,9 +23,20 @@
To compile this driver as a module, choose M here: the
module will be called physmap.
+config MTD_PHYSMAP_COMPAT
+ bool "Physmap compat support"
+ depends on MTD_PHYSMAP
+ default n
+ help
+ Setup a simple mapping via the Kconfig options. Normally the
+ physmap configuration options are done via your board's
+ resource file.
+
+ If unsure, say N here.
+
config MTD_PHYSMAP_START
hex "Physical start address of flash mapping"
- depends on MTD_PHYSMAP
+ depends on MTD_PHYSMAP_COMPAT
default "0x8000000"
help
This is the physical memory location at which the flash chips
@@ -37,7 +48,7 @@
config MTD_PHYSMAP_LEN
hex "Physical length of flash mapping"
- depends on MTD_PHYSMAP
+ depends on MTD_PHYSMAP_COMPAT
default "0"
help
This is the total length of the mapping of the flash chips on
@@ -51,7 +62,7 @@
config MTD_PHYSMAP_BANKWIDTH
int "Bank width in octets"
- depends on MTD_PHYSMAP
+ depends on MTD_PHYSMAP_COMPAT
default "2"
help
This is the total width of the data bus of the flash devices
diff --git a/drivers/mtd/maps/alchemy-flash.c b/drivers/mtd/maps/alchemy-flash.c
index 82811bc..845ad4f 100644
--- a/drivers/mtd/maps/alchemy-flash.c
+++ b/drivers/mtd/maps/alchemy-flash.c
@@ -111,7 +111,7 @@
static struct mtd_info *mymtd;
-int __init alchemy_mtd_init(void)
+static int __init alchemy_mtd_init(void)
{
struct mtd_partition *parts;
int nb_parts = 0;
diff --git a/drivers/mtd/maps/amd76xrom.c b/drivers/mtd/maps/amd76xrom.c
index d1eec7d..237733d 100644
--- a/drivers/mtd/maps/amd76xrom.c
+++ b/drivers/mtd/maps/amd76xrom.c
@@ -232,8 +232,8 @@
/* Trim the size if we are larger than the map */
if (map->mtd->size > map->map.size) {
printk(KERN_WARNING MOD_NAME
- " rom(%u) larger than window(%lu). fixing...\n",
- map->mtd->size, map->map.size);
+ " rom(%llu) larger than window(%lu). fixing...\n",
+ (unsigned long long)map->mtd->size, map->map.size);
map->mtd->size = map->map.size;
}
if (window->rsrc.parent) {
diff --git a/drivers/mtd/maps/cfi_flagadm.c b/drivers/mtd/maps/cfi_flagadm.c
index 0ecc3f6..b4ed816 100644
--- a/drivers/mtd/maps/cfi_flagadm.c
+++ b/drivers/mtd/maps/cfi_flagadm.c
@@ -88,7 +88,7 @@
static struct mtd_info *mymtd;
-int __init init_flagadm(void)
+static int __init init_flagadm(void)
{
printk(KERN_NOTICE "FlagaDM flash device: %x at %x\n",
FLASH_SIZE, FLASH_PHYS_ADDR);
diff --git a/drivers/mtd/maps/ck804xrom.c b/drivers/mtd/maps/ck804xrom.c
index 1a6feb4..5f7a245 100644
--- a/drivers/mtd/maps/ck804xrom.c
+++ b/drivers/mtd/maps/ck804xrom.c
@@ -263,8 +263,8 @@
/* Trim the size if we are larger than the map */
if (map->mtd->size > map->map.size) {
printk(KERN_WARNING MOD_NAME
- " rom(%u) larger than window(%lu). fixing...\n",
- map->mtd->size, map->map.size);
+ " rom(%llu) larger than window(%lu). fixing...\n",
+ (unsigned long long)map->mtd->size, map->map.size);
map->mtd->size = map->map.size;
}
if (window->rsrc.parent) {
diff --git a/drivers/mtd/maps/dbox2-flash.c b/drivers/mtd/maps/dbox2-flash.c
index e115667..cfacfa6 100644
--- a/drivers/mtd/maps/dbox2-flash.c
+++ b/drivers/mtd/maps/dbox2-flash.c
@@ -69,7 +69,7 @@
.phys = WINDOW_ADDR,
};
-int __init init_dbox2_flash(void)
+static int __init init_dbox2_flash(void)
{
printk(KERN_NOTICE "D-Box 2 flash driver (size->0x%X mem->0x%X)\n", WINDOW_SIZE, WINDOW_ADDR);
dbox2_flash_map.virt = ioremap(WINDOW_ADDR, WINDOW_SIZE);
diff --git a/drivers/mtd/maps/edb7312.c b/drivers/mtd/maps/edb7312.c
index 9433738..be9e90b 100644
--- a/drivers/mtd/maps/edb7312.c
+++ b/drivers/mtd/maps/edb7312.c
@@ -71,7 +71,7 @@
static int mtd_parts_nb = 0;
static struct mtd_partition *mtd_parts = 0;
-int __init init_edb7312nor(void)
+static int __init init_edb7312nor(void)
{
static const char *rom_probe_types[] = PROBETYPES;
const char **type;
diff --git a/drivers/mtd/maps/esb2rom.c b/drivers/mtd/maps/esb2rom.c
index bbbcdd4..11a2f57 100644
--- a/drivers/mtd/maps/esb2rom.c
+++ b/drivers/mtd/maps/esb2rom.c
@@ -324,8 +324,8 @@
/* Trim the size if we are larger than the map */
if (map->mtd->size > map->map.size) {
printk(KERN_WARNING MOD_NAME
- " rom(%u) larger than window(%lu). fixing...\n",
- map->mtd->size, map->map.size);
+ " rom(%llu) larger than window(%lu). fixing...\n",
+ (unsigned long long)map->mtd->size, map->map.size);
map->mtd->size = map->map.size;
}
if (window->rsrc.parent) {
diff --git a/drivers/mtd/maps/fortunet.c b/drivers/mtd/maps/fortunet.c
index a8e3fde..1e43124 100644
--- a/drivers/mtd/maps/fortunet.c
+++ b/drivers/mtd/maps/fortunet.c
@@ -181,7 +181,7 @@
/* Backwards-spelling-compatibility */
__setup("MTD_Partion=", MTD_New_Partition);
-int __init init_fortunet(void)
+static int __init init_fortunet(void)
{
int ix,iy;
for(iy=ix=0;ix<MAX_NUM_REGIONS;ix++)
diff --git a/drivers/mtd/maps/h720x-flash.c b/drivers/mtd/maps/h720x-flash.c
index 3b959fa..72c724f 100644
--- a/drivers/mtd/maps/h720x-flash.c
+++ b/drivers/mtd/maps/h720x-flash.c
@@ -65,7 +65,7 @@
/*
* Initialize FLASH support
*/
-int __init h720x_mtd_init(void)
+static int __init h720x_mtd_init(void)
{
char *part_type = NULL;
diff --git a/drivers/mtd/maps/ichxrom.c b/drivers/mtd/maps/ichxrom.c
index aeb6c91..c32bc28 100644
--- a/drivers/mtd/maps/ichxrom.c
+++ b/drivers/mtd/maps/ichxrom.c
@@ -258,8 +258,8 @@
/* Trim the size if we are larger than the map */
if (map->mtd->size > map->map.size) {
printk(KERN_WARNING MOD_NAME
- " rom(%u) larger than window(%lu). fixing...\n",
- map->mtd->size, map->map.size);
+ " rom(%llu) larger than window(%lu). fixing...\n",
+ (unsigned long long)map->mtd->size, map->map.size);
map->mtd->size = map->map.size;
}
if (window->rsrc.parent) {
diff --git a/drivers/mtd/maps/impa7.c b/drivers/mtd/maps/impa7.c
index 2682ab5..998a27d 100644
--- a/drivers/mtd/maps/impa7.c
+++ b/drivers/mtd/maps/impa7.c
@@ -70,7 +70,7 @@
static const char *probes[] = { "cmdlinepart", NULL };
-int __init init_impa7(void)
+static int __init init_impa7(void)
{
static const char *rom_probe_types[] = PROBETYPES;
const char **type;
diff --git a/drivers/mtd/maps/ipaq-flash.c b/drivers/mtd/maps/ipaq-flash.c
index ed58f6a..748c85f 100644
--- a/drivers/mtd/maps/ipaq-flash.c
+++ b/drivers/mtd/maps/ipaq-flash.c
@@ -202,7 +202,7 @@
static int __init h1900_special_case(void);
-int __init ipaq_mtd_init(void)
+static int __init ipaq_mtd_init(void)
{
struct mtd_partition *parts = NULL;
int nb_parts = 0;
diff --git a/drivers/mtd/maps/mbx860.c b/drivers/mtd/maps/mbx860.c
index 706f673..0eb5a7c 100644
--- a/drivers/mtd/maps/mbx860.c
+++ b/drivers/mtd/maps/mbx860.c
@@ -55,7 +55,7 @@
.bankwidth = 4,
};
-int __init init_mbx(void)
+static int __init init_mbx(void)
{
printk(KERN_NOTICE "Motorola MBX flash device: 0x%x at 0x%x\n", WINDOW_SIZE*4, WINDOW_ADDR);
mbx_map.virt = ioremap(WINDOW_ADDR, WINDOW_SIZE * 4);
diff --git a/drivers/mtd/maps/nettel.c b/drivers/mtd/maps/nettel.c
index 965e6c6..a97133e 100644
--- a/drivers/mtd/maps/nettel.c
+++ b/drivers/mtd/maps/nettel.c
@@ -226,7 +226,7 @@
if ((amd_mtd = do_map_probe("jedec_probe", &nettel_amd_map))) {
printk(KERN_NOTICE "SNAPGEAR: AMD flash device size = %dK\n",
- amd_mtd->size>>10);
+ (int)(amd_mtd->size>>10));
amd_mtd->owner = THIS_MODULE;
@@ -357,13 +357,12 @@
*intel1par = 0;
}
- printk(KERN_NOTICE "SNAPGEAR: Intel flash device size = %dK\n",
- (intel_mtd->size >> 10));
+ printk(KERN_NOTICE "SNAPGEAR: Intel flash device size = %lldKiB\n",
+ (unsigned long long)(intel_mtd->size >> 10));
intel_mtd->owner = THIS_MODULE;
- num_intel_partitions = sizeof(nettel_intel_partitions) /
- sizeof(nettel_intel_partitions[0]);
+ num_intel_partitions = ARRAY_SIZE(nettel_intel_partitions);
if (intelboot) {
/*
diff --git a/drivers/mtd/maps/octagon-5066.c b/drivers/mtd/maps/octagon-5066.c
index 43e04c1..2b2e450 100644
--- a/drivers/mtd/maps/octagon-5066.c
+++ b/drivers/mtd/maps/octagon-5066.c
@@ -184,7 +184,7 @@
release_region(PAGE_IO, 1);
}
-int __init init_oct5066(void)
+static int __init init_oct5066(void)
{
int i;
int ret = 0;
diff --git a/drivers/mtd/maps/physmap.c b/drivers/mtd/maps/physmap.c
index 1db16e5..8774366 100644
--- a/drivers/mtd/maps/physmap.c
+++ b/drivers/mtd/maps/physmap.c
@@ -29,7 +29,6 @@
struct map_info map[MAX_RESOURCES];
#ifdef CONFIG_MTD_PARTITIONS
int nr_parts;
- struct mtd_partition *parts;
#endif
};
@@ -56,14 +55,10 @@
for (i = 0; i < MAX_RESOURCES; i++) {
if (info->mtd[i] != NULL) {
#ifdef CONFIG_MTD_PARTITIONS
- if (info->nr_parts) {
+ if (info->nr_parts || physmap_data->nr_parts)
del_mtd_partitions(info->mtd[i]);
- kfree(info->parts);
- } else if (physmap_data->nr_parts) {
- del_mtd_partitions(info->mtd[i]);
- } else {
+ else
del_mtd_device(info->mtd[i]);
- }
#else
del_mtd_device(info->mtd[i]);
#endif
@@ -73,7 +68,12 @@
return 0;
}
-static const char *rom_probe_types[] = { "cfi_probe", "jedec_probe", "map_rom", NULL };
+static const char *rom_probe_types[] = {
+ "cfi_probe",
+ "jedec_probe",
+ "qinfo_probe",
+ "map_rom",
+ NULL };
#ifdef CONFIG_MTD_PARTITIONS
static const char *part_probe_types[] = { "cmdlinepart", "RedBoot", NULL };
#endif
@@ -86,6 +86,9 @@
int err = 0;
int i;
int devices_found = 0;
+#ifdef CONFIG_MTD_PARTITIONS
+ struct mtd_partition *parts;
+#endif
physmap_data = dev->dev.platform_data;
if (physmap_data == NULL)
@@ -119,6 +122,7 @@
info->map[i].size = dev->resource[i].end - dev->resource[i].start + 1;
info->map[i].bankwidth = physmap_data->width;
info->map[i].set_vpp = physmap_data->set_vpp;
+ info->map[i].pfow_base = physmap_data->pfow_base;
info->map[i].virt = devm_ioremap(&dev->dev, info->map[i].phys,
info->map[i].size);
@@ -163,9 +167,10 @@
goto err_out;
#ifdef CONFIG_MTD_PARTITIONS
- err = parse_mtd_partitions(info->cmtd, part_probe_types, &info->parts, 0);
+ err = parse_mtd_partitions(info->cmtd, part_probe_types, &parts, 0);
if (err > 0) {
- add_mtd_partitions(info->cmtd, info->parts, err);
+ add_mtd_partitions(info->cmtd, parts, err);
+ kfree(parts);
return 0;
}
@@ -251,14 +256,7 @@
};
-#ifdef CONFIG_MTD_PHYSMAP_LEN
-#if CONFIG_MTD_PHYSMAP_LEN != 0
-#warning using PHYSMAP compat code
-#define PHYSMAP_COMPAT
-#endif
-#endif
-
-#ifdef PHYSMAP_COMPAT
+#ifdef CONFIG_MTD_PHYSMAP_COMPAT
static struct physmap_flash_data physmap_flash_data = {
.width = CONFIG_MTD_PHYSMAP_BANKWIDTH,
};
@@ -302,7 +300,7 @@
int err;
err = platform_driver_register(&physmap_flash_driver);
-#ifdef PHYSMAP_COMPAT
+#ifdef CONFIG_MTD_PHYSMAP_COMPAT
if (err == 0)
platform_device_register(&physmap_flash);
#endif
@@ -312,7 +310,7 @@
static void __exit physmap_exit(void)
{
-#ifdef PHYSMAP_COMPAT
+#ifdef CONFIG_MTD_PHYSMAP_COMPAT
platform_device_unregister(&physmap_flash);
#endif
platform_driver_unregister(&physmap_flash_driver);
@@ -326,8 +324,7 @@
MODULE_DESCRIPTION("Generic configurable MTD map driver");
/* legacy platform drivers can't hotplug or coldplg */
-#ifndef PHYSMAP_COMPAT
+#ifndef CONFIG_MTD_PHYSMAP_COMPAT
/* work with hotplug and coldplug */
MODULE_ALIAS("platform:physmap-flash");
#endif
-
diff --git a/drivers/mtd/maps/pmcmsp-flash.c b/drivers/mtd/maps/pmcmsp-flash.c
index f43ba28..4768bd5 100644
--- a/drivers/mtd/maps/pmcmsp-flash.c
+++ b/drivers/mtd/maps/pmcmsp-flash.c
@@ -48,7 +48,7 @@
#define DEBUG_MARKER printk(KERN_NOTICE "%s[%d]\n", __func__, __LINE__)
-int __init init_msp_flash(void)
+static int __init init_msp_flash(void)
{
int i, j;
int offset, coff;
diff --git a/drivers/mtd/maps/redwood.c b/drivers/mtd/maps/redwood.c
index de002eb..933c0b6 100644
--- a/drivers/mtd/maps/redwood.c
+++ b/drivers/mtd/maps/redwood.c
@@ -122,7 +122,7 @@
static struct mtd_info *redwood_mtd;
-int __init init_redwood_flash(void)
+static int __init init_redwood_flash(void)
{
int err;
diff --git a/drivers/mtd/maps/rpxlite.c b/drivers/mtd/maps/rpxlite.c
index 14d90ed..3e3ef53 100644
--- a/drivers/mtd/maps/rpxlite.c
+++ b/drivers/mtd/maps/rpxlite.c
@@ -23,7 +23,7 @@
.phys = WINDOW_ADDR,
};
-int __init init_rpxlite(void)
+static int __init init_rpxlite(void)
{
printk(KERN_NOTICE "RPX Lite or CLLF flash device: %x at %x\n", WINDOW_SIZE*4, WINDOW_ADDR);
rpxlite_map.virt = ioremap(WINDOW_ADDR, WINDOW_SIZE * 4);
diff --git a/drivers/mtd/maps/sbc8240.c b/drivers/mtd/maps/sbc8240.c
index 6e1e99c..d5374cd 100644
--- a/drivers/mtd/maps/sbc8240.c
+++ b/drivers/mtd/maps/sbc8240.c
@@ -136,7 +136,7 @@
#endif /* CONFIG_MTD_PARTITIONS */
-int __init init_sbc8240_mtd (void)
+static int __init init_sbc8240_mtd (void)
{
static struct _cjs {
u_long addr;
diff --git a/drivers/mtd/maps/scb2_flash.c b/drivers/mtd/maps/scb2_flash.c
index 21169e6..7e329f0 100644
--- a/drivers/mtd/maps/scb2_flash.c
+++ b/drivers/mtd/maps/scb2_flash.c
@@ -118,7 +118,8 @@
struct mtd_erase_region_info *region = &mtd->eraseregions[i];
if (region->numblocks * region->erasesize > mtd->size) {
- region->numblocks = (mtd->size / region->erasesize);
+ region->numblocks = ((unsigned long)mtd->size /
+ region->erasesize);
done = 1;
} else {
region->numblocks = 0;
@@ -187,8 +188,9 @@
return -ENODEV;
}
- printk(KERN_NOTICE MODNAME ": chip size 0x%x at offset 0x%x\n",
- scb2_mtd->size, SCB2_WINDOW - scb2_mtd->size);
+ printk(KERN_NOTICE MODNAME ": chip size 0x%llx at offset 0x%llx\n",
+ (unsigned long long)scb2_mtd->size,
+ (unsigned long long)(SCB2_WINDOW - scb2_mtd->size));
add_mtd_device(scb2_mtd);
diff --git a/drivers/mtd/maps/sharpsl-flash.c b/drivers/mtd/maps/sharpsl-flash.c
index 026eab0..b392f09 100644
--- a/drivers/mtd/maps/sharpsl-flash.c
+++ b/drivers/mtd/maps/sharpsl-flash.c
@@ -47,7 +47,7 @@
}
};
-int __init init_sharpsl(void)
+static int __init init_sharpsl(void)
{
struct mtd_partition *parts;
int nb_parts = 0;
diff --git a/drivers/mtd/maps/tqm8xxl.c b/drivers/mtd/maps/tqm8xxl.c
index a5d3d85..6014698 100644
--- a/drivers/mtd/maps/tqm8xxl.c
+++ b/drivers/mtd/maps/tqm8xxl.c
@@ -109,7 +109,7 @@
};
#endif
-int __init init_tqm_mtd(void)
+static int __init init_tqm_mtd(void)
{
int idx = 0, ret = 0;
unsigned long flash_addr, flash_size, mtd_size = 0;
diff --git a/drivers/mtd/maps/uclinux.c b/drivers/mtd/maps/uclinux.c
index 0dc645f..81756e3 100644
--- a/drivers/mtd/maps/uclinux.c
+++ b/drivers/mtd/maps/uclinux.c
@@ -51,7 +51,7 @@
/****************************************************************************/
-int __init uclinux_mtd_init(void)
+static int __init uclinux_mtd_init(void)
{
struct mtd_info *mtd;
struct map_info *mapp;
@@ -94,7 +94,7 @@
/****************************************************************************/
-void __exit uclinux_mtd_cleanup(void)
+static void __exit uclinux_mtd_cleanup(void)
{
if (uclinux_ram_mtdinfo) {
del_mtd_partitions(uclinux_ram_mtdinfo);
diff --git a/drivers/mtd/maps/vmax301.c b/drivers/mtd/maps/vmax301.c
index 5a0c9a3..6d452dc 100644
--- a/drivers/mtd/maps/vmax301.c
+++ b/drivers/mtd/maps/vmax301.c
@@ -146,7 +146,7 @@
iounmap((void *)vmax_map[0].map_priv_1 - WINDOW_START);
}
-int __init init_vmax301(void)
+static int __init init_vmax301(void)
{
int i;
unsigned long iomapadr;
diff --git a/drivers/mtd/maps/wr_sbc82xx_flash.c b/drivers/mtd/maps/wr_sbc82xx_flash.c
index 413b0cf..933a2b6 100644
--- a/drivers/mtd/maps/wr_sbc82xx_flash.c
+++ b/drivers/mtd/maps/wr_sbc82xx_flash.c
@@ -74,7 +74,7 @@
} \
} while (0);
-int __init init_sbc82xx_flash(void)
+static int __init init_sbc82xx_flash(void)
{
volatile memctl_cpm2_t *mc = &cpm2_immr->im_memctl;
int bigflash;
diff --git a/drivers/mtd/mtdchar.c b/drivers/mtd/mtdchar.c
index bcffeda..e9ec59e 100644
--- a/drivers/mtd/mtdchar.c
+++ b/drivers/mtd/mtdchar.c
@@ -450,16 +450,20 @@
if (!erase)
ret = -ENOMEM;
else {
+ struct erase_info_user einfo;
+
wait_queue_head_t waitq;
DECLARE_WAITQUEUE(wait, current);
init_waitqueue_head(&waitq);
- if (copy_from_user(&erase->addr, argp,
+ if (copy_from_user(&einfo, argp,
sizeof(struct erase_info_user))) {
kfree(erase);
return -EFAULT;
}
+ erase->addr = einfo.start;
+ erase->len = einfo.length;
erase->mtd = mtd;
erase->callback = mtdchar_erase_callback;
erase->priv = (unsigned long)&waitq;
diff --git a/drivers/mtd/mtdconcat.c b/drivers/mtd/mtdconcat.c
index 1a05cf3..3dbb1b3 100644
--- a/drivers/mtd/mtdconcat.c
+++ b/drivers/mtd/mtdconcat.c
@@ -197,7 +197,7 @@
continue;
}
- size = min(total_len, (size_t)(subdev->size - to));
+ size = min_t(uint64_t, total_len, subdev->size - to);
wsize = size; /* store for future use */
entry_high = entry_low;
@@ -385,7 +385,7 @@
struct mtd_concat *concat = CONCAT(mtd);
struct mtd_info *subdev;
int i, err;
- u_int32_t length, offset = 0;
+ uint64_t length, offset = 0;
struct erase_info *erase;
if (!(mtd->flags & MTD_WRITEABLE))
@@ -518,7 +518,7 @@
return 0;
}
-static int concat_lock(struct mtd_info *mtd, loff_t ofs, size_t len)
+static int concat_lock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
{
struct mtd_concat *concat = CONCAT(mtd);
int i, err = -EINVAL;
@@ -528,7 +528,7 @@
for (i = 0; i < concat->num_subdev; i++) {
struct mtd_info *subdev = concat->subdev[i];
- size_t size;
+ uint64_t size;
if (ofs >= subdev->size) {
size = 0;
@@ -556,7 +556,7 @@
return err;
}
-static int concat_unlock(struct mtd_info *mtd, loff_t ofs, size_t len)
+static int concat_unlock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
{
struct mtd_concat *concat = CONCAT(mtd);
int i, err = 0;
@@ -566,7 +566,7 @@
for (i = 0; i < concat->num_subdev; i++) {
struct mtd_info *subdev = concat->subdev[i];
- size_t size;
+ uint64_t size;
if (ofs >= subdev->size) {
size = 0;
@@ -696,7 +696,7 @@
int i;
size_t size;
struct mtd_concat *concat;
- u_int32_t max_erasesize, curr_erasesize;
+ uint32_t max_erasesize, curr_erasesize;
int num_erase_region;
printk(KERN_NOTICE "Concatenating MTD devices:\n");
@@ -842,12 +842,14 @@
concat->mtd.erasesize = curr_erasesize;
concat->mtd.numeraseregions = 0;
} else {
+ uint64_t tmp64;
+
/*
* erase block size varies across the subdevices: allocate
* space to store the data describing the variable erase regions
*/
struct mtd_erase_region_info *erase_region_p;
- u_int32_t begin, position;
+ uint64_t begin, position;
concat->mtd.erasesize = max_erasesize;
concat->mtd.numeraseregions = num_erase_region;
@@ -879,8 +881,9 @@
erase_region_p->offset = begin;
erase_region_p->erasesize =
curr_erasesize;
- erase_region_p->numblocks =
- (position - begin) / curr_erasesize;
+ tmp64 = position - begin;
+ do_div(tmp64, curr_erasesize);
+ erase_region_p->numblocks = tmp64;
begin = position;
curr_erasesize = subdev[i]->erasesize;
@@ -897,9 +900,9 @@
erase_region_p->offset = begin;
erase_region_p->erasesize =
curr_erasesize;
- erase_region_p->numblocks =
- (position -
- begin) / curr_erasesize;
+ tmp64 = position - begin;
+ do_div(tmp64, curr_erasesize);
+ erase_region_p->numblocks = tmp64;
begin = position;
curr_erasesize =
@@ -909,14 +912,16 @@
}
position +=
subdev[i]->eraseregions[j].
- numblocks * curr_erasesize;
+ numblocks * (uint64_t)curr_erasesize;
}
}
}
/* Now write the final entry */
erase_region_p->offset = begin;
erase_region_p->erasesize = curr_erasesize;
- erase_region_p->numblocks = (position - begin) / curr_erasesize;
+ tmp64 = position - begin;
+ do_div(tmp64, curr_erasesize);
+ erase_region_p->numblocks = tmp64;
}
return &concat->mtd;
diff --git a/drivers/mtd/mtdcore.c b/drivers/mtd/mtdcore.c
index a9d2469..76fe0a1 100644
--- a/drivers/mtd/mtdcore.c
+++ b/drivers/mtd/mtdcore.c
@@ -57,6 +57,19 @@
mtd->index = i;
mtd->usecount = 0;
+ if (is_power_of_2(mtd->erasesize))
+ mtd->erasesize_shift = ffs(mtd->erasesize) - 1;
+ else
+ mtd->erasesize_shift = 0;
+
+ if (is_power_of_2(mtd->writesize))
+ mtd->writesize_shift = ffs(mtd->writesize) - 1;
+ else
+ mtd->writesize_shift = 0;
+
+ mtd->erasesize_mask = (1 << mtd->erasesize_shift) - 1;
+ mtd->writesize_mask = (1 << mtd->writesize_shift) - 1;
+
/* Some chips always power up locked. Unlock them now */
if ((mtd->flags & MTD_WRITEABLE)
&& (mtd->flags & MTD_POWERUP_LOCK) && mtd->unlock) {
@@ -344,7 +357,8 @@
if (!this)
return 0;
- return sprintf(buf, "mtd%d: %8.8x %8.8x \"%s\"\n", i, this->size,
+ return sprintf(buf, "mtd%d: %8.8llx %8.8x \"%s\"\n", i,
+ (unsigned long long)this->size,
this->erasesize, this->name);
}
diff --git a/drivers/mtd/mtdoops.c b/drivers/mtd/mtdoops.c
index aebb3b2..1a6b3be 100644
--- a/drivers/mtd/mtdoops.c
+++ b/drivers/mtd/mtdoops.c
@@ -80,9 +80,9 @@
if (ret) {
set_current_state(TASK_RUNNING);
remove_wait_queue(&wait_q, &wait);
- printk (KERN_WARNING "mtdoops: erase of region [0x%x, 0x%x] "
+ printk (KERN_WARNING "mtdoops: erase of region [0x%llx, 0x%llx] "
"on \"%s\" failed\n",
- erase.addr, erase.len, mtd->name);
+ (unsigned long long)erase.addr, (unsigned long long)erase.len, mtd->name);
return ret;
}
@@ -289,7 +289,10 @@
}
cxt->mtd = mtd;
- cxt->oops_pages = mtd->size / OOPS_PAGE_SIZE;
+ if (mtd->size > INT_MAX)
+ cxt->oops_pages = INT_MAX / OOPS_PAGE_SIZE;
+ else
+ cxt->oops_pages = (int)mtd->size / OOPS_PAGE_SIZE;
find_next_position(cxt);
diff --git a/drivers/mtd/mtdpart.c b/drivers/mtd/mtdpart.c
index 3728913..144e6b6 100644
--- a/drivers/mtd/mtdpart.c
+++ b/drivers/mtd/mtdpart.c
@@ -26,7 +26,7 @@
struct mtd_part {
struct mtd_info mtd;
struct mtd_info *master;
- u_int32_t offset;
+ uint64_t offset;
int index;
struct list_head list;
int registered;
@@ -235,7 +235,7 @@
}
EXPORT_SYMBOL_GPL(mtd_erase_callback);
-static int part_lock(struct mtd_info *mtd, loff_t ofs, size_t len)
+static int part_lock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
{
struct mtd_part *part = PART(mtd);
if ((len + ofs) > mtd->size)
@@ -243,7 +243,7 @@
return part->master->lock(part->master, ofs + part->offset, len);
}
-static int part_unlock(struct mtd_info *mtd, loff_t ofs, size_t len)
+static int part_unlock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
{
struct mtd_part *part = PART(mtd);
if ((len + ofs) > mtd->size)
@@ -317,7 +317,7 @@
static struct mtd_part *add_one_partition(struct mtd_info *master,
const struct mtd_partition *part, int partno,
- u_int32_t cur_offset)
+ uint64_t cur_offset)
{
struct mtd_part *slave;
@@ -395,19 +395,19 @@
slave->offset = cur_offset;
if (slave->offset == MTDPART_OFS_NXTBLK) {
slave->offset = cur_offset;
- if ((cur_offset % master->erasesize) != 0) {
+ if (mtd_mod_by_eb(cur_offset, master) != 0) {
/* Round up to next erasesize */
- slave->offset = ((cur_offset / master->erasesize) + 1) * master->erasesize;
+ slave->offset = (mtd_div_by_eb(cur_offset, master) + 1) * master->erasesize;
printk(KERN_NOTICE "Moving partition %d: "
- "0x%08x -> 0x%08x\n", partno,
- cur_offset, slave->offset);
+ "0x%012llx -> 0x%012llx\n", partno,
+ (unsigned long long)cur_offset, (unsigned long long)slave->offset);
}
}
if (slave->mtd.size == MTDPART_SIZ_FULL)
slave->mtd.size = master->size - slave->offset;
- printk(KERN_NOTICE "0x%08x-0x%08x : \"%s\"\n", slave->offset,
- slave->offset + slave->mtd.size, slave->mtd.name);
+ printk(KERN_NOTICE "0x%012llx-0x%012llx : \"%s\"\n", (unsigned long long)slave->offset,
+ (unsigned long long)(slave->offset + slave->mtd.size), slave->mtd.name);
/* let's do some sanity checks */
if (slave->offset >= master->size) {
@@ -420,13 +420,13 @@
}
if (slave->offset + slave->mtd.size > master->size) {
slave->mtd.size = master->size - slave->offset;
- printk(KERN_WARNING"mtd: partition \"%s\" extends beyond the end of device \"%s\" -- size truncated to %#x\n",
- part->name, master->name, slave->mtd.size);
+ printk(KERN_WARNING"mtd: partition \"%s\" extends beyond the end of device \"%s\" -- size truncated to %#llx\n",
+ part->name, master->name, (unsigned long long)slave->mtd.size);
}
if (master->numeraseregions > 1) {
/* Deal with variable erase size stuff */
int i, max = master->numeraseregions;
- u32 end = slave->offset + slave->mtd.size;
+ u64 end = slave->offset + slave->mtd.size;
struct mtd_erase_region_info *regions = master->eraseregions;
/* Find the first erase regions which is part of this
@@ -449,7 +449,7 @@
}
if ((slave->mtd.flags & MTD_WRITEABLE) &&
- (slave->offset % slave->mtd.erasesize)) {
+ mtd_mod_by_eb(slave->offset, &slave->mtd)) {
/* Doesn't start on a boundary of major erase size */
/* FIXME: Let it be writable if it is on a boundary of
* _minor_ erase size though */
@@ -458,7 +458,7 @@
part->name);
}
if ((slave->mtd.flags & MTD_WRITEABLE) &&
- (slave->mtd.size % slave->mtd.erasesize)) {
+ mtd_mod_by_eb(slave->mtd.size, &slave->mtd)) {
slave->mtd.flags &= ~MTD_WRITEABLE;
printk(KERN_WARNING"mtd: partition \"%s\" doesn't end on an erase block -- force read-only\n",
part->name);
@@ -466,7 +466,7 @@
slave->mtd.ecclayout = master->ecclayout;
if (master->block_isbad) {
- uint32_t offs = 0;
+ uint64_t offs = 0;
while (offs < slave->mtd.size) {
if (master->block_isbad(master,
@@ -501,7 +501,7 @@
int nbparts)
{
struct mtd_part *slave;
- u_int32_t cur_offset = 0;
+ uint64_t cur_offset = 0;
int i;
printk(KERN_NOTICE "Creating %d MTD partitions on \"%s\":\n", nbparts, master->name);
diff --git a/drivers/mtd/nand/Kconfig b/drivers/mtd/nand/Kconfig
index f8ae040..8b12e6e 100644
--- a/drivers/mtd/nand/Kconfig
+++ b/drivers/mtd/nand/Kconfig
@@ -163,6 +163,13 @@
incorrect ECC generation, and if using these, the default of
software ECC is preferable.
+config MTD_NAND_NDFC
+ tristate "NDFC NanD Flash Controller"
+ depends on 4xx
+ select MTD_NAND_ECC_SMC
+ help
+ NDFC Nand Flash Controllers are integrated in IBM/AMCC's 4xx SoCs
+
config MTD_NAND_S3C2410_CLKSTOP
bool "S3C2410 NAND IDLE clock stop"
depends on MTD_NAND_S3C2410
diff --git a/drivers/mtd/nand/alauda.c b/drivers/mtd/nand/alauda.c
index 9623803..6d96491 100644
--- a/drivers/mtd/nand/alauda.c
+++ b/drivers/mtd/nand/alauda.c
@@ -676,11 +676,11 @@
goto error;
al->write_out = usb_sndbulkpipe(al->dev,
- ep_wr->bEndpointAddress & USB_ENDPOINT_NUMBER_MASK);
+ usb_endpoint_num(ep_wr));
al->bulk_in = usb_rcvbulkpipe(al->dev,
- ep_in->bEndpointAddress & USB_ENDPOINT_NUMBER_MASK);
+ usb_endpoint_num(ep_in));
al->bulk_out = usb_sndbulkpipe(al->dev,
- ep_out->bEndpointAddress & USB_ENDPOINT_NUMBER_MASK);
+ usb_endpoint_num(ep_out));
/* second device is identical up to now */
memcpy(al+1, al, sizeof(*al));
diff --git a/drivers/mtd/nand/cafe_nand.c b/drivers/mtd/nand/cafe_nand.c
index b8064bf..22a6b2e 100644
--- a/drivers/mtd/nand/cafe_nand.c
+++ b/drivers/mtd/nand/cafe_nand.c
@@ -90,7 +90,7 @@
module_param_array(timing, int, &numtimings, 0644);
#ifdef CONFIG_MTD_PARTITIONS
-static const char *part_probes[] = { "RedBoot", NULL };
+static const char *part_probes[] = { "cmdlinepart", "RedBoot", NULL };
#endif
/* Hrm. Why isn't this already conditional on something in the struct device? */
@@ -805,10 +805,13 @@
add_mtd_device(mtd);
#ifdef CONFIG_MTD_PARTITIONS
+#ifdef CONFIG_MTD_CMDLINE_PARTS
+ mtd->name = "cafe_nand";
+#endif
nr_parts = parse_mtd_partitions(mtd, part_probes, &parts, 0);
if (nr_parts > 0) {
cafe->parts = parts;
- dev_info(&cafe->pdev->dev, "%d RedBoot partitions found\n", nr_parts);
+ dev_info(&cafe->pdev->dev, "%d partitions found\n", nr_parts);
add_mtd_partitions(mtd, parts, nr_parts);
}
#endif
diff --git a/drivers/mtd/nand/fsl_elbc_nand.c b/drivers/mtd/nand/fsl_elbc_nand.c
index 4aa5bd6..65929db 100644
--- a/drivers/mtd/nand/fsl_elbc_nand.c
+++ b/drivers/mtd/nand/fsl_elbc_nand.c
@@ -777,7 +777,9 @@
/* Fill in fsl_elbc_mtd structure */
priv->mtd.priv = chip;
priv->mtd.owner = THIS_MODULE;
- priv->fmr = 0; /* rest filled in later */
+
+ /* Set the ECCM according to the settings in bootloader.*/
+ priv->fmr = in_be32(&lbc->fmr) & FMR_ECCM;
/* fill in nand_chip structure */
/* set up function call table */
diff --git a/drivers/mtd/nand/nand_base.c b/drivers/mtd/nand/nand_base.c
index 0a9c9cd..0c3afcc 100644
--- a/drivers/mtd/nand/nand_base.c
+++ b/drivers/mtd/nand/nand_base.c
@@ -2014,13 +2014,14 @@
int nand_erase_nand(struct mtd_info *mtd, struct erase_info *instr,
int allowbbt)
{
- int page, len, status, pages_per_block, ret, chipnr;
+ int page, status, pages_per_block, ret, chipnr;
struct nand_chip *chip = mtd->priv;
- int rewrite_bbt[NAND_MAX_CHIPS]={0};
+ loff_t rewrite_bbt[NAND_MAX_CHIPS]={0};
unsigned int bbt_masked_page = 0xffffffff;
+ loff_t len;
- DEBUG(MTD_DEBUG_LEVEL3, "nand_erase: start = 0x%08x, len = %i\n",
- (unsigned int)instr->addr, (unsigned int)instr->len);
+ DEBUG(MTD_DEBUG_LEVEL3, "nand_erase: start = 0x%012llx, len = %llu\n",
+ (unsigned long long)instr->addr, (unsigned long long)instr->len);
/* Start address must align on block boundary */
if (instr->addr & ((1 << chip->phys_erase_shift) - 1)) {
@@ -2116,7 +2117,8 @@
DEBUG(MTD_DEBUG_LEVEL0, "nand_erase: "
"Failed erase, page 0x%08x\n", page);
instr->state = MTD_ERASE_FAILED;
- instr->fail_addr = (page << chip->page_shift);
+ instr->fail_addr =
+ ((loff_t)page << chip->page_shift);
goto erase_exit;
}
@@ -2126,7 +2128,8 @@
*/
if (bbt_masked_page != 0xffffffff &&
(page & BBT_PAGE_MASK) == bbt_masked_page)
- rewrite_bbt[chipnr] = (page << chip->page_shift);
+ rewrite_bbt[chipnr] =
+ ((loff_t)page << chip->page_shift);
/* Increment page address and decrement length */
len -= (1 << chip->phys_erase_shift);
@@ -2173,7 +2176,7 @@
continue;
/* update the BBT for chip */
DEBUG(MTD_DEBUG_LEVEL0, "nand_erase_nand: nand_update_bbt "
- "(%d:0x%0x 0x%0x)\n", chipnr, rewrite_bbt[chipnr],
+ "(%d:0x%0llx 0x%0x)\n", chipnr, rewrite_bbt[chipnr],
chip->bbt_td->pages[chipnr]);
nand_update_bbt(mtd, rewrite_bbt[chipnr]);
}
@@ -2365,7 +2368,7 @@
if (!mtd->name)
mtd->name = type->name;
- chip->chipsize = type->chipsize << 20;
+ chip->chipsize = (uint64_t)type->chipsize << 20;
/* Newer devices have all the information in additional id bytes */
if (!type->pagesize) {
@@ -2423,7 +2426,10 @@
chip->bbt_erase_shift = chip->phys_erase_shift =
ffs(mtd->erasesize) - 1;
- chip->chip_shift = ffs(chip->chipsize) - 1;
+ if (chip->chipsize & 0xffffffff)
+ chip->chip_shift = ffs((unsigned)chip->chipsize) - 1;
+ else
+ chip->chip_shift = ffs((unsigned)(chip->chipsize >> 32)) + 32 - 1;
/* Set the bad block position */
chip->badblockpos = mtd->writesize > 512 ?
@@ -2517,7 +2523,6 @@
/**
* nand_scan_tail - [NAND Interface] Scan for the NAND device
* @mtd: MTD device structure
- * @maxchips: Number of chips to scan for
*
* This is the second phase of the normal nand_scan() function. It
* fills out all the uninitialized function pointers with the defaults
diff --git a/drivers/mtd/nand/nand_bbt.c b/drivers/mtd/nand/nand_bbt.c
index 0b1c485..55c23e5 100644
--- a/drivers/mtd/nand/nand_bbt.c
+++ b/drivers/mtd/nand/nand_bbt.c
@@ -171,16 +171,16 @@
if (tmp == msk)
continue;
if (reserved_block_code && (tmp == reserved_block_code)) {
- printk(KERN_DEBUG "nand_read_bbt: Reserved block at 0x%08x\n",
- ((offs << 2) + (act >> 1)) << this->bbt_erase_shift);
+ printk(KERN_DEBUG "nand_read_bbt: Reserved block at 0x%012llx\n",
+ (loff_t)((offs << 2) + (act >> 1)) << this->bbt_erase_shift);
this->bbt[offs + (act >> 3)] |= 0x2 << (act & 0x06);
mtd->ecc_stats.bbtblocks++;
continue;
}
/* Leave it for now, if its matured we can move this
* message to MTD_DEBUG_LEVEL0 */
- printk(KERN_DEBUG "nand_read_bbt: Bad block at 0x%08x\n",
- ((offs << 2) + (act >> 1)) << this->bbt_erase_shift);
+ printk(KERN_DEBUG "nand_read_bbt: Bad block at 0x%012llx\n",
+ (loff_t)((offs << 2) + (act >> 1)) << this->bbt_erase_shift);
/* Factory marked bad or worn out ? */
if (tmp == 0)
this->bbt[offs + (act >> 3)] |= 0x3 << (act & 0x06);
@@ -284,7 +284,7 @@
/* Read the primary version, if available */
if (td->options & NAND_BBT_VERSION) {
- scan_read_raw(mtd, buf, td->pages[0] << this->page_shift,
+ scan_read_raw(mtd, buf, (loff_t)td->pages[0] << this->page_shift,
mtd->writesize);
td->version[0] = buf[mtd->writesize + td->veroffs];
printk(KERN_DEBUG "Bad block table at page %d, version 0x%02X\n",
@@ -293,7 +293,7 @@
/* Read the mirror version, if available */
if (md && (md->options & NAND_BBT_VERSION)) {
- scan_read_raw(mtd, buf, md->pages[0] << this->page_shift,
+ scan_read_raw(mtd, buf, (loff_t)md->pages[0] << this->page_shift,
mtd->writesize);
md->version[0] = buf[mtd->writesize + md->veroffs];
printk(KERN_DEBUG "Bad block table at page %d, version 0x%02X\n",
@@ -411,7 +411,7 @@
numblocks = this->chipsize >> (this->bbt_erase_shift - 1);
startblock = chip * numblocks;
numblocks += startblock;
- from = startblock << (this->bbt_erase_shift - 1);
+ from = (loff_t)startblock << (this->bbt_erase_shift - 1);
}
for (i = startblock; i < numblocks;) {
@@ -428,8 +428,8 @@
if (ret) {
this->bbt[i >> 3] |= 0x03 << (i & 0x6);
- printk(KERN_WARNING "Bad eraseblock %d at 0x%08x\n",
- i >> 1, (unsigned int)from);
+ printk(KERN_WARNING "Bad eraseblock %d at 0x%012llx\n",
+ i >> 1, (unsigned long long)from);
mtd->ecc_stats.badblocks++;
}
@@ -495,7 +495,7 @@
for (block = 0; block < td->maxblocks; block++) {
int actblock = startblock + dir * block;
- loff_t offs = actblock << this->bbt_erase_shift;
+ loff_t offs = (loff_t)actblock << this->bbt_erase_shift;
/* Read first page */
scan_read_raw(mtd, buf, offs, mtd->writesize);
@@ -719,7 +719,7 @@
memset(&einfo, 0, sizeof(einfo));
einfo.mtd = mtd;
- einfo.addr = (unsigned long)to;
+ einfo.addr = to;
einfo.len = 1 << this->bbt_erase_shift;
res = nand_erase_nand(mtd, &einfo, 1);
if (res < 0)
@@ -729,8 +729,8 @@
if (res < 0)
goto outerr;
- printk(KERN_DEBUG "Bad block table written to 0x%08x, version "
- "0x%02X\n", (unsigned int)to, td->version[chip]);
+ printk(KERN_DEBUG "Bad block table written to 0x%012llx, version "
+ "0x%02X\n", (unsigned long long)to, td->version[chip]);
/* Mark it as used */
td->pages[chip] = page;
@@ -910,7 +910,7 @@
newval = oldval | (0x2 << (block & 0x06));
this->bbt[(block >> 3)] = newval;
if ((oldval != newval) && td->reserved_block_code)
- nand_update_bbt(mtd, block << (this->bbt_erase_shift - 1));
+ nand_update_bbt(mtd, (loff_t)block << (this->bbt_erase_shift - 1));
continue;
}
update = 0;
@@ -931,7 +931,7 @@
new ones have been marked, then we need to update the stored
bbts. This should only happen once. */
if (update && td->reserved_block_code)
- nand_update_bbt(mtd, (block - 2) << (this->bbt_erase_shift - 1));
+ nand_update_bbt(mtd, (loff_t)(block - 2) << (this->bbt_erase_shift - 1));
}
}
@@ -1027,7 +1027,6 @@
if (!this->bbt || !td)
return -EINVAL;
- len = mtd->size >> (this->bbt_erase_shift + 2);
/* Allocate a temporary buffer for one eraseblock incl. oob */
len = (1 << this->bbt_erase_shift);
len += (len >> this->page_shift) * mtd->oobsize;
diff --git a/drivers/mtd/nand/nandsim.c b/drivers/mtd/nand/nandsim.c
index ae7c577..cd0711b 100644
--- a/drivers/mtd/nand/nandsim.c
+++ b/drivers/mtd/nand/nandsim.c
@@ -38,6 +38,9 @@
#include <linux/delay.h>
#include <linux/list.h>
#include <linux/random.h>
+#include <linux/sched.h>
+#include <linux/fs.h>
+#include <linux/pagemap.h>
/* Default simulator parameters values */
#if !defined(CONFIG_NANDSIM_FIRST_ID_BYTE) || \
@@ -100,6 +103,7 @@
static char *gravepages = NULL;
static unsigned int rptwear = 0;
static unsigned int overridesize = 0;
+static char *cache_file = NULL;
module_param(first_id_byte, uint, 0400);
module_param(second_id_byte, uint, 0400);
@@ -122,12 +126,13 @@
module_param(gravepages, charp, 0400);
module_param(rptwear, uint, 0400);
module_param(overridesize, uint, 0400);
+module_param(cache_file, charp, 0400);
MODULE_PARM_DESC(first_id_byte, "The first byte returned by NAND Flash 'read ID' command (manufacturer ID)");
MODULE_PARM_DESC(second_id_byte, "The second byte returned by NAND Flash 'read ID' command (chip ID)");
MODULE_PARM_DESC(third_id_byte, "The third byte returned by NAND Flash 'read ID' command");
MODULE_PARM_DESC(fourth_id_byte, "The fourth byte returned by NAND Flash 'read ID' command");
-MODULE_PARM_DESC(access_delay, "Initial page access delay (microiseconds)");
+MODULE_PARM_DESC(access_delay, "Initial page access delay (microseconds)");
MODULE_PARM_DESC(programm_delay, "Page programm delay (microseconds");
MODULE_PARM_DESC(erase_delay, "Sector erase delay (milliseconds)");
MODULE_PARM_DESC(output_cycle, "Word output (from flash) time (nanodeconds)");
@@ -153,6 +158,7 @@
MODULE_PARM_DESC(overridesize, "Specifies the NAND Flash size overriding the ID bytes. "
"The size is specified in erase blocks and as the exponent of a power of two"
" e.g. 5 means a size of 32 erase blocks");
+MODULE_PARM_DESC(cache_file, "File to use to cache nand pages instead of memory");
/* The largest possible page size */
#define NS_LARGEST_PAGE_SIZE 2048
@@ -266,6 +272,9 @@
*/
#define NS_MAX_PREVSTATES 1
+/* Maximum page cache pages needed to read or write a NAND page to the cache_file */
+#define NS_MAX_HELD_PAGES 16
+
/*
* A union to represent flash memory contents and flash buffer.
*/
@@ -295,6 +304,9 @@
/* The simulated NAND flash pages array */
union ns_mem *pages;
+ /* Slab allocator for nand pages */
+ struct kmem_cache *nand_pages_slab;
+
/* Internal buffer of page + OOB size bytes */
union ns_mem buf;
@@ -335,6 +347,13 @@
int ale; /* address Latch Enable */
int wp; /* write Protect */
} lines;
+
+ /* Fields needed when using a cache file */
+ struct file *cfile; /* Open file */
+ unsigned char *pages_written; /* Which pages have been written */
+ void *file_buf;
+ struct page *held_pages[NS_MAX_HELD_PAGES];
+ int held_cnt;
};
/*
@@ -420,25 +439,69 @@
static u_char ns_verify_buf[NS_LARGEST_PAGE_SIZE];
/*
- * Allocate array of page pointers and initialize the array to NULL
- * pointers.
+ * Allocate array of page pointers, create slab allocation for an array
+ * and initialize the array by NULL pointers.
*
* RETURNS: 0 if success, -ENOMEM if memory alloc fails.
*/
static int alloc_device(struct nandsim *ns)
{
- int i;
+ struct file *cfile;
+ int i, err;
+
+ if (cache_file) {
+ cfile = filp_open(cache_file, O_CREAT | O_RDWR | O_LARGEFILE, 0600);
+ if (IS_ERR(cfile))
+ return PTR_ERR(cfile);
+ if (!cfile->f_op || (!cfile->f_op->read && !cfile->f_op->aio_read)) {
+ NS_ERR("alloc_device: cache file not readable\n");
+ err = -EINVAL;
+ goto err_close;
+ }
+ if (!cfile->f_op->write && !cfile->f_op->aio_write) {
+ NS_ERR("alloc_device: cache file not writeable\n");
+ err = -EINVAL;
+ goto err_close;
+ }
+ ns->pages_written = vmalloc(ns->geom.pgnum);
+ if (!ns->pages_written) {
+ NS_ERR("alloc_device: unable to allocate pages written array\n");
+ err = -ENOMEM;
+ goto err_close;
+ }
+ ns->file_buf = kmalloc(ns->geom.pgszoob, GFP_KERNEL);
+ if (!ns->file_buf) {
+ NS_ERR("alloc_device: unable to allocate file buf\n");
+ err = -ENOMEM;
+ goto err_free;
+ }
+ ns->cfile = cfile;
+ memset(ns->pages_written, 0, ns->geom.pgnum);
+ return 0;
+ }
ns->pages = vmalloc(ns->geom.pgnum * sizeof(union ns_mem));
if (!ns->pages) {
- NS_ERR("alloc_map: unable to allocate page array\n");
+ NS_ERR("alloc_device: unable to allocate page array\n");
return -ENOMEM;
}
for (i = 0; i < ns->geom.pgnum; i++) {
ns->pages[i].byte = NULL;
}
+ ns->nand_pages_slab = kmem_cache_create("nandsim",
+ ns->geom.pgszoob, 0, 0, NULL);
+ if (!ns->nand_pages_slab) {
+ NS_ERR("cache_create: unable to create kmem_cache\n");
+ return -ENOMEM;
+ }
return 0;
+
+err_free:
+ vfree(ns->pages_written);
+err_close:
+ filp_close(cfile, NULL);
+ return err;
}
/*
@@ -448,11 +511,20 @@
{
int i;
+ if (ns->cfile) {
+ kfree(ns->file_buf);
+ vfree(ns->pages_written);
+ filp_close(ns->cfile, NULL);
+ return;
+ }
+
if (ns->pages) {
for (i = 0; i < ns->geom.pgnum; i++) {
if (ns->pages[i].byte)
- kfree(ns->pages[i].byte);
+ kmem_cache_free(ns->nand_pages_slab,
+ ns->pages[i].byte);
}
+ kmem_cache_destroy(ns->nand_pages_slab);
vfree(ns->pages);
}
}
@@ -464,7 +536,7 @@
return kstrdup(buf, GFP_KERNEL);
}
-static u_int64_t divide(u_int64_t n, u_int32_t d)
+static uint64_t divide(uint64_t n, uint32_t d)
{
do_div(n, d);
return n;
@@ -480,8 +552,8 @@
struct nand_chip *chip = (struct nand_chip *)mtd->priv;
struct nandsim *ns = (struct nandsim *)(chip->priv);
int i, ret = 0;
- u_int64_t remains;
- u_int64_t next_offset;
+ uint64_t remains;
+ uint64_t next_offset;
if (NS_IS_INITIALIZED(ns)) {
NS_ERR("init_nandsim: nandsim is already initialized\n");
@@ -548,7 +620,7 @@
remains = ns->geom.totsz;
next_offset = 0;
for (i = 0; i < parts_num; ++i) {
- u_int64_t part_sz = (u_int64_t)parts[i] * ns->geom.secsz;
+ uint64_t part_sz = (uint64_t)parts[i] * ns->geom.secsz;
if (!part_sz || part_sz > remains) {
NS_ERR("bad partition size.\n");
@@ -1211,6 +1283,97 @@
return -1;
}
+static void put_pages(struct nandsim *ns)
+{
+ int i;
+
+ for (i = 0; i < ns->held_cnt; i++)
+ page_cache_release(ns->held_pages[i]);
+}
+
+/* Get page cache pages in advance to provide NOFS memory allocation */
+static int get_pages(struct nandsim *ns, struct file *file, size_t count, loff_t pos)
+{
+ pgoff_t index, start_index, end_index;
+ struct page *page;
+ struct address_space *mapping = file->f_mapping;
+
+ start_index = pos >> PAGE_CACHE_SHIFT;
+ end_index = (pos + count - 1) >> PAGE_CACHE_SHIFT;
+ if (end_index - start_index + 1 > NS_MAX_HELD_PAGES)
+ return -EINVAL;
+ ns->held_cnt = 0;
+ for (index = start_index; index <= end_index; index++) {
+ page = find_get_page(mapping, index);
+ if (page == NULL) {
+ page = find_or_create_page(mapping, index, GFP_NOFS);
+ if (page == NULL) {
+ write_inode_now(mapping->host, 1);
+ page = find_or_create_page(mapping, index, GFP_NOFS);
+ }
+ if (page == NULL) {
+ put_pages(ns);
+ return -ENOMEM;
+ }
+ unlock_page(page);
+ }
+ ns->held_pages[ns->held_cnt++] = page;
+ }
+ return 0;
+}
+
+static int set_memalloc(void)
+{
+ if (current->flags & PF_MEMALLOC)
+ return 0;
+ current->flags |= PF_MEMALLOC;
+ return 1;
+}
+
+static void clear_memalloc(int memalloc)
+{
+ if (memalloc)
+ current->flags &= ~PF_MEMALLOC;
+}
+
+static ssize_t read_file(struct nandsim *ns, struct file *file, void *buf, size_t count, loff_t *pos)
+{
+ mm_segment_t old_fs;
+ ssize_t tx;
+ int err, memalloc;
+
+ err = get_pages(ns, file, count, *pos);
+ if (err)
+ return err;
+ old_fs = get_fs();
+ set_fs(get_ds());
+ memalloc = set_memalloc();
+ tx = vfs_read(file, (char __user *)buf, count, pos);
+ clear_memalloc(memalloc);
+ set_fs(old_fs);
+ put_pages(ns);
+ return tx;
+}
+
+static ssize_t write_file(struct nandsim *ns, struct file *file, void *buf, size_t count, loff_t *pos)
+{
+ mm_segment_t old_fs;
+ ssize_t tx;
+ int err, memalloc;
+
+ err = get_pages(ns, file, count, *pos);
+ if (err)
+ return err;
+ old_fs = get_fs();
+ set_fs(get_ds());
+ memalloc = set_memalloc();
+ tx = vfs_write(file, (char __user *)buf, count, pos);
+ clear_memalloc(memalloc);
+ set_fs(old_fs);
+ put_pages(ns);
+ return tx;
+}
+
/*
* Returns a pointer to the current page.
*/
@@ -1227,6 +1390,38 @@
return NS_GET_PAGE(ns)->byte + ns->regs.column + ns->regs.off;
}
+int do_read_error(struct nandsim *ns, int num)
+{
+ unsigned int page_no = ns->regs.row;
+
+ if (read_error(page_no)) {
+ int i;
+ memset(ns->buf.byte, 0xFF, num);
+ for (i = 0; i < num; ++i)
+ ns->buf.byte[i] = random32();
+ NS_WARN("simulating read error in page %u\n", page_no);
+ return 1;
+ }
+ return 0;
+}
+
+void do_bit_flips(struct nandsim *ns, int num)
+{
+ if (bitflips && random32() < (1 << 22)) {
+ int flips = 1;
+ if (bitflips > 1)
+ flips = (random32() % (int) bitflips) + 1;
+ while (flips--) {
+ int pos = random32() % (num * 8);
+ ns->buf.byte[pos / 8] ^= (1 << (pos % 8));
+ NS_WARN("read_page: flipping bit %d in page %d "
+ "reading from %d ecc: corrected=%u failed=%u\n",
+ pos, ns->regs.row, ns->regs.column + ns->regs.off,
+ nsmtd->ecc_stats.corrected, nsmtd->ecc_stats.failed);
+ }
+ }
+}
+
/*
* Fill the NAND buffer with data read from the specified page.
*/
@@ -1234,36 +1429,40 @@
{
union ns_mem *mypage;
+ if (ns->cfile) {
+ if (!ns->pages_written[ns->regs.row]) {
+ NS_DBG("read_page: page %d not written\n", ns->regs.row);
+ memset(ns->buf.byte, 0xFF, num);
+ } else {
+ loff_t pos;
+ ssize_t tx;
+
+ NS_DBG("read_page: page %d written, reading from %d\n",
+ ns->regs.row, ns->regs.column + ns->regs.off);
+ if (do_read_error(ns, num))
+ return;
+ pos = (loff_t)ns->regs.row * ns->geom.pgszoob + ns->regs.column + ns->regs.off;
+ tx = read_file(ns, ns->cfile, ns->buf.byte, num, &pos);
+ if (tx != num) {
+ NS_ERR("read_page: read error for page %d ret %ld\n", ns->regs.row, (long)tx);
+ return;
+ }
+ do_bit_flips(ns, num);
+ }
+ return;
+ }
+
mypage = NS_GET_PAGE(ns);
if (mypage->byte == NULL) {
NS_DBG("read_page: page %d not allocated\n", ns->regs.row);
memset(ns->buf.byte, 0xFF, num);
} else {
- unsigned int page_no = ns->regs.row;
NS_DBG("read_page: page %d allocated, reading from %d\n",
ns->regs.row, ns->regs.column + ns->regs.off);
- if (read_error(page_no)) {
- int i;
- memset(ns->buf.byte, 0xFF, num);
- for (i = 0; i < num; ++i)
- ns->buf.byte[i] = random32();
- NS_WARN("simulating read error in page %u\n", page_no);
+ if (do_read_error(ns, num))
return;
- }
memcpy(ns->buf.byte, NS_PAGE_BYTE_OFF(ns), num);
- if (bitflips && random32() < (1 << 22)) {
- int flips = 1;
- if (bitflips > 1)
- flips = (random32() % (int) bitflips) + 1;
- while (flips--) {
- int pos = random32() % (num * 8);
- ns->buf.byte[pos / 8] ^= (1 << (pos % 8));
- NS_WARN("read_page: flipping bit %d in page %d "
- "reading from %d ecc: corrected=%u failed=%u\n",
- pos, ns->regs.row, ns->regs.column + ns->regs.off,
- nsmtd->ecc_stats.corrected, nsmtd->ecc_stats.failed);
- }
- }
+ do_bit_flips(ns, num);
}
}
@@ -1275,11 +1474,20 @@
union ns_mem *mypage;
int i;
+ if (ns->cfile) {
+ for (i = 0; i < ns->geom.pgsec; i++)
+ if (ns->pages_written[ns->regs.row + i]) {
+ NS_DBG("erase_sector: freeing page %d\n", ns->regs.row + i);
+ ns->pages_written[ns->regs.row + i] = 0;
+ }
+ return;
+ }
+
mypage = NS_GET_PAGE(ns);
for (i = 0; i < ns->geom.pgsec; i++) {
if (mypage->byte != NULL) {
NS_DBG("erase_sector: freeing page %d\n", ns->regs.row+i);
- kfree(mypage->byte);
+ kmem_cache_free(ns->nand_pages_slab, mypage->byte);
mypage->byte = NULL;
}
mypage++;
@@ -1295,16 +1503,57 @@
union ns_mem *mypage;
u_char *pg_off;
+ if (ns->cfile) {
+ loff_t off, pos;
+ ssize_t tx;
+ int all;
+
+ NS_DBG("prog_page: writing page %d\n", ns->regs.row);
+ pg_off = ns->file_buf + ns->regs.column + ns->regs.off;
+ off = (loff_t)ns->regs.row * ns->geom.pgszoob + ns->regs.column + ns->regs.off;
+ if (!ns->pages_written[ns->regs.row]) {
+ all = 1;
+ memset(ns->file_buf, 0xff, ns->geom.pgszoob);
+ } else {
+ all = 0;
+ pos = off;
+ tx = read_file(ns, ns->cfile, pg_off, num, &pos);
+ if (tx != num) {
+ NS_ERR("prog_page: read error for page %d ret %ld\n", ns->regs.row, (long)tx);
+ return -1;
+ }
+ }
+ for (i = 0; i < num; i++)
+ pg_off[i] &= ns->buf.byte[i];
+ if (all) {
+ pos = (loff_t)ns->regs.row * ns->geom.pgszoob;
+ tx = write_file(ns, ns->cfile, ns->file_buf, ns->geom.pgszoob, &pos);
+ if (tx != ns->geom.pgszoob) {
+ NS_ERR("prog_page: write error for page %d ret %ld\n", ns->regs.row, (long)tx);
+ return -1;
+ }
+ ns->pages_written[ns->regs.row] = 1;
+ } else {
+ pos = off;
+ tx = write_file(ns, ns->cfile, pg_off, num, &pos);
+ if (tx != num) {
+ NS_ERR("prog_page: write error for page %d ret %ld\n", ns->regs.row, (long)tx);
+ return -1;
+ }
+ }
+ return 0;
+ }
+
mypage = NS_GET_PAGE(ns);
if (mypage->byte == NULL) {
NS_DBG("prog_page: allocating page %d\n", ns->regs.row);
/*
* We allocate memory with GFP_NOFS because a flash FS may
* utilize this. If it is holding an FS lock, then gets here,
- * then kmalloc runs writeback which goes to the FS again
- * and deadlocks. This was seen in practice.
+ * then kernel memory alloc runs writeback which goes to the FS
+ * again and deadlocks. This was seen in practice.
*/
- mypage->byte = kmalloc(ns->geom.pgszoob, GFP_NOFS);
+ mypage->byte = kmem_cache_alloc(ns->nand_pages_slab, GFP_NOFS);
if (mypage->byte == NULL) {
NS_ERR("prog_page: error allocating memory for page %d\n", ns->regs.row);
return -1;
@@ -1736,13 +1985,17 @@
/* Check if chip is expecting command */
if (NS_STATE(ns->nxstate) != STATE_UNKNOWN && !(ns->nxstate & STATE_CMD_MASK)) {
- /*
- * We are in situation when something else (not command)
- * was expected but command was input. In this case ignore
- * previous command(s)/state(s) and accept the last one.
- */
- NS_WARN("write_byte: command (%#x) wasn't expected, expected state is %s, "
- "ignore previous states\n", (uint)byte, get_state_name(ns->nxstate));
+ /* Do not warn if only 2 id bytes are read */
+ if (!(ns->regs.command == NAND_CMD_READID &&
+ NS_STATE(ns->state) == STATE_DATAOUT_ID && ns->regs.count == 2)) {
+ /*
+ * We are in situation when something else (not command)
+ * was expected but command was input. In this case ignore
+ * previous command(s)/state(s) and accept the last one.
+ */
+ NS_WARN("write_byte: command (%#x) wasn't expected, expected state is %s, "
+ "ignore previous states\n", (uint)byte, get_state_name(ns->nxstate));
+ }
switch_to_ready_state(ns, NS_STATUS_FAILED(ns));
}
@@ -2044,7 +2297,7 @@
}
if (overridesize) {
- u_int64_t new_size = (u_int64_t)nsmtd->erasesize << overridesize;
+ uint64_t new_size = (uint64_t)nsmtd->erasesize << overridesize;
if (new_size >> overridesize != nsmtd->erasesize) {
NS_ERR("overridesize is too big\n");
goto err_exit;
diff --git a/drivers/mtd/nand/ndfc.c b/drivers/mtd/nand/ndfc.c
index 955959e..582cf80 100644
--- a/drivers/mtd/nand/ndfc.c
+++ b/drivers/mtd/nand/ndfc.c
@@ -2,12 +2,20 @@
* drivers/mtd/ndfc.c
*
* Overview:
- * Platform independend driver for NDFC (NanD Flash Controller)
+ * Platform independent driver for NDFC (NanD Flash Controller)
* integrated into EP440 cores
*
+ * Ported to an OF platform driver by Sean MacLennan
+ *
+ * The NDFC supports multiple chips, but this driver only supports a
+ * single chip since I do not have access to any boards with
+ * multiple chips.
+ *
* Author: Thomas Gleixner
*
* Copyright 2006 IBM
+ * Copyright 2008 PIKA Technologies
+ * Sean MacLennan <smaclennan@pikatech.com>
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
@@ -21,27 +29,20 @@
#include <linux/mtd/partitions.h>
#include <linux/mtd/ndfc.h>
#include <linux/mtd/mtd.h>
-#include <linux/platform_device.h>
-
+#include <linux/of_platform.h>
#include <asm/io.h>
-#ifdef CONFIG_40x
-#include <asm/ibm405.h>
-#else
-#include <asm/ibm44x.h>
-#endif
-struct ndfc_nand_mtd {
- struct mtd_info mtd;
- struct nand_chip chip;
- struct platform_nand_chip *pl_chip;
-};
-
-static struct ndfc_nand_mtd ndfc_mtd[NDFC_MAX_BANKS];
struct ndfc_controller {
- void __iomem *ndfcbase;
- struct nand_hw_control ndfc_control;
- atomic_t childs_active;
+ struct of_device *ofdev;
+ void __iomem *ndfcbase;
+ struct mtd_info mtd;
+ struct nand_chip chip;
+ int chip_select;
+ struct nand_hw_control ndfc_control;
+#ifdef CONFIG_MTD_PARTITIONS
+ struct mtd_partition *parts;
+#endif
};
static struct ndfc_controller ndfc_ctrl;
@@ -50,17 +51,14 @@
{
uint32_t ccr;
struct ndfc_controller *ndfc = &ndfc_ctrl;
- struct nand_chip *nandchip = mtd->priv;
- struct ndfc_nand_mtd *nandmtd = nandchip->priv;
- struct platform_nand_chip *pchip = nandmtd->pl_chip;
- ccr = __raw_readl(ndfc->ndfcbase + NDFC_CCR);
+ ccr = in_be32(ndfc->ndfcbase + NDFC_CCR);
if (chip >= 0) {
ccr &= ~NDFC_CCR_BS_MASK;
- ccr |= NDFC_CCR_BS(chip + pchip->chip_offset);
+ ccr |= NDFC_CCR_BS(chip + ndfc->chip_select);
} else
ccr |= NDFC_CCR_RESET_CE;
- __raw_writel(ccr, ndfc->ndfcbase + NDFC_CCR);
+ out_be32(ndfc->ndfcbase + NDFC_CCR, ccr);
}
static void ndfc_hwcontrol(struct mtd_info *mtd, int cmd, unsigned int ctrl)
@@ -80,7 +78,7 @@
{
struct ndfc_controller *ndfc = &ndfc_ctrl;
- return __raw_readl(ndfc->ndfcbase + NDFC_STAT) & NDFC_STAT_IS_READY;
+ return in_be32(ndfc->ndfcbase + NDFC_STAT) & NDFC_STAT_IS_READY;
}
static void ndfc_enable_hwecc(struct mtd_info *mtd, int mode)
@@ -88,9 +86,9 @@
uint32_t ccr;
struct ndfc_controller *ndfc = &ndfc_ctrl;
- ccr = __raw_readl(ndfc->ndfcbase + NDFC_CCR);
+ ccr = in_be32(ndfc->ndfcbase + NDFC_CCR);
ccr |= NDFC_CCR_RESET_ECC;
- __raw_writel(ccr, ndfc->ndfcbase + NDFC_CCR);
+ out_be32(ndfc->ndfcbase + NDFC_CCR, ccr);
wmb();
}
@@ -102,9 +100,10 @@
uint8_t *p = (uint8_t *)&ecc;
wmb();
- ecc = __raw_readl(ndfc->ndfcbase + NDFC_ECC);
- ecc_code[0] = p[1];
- ecc_code[1] = p[2];
+ ecc = in_be32(ndfc->ndfcbase + NDFC_ECC);
+ /* The NDFC uses Smart Media (SMC) bytes order */
+ ecc_code[0] = p[2];
+ ecc_code[1] = p[1];
ecc_code[2] = p[3];
return 0;
@@ -123,7 +122,7 @@
uint32_t *p = (uint32_t *) buf;
for(;len > 0; len -= 4)
- *p++ = __raw_readl(ndfc->ndfcbase + NDFC_DATA);
+ *p++ = in_be32(ndfc->ndfcbase + NDFC_DATA);
}
static void ndfc_write_buf(struct mtd_info *mtd, const uint8_t *buf, int len)
@@ -132,7 +131,7 @@
uint32_t *p = (uint32_t *) buf;
for(;len > 0; len -= 4)
- __raw_writel(*p++, ndfc->ndfcbase + NDFC_DATA);
+ out_be32(ndfc->ndfcbase + NDFC_DATA, *p++);
}
static int ndfc_verify_buf(struct mtd_info *mtd, const uint8_t *buf, int len)
@@ -141,7 +140,7 @@
uint32_t *p = (uint32_t *) buf;
for(;len > 0; len -= 4)
- if (*p++ != __raw_readl(ndfc->ndfcbase + NDFC_DATA))
+ if (*p++ != in_be32(ndfc->ndfcbase + NDFC_DATA))
return -EFAULT;
return 0;
}
@@ -149,10 +148,19 @@
/*
* Initialize chip structure
*/
-static void ndfc_chip_init(struct ndfc_nand_mtd *mtd)
+static int ndfc_chip_init(struct ndfc_controller *ndfc,
+ struct device_node *node)
{
- struct ndfc_controller *ndfc = &ndfc_ctrl;
- struct nand_chip *chip = &mtd->chip;
+#ifdef CONFIG_MTD_PARTITIONS
+#ifdef CONFIG_MTD_CMDLINE_PARTS
+ static const char *part_types[] = { "cmdlinepart", NULL };
+#else
+ static const char *part_types[] = { NULL };
+#endif
+#endif
+ struct device_node *flash_np;
+ struct nand_chip *chip = &ndfc->chip;
+ int ret;
chip->IO_ADDR_R = ndfc->ndfcbase + NDFC_DATA;
chip->IO_ADDR_W = ndfc->ndfcbase + NDFC_DATA;
@@ -160,8 +168,6 @@
chip->dev_ready = ndfc_ready;
chip->select_chip = ndfc_select_chip;
chip->chip_delay = 50;
- chip->priv = mtd;
- chip->options = mtd->pl_chip->options;
chip->controller = &ndfc->ndfc_control;
chip->read_buf = ndfc_read_buf;
chip->write_buf = ndfc_write_buf;
@@ -172,143 +178,136 @@
chip->ecc.mode = NAND_ECC_HW;
chip->ecc.size = 256;
chip->ecc.bytes = 3;
- chip->ecclayout = chip->ecc.layout = mtd->pl_chip->ecclayout;
- mtd->mtd.priv = chip;
- mtd->mtd.owner = THIS_MODULE;
-}
-static int ndfc_chip_probe(struct platform_device *pdev)
-{
- struct platform_nand_chip *nc = pdev->dev.platform_data;
- struct ndfc_chip_settings *settings = nc->priv;
- struct ndfc_controller *ndfc = &ndfc_ctrl;
- struct ndfc_nand_mtd *nandmtd;
+ ndfc->mtd.priv = chip;
+ ndfc->mtd.owner = THIS_MODULE;
- if (nc->chip_offset >= NDFC_MAX_BANKS || nc->nr_chips > NDFC_MAX_BANKS)
- return -EINVAL;
-
- /* Set the bank settings */
- __raw_writel(settings->bank_settings,
- ndfc->ndfcbase + NDFC_BCFG0 + (nc->chip_offset << 2));
-
- nandmtd = &ndfc_mtd[pdev->id];
- if (nandmtd->pl_chip)
- return -EBUSY;
-
- nandmtd->pl_chip = nc;
- ndfc_chip_init(nandmtd);
-
- /* Scan for chips */
- if (nand_scan(&nandmtd->mtd, nc->nr_chips)) {
- nandmtd->pl_chip = NULL;
+ flash_np = of_get_next_child(node, NULL);
+ if (!flash_np)
return -ENODEV;
+
+ ndfc->mtd.name = kasprintf(GFP_KERNEL, "%s.%s",
+ ndfc->ofdev->dev.bus_id, flash_np->name);
+ if (!ndfc->mtd.name) {
+ ret = -ENOMEM;
+ goto err;
}
+ ret = nand_scan(&ndfc->mtd, 1);
+ if (ret)
+ goto err;
+
#ifdef CONFIG_MTD_PARTITIONS
- printk("Number of partitions %d\n", nc->nr_partitions);
- if (nc->nr_partitions) {
- /* Add the full device, so complete dumps can be made */
- add_mtd_device(&nandmtd->mtd);
- add_mtd_partitions(&nandmtd->mtd, nc->partitions,
- nc->nr_partitions);
+ ret = parse_mtd_partitions(&ndfc->mtd, part_types, &ndfc->parts, 0);
+ if (ret < 0)
+ goto err;
- } else
-#else
- add_mtd_device(&nandmtd->mtd);
-#endif
-
- atomic_inc(&ndfc->childs_active);
- return 0;
-}
-
-static int ndfc_chip_remove(struct platform_device *pdev)
-{
- return 0;
-}
-
-static int ndfc_nand_probe(struct platform_device *pdev)
-{
- struct platform_nand_ctrl *nc = pdev->dev.platform_data;
- struct ndfc_controller_settings *settings = nc->priv;
- struct resource *res = pdev->resource;
- struct ndfc_controller *ndfc = &ndfc_ctrl;
- unsigned long long phys = settings->ndfc_erpn | res->start;
-
-#ifndef CONFIG_PHYS_64BIT
- ndfc->ndfcbase = ioremap((phys_addr_t)phys, res->end - res->start + 1);
-#else
- ndfc->ndfcbase = ioremap64(phys, res->end - res->start + 1);
-#endif
- if (!ndfc->ndfcbase) {
- printk(KERN_ERR "NDFC: ioremap failed\n");
- return -EIO;
+#ifdef CONFIG_MTD_OF_PARTS
+ if (ret == 0) {
+ ret = of_mtd_parse_partitions(&ndfc->ofdev->dev, flash_np,
+ &ndfc->parts);
+ if (ret < 0)
+ goto err;
}
+#endif
- __raw_writel(settings->ccr_settings, ndfc->ndfcbase + NDFC_CCR);
+ if (ret > 0)
+ ret = add_mtd_partitions(&ndfc->mtd, ndfc->parts, ret);
+ else
+#endif
+ ret = add_mtd_device(&ndfc->mtd);
+
+err:
+ of_node_put(flash_np);
+ if (ret)
+ kfree(ndfc->mtd.name);
+ return ret;
+}
+
+static int __devinit ndfc_probe(struct of_device *ofdev,
+ const struct of_device_id *match)
+{
+ struct ndfc_controller *ndfc = &ndfc_ctrl;
+ const u32 *reg;
+ u32 ccr;
+ int err, len;
spin_lock_init(&ndfc->ndfc_control.lock);
init_waitqueue_head(&ndfc->ndfc_control.wq);
+ ndfc->ofdev = ofdev;
+ dev_set_drvdata(&ofdev->dev, ndfc);
- platform_set_drvdata(pdev, ndfc);
-
- printk("NDFC NAND Driver initialized. Chip-Rev: 0x%08x\n",
- __raw_readl(ndfc->ndfcbase + NDFC_REVID));
-
- return 0;
-}
-
-static int ndfc_nand_remove(struct platform_device *pdev)
-{
- struct ndfc_controller *ndfc = platform_get_drvdata(pdev);
-
- if (atomic_read(&ndfc->childs_active))
- return -EBUSY;
-
- if (ndfc) {
- platform_set_drvdata(pdev, NULL);
- iounmap(ndfc_ctrl.ndfcbase);
- ndfc_ctrl.ndfcbase = NULL;
+ /* Read the reg property to get the chip select */
+ reg = of_get_property(ofdev->node, "reg", &len);
+ if (reg == NULL || len != 12) {
+ dev_err(&ofdev->dev, "unable read reg property (%d)\n", len);
+ return -ENOENT;
}
+ ndfc->chip_select = reg[0];
+
+ ndfc->ndfcbase = of_iomap(ofdev->node, 0);
+ if (!ndfc->ndfcbase) {
+ dev_err(&ofdev->dev, "failed to get memory\n");
+ return -EIO;
+ }
+
+ ccr = NDFC_CCR_BS(ndfc->chip_select);
+
+ /* It is ok if ccr does not exist - just default to 0 */
+ reg = of_get_property(ofdev->node, "ccr", NULL);
+ if (reg)
+ ccr |= *reg;
+
+ out_be32(ndfc->ndfcbase + NDFC_CCR, ccr);
+
+ /* Set the bank settings if given */
+ reg = of_get_property(ofdev->node, "bank-settings", NULL);
+ if (reg) {
+ int offset = NDFC_BCFG0 + (ndfc->chip_select << 2);
+ out_be32(ndfc->ndfcbase + offset, *reg);
+ }
+
+ err = ndfc_chip_init(ndfc, ofdev->node);
+ if (err) {
+ iounmap(ndfc->ndfcbase);
+ return err;
+ }
+
return 0;
}
-/* driver device registration */
+static int __devexit ndfc_remove(struct of_device *ofdev)
+{
+ struct ndfc_controller *ndfc = dev_get_drvdata(&ofdev->dev);
-static struct platform_driver ndfc_chip_driver = {
- .probe = ndfc_chip_probe,
- .remove = ndfc_chip_remove,
- .driver = {
- .name = "ndfc-chip",
- .owner = THIS_MODULE,
- },
+ nand_release(&ndfc->mtd);
+
+ return 0;
+}
+
+static const struct of_device_id ndfc_match[] = {
+ { .compatible = "ibm,ndfc", },
+ {}
};
+MODULE_DEVICE_TABLE(of, ndfc_match);
-static struct platform_driver ndfc_nand_driver = {
- .probe = ndfc_nand_probe,
- .remove = ndfc_nand_remove,
- .driver = {
- .name = "ndfc-nand",
- .owner = THIS_MODULE,
+static struct of_platform_driver ndfc_driver = {
+ .driver = {
+ .name = "ndfc",
},
+ .match_table = ndfc_match,
+ .probe = ndfc_probe,
+ .remove = __devexit_p(ndfc_remove),
};
static int __init ndfc_nand_init(void)
{
- int ret;
-
- spin_lock_init(&ndfc_ctrl.ndfc_control.lock);
- init_waitqueue_head(&ndfc_ctrl.ndfc_control.wq);
-
- ret = platform_driver_register(&ndfc_nand_driver);
- if (!ret)
- ret = platform_driver_register(&ndfc_chip_driver);
- return ret;
+ return of_register_platform_driver(&ndfc_driver);
}
static void __exit ndfc_nand_exit(void)
{
- platform_driver_unregister(&ndfc_chip_driver);
- platform_driver_unregister(&ndfc_nand_driver);
+ of_unregister_platform_driver(&ndfc_driver);
}
module_init(ndfc_nand_init);
@@ -316,6 +315,4 @@
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Thomas Gleixner <tglx@linutronix.de>");
-MODULE_DESCRIPTION("Platform driver for NDFC");
-MODULE_ALIAS("platform:ndfc-chip");
-MODULE_ALIAS("platform:ndfc-nand");
+MODULE_DESCRIPTION("OF Platform driver for NDFC");
diff --git a/drivers/mtd/nand/pxa3xx_nand.c b/drivers/mtd/nand/pxa3xx_nand.c
index fc41444..cc55cbc 100644
--- a/drivers/mtd/nand/pxa3xx_nand.c
+++ b/drivers/mtd/nand/pxa3xx_nand.c
@@ -298,7 +298,7 @@
#define NDTR1_tAR(c) (min((c), 15) << 0)
/* convert nano-seconds to nand flash controller clock cycles */
-#define ns2cycle(ns, clk) (int)(((ns) * (clk / 1000000) / 1000) + 1)
+#define ns2cycle(ns, clk) (int)(((ns) * (clk / 1000000) / 1000) - 1)
static void pxa3xx_nand_set_timing(struct pxa3xx_nand_info *info,
const struct pxa3xx_nand_timing *t)
@@ -368,14 +368,14 @@
/* large block, 2 cycles for column address
* row address starts from 3rd cycle
*/
- info->ndcb1 |= (page_addr << 16) | (column & 0xffff);
+ info->ndcb1 |= page_addr << 16;
if (info->row_addr_cycles == 3)
info->ndcb2 = (page_addr >> 16) & 0xff;
} else
/* small block, 1 cycles for column address
* row address starts from 2nd cycle
*/
- info->ndcb1 = (page_addr << 8) | (column & 0xff);
+ info->ndcb1 = page_addr << 8;
if (cmd == cmdset->program)
info->ndcb0 |= NDCB0_CMD_TYPE(1) | NDCB0_AUTO_RS;
diff --git a/drivers/mtd/nand/sharpsl.c b/drivers/mtd/nand/sharpsl.c
index 30a518e..54ec754 100644
--- a/drivers/mtd/nand/sharpsl.c
+++ b/drivers/mtd/nand/sharpsl.c
@@ -2,6 +2,7 @@
* drivers/mtd/nand/sharpsl.c
*
* Copyright (C) 2004 Richard Purdie
+ * Copyright (C) 2008 Dmitry Baryshkov
*
* Based on Sharp's NAND driver sharp_sl.c
*
@@ -19,22 +20,31 @@
#include <linux/mtd/nand.h>
#include <linux/mtd/nand_ecc.h>
#include <linux/mtd/partitions.h>
+#include <linux/mtd/sharpsl.h>
#include <linux/interrupt.h>
+#include <linux/platform_device.h>
+
#include <asm/io.h>
#include <mach/hardware.h>
#include <asm/mach-types.h>
-static void __iomem *sharpsl_io_base;
-static int sharpsl_phys_base = 0x0C000000;
+struct sharpsl_nand {
+ struct mtd_info mtd;
+ struct nand_chip chip;
+
+ void __iomem *io;
+};
+
+#define mtd_to_sharpsl(_mtd) container_of(_mtd, struct sharpsl_nand, mtd)
/* register offset */
-#define ECCLPLB sharpsl_io_base+0x00 /* line parity 7 - 0 bit */
-#define ECCLPUB sharpsl_io_base+0x04 /* line parity 15 - 8 bit */
-#define ECCCP sharpsl_io_base+0x08 /* column parity 5 - 0 bit */
-#define ECCCNTR sharpsl_io_base+0x0C /* ECC byte counter */
-#define ECCCLRR sharpsl_io_base+0x10 /* cleare ECC */
-#define FLASHIO sharpsl_io_base+0x14 /* Flash I/O */
-#define FLASHCTL sharpsl_io_base+0x18 /* Flash Control */
+#define ECCLPLB 0x00 /* line parity 7 - 0 bit */
+#define ECCLPUB 0x04 /* line parity 15 - 8 bit */
+#define ECCCP 0x08 /* column parity 5 - 0 bit */
+#define ECCCNTR 0x0C /* ECC byte counter */
+#define ECCCLRR 0x10 /* cleare ECC */
+#define FLASHIO 0x14 /* Flash I/O */
+#define FLASHCTL 0x18 /* Flash Control */
/* Flash control bit */
#define FLRYBY (1 << 5)
@@ -45,35 +55,6 @@
#define FLCE0 (1 << 0)
/*
- * MTD structure for SharpSL
- */
-static struct mtd_info *sharpsl_mtd = NULL;
-
-/*
- * Define partitions for flash device
- */
-#define DEFAULT_NUM_PARTITIONS 3
-
-static int nr_partitions;
-static struct mtd_partition sharpsl_nand_default_partition_info[] = {
- {
- .name = "System Area",
- .offset = 0,
- .size = 7 * 1024 * 1024,
- },
- {
- .name = "Root Filesystem",
- .offset = 7 * 1024 * 1024,
- .size = 30 * 1024 * 1024,
- },
- {
- .name = "Home Filesystem",
- .offset = MTDPART_OFS_APPEND,
- .size = MTDPART_SIZ_FULL,
- },
-};
-
-/*
* hardware specific access to control-lines
* ctrl:
* NAND_CNE: bit 0 -> ! bit 0 & 4
@@ -84,6 +65,7 @@
static void sharpsl_nand_hwcontrol(struct mtd_info *mtd, int cmd,
unsigned int ctrl)
{
+ struct sharpsl_nand *sharpsl = mtd_to_sharpsl(mtd);
struct nand_chip *chip = mtd->priv;
if (ctrl & NAND_CTRL_CHANGE) {
@@ -93,103 +75,97 @@
bits ^= 0x11;
- writeb((readb(FLASHCTL) & ~0x17) | bits, FLASHCTL);
+ writeb((readb(sharpsl->io + FLASHCTL) & ~0x17) | bits, sharpsl->io + FLASHCTL);
}
if (cmd != NAND_CMD_NONE)
writeb(cmd, chip->IO_ADDR_W);
}
-static uint8_t scan_ff_pattern[] = { 0xff, 0xff };
-
-static struct nand_bbt_descr sharpsl_bbt = {
- .options = 0,
- .offs = 4,
- .len = 2,
- .pattern = scan_ff_pattern
-};
-
-static struct nand_bbt_descr sharpsl_akita_bbt = {
- .options = 0,
- .offs = 4,
- .len = 1,
- .pattern = scan_ff_pattern
-};
-
-static struct nand_ecclayout akita_oobinfo = {
- .eccbytes = 24,
- .eccpos = {
- 0x5, 0x1, 0x2, 0x3, 0x6, 0x7, 0x15, 0x11,
- 0x12, 0x13, 0x16, 0x17, 0x25, 0x21, 0x22, 0x23,
- 0x26, 0x27, 0x35, 0x31, 0x32, 0x33, 0x36, 0x37},
- .oobfree = {{0x08, 0x09}}
-};
-
static int sharpsl_nand_dev_ready(struct mtd_info *mtd)
{
- return !((readb(FLASHCTL) & FLRYBY) == 0);
+ struct sharpsl_nand *sharpsl = mtd_to_sharpsl(mtd);
+ return !((readb(sharpsl->io + FLASHCTL) & FLRYBY) == 0);
}
static void sharpsl_nand_enable_hwecc(struct mtd_info *mtd, int mode)
{
- writeb(0, ECCCLRR);
+ struct sharpsl_nand *sharpsl = mtd_to_sharpsl(mtd);
+ writeb(0, sharpsl->io + ECCCLRR);
}
static int sharpsl_nand_calculate_ecc(struct mtd_info *mtd, const u_char * dat, u_char * ecc_code)
{
- ecc_code[0] = ~readb(ECCLPUB);
- ecc_code[1] = ~readb(ECCLPLB);
- ecc_code[2] = (~readb(ECCCP) << 2) | 0x03;
- return readb(ECCCNTR) != 0;
+ struct sharpsl_nand *sharpsl = mtd_to_sharpsl(mtd);
+ ecc_code[0] = ~readb(sharpsl->io + ECCLPUB);
+ ecc_code[1] = ~readb(sharpsl->io + ECCLPLB);
+ ecc_code[2] = (~readb(sharpsl->io + ECCCP) << 2) | 0x03;
+ return readb(sharpsl->io + ECCCNTR) != 0;
}
#ifdef CONFIG_MTD_PARTITIONS
-const char *part_probes[] = { "cmdlinepart", NULL };
+static const char *part_probes[] = { "cmdlinepart", NULL };
#endif
/*
* Main initialization routine
*/
-static int __init sharpsl_nand_init(void)
+static int __devinit sharpsl_nand_probe(struct platform_device *pdev)
{
struct nand_chip *this;
+#ifdef CONFIG_MTD_PARTITIONS
struct mtd_partition *sharpsl_partition_info;
+ int nr_partitions;
+#endif
+ struct resource *r;
int err = 0;
+ struct sharpsl_nand *sharpsl;
+ struct sharpsl_nand_platform_data *data = pdev->dev.platform_data;
+
+ if (!data) {
+ dev_err(&pdev->dev, "no platform data!\n");
+ return -EINVAL;
+ }
/* Allocate memory for MTD device structure and private data */
- sharpsl_mtd = kmalloc(sizeof(struct mtd_info) + sizeof(struct nand_chip), GFP_KERNEL);
- if (!sharpsl_mtd) {
+ sharpsl = kzalloc(sizeof(struct sharpsl_nand), GFP_KERNEL);
+ if (!sharpsl) {
printk("Unable to allocate SharpSL NAND MTD device structure.\n");
return -ENOMEM;
}
+ r = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!r) {
+ dev_err(&pdev->dev, "no io memory resource defined!\n");
+ err = -ENODEV;
+ goto err_get_res;
+ }
+
/* map physical address */
- sharpsl_io_base = ioremap(sharpsl_phys_base, 0x1000);
- if (!sharpsl_io_base) {
+ sharpsl->io = ioremap(r->start, resource_size(r));
+ if (!sharpsl->io) {
printk("ioremap to access Sharp SL NAND chip failed\n");
- kfree(sharpsl_mtd);
- return -EIO;
+ err = -EIO;
+ goto err_ioremap;
}
/* Get pointer to private data */
- this = (struct nand_chip *)(&sharpsl_mtd[1]);
-
- /* Initialize structures */
- memset(sharpsl_mtd, 0, sizeof(struct mtd_info));
- memset(this, 0, sizeof(struct nand_chip));
+ this = (struct nand_chip *)(&sharpsl->chip);
/* Link the private data with the MTD structure */
- sharpsl_mtd->priv = this;
- sharpsl_mtd->owner = THIS_MODULE;
+ sharpsl->mtd.priv = this;
+ sharpsl->mtd.owner = THIS_MODULE;
+
+ platform_set_drvdata(pdev, sharpsl);
/*
* PXA initialize
*/
- writeb(readb(FLASHCTL) | FLWP, FLASHCTL);
+ writeb(readb(sharpsl->io + FLASHCTL) | FLWP, sharpsl->io + FLASHCTL);
/* Set address of NAND IO lines */
- this->IO_ADDR_R = FLASHIO;
- this->IO_ADDR_W = FLASHIO;
+ this->IO_ADDR_R = sharpsl->io + FLASHIO;
+ this->IO_ADDR_W = sharpsl->io + FLASHIO;
/* Set address of hardware control function */
this->cmd_ctrl = sharpsl_nand_hwcontrol;
this->dev_ready = sharpsl_nand_dev_ready;
@@ -199,68 +175,89 @@
this->ecc.mode = NAND_ECC_HW;
this->ecc.size = 256;
this->ecc.bytes = 3;
- this->badblock_pattern = &sharpsl_bbt;
- if (machine_is_akita() || machine_is_borzoi()) {
- this->badblock_pattern = &sharpsl_akita_bbt;
- this->ecc.layout = &akita_oobinfo;
- }
+ this->badblock_pattern = data->badblock_pattern;
+ this->ecc.layout = data->ecc_layout;
this->ecc.hwctl = sharpsl_nand_enable_hwecc;
this->ecc.calculate = sharpsl_nand_calculate_ecc;
this->ecc.correct = nand_correct_data;
/* Scan to find existence of the device */
- err = nand_scan(sharpsl_mtd, 1);
- if (err) {
- iounmap(sharpsl_io_base);
- kfree(sharpsl_mtd);
- return err;
- }
+ err = nand_scan(&sharpsl->mtd, 1);
+ if (err)
+ goto err_scan;
/* Register the partitions */
- sharpsl_mtd->name = "sharpsl-nand";
- nr_partitions = parse_mtd_partitions(sharpsl_mtd, part_probes, &sharpsl_partition_info, 0);
-
+ sharpsl->mtd.name = "sharpsl-nand";
+#ifdef CONFIG_MTD_PARTITIONS
+ nr_partitions = parse_mtd_partitions(&sharpsl->mtd, part_probes, &sharpsl_partition_info, 0);
if (nr_partitions <= 0) {
- nr_partitions = DEFAULT_NUM_PARTITIONS;
- sharpsl_partition_info = sharpsl_nand_default_partition_info;
- if (machine_is_poodle()) {
- sharpsl_partition_info[1].size = 22 * 1024 * 1024;
- } else if (machine_is_corgi() || machine_is_shepherd()) {
- sharpsl_partition_info[1].size = 25 * 1024 * 1024;
- } else if (machine_is_husky()) {
- sharpsl_partition_info[1].size = 53 * 1024 * 1024;
- } else if (machine_is_spitz()) {
- sharpsl_partition_info[1].size = 5 * 1024 * 1024;
- } else if (machine_is_akita()) {
- sharpsl_partition_info[1].size = 58 * 1024 * 1024;
- } else if (machine_is_borzoi()) {
- sharpsl_partition_info[1].size = 32 * 1024 * 1024;
- }
+ nr_partitions = data->nr_partitions;
+ sharpsl_partition_info = data->partitions;
}
- add_mtd_partitions(sharpsl_mtd, sharpsl_partition_info, nr_partitions);
+ if (nr_partitions > 0)
+ err = add_mtd_partitions(&sharpsl->mtd, sharpsl_partition_info, nr_partitions);
+ else
+#endif
+ err = add_mtd_device(&sharpsl->mtd);
+ if (err)
+ goto err_add;
/* Return happy */
return 0;
-}
-module_init(sharpsl_nand_init);
+err_add:
+ nand_release(&sharpsl->mtd);
+
+err_scan:
+ platform_set_drvdata(pdev, NULL);
+ iounmap(sharpsl->io);
+err_ioremap:
+err_get_res:
+ kfree(sharpsl);
+ return err;
+}
/*
* Clean up routine
*/
-static void __exit sharpsl_nand_cleanup(void)
+static int __devexit sharpsl_nand_remove(struct platform_device *pdev)
{
- /* Release resources, unregister device */
- nand_release(sharpsl_mtd);
+ struct sharpsl_nand *sharpsl = platform_get_drvdata(pdev);
- iounmap(sharpsl_io_base);
+ /* Release resources, unregister device */
+ nand_release(&sharpsl->mtd);
+
+ platform_set_drvdata(pdev, NULL);
+
+ iounmap(sharpsl->io);
/* Free the MTD device structure */
- kfree(sharpsl_mtd);
+ kfree(sharpsl);
+
+ return 0;
}
-module_exit(sharpsl_nand_cleanup);
+static struct platform_driver sharpsl_nand_driver = {
+ .driver = {
+ .name = "sharpsl-nand",
+ .owner = THIS_MODULE,
+ },
+ .probe = sharpsl_nand_probe,
+ .remove = __devexit_p(sharpsl_nand_remove),
+};
+
+static int __init sharpsl_nand_init(void)
+{
+ return platform_driver_register(&sharpsl_nand_driver);
+}
+module_init(sharpsl_nand_init);
+
+static void __exit sharpsl_nand_exit(void)
+{
+ platform_driver_unregister(&sharpsl_nand_driver);
+}
+module_exit(sharpsl_nand_exit);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Richard Purdie <rpurdie@rpsys.net>");
diff --git a/drivers/mtd/nftlcore.c b/drivers/mtd/nftlcore.c
index 320b929..d1c4546 100644
--- a/drivers/mtd/nftlcore.c
+++ b/drivers/mtd/nftlcore.c
@@ -39,7 +39,7 @@
struct NFTLrecord *nftl;
unsigned long temp;
- if (mtd->type != MTD_NANDFLASH)
+ if (mtd->type != MTD_NANDFLASH || mtd->size > UINT_MAX)
return;
/* OK, this is moderately ugly. But probably safe. Alternatives? */
if (memcmp(mtd->name, "DiskOnChip", 10))
diff --git a/drivers/mtd/nftlmount.c b/drivers/mtd/nftlmount.c
index ccc4f20..8b22b18 100644
--- a/drivers/mtd/nftlmount.c
+++ b/drivers/mtd/nftlmount.c
@@ -51,7 +51,7 @@
the mtd device accordingly. We could even get rid of
nftl->EraseSize if there were any point in doing so. */
nftl->EraseSize = nftl->mbd.mtd->erasesize;
- nftl->nb_blocks = nftl->mbd.mtd->size / nftl->EraseSize;
+ nftl->nb_blocks = (u32)nftl->mbd.mtd->size / nftl->EraseSize;
nftl->MediaUnit = BLOCK_NIL;
nftl->SpareMediaUnit = BLOCK_NIL;
@@ -168,7 +168,7 @@
printk(KERN_NOTICE "WARNING: Support for NFTL with UnitSizeFactor 0x%02x is experimental\n",
mh->UnitSizeFactor);
nftl->EraseSize = nftl->mbd.mtd->erasesize << (0xff - mh->UnitSizeFactor);
- nftl->nb_blocks = nftl->mbd.mtd->size / nftl->EraseSize;
+ nftl->nb_blocks = (u32)nftl->mbd.mtd->size / nftl->EraseSize;
}
#endif
nftl->nb_boot_blocks = le16_to_cpu(mh->FirstPhysicalEUN);
diff --git a/drivers/mtd/onenand/onenand_base.c b/drivers/mtd/onenand/onenand_base.c
index 90ed319..529af27 100644
--- a/drivers/mtd/onenand/onenand_base.c
+++ b/drivers/mtd/onenand/onenand_base.c
@@ -1772,7 +1772,7 @@
int len;
int ret = 0;
- DEBUG(MTD_DEBUG_LEVEL3, "onenand_erase: start = 0x%08x, len = %i\n", (unsigned int) instr->addr, (unsigned int) instr->len);
+ DEBUG(MTD_DEBUG_LEVEL3, "onenand_erase: start = 0x%012llx, len = %llu\n", (unsigned long long) instr->addr, (unsigned long long) instr->len);
block_size = (1 << this->erase_shift);
@@ -1810,7 +1810,7 @@
/* Check if we have a bad block, we do not erase bad blocks */
if (onenand_block_isbad_nolock(mtd, addr, 0)) {
- printk (KERN_WARNING "onenand_erase: attempt to erase a bad block at addr 0x%08x\n", (unsigned int) addr);
+ printk (KERN_WARNING "onenand_erase: attempt to erase a bad block at addr 0x%012llx\n", (unsigned long long) addr);
instr->state = MTD_ERASE_FAILED;
goto erase_exit;
}
@@ -2029,7 +2029,7 @@
*
* Lock one or more blocks
*/
-static int onenand_lock(struct mtd_info *mtd, loff_t ofs, size_t len)
+static int onenand_lock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
{
int ret;
@@ -2047,7 +2047,7 @@
*
* Unlock one or more blocks
*/
-static int onenand_unlock(struct mtd_info *mtd, loff_t ofs, size_t len)
+static int onenand_unlock(struct mtd_info *mtd, loff_t ofs, uint64_t len)
{
int ret;
diff --git a/drivers/mtd/rfd_ftl.c b/drivers/mtd/rfd_ftl.c
index e538c0a..d2aa9c46 100644
--- a/drivers/mtd/rfd_ftl.c
+++ b/drivers/mtd/rfd_ftl.c
@@ -21,8 +21,6 @@
#include <asm/types.h>
-#define const_cpu_to_le16 __constant_cpu_to_le16
-
static int block_size = 0;
module_param(block_size, int, 0);
MODULE_PARM_DESC(block_size, "Block size to use by RFD, defaults to erase unit size");
@@ -156,7 +154,7 @@
size_t retlen;
sectors_per_block = part->block_size / SECTOR_SIZE;
- part->total_blocks = part->mbd.mtd->size / part->block_size;
+ part->total_blocks = (u32)part->mbd.mtd->size / part->block_size;
if (part->total_blocks < 2)
return -ENOENT;
@@ -276,16 +274,17 @@
part = (struct partition*)erase->priv;
- i = erase->addr / part->block_size;
- if (i >= part->total_blocks || part->blocks[i].offset != erase->addr) {
- printk(KERN_ERR PREFIX "erase callback for unknown offset %x "
- "on '%s'\n", erase->addr, part->mbd.mtd->name);
+ i = (u32)erase->addr / part->block_size;
+ if (i >= part->total_blocks || part->blocks[i].offset != erase->addr ||
+ erase->addr > UINT_MAX) {
+ printk(KERN_ERR PREFIX "erase callback for unknown offset %llx "
+ "on '%s'\n", (unsigned long long)erase->addr, part->mbd.mtd->name);
return;
}
if (erase->state != MTD_ERASE_DONE) {
- printk(KERN_WARNING PREFIX "erase failed at 0x%x on '%s', "
- "state %d\n", erase->addr,
+ printk(KERN_WARNING PREFIX "erase failed at 0x%llx on '%s', "
+ "state %d\n", (unsigned long long)erase->addr,
part->mbd.mtd->name, erase->state);
part->blocks[i].state = BLOCK_FAILED;
@@ -297,7 +296,7 @@
return;
}
- magic = const_cpu_to_le16(RFD_MAGIC);
+ magic = cpu_to_le16(RFD_MAGIC);
part->blocks[i].state = BLOCK_ERASED;
part->blocks[i].free_sectors = part->data_sectors_per_block;
@@ -345,9 +344,9 @@
rc = part->mbd.mtd->erase(part->mbd.mtd, erase);
if (rc) {
- printk(KERN_ERR PREFIX "erase of region %x,%x on '%s' "
- "failed\n", erase->addr, erase->len,
- part->mbd.mtd->name);
+ printk(KERN_ERR PREFIX "erase of region %llx,%llx on '%s' "
+ "failed\n", (unsigned long long)erase->addr,
+ (unsigned long long)erase->len, part->mbd.mtd->name);
kfree(erase);
}
@@ -587,7 +586,7 @@
int block, offset, rc;
u_long addr;
size_t retlen;
- u16 del = const_cpu_to_le16(SECTOR_DELETED);
+ u16 del = cpu_to_le16(SECTOR_DELETED);
block = old_addr / part->block_size;
offset = (old_addr % part->block_size) / SECTOR_SIZE -
@@ -763,7 +762,7 @@
{
struct partition *part;
- if (mtd->type != MTD_NORFLASH)
+ if (mtd->type != MTD_NORFLASH || mtd->size > UINT_MAX)
return;
part = kzalloc(sizeof(struct partition), GFP_KERNEL);
diff --git a/drivers/mtd/ssfdc.c b/drivers/mtd/ssfdc.c
index 33a5d6e..3f67e00 100644
--- a/drivers/mtd/ssfdc.c
+++ b/drivers/mtd/ssfdc.c
@@ -294,7 +294,8 @@
int cis_sector;
/* Check for small page NAND flash */
- if (mtd->type != MTD_NANDFLASH || mtd->oobsize != OOB_SIZE)
+ if (mtd->type != MTD_NANDFLASH || mtd->oobsize != OOB_SIZE ||
+ mtd->size > UINT_MAX)
return;
/* Check for SSDFC format by reading CIS/IDI sector */
@@ -316,7 +317,7 @@
ssfdc->cis_block = cis_sector / (mtd->erasesize >> SECTOR_SHIFT);
ssfdc->erase_size = mtd->erasesize;
- ssfdc->map_len = mtd->size / mtd->erasesize;
+ ssfdc->map_len = (u32)mtd->size / mtd->erasesize;
DEBUG(MTD_DEBUG_LEVEL1,
"SSFDC_RO: cis_block=%d,erase_size=%d,map_len=%d,n_zones=%d\n",
@@ -327,7 +328,7 @@
ssfdc->heads = 16;
ssfdc->sectors = 32;
get_chs(mtd->size, NULL, &ssfdc->heads, &ssfdc->sectors);
- ssfdc->cylinders = (unsigned short)((mtd->size >> SECTOR_SHIFT) /
+ ssfdc->cylinders = (unsigned short)(((u32)mtd->size >> SECTOR_SHIFT) /
((long)ssfdc->sectors * (long)ssfdc->heads));
DEBUG(MTD_DEBUG_LEVEL1, "SSFDC_RO: using C:%d H:%d S:%d == %ld sects\n",
diff --git a/drivers/mtd/tests/Makefile b/drivers/mtd/tests/Makefile
new file mode 100644
index 0000000..c1d5013
--- /dev/null
+++ b/drivers/mtd/tests/Makefile
@@ -0,0 +1,7 @@
+obj-$(CONFIG_MTD_TESTS) += mtd_oobtest.o
+obj-$(CONFIG_MTD_TESTS) += mtd_pagetest.o
+obj-$(CONFIG_MTD_TESTS) += mtd_readtest.o
+obj-$(CONFIG_MTD_TESTS) += mtd_speedtest.o
+obj-$(CONFIG_MTD_TESTS) += mtd_stresstest.o
+obj-$(CONFIG_MTD_TESTS) += mtd_subpagetest.o
+obj-$(CONFIG_MTD_TESTS) += mtd_torturetest.o
diff --git a/drivers/mtd/tests/mtd_oobtest.c b/drivers/mtd/tests/mtd_oobtest.c
new file mode 100644
index 0000000..afbc3f8
--- /dev/null
+++ b/drivers/mtd/tests/mtd_oobtest.c
@@ -0,0 +1,742 @@
+/*
+ * Copyright (C) 2006-2008 Nokia Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; see the file COPYING. If not, write to the Free Software
+ * Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Test OOB read and write on MTD device.
+ *
+ * Author: Adrian Hunter <ext-adrian.hunter@nokia.com>
+ */
+
+#include <asm/div64.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/err.h>
+#include <linux/mtd/mtd.h>
+#include <linux/sched.h>
+
+#define PRINT_PREF KERN_INFO "mtd_oobtest: "
+
+static int dev;
+module_param(dev, int, S_IRUGO);
+MODULE_PARM_DESC(dev, "MTD device number to use");
+
+static struct mtd_info *mtd;
+static unsigned char *readbuf;
+static unsigned char *writebuf;
+static unsigned char *bbt;
+
+static int ebcnt;
+static int pgcnt;
+static int errcnt;
+static int use_offset;
+static int use_len;
+static int use_len_max;
+static int vary_offset;
+static unsigned long next = 1;
+
+static inline unsigned int simple_rand(void)
+{
+ next = next * 1103515245 + 12345;
+ return (unsigned int)((next / 65536) % 32768);
+}
+
+static inline void simple_srand(unsigned long seed)
+{
+ next = seed;
+}
+
+static void set_random_data(unsigned char *buf, size_t len)
+{
+ size_t i;
+
+ for (i = 0; i < len; ++i)
+ buf[i] = simple_rand();
+}
+
+static int erase_eraseblock(int ebnum)
+{
+ int err;
+ struct erase_info ei;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ memset(&ei, 0, sizeof(struct erase_info));
+ ei.mtd = mtd;
+ ei.addr = addr;
+ ei.len = mtd->erasesize;
+
+ err = mtd->erase(mtd, &ei);
+ if (err) {
+ printk(PRINT_PREF "error %d while erasing EB %d\n", err, ebnum);
+ return err;
+ }
+
+ if (ei.state == MTD_ERASE_FAILED) {
+ printk(PRINT_PREF "some erase error occurred at EB %d\n",
+ ebnum);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+static int erase_whole_device(void)
+{
+ int err;
+ unsigned int i;
+
+ printk(PRINT_PREF "erasing whole device\n");
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = erase_eraseblock(i);
+ if (err)
+ return err;
+ cond_resched();
+ }
+ printk(PRINT_PREF "erased %u eraseblocks\n", i);
+ return 0;
+}
+
+static void do_vary_offset(void)
+{
+ use_len -= 1;
+ if (use_len < 1) {
+ use_offset += 1;
+ if (use_offset >= use_len_max)
+ use_offset = 0;
+ use_len = use_len_max - use_offset;
+ }
+}
+
+static int write_eraseblock(int ebnum)
+{
+ int i;
+ struct mtd_oob_ops ops;
+ int err = 0;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ for (i = 0; i < pgcnt; ++i, addr += mtd->writesize) {
+ set_random_data(writebuf, use_len);
+ ops.mode = MTD_OOB_AUTO;
+ ops.len = 0;
+ ops.retlen = 0;
+ ops.ooblen = use_len;
+ ops.oobretlen = 0;
+ ops.ooboffs = use_offset;
+ ops.datbuf = 0;
+ ops.oobbuf = writebuf;
+ err = mtd->write_oob(mtd, addr, &ops);
+ if (err || ops.oobretlen != use_len) {
+ printk(PRINT_PREF "error: writeoob failed at %#llx\n",
+ (long long)addr);
+ printk(PRINT_PREF "error: use_len %d, use_offset %d\n",
+ use_len, use_offset);
+ errcnt += 1;
+ return err ? err : -1;
+ }
+ if (vary_offset)
+ do_vary_offset();
+ }
+
+ return err;
+}
+
+static int write_whole_device(void)
+{
+ int err;
+ unsigned int i;
+
+ printk(PRINT_PREF "writing OOBs of whole device\n");
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = write_eraseblock(i);
+ if (err)
+ return err;
+ if (i % 256 == 0)
+ printk(PRINT_PREF "written up to eraseblock %u\n", i);
+ cond_resched();
+ }
+ printk(PRINT_PREF "written %u eraseblocks\n", i);
+ return 0;
+}
+
+static int verify_eraseblock(int ebnum)
+{
+ int i;
+ struct mtd_oob_ops ops;
+ int err = 0;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ for (i = 0; i < pgcnt; ++i, addr += mtd->writesize) {
+ set_random_data(writebuf, use_len);
+ ops.mode = MTD_OOB_AUTO;
+ ops.len = 0;
+ ops.retlen = 0;
+ ops.ooblen = use_len;
+ ops.oobretlen = 0;
+ ops.ooboffs = use_offset;
+ ops.datbuf = 0;
+ ops.oobbuf = readbuf;
+ err = mtd->read_oob(mtd, addr, &ops);
+ if (err || ops.oobretlen != use_len) {
+ printk(PRINT_PREF "error: readoob failed at %#llx\n",
+ (long long)addr);
+ errcnt += 1;
+ return err ? err : -1;
+ }
+ if (memcmp(readbuf, writebuf, use_len)) {
+ printk(PRINT_PREF "error: verify failed at %#llx\n",
+ (long long)addr);
+ errcnt += 1;
+ if (errcnt > 1000) {
+ printk(PRINT_PREF "error: too many errors\n");
+ return -1;
+ }
+ }
+ if (use_offset != 0 || use_len < mtd->ecclayout->oobavail) {
+ int k;
+
+ ops.mode = MTD_OOB_AUTO;
+ ops.len = 0;
+ ops.retlen = 0;
+ ops.ooblen = mtd->ecclayout->oobavail;
+ ops.oobretlen = 0;
+ ops.ooboffs = 0;
+ ops.datbuf = 0;
+ ops.oobbuf = readbuf;
+ err = mtd->read_oob(mtd, addr, &ops);
+ if (err || ops.oobretlen != mtd->ecclayout->oobavail) {
+ printk(PRINT_PREF "error: readoob failed at "
+ "%#llx\n", (long long)addr);
+ errcnt += 1;
+ return err ? err : -1;
+ }
+ if (memcmp(readbuf + use_offset, writebuf, use_len)) {
+ printk(PRINT_PREF "error: verify failed at "
+ "%#llx\n", (long long)addr);
+ errcnt += 1;
+ if (errcnt > 1000) {
+ printk(PRINT_PREF "error: too many "
+ "errors\n");
+ return -1;
+ }
+ }
+ for (k = 0; k < use_offset; ++k)
+ if (readbuf[k] != 0xff) {
+ printk(PRINT_PREF "error: verify 0xff "
+ "failed at %#llx\n",
+ (long long)addr);
+ errcnt += 1;
+ if (errcnt > 1000) {
+ printk(PRINT_PREF "error: too "
+ "many errors\n");
+ return -1;
+ }
+ }
+ for (k = use_offset + use_len;
+ k < mtd->ecclayout->oobavail; ++k)
+ if (readbuf[k] != 0xff) {
+ printk(PRINT_PREF "error: verify 0xff "
+ "failed at %#llx\n",
+ (long long)addr);
+ errcnt += 1;
+ if (errcnt > 1000) {
+ printk(PRINT_PREF "error: too "
+ "many errors\n");
+ return -1;
+ }
+ }
+ }
+ if (vary_offset)
+ do_vary_offset();
+ }
+ return err;
+}
+
+static int verify_eraseblock_in_one_go(int ebnum)
+{
+ struct mtd_oob_ops ops;
+ int err = 0;
+ loff_t addr = ebnum * mtd->erasesize;
+ size_t len = mtd->ecclayout->oobavail * pgcnt;
+
+ set_random_data(writebuf, len);
+ ops.mode = MTD_OOB_AUTO;
+ ops.len = 0;
+ ops.retlen = 0;
+ ops.ooblen = len;
+ ops.oobretlen = 0;
+ ops.ooboffs = 0;
+ ops.datbuf = 0;
+ ops.oobbuf = readbuf;
+ err = mtd->read_oob(mtd, addr, &ops);
+ if (err || ops.oobretlen != len) {
+ printk(PRINT_PREF "error: readoob failed at %#llx\n",
+ (long long)addr);
+ errcnt += 1;
+ return err ? err : -1;
+ }
+ if (memcmp(readbuf, writebuf, len)) {
+ printk(PRINT_PREF "error: verify failed at %#llx\n",
+ (long long)addr);
+ errcnt += 1;
+ if (errcnt > 1000) {
+ printk(PRINT_PREF "error: too many errors\n");
+ return -1;
+ }
+ }
+
+ return err;
+}
+
+static int verify_all_eraseblocks(void)
+{
+ int err;
+ unsigned int i;
+
+ printk(PRINT_PREF "verifying all eraseblocks\n");
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = verify_eraseblock(i);
+ if (err)
+ return err;
+ if (i % 256 == 0)
+ printk(PRINT_PREF "verified up to eraseblock %u\n", i);
+ cond_resched();
+ }
+ printk(PRINT_PREF "verified %u eraseblocks\n", i);
+ return 0;
+}
+
+static int is_block_bad(int ebnum)
+{
+ int ret;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ ret = mtd->block_isbad(mtd, addr);
+ if (ret)
+ printk(PRINT_PREF "block %d is bad\n", ebnum);
+ return ret;
+}
+
+static int scan_for_bad_eraseblocks(void)
+{
+ int i, bad = 0;
+
+ bbt = kmalloc(ebcnt, GFP_KERNEL);
+ if (!bbt) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ return -ENOMEM;
+ }
+ memset(bbt, 0 , ebcnt);
+
+ printk(PRINT_PREF "scanning for bad eraseblocks\n");
+ for (i = 0; i < ebcnt; ++i) {
+ bbt[i] = is_block_bad(i) ? 1 : 0;
+ if (bbt[i])
+ bad += 1;
+ cond_resched();
+ }
+ printk(PRINT_PREF "scanned %d eraseblocks, %d are bad\n", i, bad);
+ return 0;
+}
+
+static int __init mtd_oobtest_init(void)
+{
+ int err = 0;
+ unsigned int i;
+ uint64_t tmp;
+ struct mtd_oob_ops ops;
+ loff_t addr = 0, addr0;
+
+ printk(KERN_INFO "\n");
+ printk(KERN_INFO "=================================================\n");
+ printk(PRINT_PREF "MTD device: %d\n", dev);
+
+ mtd = get_mtd_device(NULL, dev);
+ if (IS_ERR(mtd)) {
+ err = PTR_ERR(mtd);
+ printk(PRINT_PREF "error: cannot get MTD device\n");
+ return err;
+ }
+
+ if (mtd->type != MTD_NANDFLASH) {
+ printk(PRINT_PREF "this test requires NAND flash\n");
+ goto out;
+ }
+
+ tmp = mtd->size;
+ do_div(tmp, mtd->erasesize);
+ ebcnt = tmp;
+ pgcnt = mtd->erasesize / mtd->writesize;
+
+ printk(PRINT_PREF "MTD device size %llu, eraseblock size %u, "
+ "page size %u, count of eraseblocks %u, pages per "
+ "eraseblock %u, OOB size %u\n",
+ (unsigned long long)mtd->size, mtd->erasesize,
+ mtd->writesize, ebcnt, pgcnt, mtd->oobsize);
+
+ err = -ENOMEM;
+ mtd->erasesize = mtd->erasesize;
+ readbuf = kmalloc(mtd->erasesize, GFP_KERNEL);
+ if (!readbuf) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ goto out;
+ }
+ writebuf = kmalloc(mtd->erasesize, GFP_KERNEL);
+ if (!writebuf) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ goto out;
+ }
+
+ err = scan_for_bad_eraseblocks();
+ if (err)
+ goto out;
+
+ use_offset = 0;
+ use_len = mtd->ecclayout->oobavail;
+ use_len_max = mtd->ecclayout->oobavail;
+ vary_offset = 0;
+
+ /* First test: write all OOB, read it back and verify */
+ printk(PRINT_PREF "test 1 of 5\n");
+
+ err = erase_whole_device();
+ if (err)
+ goto out;
+
+ simple_srand(1);
+ err = write_whole_device();
+ if (err)
+ goto out;
+
+ simple_srand(1);
+ err = verify_all_eraseblocks();
+ if (err)
+ goto out;
+
+ /*
+ * Second test: write all OOB, a block at a time, read it back and
+ * verify.
+ */
+ printk(PRINT_PREF "test 2 of 5\n");
+
+ err = erase_whole_device();
+ if (err)
+ goto out;
+
+ simple_srand(3);
+ err = write_whole_device();
+ if (err)
+ goto out;
+
+ /* Check all eraseblocks */
+ simple_srand(3);
+ printk(PRINT_PREF "verifying all eraseblocks\n");
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = verify_eraseblock_in_one_go(i);
+ if (err)
+ goto out;
+ if (i % 256 == 0)
+ printk(PRINT_PREF "verified up to eraseblock %u\n", i);
+ cond_resched();
+ }
+ printk(PRINT_PREF "verified %u eraseblocks\n", i);
+
+ /*
+ * Third test: write OOB at varying offsets and lengths, read it back
+ * and verify.
+ */
+ printk(PRINT_PREF "test 3 of 5\n");
+
+ err = erase_whole_device();
+ if (err)
+ goto out;
+
+ /* Write all eraseblocks */
+ use_offset = 0;
+ use_len = mtd->ecclayout->oobavail;
+ use_len_max = mtd->ecclayout->oobavail;
+ vary_offset = 1;
+ simple_srand(5);
+ printk(PRINT_PREF "writing OOBs of whole device\n");
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = write_eraseblock(i);
+ if (err)
+ goto out;
+ if (i % 256 == 0)
+ printk(PRINT_PREF "written up to eraseblock %u\n", i);
+ cond_resched();
+ }
+ printk(PRINT_PREF "written %u eraseblocks\n", i);
+
+ /* Check all eraseblocks */
+ use_offset = 0;
+ use_len = mtd->ecclayout->oobavail;
+ use_len_max = mtd->ecclayout->oobavail;
+ vary_offset = 1;
+ simple_srand(5);
+ err = verify_all_eraseblocks();
+ if (err)
+ goto out;
+
+ use_offset = 0;
+ use_len = mtd->ecclayout->oobavail;
+ use_len_max = mtd->ecclayout->oobavail;
+ vary_offset = 0;
+
+ /* Fourth test: try to write off end of device */
+ printk(PRINT_PREF "test 4 of 5\n");
+
+ err = erase_whole_device();
+ if (err)
+ goto out;
+
+ addr0 = 0;
+ for (i = 0; bbt[i] && i < ebcnt; ++i)
+ addr0 += mtd->erasesize;
+
+ /* Attempt to write off end of OOB */
+ ops.mode = MTD_OOB_AUTO;
+ ops.len = 0;
+ ops.retlen = 0;
+ ops.ooblen = 1;
+ ops.oobretlen = 0;
+ ops.ooboffs = mtd->ecclayout->oobavail;
+ ops.datbuf = 0;
+ ops.oobbuf = writebuf;
+ printk(PRINT_PREF "attempting to start write past end of OOB\n");
+ printk(PRINT_PREF "an error is expected...\n");
+ err = mtd->write_oob(mtd, addr0, &ops);
+ if (err) {
+ printk(PRINT_PREF "error occurred as expected\n");
+ err = 0;
+ } else {
+ printk(PRINT_PREF "error: can write past end of OOB\n");
+ errcnt += 1;
+ }
+
+ /* Attempt to read off end of OOB */
+ ops.mode = MTD_OOB_AUTO;
+ ops.len = 0;
+ ops.retlen = 0;
+ ops.ooblen = 1;
+ ops.oobretlen = 0;
+ ops.ooboffs = mtd->ecclayout->oobavail;
+ ops.datbuf = 0;
+ ops.oobbuf = readbuf;
+ printk(PRINT_PREF "attempting to start read past end of OOB\n");
+ printk(PRINT_PREF "an error is expected...\n");
+ err = mtd->read_oob(mtd, addr0, &ops);
+ if (err) {
+ printk(PRINT_PREF "error occurred as expected\n");
+ err = 0;
+ } else {
+ printk(PRINT_PREF "error: can read past end of OOB\n");
+ errcnt += 1;
+ }
+
+ if (bbt[ebcnt - 1])
+ printk(PRINT_PREF "skipping end of device tests because last "
+ "block is bad\n");
+ else {
+ /* Attempt to write off end of device */
+ ops.mode = MTD_OOB_AUTO;
+ ops.len = 0;
+ ops.retlen = 0;
+ ops.ooblen = mtd->ecclayout->oobavail + 1;
+ ops.oobretlen = 0;
+ ops.ooboffs = 0;
+ ops.datbuf = 0;
+ ops.oobbuf = writebuf;
+ printk(PRINT_PREF "attempting to write past end of device\n");
+ printk(PRINT_PREF "an error is expected...\n");
+ err = mtd->write_oob(mtd, mtd->size - mtd->writesize, &ops);
+ if (err) {
+ printk(PRINT_PREF "error occurred as expected\n");
+ err = 0;
+ } else {
+ printk(PRINT_PREF "error: wrote past end of device\n");
+ errcnt += 1;
+ }
+
+ /* Attempt to read off end of device */
+ ops.mode = MTD_OOB_AUTO;
+ ops.len = 0;
+ ops.retlen = 0;
+ ops.ooblen = mtd->ecclayout->oobavail + 1;
+ ops.oobretlen = 0;
+ ops.ooboffs = 0;
+ ops.datbuf = 0;
+ ops.oobbuf = readbuf;
+ printk(PRINT_PREF "attempting to read past end of device\n");
+ printk(PRINT_PREF "an error is expected...\n");
+ err = mtd->read_oob(mtd, mtd->size - mtd->writesize, &ops);
+ if (err) {
+ printk(PRINT_PREF "error occurred as expected\n");
+ err = 0;
+ } else {
+ printk(PRINT_PREF "error: read past end of device\n");
+ errcnt += 1;
+ }
+
+ err = erase_eraseblock(ebcnt - 1);
+ if (err)
+ goto out;
+
+ /* Attempt to write off end of device */
+ ops.mode = MTD_OOB_AUTO;
+ ops.len = 0;
+ ops.retlen = 0;
+ ops.ooblen = mtd->ecclayout->oobavail;
+ ops.oobretlen = 0;
+ ops.ooboffs = 1;
+ ops.datbuf = 0;
+ ops.oobbuf = writebuf;
+ printk(PRINT_PREF "attempting to write past end of device\n");
+ printk(PRINT_PREF "an error is expected...\n");
+ err = mtd->write_oob(mtd, mtd->size - mtd->writesize, &ops);
+ if (err) {
+ printk(PRINT_PREF "error occurred as expected\n");
+ err = 0;
+ } else {
+ printk(PRINT_PREF "error: wrote past end of device\n");
+ errcnt += 1;
+ }
+
+ /* Attempt to read off end of device */
+ ops.mode = MTD_OOB_AUTO;
+ ops.len = 0;
+ ops.retlen = 0;
+ ops.ooblen = mtd->ecclayout->oobavail;
+ ops.oobretlen = 0;
+ ops.ooboffs = 1;
+ ops.datbuf = 0;
+ ops.oobbuf = readbuf;
+ printk(PRINT_PREF "attempting to read past end of device\n");
+ printk(PRINT_PREF "an error is expected...\n");
+ err = mtd->read_oob(mtd, mtd->size - mtd->writesize, &ops);
+ if (err) {
+ printk(PRINT_PREF "error occurred as expected\n");
+ err = 0;
+ } else {
+ printk(PRINT_PREF "error: read past end of device\n");
+ errcnt += 1;
+ }
+ }
+
+ /* Fifth test: write / read across block boundaries */
+ printk(PRINT_PREF "test 5 of 5\n");
+
+ /* Erase all eraseblocks */
+ err = erase_whole_device();
+ if (err)
+ goto out;
+
+ /* Write all eraseblocks */
+ simple_srand(11);
+ printk(PRINT_PREF "writing OOBs of whole device\n");
+ for (i = 0; i < ebcnt - 1; ++i) {
+ int cnt = 2;
+ int pg;
+ size_t sz = mtd->ecclayout->oobavail;
+ if (bbt[i] || bbt[i + 1])
+ continue;
+ addr = (i + 1) * mtd->erasesize - mtd->writesize;
+ for (pg = 0; pg < cnt; ++pg) {
+ set_random_data(writebuf, sz);
+ ops.mode = MTD_OOB_AUTO;
+ ops.len = 0;
+ ops.retlen = 0;
+ ops.ooblen = sz;
+ ops.oobretlen = 0;
+ ops.ooboffs = 0;
+ ops.datbuf = 0;
+ ops.oobbuf = writebuf;
+ err = mtd->write_oob(mtd, addr, &ops);
+ if (err)
+ goto out;
+ if (i % 256 == 0)
+ printk(PRINT_PREF "written up to eraseblock "
+ "%u\n", i);
+ cond_resched();
+ addr += mtd->writesize;
+ }
+ }
+ printk(PRINT_PREF "written %u eraseblocks\n", i);
+
+ /* Check all eraseblocks */
+ simple_srand(11);
+ printk(PRINT_PREF "verifying all eraseblocks\n");
+ for (i = 0; i < ebcnt - 1; ++i) {
+ if (bbt[i] || bbt[i + 1])
+ continue;
+ set_random_data(writebuf, mtd->ecclayout->oobavail * 2);
+ addr = (i + 1) * mtd->erasesize - mtd->writesize;
+ ops.mode = MTD_OOB_AUTO;
+ ops.len = 0;
+ ops.retlen = 0;
+ ops.ooblen = mtd->ecclayout->oobavail * 2;
+ ops.oobretlen = 0;
+ ops.ooboffs = 0;
+ ops.datbuf = 0;
+ ops.oobbuf = readbuf;
+ err = mtd->read_oob(mtd, addr, &ops);
+ if (err)
+ goto out;
+ if (memcmp(readbuf, writebuf, mtd->ecclayout->oobavail * 2)) {
+ printk(PRINT_PREF "error: verify failed at %#llx\n",
+ (long long)addr);
+ errcnt += 1;
+ if (errcnt > 1000) {
+ printk(PRINT_PREF "error: too many errors\n");
+ goto out;
+ }
+ }
+ if (i % 256 == 0)
+ printk(PRINT_PREF "verified up to eraseblock %u\n", i);
+ cond_resched();
+ }
+ printk(PRINT_PREF "verified %u eraseblocks\n", i);
+
+ printk(PRINT_PREF "finished with %d errors\n", errcnt);
+out:
+ kfree(bbt);
+ kfree(writebuf);
+ kfree(readbuf);
+ put_mtd_device(mtd);
+ if (err)
+ printk(PRINT_PREF "error %d occurred\n", err);
+ printk(KERN_INFO "=================================================\n");
+ return err;
+}
+module_init(mtd_oobtest_init);
+
+static void __exit mtd_oobtest_exit(void)
+{
+ return;
+}
+module_exit(mtd_oobtest_exit);
+
+MODULE_DESCRIPTION("Out-of-band test module");
+MODULE_AUTHOR("Adrian Hunter");
+MODULE_LICENSE("GPL");
diff --git a/drivers/mtd/tests/mtd_pagetest.c b/drivers/mtd/tests/mtd_pagetest.c
new file mode 100644
index 0000000..9648818
--- /dev/null
+++ b/drivers/mtd/tests/mtd_pagetest.c
@@ -0,0 +1,632 @@
+/*
+ * Copyright (C) 2006-2008 Nokia Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; see the file COPYING. If not, write to the Free Software
+ * Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Test page read and write on MTD device.
+ *
+ * Author: Adrian Hunter <ext-adrian.hunter@nokia.com>
+ */
+
+#include <asm/div64.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/err.h>
+#include <linux/mtd/mtd.h>
+#include <linux/sched.h>
+
+#define PRINT_PREF KERN_INFO "mtd_pagetest: "
+
+static int dev;
+module_param(dev, int, S_IRUGO);
+MODULE_PARM_DESC(dev, "MTD device number to use");
+
+static struct mtd_info *mtd;
+static unsigned char *twopages;
+static unsigned char *writebuf;
+static unsigned char *boundary;
+static unsigned char *bbt;
+
+static int pgsize;
+static int bufsize;
+static int ebcnt;
+static int pgcnt;
+static int errcnt;
+static unsigned long next = 1;
+
+static inline unsigned int simple_rand(void)
+{
+ next = next * 1103515245 + 12345;
+ return (unsigned int)((next / 65536) % 32768);
+}
+
+static inline void simple_srand(unsigned long seed)
+{
+ next = seed;
+}
+
+static void set_random_data(unsigned char *buf, size_t len)
+{
+ size_t i;
+
+ for (i = 0; i < len; ++i)
+ buf[i] = simple_rand();
+}
+
+static int erase_eraseblock(int ebnum)
+{
+ int err;
+ struct erase_info ei;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ memset(&ei, 0, sizeof(struct erase_info));
+ ei.mtd = mtd;
+ ei.addr = addr;
+ ei.len = mtd->erasesize;
+
+ err = mtd->erase(mtd, &ei);
+ if (err) {
+ printk(PRINT_PREF "error %d while erasing EB %d\n", err, ebnum);
+ return err;
+ }
+
+ if (ei.state == MTD_ERASE_FAILED) {
+ printk(PRINT_PREF "some erase error occurred at EB %d\n",
+ ebnum);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+static int write_eraseblock(int ebnum)
+{
+ int err = 0;
+ size_t written = 0;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ set_random_data(writebuf, mtd->erasesize);
+ cond_resched();
+ err = mtd->write(mtd, addr, mtd->erasesize, &written, writebuf);
+ if (err || written != mtd->erasesize)
+ printk(PRINT_PREF "error: write failed at %#llx\n",
+ (long long)addr);
+
+ return err;
+}
+
+static int verify_eraseblock(int ebnum)
+{
+ uint32_t j;
+ size_t read = 0;
+ int err = 0, i;
+ loff_t addr0, addrn;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ addr0 = 0;
+ for (i = 0; bbt[i] && i < ebcnt; ++i)
+ addr0 += mtd->erasesize;
+
+ addrn = mtd->size;
+ for (i = 0; bbt[ebcnt - i - 1] && i < ebcnt; ++i)
+ addrn -= mtd->erasesize;
+
+ set_random_data(writebuf, mtd->erasesize);
+ for (j = 0; j < pgcnt - 1; ++j, addr += pgsize) {
+ /* Do a read to set the internal dataRAMs to different data */
+ err = mtd->read(mtd, addr0, bufsize, &read, twopages);
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != bufsize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ (long long)addr0);
+ return err;
+ }
+ err = mtd->read(mtd, addrn - bufsize, bufsize, &read, twopages);
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != bufsize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ (long long)(addrn - bufsize));
+ return err;
+ }
+ memset(twopages, 0, bufsize);
+ read = 0;
+ err = mtd->read(mtd, addr, bufsize, &read, twopages);
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != bufsize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ (long long)addr);
+ break;
+ }
+ if (memcmp(twopages, writebuf + (j * pgsize), bufsize)) {
+ printk(PRINT_PREF "error: verify failed at %#llx\n",
+ (long long)addr);
+ errcnt += 1;
+ }
+ }
+ /* Check boundary between eraseblocks */
+ if (addr <= addrn - pgsize - pgsize && !bbt[ebnum + 1]) {
+ unsigned long oldnext = next;
+ /* Do a read to set the internal dataRAMs to different data */
+ err = mtd->read(mtd, addr0, bufsize, &read, twopages);
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != bufsize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ (long long)addr0);
+ return err;
+ }
+ err = mtd->read(mtd, addrn - bufsize, bufsize, &read, twopages);
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != bufsize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ (long long)(addrn - bufsize));
+ return err;
+ }
+ memset(twopages, 0, bufsize);
+ read = 0;
+ err = mtd->read(mtd, addr, bufsize, &read, twopages);
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != bufsize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ (long long)addr);
+ return err;
+ }
+ memcpy(boundary, writebuf + mtd->erasesize - pgsize, pgsize);
+ set_random_data(boundary + pgsize, pgsize);
+ if (memcmp(twopages, boundary, bufsize)) {
+ printk(PRINT_PREF "error: verify failed at %#llx\n",
+ (long long)addr);
+ errcnt += 1;
+ }
+ next = oldnext;
+ }
+ return err;
+}
+
+static int crosstest(void)
+{
+ size_t read = 0;
+ int err = 0, i;
+ loff_t addr, addr0, addrn;
+ unsigned char *pp1, *pp2, *pp3, *pp4;
+
+ printk(PRINT_PREF "crosstest\n");
+ pp1 = kmalloc(pgsize * 4, GFP_KERNEL);
+ if (!pp1) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ return -ENOMEM;
+ }
+ pp2 = pp1 + pgsize;
+ pp3 = pp2 + pgsize;
+ pp4 = pp3 + pgsize;
+ memset(pp1, 0, pgsize * 4);
+
+ addr0 = 0;
+ for (i = 0; bbt[i] && i < ebcnt; ++i)
+ addr0 += mtd->erasesize;
+
+ addrn = mtd->size;
+ for (i = 0; bbt[ebcnt - i - 1] && i < ebcnt; ++i)
+ addrn -= mtd->erasesize;
+
+ /* Read 2nd-to-last page to pp1 */
+ read = 0;
+ addr = addrn - pgsize - pgsize;
+ err = mtd->read(mtd, addr, pgsize, &read, pp1);
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != pgsize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ (long long)addr);
+ kfree(pp1);
+ return err;
+ }
+
+ /* Read 3rd-to-last page to pp1 */
+ read = 0;
+ addr = addrn - pgsize - pgsize - pgsize;
+ err = mtd->read(mtd, addr, pgsize, &read, pp1);
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != pgsize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ (long long)addr);
+ kfree(pp1);
+ return err;
+ }
+
+ /* Read first page to pp2 */
+ read = 0;
+ addr = addr0;
+ printk(PRINT_PREF "reading page at %#llx\n", (long long)addr);
+ err = mtd->read(mtd, addr, pgsize, &read, pp2);
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != pgsize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ (long long)addr);
+ kfree(pp1);
+ return err;
+ }
+
+ /* Read last page to pp3 */
+ read = 0;
+ addr = addrn - pgsize;
+ printk(PRINT_PREF "reading page at %#llx\n", (long long)addr);
+ err = mtd->read(mtd, addr, pgsize, &read, pp3);
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != pgsize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ (long long)addr);
+ kfree(pp1);
+ return err;
+ }
+
+ /* Read first page again to pp4 */
+ read = 0;
+ addr = addr0;
+ printk(PRINT_PREF "reading page at %#llx\n", (long long)addr);
+ err = mtd->read(mtd, addr, pgsize, &read, pp4);
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != pgsize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ (long long)addr);
+ kfree(pp1);
+ return err;
+ }
+
+ /* pp2 and pp4 should be the same */
+ printk(PRINT_PREF "verifying pages read at %#llx match\n",
+ (long long)addr0);
+ if (memcmp(pp2, pp4, pgsize)) {
+ printk(PRINT_PREF "verify failed!\n");
+ errcnt += 1;
+ } else if (!err)
+ printk(PRINT_PREF "crosstest ok\n");
+ kfree(pp1);
+ return err;
+}
+
+static int erasecrosstest(void)
+{
+ size_t read = 0, written = 0;
+ int err = 0, i, ebnum, ok = 1, ebnum2;
+ loff_t addr0;
+ char *readbuf = twopages;
+
+ printk(PRINT_PREF "erasecrosstest\n");
+
+ ebnum = 0;
+ addr0 = 0;
+ for (i = 0; bbt[i] && i < ebcnt; ++i) {
+ addr0 += mtd->erasesize;
+ ebnum += 1;
+ }
+
+ ebnum2 = ebcnt - 1;
+ while (ebnum2 && bbt[ebnum2])
+ ebnum2 -= 1;
+
+ printk(PRINT_PREF "erasing block %d\n", ebnum);
+ err = erase_eraseblock(ebnum);
+ if (err)
+ return err;
+
+ printk(PRINT_PREF "writing 1st page of block %d\n", ebnum);
+ set_random_data(writebuf, pgsize);
+ strcpy(writebuf, "There is no data like this!");
+ err = mtd->write(mtd, addr0, pgsize, &written, writebuf);
+ if (err || written != pgsize) {
+ printk(PRINT_PREF "error: write failed at %#llx\n",
+ (long long)addr0);
+ return err ? err : -1;
+ }
+
+ printk(PRINT_PREF "reading 1st page of block %d\n", ebnum);
+ memset(readbuf, 0, pgsize);
+ err = mtd->read(mtd, addr0, pgsize, &read, readbuf);
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != pgsize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ (long long)addr0);
+ return err ? err : -1;
+ }
+
+ printk(PRINT_PREF "verifying 1st page of block %d\n", ebnum);
+ if (memcmp(writebuf, readbuf, pgsize)) {
+ printk(PRINT_PREF "verify failed!\n");
+ errcnt += 1;
+ ok = 0;
+ return err;
+ }
+
+ printk(PRINT_PREF "erasing block %d\n", ebnum);
+ err = erase_eraseblock(ebnum);
+ if (err)
+ return err;
+
+ printk(PRINT_PREF "writing 1st page of block %d\n", ebnum);
+ set_random_data(writebuf, pgsize);
+ strcpy(writebuf, "There is no data like this!");
+ err = mtd->write(mtd, addr0, pgsize, &written, writebuf);
+ if (err || written != pgsize) {
+ printk(PRINT_PREF "error: write failed at %#llx\n",
+ (long long)addr0);
+ return err ? err : -1;
+ }
+
+ printk(PRINT_PREF "erasing block %d\n", ebnum2);
+ err = erase_eraseblock(ebnum2);
+ if (err)
+ return err;
+
+ printk(PRINT_PREF "reading 1st page of block %d\n", ebnum);
+ memset(readbuf, 0, pgsize);
+ err = mtd->read(mtd, addr0, pgsize, &read, readbuf);
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != pgsize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ (long long)addr0);
+ return err ? err : -1;
+ }
+
+ printk(PRINT_PREF "verifying 1st page of block %d\n", ebnum);
+ if (memcmp(writebuf, readbuf, pgsize)) {
+ printk(PRINT_PREF "verify failed!\n");
+ errcnt += 1;
+ ok = 0;
+ }
+
+ if (ok && !err)
+ printk(PRINT_PREF "erasecrosstest ok\n");
+ return err;
+}
+
+static int erasetest(void)
+{
+ size_t read = 0, written = 0;
+ int err = 0, i, ebnum, ok = 1;
+ loff_t addr0;
+
+ printk(PRINT_PREF "erasetest\n");
+
+ ebnum = 0;
+ addr0 = 0;
+ for (i = 0; bbt[i] && i < ebcnt; ++i) {
+ addr0 += mtd->erasesize;
+ ebnum += 1;
+ }
+
+ printk(PRINT_PREF "erasing block %d\n", ebnum);
+ err = erase_eraseblock(ebnum);
+ if (err)
+ return err;
+
+ printk(PRINT_PREF "writing 1st page of block %d\n", ebnum);
+ set_random_data(writebuf, pgsize);
+ err = mtd->write(mtd, addr0, pgsize, &written, writebuf);
+ if (err || written != pgsize) {
+ printk(PRINT_PREF "error: write failed at %#llx\n",
+ (long long)addr0);
+ return err ? err : -1;
+ }
+
+ printk(PRINT_PREF "erasing block %d\n", ebnum);
+ err = erase_eraseblock(ebnum);
+ if (err)
+ return err;
+
+ printk(PRINT_PREF "reading 1st page of block %d\n", ebnum);
+ err = mtd->read(mtd, addr0, pgsize, &read, twopages);
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != pgsize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ (long long)addr0);
+ return err ? err : -1;
+ }
+
+ printk(PRINT_PREF "verifying 1st page of block %d is all 0xff\n",
+ ebnum);
+ for (i = 0; i < pgsize; ++i)
+ if (twopages[i] != 0xff) {
+ printk(PRINT_PREF "verifying all 0xff failed at %d\n",
+ i);
+ errcnt += 1;
+ ok = 0;
+ break;
+ }
+
+ if (ok && !err)
+ printk(PRINT_PREF "erasetest ok\n");
+
+ return err;
+}
+
+static int is_block_bad(int ebnum)
+{
+ loff_t addr = ebnum * mtd->erasesize;
+ int ret;
+
+ ret = mtd->block_isbad(mtd, addr);
+ if (ret)
+ printk(PRINT_PREF "block %d is bad\n", ebnum);
+ return ret;
+}
+
+static int scan_for_bad_eraseblocks(void)
+{
+ int i, bad = 0;
+
+ bbt = kmalloc(ebcnt, GFP_KERNEL);
+ if (!bbt) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ return -ENOMEM;
+ }
+ memset(bbt, 0 , ebcnt);
+
+ printk(PRINT_PREF "scanning for bad eraseblocks\n");
+ for (i = 0; i < ebcnt; ++i) {
+ bbt[i] = is_block_bad(i) ? 1 : 0;
+ if (bbt[i])
+ bad += 1;
+ cond_resched();
+ }
+ printk(PRINT_PREF "scanned %d eraseblocks, %d are bad\n", i, bad);
+ return 0;
+}
+
+static int __init mtd_pagetest_init(void)
+{
+ int err = 0;
+ uint64_t tmp;
+ uint32_t i;
+
+ printk(KERN_INFO "\n");
+ printk(KERN_INFO "=================================================\n");
+ printk(PRINT_PREF "MTD device: %d\n", dev);
+
+ mtd = get_mtd_device(NULL, dev);
+ if (IS_ERR(mtd)) {
+ err = PTR_ERR(mtd);
+ printk(PRINT_PREF "error: cannot get MTD device\n");
+ return err;
+ }
+
+ if (mtd->type != MTD_NANDFLASH) {
+ printk(PRINT_PREF "this test requires NAND flash\n");
+ goto out;
+ }
+
+ tmp = mtd->size;
+ do_div(tmp, mtd->erasesize);
+ ebcnt = tmp;
+ pgcnt = mtd->erasesize / mtd->writesize;
+
+ printk(PRINT_PREF "MTD device size %llu, eraseblock size %u, "
+ "page size %u, count of eraseblocks %u, pages per "
+ "eraseblock %u, OOB size %u\n",
+ (unsigned long long)mtd->size, mtd->erasesize,
+ pgsize, ebcnt, pgcnt, mtd->oobsize);
+
+ err = -ENOMEM;
+ bufsize = pgsize * 2;
+ writebuf = kmalloc(mtd->erasesize, GFP_KERNEL);
+ if (!writebuf) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ goto out;
+ }
+ twopages = kmalloc(bufsize, GFP_KERNEL);
+ if (!twopages) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ goto out;
+ }
+ boundary = kmalloc(bufsize, GFP_KERNEL);
+ if (!boundary) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ goto out;
+ }
+
+ err = scan_for_bad_eraseblocks();
+ if (err)
+ goto out;
+
+ /* Erase all eraseblocks */
+ printk(PRINT_PREF "erasing whole device\n");
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = erase_eraseblock(i);
+ if (err)
+ goto out;
+ cond_resched();
+ }
+ printk(PRINT_PREF "erased %u eraseblocks\n", i);
+
+ /* Write all eraseblocks */
+ simple_srand(1);
+ printk(PRINT_PREF "writing whole device\n");
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = write_eraseblock(i);
+ if (err)
+ goto out;
+ if (i % 256 == 0)
+ printk(PRINT_PREF "written up to eraseblock %u\n", i);
+ cond_resched();
+ }
+ printk(PRINT_PREF "written %u eraseblocks\n", i);
+
+ /* Check all eraseblocks */
+ simple_srand(1);
+ printk(PRINT_PREF "verifying all eraseblocks\n");
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = verify_eraseblock(i);
+ if (err)
+ goto out;
+ if (i % 256 == 0)
+ printk(PRINT_PREF "verified up to eraseblock %u\n", i);
+ cond_resched();
+ }
+ printk(PRINT_PREF "verified %u eraseblocks\n", i);
+
+ err = crosstest();
+ if (err)
+ goto out;
+
+ err = erasecrosstest();
+ if (err)
+ goto out;
+
+ err = erasetest();
+ if (err)
+ goto out;
+
+ printk(PRINT_PREF "finished with %d errors\n", errcnt);
+out:
+
+ kfree(bbt);
+ kfree(boundary);
+ kfree(twopages);
+ kfree(writebuf);
+ put_mtd_device(mtd);
+ if (err)
+ printk(PRINT_PREF "error %d occurred\n", err);
+ printk(KERN_INFO "=================================================\n");
+ return err;
+}
+module_init(mtd_pagetest_init);
+
+static void __exit mtd_pagetest_exit(void)
+{
+ return;
+}
+module_exit(mtd_pagetest_exit);
+
+MODULE_DESCRIPTION("NAND page test");
+MODULE_AUTHOR("Adrian Hunter");
+MODULE_LICENSE("GPL");
diff --git a/drivers/mtd/tests/mtd_readtest.c b/drivers/mtd/tests/mtd_readtest.c
new file mode 100644
index 0000000..645e77f
--- /dev/null
+++ b/drivers/mtd/tests/mtd_readtest.c
@@ -0,0 +1,253 @@
+/*
+ * Copyright (C) 2006-2008 Nokia Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; see the file COPYING. If not, write to the Free Software
+ * Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Check MTD device read.
+ *
+ * Author: Adrian Hunter <ext-adrian.hunter@nokia.com>
+ */
+
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/err.h>
+#include <linux/mtd/mtd.h>
+#include <linux/sched.h>
+
+#define PRINT_PREF KERN_INFO "mtd_readtest: "
+
+static int dev;
+module_param(dev, int, S_IRUGO);
+MODULE_PARM_DESC(dev, "MTD device number to use");
+
+static struct mtd_info *mtd;
+static unsigned char *iobuf;
+static unsigned char *iobuf1;
+static unsigned char *bbt;
+
+static int pgsize;
+static int ebcnt;
+static int pgcnt;
+
+static int read_eraseblock_by_page(int ebnum)
+{
+ size_t read = 0;
+ int i, ret, err = 0;
+ loff_t addr = ebnum * mtd->erasesize;
+ void *buf = iobuf;
+ void *oobbuf = iobuf1;
+
+ for (i = 0; i < pgcnt; i++) {
+ memset(buf, 0 , pgcnt);
+ ret = mtd->read(mtd, addr, pgsize, &read, buf);
+ if (ret == -EUCLEAN)
+ ret = 0;
+ if (ret || read != pgsize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ (long long)addr);
+ if (!err)
+ err = ret;
+ if (!err)
+ err = -EINVAL;
+ }
+ if (mtd->oobsize) {
+ struct mtd_oob_ops ops;
+
+ ops.mode = MTD_OOB_PLACE;
+ ops.len = 0;
+ ops.retlen = 0;
+ ops.ooblen = mtd->oobsize;
+ ops.oobretlen = 0;
+ ops.ooboffs = 0;
+ ops.datbuf = 0;
+ ops.oobbuf = oobbuf;
+ ret = mtd->read_oob(mtd, addr, &ops);
+ if (ret || ops.oobretlen != mtd->oobsize) {
+ printk(PRINT_PREF "error: read oob failed at "
+ "%#llx\n", (long long)addr);
+ if (!err)
+ err = ret;
+ if (!err)
+ err = -EINVAL;
+ }
+ oobbuf += mtd->oobsize;
+ }
+ addr += pgsize;
+ buf += pgsize;
+ }
+
+ return err;
+}
+
+static void dump_eraseblock(int ebnum)
+{
+ int i, j, n;
+ char line[128];
+ int pg, oob;
+
+ printk(PRINT_PREF "dumping eraseblock %d\n", ebnum);
+ n = mtd->erasesize;
+ for (i = 0; i < n;) {
+ char *p = line;
+
+ p += sprintf(p, "%05x: ", i);
+ for (j = 0; j < 32 && i < n; j++, i++)
+ p += sprintf(p, "%02x", (unsigned int)iobuf[i]);
+ printk(KERN_CRIT "%s\n", line);
+ cond_resched();
+ }
+ if (!mtd->oobsize)
+ return;
+ printk(PRINT_PREF "dumping oob from eraseblock %d\n", ebnum);
+ n = mtd->oobsize;
+ for (pg = 0, i = 0; pg < pgcnt; pg++)
+ for (oob = 0; oob < n;) {
+ char *p = line;
+
+ p += sprintf(p, "%05x: ", i);
+ for (j = 0; j < 32 && oob < n; j++, oob++, i++)
+ p += sprintf(p, "%02x",
+ (unsigned int)iobuf1[i]);
+ printk(KERN_CRIT "%s\n", line);
+ cond_resched();
+ }
+}
+
+static int is_block_bad(int ebnum)
+{
+ loff_t addr = ebnum * mtd->erasesize;
+ int ret;
+
+ ret = mtd->block_isbad(mtd, addr);
+ if (ret)
+ printk(PRINT_PREF "block %d is bad\n", ebnum);
+ return ret;
+}
+
+static int scan_for_bad_eraseblocks(void)
+{
+ int i, bad = 0;
+
+ bbt = kmalloc(ebcnt, GFP_KERNEL);
+ if (!bbt) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ return -ENOMEM;
+ }
+ memset(bbt, 0 , ebcnt);
+
+ printk(PRINT_PREF "scanning for bad eraseblocks\n");
+ for (i = 0; i < ebcnt; ++i) {
+ bbt[i] = is_block_bad(i) ? 1 : 0;
+ if (bbt[i])
+ bad += 1;
+ cond_resched();
+ }
+ printk(PRINT_PREF "scanned %d eraseblocks, %d are bad\n", i, bad);
+ return 0;
+}
+
+static int __init mtd_readtest_init(void)
+{
+ uint64_t tmp;
+ int err, i;
+
+ printk(KERN_INFO "\n");
+ printk(KERN_INFO "=================================================\n");
+ printk(PRINT_PREF "MTD device: %d\n", dev);
+
+ mtd = get_mtd_device(NULL, dev);
+ if (IS_ERR(mtd)) {
+ err = PTR_ERR(mtd);
+ printk(PRINT_PREF "error: Cannot get MTD device\n");
+ return err;
+ }
+
+ if (mtd->writesize == 1) {
+ printk(PRINT_PREF "not NAND flash, assume page size is 512 "
+ "bytes.\n");
+ pgsize = 512;
+ } else
+ pgsize = mtd->writesize;
+
+ tmp = mtd->size;
+ do_div(tmp, mtd->erasesize);
+ ebcnt = tmp;
+ pgcnt = mtd->erasesize / mtd->writesize;
+
+ printk(PRINT_PREF "MTD device size %llu, eraseblock size %u, "
+ "page size %u, count of eraseblocks %u, pages per "
+ "eraseblock %u, OOB size %u\n",
+ (unsigned long long)mtd->size, mtd->erasesize,
+ pgsize, ebcnt, pgcnt, mtd->oobsize);
+
+ err = -ENOMEM;
+ iobuf = kmalloc(mtd->erasesize, GFP_KERNEL);
+ if (!iobuf) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ goto out;
+ }
+ iobuf1 = kmalloc(mtd->erasesize, GFP_KERNEL);
+ if (!iobuf1) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ goto out;
+ }
+
+ err = scan_for_bad_eraseblocks();
+ if (err)
+ goto out;
+
+ /* Read all eraseblocks 1 page at a time */
+ printk(PRINT_PREF "testing page read\n");
+ for (i = 0; i < ebcnt; ++i) {
+ int ret;
+
+ if (bbt[i])
+ continue;
+ ret = read_eraseblock_by_page(i);
+ if (ret) {
+ dump_eraseblock(i);
+ if (!err)
+ err = ret;
+ }
+ cond_resched();
+ }
+
+ if (err)
+ printk(PRINT_PREF "finished with errors\n");
+ else
+ printk(PRINT_PREF "finished\n");
+
+out:
+
+ kfree(iobuf);
+ kfree(iobuf1);
+ kfree(bbt);
+ put_mtd_device(mtd);
+ if (err)
+ printk(PRINT_PREF "error %d occurred\n", err);
+ printk(KERN_INFO "=================================================\n");
+ return err;
+}
+module_init(mtd_readtest_init);
+
+static void __exit mtd_readtest_exit(void)
+{
+ return;
+}
+module_exit(mtd_readtest_exit);
+
+MODULE_DESCRIPTION("Read test module");
+MODULE_AUTHOR("Adrian Hunter");
+MODULE_LICENSE("GPL");
diff --git a/drivers/mtd/tests/mtd_speedtest.c b/drivers/mtd/tests/mtd_speedtest.c
new file mode 100644
index 0000000..141363a
--- /dev/null
+++ b/drivers/mtd/tests/mtd_speedtest.c
@@ -0,0 +1,502 @@
+/*
+ * Copyright (C) 2007 Nokia Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; see the file COPYING. If not, write to the Free Software
+ * Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Test read and write speed of a MTD device.
+ *
+ * Author: Adrian Hunter <ext-adrian.hunter@nokia.com>
+ */
+
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/err.h>
+#include <linux/mtd/mtd.h>
+#include <linux/sched.h>
+
+#define PRINT_PREF KERN_INFO "mtd_speedtest: "
+
+static int dev;
+module_param(dev, int, S_IRUGO);
+MODULE_PARM_DESC(dev, "MTD device number to use");
+
+static struct mtd_info *mtd;
+static unsigned char *iobuf;
+static unsigned char *bbt;
+
+static int pgsize;
+static int ebcnt;
+static int pgcnt;
+static int goodebcnt;
+static struct timeval start, finish;
+static unsigned long next = 1;
+
+static inline unsigned int simple_rand(void)
+{
+ next = next * 1103515245 + 12345;
+ return (unsigned int)((next / 65536) % 32768);
+}
+
+static inline void simple_srand(unsigned long seed)
+{
+ next = seed;
+}
+
+static void set_random_data(unsigned char *buf, size_t len)
+{
+ size_t i;
+
+ for (i = 0; i < len; ++i)
+ buf[i] = simple_rand();
+}
+
+static int erase_eraseblock(int ebnum)
+{
+ int err;
+ struct erase_info ei;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ memset(&ei, 0, sizeof(struct erase_info));
+ ei.mtd = mtd;
+ ei.addr = addr;
+ ei.len = mtd->erasesize;
+
+ err = mtd->erase(mtd, &ei);
+ if (err) {
+ printk(PRINT_PREF "error %d while erasing EB %d\n", err, ebnum);
+ return err;
+ }
+
+ if (ei.state == MTD_ERASE_FAILED) {
+ printk(PRINT_PREF "some erase error occurred at EB %d\n",
+ ebnum);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+static int erase_whole_device(void)
+{
+ int err;
+ unsigned int i;
+
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = erase_eraseblock(i);
+ if (err)
+ return err;
+ cond_resched();
+ }
+ return 0;
+}
+
+static int write_eraseblock(int ebnum)
+{
+ size_t written = 0;
+ int err = 0;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ err = mtd->write(mtd, addr, mtd->erasesize, &written, iobuf);
+ if (err || written != mtd->erasesize) {
+ printk(PRINT_PREF "error: write failed at %#llx\n", addr);
+ if (!err)
+ err = -EINVAL;
+ }
+
+ return err;
+}
+
+static int write_eraseblock_by_page(int ebnum)
+{
+ size_t written = 0;
+ int i, err = 0;
+ loff_t addr = ebnum * mtd->erasesize;
+ void *buf = iobuf;
+
+ for (i = 0; i < pgcnt; i++) {
+ err = mtd->write(mtd, addr, pgsize, &written, buf);
+ if (err || written != pgsize) {
+ printk(PRINT_PREF "error: write failed at %#llx\n",
+ addr);
+ if (!err)
+ err = -EINVAL;
+ break;
+ }
+ addr += pgsize;
+ buf += pgsize;
+ }
+
+ return err;
+}
+
+static int write_eraseblock_by_2pages(int ebnum)
+{
+ size_t written = 0, sz = pgsize * 2;
+ int i, n = pgcnt / 2, err = 0;
+ loff_t addr = ebnum * mtd->erasesize;
+ void *buf = iobuf;
+
+ for (i = 0; i < n; i++) {
+ err = mtd->write(mtd, addr, sz, &written, buf);
+ if (err || written != sz) {
+ printk(PRINT_PREF "error: write failed at %#llx\n",
+ addr);
+ if (!err)
+ err = -EINVAL;
+ return err;
+ }
+ addr += sz;
+ buf += sz;
+ }
+ if (pgcnt % 2) {
+ err = mtd->write(mtd, addr, pgsize, &written, buf);
+ if (err || written != pgsize) {
+ printk(PRINT_PREF "error: write failed at %#llx\n",
+ addr);
+ if (!err)
+ err = -EINVAL;
+ }
+ }
+
+ return err;
+}
+
+static int read_eraseblock(int ebnum)
+{
+ size_t read = 0;
+ int err = 0;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ err = mtd->read(mtd, addr, mtd->erasesize, &read, iobuf);
+ /* Ignore corrected ECC errors */
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != mtd->erasesize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n", addr);
+ if (!err)
+ err = -EINVAL;
+ }
+
+ return err;
+}
+
+static int read_eraseblock_by_page(int ebnum)
+{
+ size_t read = 0;
+ int i, err = 0;
+ loff_t addr = ebnum * mtd->erasesize;
+ void *buf = iobuf;
+
+ for (i = 0; i < pgcnt; i++) {
+ err = mtd->read(mtd, addr, pgsize, &read, buf);
+ /* Ignore corrected ECC errors */
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != pgsize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ addr);
+ if (!err)
+ err = -EINVAL;
+ break;
+ }
+ addr += pgsize;
+ buf += pgsize;
+ }
+
+ return err;
+}
+
+static int read_eraseblock_by_2pages(int ebnum)
+{
+ size_t read = 0, sz = pgsize * 2;
+ int i, n = pgcnt / 2, err = 0;
+ loff_t addr = ebnum * mtd->erasesize;
+ void *buf = iobuf;
+
+ for (i = 0; i < n; i++) {
+ err = mtd->read(mtd, addr, sz, &read, buf);
+ /* Ignore corrected ECC errors */
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != sz) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ addr);
+ if (!err)
+ err = -EINVAL;
+ return err;
+ }
+ addr += sz;
+ buf += sz;
+ }
+ if (pgcnt % 2) {
+ err = mtd->read(mtd, addr, pgsize, &read, buf);
+ /* Ignore corrected ECC errors */
+ if (err == -EUCLEAN)
+ err = 0;
+ if (err || read != pgsize) {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ addr);
+ if (!err)
+ err = -EINVAL;
+ }
+ }
+
+ return err;
+}
+
+static int is_block_bad(int ebnum)
+{
+ loff_t addr = ebnum * mtd->erasesize;
+ int ret;
+
+ ret = mtd->block_isbad(mtd, addr);
+ if (ret)
+ printk(PRINT_PREF "block %d is bad\n", ebnum);
+ return ret;
+}
+
+static inline void start_timing(void)
+{
+ do_gettimeofday(&start);
+}
+
+static inline void stop_timing(void)
+{
+ do_gettimeofday(&finish);
+}
+
+static long calc_speed(void)
+{
+ long ms, k, speed;
+
+ ms = (finish.tv_sec - start.tv_sec) * 1000 +
+ (finish.tv_usec - start.tv_usec) / 1000;
+ k = goodebcnt * mtd->erasesize / 1024;
+ speed = (k * 1000) / ms;
+ return speed;
+}
+
+static int scan_for_bad_eraseblocks(void)
+{
+ int i, bad = 0;
+
+ bbt = kmalloc(ebcnt, GFP_KERNEL);
+ if (!bbt) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ return -ENOMEM;
+ }
+ memset(bbt, 0 , ebcnt);
+
+ printk(PRINT_PREF "scanning for bad eraseblocks\n");
+ for (i = 0; i < ebcnt; ++i) {
+ bbt[i] = is_block_bad(i) ? 1 : 0;
+ if (bbt[i])
+ bad += 1;
+ cond_resched();
+ }
+ printk(PRINT_PREF "scanned %d eraseblocks, %d are bad\n", i, bad);
+ goodebcnt = ebcnt - bad;
+ return 0;
+}
+
+static int __init mtd_speedtest_init(void)
+{
+ int err, i;
+ long speed;
+ uint64_t tmp;
+
+ printk(KERN_INFO "\n");
+ printk(KERN_INFO "=================================================\n");
+ printk(PRINT_PREF "MTD device: %d\n", dev);
+
+ mtd = get_mtd_device(NULL, dev);
+ if (IS_ERR(mtd)) {
+ err = PTR_ERR(mtd);
+ printk(PRINT_PREF "error: cannot get MTD device\n");
+ return err;
+ }
+
+ if (mtd->writesize == 1) {
+ printk(PRINT_PREF "not NAND flash, assume page size is 512 "
+ "bytes.\n");
+ pgsize = 512;
+ } else
+ pgsize = mtd->writesize;
+
+ tmp = mtd->size;
+ do_div(tmp, mtd->erasesize);
+ ebcnt = tmp;
+ pgcnt = mtd->erasesize / mtd->writesize;
+
+ printk(PRINT_PREF "MTD device size %llu, eraseblock size %u, "
+ "page size %u, count of eraseblocks %u, pages per "
+ "eraseblock %u, OOB size %u\n",
+ (unsigned long long)mtd->size, mtd->erasesize,
+ pgsize, ebcnt, pgcnt, mtd->oobsize);
+
+ err = -ENOMEM;
+ iobuf = kmalloc(mtd->erasesize, GFP_KERNEL);
+ if (!iobuf) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ goto out;
+ }
+
+ simple_srand(1);
+ set_random_data(iobuf, mtd->erasesize);
+
+ err = scan_for_bad_eraseblocks();
+ if (err)
+ goto out;
+
+ err = erase_whole_device();
+ if (err)
+ goto out;
+
+ /* Write all eraseblocks, 1 eraseblock at a time */
+ printk(PRINT_PREF "testing eraseblock write speed\n");
+ start_timing();
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = write_eraseblock(i);
+ if (err)
+ goto out;
+ cond_resched();
+ }
+ stop_timing();
+ speed = calc_speed();
+ printk(PRINT_PREF "eraseblock write speed is %ld KiB/s\n", speed);
+
+ /* Read all eraseblocks, 1 eraseblock at a time */
+ printk(PRINT_PREF "testing eraseblock read speed\n");
+ start_timing();
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = read_eraseblock(i);
+ if (err)
+ goto out;
+ cond_resched();
+ }
+ stop_timing();
+ speed = calc_speed();
+ printk(PRINT_PREF "eraseblock read speed is %ld KiB/s\n", speed);
+
+ err = erase_whole_device();
+ if (err)
+ goto out;
+
+ /* Write all eraseblocks, 1 page at a time */
+ printk(PRINT_PREF "testing page write speed\n");
+ start_timing();
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = write_eraseblock_by_page(i);
+ if (err)
+ goto out;
+ cond_resched();
+ }
+ stop_timing();
+ speed = calc_speed();
+ printk(PRINT_PREF "page write speed is %ld KiB/s\n", speed);
+
+ /* Read all eraseblocks, 1 page at a time */
+ printk(PRINT_PREF "testing page read speed\n");
+ start_timing();
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = read_eraseblock_by_page(i);
+ if (err)
+ goto out;
+ cond_resched();
+ }
+ stop_timing();
+ speed = calc_speed();
+ printk(PRINT_PREF "page read speed is %ld KiB/s\n", speed);
+
+ err = erase_whole_device();
+ if (err)
+ goto out;
+
+ /* Write all eraseblocks, 2 pages at a time */
+ printk(PRINT_PREF "testing 2 page write speed\n");
+ start_timing();
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = write_eraseblock_by_2pages(i);
+ if (err)
+ goto out;
+ cond_resched();
+ }
+ stop_timing();
+ speed = calc_speed();
+ printk(PRINT_PREF "2 page write speed is %ld KiB/s\n", speed);
+
+ /* Read all eraseblocks, 2 pages at a time */
+ printk(PRINT_PREF "testing 2 page read speed\n");
+ start_timing();
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = read_eraseblock_by_2pages(i);
+ if (err)
+ goto out;
+ cond_resched();
+ }
+ stop_timing();
+ speed = calc_speed();
+ printk(PRINT_PREF "2 page read speed is %ld KiB/s\n", speed);
+
+ /* Erase all eraseblocks */
+ printk(PRINT_PREF "Testing erase speed\n");
+ start_timing();
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = erase_eraseblock(i);
+ if (err)
+ goto out;
+ cond_resched();
+ }
+ stop_timing();
+ speed = calc_speed();
+ printk(PRINT_PREF "erase speed is %ld KiB/s\n", speed);
+
+ printk(PRINT_PREF "finished\n");
+out:
+ kfree(iobuf);
+ kfree(bbt);
+ put_mtd_device(mtd);
+ if (err)
+ printk(PRINT_PREF "error %d occurred\n", err);
+ printk(KERN_INFO "=================================================\n");
+ return err;
+}
+module_init(mtd_speedtest_init);
+
+static void __exit mtd_speedtest_exit(void)
+{
+ return;
+}
+module_exit(mtd_speedtest_exit);
+
+MODULE_DESCRIPTION("Speed test module");
+MODULE_AUTHOR("Adrian Hunter");
+MODULE_LICENSE("GPL");
diff --git a/drivers/mtd/tests/mtd_stresstest.c b/drivers/mtd/tests/mtd_stresstest.c
new file mode 100644
index 0000000..6392047
--- /dev/null
+++ b/drivers/mtd/tests/mtd_stresstest.c
@@ -0,0 +1,330 @@
+/*
+ * Copyright (C) 2006-2008 Nokia Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; see the file COPYING. If not, write to the Free Software
+ * Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Test random reads, writes and erases on MTD device.
+ *
+ * Author: Adrian Hunter <ext-adrian.hunter@nokia.com>
+ */
+
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/err.h>
+#include <linux/mtd/mtd.h>
+#include <linux/sched.h>
+#include <linux/vmalloc.h>
+
+#define PRINT_PREF KERN_INFO "mtd_stresstest: "
+
+static int dev;
+module_param(dev, int, S_IRUGO);
+MODULE_PARM_DESC(dev, "MTD device number to use");
+
+static int count = 10000;
+module_param(count, int, S_IRUGO);
+MODULE_PARM_DESC(count, "Number of operations to do (default is 10000)");
+
+static struct mtd_info *mtd;
+static unsigned char *writebuf;
+static unsigned char *readbuf;
+static unsigned char *bbt;
+static int *offsets;
+
+static int pgsize;
+static int bufsize;
+static int ebcnt;
+static int pgcnt;
+static unsigned long next = 1;
+
+static inline unsigned int simple_rand(void)
+{
+ next = next * 1103515245 + 12345;
+ return (unsigned int)((next / 65536) % 32768);
+}
+
+static inline void simple_srand(unsigned long seed)
+{
+ next = seed;
+}
+
+static int rand_eb(void)
+{
+ int eb;
+
+again:
+ if (ebcnt < 32768)
+ eb = simple_rand();
+ else
+ eb = (simple_rand() << 15) | simple_rand();
+ /* Read or write up 2 eraseblocks at a time - hence 'ebcnt - 1' */
+ eb %= (ebcnt - 1);
+ if (bbt[eb])
+ goto again;
+ return eb;
+}
+
+static int rand_offs(void)
+{
+ int offs;
+
+ if (bufsize < 32768)
+ offs = simple_rand();
+ else
+ offs = (simple_rand() << 15) | simple_rand();
+ offs %= bufsize;
+ return offs;
+}
+
+static int rand_len(int offs)
+{
+ int len;
+
+ if (bufsize < 32768)
+ len = simple_rand();
+ else
+ len = (simple_rand() << 15) | simple_rand();
+ len %= (bufsize - offs);
+ return len;
+}
+
+static int erase_eraseblock(int ebnum)
+{
+ int err;
+ struct erase_info ei;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ memset(&ei, 0, sizeof(struct erase_info));
+ ei.mtd = mtd;
+ ei.addr = addr;
+ ei.len = mtd->erasesize;
+
+ err = mtd->erase(mtd, &ei);
+ if (unlikely(err)) {
+ printk(PRINT_PREF "error %d while erasing EB %d\n", err, ebnum);
+ return err;
+ }
+
+ if (unlikely(ei.state == MTD_ERASE_FAILED)) {
+ printk(PRINT_PREF "some erase error occurred at EB %d\n",
+ ebnum);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+static int is_block_bad(int ebnum)
+{
+ loff_t addr = ebnum * mtd->erasesize;
+ int ret;
+
+ ret = mtd->block_isbad(mtd, addr);
+ if (ret)
+ printk(PRINT_PREF "block %d is bad\n", ebnum);
+ return ret;
+}
+
+static int do_read(void)
+{
+ size_t read = 0;
+ int eb = rand_eb();
+ int offs = rand_offs();
+ int len = rand_len(offs), err;
+ loff_t addr;
+
+ if (bbt[eb + 1]) {
+ if (offs >= mtd->erasesize)
+ offs -= mtd->erasesize;
+ if (offs + len > mtd->erasesize)
+ len = mtd->erasesize - offs;
+ }
+ addr = eb * mtd->erasesize + offs;
+ err = mtd->read(mtd, addr, len, &read, readbuf);
+ if (err == -EUCLEAN)
+ err = 0;
+ if (unlikely(err || read != len)) {
+ printk(PRINT_PREF "error: read failed at 0x%llx\n",
+ (long long)addr);
+ if (!err)
+ err = -EINVAL;
+ return err;
+ }
+ return 0;
+}
+
+static int do_write(void)
+{
+ int eb = rand_eb(), offs, err, len;
+ size_t written = 0;
+ loff_t addr;
+
+ offs = offsets[eb];
+ if (offs >= mtd->erasesize) {
+ err = erase_eraseblock(eb);
+ if (err)
+ return err;
+ offs = offsets[eb] = 0;
+ }
+ len = rand_len(offs);
+ len = ((len + pgsize - 1) / pgsize) * pgsize;
+ if (offs + len > mtd->erasesize) {
+ if (bbt[eb + 1])
+ len = mtd->erasesize - offs;
+ else {
+ err = erase_eraseblock(eb + 1);
+ if (err)
+ return err;
+ offsets[eb + 1] = 0;
+ }
+ }
+ addr = eb * mtd->erasesize + offs;
+ err = mtd->write(mtd, addr, len, &written, writebuf);
+ if (unlikely(err || written != len)) {
+ printk(PRINT_PREF "error: write failed at 0x%llx\n",
+ (long long)addr);
+ if (!err)
+ err = -EINVAL;
+ return err;
+ }
+ offs += len;
+ while (offs > mtd->erasesize) {
+ offsets[eb++] = mtd->erasesize;
+ offs -= mtd->erasesize;
+ }
+ offsets[eb] = offs;
+ return 0;
+}
+
+static int do_operation(void)
+{
+ if (simple_rand() & 1)
+ return do_read();
+ else
+ return do_write();
+}
+
+static int scan_for_bad_eraseblocks(void)
+{
+ int i, bad = 0;
+
+ bbt = kmalloc(ebcnt, GFP_KERNEL);
+ if (!bbt) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ return -ENOMEM;
+ }
+ memset(bbt, 0 , ebcnt);
+
+ printk(PRINT_PREF "scanning for bad eraseblocks\n");
+ for (i = 0; i < ebcnt; ++i) {
+ bbt[i] = is_block_bad(i) ? 1 : 0;
+ if (bbt[i])
+ bad += 1;
+ cond_resched();
+ }
+ printk(PRINT_PREF "scanned %d eraseblocks, %d are bad\n", i, bad);
+ return 0;
+}
+
+static int __init mtd_stresstest_init(void)
+{
+ int err;
+ int i, op;
+ uint64_t tmp;
+
+ printk(KERN_INFO "\n");
+ printk(KERN_INFO "=================================================\n");
+ printk(PRINT_PREF "MTD device: %d\n", dev);
+
+ mtd = get_mtd_device(NULL, dev);
+ if (IS_ERR(mtd)) {
+ err = PTR_ERR(mtd);
+ printk(PRINT_PREF "error: cannot get MTD device\n");
+ return err;
+ }
+
+ if (mtd->writesize == 1) {
+ printk(PRINT_PREF "not NAND flash, assume page size is 512 "
+ "bytes.\n");
+ pgsize = 512;
+ } else
+ pgsize = mtd->writesize;
+
+ tmp = mtd->size;
+ do_div(tmp, mtd->erasesize);
+ ebcnt = tmp;
+ pgcnt = mtd->erasesize / mtd->writesize;
+
+ printk(PRINT_PREF "MTD device size %llu, eraseblock size %u, "
+ "page size %u, count of eraseblocks %u, pages per "
+ "eraseblock %u, OOB size %u\n",
+ (unsigned long long)mtd->size, mtd->erasesize,
+ pgsize, ebcnt, pgcnt, mtd->oobsize);
+
+ /* Read or write up 2 eraseblocks at a time */
+ bufsize = mtd->erasesize * 2;
+
+ err = -ENOMEM;
+ readbuf = vmalloc(bufsize);
+ writebuf = vmalloc(bufsize);
+ offsets = kmalloc(ebcnt * sizeof(int), GFP_KERNEL);
+ if (!readbuf || !writebuf || !offsets) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ goto out;
+ }
+ for (i = 0; i < ebcnt; i++)
+ offsets[i] = mtd->erasesize;
+ simple_srand(current->pid);
+ for (i = 0; i < bufsize; i++)
+ writebuf[i] = simple_rand();
+
+ err = scan_for_bad_eraseblocks();
+ if (err)
+ goto out;
+
+ /* Do operations */
+ printk(PRINT_PREF "doing operations\n");
+ for (op = 0; op < count; op++) {
+ if ((op & 1023) == 0)
+ printk(PRINT_PREF "%d operations done\n", op);
+ err = do_operation();
+ if (err)
+ goto out;
+ cond_resched();
+ }
+ printk(PRINT_PREF "finished, %d operations done\n", op);
+
+out:
+ kfree(offsets);
+ kfree(bbt);
+ vfree(writebuf);
+ vfree(readbuf);
+ put_mtd_device(mtd);
+ if (err)
+ printk(PRINT_PREF "error %d occurred\n", err);
+ printk(KERN_INFO "=================================================\n");
+ return err;
+}
+module_init(mtd_stresstest_init);
+
+static void __exit mtd_stresstest_exit(void)
+{
+ return;
+}
+module_exit(mtd_stresstest_exit);
+
+MODULE_DESCRIPTION("Stress test module");
+MODULE_AUTHOR("Adrian Hunter");
+MODULE_LICENSE("GPL");
diff --git a/drivers/mtd/tests/mtd_subpagetest.c b/drivers/mtd/tests/mtd_subpagetest.c
new file mode 100644
index 0000000..5b88972
--- /dev/null
+++ b/drivers/mtd/tests/mtd_subpagetest.c
@@ -0,0 +1,525 @@
+/*
+ * Copyright (C) 2006-2007 Nokia Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; see the file COPYING. If not, write to the Free Software
+ * Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Test sub-page read and write on MTD device.
+ * Author: Adrian Hunter <ext-adrian.hunter@nokia.com>
+ *
+ */
+
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/err.h>
+#include <linux/mtd/mtd.h>
+#include <linux/sched.h>
+
+#define PRINT_PREF KERN_INFO "mtd_subpagetest: "
+
+static int dev;
+module_param(dev, int, S_IRUGO);
+MODULE_PARM_DESC(dev, "MTD device number to use");
+
+static struct mtd_info *mtd;
+static unsigned char *writebuf;
+static unsigned char *readbuf;
+static unsigned char *bbt;
+
+static int subpgsize;
+static int bufsize;
+static int ebcnt;
+static int pgcnt;
+static int errcnt;
+static unsigned long next = 1;
+
+static inline unsigned int simple_rand(void)
+{
+ next = next * 1103515245 + 12345;
+ return (unsigned int)((next / 65536) % 32768);
+}
+
+static inline void simple_srand(unsigned long seed)
+{
+ next = seed;
+}
+
+static void set_random_data(unsigned char *buf, size_t len)
+{
+ size_t i;
+
+ for (i = 0; i < len; ++i)
+ buf[i] = simple_rand();
+}
+
+static inline void clear_data(unsigned char *buf, size_t len)
+{
+ memset(buf, 0, len);
+}
+
+static int erase_eraseblock(int ebnum)
+{
+ int err;
+ struct erase_info ei;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ memset(&ei, 0, sizeof(struct erase_info));
+ ei.mtd = mtd;
+ ei.addr = addr;
+ ei.len = mtd->erasesize;
+
+ err = mtd->erase(mtd, &ei);
+ if (err) {
+ printk(PRINT_PREF "error %d while erasing EB %d\n", err, ebnum);
+ return err;
+ }
+
+ if (ei.state == MTD_ERASE_FAILED) {
+ printk(PRINT_PREF "some erase error occurred at EB %d\n",
+ ebnum);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+static int erase_whole_device(void)
+{
+ int err;
+ unsigned int i;
+
+ printk(PRINT_PREF "erasing whole device\n");
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = erase_eraseblock(i);
+ if (err)
+ return err;
+ cond_resched();
+ }
+ printk(PRINT_PREF "erased %u eraseblocks\n", i);
+ return 0;
+}
+
+static int write_eraseblock(int ebnum)
+{
+ size_t written = 0;
+ int err = 0;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ set_random_data(writebuf, subpgsize);
+ err = mtd->write(mtd, addr, subpgsize, &written, writebuf);
+ if (unlikely(err || written != subpgsize)) {
+ printk(PRINT_PREF "error: write failed at %#llx\n",
+ (long long)addr);
+ if (written != subpgsize) {
+ printk(PRINT_PREF " write size: %#x\n", subpgsize);
+ printk(PRINT_PREF " written: %#zx\n", written);
+ }
+ return err ? err : -1;
+ }
+
+ addr += subpgsize;
+
+ set_random_data(writebuf, subpgsize);
+ err = mtd->write(mtd, addr, subpgsize, &written, writebuf);
+ if (unlikely(err || written != subpgsize)) {
+ printk(PRINT_PREF "error: write failed at %#llx\n",
+ (long long)addr);
+ if (written != subpgsize) {
+ printk(PRINT_PREF " write size: %#x\n", subpgsize);
+ printk(PRINT_PREF " written: %#zx\n", written);
+ }
+ return err ? err : -1;
+ }
+
+ return err;
+}
+
+static int write_eraseblock2(int ebnum)
+{
+ size_t written = 0;
+ int err = 0, k;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ for (k = 1; k < 33; ++k) {
+ if (addr + (subpgsize * k) > (ebnum + 1) * mtd->erasesize)
+ break;
+ set_random_data(writebuf, subpgsize * k);
+ err = mtd->write(mtd, addr, subpgsize * k, &written, writebuf);
+ if (unlikely(err || written != subpgsize * k)) {
+ printk(PRINT_PREF "error: write failed at %#llx\n",
+ (long long)addr);
+ if (written != subpgsize) {
+ printk(PRINT_PREF " write size: %#x\n",
+ subpgsize * k);
+ printk(PRINT_PREF " written: %#08zx\n",
+ written);
+ }
+ return err ? err : -1;
+ }
+ addr += subpgsize * k;
+ }
+
+ return err;
+}
+
+static void print_subpage(unsigned char *p)
+{
+ int i, j;
+
+ for (i = 0; i < subpgsize; ) {
+ for (j = 0; i < subpgsize && j < 32; ++i, ++j)
+ printk("%02x", *p++);
+ printk("\n");
+ }
+}
+
+static int verify_eraseblock(int ebnum)
+{
+ size_t read = 0;
+ int err = 0;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ set_random_data(writebuf, subpgsize);
+ clear_data(readbuf, subpgsize);
+ read = 0;
+ err = mtd->read(mtd, addr, subpgsize, &read, readbuf);
+ if (unlikely(err || read != subpgsize)) {
+ if (err == -EUCLEAN && read == subpgsize) {
+ printk(PRINT_PREF "ECC correction at %#llx\n",
+ (long long)addr);
+ err = 0;
+ } else {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ (long long)addr);
+ return err ? err : -1;
+ }
+ }
+ if (unlikely(memcmp(readbuf, writebuf, subpgsize))) {
+ printk(PRINT_PREF "error: verify failed at %#llx\n",
+ (long long)addr);
+ printk(PRINT_PREF "------------- written----------------\n");
+ print_subpage(writebuf);
+ printk(PRINT_PREF "------------- read ------------------\n");
+ print_subpage(readbuf);
+ printk(PRINT_PREF "-------------------------------------\n");
+ errcnt += 1;
+ }
+
+ addr += subpgsize;
+
+ set_random_data(writebuf, subpgsize);
+ clear_data(readbuf, subpgsize);
+ read = 0;
+ err = mtd->read(mtd, addr, subpgsize, &read, readbuf);
+ if (unlikely(err || read != subpgsize)) {
+ if (err == -EUCLEAN && read == subpgsize) {
+ printk(PRINT_PREF "ECC correction at %#llx\n",
+ (long long)addr);
+ err = 0;
+ } else {
+ printk(PRINT_PREF "error: read failed at %#llx\n",
+ (long long)addr);
+ return err ? err : -1;
+ }
+ }
+ if (unlikely(memcmp(readbuf, writebuf, subpgsize))) {
+ printk(PRINT_PREF "error: verify failed at %#llx\n",
+ (long long)addr);
+ printk(PRINT_PREF "------------- written----------------\n");
+ print_subpage(writebuf);
+ printk(PRINT_PREF "------------- read ------------------\n");
+ print_subpage(readbuf);
+ printk(PRINT_PREF "-------------------------------------\n");
+ errcnt += 1;
+ }
+
+ return err;
+}
+
+static int verify_eraseblock2(int ebnum)
+{
+ size_t read = 0;
+ int err = 0, k;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ for (k = 1; k < 33; ++k) {
+ if (addr + (subpgsize * k) > (ebnum + 1) * mtd->erasesize)
+ break;
+ set_random_data(writebuf, subpgsize * k);
+ clear_data(readbuf, subpgsize * k);
+ read = 0;
+ err = mtd->read(mtd, addr, subpgsize * k, &read, readbuf);
+ if (unlikely(err || read != subpgsize * k)) {
+ if (err == -EUCLEAN && read == subpgsize * k) {
+ printk(PRINT_PREF "ECC correction at %#llx\n",
+ (long long)addr);
+ err = 0;
+ } else {
+ printk(PRINT_PREF "error: read failed at "
+ "%#llx\n", (long long)addr);
+ return err ? err : -1;
+ }
+ }
+ if (unlikely(memcmp(readbuf, writebuf, subpgsize * k))) {
+ printk(PRINT_PREF "error: verify failed at %#llx\n",
+ (long long)addr);
+ errcnt += 1;
+ }
+ addr += subpgsize * k;
+ }
+
+ return err;
+}
+
+static int verify_eraseblock_ff(int ebnum)
+{
+ uint32_t j;
+ size_t read = 0;
+ int err = 0;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ memset(writebuf, 0xff, subpgsize);
+ for (j = 0; j < mtd->erasesize / subpgsize; ++j) {
+ clear_data(readbuf, subpgsize);
+ read = 0;
+ err = mtd->read(mtd, addr, subpgsize, &read, readbuf);
+ if (unlikely(err || read != subpgsize)) {
+ if (err == -EUCLEAN && read == subpgsize) {
+ printk(PRINT_PREF "ECC correction at %#llx\n",
+ (long long)addr);
+ err = 0;
+ } else {
+ printk(PRINT_PREF "error: read failed at "
+ "%#llx\n", (long long)addr);
+ return err ? err : -1;
+ }
+ }
+ if (unlikely(memcmp(readbuf, writebuf, subpgsize))) {
+ printk(PRINT_PREF "error: verify 0xff failed at "
+ "%#llx\n", (long long)addr);
+ errcnt += 1;
+ }
+ addr += subpgsize;
+ }
+
+ return err;
+}
+
+static int verify_all_eraseblocks_ff(void)
+{
+ int err;
+ unsigned int i;
+
+ printk(PRINT_PREF "verifying all eraseblocks for 0xff\n");
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = verify_eraseblock_ff(i);
+ if (err)
+ return err;
+ if (i % 256 == 0)
+ printk(PRINT_PREF "verified up to eraseblock %u\n", i);
+ cond_resched();
+ }
+ printk(PRINT_PREF "verified %u eraseblocks\n", i);
+ return 0;
+}
+
+static int is_block_bad(int ebnum)
+{
+ loff_t addr = ebnum * mtd->erasesize;
+ int ret;
+
+ ret = mtd->block_isbad(mtd, addr);
+ if (ret)
+ printk(PRINT_PREF "block %d is bad\n", ebnum);
+ return ret;
+}
+
+static int scan_for_bad_eraseblocks(void)
+{
+ int i, bad = 0;
+
+ bbt = kmalloc(ebcnt, GFP_KERNEL);
+ if (!bbt) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ return -ENOMEM;
+ }
+ memset(bbt, 0 , ebcnt);
+
+ printk(PRINT_PREF "scanning for bad eraseblocks\n");
+ for (i = 0; i < ebcnt; ++i) {
+ bbt[i] = is_block_bad(i) ? 1 : 0;
+ if (bbt[i])
+ bad += 1;
+ cond_resched();
+ }
+ printk(PRINT_PREF "scanned %d eraseblocks, %d are bad\n", i, bad);
+ return 0;
+}
+
+static int __init mtd_subpagetest_init(void)
+{
+ int err = 0;
+ uint32_t i;
+ uint64_t tmp;
+
+ printk(KERN_INFO "\n");
+ printk(KERN_INFO "=================================================\n");
+ printk(PRINT_PREF "MTD device: %d\n", dev);
+
+ mtd = get_mtd_device(NULL, dev);
+ if (IS_ERR(mtd)) {
+ err = PTR_ERR(mtd);
+ printk(PRINT_PREF "error: cannot get MTD device\n");
+ return err;
+ }
+
+ if (mtd->type != MTD_NANDFLASH) {
+ printk(PRINT_PREF "this test requires NAND flash\n");
+ goto out;
+ }
+
+ subpgsize = mtd->writesize >> mtd->subpage_sft;
+ printk(PRINT_PREF "MTD device size %llu, eraseblock size %u, "
+ "page size %u, subpage size %u, count of eraseblocks %u, "
+ "pages per eraseblock %u, OOB size %u\n",
+ (unsigned long long)mtd->size, mtd->erasesize,
+ mtd->writesize, subpgsize, ebcnt, pgcnt, mtd->oobsize);
+
+ err = -ENOMEM;
+ bufsize = subpgsize * 32;
+ writebuf = kmalloc(bufsize, GFP_KERNEL);
+ if (!writebuf) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ goto out;
+ }
+ readbuf = kmalloc(bufsize, GFP_KERNEL);
+ if (!readbuf) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ goto out;
+ }
+
+ tmp = mtd->size;
+ do_div(tmp, mtd->erasesize);
+ ebcnt = tmp;
+ pgcnt = mtd->erasesize / mtd->writesize;
+
+ err = scan_for_bad_eraseblocks();
+ if (err)
+ goto out;
+
+ err = erase_whole_device();
+ if (err)
+ goto out;
+
+ printk(PRINT_PREF "writing whole device\n");
+ simple_srand(1);
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = write_eraseblock(i);
+ if (unlikely(err))
+ goto out;
+ if (i % 256 == 0)
+ printk(PRINT_PREF "written up to eraseblock %u\n", i);
+ cond_resched();
+ }
+ printk(PRINT_PREF "written %u eraseblocks\n", i);
+
+ simple_srand(1);
+ printk(PRINT_PREF "verifying all eraseblocks\n");
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = verify_eraseblock(i);
+ if (unlikely(err))
+ goto out;
+ if (i % 256 == 0)
+ printk(PRINT_PREF "verified up to eraseblock %u\n", i);
+ cond_resched();
+ }
+ printk(PRINT_PREF "verified %u eraseblocks\n", i);
+
+ err = erase_whole_device();
+ if (err)
+ goto out;
+
+ err = verify_all_eraseblocks_ff();
+ if (err)
+ goto out;
+
+ /* Write all eraseblocks */
+ simple_srand(3);
+ printk(PRINT_PREF "writing whole device\n");
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = write_eraseblock2(i);
+ if (unlikely(err))
+ goto out;
+ if (i % 256 == 0)
+ printk(PRINT_PREF "written up to eraseblock %u\n", i);
+ cond_resched();
+ }
+ printk(PRINT_PREF "written %u eraseblocks\n", i);
+
+ /* Check all eraseblocks */
+ simple_srand(3);
+ printk(PRINT_PREF "verifying all eraseblocks\n");
+ for (i = 0; i < ebcnt; ++i) {
+ if (bbt[i])
+ continue;
+ err = verify_eraseblock2(i);
+ if (unlikely(err))
+ goto out;
+ if (i % 256 == 0)
+ printk(PRINT_PREF "verified up to eraseblock %u\n", i);
+ cond_resched();
+ }
+ printk(PRINT_PREF "verified %u eraseblocks\n", i);
+
+ err = erase_whole_device();
+ if (err)
+ goto out;
+
+ err = verify_all_eraseblocks_ff();
+ if (err)
+ goto out;
+
+ printk(PRINT_PREF "finished with %d errors\n", errcnt);
+
+out:
+ kfree(bbt);
+ kfree(readbuf);
+ kfree(writebuf);
+ put_mtd_device(mtd);
+ if (err)
+ printk(PRINT_PREF "error %d occurred\n", err);
+ printk(KERN_INFO "=================================================\n");
+ return err;
+}
+module_init(mtd_subpagetest_init);
+
+static void __exit mtd_subpagetest_exit(void)
+{
+ return;
+}
+module_exit(mtd_subpagetest_exit);
+
+MODULE_DESCRIPTION("Subpage test module");
+MODULE_AUTHOR("Adrian Hunter");
+MODULE_LICENSE("GPL");
diff --git a/drivers/mtd/tests/mtd_torturetest.c b/drivers/mtd/tests/mtd_torturetest.c
new file mode 100644
index 0000000..631a0ab
--- /dev/null
+++ b/drivers/mtd/tests/mtd_torturetest.c
@@ -0,0 +1,530 @@
+/*
+ * Copyright (C) 2006-2008 Artem Bityutskiy
+ * Copyright (C) 2006-2008 Jarkko Lavinen
+ * Copyright (C) 2006-2008 Adrian Hunter
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; see the file COPYING. If not, write to the Free Software
+ * Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Authors: Artem Bityutskiy, Jarkko Lavinen, Adria Hunter
+ *
+ * WARNING: this test program may kill your flash and your device. Do not
+ * use it unless you know what you do. Authors are not responsible for any
+ * damage caused by this program.
+ */
+
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/err.h>
+#include <linux/mtd/mtd.h>
+#include <linux/sched.h>
+
+#define PRINT_PREF KERN_INFO "mtd_torturetest: "
+#define RETRIES 3
+
+static int eb = 8;
+module_param(eb, int, S_IRUGO);
+MODULE_PARM_DESC(eb, "eraseblock number within the selected MTD device");
+
+static int ebcnt = 32;
+module_param(ebcnt, int, S_IRUGO);
+MODULE_PARM_DESC(ebcnt, "number of consecutive eraseblocks to torture");
+
+static int pgcnt;
+module_param(pgcnt, int, S_IRUGO);
+MODULE_PARM_DESC(pgcnt, "number of pages per eraseblock to torture (0 => all)");
+
+static int dev;
+module_param(dev, int, S_IRUGO);
+MODULE_PARM_DESC(dev, "MTD device number to use");
+
+static int gran = 512;
+module_param(gran, int, S_IRUGO);
+MODULE_PARM_DESC(gran, "how often the status information should be printed");
+
+static int check = 1;
+module_param(check, int, S_IRUGO);
+MODULE_PARM_DESC(check, "if the written data should be checked");
+
+static unsigned int cycles_count;
+module_param(cycles_count, uint, S_IRUGO);
+MODULE_PARM_DESC(cycles_count, "how many erase cycles to do "
+ "(infinite by default)");
+
+static struct mtd_info *mtd;
+
+/* This buffer contains 0x555555...0xAAAAAA... pattern */
+static unsigned char *patt_5A5;
+/* This buffer contains 0xAAAAAA...0x555555... pattern */
+static unsigned char *patt_A5A;
+/* This buffer contains all 0xFF bytes */
+static unsigned char *patt_FF;
+/* This a temporary buffer is use when checking data */
+static unsigned char *check_buf;
+/* How many erase cycles were done */
+static unsigned int erase_cycles;
+
+static int pgsize;
+static struct timeval start, finish;
+
+static void report_corrupt(unsigned char *read, unsigned char *written);
+
+static inline void start_timing(void)
+{
+ do_gettimeofday(&start);
+}
+
+static inline void stop_timing(void)
+{
+ do_gettimeofday(&finish);
+}
+
+/*
+ * Erase eraseblock number @ebnum.
+ */
+static inline int erase_eraseblock(int ebnum)
+{
+ int err;
+ struct erase_info ei;
+ loff_t addr = ebnum * mtd->erasesize;
+
+ memset(&ei, 0, sizeof(struct erase_info));
+ ei.mtd = mtd;
+ ei.addr = addr;
+ ei.len = mtd->erasesize;
+
+ err = mtd->erase(mtd, &ei);
+ if (err) {
+ printk(PRINT_PREF "error %d while erasing EB %d\n", err, ebnum);
+ return err;
+ }
+
+ if (ei.state == MTD_ERASE_FAILED) {
+ printk(PRINT_PREF "some erase error occurred at EB %d\n",
+ ebnum);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+/*
+ * Check that the contents of eraseblock number @enbum is equivalent to the
+ * @buf buffer.
+ */
+static inline int check_eraseblock(int ebnum, unsigned char *buf)
+{
+ int err, retries = 0;
+ size_t read = 0;
+ loff_t addr = ebnum * mtd->erasesize;
+ size_t len = mtd->erasesize;
+
+ if (pgcnt) {
+ addr = (ebnum + 1) * mtd->erasesize - pgcnt * pgsize;
+ len = pgcnt * pgsize;
+ }
+
+retry:
+ err = mtd->read(mtd, addr, len, &read, check_buf);
+ if (err == -EUCLEAN)
+ printk(PRINT_PREF "single bit flip occurred at EB %d "
+ "MTD reported that it was fixed.\n", ebnum);
+ else if (err) {
+ printk(PRINT_PREF "error %d while reading EB %d, "
+ "read %zd\n", err, ebnum, read);
+ return err;
+ }
+
+ if (read != len) {
+ printk(PRINT_PREF "failed to read %zd bytes from EB %d, "
+ "read only %zd, but no error reported\n",
+ len, ebnum, read);
+ return -EIO;
+ }
+
+ if (memcmp(buf, check_buf, len)) {
+ printk(PRINT_PREF "read wrong data from EB %d\n", ebnum);
+ report_corrupt(check_buf, buf);
+
+ if (retries++ < RETRIES) {
+ /* Try read again */
+ yield();
+ printk(PRINT_PREF "re-try reading data from EB %d\n",
+ ebnum);
+ goto retry;
+ } else {
+ printk(PRINT_PREF "retried %d times, still errors, "
+ "give-up\n", RETRIES);
+ return -EINVAL;
+ }
+ }
+
+ if (retries != 0)
+ printk(PRINT_PREF "only attempt number %d was OK (!!!)\n",
+ retries);
+
+ return 0;
+}
+
+static inline int write_pattern(int ebnum, void *buf)
+{
+ int err;
+ size_t written = 0;
+ loff_t addr = ebnum * mtd->erasesize;
+ size_t len = mtd->erasesize;
+
+ if (pgcnt) {
+ addr = (ebnum + 1) * mtd->erasesize - pgcnt * pgsize;
+ len = pgcnt * pgsize;
+ }
+ err = mtd->write(mtd, addr, len, &written, buf);
+ if (err) {
+ printk(PRINT_PREF "error %d while writing EB %d, written %zd"
+ " bytes\n", err, ebnum, written);
+ return err;
+ }
+ if (written != len) {
+ printk(PRINT_PREF "written only %zd bytes of %zd, but no error"
+ " reported\n", written, len);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+static int __init tort_init(void)
+{
+ int err = 0, i, infinite = !cycles_count;
+ int bad_ebs[ebcnt];
+
+ printk(KERN_INFO "\n");
+ printk(KERN_INFO "=================================================\n");
+ printk(PRINT_PREF "Warning: this program is trying to wear out your "
+ "flash, stop it if this is not wanted.\n");
+ printk(PRINT_PREF "MTD device: %d\n", dev);
+ printk(PRINT_PREF "torture %d eraseblocks (%d-%d) of mtd%d\n",
+ ebcnt, eb, eb + ebcnt - 1, dev);
+ if (pgcnt)
+ printk(PRINT_PREF "torturing just %d pages per eraseblock\n",
+ pgcnt);
+ printk(PRINT_PREF "write verify %s\n", check ? "enabled" : "disabled");
+
+ mtd = get_mtd_device(NULL, dev);
+ if (IS_ERR(mtd)) {
+ err = PTR_ERR(mtd);
+ printk(PRINT_PREF "error: cannot get MTD device\n");
+ return err;
+ }
+
+ if (mtd->writesize == 1) {
+ printk(PRINT_PREF "not NAND flash, assume page size is 512 "
+ "bytes.\n");
+ pgsize = 512;
+ } else
+ pgsize = mtd->writesize;
+
+ if (pgcnt && (pgcnt > mtd->erasesize / pgsize || pgcnt < 0)) {
+ printk(PRINT_PREF "error: invalid pgcnt value %d\n", pgcnt);
+ goto out_mtd;
+ }
+
+ err = -ENOMEM;
+ patt_5A5 = kmalloc(mtd->erasesize, GFP_KERNEL);
+ if (!patt_5A5) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ goto out_mtd;
+ }
+
+ patt_A5A = kmalloc(mtd->erasesize, GFP_KERNEL);
+ if (!patt_A5A) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ goto out_patt_5A5;
+ }
+
+ patt_FF = kmalloc(mtd->erasesize, GFP_KERNEL);
+ if (!patt_FF) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ goto out_patt_A5A;
+ }
+
+ check_buf = kmalloc(mtd->erasesize, GFP_KERNEL);
+ if (!check_buf) {
+ printk(PRINT_PREF "error: cannot allocate memory\n");
+ goto out_patt_FF;
+ }
+
+ err = 0;
+
+ /* Initialize patterns */
+ memset(patt_FF, 0xFF, mtd->erasesize);
+ for (i = 0; i < mtd->erasesize / pgsize; i++) {
+ if (!(i & 1)) {
+ memset(patt_5A5 + i * pgsize, 0x55, pgsize);
+ memset(patt_A5A + i * pgsize, 0xAA, pgsize);
+ } else {
+ memset(patt_5A5 + i * pgsize, 0xAA, pgsize);
+ memset(patt_A5A + i * pgsize, 0x55, pgsize);
+ }
+ }
+
+ /*
+ * Check if there is a bad eraseblock among those we are going to test.
+ */
+ memset(&bad_ebs[0], 0, sizeof(int) * ebcnt);
+ if (mtd->block_isbad) {
+ for (i = eb; i < eb + ebcnt; i++) {
+ err = mtd->block_isbad(mtd,
+ (loff_t)i * mtd->erasesize);
+
+ if (err < 0) {
+ printk(PRINT_PREF "block_isbad() returned %d "
+ "for EB %d\n", err, i);
+ goto out;
+ }
+
+ if (err) {
+ printk("EB %d is bad. Skip it.\n", i);
+ bad_ebs[i - eb] = 1;
+ }
+ }
+ }
+
+ start_timing();
+ while (1) {
+ int i;
+ void *patt;
+
+ /* Erase all eraseblocks */
+ for (i = eb; i < eb + ebcnt; i++) {
+ if (bad_ebs[i - eb])
+ continue;
+ err = erase_eraseblock(i);
+ if (err)
+ goto out;
+ cond_resched();
+ }
+
+ /* Check if the eraseblocks contain only 0xFF bytes */
+ if (check) {
+ for (i = eb; i < eb + ebcnt; i++) {
+ if (bad_ebs[i - eb])
+ continue;
+ err = check_eraseblock(i, patt_FF);
+ if (err) {
+ printk(PRINT_PREF "verify failed"
+ " for 0xFF... pattern\n");
+ goto out;
+ }
+ cond_resched();
+ }
+ }
+
+ /* Write the pattern */
+ for (i = eb; i < eb + ebcnt; i++) {
+ if (bad_ebs[i - eb])
+ continue;
+ if ((eb + erase_cycles) & 1)
+ patt = patt_5A5;
+ else
+ patt = patt_A5A;
+ err = write_pattern(i, patt);
+ if (err)
+ goto out;
+ cond_resched();
+ }
+
+ /* Verify what we wrote */
+ if (check) {
+ for (i = eb; i < eb + ebcnt; i++) {
+ if (bad_ebs[i - eb])
+ continue;
+ if ((eb + erase_cycles) & 1)
+ patt = patt_5A5;
+ else
+ patt = patt_A5A;
+ err = check_eraseblock(i, patt);
+ if (err) {
+ printk(PRINT_PREF "verify failed for %s"
+ " pattern\n",
+ ((eb + erase_cycles) & 1) ?
+ "0x55AA55..." : "0xAA55AA...");
+ goto out;
+ }
+ cond_resched();
+ }
+ }
+
+ erase_cycles += 1;
+
+ if (erase_cycles % gran == 0) {
+ long ms;
+
+ stop_timing();
+ ms = (finish.tv_sec - start.tv_sec) * 1000 +
+ (finish.tv_usec - start.tv_usec) / 1000;
+ printk(PRINT_PREF "%08u erase cycles done, took %lu "
+ "milliseconds (%lu seconds)\n",
+ erase_cycles, ms, ms / 1000);
+ start_timing();
+ }
+
+ if (!infinite && --cycles_count == 0)
+ break;
+ }
+out:
+
+ printk(PRINT_PREF "finished after %u erase cycles\n",
+ erase_cycles);
+ kfree(check_buf);
+out_patt_FF:
+ kfree(patt_FF);
+out_patt_A5A:
+ kfree(patt_A5A);
+out_patt_5A5:
+ kfree(patt_5A5);
+out_mtd:
+ put_mtd_device(mtd);
+ if (err)
+ printk(PRINT_PREF "error %d occurred during torturing\n", err);
+ printk(KERN_INFO "=================================================\n");
+ return err;
+}
+module_init(tort_init);
+
+static void __exit tort_exit(void)
+{
+ return;
+}
+module_exit(tort_exit);
+
+static int countdiffs(unsigned char *buf, unsigned char *check_buf,
+ unsigned offset, unsigned len, unsigned *bytesp,
+ unsigned *bitsp);
+static void print_bufs(unsigned char *read, unsigned char *written, int start,
+ int len);
+
+/*
+ * Report the detailed information about how the read EB differs from what was
+ * written.
+ */
+static void report_corrupt(unsigned char *read, unsigned char *written)
+{
+ int i;
+ int bytes, bits, pages, first;
+ int offset, len;
+ size_t check_len = mtd->erasesize;
+
+ if (pgcnt)
+ check_len = pgcnt * pgsize;
+
+ bytes = bits = pages = 0;
+ for (i = 0; i < check_len; i += pgsize)
+ if (countdiffs(written, read, i, pgsize, &bytes,
+ &bits) >= 0)
+ pages++;
+
+ printk(PRINT_PREF "verify fails on %d pages, %d bytes/%d bits\n",
+ pages, bytes, bits);
+ printk(PRINT_PREF "The following is a list of all differences between"
+ " what was read from flash and what was expected\n");
+
+ for (i = 0; i < check_len; i += pgsize) {
+ cond_resched();
+ bytes = bits = 0;
+ first = countdiffs(written, read, i, pgsize, &bytes,
+ &bits);
+ if (first < 0)
+ continue;
+
+ printk("-------------------------------------------------------"
+ "----------------------------------\n");
+
+ printk(PRINT_PREF "Page %zd has %d bytes/%d bits failing verify,"
+ " starting at offset 0x%x\n",
+ (mtd->erasesize - check_len + i) / pgsize,
+ bytes, bits, first);
+
+ offset = first & ~0x7;
+ len = ((first + bytes) | 0x7) + 1 - offset;
+
+ print_bufs(read, written, offset, len);
+ }
+}
+
+static void print_bufs(unsigned char *read, unsigned char *written, int start,
+ int len)
+{
+ int i = 0, j1, j2;
+ char *diff;
+
+ printk("Offset Read Written\n");
+ while (i < len) {
+ printk("0x%08x: ", start + i);
+ diff = " ";
+ for (j1 = 0; j1 < 8 && i + j1 < len; j1++) {
+ printk(" %02x", read[start + i + j1]);
+ if (read[start + i + j1] != written[start + i + j1])
+ diff = "***";
+ }
+
+ while (j1 < 8) {
+ printk(" ");
+ j1 += 1;
+ }
+
+ printk(" %s ", diff);
+
+ for (j2 = 0; j2 < 8 && i + j2 < len; j2++)
+ printk(" %02x", written[start + i + j2]);
+ printk("\n");
+ i += 8;
+ }
+}
+
+/*
+ * Count the number of differing bytes and bits and return the first differing
+ * offset.
+ */
+static int countdiffs(unsigned char *buf, unsigned char *check_buf,
+ unsigned offset, unsigned len, unsigned *bytesp,
+ unsigned *bitsp)
+{
+ unsigned i, bit;
+ int first = -1;
+
+ for (i = offset; i < offset + len; i++)
+ if (buf[i] != check_buf[i]) {
+ first = i;
+ break;
+ }
+
+ while (i < offset + len) {
+ if (buf[i] != check_buf[i]) {
+ (*bytesp)++;
+ bit = 1;
+ while (bit < 256) {
+ if ((buf[i] & bit) != (check_buf[i] & bit))
+ (*bitsp)++;
+ bit <<= 1;
+ }
+ }
+ i++;
+ }
+
+ return first;
+}
+
+MODULE_DESCRIPTION("Eraseblock torturing module");
+MODULE_AUTHOR("Artem Bityutskiy, Jarkko Lavinen, Adrian Hunter");
+MODULE_LICENSE("GPL");
diff --git a/drivers/mtd/ubi/build.c b/drivers/mtd/ubi/build.c
index 7caf22c..9082768 100644
--- a/drivers/mtd/ubi/build.c
+++ b/drivers/mtd/ubi/build.c
@@ -561,7 +561,7 @@
*/
ubi->peb_size = ubi->mtd->erasesize;
- ubi->peb_count = ubi->mtd->size / ubi->mtd->erasesize;
+ ubi->peb_count = mtd_div_by_eb(ubi->mtd->size, ubi->mtd);
ubi->flash_size = ubi->mtd->size;
if (ubi->mtd->block_isbad && ubi->mtd->block_markbad)
diff --git a/drivers/mtd/ubi/gluebi.c b/drivers/mtd/ubi/gluebi.c
index 605812b..6dd4f5e 100644
--- a/drivers/mtd/ubi/gluebi.c
+++ b/drivers/mtd/ubi/gluebi.c
@@ -215,7 +215,8 @@
struct ubi_volume *vol;
struct ubi_device *ubi;
- dbg_gen("erase %u bytes at offset %u", instr->len, instr->addr);
+ dbg_gen("erase %llu bytes at offset %llu", (unsigned long long)instr->len,
+ (unsigned long long)instr->addr);
if (instr->addr < 0 || instr->addr > mtd->size - mtd->erasesize)
return -EINVAL;
@@ -223,11 +224,11 @@
if (instr->len < 0 || instr->addr + instr->len > mtd->size)
return -EINVAL;
- if (instr->addr % mtd->writesize || instr->len % mtd->writesize)
+ if (mtd_mod_by_ws(instr->addr, mtd) || mtd_mod_by_ws(instr->len, mtd))
return -EINVAL;
- lnum = instr->addr / mtd->erasesize;
- count = instr->len / mtd->erasesize;
+ lnum = mtd_div_by_eb(instr->addr, mtd);
+ count = mtd_div_by_eb(instr->len, mtd);
vol = container_of(mtd, struct ubi_volume, gluebi_mtd);
ubi = vol->ubi;
@@ -255,7 +256,7 @@
out_err:
instr->state = MTD_ERASE_FAILED;
- instr->fail_addr = lnum * mtd->erasesize;
+ instr->fail_addr = (long long)lnum * mtd->erasesize;
return err;
}
@@ -294,7 +295,7 @@
* bytes.
*/
if (vol->vol_type == UBI_DYNAMIC_VOLUME)
- mtd->size = vol->usable_leb_size * vol->reserved_pebs;
+ mtd->size = (long long)vol->usable_leb_size * vol->reserved_pebs;
else
mtd->size = vol->used_bytes;
@@ -304,8 +305,8 @@
return -ENFILE;
}
- dbg_gen("added mtd%d (\"%s\"), size %u, EB size %u",
- mtd->index, mtd->name, mtd->size, mtd->erasesize);
+ dbg_gen("added mtd%d (\"%s\"), size %llu, EB size %u",
+ mtd->index, mtd->name, (unsigned long long)mtd->size, mtd->erasesize);
return 0;
}
diff --git a/drivers/mtd/ubi/kapi.c b/drivers/mtd/ubi/kapi.c
index 5d9bcf1..4abbe57 100644
--- a/drivers/mtd/ubi/kapi.c
+++ b/drivers/mtd/ubi/kapi.c
@@ -564,7 +564,7 @@
* @dtype: expected data type
*
* This function maps an un-mapped logical eraseblock @lnum to a physical
- * eraseblock. This means, that after a successfull invocation of this
+ * eraseblock. This means, that after a successful invocation of this
* function the logical eraseblock @lnum will be empty (contain only %0xFF
* bytes) and be mapped to a physical eraseblock, even if an unclean reboot
* happens.
diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig
index 97ea7c6..65afda4 100644
--- a/drivers/net/Kconfig
+++ b/drivers/net/Kconfig
@@ -830,7 +830,7 @@
config BFIN_MAC
tristate "Blackfin on-chip MAC support"
- depends on NET_ETHERNET && (BF526 || BF527 || BF536 || BF537)
+ depends on NET_ETHERNET && (BF516 || BF518 || BF526 || BF527 || BF536 || BF537)
select CRC32
select MII
select PHYLIB
diff --git a/drivers/net/acenic_firmware.h b/drivers/net/acenic_firmware.h
deleted file mode 100644
index fd41f78..0000000
--- a/drivers/net/acenic_firmware.h
+++ /dev/null
@@ -1,9456 +0,0 @@
-/*
- * Declare these here even if Tigon I support is disabled to avoid
- * the compiler complaining about undefined symbols.
- */
-#define tigonFwReleaseMajor 0xc
-#define tigonFwReleaseMinor 0x4
-#define tigonFwReleaseFix 0xb
-#define tigonFwStartAddr 0x00004000
-#define tigonFwTextAddr 0x00004000
-#define tigonFwTextLen 0x11140
-#define tigonFwRodataAddr 0x00015140
-#define tigonFwRodataLen 0xac0
-#define tigonFwDataAddr 0x00015c20
-#define tigonFwDataLen 0x170
-#define tigonFwSbssAddr 0x00015d90
-#define tigonFwSbssLen 0x38
-#define tigonFwBssAddr 0x00015dd0
-#define tigonFwBssLen 0x2080
-#ifdef CONFIG_ACENIC_OMIT_TIGON_I
-#define tigonFwText NULL
-#define tigonFwData NULL
-#define tigonFwRodata NULL
-#else
-/* Generated by genfw.c */
-static u32 tigonFwText[(MAX_TEXT_LEN/4) + 1] __devinitdata = {
-0x10000003,
-0x0, 0xd, 0xd, 0x3c1d0001,
-0x8fbd5c54, 0x3a0f021, 0x3c100000, 0x26104000,
-0xc00100c, 0x0, 0xd, 0x27bdffd8,
-0x3c1cc000, 0x3c1b0013, 0x377bd800, 0xd021,
-0x3c170013, 0x36f75418, 0x2e02021, 0x340583e8,
-0xafbf0024, 0xc002488, 0xafb00020, 0xc0023e8,
-0x0, 0x3c040001, 0x248451a4, 0x24050001,
-0x2e03021, 0x3821, 0x3c100001, 0x26107e50,
-0xafb00010, 0xc002403, 0xafbb0014, 0x3c02000f,
-0x3442ffff, 0x2021024, 0x362102b, 0x10400009,
-0x24050003, 0x3c040001, 0x248451b0, 0x2003021,
-0x3603821, 0x3c020010, 0xafa20010, 0xc002403,
-0xafa00014, 0x2021, 0x3405c000, 0x3c010001,
-0x370821, 0xa02083b0, 0x3c010001, 0x370821,
-0xa02083b2, 0x3c010001, 0x370821, 0xa02083b3,
-0x3c010001, 0x370821, 0xac2083b4, 0xa2e004d8,
-0x418c0, 0x24840001, 0x771021, 0xac40727c,
-0x771021, 0xac407280, 0x2e31021, 0xa445727c,
-0x2c820020, 0x1440fff7, 0x418c0, 0x2021,
-0x3405c000, 0x418c0, 0x24840001, 0x771021,
-0xac40737c, 0x771021, 0xac407380, 0x2e31021,
-0xa445737c, 0x2c820080, 0x5440fff7, 0x418c0,
-0xaf800054, 0xaf80011c, 0x8f820044, 0x34420040,
-0xaf820044, 0x8f820044, 0x34420020, 0xaf820044,
-0x8f420218, 0x30420002, 0x10400009, 0x0,
-0x8f420220, 0x3c030002, 0x34630004, 0x431025,
-0xaee204c4, 0x8f42021c, 0x8001074, 0x34420004,
-0x8f420220, 0x3c030002, 0x34630006, 0x431025,
-0xaee204c4, 0x8f42021c, 0x34420006, 0xaee204cc,
-0x8f420218, 0x30420010, 0x1040000a, 0x0,
-0x8f42021c, 0x34420004, 0xaee204c8, 0x8f420220,
-0x3c03000a, 0x34630004, 0x431025, 0x800108a,
-0xaee204c0, 0x8f420220, 0x3c03000a, 0x34630006,
-0x431025, 0xaee204c0, 0x8f42021c, 0x34420006,
-0xaee204c8, 0x8f420218, 0x30420200, 0x10400003,
-0x24020001, 0x8001091, 0xa2e27248, 0xa2e07248,
-0x24020001, 0xaf8200a0, 0xaf8200b0, 0x8f830054,
-0x8f820054, 0x8001099, 0x24630064, 0x8f820054,
-0x621023, 0x2c420065, 0x1440fffc, 0x0,
-0xaf800044, 0x8f420208, 0x8f43020c, 0xaee20010,
-0xaee30014, 0x8ee40010, 0x8ee50014, 0x26e20030,
-0xaee20028, 0x24020490, 0xaee20018, 0xaf840090,
-0xaf850094, 0x8ee20028, 0xaf8200b4, 0x96e2001a,
-0xaf82009c, 0x8f8200b0, 0x8ee304cc, 0x431025,
-0xaf8200b0, 0x8f8200b0, 0x30420004, 0x1440fffd,
-0x0, 0x8ee20450, 0x8ee30454, 0xaee304fc,
-0x8ee204fc, 0x2442e000, 0x2c422001, 0x1440000d,
-0x26e40030, 0x8ee20450, 0x8ee30454, 0x3c040001,
-0x248451bc, 0x3c050001, 0xafa00010, 0xafa00014,
-0x8ee704fc, 0x34a5f000, 0xc002403, 0x603021,
-0x26e40030, 0xc002488, 0x24050400, 0x27440080,
-0xc002488, 0x24050080, 0x26e4777c, 0xc002488,
-0x24050400, 0x8f42025c, 0x26e40094, 0xaee20060,
-0x8f420260, 0x27450200, 0x24060008, 0xaee20068,
-0x24020006, 0xc00249a, 0xaee20064, 0x3c023b9a,
-0x3442ca00, 0x2021, 0x24030002, 0xaee30074,
-0xaee30070, 0xaee2006c, 0x240203e8, 0xaee20104,
-0x24020001, 0xaee30100, 0xaee2010c, 0x3c030001,
-0x641821, 0x90635c20, 0x2e41021, 0x24840001,
-0xa043009c, 0x2c82000f, 0x1440fff8, 0x0,
-0x8f820040, 0x2e41821, 0x24840001, 0x21702,
-0x24420030, 0xa062009c, 0x2e41021, 0xa040009c,
-0x96e2046a, 0x30420003, 0x14400009, 0x0,
-0x96e2047a, 0x30420003, 0x50400131, 0x3c030800,
-0x96e2046a, 0x30420003, 0x1040002a, 0x3c020700,
-0x96e2047a, 0x30420003, 0x10400026, 0x3c020700,
-0x96e3047a, 0x96e2046a, 0x14620022, 0x3c020700,
-0x8ee204c0, 0x24030001, 0xa2e34e20, 0x34420e00,
-0xaee204c0, 0x8f420218, 0x30420100, 0x10400005,
-0x0, 0x3c020001, 0x2442e168, 0x800111d,
-0x21100, 0x3c020001, 0x2442d35c, 0x21100,
-0x21182, 0x3c030800, 0x431025, 0x3c010001,
-0xac221238, 0x3c020001, 0x2442f680, 0x21100,
-0x21182, 0x3c030800, 0x431025, 0x3c010001,
-0xac221278, 0x8ee20000, 0x34424000, 0x8001238,
-0xaee20000, 0x34423000, 0xafa20018, 0x8ee20608,
-0x8f430228, 0x24420001, 0x304900ff, 0x512300e2,
-0xafa00010, 0x8ee20608, 0x210c0, 0x571021,
-0x8fa30018, 0x8fa4001c, 0xac43060c, 0xac440610,
-0x8f870120, 0x27623800, 0x24e80020, 0x102102b,
-0x50400001, 0x27683000, 0x8f820128, 0x11020004,
-0x0, 0x8f820124, 0x15020007, 0x1021,
-0x8ee201a4, 0x3021, 0x24420001, 0xaee201a4,
-0x80011a0, 0x8ee201a4, 0x8ee40608, 0x420c0,
-0x801821, 0x8ee40430, 0x8ee50434, 0xa32821,
-0xa3302b, 0x822021, 0x862021, 0xace40000,
-0xace50004, 0x8ee30608, 0x24020008, 0xa4e2000e,
-0x2402000d, 0xace20018, 0xace9001c, 0x318c0,
-0x2463060c, 0x2e31021, 0xace20008, 0x8ee204c4,
-0xace20010, 0xaf880120, 0x92e24e20, 0x14400037,
-0x24060001, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0x8c830000, 0x24020007, 0x1462001f,
-0x0, 0x8ee34e30, 0x8ee24e34, 0x1062001b,
-0x24030040, 0x8c820004, 0x24420001, 0xac820004,
-0x8ee24e34, 0x8ee54e30, 0x24420001, 0x10430007,
-0x0, 0x8ee24e34, 0x24420001, 0x10a20005,
-0x0, 0x800118a, 0x0, 0x14a00005,
-0x0, 0x8f820128, 0x24420020, 0xaf820128,
-0x8f820128, 0x8c820004, 0x2c420011, 0x50400013,
-0xac800000, 0x80011a0, 0x0, 0x8ee24e30,
-0x24030040, 0x24420001, 0x50430003, 0x1021,
-0x8ee24e30, 0x24420001, 0xaee24e30, 0x8ee24e30,
-0x210c0, 0x24425038, 0x2e22021, 0x24020007,
-0xac820000, 0x24020001, 0xac820004, 0x54c0000c,
-0xaee90608, 0x3c040001, 0x248451c8, 0xafa00010,
-0xafa00014, 0x8ee60608, 0x8f470228, 0x3c050009,
-0xc002403, 0x34a5f000, 0x8001223, 0x0,
-0x8f830120, 0x27623800, 0x24660020, 0xc2102b,
-0x50400001, 0x27663000, 0x8f820128, 0x10c20004,
-0x0, 0x8f820124, 0x14c20007, 0x0,
-0x8ee201a4, 0x3021, 0x24420001, 0xaee201a4,
-0x8001207, 0x8ee201a4, 0x8ee20608, 0xac62001c,
-0x8ee404a0, 0x8ee504a4, 0x2462001c, 0xac620008,
-0x24020008, 0xa462000e, 0x24020011, 0xac620018,
-0xac640000, 0xac650004, 0x8ee204c4, 0xac620010,
-0xaf860120, 0x92e24e20, 0x14400037, 0x24060001,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x8c830000, 0x24020012, 0x1462001f, 0x0,
-0x8ee34e30, 0x8ee24e34, 0x1062001b, 0x24030040,
-0x8c820004, 0x24420001, 0xac820004, 0x8ee24e34,
-0x8ee54e30, 0x24420001, 0x10430007, 0x0,
-0x8ee24e34, 0x24420001, 0x10a20005, 0x0,
-0x80011f1, 0x0, 0x14a00005, 0x0,
-0x8f820128, 0x24420020, 0xaf820128, 0x8f820128,
-0x8c820004, 0x2c420011, 0x50400013, 0xac800000,
-0x8001207, 0x0, 0x8ee24e30, 0x24030040,
-0x24420001, 0x50430003, 0x1021, 0x8ee24e30,
-0x24420001, 0xaee24e30, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x24020012, 0xac820000,
-0x24020001, 0xac820004, 0x14c0001b, 0x0,
-0x3c040001, 0x248451d0, 0xafa00010, 0xafa00014,
-0x8ee60608, 0x8f470228, 0x3c050009, 0xc002403,
-0x34a5f001, 0x8ee201b0, 0x24420001, 0xaee201b0,
-0x8001223, 0x8ee201b0, 0x3c040001, 0x248451dc,
-0xafa00014, 0x8ee60608, 0x8f470228, 0x3c050009,
-0xc002403, 0x34a5f005, 0x8ee201ac, 0x24420001,
-0xaee201ac, 0x8ee201ac, 0x8ee20160, 0x3c040001,
-0x248451e8, 0x3405f001, 0x24420001, 0xaee20160,
-0x8ee20160, 0x3021, 0x3821, 0xafa00010,
-0xc002403, 0xafa00014, 0x8001238, 0x0,
-0x3c020001, 0x2442f5a8, 0x21100, 0x21182,
-0x431025, 0x3c010001, 0xac221278, 0x96e2045a,
-0x30420003, 0x10400025, 0x3c050fff, 0x8ee204c8,
-0x34a5ffff, 0x34420a00, 0xaee204c8, 0x8ee304c8,
-0x3c040001, 0x248451f4, 0x24020001, 0xa2e204ec,
-0xa2e204ed, 0x3c020002, 0x621825, 0x3c020001,
-0x2442a390, 0x451024, 0x21082, 0xaee304c8,
-0x3c030800, 0x431025, 0x3c010001, 0xac221220,
-0x3c020001, 0x2442add4, 0x451024, 0x21082,
-0x431025, 0x3c010001, 0xac221280, 0x96e6045a,
-0x3821, 0x24050011, 0xafa00010, 0xc002403,
-0xafa00014, 0x8001268, 0x0, 0x3c020001,
-0x2442a9d4, 0x21100, 0x21182, 0x3c030800,
-0x431025, 0x3c010001, 0xac221280, 0x96e2046a,
-0x30420010, 0x14400009, 0x0, 0x96e2047a,
-0x30420010, 0x10400112, 0x0, 0x96e2046a,
-0x30420010, 0x10400005, 0x3c020700, 0x96e2047a,
-0x30420010, 0x14400102, 0x3c020700, 0x34423000,
-0xafa20018, 0x8ee20608, 0x8f430228, 0x24420001,
-0x304900ff, 0x512300e2, 0xafa00010, 0x8ee20608,
-0x210c0, 0x571021, 0x8fa30018, 0x8fa4001c,
-0xac43060c, 0xac440610, 0x8f870120, 0x27623800,
-0x24e80020, 0x102102b, 0x50400001, 0x27683000,
-0x8f820128, 0x11020004, 0x0, 0x8f820124,
-0x15020007, 0x1021, 0x8ee201a4, 0x3021,
-0x24420001, 0xaee201a4, 0x80012ea, 0x8ee201a4,
-0x8ee40608, 0x420c0, 0x801821, 0x8ee40430,
-0x8ee50434, 0xa32821, 0xa3302b, 0x822021,
-0x862021, 0xace40000, 0xace50004, 0x8ee30608,
-0x24020008, 0xa4e2000e, 0x2402000d, 0xace20018,
-0xace9001c, 0x318c0, 0x2463060c, 0x2e31021,
-0xace20008, 0x8ee204c4, 0xace20010, 0xaf880120,
-0x92e24e20, 0x14400037, 0x24060001, 0x8ee24e30,
-0x210c0, 0x24425038, 0x2e22021, 0x8c830000,
-0x24020007, 0x1462001f, 0x0, 0x8ee34e30,
-0x8ee24e34, 0x1062001b, 0x24030040, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee24e34, 0x8ee54e30,
-0x24420001, 0x10430007, 0x0, 0x8ee24e34,
-0x24420001, 0x10a20005, 0x0, 0x80012d4,
-0x0, 0x14a00005, 0x0, 0x8f820128,
-0x24420020, 0xaf820128, 0x8f820128, 0x8c820004,
-0x2c420011, 0x50400013, 0xac800000, 0x80012ea,
-0x0, 0x8ee24e30, 0x24030040, 0x24420001,
-0x50430003, 0x1021, 0x8ee24e30, 0x24420001,
-0xaee24e30, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0x24020007, 0xac820000, 0x24020001,
-0xac820004, 0x54c0000c, 0xaee90608, 0x3c040001,
-0x248451c8, 0xafa00010, 0xafa00014, 0x8ee60608,
-0x8f470228, 0x3c050009, 0xc002403, 0x34a5f000,
-0x800136d, 0x0, 0x8f830120, 0x27623800,
-0x24660020, 0xc2102b, 0x50400001, 0x27663000,
-0x8f820128, 0x10c20004, 0x0, 0x8f820124,
-0x14c20007, 0x0, 0x8ee201a4, 0x3021,
-0x24420001, 0xaee201a4, 0x8001351, 0x8ee201a4,
-0x8ee20608, 0xac62001c, 0x8ee404a0, 0x8ee504a4,
-0x2462001c, 0xac620008, 0x24020008, 0xa462000e,
-0x24020011, 0xac620018, 0xac640000, 0xac650004,
-0x8ee204c4, 0xac620010, 0xaf860120, 0x92e24e20,
-0x14400037, 0x24060001, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x8c830000, 0x24020012,
-0x1462001f, 0x0, 0x8ee34e30, 0x8ee24e34,
-0x1062001b, 0x24030040, 0x8c820004, 0x24420001,
-0xac820004, 0x8ee24e34, 0x8ee54e30, 0x24420001,
-0x10430007, 0x0, 0x8ee24e34, 0x24420001,
-0x10a20005, 0x0, 0x800133b, 0x0,
-0x14a00005, 0x0, 0x8f820128, 0x24420020,
-0xaf820128, 0x8f820128, 0x8c820004, 0x2c420011,
-0x50400013, 0xac800000, 0x8001351, 0x0,
-0x8ee24e30, 0x24030040, 0x24420001, 0x50430003,
-0x1021, 0x8ee24e30, 0x24420001, 0xaee24e30,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x24020012, 0xac820000, 0x24020001, 0xac820004,
-0x14c0001b, 0x0, 0x3c040001, 0x248451d0,
-0xafa00010, 0xafa00014, 0x8ee60608, 0x8f470228,
-0x3c050009, 0xc002403, 0x34a5f001, 0x8ee201b0,
-0x24420001, 0xaee201b0, 0x800136d, 0x8ee201b0,
-0x3c040001, 0x248451dc, 0xafa00014, 0x8ee60608,
-0x8f470228, 0x3c050009, 0xc002403, 0x34a5f005,
-0x8ee201ac, 0x24420001, 0xaee201ac, 0x8ee201ac,
-0x8ee20160, 0x3c040001, 0x248451e8, 0x3405f002,
-0x24420001, 0xaee20160, 0x8ee20160, 0x3021,
-0x3821, 0xafa00010, 0xc002403, 0xafa00014,
-0x96e6047a, 0x96e7046a, 0x3c040001, 0x24845200,
-0x24050012, 0xafa00010, 0xc002403, 0xafa00014,
-0xc004500, 0x0, 0xc002318, 0x0,
-0x3c060001, 0x34c63800, 0xaee00608, 0xaf400228,
-0xaf40022c, 0x96e30458, 0x8ee40000, 0x3c0512d8,
-0x34a5c358, 0x27623800, 0xaee27258, 0x27623800,
-0xaee27260, 0x27623800, 0xaee27264, 0x3661021,
-0xaee27270, 0x2402ffff, 0xaee004d4, 0xaee004e0,
-0xaee004e4, 0xaee004f0, 0xa2e004f4, 0xaee00e0c,
-0xaee00e18, 0xaee00e10, 0xaee00e14, 0xaee00e1c,
-0xaee0724c, 0xaee05244, 0xaee05240, 0xaee0523c,
-0xaee07250, 0xaee07254, 0xaee0725c, 0xaee07268,
-0xaee004d0, 0x2463ffff, 0x852025, 0xaee304f8,
-0xaee40000, 0xaf800060, 0xaf820064, 0x3c020100,
-0xafa20018, 0x8ee20608, 0x8f430228, 0x24420001,
-0x304900ff, 0x512300e2, 0xafa00010, 0x8ee20608,
-0x210c0, 0x571021, 0x8fa30018, 0x8fa4001c,
-0xac43060c, 0xac440610, 0x8f870120, 0x27623800,
-0x24e80020, 0x102102b, 0x50400001, 0x27683000,
-0x8f820128, 0x11020004, 0x0, 0x8f820124,
-0x15020007, 0x1021, 0x8ee201a4, 0x3021,
-0x24420001, 0xaee201a4, 0x8001422, 0x8ee201a4,
-0x8ee40608, 0x420c0, 0x801821, 0x8ee40430,
-0x8ee50434, 0xa32821, 0xa3302b, 0x822021,
-0x862021, 0xace40000, 0xace50004, 0x8ee30608,
-0x24020008, 0xa4e2000e, 0x2402000d, 0xace20018,
-0xace9001c, 0x318c0, 0x2463060c, 0x2e31021,
-0xace20008, 0x8ee204c4, 0xace20010, 0xaf880120,
-0x92e24e20, 0x14400037, 0x24060001, 0x8ee24e30,
-0x210c0, 0x24425038, 0x2e22021, 0x8c830000,
-0x24020007, 0x1462001f, 0x0, 0x8ee34e30,
-0x8ee24e34, 0x1062001b, 0x24030040, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee24e34, 0x8ee54e30,
-0x24420001, 0x10430007, 0x0, 0x8ee24e34,
-0x24420001, 0x10a20005, 0x0, 0x800140c,
-0x0, 0x14a00005, 0x0, 0x8f820128,
-0x24420020, 0xaf820128, 0x8f820128, 0x8c820004,
-0x2c420011, 0x50400013, 0xac800000, 0x8001422,
-0x0, 0x8ee24e30, 0x24030040, 0x24420001,
-0x50430003, 0x1021, 0x8ee24e30, 0x24420001,
-0xaee24e30, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0x24020007, 0xac820000, 0x24020001,
-0xac820004, 0x54c0000c, 0xaee90608, 0x3c040001,
-0x248451c8, 0xafa00010, 0xafa00014, 0x8ee60608,
-0x8f470228, 0x3c050009, 0xc002403, 0x34a5f000,
-0x80014a5, 0x0, 0x8f830120, 0x27623800,
-0x24660020, 0xc2102b, 0x50400001, 0x27663000,
-0x8f820128, 0x10c20004, 0x0, 0x8f820124,
-0x14c20007, 0x0, 0x8ee201a4, 0x3021,
-0x24420001, 0xaee201a4, 0x8001489, 0x8ee201a4,
-0x8ee20608, 0xac62001c, 0x8ee404a0, 0x8ee504a4,
-0x2462001c, 0xac620008, 0x24020008, 0xa462000e,
-0x24020011, 0xac620018, 0xac640000, 0xac650004,
-0x8ee204c4, 0xac620010, 0xaf860120, 0x92e24e20,
-0x14400037, 0x24060001, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x8c830000, 0x24020012,
-0x1462001f, 0x0, 0x8ee34e30, 0x8ee24e34,
-0x1062001b, 0x24030040, 0x8c820004, 0x24420001,
-0xac820004, 0x8ee24e34, 0x8ee54e30, 0x24420001,
-0x10430007, 0x0, 0x8ee24e34, 0x24420001,
-0x10a20005, 0x0, 0x8001473, 0x0,
-0x14a00005, 0x0, 0x8f820128, 0x24420020,
-0xaf820128, 0x8f820128, 0x8c820004, 0x2c420011,
-0x50400013, 0xac800000, 0x8001489, 0x0,
-0x8ee24e30, 0x24030040, 0x24420001, 0x50430003,
-0x1021, 0x8ee24e30, 0x24420001, 0xaee24e30,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x24020012, 0xac820000, 0x24020001, 0xac820004,
-0x14c0001b, 0x0, 0x3c040001, 0x248451d0,
-0xafa00010, 0xafa00014, 0x8ee60608, 0x8f470228,
-0x3c050009, 0xc002403, 0x34a5f001, 0x8ee201b0,
-0x24420001, 0xaee201b0, 0x80014a5, 0x8ee201b0,
-0x3c040001, 0x248451dc, 0xafa00014, 0x8ee60608,
-0x8f470228, 0x3c050009, 0xc002403, 0x34a5f005,
-0x8ee201ac, 0x24420001, 0xaee201ac, 0x8ee201ac,
-0x8ee20154, 0x24420001, 0xaee20154, 0xc0014dc,
-0x8ee20154, 0x8f8200a0, 0x30420004, 0x1440fffd,
-0x0, 0x8f820040, 0x30420001, 0x14400008,
-0x0, 0x8f430104, 0x24020001, 0x10620004,
-0x0, 0x8f420264, 0x10400006, 0x0,
-0x8ee2017c, 0x24420001, 0xaee2017c, 0x80014c5,
-0x8ee2017c, 0x8f820044, 0x34420004, 0xaf820044,
-0x8ee20178, 0x24420001, 0xaee20178, 0x8ee20178,
-0x8f8200d8, 0x8f8300d4, 0x431023, 0xaee2726c,
-0x8ee2726c, 0x1c400003, 0x3c030001, 0x431021,
-0xaee2726c, 0xc004064, 0x0, 0xc004440,
-0xaf800228, 0x8fbf0024, 0x8fb00020, 0x3e00008,
-0x27bd0028, 0x3e00008, 0x0, 0x3e00008,
-0x0, 0x0, 0x0, 0x2402002c,
-0xaf820050, 0xaee07274, 0x8f420238, 0xaee27278,
-0x8f820054, 0x24420067, 0xaf820058, 0xaee07b88,
-0xaee07b8c, 0xaee07b84, 0x3c010001, 0x370821,
-0xac2083bc, 0x3c010001, 0x370821, 0x3e00008,
-0xa02083b9, 0x27bdffd8, 0xafbf0024, 0xafb00020,
-0x8f820054, 0x3c030001, 0x8c635cd8, 0x24420067,
-0x1060000d, 0xaf820058, 0x3c020001, 0x571021,
-0x904283b8, 0x10400005, 0x3c030200, 0x3c010001,
-0x370821, 0x8001503, 0xa02083b8, 0x8ee20000,
-0x431025, 0xaee20000, 0x8f420218, 0x30420100,
-0x104000c6, 0x0, 0x8f8200b0, 0x30420004,
-0x104000c2, 0x0, 0x3c030001, 0x771821,
-0x8c6383d0, 0x8f820104, 0x146200b4, 0x0,
-0x3c030001, 0x771821, 0x8c6383d4, 0x8f8200b4,
-0x146200ae, 0x0, 0x8f8200b0, 0x3c030080,
-0x431024, 0x1040000d, 0x0, 0x8f82011c,
-0x34420002, 0xaf82011c, 0x8f8200b0, 0x2403fffb,
-0x431024, 0xaf8200b0, 0x8f82011c, 0x2403fffd,
-0x431024, 0x80015cc, 0xaf82011c, 0x3c030001,
-0x771821, 0x8c6383d0, 0x8f820104, 0x14620082,
-0x0, 0x3c030001, 0x771821, 0x8c6383d4,
-0x8f8200b4, 0x1462007c, 0x0, 0x3c070001,
-0xf73821, 0x8ce783d0, 0x8f8200b0, 0x3c040001,
-0x24845270, 0xafa00014, 0xafa20010, 0x8f8600b0,
-0x3c050005, 0xc002403, 0x34a50900, 0x8f82011c,
-0x34420002, 0xaf82011c, 0x8f830104, 0x8f8200b0,
-0x34420001, 0xaf8200b0, 0xaf830104, 0x8f830120,
-0x27623800, 0x24660020, 0xc2102b, 0x50400001,
-0x27663000, 0x8f820128, 0x10c20004, 0x0,
-0x8f820124, 0x14c20006, 0x0, 0x8ee201a4,
-0x24420001, 0xaee201a4, 0x80015a0, 0x8ee201a4,
-0x8f440208, 0x8f45020c, 0x26e20030, 0xac620008,
-0x24020400, 0xa462000e, 0x2402000f, 0xac620018,
-0xac60001c, 0xac640000, 0xac650004, 0x8ee204c4,
-0xac620010, 0xaf860120, 0x92e24e20, 0x14400037,
-0x0, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0x8c830000, 0x24020007, 0x1462001f,
-0x0, 0x8ee34e30, 0x8ee24e34, 0x1062001b,
-0x24030040, 0x8c820004, 0x24420001, 0xac820004,
-0x8ee24e34, 0x8ee54e30, 0x24420001, 0x10430007,
-0x0, 0x8ee24e34, 0x24420001, 0x10a20005,
-0x0, 0x800158a, 0x0, 0x14a00005,
-0x0, 0x8f820128, 0x24420020, 0xaf820128,
-0x8f820128, 0x8c820004, 0x2c420011, 0x50400013,
-0xac800000, 0x80015a0, 0x0, 0x8ee24e30,
-0x24030040, 0x24420001, 0x50430003, 0x1021,
-0x8ee24e30, 0x24420001, 0xaee24e30, 0x8ee24e30,
-0x210c0, 0x24425038, 0x2e22021, 0x24020007,
-0xac820000, 0x24020001, 0xac820004, 0x8f82011c,
-0x2403fffd, 0x431024, 0xaf82011c, 0x8ee201e4,
-0x3c070001, 0xf73821, 0x8ce783d0, 0x24420001,
-0xaee201e4, 0x8ee201e4, 0x3c040001, 0x2484527c,
-0x80015bd, 0xafa00010, 0x8f820104, 0x3c010001,
-0x370821, 0xac2283d0, 0x8f8200b4, 0x3c070001,
-0xf73821, 0x8ce783d0, 0x3c040001, 0x24845284,
-0x3c010001, 0x370821, 0xac2283d4, 0xafa00010,
-0xafa00014, 0x8f8600b0, 0x3c050005, 0xc002403,
-0x34a50900, 0x80015cc, 0x0, 0x8f820104,
-0x3c010001, 0x370821, 0xac2283d0, 0x8f8200b4,
-0x3c010001, 0x370821, 0xac2283d4, 0x8ee27274,
-0x92e304f4, 0x24420067, 0x14600006, 0xaee27274,
-0x8ee27274, 0x8f430234, 0x43102b, 0x1440007b,
-0x0, 0x8ee304e4, 0x8ee204f8, 0x14620004,
-0x0, 0x92e204f4, 0x50400074, 0xa2e004f4,
-0x8f830120, 0x27623800, 0x24660020, 0xc2102b,
-0x50400001, 0x27663000, 0x8f820128, 0x10c20004,
-0x0, 0x8f820124, 0x14c20007, 0x0,
-0x8ee201a4, 0x8021, 0x24420001, 0xaee201a4,
-0x8001637, 0x8ee201a4, 0x8ee204e4, 0xac62001c,
-0x8ee404b0, 0x8ee504b4, 0x2462001c, 0xac620008,
-0x24020008, 0xa462000e, 0x24020011, 0xac620018,
-0xac640000, 0xac650004, 0x8ee204c4, 0xac620010,
-0xaf860120, 0x92e24e20, 0x14400037, 0x24100001,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x8c830000, 0x24020012, 0x1462001f, 0x0,
-0x8ee34e30, 0x8ee24e34, 0x1062001b, 0x24030040,
-0x8c820004, 0x24420001, 0xac820004, 0x8ee24e34,
-0x8ee54e30, 0x24420001, 0x10430007, 0x0,
-0x8ee24e34, 0x24420001, 0x10a20005, 0x0,
-0x8001621, 0x0, 0x14a00005, 0x0,
-0x8f820128, 0x24420020, 0xaf820128, 0x8f820128,
-0x8c820004, 0x2c420011, 0x50400013, 0xac800000,
-0x8001637, 0x0, 0x8ee24e30, 0x24030040,
-0x24420001, 0x50430003, 0x1021, 0x8ee24e30,
-0x24420001, 0xaee24e30, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x24020012, 0xac820000,
-0x24020001, 0xac820004, 0x5600000b, 0x24100001,
-0x8ee204e4, 0x3c040001, 0x2484528c, 0xafa00014,
-0xafa20010, 0x8ee60608, 0x8f470228, 0x3c050009,
-0xc002403, 0x34a5f006, 0x16000003, 0x24020001,
-0x8001650, 0xa2e204f4, 0x8ee20170, 0x24420001,
-0xaee20170, 0x8ee20170, 0x8ee204e4, 0xa2e004f4,
-0xaee004f0, 0xaee07274, 0xaee204f8, 0x8ee20e1c,
-0x1040006d, 0x0, 0x8f830120, 0x27623800,
-0x24660020, 0xc2102b, 0x50400001, 0x27663000,
-0x8f820128, 0x10c20004, 0x0, 0x8f820124,
-0x14c20007, 0x0, 0x8ee201a4, 0x8021,
-0x24420001, 0xaee201a4, 0x80016ad, 0x8ee201a4,
-0x8ee2724c, 0xac62001c, 0x8ee404a8, 0x8ee504ac,
-0x2462001c, 0xac620008, 0x24020008, 0xa462000e,
-0x24020011, 0xac620018, 0xac640000, 0xac650004,
-0x8ee204c4, 0xac620010, 0xaf860120, 0x92e24e20,
-0x14400037, 0x24100001, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x8c830000, 0x24020012,
-0x1462001f, 0x0, 0x8ee34e30, 0x8ee24e34,
-0x1062001b, 0x24030040, 0x8c820004, 0x24420001,
-0xac820004, 0x8ee24e34, 0x8ee54e30, 0x24420001,
-0x10430007, 0x0, 0x8ee24e34, 0x24420001,
-0x10a20005, 0x0, 0x8001697, 0x0,
-0x14a00005, 0x0, 0x8f820128, 0x24420020,
-0xaf820128, 0x8f820128, 0x8c820004, 0x2c420011,
-0x50400013, 0xac800000, 0x80016ad, 0x0,
-0x8ee24e30, 0x24030040, 0x24420001, 0x50430003,
-0x1021, 0x8ee24e30, 0x24420001, 0xaee24e30,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x24020012, 0xac820000, 0x24020001, 0xac820004,
-0x5600000b, 0x24100001, 0x8ee2724c, 0x3c040001,
-0x24845298, 0xafa00014, 0xafa20010, 0x8ee6724c,
-0x8f470280, 0x3c050009, 0xc002403, 0x34a5f008,
-0x56000001, 0xaee00e1c, 0x8ee20174, 0x24420001,
-0xaee20174, 0x8ee20174, 0x8ee24e24, 0x10400019,
-0x0, 0xaee04e24, 0x8f820040, 0x30420001,
-0x14400008, 0x0, 0x8f430104, 0x24020001,
-0x10620004, 0x0, 0x8f420264, 0x10400006,
-0x0, 0x8ee2017c, 0x24420001, 0xaee2017c,
-0x80016da, 0x8ee2017c, 0x8f820044, 0x34420004,
-0xaf820044, 0x8ee20178, 0x24420001, 0xaee20178,
-0x8ee20178, 0x8ee27278, 0x2442ff99, 0xaee27278,
-0x8ee27278, 0x1c4002ad, 0x0, 0x8f420238,
-0x104002aa, 0x0, 0x3c020001, 0x571021,
-0x904283e0, 0x144002a5, 0x0, 0x8f420080,
-0xaee2004c, 0x8f4200c0, 0xaee20048, 0x8f420084,
-0xaee20038, 0x8f420084, 0xaee20244, 0x8f420088,
-0xaee20248, 0x8f42008c, 0xaee2024c, 0x8f420090,
-0xaee20250, 0x8f420094, 0xaee20254, 0x8f420098,
-0xaee20258, 0x8f42009c, 0xaee2025c, 0x8f4200a0,
-0xaee20260, 0x8f4200a4, 0xaee20264, 0x8f4200a8,
-0xaee20268, 0x8f4200ac, 0xaee2026c, 0x8f4200b0,
-0xaee20270, 0x8f4200b4, 0xaee20274, 0x8f4200b8,
-0xaee20278, 0x8f4200bc, 0x24040001, 0xaee2027c,
-0xaee0003c, 0x41080, 0x571021, 0x8ee3003c,
-0x8c420244, 0x24840001, 0x621821, 0x2c82000f,
-0xaee3003c, 0x1440fff8, 0x41080, 0x8f4200cc,
-0xaee20050, 0x8f4200d0, 0xaee20054, 0x8f830120,
-0x27623800, 0x24660020, 0xc2102b, 0x50400001,
-0x27663000, 0x8f820128, 0x10c20004, 0x0,
-0x8f820124, 0x14c20007, 0x0, 0x8ee201a4,
-0x8021, 0x24420001, 0xaee201a4, 0x8001775,
-0x8ee201a4, 0x8f440208, 0x8f45020c, 0x26e20030,
-0xac620008, 0x24020400, 0xa462000e, 0x2402000f,
-0xac620018, 0xac60001c, 0xac640000, 0xac650004,
-0x8ee204c4, 0xac620010, 0xaf860120, 0x92e24e20,
-0x14400037, 0x24100001, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x8c830000, 0x24020007,
-0x1462001f, 0x0, 0x8ee34e30, 0x8ee24e34,
-0x1062001b, 0x24030040, 0x8c820004, 0x24420001,
-0xac820004, 0x8ee24e34, 0x8ee54e30, 0x24420001,
-0x10430007, 0x0, 0x8ee24e34, 0x24420001,
-0x10a20005, 0x0, 0x800175f, 0x0,
-0x14a00005, 0x0, 0x8f820128, 0x24420020,
-0xaf820128, 0x8f820128, 0x8c820004, 0x2c420011,
-0x50400013, 0xac800000, 0x8001775, 0x0,
-0x8ee24e30, 0x24030040, 0x24420001, 0x50430003,
-0x1021, 0x8ee24e30, 0x24420001, 0xaee24e30,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x24020007, 0xac820000, 0x24020001, 0xac820004,
-0x12000212, 0x3c020400, 0xafa20018, 0x3c020001,
-0x571021, 0x904283b0, 0x1040010b, 0x0,
-0x8ee20608, 0x8f430228, 0x24420001, 0x304a00ff,
-0x514300fd, 0xafa00010, 0x8ee20608, 0x210c0,
-0x571021, 0x8fa30018, 0x8fa4001c, 0xac43060c,
-0xac440610, 0x8f830054, 0x8f820054, 0x24690032,
-0x1221023, 0x2c420033, 0x1040006a, 0x5821,
-0x24180008, 0x240f000d, 0x240d0007, 0x240c0040,
-0x240e0001, 0x8f870120, 0x27623800, 0x24e80020,
-0x102102b, 0x50400001, 0x27683000, 0x8f820128,
-0x11020004, 0x0, 0x8f820124, 0x15020007,
-0x1021, 0x8ee201a4, 0x8021, 0x24420001,
-0xaee201a4, 0x80017f3, 0x8ee201a4, 0x8ee40608,
-0x420c0, 0x801821, 0x8ee40430, 0x8ee50434,
-0xa32821, 0xa3302b, 0x822021, 0x862021,
-0xace40000, 0xace50004, 0x8ee20608, 0xa4f8000e,
-0xacef0018, 0xacea001c, 0x210c0, 0x2442060c,
-0x2e21021, 0xace20008, 0x8ee204c4, 0xace20010,
-0xaf880120, 0x92e24e20, 0x14400033, 0x24100001,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x8c820000, 0x144d001f, 0x0, 0x8ee34e30,
-0x8ee24e34, 0x1062001b, 0x0, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee24e34, 0x8ee34e30,
-0x24420001, 0x104c0007, 0x0, 0x8ee24e34,
-0x24420001, 0x10620005, 0x0, 0x80017e0,
-0x0, 0x14600005, 0x0, 0x8f820128,
-0x24420020, 0xaf820128, 0x8f820128, 0x8c820004,
-0x2c420011, 0x50400010, 0xac800000, 0x80017f3,
-0x0, 0x8ee24e30, 0x24420001, 0x504c0003,
-0x1021, 0x8ee24e30, 0x24420001, 0xaee24e30,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0xac8d0000, 0xac8e0004, 0x56000006, 0x240b0001,
-0x8f820054, 0x1221023, 0x2c420033, 0x1440ff9d,
-0x0, 0x316300ff, 0x24020001, 0x14620077,
-0x3c050009, 0xaeea0608, 0x8f830054, 0x8f820054,
-0x24690032, 0x1221023, 0x2c420033, 0x10400061,
-0x5821, 0x240d0008, 0x240c0011, 0x24080012,
-0x24070040, 0x240a0001, 0x8f830120, 0x27623800,
-0x24660020, 0xc2102b, 0x50400001, 0x27663000,
-0x8f820128, 0x10c20004, 0x0, 0x8f820124,
-0x14c20007, 0x0, 0x8ee201a4, 0x8021,
-0x24420001, 0xaee201a4, 0x800185f, 0x8ee201a4,
-0x8ee20608, 0xac62001c, 0x8ee404a0, 0x8ee504a4,
-0x2462001c, 0xac620008, 0xa46d000e, 0xac6c0018,
-0xac640000, 0xac650004, 0x8ee204c4, 0xac620010,
-0xaf860120, 0x92e24e20, 0x14400033, 0x24100001,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x8c820000, 0x1448001f, 0x0, 0x8ee34e30,
-0x8ee24e34, 0x1062001b, 0x0, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee24e34, 0x8ee34e30,
-0x24420001, 0x10470007, 0x0, 0x8ee24e34,
-0x24420001, 0x10620005, 0x0, 0x800184c,
-0x0, 0x14600005, 0x0, 0x8f820128,
-0x24420020, 0xaf820128, 0x8f820128, 0x8c820004,
-0x2c420011, 0x50400010, 0xac800000, 0x800185f,
-0x0, 0x8ee24e30, 0x24420001, 0x50470003,
-0x1021, 0x8ee24e30, 0x24420001, 0xaee24e30,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0xac880000, 0xac8a0004, 0x56000006, 0x240b0001,
-0x8f820054, 0x1221023, 0x2c420033, 0x1440ffa6,
-0x0, 0x316300ff, 0x24020001, 0x14620003,
-0x3c050009, 0x800197c, 0x24100001, 0x3c040001,
-0x248452a4, 0xafa00010, 0xafa00014, 0x8f860120,
-0x8f870124, 0x800187b, 0x34a5f011, 0x3c040001,
-0x248452b0, 0xafa00010, 0xafa00014, 0x8f860120,
-0x8f870124, 0x34a5f010, 0xc002403, 0x8021,
-0x800197c, 0x0, 0x3c040001, 0x248452bc,
-0xafa00014, 0x8ee60608, 0x8f470228, 0x3c050009,
-0x8001975, 0x34a5f00f, 0x8ee20608, 0x8f430228,
-0x24420001, 0x304900ff, 0x512300e2, 0xafa00010,
-0x8ee20608, 0x210c0, 0x571021, 0x8fa30018,
-0x8fa4001c, 0xac43060c, 0xac440610, 0x8f870120,
-0x27623800, 0x24e80020, 0x102102b, 0x50400001,
-0x27683000, 0x8f820128, 0x11020004, 0x0,
-0x8f820124, 0x15020007, 0x1021, 0x8ee201a4,
-0x8021, 0x24420001, 0xaee201a4, 0x80018f7,
-0x8ee201a4, 0x8ee40608, 0x420c0, 0x801821,
-0x8ee40430, 0x8ee50434, 0xa32821, 0xa3302b,
-0x822021, 0x862021, 0xace40000, 0xace50004,
-0x8ee30608, 0x24020008, 0xa4e2000e, 0x2402000d,
-0xace20018, 0xace9001c, 0x318c0, 0x2463060c,
-0x2e31021, 0xace20008, 0x8ee204c4, 0xace20010,
-0xaf880120, 0x92e24e20, 0x14400037, 0x24100001,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x8c830000, 0x24020007, 0x1462001f, 0x0,
-0x8ee34e30, 0x8ee24e34, 0x1062001b, 0x24030040,
-0x8c820004, 0x24420001, 0xac820004, 0x8ee24e34,
-0x8ee54e30, 0x24420001, 0x10430007, 0x0,
-0x8ee24e34, 0x24420001, 0x10a20005, 0x0,
-0x80018e1, 0x0, 0x14a00005, 0x0,
-0x8f820128, 0x24420020, 0xaf820128, 0x8f820128,
-0x8c820004, 0x2c420011, 0x50400013, 0xac800000,
-0x80018f7, 0x0, 0x8ee24e30, 0x24030040,
-0x24420001, 0x50430003, 0x1021, 0x8ee24e30,
-0x24420001, 0xaee24e30, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x24020007, 0xac820000,
-0x24020001, 0xac820004, 0x5600000c, 0xaee90608,
-0x3c040001, 0x248452c8, 0xafa00010, 0xafa00014,
-0x8ee60608, 0x8f470228, 0x3c050009, 0xc002403,
-0x34a5f000, 0x800197c, 0x0, 0x8f830120,
-0x27623800, 0x24660020, 0xc2102b, 0x50400001,
-0x27663000, 0x8f820128, 0x10c20004, 0x0,
-0x8f820124, 0x14c20007, 0x0, 0x8ee201a4,
-0x8021, 0x24420001, 0xaee201a4, 0x800195e,
-0x8ee201a4, 0x8ee20608, 0xac62001c, 0x8ee404a0,
-0x8ee504a4, 0x2462001c, 0xac620008, 0x24020008,
-0xa462000e, 0x24020011, 0xac620018, 0xac640000,
-0xac650004, 0x8ee204c4, 0xac620010, 0xaf860120,
-0x92e24e20, 0x14400037, 0x24100001, 0x8ee24e30,
-0x210c0, 0x24425038, 0x2e22021, 0x8c830000,
-0x24020012, 0x1462001f, 0x0, 0x8ee34e30,
-0x8ee24e34, 0x1062001b, 0x24030040, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee24e34, 0x8ee54e30,
-0x24420001, 0x10430007, 0x0, 0x8ee24e34,
-0x24420001, 0x10a20005, 0x0, 0x8001948,
-0x0, 0x14a00005, 0x0, 0x8f820128,
-0x24420020, 0xaf820128, 0x8f820128, 0x8c820004,
-0x2c420011, 0x50400013, 0xac800000, 0x800195e,
-0x0, 0x8ee24e30, 0x24030040, 0x24420001,
-0x50430003, 0x1021, 0x8ee24e30, 0x24420001,
-0xaee24e30, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0x24020012, 0xac820000, 0x24020001,
-0xac820004, 0x5600001d, 0x24100001, 0x3c040001,
-0x248452d0, 0xafa00010, 0xafa00014, 0x8ee60608,
-0x8f470228, 0x3c050009, 0xc002403, 0x34a5f001,
-0x8ee201b0, 0x24420001, 0xaee201b0, 0x800197c,
-0x8ee201b0, 0x3c040001, 0x248452dc, 0xafa00014,
-0x8ee60608, 0x8f470228, 0x3c050009, 0x34a5f005,
-0xc002403, 0x0, 0x8ee201ac, 0x8021,
-0x24420001, 0xaee201ac, 0x8ee201ac, 0x1200000c,
-0x24020001, 0x3c010001, 0x370821, 0xa02083b0,
-0x8f420238, 0x8ee30158, 0x24630001, 0xaee30158,
-0x8ee30158, 0x800198c, 0xaee27278, 0x24020001,
-0x3c010001, 0x370821, 0xa02283b0, 0x3c020001,
-0x8c425cd8, 0x10400187, 0x0, 0x8ee27b84,
-0x24430001, 0x284200c9, 0x144001a4, 0xaee37b84,
-0x8ee204d4, 0x30420002, 0x14400119, 0xaee07b84,
-0x8ee204d4, 0x3c030600, 0x34631000, 0x34420002,
-0xaee204d4, 0xafa30018, 0x8ee20608, 0x8f430228,
-0x24420001, 0x304a00ff, 0x514300fd, 0xafa00010,
-0x8ee20608, 0x210c0, 0x571021, 0x8fa30018,
-0x8fa4001c, 0xac43060c, 0xac440610, 0x8f830054,
-0x8f820054, 0x24690032, 0x1221023, 0x2c420033,
-0x1040006a, 0x5821, 0x24180008, 0x240f000d,
-0x240d0007, 0x240c0040, 0x240e0001, 0x8f870120,
-0x27623800, 0x24e80020, 0x102102b, 0x50400001,
-0x27683000, 0x8f820128, 0x11020004, 0x0,
-0x8f820124, 0x15020007, 0x1021, 0x8ee201a4,
-0x8021, 0x24420001, 0xaee201a4, 0x8001a15,
-0x8ee201a4, 0x8ee40608, 0x420c0, 0x801821,
-0x8ee40430, 0x8ee50434, 0xa32821, 0xa3302b,
-0x822021, 0x862021, 0xace40000, 0xace50004,
-0x8ee20608, 0xa4f8000e, 0xacef0018, 0xacea001c,
-0x210c0, 0x2442060c, 0x2e21021, 0xace20008,
-0x8ee204c4, 0xace20010, 0xaf880120, 0x92e24e20,
-0x14400033, 0x24100001, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x8c820000, 0x144d001f,
-0x0, 0x8ee34e30, 0x8ee24e34, 0x1062001b,
-0x0, 0x8c820004, 0x24420001, 0xac820004,
-0x8ee24e34, 0x8ee34e30, 0x24420001, 0x104c0007,
-0x0, 0x8ee24e34, 0x24420001, 0x10620005,
-0x0, 0x8001a02, 0x0, 0x14600005,
-0x0, 0x8f820128, 0x24420020, 0xaf820128,
-0x8f820128, 0x8c820004, 0x2c420011, 0x50400010,
-0xac800000, 0x8001a15, 0x0, 0x8ee24e30,
-0x24420001, 0x504c0003, 0x1021, 0x8ee24e30,
-0x24420001, 0xaee24e30, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0xac8d0000, 0xac8e0004,
-0x56000006, 0x240b0001, 0x8f820054, 0x1221023,
-0x2c420033, 0x1440ff9d, 0x0, 0x316300ff,
-0x24020001, 0x54620078, 0xafa00010, 0xaeea0608,
-0x8f830054, 0x8f820054, 0x24690032, 0x1221023,
-0x2c420033, 0x10400061, 0x5821, 0x240d0008,
-0x240c0011, 0x24080012, 0x24070040, 0x240a0001,
-0x8f830120, 0x27623800, 0x24660020, 0xc2102b,
-0x50400001, 0x27663000, 0x8f820128, 0x10c20004,
-0x0, 0x8f820124, 0x14c20007, 0x0,
-0x8ee201a4, 0x8021, 0x24420001, 0xaee201a4,
-0x8001a81, 0x8ee201a4, 0x8ee20608, 0xac62001c,
-0x8ee404a0, 0x8ee504a4, 0x2462001c, 0xac620008,
-0xa46d000e, 0xac6c0018, 0xac640000, 0xac650004,
-0x8ee204c4, 0xac620010, 0xaf860120, 0x92e24e20,
-0x14400033, 0x24100001, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x8c820000, 0x1448001f,
-0x0, 0x8ee34e30, 0x8ee24e34, 0x1062001b,
-0x0, 0x8c820004, 0x24420001, 0xac820004,
-0x8ee24e34, 0x8ee34e30, 0x24420001, 0x10470007,
-0x0, 0x8ee24e34, 0x24420001, 0x10620005,
-0x0, 0x8001a6e, 0x0, 0x14600005,
-0x0, 0x8f820128, 0x24420020, 0xaf820128,
-0x8f820128, 0x8c820004, 0x2c420011, 0x50400010,
-0xac800000, 0x8001a81, 0x0, 0x8ee24e30,
-0x24420001, 0x50470003, 0x1021, 0x8ee24e30,
-0x24420001, 0xaee24e30, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0xac880000, 0xac8a0004,
-0x56000006, 0x240b0001, 0x8f820054, 0x1221023,
-0x2c420033, 0x1440ffa6, 0x0, 0x316300ff,
-0x24020001, 0x10620022, 0x0, 0x3c040001,
-0x248452a4, 0xafa00010, 0xafa00014, 0x8f860120,
-0x8f870124, 0x3c050009, 0xc002403, 0x34a5f011,
-0x8001aad, 0x0, 0x3c040001, 0x248452b0,
-0xafa00014, 0x8f860120, 0x8f870124, 0x3c050009,
-0xc002403, 0x34a5f010, 0x8001aad, 0x0,
-0x3c040001, 0x248452bc, 0xafa00014, 0x8ee60608,
-0x8f470228, 0x3c050009, 0xc002403, 0x34a5f00f,
-0x8ee201ac, 0x24420001, 0xaee201ac, 0x8ee201ac,
-0x8ee2015c, 0x24420001, 0xaee2015c, 0x8ee2015c,
-0x8ee204d4, 0x30420001, 0x10400055, 0x0,
-0x8f420218, 0x30420080, 0x10400029, 0x0,
-0x8f820044, 0x34420040, 0xaf820044, 0x8ee27b7c,
-0x402821, 0x8ee200c0, 0x8ee300c4, 0x24060000,
-0x2407ffff, 0x2021, 0x461024, 0x1444000d,
-0x671824, 0x1465000b, 0x0, 0x8ee27b80,
-0x402821, 0x8ee200e0, 0x8ee300e4, 0x2021,
-0x461024, 0x14440003, 0x671824, 0x1065000b,
-0x0, 0x8ee200c0, 0x8ee300c4, 0x8ee400e0,
-0x8ee500e4, 0xaee37b7c, 0xaee57b80, 0x8f820044,
-0x38420020, 0x8001b38, 0xaf820044, 0x8f820044,
-0x2403ffdf, 0x431024, 0x8001b38, 0xaf820044,
-0x8f820044, 0x2403ffdf, 0x431024, 0xaf820044,
-0x8ee27b7c, 0x402821, 0x8ee200c0, 0x8ee300c4,
-0x24060000, 0x2407ffff, 0x2021, 0x461024,
-0x1444000d, 0x671824, 0x1465000b, 0x0,
-0x8ee27b80, 0x402821, 0x8ee200e0, 0x8ee300e4,
-0x2021, 0x461024, 0x14440003, 0x671824,
-0x1065000b, 0x0, 0x8ee200c0, 0x8ee300c4,
-0x8ee400e0, 0x8ee500e4, 0xaee37b7c, 0xaee57b80,
-0x8f820044, 0x38420040, 0x8001b38, 0xaf820044,
-0x8f820044, 0x34420040, 0x8001b38, 0xaf820044,
-0x8f820044, 0x34420040, 0xaf820044, 0x8ee27b8c,
-0x24430001, 0x28420015, 0x14400028, 0xaee37b8c,
-0x8f820044, 0x38420020, 0xaf820044, 0x8001b38,
-0xaee07b8c, 0x8ee204d4, 0x30420001, 0x10400011,
-0x0, 0x8f420218, 0x30420080, 0x10400009,
-0x0, 0x8f820044, 0x34420020, 0xaf820044,
-0x8f820044, 0x2403ffbf, 0x431024, 0x8001b36,
-0xaf820044, 0x8f820044, 0x34420060, 0x8001b36,
-0xaf820044, 0x8f820044, 0x34420040, 0xaf820044,
-0x8ee27b88, 0x24430001, 0x28421389, 0x14400005,
-0xaee37b88, 0x8f820044, 0x38420020, 0xaf820044,
-0xaee07b88, 0xc004603, 0x0, 0x8fbf0024,
-0x8fb00020, 0x3e00008, 0x27bd0028, 0x27bdffb8,
-0xafbf0044, 0xafb60040, 0xafb5003c, 0xafb40038,
-0xafb30034, 0xafb20030, 0xafb1002c, 0xafb00028,
-0x8f960064, 0x32c20004, 0x1040000c, 0x24020004,
-0xaf820064, 0x8f420114, 0xaee204e0, 0x8f820060,
-0x34420008, 0xaf820060, 0x8ee2016c, 0x24420001,
-0xaee2016c, 0x80022f4, 0x8ee2016c, 0x32c20001,
-0x10400004, 0x24020001, 0xaf820064, 0x80022f4,
-0x0, 0x32c20002, 0x1440000c, 0x3c050003,
-0x3c040001, 0x24845354, 0x34a50001, 0x2c03021,
-0x3821, 0xafa00010, 0xc002403, 0xafa00014,
-0x2402fff8, 0x80022f4, 0xaf820064, 0x8f43022c,
-0x8f42010c, 0x5062000c, 0xafa00010, 0x8f42022c,
-0x21080, 0x5a1021, 0x8c420300, 0xafa20020,
-0x8f42022c, 0x24070001, 0x24420001, 0x3042003f,
-0x8001b80, 0xaf42022c, 0x3c040001, 0x24845360,
-0xafa00014, 0x8f46022c, 0x8f47010c, 0x3c050003,
-0xc002403, 0x34a5f01f, 0x3821, 0x14e00003,
-0x0, 0x80022ed, 0xaf960064, 0x93a20020,
-0x2443ffff, 0x2c620011, 0x10400658, 0x31080,
-0x3c010001, 0x220821, 0x8c225418, 0x400008,
-0x0, 0x8fa20020, 0x30420fff, 0xaee20e0c,
-0x8f820060, 0x34420200, 0xaf820060, 0x8ee20118,
-0x24420001, 0xaee20118, 0x80022e8, 0x8ee20118,
-0x8fa20020, 0x24030001, 0x3c010001, 0x370821,
-0xa02383b1, 0x30420fff, 0xaee25238, 0x8f820060,
-0x34420100, 0xaf820060, 0x8ee20144, 0x24420001,
-0xaee20144, 0x80022e8, 0x8ee20144, 0x8fa20020,
-0x21200, 0x22502, 0x24020001, 0x10820005,
-0x24020002, 0x10820009, 0x2402fffe, 0x8001bc9,
-0xafa00010, 0x8ee204d4, 0xaee40070, 0xaee40074,
-0x34420001, 0x8001bbd, 0xaee204d4, 0x8ee304d4,
-0xaee40070, 0xaee40074, 0x621824, 0xaee304d4,
-0x8f840054, 0x41442, 0x41c82, 0x431021,
-0x41cc2, 0x431023, 0x41d02, 0x431021,
-0x41d42, 0x431023, 0x8001bd0, 0xaee20078,
-0x3c040001, 0x2484536c, 0xafa00014, 0x8fa60020,
-0x3c050003, 0xc002403, 0x34a50004, 0x8ee20110,
-0x24420001, 0xaee20110, 0x80022e8, 0x8ee20110,
-0x27440212, 0xc0022fe, 0x24050006, 0x3049001f,
-0x920c0, 0x2e41021, 0x9442727c, 0x30424000,
-0x1040000a, 0x971021, 0x97430212, 0xa443727e,
-0x8f430214, 0x971021, 0xac437280, 0x2e41821,
-0x34028000, 0x8001c79, 0xa462727c, 0x9443727e,
-0x97420212, 0x14620006, 0x2e41021, 0x971021,
-0x8c437280, 0x8f420214, 0x1062009f, 0x2e41021,
-0x9442727c, 0x30428000, 0x1040002a, 0x2406ffff,
-0x2021, 0x410c0, 0x2e21021, 0x9442737c,
-0x30424000, 0x54400005, 0x803021, 0x24840001,
-0x2c820080, 0x1440fff8, 0x410c0, 0x4c10010,
-0x618c0, 0x610c0, 0x571821, 0x8c63737c,
-0x571021, 0xafa30010, 0x8c427380, 0x3c040001,
-0x24845378, 0xafa20014, 0x8f470214, 0x3c050003,
-0xc002403, 0x34a50013, 0x8001c90, 0x3c020800,
-0x97440212, 0x771021, 0xa444737e, 0x8f440214,
-0x771021, 0x2e31821, 0xac447380, 0x34028000,
-0xa462737c, 0x910c0, 0x2e21021, 0x8001c79,
-0xa446727c, 0x2e41021, 0x9445727c, 0x8001c2e,
-0x510c0, 0x9443737e, 0x97420212, 0x14620006,
-0x510c0, 0x971021, 0x8c437380, 0x8f420214,
-0x10620065, 0x510c0, 0x2e21021, 0x9445737c,
-0x510c0, 0x2e21021, 0x9442737c, 0x30428000,
-0x1040fff0, 0x971021, 0x520c0, 0x971021,
-0x9443737e, 0x97420212, 0x14620006, 0x2406ffff,
-0x971021, 0x8c437380, 0x8f420214, 0x10620053,
-0x3c020800, 0x2021, 0x410c0, 0x2e21021,
-0x9442737c, 0x30424000, 0x54400005, 0x803021,
-0x24840001, 0x2c820080, 0x1440fff8, 0x410c0,
-0x4c10023, 0x618c0, 0x910c0, 0x571821,
-0x8c63727c, 0x571021, 0xafa30010, 0x8c427280,
-0x3c040001, 0x24845384, 0xafa20014, 0x8f470214,
-0x3c050003, 0xc002403, 0x34a5f017, 0x8001c90,
-0x3c020800, 0x8f430210, 0xb71021, 0xac43777c,
-0x8f430214, 0xb71021, 0xac437780, 0x3c020001,
-0x571021, 0x8c4283b4, 0x24420001, 0x3c010001,
-0x370821, 0xac2283b4, 0x3c030001, 0x771821,
-0x8c6383b4, 0x2e51021, 0x8001c82, 0xa443777c,
-0x97440212, 0x771021, 0xa444737e, 0x8f440214,
-0x771021, 0x2e31821, 0xac447380, 0x34028000,
-0xa462737c, 0x510c0, 0x2e21021, 0xa446737c,
-0x2021, 0x428c0, 0x2e51021, 0x9442777c,
-0x1040ffdc, 0x24840001, 0x2c820080, 0x5440fffa,
-0x428c0, 0x92e204d8, 0x10400006, 0x24020001,
-0x8ee304dc, 0x1221004, 0x621825, 0x8001c8f,
-0xaee304dc, 0x8f830228, 0x24020001, 0x1221004,
-0x621825, 0xaf830228, 0x3c020800, 0x34421000,
-0xafa20018, 0x8ee20608, 0x8f430228, 0x24420001,
-0x304a00ff, 0x514300fd, 0xafa00010, 0x8ee20608,
-0x210c0, 0x571021, 0x8fa30018, 0x8fa4001c,
-0xac43060c, 0xac440610, 0x8f830054, 0x8f820054,
-0x24690032, 0x1221023, 0x2c420033, 0x1040006a,
-0x5821, 0x24100008, 0x240f000d, 0x240d0007,
-0x240c0040, 0x240e0001, 0x8f870120, 0x27623800,
-0x24e80020, 0x102102b, 0x50400001, 0x27683000,
-0x8f820128, 0x11020004, 0x0, 0x8f820124,
-0x15020007, 0x1021, 0x8ee201a4, 0x3821,
-0x24420001, 0xaee201a4, 0x8001d08, 0x8ee201a4,
-0x8ee40608, 0x420c0, 0x801821, 0x8ee40430,
-0x8ee50434, 0xa32821, 0xa3302b, 0x822021,
-0x862021, 0xace40000, 0xace50004, 0x8ee20608,
-0xa4f0000e, 0xacef0018, 0xacea001c, 0x210c0,
-0x2442060c, 0x2e21021, 0xace20008, 0x8ee204c4,
-0xace20010, 0xaf880120, 0x92e24e20, 0x14400033,
-0x24070001, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0x8c820000, 0x144d001f, 0x0,
-0x8ee34e30, 0x8ee24e34, 0x1062001b, 0x0,
-0x8c820004, 0x24420001, 0xac820004, 0x8ee24e34,
-0x8ee34e30, 0x24420001, 0x104c0007, 0x0,
-0x8ee24e34, 0x24420001, 0x10620005, 0x0,
-0x8001cf5, 0x0, 0x14600005, 0x0,
-0x8f820128, 0x24420020, 0xaf820128, 0x8f820128,
-0x8c820004, 0x2c420011, 0x50400010, 0xac800000,
-0x8001d08, 0x0, 0x8ee24e30, 0x24420001,
-0x504c0003, 0x1021, 0x8ee24e30, 0x24420001,
-0xaee24e30, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0xac8d0000, 0xac8e0004, 0x54e00006,
-0x240b0001, 0x8f820054, 0x1221023, 0x2c420033,
-0x1440ff9d, 0x0, 0x316300ff, 0x24020001,
-0x54620078, 0xafa00010, 0xaeea0608, 0x8f830054,
-0x8f820054, 0x24690032, 0x1221023, 0x2c420033,
-0x10400061, 0x5821, 0x240e0008, 0x240d0011,
-0x240a0012, 0x24080040, 0x240c0001, 0x8f830120,
-0x27623800, 0x24660020, 0xc2102b, 0x50400001,
-0x27663000, 0x8f820128, 0x10c20004, 0x0,
-0x8f820124, 0x14c20007, 0x0, 0x8ee201a4,
-0x3821, 0x24420001, 0xaee201a4, 0x8001d74,
-0x8ee201a4, 0x8ee20608, 0xac62001c, 0x8ee404a0,
-0x8ee504a4, 0x2462001c, 0xac620008, 0xa46e000e,
-0xac6d0018, 0xac640000, 0xac650004, 0x8ee204c4,
-0xac620010, 0xaf860120, 0x92e24e20, 0x14400033,
-0x24070001, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0x8c820000, 0x144a001f, 0x0,
-0x8ee34e30, 0x8ee24e34, 0x1062001b, 0x0,
-0x8c820004, 0x24420001, 0xac820004, 0x8ee24e34,
-0x8ee34e30, 0x24420001, 0x10480007, 0x0,
-0x8ee24e34, 0x24420001, 0x10620005, 0x0,
-0x8001d61, 0x0, 0x14600005, 0x0,
-0x8f820128, 0x24420020, 0xaf820128, 0x8f820128,
-0x8c820004, 0x2c420011, 0x50400010, 0xac800000,
-0x8001d74, 0x0, 0x8ee24e30, 0x24420001,
-0x50480003, 0x1021, 0x8ee24e30, 0x24420001,
-0xaee24e30, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0xac8a0000, 0xac8c0004, 0x54e00006,
-0x240b0001, 0x8f820054, 0x1221023, 0x2c420033,
-0x1440ffa6, 0x0, 0x316300ff, 0x24020001,
-0x10620022, 0x0, 0x3c040001, 0x24845390,
-0xafa00010, 0xafa00014, 0x8f860120, 0x8f870124,
-0x3c050009, 0xc002403, 0x34a5f011, 0x8001da0,
-0x0, 0x3c040001, 0x2484539c, 0xafa00014,
-0x8f860120, 0x8f870124, 0x3c050009, 0xc002403,
-0x34a5f010, 0x8001da0, 0x0, 0x3c040001,
-0x248453a8, 0xafa00014, 0x8ee60608, 0x8f470228,
-0x3c050009, 0xc002403, 0x34a5f00f, 0x8ee201ac,
-0x24420001, 0xaee201ac, 0x8ee201ac, 0x8ee20124,
-0x24420001, 0xaee20124, 0x8001f97, 0x8ee20124,
-0x27440212, 0xc0022fe, 0x24050006, 0x3049001f,
-0x928c0, 0x2e51021, 0x9442727c, 0x30428000,
-0x1040002f, 0x2e51021, 0x9442727c, 0x30424000,
-0x1440001c, 0xb71021, 0x9443727e, 0x97420212,
-0x14620018, 0xb71021, 0x8c437280, 0x8f420214,
-0x54620016, 0xafa20010, 0x92e204d8, 0x10400007,
-0x24020001, 0x8ee304dc, 0x1221004, 0x21027,
-0x621824, 0x8001dc9, 0xaee304dc, 0x8f830228,
-0x1221004, 0x21027, 0x621824, 0xaf830228,
-0x910c0, 0x2e21821, 0x3402c000, 0x8001e4e,
-0xa462727c, 0x8f420214, 0xafa20010, 0x910c0,
-0x571021, 0x8c42727c, 0x3c040001, 0x248453b4,
-0x3c050003, 0xafa20014, 0x8f470210, 0x34a5f01c,
-0xc002403, 0x1203021, 0x8001e83, 0x3c020800,
-0xb71021, 0x9443727e, 0x97420212, 0x14620019,
-0x918c0, 0xb71021, 0x8c437280, 0x8f420214,
-0x14620014, 0x918c0, 0x2e51021, 0x9447727c,
-0x720c0, 0x971021, 0x9443737e, 0xb71021,
-0xa443727e, 0x971021, 0x8c437380, 0xb71021,
-0xac437280, 0x2e41021, 0x9443737c, 0x2e51021,
-0xa443727c, 0x2e41821, 0x3402c000, 0x8001e4e,
-0xa462737c, 0x2e31021, 0x9447727c, 0x3021,
-0x720c0, 0x2e41021, 0x9442737c, 0x4021,
-0x30428000, 0x14400025, 0xe02821, 0x605021,
-0x340bc000, 0x971021, 0x9443737e, 0x97420212,
-0x54620015, 0xe02821, 0x971021, 0x8c437380,
-0x8f420214, 0x54620010, 0xe02821, 0x11000006,
-0x2e41021, 0x9443737c, 0x510c0, 0x2e21021,
-0x8001e1a, 0xa443737c, 0x9443737c, 0x2ea1021,
-0xa443727c, 0x710c0, 0x2e21021, 0xa44b737c,
-0x8001e28, 0x24060001, 0x510c0, 0x2e21021,
-0x9447737c, 0x720c0, 0x2e41021, 0x9442737c,
-0x30428000, 0x1040ffdf, 0x25080001, 0x30c200ff,
-0x14400025, 0x2021, 0x720c0, 0x971021,
-0x9443737e, 0x97420212, 0x1462000f, 0x910c0,
-0x971021, 0x8c437380, 0x8f420214, 0x1462000a,
-0x910c0, 0x2e41821, 0x3402c000, 0x15000015,
-0xa462737c, 0x910c0, 0x2e21821, 0x34028000,
-0x8001e4e, 0xa462727c, 0x571021, 0x8c42727c,
-0x3c040001, 0x248453c0, 0x3c050003, 0xafa20010,
-0x710c0, 0x571021, 0x8c42737c, 0x34a5001e,
-0x1203021, 0xc002403, 0xafa20014, 0x8001e83,
-0x3c020800, 0x2021, 0x428c0, 0xb71021,
-0x9443777e, 0x97420212, 0x5462002b, 0x24840001,
-0xb71021, 0x8c437780, 0x8f420214, 0x54620026,
-0x24840001, 0x3c020001, 0x571021, 0x8c4283b4,
-0x2442ffff, 0x3c010001, 0x370821, 0xac2283b4,
-0x3c020001, 0x571021, 0x8c4283b4, 0x809021,
-0x242102b, 0x1040000e, 0x24b1777c, 0x24b07784,
-0x2f02021, 0x2f12821, 0xc002490, 0x24060008,
-0x26310008, 0x3c020001, 0x571021, 0x8c4283b4,
-0x26520001, 0x242102b, 0x1440fff5, 0x26100008,
-0x3c040001, 0x972021, 0x8c8483b4, 0x24050008,
-0x420c0, 0x2484777c, 0xc002488, 0x2e42021,
-0x8001e83, 0x3c020800, 0x2c820080, 0x1440ffcf,
-0x428c0, 0x3c020800, 0x34422000, 0xafa20018,
-0x8ee20608, 0x8f430228, 0x24420001, 0x304a00ff,
-0x514300fd, 0xafa00010, 0x8ee20608, 0x210c0,
-0x571021, 0x8fa30018, 0x8fa4001c, 0xac43060c,
-0xac440610, 0x8f830054, 0x8f820054, 0x24690032,
-0x1221023, 0x2c420033, 0x1040006a, 0x5821,
-0x24100008, 0x240f000d, 0x240d0007, 0x240c0040,
-0x240e0001, 0x8f870120, 0x27623800, 0x24e80020,
-0x102102b, 0x50400001, 0x27683000, 0x8f820128,
-0x11020004, 0x0, 0x8f820124, 0x15020007,
-0x1021, 0x8ee201a4, 0x3821, 0x24420001,
-0xaee201a4, 0x8001efb, 0x8ee201a4, 0x8ee40608,
-0x420c0, 0x801821, 0x8ee40430, 0x8ee50434,
-0xa32821, 0xa3302b, 0x822021, 0x862021,
-0xace40000, 0xace50004, 0x8ee20608, 0xa4f0000e,
-0xacef0018, 0xacea001c, 0x210c0, 0x2442060c,
-0x2e21021, 0xace20008, 0x8ee204c4, 0xace20010,
-0xaf880120, 0x92e24e20, 0x14400033, 0x24070001,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x8c820000, 0x144d001f, 0x0, 0x8ee34e30,
-0x8ee24e34, 0x1062001b, 0x0, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee24e34, 0x8ee34e30,
-0x24420001, 0x104c0007, 0x0, 0x8ee24e34,
-0x24420001, 0x10620005, 0x0, 0x8001ee8,
-0x0, 0x14600005, 0x0, 0x8f820128,
-0x24420020, 0xaf820128, 0x8f820128, 0x8c820004,
-0x2c420011, 0x50400010, 0xac800000, 0x8001efb,
-0x0, 0x8ee24e30, 0x24420001, 0x504c0003,
-0x1021, 0x8ee24e30, 0x24420001, 0xaee24e30,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0xac8d0000, 0xac8e0004, 0x54e00006, 0x240b0001,
-0x8f820054, 0x1221023, 0x2c420033, 0x1440ff9d,
-0x0, 0x316300ff, 0x24020001, 0x54620078,
-0xafa00010, 0xaeea0608, 0x8f830054, 0x8f820054,
-0x24690032, 0x1221023, 0x2c420033, 0x10400061,
-0x5821, 0x240e0008, 0x240d0011, 0x240a0012,
-0x24080040, 0x240c0001, 0x8f830120, 0x27623800,
-0x24660020, 0xc2102b, 0x50400001, 0x27663000,
-0x8f820128, 0x10c20004, 0x0, 0x8f820124,
-0x14c20007, 0x0, 0x8ee201a4, 0x3821,
-0x24420001, 0xaee201a4, 0x8001f67, 0x8ee201a4,
-0x8ee20608, 0xac62001c, 0x8ee404a0, 0x8ee504a4,
-0x2462001c, 0xac620008, 0xa46e000e, 0xac6d0018,
-0xac640000, 0xac650004, 0x8ee204c4, 0xac620010,
-0xaf860120, 0x92e24e20, 0x14400033, 0x24070001,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x8c820000, 0x144a001f, 0x0, 0x8ee34e30,
-0x8ee24e34, 0x1062001b, 0x0, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee24e34, 0x8ee34e30,
-0x24420001, 0x10480007, 0x0, 0x8ee24e34,
-0x24420001, 0x10620005, 0x0, 0x8001f54,
-0x0, 0x14600005, 0x0, 0x8f820128,
-0x24420020, 0xaf820128, 0x8f820128, 0x8c820004,
-0x2c420011, 0x50400010, 0xac800000, 0x8001f67,
-0x0, 0x8ee24e30, 0x24420001, 0x50480003,
-0x1021, 0x8ee24e30, 0x24420001, 0xaee24e30,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0xac8a0000, 0xac8c0004, 0x54e00006, 0x240b0001,
-0x8f820054, 0x1221023, 0x2c420033, 0x1440ffa6,
-0x0, 0x316300ff, 0x24020001, 0x10620022,
-0x0, 0x3c040001, 0x24845390, 0xafa00010,
-0xafa00014, 0x8f860120, 0x8f870124, 0x3c050009,
-0xc002403, 0x34a5f011, 0x8001f93, 0x0,
-0x3c040001, 0x2484539c, 0xafa00014, 0x8f860120,
-0x8f870124, 0x3c050009, 0xc002403, 0x34a5f010,
-0x8001f93, 0x0, 0x3c040001, 0x248453a8,
-0xafa00014, 0x8ee60608, 0x8f470228, 0x3c050009,
-0xc002403, 0x34a5f00f, 0x8ee201ac, 0x24420001,
-0xaee201ac, 0x8ee201ac, 0x8ee20128, 0x24420001,
-0xaee20128, 0x8ee20128, 0x8ee20164, 0x24420001,
-0xaee20164, 0x80022e8, 0x8ee20164, 0x8fa20020,
-0x21200, 0x21d02, 0x24020001, 0x10620005,
-0x24020002, 0x1062000d, 0x0, 0x8001fb7,
-0xafa00010, 0x92e204d8, 0x14400006, 0x24020001,
-0x8f820228, 0xaee204dc, 0x2402ffff, 0xaf820228,
-0x24020001, 0x8001fbe, 0xa2e204d8, 0x92e204d8,
-0x5040000c, 0xa2e004d8, 0x8ee204dc, 0xaf820228,
-0x8001fbe, 0xa2e004d8, 0x3c040001, 0x248453c8,
-0xafa00014, 0x8fa60020, 0x3c050003, 0xc002403,
-0x34a5f009, 0x8ee2013c, 0x24420001, 0xaee2013c,
-0x80022e8, 0x8ee2013c, 0x8fa20020, 0x21200,
-0x22502, 0x24020001, 0x10820005, 0x24020002,
-0x1082000f, 0x0, 0x8001fe3, 0xafa00010,
-0x8f820220, 0x3c0308ff, 0x3463ffff, 0x431024,
-0x34420008, 0xaf820220, 0x24020001, 0x3c010001,
-0x370821, 0xa02283b2, 0x8001fea, 0xaee40108,
-0x8f820220, 0x3c0308ff, 0x3463fff7, 0x431024,
-0xaf820220, 0x3c010001, 0x370821, 0xa02083b2,
-0x8001fea, 0xaee40108, 0x3c040001, 0x248453d4,
-0xafa00014, 0x8fa60020, 0x3c050003, 0xc002403,
-0x34a5f00a, 0x8ee2012c, 0x24420001, 0xaee2012c,
-0x80022e8, 0x8ee2012c, 0x8fa20020, 0x21200,
-0x21d02, 0x24020001, 0x10620005, 0x24020002,
-0x1062000e, 0x0, 0x8002011, 0xafa00010,
-0x8f820220, 0x3c0308ff, 0x3463ffff, 0x431024,
-0x34420008, 0xaf820220, 0x24020001, 0x3c010001,
-0x370821, 0x8002018, 0xa02283b3, 0x3c020001,
-0x571021, 0x904283b2, 0x3c010001, 0x370821,
-0x1440000e, 0xa02083b3, 0x8f820220, 0x3c0308ff,
-0x3463fff7, 0x431024, 0x8002018, 0xaf820220,
-0x3c040001, 0x248453e0, 0xafa00014, 0x8fa60020,
-0x3c050003, 0xc002403, 0x34a5f00b, 0x8ee20114,
-0x24420001, 0xaee20114, 0x80022e8, 0x8ee20114,
-0x27840208, 0x27450200, 0xc00249a, 0x24060008,
-0x26e40094, 0x27450200, 0xc00249a, 0x24060008,
-0x8ee20134, 0x24420001, 0xaee20134, 0x80022e8,
-0x8ee20134, 0x8f460248, 0x2021, 0xc005108,
-0x24050004, 0x8ee20130, 0x24420001, 0xaee20130,
-0x80022e8, 0x8ee20130, 0x8ef301cc, 0x8ef401d0,
-0x8ef501d8, 0x8ee20140, 0x26e40030, 0x24420001,
-0xaee20140, 0x8ef00140, 0x8ef10074, 0x8ef20070,
-0xc002488, 0x24050400, 0xaef301cc, 0xaef401d0,
-0xaef501d8, 0xaef00140, 0xaef10074, 0xaef20070,
-0x8f42025c, 0x26e40094, 0xaee20060, 0x8f420260,
-0x27450200, 0x24060008, 0xaee20068, 0x24020006,
-0xc00249a, 0xaee20064, 0x3c023b9a, 0x3442ca00,
-0xaee2006c, 0x240203e8, 0x24040002, 0x24030001,
-0xaee20104, 0xaee40100, 0xaee3010c, 0x8f820220,
-0x30420008, 0x10400004, 0x0, 0xaee30108,
-0x8002061, 0x2021, 0xaee40108, 0x2021,
-0x3c030001, 0x641821, 0x90635c30, 0x2e41021,
-0x24840001, 0xa043009c, 0x2c82000f, 0x1440fff8,
-0x0, 0x8f820040, 0x2e41821, 0x24840001,
-0x21702, 0x24420030, 0xa062009c, 0x2e41021,
-0x80022e8, 0xa040009c, 0x24020001, 0x3c010001,
-0x370821, 0xa02283e0, 0x240b0400, 0x24080014,
-0x240a0040, 0x24090001, 0x8f830100, 0x27623000,
-0x24660020, 0xc2102b, 0x50400001, 0x27662800,
-0x8f820108, 0x10c20004, 0x0, 0x8f820104,
-0x14c20007, 0x26e20030, 0x8ee201a8, 0x3821,
-0x24420001, 0xaee201a8, 0x80020a8, 0x8ee201a8,
-0x8ee404b8, 0x8ee504bc, 0xac620008, 0xa46b000e,
-0xac680018, 0xac60001c, 0xac640000, 0xac650004,
-0x8ee204cc, 0xac620010, 0xaf860100, 0x92e204ec,
-0x1440000e, 0x24070001, 0x8ee24e28, 0x24420001,
-0x504a0003, 0x1021, 0x8ee24e28, 0x24420001,
-0xaee24e28, 0x8ee24e28, 0x210c0, 0x24424e38,
-0x2e21021, 0xac480000, 0xac490004, 0x10e0ffd2,
-0x0, 0x80022e8, 0x0, 0x3c020900,
-0xaee05238, 0xaee0523c, 0xaee05240, 0xaee05244,
-0xaee001d0, 0x3c010001, 0x370821, 0xa02083b1,
-0xafa20018, 0x8ee20608, 0x8f430228, 0x24420001,
-0x304a00ff, 0x514300fd, 0xafa00010, 0x8ee20608,
-0x210c0, 0x571021, 0x8fa30018, 0x8fa4001c,
-0xac43060c, 0xac440610, 0x8f830054, 0x8f820054,
-0x24690032, 0x1221023, 0x2c420033, 0x1040006a,
-0x5821, 0x24100008, 0x240f000d, 0x240d0007,
-0x240c0040, 0x240e0001, 0x8f870120, 0x27623800,
-0x24e80020, 0x102102b, 0x50400001, 0x27683000,
-0x8f820128, 0x11020004, 0x0, 0x8f820124,
-0x15020007, 0x1021, 0x8ee201a4, 0x3821,
-0x24420001, 0xaee201a4, 0x800212c, 0x8ee201a4,
-0x8ee40608, 0x420c0, 0x801821, 0x8ee40430,
-0x8ee50434, 0xa32821, 0xa3302b, 0x822021,
-0x862021, 0xace40000, 0xace50004, 0x8ee20608,
-0xa4f0000e, 0xacef0018, 0xacea001c, 0x210c0,
-0x2442060c, 0x2e21021, 0xace20008, 0x8ee204c4,
-0xace20010, 0xaf880120, 0x92e24e20, 0x14400033,
-0x24070001, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0x8c820000, 0x144d001f, 0x0,
-0x8ee34e30, 0x8ee24e34, 0x1062001b, 0x0,
-0x8c820004, 0x24420001, 0xac820004, 0x8ee24e34,
-0x8ee34e30, 0x24420001, 0x104c0007, 0x0,
-0x8ee24e34, 0x24420001, 0x10620005, 0x0,
-0x8002119, 0x0, 0x14600005, 0x0,
-0x8f820128, 0x24420020, 0xaf820128, 0x8f820128,
-0x8c820004, 0x2c420011, 0x50400010, 0xac800000,
-0x800212c, 0x0, 0x8ee24e30, 0x24420001,
-0x504c0003, 0x1021, 0x8ee24e30, 0x24420001,
-0xaee24e30, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0xac8d0000, 0xac8e0004, 0x54e00006,
-0x240b0001, 0x8f820054, 0x1221023, 0x2c420033,
-0x1440ff9d, 0x0, 0x316300ff, 0x24020001,
-0x54620078, 0xafa00010, 0xaeea0608, 0x8f830054,
-0x8f820054, 0x24690032, 0x1221023, 0x2c420033,
-0x10400061, 0x5821, 0x240e0008, 0x240d0011,
-0x240a0012, 0x24080040, 0x240c0001, 0x8f830120,
-0x27623800, 0x24660020, 0xc2102b, 0x50400001,
-0x27663000, 0x8f820128, 0x10c20004, 0x0,
-0x8f820124, 0x14c20007, 0x0, 0x8ee201a4,
-0x3821, 0x24420001, 0xaee201a4, 0x8002198,
-0x8ee201a4, 0x8ee20608, 0xac62001c, 0x8ee404a0,
-0x8ee504a4, 0x2462001c, 0xac620008, 0xa46e000e,
-0xac6d0018, 0xac640000, 0xac650004, 0x8ee204c4,
-0xac620010, 0xaf860120, 0x92e24e20, 0x14400033,
-0x24070001, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0x8c820000, 0x144a001f, 0x0,
-0x8ee34e30, 0x8ee24e34, 0x1062001b, 0x0,
-0x8c820004, 0x24420001, 0xac820004, 0x8ee24e34,
-0x8ee34e30, 0x24420001, 0x10480007, 0x0,
-0x8ee24e34, 0x24420001, 0x10620005, 0x0,
-0x8002185, 0x0, 0x14600005, 0x0,
-0x8f820128, 0x24420020, 0xaf820128, 0x8f820128,
-0x8c820004, 0x2c420011, 0x50400010, 0xac800000,
-0x8002198, 0x0, 0x8ee24e30, 0x24420001,
-0x50480003, 0x1021, 0x8ee24e30, 0x24420001,
-0xaee24e30, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0xac8a0000, 0xac8c0004, 0x54e00006,
-0x240b0001, 0x8f820054, 0x1221023, 0x2c420033,
-0x1440ffa6, 0x0, 0x316300ff, 0x24020001,
-0x10620022, 0x0, 0x3c040001, 0x24845390,
-0xafa00010, 0xafa00014, 0x8f860120, 0x8f870124,
-0x3c050009, 0xc002403, 0x34a5f011, 0x80021c4,
-0x0, 0x3c040001, 0x2484539c, 0xafa00014,
-0x8f860120, 0x8f870124, 0x3c050009, 0xc002403,
-0x34a5f010, 0x80021c4, 0x0, 0x3c040001,
-0x248453a8, 0xafa00014, 0x8ee60608, 0x8f470228,
-0x3c050009, 0xc002403, 0x34a5f00f, 0x8ee201ac,
-0x24420001, 0xaee201ac, 0x8ee201ac, 0x8ee20120,
-0x24420001, 0xaee20120, 0x8ee20120, 0x8ee20168,
-0x24420001, 0xaee20168, 0x80022e8, 0x8ee20168,
-0x8f42025c, 0x26e40094, 0xaee20060, 0x8f420260,
-0x27450200, 0x24060008, 0xc00249a, 0xaee20068,
-0x8f820220, 0x30420008, 0x14400002, 0x24020001,
-0x24020002, 0xaee20108, 0x8ee2011c, 0x24420001,
-0xaee2011c, 0x80022e8, 0x8ee2011c, 0x3c040001,
-0x248453ec, 0xafa00010, 0xafa00014, 0x8fa60020,
-0x3c050003, 0xc002403, 0x34a5f00f, 0x93a20020,
-0x3c030700, 0x34631000, 0x431025, 0xafa20018,
-0x8ee20608, 0x8f430228, 0x24420001, 0x304900ff,
-0x512300e2, 0xafa00010, 0x8ee20608, 0x210c0,
-0x571021, 0x8fa30018, 0x8fa4001c, 0xac43060c,
-0xac440610, 0x8f870120, 0x27623800, 0x24e80020,
-0x102102b, 0x50400001, 0x27683000, 0x8f820128,
-0x11020004, 0x0, 0x8f820124, 0x15020007,
-0x1021, 0x8ee201a4, 0x3821, 0x24420001,
-0xaee201a4, 0x800225d, 0x8ee201a4, 0x8ee40608,
-0x420c0, 0x801821, 0x8ee40430, 0x8ee50434,
-0xa32821, 0xa3302b, 0x822021, 0x862021,
-0xace40000, 0xace50004, 0x8ee30608, 0x24020008,
-0xa4e2000e, 0x2402000d, 0xace20018, 0xace9001c,
-0x318c0, 0x2463060c, 0x2e31021, 0xace20008,
-0x8ee204c4, 0xace20010, 0xaf880120, 0x92e24e20,
-0x14400037, 0x24070001, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x8c830000, 0x24020007,
-0x1462001f, 0x0, 0x8ee34e30, 0x8ee24e34,
-0x1062001b, 0x24030040, 0x8c820004, 0x24420001,
-0xac820004, 0x8ee24e34, 0x8ee54e30, 0x24420001,
-0x10430007, 0x0, 0x8ee24e34, 0x24420001,
-0x10a20005, 0x0, 0x8002247, 0x0,
-0x14a00005, 0x0, 0x8f820128, 0x24420020,
-0xaf820128, 0x8f820128, 0x8c820004, 0x2c420011,
-0x50400013, 0xac800000, 0x800225d, 0x0,
-0x8ee24e30, 0x24030040, 0x24420001, 0x50430003,
-0x1021, 0x8ee24e30, 0x24420001, 0xaee24e30,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x24020007, 0xac820000, 0x24020001, 0xac820004,
-0x54e0000c, 0xaee90608, 0x3c040001, 0x248453f4,
-0xafa00010, 0xafa00014, 0x8ee60608, 0x8f470228,
-0x3c050009, 0xc002403, 0x34a5f000, 0x80022e0,
-0x0, 0x8f830120, 0x27623800, 0x24660020,
-0xc2102b, 0x50400001, 0x27663000, 0x8f820128,
-0x10c20004, 0x0, 0x8f820124, 0x14c20007,
-0x0, 0x8ee201a4, 0x3821, 0x24420001,
-0xaee201a4, 0x80022c4, 0x8ee201a4, 0x8ee20608,
-0xac62001c, 0x8ee404a0, 0x8ee504a4, 0x2462001c,
-0xac620008, 0x24020008, 0xa462000e, 0x24020011,
-0xac620018, 0xac640000, 0xac650004, 0x8ee204c4,
-0xac620010, 0xaf860120, 0x92e24e20, 0x14400037,
-0x24070001, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0x8c830000, 0x24020012, 0x1462001f,
-0x0, 0x8ee34e30, 0x8ee24e34, 0x1062001b,
-0x24030040, 0x8c820004, 0x24420001, 0xac820004,
-0x8ee24e34, 0x8ee54e30, 0x24420001, 0x10430007,
-0x0, 0x8ee24e34, 0x24420001, 0x10a20005,
-0x0, 0x80022ae, 0x0, 0x14a00005,
-0x0, 0x8f820128, 0x24420020, 0xaf820128,
-0x8f820128, 0x8c820004, 0x2c420011, 0x50400013,
-0xac800000, 0x80022c4, 0x0, 0x8ee24e30,
-0x24030040, 0x24420001, 0x50430003, 0x1021,
-0x8ee24e30, 0x24420001, 0xaee24e30, 0x8ee24e30,
-0x210c0, 0x24425038, 0x2e22021, 0x24020012,
-0xac820000, 0x24020001, 0xac820004, 0x14e0001b,
-0x0, 0x3c040001, 0x248453fc, 0xafa00010,
-0xafa00014, 0x8ee60608, 0x8f470228, 0x3c050009,
-0xc002403, 0x34a5f001, 0x8ee201b0, 0x24420001,
-0xaee201b0, 0x80022e0, 0x8ee201b0, 0x3c040001,
-0x24845408, 0xafa00014, 0x8ee60608, 0x8f470228,
-0x3c050009, 0xc002403, 0x34a5f005, 0x8ee201ac,
-0x24420001, 0xaee201ac, 0x8ee201ac, 0x8ee20150,
-0x24420001, 0xaee20150, 0x8ee20150, 0x8ee20160,
-0x24420001, 0xaee20160, 0x8ee20160, 0x8f43022c,
-0x8f42010c, 0x14620009, 0x24020002, 0xaf820064,
-0x8f820064, 0x14400005, 0x0, 0x8f43022c,
-0x8f42010c, 0x1462f875, 0x0, 0x8fbf0044,
-0x8fb60040, 0x8fb5003c, 0x8fb40038, 0x8fb30034,
-0x8fb20030, 0x8fb1002c, 0x8fb00028, 0x3e00008,
-0x27bd0048, 0x27bdfff8, 0x2408ffff, 0x10a00014,
-0x4821, 0x3c0aedb8, 0x354a8320, 0x90870000,
-0x24840001, 0x3021, 0x1071026, 0x30420001,
-0x10400002, 0x81842, 0x6a1826, 0x604021,
-0x24c60001, 0x2cc20008, 0x1440fff7, 0x73842,
-0x25290001, 0x125102b, 0x1440fff0, 0x0,
-0x1001021, 0x3e00008, 0x27bd0008, 0x27bdffe8,
-0x27642800, 0xafbf0010, 0xc002488, 0x24051000,
-0x24020021, 0xaf800100, 0xaf800104, 0xaf800108,
-0xaf800110, 0xaf800114, 0xaf800118, 0xaf800120,
-0xaf800124, 0xaf800128, 0xaf800130, 0xaf800134,
-0xaf800138, 0xaee04e28, 0xaee04e2c, 0xaee04e30,
-0xaee04e34, 0xaf82011c, 0x8f420218, 0x30420040,
-0x10400004, 0x0, 0x8f82011c, 0x34420004,
-0xaf82011c, 0x8fbf0010, 0x3e00008, 0x27bd0018,
-0x27bdffe0, 0xafbf0018, 0x8f820104, 0xafa20010,
-0x8f820100, 0x3c050002, 0xafa20014, 0x8f8600b0,
-0x8f87011c, 0x3c040001, 0x248454c0, 0xc002403,
-0x34a5f000, 0x8f8300b0, 0x3c027f00, 0x621824,
-0x3c020400, 0x10620029, 0x43102b, 0x14400008,
-0x3c022000, 0x3c020100, 0x10620024, 0x3c020200,
-0x10620011, 0x0, 0x8002374, 0x0,
-0x10620008, 0x3c024000, 0x1462001c, 0x0,
-0x8ee20190, 0x24420001, 0xaee20190, 0x8002374,
-0x8ee20190, 0x8ee2018c, 0x24420001, 0xaee2018c,
-0x8002374, 0x8ee2018c, 0x8f82011c, 0x34420002,
-0xaf82011c, 0x8f830104, 0x8f8200b0, 0x34420001,
-0xaf8200b0, 0xaf830104, 0x8f82011c, 0x2403fffd,
-0x431024, 0xaf82011c, 0x8ee201a0, 0x24420001,
-0xaee201a0, 0x8002377, 0x8ee201a0, 0x8f8200b0,
-0x34420001, 0xaf8200b0, 0x8fbf0018, 0x3e00008,
-0x27bd0020, 0x27bdffe0, 0xafbf001c, 0xafb00018,
-0x8f820120, 0xafa20010, 0x8f820124, 0x3c050001,
-0xafa20014, 0x8f8600a0, 0x8f87011c, 0x3c040001,
-0x248454cc, 0xc002403, 0x34a5f000, 0x8f8300a0,
-0x3c027f00, 0x621824, 0x3c020400, 0x10620053,
-0x8021, 0x43102b, 0x14400008, 0x3c042000,
-0x3c020100, 0x1062004d, 0x3c020200, 0x1062003a,
-0x0, 0x80023e0, 0x0, 0x10640003,
-0x3c024000, 0x14620045, 0x0, 0x8f8200a0,
-0x441024, 0x10400006, 0x0, 0x8ee20194,
-0x24420001, 0xaee20194, 0x80023a9, 0x8ee20194,
-0x8ee20198, 0x24420001, 0xaee20198, 0x8ee20198,
-0x8f82011c, 0x34420002, 0xaf82011c, 0x8f82011c,
-0x30420200, 0x1040001b, 0x0, 0x8f8300a0,
-0x8f840124, 0x8f8200ac, 0x14400007, 0x24020001,
-0x3c020001, 0x3442f000, 0x621024, 0x50400001,
-0x24100001, 0x24020001, 0x1200000d, 0xaf8200a0,
-0x8f820124, 0x2442ffe0, 0xaf820124, 0x8f820124,
-0x8f820124, 0x27633000, 0x43102b, 0x10400005,
-0x276237e0, 0xaf820124, 0x80023ca, 0x0,
-0xaf840124, 0x8f82011c, 0x2403fffd, 0x431024,
-0x80023e3, 0xaf82011c, 0x8f82011c, 0x34420002,
-0xaf82011c, 0x8f830124, 0x8f8200a0, 0x34420001,
-0xaf8200a0, 0xaf830124, 0x8f82011c, 0x2403fffd,
-0x431024, 0xaf82011c, 0x8ee2019c, 0x24420001,
-0xaee2019c, 0x80023e3, 0x8ee2019c, 0x8f8200a0,
-0x34420001, 0xaf8200a0, 0x8fbf001c, 0x8fb00018,
-0x3e00008, 0x27bd0020, 0x0, 0x3c020001,
-0x8c425c58, 0x27bdffe8, 0xafbf0014, 0x14400012,
-0xafb00010, 0x3c100001, 0x26105dd0, 0x2002021,
-0xc002488, 0x24052000, 0x26021fe0, 0x3c010001,
-0xac225d94, 0x3c010001, 0xac225d90, 0xaf420250,
-0x24022000, 0xaf500254, 0xaf420258, 0x24020001,
-0x3c010001, 0xac225c58, 0x8fbf0014, 0x8fb00010,
-0x3e00008, 0x27bd0018, 0x3c030001, 0x8c635d94,
-0x8c820000, 0x8fa80010, 0x8fa90014, 0xac620000,
-0x3c020001, 0x8c425d94, 0x8c830004, 0xac430004,
-0xac450008, 0x8f840054, 0x2443ffe0, 0xac460010,
-0xac470014, 0xac480018, 0xac49001c, 0x3c010001,
-0xac235d94, 0xac44000c, 0x3c020001, 0x24425dd0,
-0x62182b, 0x10600005, 0x0, 0x3c020001,
-0x8c425d90, 0x3c010001, 0xac225d94, 0x3c030001,
-0x8c635d94, 0x3c020001, 0x8c425c40, 0xac620000,
-0x3c030001, 0x8c635d94, 0x3c020001, 0x8c425c40,
-0xac620004, 0x3e00008, 0xaf430250, 0x3c030001,
-0x8c635d94, 0x3c020001, 0x8c425c40, 0x27bdffd0,
-0xafb40020, 0x8fb40040, 0xafb00010, 0x808021,
-0xafb50024, 0x8fb50044, 0x8fa40048, 0xafb10014,
-0xa08821, 0xafbf0028, 0xafb3001c, 0xafb20018,
-0xac620000, 0x3c050001, 0x8ca55d94, 0x3c020001,
-0x8c425c40, 0xc09021, 0xe09821, 0x10800006,
-0xaca20004, 0x24a50008, 0xc002490, 0x24060018,
-0x800244e, 0x0, 0x24a40008, 0xc002488,
-0x24050018, 0x3c020001, 0x8c425d94, 0x3c050001,
-0x24a55dd0, 0x2442ffe0, 0x3c010001, 0xac225d94,
-0x45102b, 0x10400005, 0x0, 0x3c020001,
-0x8c425d90, 0x3c010001, 0xac225d94, 0x3c030001,
-0x8c635d94, 0x8e020000, 0xac620000, 0x3c030001,
-0x8c635d94, 0x8e020004, 0xac620004, 0xac710008,
-0x8f840054, 0x2462ffe0, 0x3c010001, 0xac225d94,
-0x45102b, 0xac720010, 0xac730014, 0xac740018,
-0xac75001c, 0x10400005, 0xac64000c, 0x3c020001,
-0x8c425d90, 0x3c010001, 0xac225d94, 0x3c030001,
-0x8c635d94, 0x3c020001, 0x8c425c40, 0xac620000,
-0x3c030001, 0x8c635d94, 0x3c020001, 0x8c425c40,
-0xac620004, 0xaf430250, 0x8fbf0028, 0x8fb50024,
-0x8fb40020, 0x8fb3001c, 0x8fb20018, 0x8fb10014,
-0x8fb00010, 0x3e00008, 0x27bd0030, 0x10a00005,
-0x0, 0xac800000, 0x24a5fffc, 0x14a0fffd,
-0x24840004, 0x3e00008, 0x0, 0x10c00007,
-0x0, 0x8c820000, 0x24840004, 0x24c6fffc,
-0xaca20000, 0x14c0fffb, 0x24a50004, 0x3e00008,
-0x0, 0x10c00007, 0x0, 0x8ca20000,
-0x24a50004, 0x24c6fffc, 0xac820000, 0x14c0fffb,
-0x24840004, 0x3e00008, 0x0, 0x3e00008,
-0x0, 0x27bdffd8, 0xafbf0020, 0x8ee304e4,
-0x8ee204e0, 0x10620436, 0x0, 0x8ee204e4,
-0x8ee304fc, 0x21100, 0x626021, 0x95870008,
-0x8d8a0000, 0x8d8b0004, 0x958d000a, 0x8ee2725c,
-0x8ee3726c, 0x30e4ffff, 0x441021, 0x62182b,
-0x10600015, 0x31a20004, 0x8f8200d8, 0x8ee37258,
-0x431023, 0xaee2726c, 0x8ee2726c, 0x1c400003,
-0x3c030001, 0x431021, 0xaee2726c, 0x8ee2725c,
-0x8ee3726c, 0x441021, 0x62182b, 0x10600006,
-0x31a20004, 0x8ee201b8, 0x24420001, 0xaee201b8,
-0x80028e1, 0x8ee201b8, 0x10400240, 0x31a20200,
-0x1040014d, 0x4821, 0x96e2045a, 0x30420010,
-0x10400149, 0x0, 0x8f840100, 0x27623000,
-0x24850020, 0xa2102b, 0x50400001, 0x27652800,
-0x8f820108, 0x10a20004, 0x0, 0x8f820104,
-0x14a20006, 0x2402000c, 0x8ee201a8, 0x24420001,
-0xaee201a8, 0x800252c, 0x8ee201a8, 0xac8a0000,
-0xac8b0004, 0x8ee37264, 0x24060005, 0xa482000e,
-0xac860018, 0xac830008, 0x8ee204e4, 0xac82001c,
-0x8ee204c8, 0xac820010, 0xaf850100, 0x92e204ec,
-0x14400036, 0x24090001, 0x8ee24e28, 0x210c0,
-0x24424e38, 0x2e22021, 0x8c820000, 0x1446001f,
-0x0, 0x8ee34e28, 0x8ee24e2c, 0x1062001b,
-0x24030040, 0x8c820004, 0x24420001, 0xac820004,
-0x8ee24e2c, 0x8ee54e28, 0x24420001, 0x10430007,
-0x0, 0x8ee24e2c, 0x24420001, 0x10a20005,
-0x0, 0x8002516, 0x0, 0x14a00005,
-0x0, 0x8f820108, 0x24420020, 0xaf820108,
-0x8f820108, 0x8c820004, 0x2c420011, 0x50400013,
-0xac800000, 0x800252c, 0x0, 0x8ee24e28,
-0x24030040, 0x24420001, 0x50430003, 0x1021,
-0x8ee24e28, 0x24420001, 0xaee24e28, 0x8ee24e28,
-0x210c0, 0x24424e38, 0x2e22021, 0x24020005,
-0xac820000, 0x24020001, 0xac820004, 0x1520000a,
-0x3c040001, 0xafab0010, 0x8ee27264, 0x3c040001,
-0x24845730, 0x3c050004, 0xafa20014, 0x8ee604e4,
-0x80028be, 0x34a5f114, 0x8ee27264, 0x34843800,
-0x3641821, 0x24420010, 0x43102b, 0x14400073,
-0x0, 0x8ee27264, 0x24480010, 0x3641021,
-0x102102b, 0x14400002, 0x3c02ffff, 0x1024021,
-0x8f850100, 0x27623000, 0x24a60020, 0xc2102b,
-0x50400001, 0x27662800, 0x8f820108, 0x10c20004,
-0x0, 0x8f820104, 0x14c20007, 0x2563000c,
-0x8ee201a8, 0x4821, 0x24420001, 0xaee201a8,
-0x80025a0, 0x8ee201a8, 0x2c64000c, 0x1441021,
-0xaca20000, 0xaca30004, 0x24e2fff4, 0xa4a2000e,
-0x24020006, 0xaca80008, 0xaca20018, 0x8ee204e4,
-0xaca2001c, 0x8ee204c8, 0x3c030002, 0x431025,
-0xaca20010, 0xaf860100, 0x92e204ec, 0x14400037,
-0x24090001, 0x8ee24e28, 0x210c0, 0x24424e38,
-0x2e22021, 0x8c830000, 0x24020005, 0x1462001f,
-0x0, 0x8ee34e28, 0x8ee24e2c, 0x1062001b,
-0x24030040, 0x8c820004, 0x24420001, 0xac820004,
-0x8ee24e2c, 0x8ee54e28, 0x24420001, 0x10430007,
-0x0, 0x8ee24e2c, 0x24420001, 0x10a20005,
-0x0, 0x800258a, 0x0, 0x14a00005,
-0x0, 0x8f820108, 0x24420020, 0xaf820108,
-0x8f820108, 0x8c820004, 0x2c420011, 0x50400013,
-0xac800000, 0x80025a0, 0x0, 0x8ee24e28,
-0x24030040, 0x24420001, 0x50430003, 0x1021,
-0x8ee24e28, 0x24420001, 0xaee24e28, 0x8ee24e28,
-0x210c0, 0x24424e38, 0x2e22021, 0x24020005,
-0xac820000, 0x24020001, 0xac820004, 0x1520000a,
-0x2508fffc, 0xafab0010, 0x8ee27264, 0x3c040001,
-0x24845730, 0x3c050004, 0xafa20014, 0x8ee604e4,
-0x80028be, 0x34a5f125, 0x34028100, 0xa5020000,
-0x9582000e, 0x800261d, 0xa5020002, 0x8f850100,
-0x27623000, 0x24a60020, 0xc2102b, 0x50400001,
-0x27662800, 0x8f820108, 0x10c20004, 0x0,
-0x8f820104, 0x14c20007, 0x2563000c, 0x8ee201a8,
-0x4821, 0x24420001, 0xaee201a8, 0x800260d,
-0x8ee201a8, 0x2c64000c, 0x1441021, 0xaca20000,
-0xaca30004, 0x8ee37264, 0x24e2fff4, 0xa4a2000e,
-0x24020006, 0xaca20018, 0x24630010, 0xaca30008,
-0x8ee204e4, 0xaca2001c, 0x8ee204c8, 0x3c030002,
-0x431025, 0xaca20010, 0xaf860100, 0x92e204ec,
-0x14400037, 0x24090001, 0x8ee24e28, 0x210c0,
-0x24424e38, 0x2e22021, 0x8c830000, 0x24020005,
-0x1462001f, 0x0, 0x8ee34e28, 0x8ee24e2c,
-0x1062001b, 0x24030040, 0x8c820004, 0x24420001,
-0xac820004, 0x8ee24e2c, 0x8ee54e28, 0x24420001,
-0x10430007, 0x0, 0x8ee24e2c, 0x24420001,
-0x10a20005, 0x0, 0x80025f7, 0x0,
-0x14a00005, 0x0, 0x8f820108, 0x24420020,
-0xaf820108, 0x8f820108, 0x8c820004, 0x2c420011,
-0x50400013, 0xac800000, 0x800260d, 0x0,
-0x8ee24e28, 0x24030040, 0x24420001, 0x50430003,
-0x1021, 0x8ee24e28, 0x24420001, 0xaee24e28,
-0x8ee24e28, 0x210c0, 0x24424e38, 0x2e22021,
-0x24020005, 0xac820000, 0x24020001, 0xac820004,
-0x1520000a, 0x34028100, 0xafab0010, 0x8ee27264,
-0x3c040001, 0x24845730, 0x3c050004, 0xafa20014,
-0x8ee604e4, 0x80028be, 0x34a5f015, 0x8ee37264,
-0xa462000c, 0x8ee37264, 0x9582000e, 0xa462000e,
-0x8002681, 0x24e70004, 0x8f840100, 0x27623000,
-0x24850020, 0xa2102b, 0x50400001, 0x27652800,
-0x8f820108, 0x10a20004, 0x0, 0x8f820104,
-0x14a20007, 0x24020006, 0x8ee201a8, 0x4821,
-0x24420001, 0xaee201a8, 0x8002677, 0x8ee201a8,
-0xac8a0000, 0xac8b0004, 0x8ee37264, 0xa487000e,
-0xac820018, 0xac830008, 0x8ee204e4, 0xac82001c,
-0x8ee204c8, 0x3c030002, 0x431025, 0xac820010,
-0xaf850100, 0x92e204ec, 0x14400037, 0x24090001,
-0x8ee24e28, 0x210c0, 0x24424e38, 0x2e22021,
-0x8c830000, 0x24020005, 0x1462001f, 0x0,
-0x8ee34e28, 0x8ee24e2c, 0x1062001b, 0x24030040,
-0x8c820004, 0x24420001, 0xac820004, 0x8ee24e2c,
-0x8ee54e28, 0x24420001, 0x10430007, 0x0,
-0x8ee24e2c, 0x24420001, 0x10a20005, 0x0,
-0x8002661, 0x0, 0x14a00005, 0x0,
-0x8f820108, 0x24420020, 0xaf820108, 0x8f820108,
-0x8c820004, 0x2c420011, 0x50400013, 0xac800000,
-0x8002677, 0x0, 0x8ee24e28, 0x24030040,
-0x24420001, 0x50430003, 0x1021, 0x8ee24e28,
-0x24420001, 0xaee24e28, 0x8ee24e28, 0x210c0,
-0x24424e38, 0x2e22021, 0x24020005, 0xac820000,
-0x24020001, 0xac820004, 0x15200009, 0x3c050004,
-0xafab0010, 0x8ee27264, 0x3c040001, 0x24845730,
-0xafa20014, 0x8ee604e4, 0x80028be, 0x34a5f004,
-0x8ee2725c, 0x30e7ffff, 0x471021, 0xaee2725c,
-0x8ee204e4, 0x8ee304fc, 0x8ee47258, 0x21100,
-0x431021, 0xac44000c, 0x8ee27258, 0xafa20018,
-0x8ee3725c, 0xafa3001c, 0x8ee2725c, 0x2c42003c,
-0x10400004, 0x24620001, 0x2403fffe, 0x431024,
-0xafa2001c, 0x8ee27264, 0x3c060001, 0x34c63800,
-0x8ee3725c, 0x2405fff8, 0x471021, 0x24420007,
-0x451024, 0x24630007, 0xaee27258, 0x8ee2726c,
-0x8ee47258, 0x651824, 0x431023, 0xaee2726c,
-0x3661021, 0x82202b, 0x14800004, 0x3c03ffff,
-0x8ee27258, 0x431021, 0xaee27258, 0x8ee27258,
-0xaee27264, 0x8f8200f0, 0x24470008, 0x27621800,
-0xe2102b, 0x50400001, 0x27671000, 0x8f8200f4,
-0x14e20007, 0x0, 0x8ee201b4, 0x4821,
-0x24420001, 0xaee201b4, 0x80026c4, 0x8ee201b4,
-0x8f8200f0, 0x24090001, 0x8fa30018, 0x8fa4001c,
-0xac430000, 0xac440004, 0xaf8700f0, 0x15200012,
-0xd1142, 0x8f8200f0, 0xafa20010, 0x8f8200f4,
-0x3c040001, 0x2484573c, 0xafa20014, 0x8fa60018,
-0x8fa7001c, 0x3c050004, 0xc002403, 0x34a5f005,
-0x8ee20088, 0x24420001, 0xaee20088, 0x8ee20088,
-0x80028d3, 0xaee0725c, 0x30430003, 0x24020002,
-0x10620016, 0x28620003, 0x10400005, 0x24020001,
-0x10620008, 0x0, 0x8002703, 0x0,
-0x24020003, 0x10620017, 0x0, 0x8002703,
-0x0, 0x8ee200e8, 0x8ee300ec, 0x24630001,
-0x2c640001, 0x441021, 0xaee200e8, 0xaee300ec,
-0x8ee200e8, 0x8002703, 0x8ee300ec, 0x8ee200f0,
-0x8ee300f4, 0x24630001, 0x2c640001, 0x441021,
-0xaee200f0, 0xaee300f4, 0x8ee200f0, 0x8002703,
-0x8ee300f4, 0x8ee200f8, 0x8ee300fc, 0x24630001,
-0x2c640001, 0x441021, 0xaee200f8, 0xaee300fc,
-0x8ee200f8, 0x8ee300fc, 0x8ee2725c, 0x8ee400e0,
-0x8ee500e4, 0x401821, 0x1021, 0xa32821,
-0xa3302b, 0x822021, 0x862021, 0xaee400e0,
-0xaee500e4, 0x80028d3, 0xaee0725c, 0x30e2ffff,
-0x104001c1, 0x31a20200, 0x1040014d, 0x4821,
-0x96e2045a, 0x30420010, 0x10400149, 0x0,
-0x8f840100, 0x27623000, 0x24850020, 0xa2102b,
-0x50400001, 0x27652800, 0x8f820108, 0x10a20004,
-0x0, 0x8f820104, 0x14a20006, 0x2402000c,
-0x8ee201a8, 0x24420001, 0xaee201a8, 0x800276e,
-0x8ee201a8, 0xac8a0000, 0xac8b0004, 0x8ee37264,
-0x24060005, 0xa482000e, 0xac860018, 0xac830008,
-0x8ee204e4, 0xac82001c, 0x8ee204c8, 0xac820010,
-0xaf850100, 0x92e204ec, 0x14400036, 0x24090001,
-0x8ee24e28, 0x210c0, 0x24424e38, 0x2e22021,
-0x8c820000, 0x1446001f, 0x0, 0x8ee34e28,
-0x8ee24e2c, 0x1062001b, 0x24030040, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee24e2c, 0x8ee54e28,
-0x24420001, 0x10430007, 0x0, 0x8ee24e2c,
-0x24420001, 0x10a20005, 0x0, 0x8002758,
-0x0, 0x14a00005, 0x0, 0x8f820108,
-0x24420020, 0xaf820108, 0x8f820108, 0x8c820004,
-0x2c420011, 0x50400013, 0xac800000, 0x800276e,
-0x0, 0x8ee24e28, 0x24030040, 0x24420001,
-0x50430003, 0x1021, 0x8ee24e28, 0x24420001,
-0xaee24e28, 0x8ee24e28, 0x210c0, 0x24424e38,
-0x2e22021, 0x24020005, 0xac820000, 0x24020001,
-0xac820004, 0x1520000a, 0x3c040001, 0xafab0010,
-0x8ee27264, 0x3c040001, 0x24845730, 0x3c050004,
-0xafa20014, 0x8ee604e4, 0x80028be, 0x34a5f014,
-0x8ee27264, 0x34843800, 0x3641821, 0x24420010,
-0x43102b, 0x14400073, 0x0, 0x8ee27264,
-0x24480010, 0x3641021, 0x102102b, 0x14400002,
-0x3c02ffff, 0x1024021, 0x8f850100, 0x27623000,
-0x24a60020, 0xc2102b, 0x50400001, 0x27662800,
-0x8f820108, 0x10c20004, 0x0, 0x8f820104,
-0x14c20007, 0x2563000c, 0x8ee201a8, 0x4821,
-0x24420001, 0xaee201a8, 0x80027e2, 0x8ee201a8,
-0x2c64000c, 0x1441021, 0xaca20000, 0xaca30004,
-0x24e2fff4, 0xa4a2000e, 0x24020006, 0xaca80008,
-0xaca20018, 0x8ee204e4, 0xaca2001c, 0x8ee204c8,
-0x3c030002, 0x431025, 0xaca20010, 0xaf860100,
-0x92e204ec, 0x14400037, 0x24090001, 0x8ee24e28,
-0x210c0, 0x24424e38, 0x2e22021, 0x8c830000,
-0x24020005, 0x1462001f, 0x0, 0x8ee34e28,
-0x8ee24e2c, 0x1062001b, 0x24030040, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee24e2c, 0x8ee54e28,
-0x24420001, 0x10430007, 0x0, 0x8ee24e2c,
-0x24420001, 0x10a20005, 0x0, 0x80027cc,
-0x0, 0x14a00005, 0x0, 0x8f820108,
-0x24420020, 0xaf820108, 0x8f820108, 0x8c820004,
-0x2c420011, 0x50400013, 0xac800000, 0x80027e2,
-0x0, 0x8ee24e28, 0x24030040, 0x24420001,
-0x50430003, 0x1021, 0x8ee24e28, 0x24420001,
-0xaee24e28, 0x8ee24e28, 0x210c0, 0x24424e38,
-0x2e22021, 0x24020005, 0xac820000, 0x24020001,
-0xac820004, 0x1520000a, 0x2508fffc, 0xafab0010,
-0x8ee27264, 0x3c040001, 0x24845730, 0x3c050004,
-0xafa20014, 0x8ee604e4, 0x80028be, 0x34a5f015,
-0x34028100, 0xa5020000, 0x9582000e, 0x800285f,
-0xa5020002, 0x8f850100, 0x27623000, 0x24a60020,
-0xc2102b, 0x50400001, 0x27662800, 0x8f820108,
-0x10c20004, 0x0, 0x8f820104, 0x14c20007,
-0x2563000c, 0x8ee201a8, 0x4821, 0x24420001,
-0xaee201a8, 0x800284f, 0x8ee201a8, 0x2c64000c,
-0x1441021, 0xaca20000, 0xaca30004, 0x8ee37264,
-0x24e2fff4, 0xa4a2000e, 0x24020006, 0xaca20018,
-0x24630010, 0xaca30008, 0x8ee204e4, 0xaca2001c,
-0x8ee204c8, 0x3c030002, 0x431025, 0xaca20010,
-0xaf860100, 0x92e204ec, 0x14400037, 0x24090001,
-0x8ee24e28, 0x210c0, 0x24424e38, 0x2e22021,
-0x8c830000, 0x24020005, 0x1462001f, 0x0,
-0x8ee34e28, 0x8ee24e2c, 0x1062001b, 0x24030040,
-0x8c820004, 0x24420001, 0xac820004, 0x8ee24e2c,
-0x8ee54e28, 0x24420001, 0x10430007, 0x0,
-0x8ee24e2c, 0x24420001, 0x10a20005, 0x0,
-0x8002839, 0x0, 0x14a00005, 0x0,
-0x8f820108, 0x24420020, 0xaf820108, 0x8f820108,
-0x8c820004, 0x2c420011, 0x50400013, 0xac800000,
-0x800284f, 0x0, 0x8ee24e28, 0x24030040,
-0x24420001, 0x50430003, 0x1021, 0x8ee24e28,
-0x24420001, 0xaee24e28, 0x8ee24e28, 0x210c0,
-0x24424e38, 0x2e22021, 0x24020005, 0xac820000,
-0x24020001, 0xac820004, 0x1520000a, 0x34028100,
-0xafab0010, 0x8ee27264, 0x3c040001, 0x24845730,
-0x3c050004, 0xafa20014, 0x8ee604e4, 0x80028be,
-0x34a5f016, 0x8ee37264, 0xa462000c, 0x8ee37264,
-0x9582000e, 0xa462000e, 0x80028c2, 0x24e70004,
-0x8f830100, 0x27623000, 0x24640020, 0x82102b,
-0x50400001, 0x27642800, 0x8f820108, 0x10820004,
-0x0, 0x8f820104, 0x14820007, 0x24050005,
-0x8ee201a8, 0x4821, 0x24420001, 0xaee201a8,
-0x80028b6, 0x8ee201a8, 0xac6a0000, 0xac6b0004,
-0x8ee27264, 0xa467000e, 0xac650018, 0xac620008,
-0x8ee204e4, 0xac62001c, 0x8ee204c8, 0xac620010,
-0xaf840100, 0x92e204ec, 0x14400036, 0x24090001,
-0x8ee24e28, 0x210c0, 0x24424e38, 0x2e22021,
-0x8c820000, 0x1445001f, 0x0, 0x8ee34e28,
-0x8ee24e2c, 0x1062001b, 0x24030040, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee24e2c, 0x8ee54e28,
-0x24420001, 0x10430007, 0x0, 0x8ee24e2c,
-0x24420001, 0x10a20005, 0x0, 0x80028a0,
-0x0, 0x14a00005, 0x0, 0x8f820108,
-0x24420020, 0xaf820108, 0x8f820108, 0x8c820004,
-0x2c420011, 0x50400013, 0xac800000, 0x80028b6,
-0x0, 0x8ee24e28, 0x24030040, 0x24420001,
-0x50430003, 0x1021, 0x8ee24e28, 0x24420001,
-0xaee24e28, 0x8ee24e28, 0x210c0, 0x24424e38,
-0x2e22021, 0x24020005, 0xac820000, 0x24020001,
-0xac820004, 0x1520000b, 0x3c050004, 0x3c040001,
-0x24845748, 0xafab0010, 0xafa00014, 0x8ee604e4,
-0x34a5f017, 0xc002403, 0x30e7ffff, 0x80028e1,
-0x0, 0x8ee27264, 0x3c050001, 0x30e4ffff,
-0x441021, 0xaee27264, 0x8ee2725c, 0x8ee37264,
-0x34a53800, 0x441021, 0xaee2725c, 0x3651021,
-0x62182b, 0x14600004, 0x3c03ffff, 0x8ee27264,
-0x431021, 0xaee27264, 0x8ee304e4, 0x96e20458,
-0x24630001, 0x2442ffff, 0x621824, 0xaee304e4,
-0x8ee304e4, 0x8ee204e0, 0x14620005, 0x0,
-0x8f820060, 0x2403fff7, 0x431024, 0xaf820060,
-0x8fbf0020, 0x3e00008, 0x27bd0028, 0x27bdffe0,
-0xafbf0018, 0x8ee304e8, 0x8ee204e0, 0x10620189,
-0x0, 0x8ee204e8, 0x8ee304fc, 0x21100,
-0x621821, 0x94670008, 0x92e204ed, 0x8c680000,
-0x8c690004, 0x10400023, 0x946a000a, 0x8ee204c8,
-0x34460400, 0x31420200, 0x1040001f, 0x0,
-0x96e2045a, 0x30420010, 0x1040001b, 0x3c028000,
-0x3c010001, 0x370821, 0xac2283d8, 0x8ee27264,
-0x9464000e, 0x3c050001, 0x34a53800, 0x24420004,
-0xaee27264, 0x8ee37264, 0x42400, 0x3651021,
-0x3c010001, 0x370821, 0xac2483dc, 0x62182b,
-0x14600005, 0x24e70004, 0x8ee27264, 0x3c03ffff,
-0x431021, 0xaee27264, 0x8ee27264, 0x8002917,
-0xaee27258, 0x8ee604c8, 0x8ee2726c, 0x30e4ffff,
-0x44102a, 0x10400015, 0x0, 0x8f8200d8,
-0x8ee37258, 0x431023, 0xaee2726c, 0x8ee2726c,
-0x1c400007, 0x44102a, 0x8ee2726c, 0x3c030001,
-0x431021, 0xaee2726c, 0x8ee2726c, 0x44102a,
-0x10400006, 0x0, 0x8ee201b8, 0x24420001,
-0xaee201b8, 0x8002a72, 0x8ee201b8, 0x3c020001,
-0x571021, 0x8c4283d8, 0x54400001, 0x24e7fffc,
-0x31420004, 0x104000b9, 0x30e2ffff, 0x3c020001,
-0x571021, 0x8c4283d8, 0x1040002f, 0x5021,
-0x8f840100, 0x27623000, 0x24850020, 0xa2102b,
-0x50400001, 0x27652800, 0x8f820108, 0x10a20032,
-0x0, 0x8f820104, 0x10a2002f, 0x24020015,
-0xac880000, 0xac890004, 0x8ee37264, 0xa487000e,
-0xac820018, 0xac830008, 0x8ee204e8, 0x3c030001,
-0x771821, 0x8c6383dc, 0xac860010, 0x431025,
-0xac82001c, 0xaf850100, 0x92e204ec, 0x14400066,
-0x240a0001, 0x8ee24e28, 0x24030040, 0x24420001,
-0x50430003, 0x1021, 0x8ee24e28, 0x24420001,
-0xaee24e28, 0x8ee24e28, 0x210c0, 0x24424e38,
-0x2e21821, 0x24020015, 0xac620000, 0x24020001,
-0x80029bf, 0xac620004, 0x8f840100, 0x27623000,
-0x24850020, 0xa2102b, 0x50400001, 0x27652800,
-0x8f820108, 0x10a20004, 0x0, 0x8f820104,
-0x14a20006, 0x24020006, 0x8ee201a8, 0x24420001,
-0xaee201a8, 0x80029bf, 0x8ee201a8, 0xac880000,
-0xac890004, 0x8ee37264, 0xa487000e, 0xac820018,
-0xac830008, 0x8ee204e8, 0xac860010, 0xac82001c,
-0xaf850100, 0x92e204ec, 0x14400037, 0x240a0001,
-0x8ee24e28, 0x210c0, 0x24424e38, 0x2e22021,
-0x8c830000, 0x24020005, 0x1462001f, 0x0,
-0x8ee34e28, 0x8ee24e2c, 0x1062001b, 0x24030040,
-0x8c820004, 0x24420001, 0xac820004, 0x8ee24e2c,
-0x8ee54e28, 0x24420001, 0x10430007, 0x0,
-0x8ee24e2c, 0x24420001, 0x10a20005, 0x0,
-0x80029a9, 0x0, 0x14a00005, 0x0,
-0x8f820108, 0x24420020, 0xaf820108, 0x8f820108,
-0x8c820004, 0x2c420011, 0x50400013, 0xac800000,
-0x80029bf, 0x0, 0x8ee24e28, 0x24030040,
-0x24420001, 0x50430003, 0x1021, 0x8ee24e28,
-0x24420001, 0xaee24e28, 0x8ee24e28, 0x210c0,
-0x24424e38, 0x2e22021, 0x24020005, 0xac820000,
-0x24020001, 0xac820004, 0x1540000a, 0x24020001,
-0xafa90010, 0x8ee27264, 0x3c040001, 0x24845730,
-0x3c050004, 0xafa20014, 0x8ee604e4, 0x8002a4f,
-0x34a5f204, 0xa2e204ed, 0x8ee204e8, 0x8ee304fc,
-0x8ee47258, 0x3c060001, 0x34c63800, 0x3c010001,
-0x370821, 0xac2083d8, 0x3c010001, 0x370821,
-0xac2083dc, 0x21100, 0x431021, 0xac44000c,
-0x8ee27264, 0x2405fff8, 0x30e3ffff, 0x431021,
-0x24420007, 0x451024, 0x24630007, 0xaee27258,
-0x8ee2726c, 0x8ee47258, 0x651824, 0x431023,
-0xaee2726c, 0x3661021, 0x82202b, 0x14800004,
-0x3c03ffff, 0x8ee27258, 0x431021, 0xaee27258,
-0x8ee27258, 0x8002a64, 0xaee27264, 0x10400073,
-0x0, 0x8f830100, 0x27623000, 0x24640020,
-0x82102b, 0x14400002, 0x5021, 0x27642800,
-0x8f820108, 0x10820004, 0x0, 0x8f820104,
-0x14820006, 0x24050005, 0x8ee201a8, 0x24420001,
-0xaee201a8, 0x8002a46, 0x8ee201a8, 0xac680000,
-0xac690004, 0x8ee27264, 0xa467000e, 0xac650018,
-0xac620008, 0x8ee204e8, 0xac660010, 0xac62001c,
-0xaf840100, 0x92e204ec, 0x14400036, 0x240a0001,
-0x8ee24e28, 0x210c0, 0x24424e38, 0x2e22021,
-0x8c820000, 0x1445001f, 0x0, 0x8ee34e28,
-0x8ee24e2c, 0x1062001b, 0x24030040, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee24e2c, 0x8ee54e28,
-0x24420001, 0x10430007, 0x0, 0x8ee24e2c,
-0x24420001, 0x10a20005, 0x0, 0x8002a30,
-0x0, 0x14a00005, 0x0, 0x8f820108,
-0x24420020, 0xaf820108, 0x8f820108, 0x8c820004,
-0x2c420011, 0x50400013, 0xac800000, 0x8002a46,
-0x0, 0x8ee24e28, 0x24030040, 0x24420001,
-0x50430003, 0x1021, 0x8ee24e28, 0x24420001,
-0xaee24e28, 0x8ee24e28, 0x210c0, 0x24424e38,
-0x2e22021, 0x24020005, 0xac820000, 0x24020001,
-0xac820004, 0x1540000c, 0x30e5ffff, 0x3c040001,
-0x24845748, 0x3c050004, 0xafa90010, 0xafa00014,
-0x8ee604e4, 0x34a5f237, 0xc002403, 0x30e7ffff,
-0x8002a72, 0x0, 0x8ee27264, 0x451021,
-0xaee27264, 0x8ee2726c, 0x8ee37264, 0x3c040001,
-0x34843800, 0xa2e004ed, 0x451023, 0xaee2726c,
-0x3641021, 0x62182b, 0x14600004, 0x3c03ffff,
-0x8ee27264, 0x431021, 0xaee27264, 0x8ee304e8,
-0x96e20458, 0x24630001, 0x2442ffff, 0x621824,
-0xaee304e8, 0x8ee304e8, 0x8ee204e0, 0x14620005,
-0x0, 0x8f820060, 0x2403fff7, 0x431024,
-0xaf820060, 0x8fbf0018, 0x3e00008, 0x27bd0020,
-0x27bdffe0, 0xafbf001c, 0xafb00018, 0x8f820100,
-0x8ee34e2c, 0x8f820104, 0x8f850108, 0x24020040,
-0x24630001, 0x50620003, 0x1021, 0x8ee24e2c,
-0x24420001, 0xaee24e2c, 0x8ee24e2c, 0x8ee34e2c,
-0x210c0, 0x24424e38, 0x2e22021, 0x8ee24e28,
-0x8c870004, 0x14620007, 0xa03021, 0x8f820108,
-0x24420020, 0xaf820108, 0x8f820108, 0x8002aa2,
-0xac800000, 0x8ee24e2c, 0x24030040, 0x24420001,
-0x50430003, 0x1021, 0x8ee24e2c, 0x24420001,
-0x210c0, 0x24424e38, 0x2e22021, 0x8c820004,
-0x8f830108, 0x21140, 0x621821, 0xaf830108,
-0xac800000, 0x8cc20018, 0x2443fffe, 0x2c620013,
-0x104000c1, 0x31080, 0x3c010001, 0x220821,
-0x8c225770, 0x400008, 0x0, 0x8ee204f0,
-0x471021, 0xaee204f0, 0x8ee204f0, 0x8f43023c,
-0x43102b, 0x144000be, 0x0, 0x8ee304e4,
-0x8ee204f8, 0x506200ba, 0xa2e004f4, 0x8f830120,
-0x27623800, 0x24660020, 0xc2102b, 0x50400001,
-0x27663000, 0x8f820128, 0x10c20004, 0x0,
-0x8f820124, 0x14c20007, 0x0, 0x8ee201a4,
-0x8021, 0x24420001, 0xaee201a4, 0x8002b12,
-0x8ee201a4, 0x8ee204e4, 0xac62001c, 0x8ee404b0,
-0x8ee504b4, 0x2462001c, 0xac620008, 0x24020008,
-0xa462000e, 0x24020011, 0xac620018, 0xac640000,
-0xac650004, 0x8ee204c4, 0xac620010, 0xaf860120,
-0x92e24e20, 0x14400037, 0x24100001, 0x8ee24e30,
-0x210c0, 0x24425038, 0x2e22021, 0x8c830000,
-0x24020012, 0x1462001f, 0x0, 0x8ee34e30,
-0x8ee24e34, 0x1062001b, 0x24030040, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee24e34, 0x8ee54e30,
-0x24420001, 0x10430007, 0x0, 0x8ee24e34,
-0x24420001, 0x10a20005, 0x0, 0x8002afc,
-0x0, 0x14a00005, 0x0, 0x8f820128,
-0x24420020, 0xaf820128, 0x8f820128, 0x8c820004,
-0x2c420011, 0x50400013, 0xac800000, 0x8002b12,
-0x0, 0x8ee24e30, 0x24030040, 0x24420001,
-0x50430003, 0x1021, 0x8ee24e30, 0x24420001,
-0xaee24e30, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0x24020012, 0xac820000, 0x24020001,
-0xac820004, 0x5600000b, 0x24100001, 0x8ee204e4,
-0x3c040001, 0x24845754, 0xafa00014, 0xafa20010,
-0x8ee60608, 0x8f470228, 0x3c050009, 0xc002403,
-0x34a5f006, 0x16000003, 0x24020001, 0x8002b71,
-0xa2e204f4, 0x8ee20170, 0x24420001, 0xaee20170,
-0x8ee20170, 0x8ee204e4, 0xa2e004f4, 0xaee004f0,
-0xaee204f8, 0x8f42023c, 0x50400045, 0xaee07274,
-0x8ee20184, 0x24420001, 0xaee20184, 0x8ee20184,
-0x8002b71, 0xaee07274, 0x8ee20504, 0x24030040,
-0x24420001, 0x50430003, 0x1021, 0x8ee20504,
-0x24420001, 0xaee20504, 0x8ee20504, 0x8cc30018,
-0x21080, 0x571021, 0x8c440508, 0x24020003,
-0x1462000f, 0x0, 0x3c020001, 0x571021,
-0x904283b1, 0x10400014, 0x0, 0x8ee201d0,
-0x8ee35240, 0x441021, 0xaee201d0, 0x8ee201d8,
-0x641821, 0x306300ff, 0x8002b59, 0xaee35240,
-0x8ee201cc, 0x8ee30e10, 0x441021, 0xaee201cc,
-0x8ee201d8, 0x641821, 0x306301ff, 0xaee30e10,
-0x441021, 0xaee201d8, 0x8ee20000, 0x34420040,
-0x8002b71, 0xaee20000, 0x8ee2014c, 0x3c010001,
-0x370821, 0xa02083e0, 0x24420001, 0xaee2014c,
-0x8002b71, 0x8ee2014c, 0x94c7000e, 0x8cc2001c,
-0x3c040001, 0x24845760, 0xafa60014, 0xafa20010,
-0x8cc60018, 0x3c050008, 0xc002403, 0x34a50910,
-0x8fbf001c, 0x8fb00018, 0x3e00008, 0x27bd0020,
-0x27bdff98, 0xafbf0060, 0xafbe005c, 0xafb60058,
-0xafb50054, 0xafb40050, 0xafb3004c, 0xafb20048,
-0xafb10044, 0xafb00040, 0x8f830108, 0x8f820104,
-0xafa00024, 0x106203e7, 0xafa0002c, 0x3c1e0001,
-0x37de3800, 0x3c0bffff, 0x8f930108, 0x8e620018,
-0x8f830104, 0x2443fffe, 0x2c620014, 0x104003cf,
-0x31080, 0x3c010001, 0x220821, 0x8c2257c0,
-0x400008, 0x0, 0x9663000e, 0x8ee2725c,
-0x8ee404f0, 0x431021, 0xaee2725c, 0x8e63001c,
-0x96e20458, 0x24840001, 0xaee404f0, 0x24630001,
-0x2442ffff, 0x621824, 0xaee304e4, 0x8f42023c,
-0x82202b, 0x148003b9, 0x0, 0x8f830120,
-0x27623800, 0x24660020, 0xc2102b, 0x50400001,
-0x27663000, 0x8f820128, 0x10c20004, 0x0,
-0x8f820124, 0x14c20007, 0x0, 0x8ee201a4,
-0x8021, 0x24420001, 0xaee201a4, 0x8002bfe,
-0x8ee201a4, 0x8ee204e4, 0xac62001c, 0x8ee404b0,
-0x8ee504b4, 0x2462001c, 0xac620008, 0x24020008,
-0xa462000e, 0x24020011, 0xac620018, 0xac640000,
-0xac650004, 0x8ee204c4, 0xac620010, 0xaf860120,
-0x92e24e20, 0x14400037, 0x24100001, 0x8ee24e30,
-0x210c0, 0x24425038, 0x2e22021, 0x8c830000,
-0x24020012, 0x1462001f, 0x0, 0x8ee34e30,
-0x8ee24e34, 0x1062001b, 0x240c0040, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee24e34, 0x8ee34e30,
-0x24420001, 0x104c0007, 0x0, 0x8ee24e34,
-0x24420001, 0x10620005, 0x0, 0x8002be8,
-0x0, 0x14600005, 0x0, 0x8f820128,
-0x24420020, 0xaf820128, 0x8f820128, 0x8c820004,
-0x2c420011, 0x50400013, 0xac800000, 0x8002bfe,
-0x0, 0x8ee24e30, 0x240c0040, 0x24420001,
-0x504c0003, 0x1021, 0x8ee24e30, 0x24420001,
-0xaee24e30, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0x24020012, 0x240c0001, 0xac820000,
-0xac8c0004, 0x5600000d, 0x24100001, 0x8ee204e4,
-0x3c040001, 0x24845754, 0xafa00014, 0xafa20010,
-0x8ee60608, 0x8f470228, 0x3c050009, 0x34a5f006,
-0xc002403, 0xafab0038, 0x8fab0038, 0x1200030a,
-0x240c0001, 0x8002f19, 0x0, 0x966c001c,
-0xafac002c, 0x9662001e, 0x3c0c8000, 0xafac0024,
-0xae62001c, 0x8e75001c, 0x8ee204fc, 0x8ee404fc,
-0x151900, 0x621021, 0x8c52000c, 0x92e27b98,
-0x641821, 0x9476000a, 0x14400003, 0x32c20002,
-0xaef27ba4, 0xaef57b9c, 0x1040004b, 0x8021,
-0x96e2045a, 0x30420002, 0x10400047, 0x0,
-0x8e63001c, 0x8ee204fc, 0x32100, 0x821021,
-0x8c42000c, 0x37e1821, 0x24420022, 0x43102b,
-0x1440000a, 0x24050014, 0x8ee204fc, 0x821021,
-0x8c44000c, 0xafab0038, 0xc002f75, 0x2484000e,
-0x8fab0038, 0x8002c52, 0x3050ffff, 0x8ee204fc,
-0x821021, 0x8c42000c, 0x9450000e, 0x94430010,
-0x94440012, 0x94450014, 0x2038021, 0x2048021,
-0x2058021, 0x94430016, 0x94440018, 0x9445001a,
-0x2038021, 0x2048021, 0x2058021, 0x9443001c,
-0x9444001e, 0x94420020, 0x2038021, 0x2048021,
-0x2028021, 0x101c02, 0x3202ffff, 0x628021,
-0x8e63001c, 0x8ee204fc, 0x102402, 0x32900,
-0xa21021, 0x8c43000c, 0x3202ffff, 0x828021,
-0x37e1021, 0x24630018, 0x62182b, 0x14600009,
-0x0, 0x8ee204fc, 0xa21021, 0x8c43000c,
-0x101027, 0x3c01ffff, 0x230821, 0x8002c6f,
-0xa4220018, 0x8ee204fc, 0xa21021, 0x8c43000c,
-0x101027, 0xa4620018, 0x96e2045a, 0x8821,
-0x30420008, 0x14400063, 0xa021, 0x8e63001c,
-0x8ee204fc, 0x33100, 0xc21021, 0x8c42000c,
-0x37e1821, 0x24420022, 0x43102b, 0x14400035,
-0x0, 0x8ee204fc, 0xc21021, 0x8c42000c,
-0x24470010, 0x37e1021, 0xe2102b, 0x50400001,
-0xeb3821, 0x8ee204fc, 0x94f10000, 0xc21021,
-0x8c42000c, 0x24470016, 0x37e1021, 0xe2102b,
-0x14400002, 0x2634ffec, 0xeb3821, 0x8ee204fc,
-0x90e30001, 0xc21021, 0x8c42000c, 0x2447001a,
-0x37e1021, 0xe2102b, 0x14400002, 0x2838821,
-0xeb3821, 0x94e20000, 0x24e70002, 0x2228821,
-0x37e1021, 0xe2102b, 0x50400001, 0xeb3821,
-0x94e20000, 0x24e70002, 0x2228821, 0x37e1021,
-0xe2102b, 0x50400001, 0xeb3821, 0x94e20000,
-0x24e70002, 0x2228821, 0x37e1021, 0xe2102b,
-0x50400001, 0xeb3821, 0x94e20000, 0x8002cd0,
-0x2228821, 0x8ee204fc, 0xc21021, 0x8c43000c,
-0x8ee204fc, 0x94710010, 0x8ee304fc, 0xc21021,
-0x8c44000c, 0xc31821, 0x8c62000c, 0x2634ffec,
-0x90840017, 0x8ee304fc, 0x9442001a, 0x2848821,
-0xc31821, 0x8c65000c, 0x8ee304fc, 0x2228821,
-0x8ee204fc, 0xc31821, 0xc21021, 0x8c44000c,
-0x8c62000c, 0x94a3001c, 0x9484001e, 0x94420020,
-0x2238821, 0x2248821, 0x2228821, 0x111c02,
-0x3222ffff, 0x628821, 0x111c02, 0x3222ffff,
-0x628821, 0x32c20001, 0x104000b2, 0x0,
-0x96e2045a, 0x30420001, 0x104000ae, 0x32c20080,
-0x10400008, 0x0, 0x92e27b98, 0x14400005,
-0x0, 0x240c0001, 0xa2ec7b98, 0xaef57b9c,
-0xaef27ba4, 0x8ee304fc, 0x151100, 0x431021,
-0x8c47000c, 0x37e1821, 0x24e2000e, 0x43102b,
-0x14400008, 0xe02021, 0x2405000e, 0xc002f75,
-0xafab0038, 0x3042ffff, 0x8fab0038, 0x8002d09,
-0x2028021, 0x94e60000, 0x24e70002, 0x94e50000,
-0x24e70002, 0x94e30000, 0x24e70002, 0x94e20000,
-0x24e70002, 0x94e40000, 0x24e70002, 0x2068021,
-0x2058021, 0x2038021, 0x2028021, 0x94e20000,
-0x94e30002, 0x2048021, 0x2028021, 0x2038021,
-0x101c02, 0x3202ffff, 0x628021, 0x101c02,
-0x3202ffff, 0x8ee47b9c, 0x628021, 0x14950004,
-0x3205ffff, 0x96620016, 0x8002d17, 0x512021,
-0x96620016, 0x542021, 0x41402, 0x3083ffff,
-0x432021, 0x852023, 0x41402, 0x822021,
-0x3084ffff, 0x50800001, 0x3404ffff, 0x8ee27ba4,
-0x24430017, 0x37e1021, 0x62102b, 0x50400001,
-0x6b1821, 0x90630000, 0x24020011, 0x14620031,
-0x24020006, 0x8ee27ba4, 0x37e1821, 0x24420028,
-0x43102b, 0x14400018, 0x0, 0x8ee27b9c,
-0x12a2000a, 0x32c20100, 0x8ee27ba4, 0x3c01ffff,
-0x220821, 0x94220028, 0x822021, 0x41c02,
-0x3082ffff, 0x622021, 0x32c20100, 0x14400004,
-0x41027, 0x92e27b98, 0x14400002, 0x41027,
-0x3044ffff, 0x8ee27ba4, 0x3c01ffff, 0x220821,
-0x8002d8a, 0xa4240028, 0x8ee27b9c, 0x12a20008,
-0x32c20100, 0x8ee27ba4, 0x94420028, 0x822021,
-0x41c02, 0x3082ffff, 0x622021, 0x32c20100,
-0x14400004, 0x41027, 0x92e27b98, 0x14400002,
-0x41027, 0x3044ffff, 0x8ee27ba4, 0x8002d8a,
-0xa4440028, 0x1462002f, 0x37e1821, 0x8ee27ba4,
-0x24420032, 0x43102b, 0x14400018, 0x0,
-0x8ee27b9c, 0x12a2000a, 0x32c20100, 0x8ee27ba4,
-0x3c01ffff, 0x220821, 0x94220032, 0x822021,
-0x41c02, 0x3082ffff, 0x622021, 0x32c20100,
-0x14400004, 0x41027, 0x92e27b98, 0x14400002,
-0x41027, 0x3044ffff, 0x8ee27ba4, 0x3c01ffff,
-0x220821, 0x8002d8a, 0xa4240032, 0x8ee27b9c,
-0x12a20008, 0x32c20100, 0x8ee27ba4, 0x94420032,
-0x822021, 0x41c02, 0x3082ffff, 0x622021,
-0x32c20100, 0x14400004, 0x41027, 0x92e27b98,
-0x14400002, 0x41027, 0x3044ffff, 0x8ee27ba4,
-0xa4440032, 0x8fac0024, 0x1180002c, 0x37e1821,
-0x8e420000, 0xae42fffc, 0x2642000a, 0x43102b,
-0x1440001b, 0x34038100, 0x26430004, 0x37e1021,
-0x62102b, 0x14400003, 0x602021, 0x6b1821,
-0x602021, 0x8c620000, 0x24630004, 0xae420000,
-0x37e1021, 0x62102b, 0x50400001, 0x6b1821,
-0x8c620000, 0xac820000, 0x34028100, 0xa4620000,
-0x24630002, 0x37e1021, 0x62102b, 0x50400001,
-0x6b1821, 0x97ac002e, 0x8002db4, 0xa46c0000,
-0x8e420004, 0x8e440008, 0xa6430008, 0x97ac002e,
-0xa64c000a, 0xae420000, 0xae440004, 0x9662000e,
-0x2652fffc, 0x24420004, 0xa662000e, 0x9662000e,
-0x8ee3725c, 0x621821, 0xaee3725c, 0xafb20018,
-0x8ee3725c, 0xafa3001c, 0x8ee2725c, 0x2c42003c,
-0x10400004, 0x24620001, 0x2403fffe, 0x431024,
-0xafa2001c, 0x32c20080, 0x1040000c, 0x32c20100,
-0x8ee27ba8, 0x24430001, 0x210c0, 0x571021,
-0xaee37ba8, 0x8fa30018, 0x8fa4001c, 0xac437bac,
-0xac447bb0, 0x8002ea0, 0xaee0725c, 0x10400072,
-0x0, 0x8ee27ba8, 0x24430001, 0x210c0,
-0x571021, 0xaee37ba8, 0x8fa30018, 0x8fa4001c,
-0xac437bac, 0xac447bb0, 0x8ee27ba8, 0x10400063,
-0x4821, 0x5021, 0x8f8200f0, 0x24480008,
-0x27621800, 0x102102b, 0x50400001, 0x27681000,
-0x8f8200f4, 0x15020007, 0x0, 0x8ee201b4,
-0x8021, 0x24420001, 0xaee201b4, 0x8002dfa,
-0x8ee201b4, 0x8f8300f0, 0x24100001, 0x1571021,
-0x8c447bac, 0x8c457bb0, 0xac640000, 0xac650004,
-0xaf8800f0, 0x16000006, 0x2ea1021, 0x8ee20088,
-0x24420001, 0xaee20088, 0x8002e3f, 0x8ee20088,
-0x8c427bb0, 0x8ee400e0, 0x8ee500e4, 0x8ee67b9c,
-0x401821, 0x1021, 0xa32821, 0xa3382b,
-0x822021, 0x872021, 0x8ee204fc, 0xc93021,
-0x63100, 0xaee400e0, 0xaee500e4, 0xc23021,
-0x94c2000a, 0x240c0002, 0x21142, 0x30430003,
-0x106c0016, 0x28620003, 0x10400005, 0x240c0001,
-0x106c0008, 0x0, 0x8002e3f, 0x0,
-0x240c0003, 0x106c0017, 0x0, 0x8002e3f,
-0x0, 0x8ee200e8, 0x8ee300ec, 0x24630001,
-0x2c640001, 0x441021, 0xaee200e8, 0xaee300ec,
-0x8ee200e8, 0x8002e3f, 0x8ee300ec, 0x8ee200f0,
-0x8ee300f4, 0x24630001, 0x2c640001, 0x441021,
-0xaee200f0, 0xaee300f4, 0x8ee200f0, 0x8002e3f,
-0x8ee300f4, 0x8ee200f8, 0x8ee300fc, 0x24630001,
-0x2c640001, 0x441021, 0xaee200f8, 0xaee300fc,
-0x8ee200f8, 0x8ee300fc, 0x8ee27ba8, 0x25290001,
-0x122102b, 0x1440ffa0, 0x254a0008, 0xa2e07b98,
-0x8002e9f, 0xaee07ba8, 0x8f8200f0, 0x24470008,
-0x27621800, 0xe2102b, 0x50400001, 0x27671000,
-0x8f8200f4, 0x14e20007, 0x0, 0x8ee201b4,
-0x8021, 0x24420001, 0xaee201b4, 0x8002e5d,
-0x8ee201b4, 0x8f8200f0, 0x24100001, 0x8fa30018,
-0x8fa4001c, 0xac430000, 0xac440004, 0xaf8700f0,
-0x16000007, 0x0, 0x8ee20088, 0x24420001,
-0xaee20088, 0x8ee20088, 0x8002ea0, 0xaee0725c,
-0x8ee2725c, 0x8ee400e0, 0x8ee500e4, 0x240c0002,
-0x401821, 0x1021, 0xa32821, 0xa3302b,
-0x822021, 0x862021, 0x161142, 0x30430003,
-0xaee400e0, 0xaee500e4, 0x106c0017, 0x2c620003,
-0x10400005, 0x240c0001, 0x106c0008, 0x0,
-0x8002ea0, 0xaee0725c, 0x240c0003, 0x106c0019,
-0x0, 0x8002ea0, 0xaee0725c, 0x8ee200e8,
-0x8ee300ec, 0x24630001, 0x2c640001, 0x441021,
-0xaee200e8, 0xaee300ec, 0x8ee200e8, 0x8ee300ec,
-0x8002ea0, 0xaee0725c, 0x8ee200f0, 0x8ee300f4,
-0x24630001, 0x2c640001, 0x441021, 0xaee200f0,
-0xaee300f4, 0x8ee200f0, 0x8ee300f4, 0x8002ea0,
-0xaee0725c, 0x8ee200f8, 0x8ee300fc, 0x24630001,
-0x2c640001, 0x441021, 0xaee200f8, 0xaee300fc,
-0x8ee200f8, 0x8ee300fc, 0xaee0725c, 0x8e62001c,
-0x96e30458, 0x8ee404f0, 0x24420001, 0x2463ffff,
-0x431024, 0x24840001, 0xaee204e4, 0xaee404f0,
-0x8f42023c, 0x82202b, 0x148000b0, 0x0,
-0x8f830120, 0x27623800, 0x24660020, 0xc2102b,
-0x50400001, 0x27663000, 0x8f820128, 0x10c20004,
-0x0, 0x8f820124, 0x14c20007, 0x0,
-0x8ee201a4, 0x8021, 0x24420001, 0xaee201a4,
-0x8002f07, 0x8ee201a4, 0x8ee204e4, 0xac62001c,
-0x8ee404b0, 0x8ee504b4, 0x2462001c, 0xac620008,
-0x24020008, 0xa462000e, 0x24020011, 0xac620018,
-0xac640000, 0xac650004, 0x8ee204c4, 0xac620010,
-0xaf860120, 0x92e24e20, 0x14400037, 0x24100001,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x8c830000, 0x24020012, 0x1462001f, 0x0,
-0x8ee34e30, 0x8ee24e34, 0x1062001b, 0x240c0040,
-0x8c820004, 0x24420001, 0xac820004, 0x8ee24e34,
-0x8ee34e30, 0x24420001, 0x104c0007, 0x0,
-0x8ee24e34, 0x24420001, 0x10620005, 0x0,
-0x8002ef1, 0x0, 0x14600005, 0x0,
-0x8f820128, 0x24420020, 0xaf820128, 0x8f820128,
-0x8c820004, 0x2c420011, 0x50400013, 0xac800000,
-0x8002f07, 0x0, 0x8ee24e30, 0x240c0040,
-0x24420001, 0x504c0003, 0x1021, 0x8ee24e30,
-0x24420001, 0xaee24e30, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x24020012, 0x240c0001,
-0xac820000, 0xac8c0004, 0x5600000d, 0x24100001,
-0x8ee204e4, 0x3c040001, 0x24845754, 0xafa00014,
-0xafa20010, 0x8ee60608, 0x8f470228, 0x3c050009,
-0x34a5f006, 0xc002403, 0xafab0038, 0x8fab0038,
-0x16000003, 0x240c0001, 0x8002f5c, 0xa2ec04f4,
-0x8ee20170, 0x24420001, 0xaee20170, 0x8ee20170,
-0x8ee204e4, 0xa2e004f4, 0xaee004f0, 0xaee07274,
-0xaee204f8, 0x8f42023c, 0x10400038, 0x0,
-0x8ee20184, 0x24420001, 0xaee20184, 0x8002f5c,
-0x8ee20184, 0x8ee20504, 0x240c0040, 0x24420001,
-0x504c0003, 0x1021, 0x8ee20504, 0x24420001,
-0xaee20504, 0x8ee20504, 0x8e630018, 0x240c0003,
-0x21080, 0x571021, 0x146c000f, 0x8c440508,
-0x3c020001, 0x571021, 0x904283b1, 0x10400014,
-0x0, 0x8ee201d0, 0x8ee35240, 0x441021,
-0xaee201d0, 0x8ee201d8, 0x641821, 0x306300ff,
-0x8002f4f, 0xaee35240, 0x8ee201cc, 0x8ee30e10,
-0x441021, 0xaee201cc, 0x8ee201d8, 0x641821,
-0x306301ff, 0xaee30e10, 0x441021, 0xaee201d8,
-0x8ee20000, 0x34420040, 0x8002f5c, 0xaee20000,
-0x8ee2014c, 0x3c010001, 0x370821, 0xa02083e0,
-0x24420001, 0xaee2014c, 0x8ee2014c, 0x8f820108,
-0x24420020, 0xaf820108, 0x8f820108, 0x8f820108,
-0x27633000, 0x43102b, 0x14400002, 0x27622800,
-0xaf820108, 0x8f830108, 0x8f820104, 0x1462fc1e,
-0x0, 0x8fbf0060, 0x8fbe005c, 0x8fb60058,
-0x8fb50054, 0x8fb40050, 0x8fb3004c, 0x8fb20048,
-0x8fb10044, 0x8fb00040, 0x3e00008, 0x27bd0068,
-0x52843, 0x10a0000d, 0x3021, 0x3c030001,
-0x34633800, 0x3c07ffff, 0x3631021, 0x82102b,
-0x50400001, 0x872021, 0x94820000, 0x24840002,
-0x24a5ffff, 0x14a0fff8, 0xc23021, 0x61c02,
-0x30c2ffff, 0x623021, 0x61c02, 0x30c2ffff,
-0x623021, 0x3e00008, 0x30c2ffff, 0x27bdff88,
-0x240f0001, 0xafbf0070, 0xafbe006c, 0xafb60068,
-0xafb50064, 0xafb40060, 0xafb3005c, 0xafb20058,
-0xafb10054, 0xafb00050, 0xa3a00027, 0xafaf002c,
-0x8ee204d4, 0x8021, 0x30420001, 0x1440002a,
-0xa3a00037, 0x8f8700e0, 0x8f8800c4, 0x8f8200e8,
-0xe22023, 0x2c821000, 0x50400001, 0x24841000,
-0x420c2, 0x801821, 0x8ee400c8, 0x8ee500cc,
-0x1021, 0xa32821, 0xa3302b, 0x822021,
-0x862021, 0xaee400c8, 0xaee500cc, 0x8f8300c8,
-0x3c02000a, 0x3442efff, 0x1032023, 0x44102b,
-0x10400003, 0x3c02000a, 0x3442f000, 0x822021,
-0x801821, 0x8ee400c0, 0x8ee500c4, 0x1021,
-0xa32821, 0xa3302b, 0x822021, 0x862021,
-0xaee400c0, 0xaee500c4, 0xaf8800c8, 0xaf8700e4,
-0x80034cc, 0xaf8700e8, 0x3c020001, 0x571021,
-0x904283c0, 0x1040000b, 0x0, 0x3c140001,
-0x297a021, 0x8e9483c4, 0x3c130001, 0x2779821,
-0x8e7383c8, 0x3c120001, 0x2579021, 0x8003193,
-0x8e5283cc, 0x8f8300e0, 0x8f8200e4, 0x10430007,
-0x8821, 0x8f8200e4, 0x24110001, 0x8c430000,
-0x8c440004, 0xafa30018, 0xafa4001c, 0x1620000e,
-0x3c02ffff, 0x8f8200c4, 0xafa20010, 0x8f8200c8,
-0x3c040001, 0x24845870, 0xafa20014, 0x8f8600e0,
-0x8f8700e4, 0x3c050006, 0xc002403, 0x34a5f000,
-0x80034cc, 0x0, 0x8fa3001c, 0x8fb20018,
-0x3074ffff, 0x2694fffc, 0x621024, 0x10400058,
-0x2409821, 0x3c020080, 0x621024, 0x1040000a,
-0x3c040040, 0x8ee2007c, 0x24420001, 0xaee2007c,
-0x8ee2007c, 0x8ee201fc, 0x24420001, 0xaee201fc,
-0x80034c6, 0x8ee201fc, 0x3c060004, 0x3c0b0001,
-0x3c0a0002, 0x3c050010, 0x3c090008, 0x8ee20080,
-0x3c080020, 0x34078000, 0x24420001, 0xaee20080,
-0x8ee20080, 0x8fa2001c, 0x441824, 0x10660021,
-0xc3102b, 0x14400007, 0x0, 0x106b0011,
-0x0, 0x106a0015, 0x0, 0x8003049,
-0x42042, 0x10650023, 0xa3102b, 0x14400005,
-0x0, 0x10690019, 0x0, 0x8003049,
-0x42042, 0x10680021, 0x0, 0x8003049,
-0x42042, 0x8ee20034, 0x24420001, 0xaee20034,
-0x8ee20034, 0x8003049, 0x42042, 0x8ee201ec,
-0x24420001, 0xaee201ec, 0x8ee201ec, 0x8003049,
-0x42042, 0x8ee201f0, 0x24420001, 0xaee201f0,
-0x8ee201f0, 0x8003049, 0x42042, 0x8ee201f4,
-0x24420001, 0xaee201f4, 0x8ee201f4, 0x8003049,
-0x42042, 0x8ee20030, 0x24420001, 0xaee20030,
-0x8ee20030, 0x8003049, 0x42042, 0x8ee201f8,
-0x24420001, 0xaee201f8, 0x8ee201f8, 0x42042,
-0x1087047c, 0x0, 0x800300e, 0x0,
-0x3c020001, 0x571021, 0x904283b2, 0x14400084,
-0x24020001, 0x3c030001, 0x771821, 0x906383b3,
-0x1462007f, 0x3c020100, 0x8e430000, 0x621024,
-0x1040006f, 0x2402ffff, 0x14620005, 0x24100001,
-0x96430004, 0x3402ffff, 0x10620075, 0x0,
-0x92e204d8, 0x14400072, 0x0, 0x3c020001,
-0x571021, 0x8c4283b4, 0x28420005, 0x10400020,
-0x3821, 0x3c020001, 0x571021, 0x8c4283b4,
-0x18400016, 0x2821, 0x96660000, 0x520c0,
-0x971021, 0x9442777e, 0x14460009, 0x971021,
-0x94437780, 0x96620002, 0x14620005, 0x971021,
-0x94437782, 0x96620004, 0x50620008, 0x24070001,
-0x3c020001, 0x571021, 0x8c4283b4, 0x24a50001,
-0xa2102a, 0x5440ffee, 0x520c0, 0x30e200ff,
-0x10400440, 0x0, 0x80030d5, 0x0,
-0x2402021, 0xc0022fe, 0x24050006, 0x3044001f,
-0x428c0, 0x2e51021, 0x9442727c, 0x30424000,
-0x14400434, 0xb71021, 0x9443727e, 0x96620000,
-0x1462000b, 0x418c0, 0xb71021, 0x94437280,
-0x96620002, 0x14620006, 0x418c0, 0xb71021,
-0x94437282, 0x96620004, 0x10620035, 0x418c0,
-0x2e31021, 0x9442727c, 0x30428000, 0x14400421,
-0x2e31021, 0x944b727c, 0x96670000, 0xb28c0,
-0xb71021, 0x9442737e, 0x80030b7, 0x3021,
-0x420c0, 0x2e41021, 0x9443737c, 0x2e41021,
-0x944b737c, 0x30638000, 0x14600010, 0xb28c0,
-0xb71021, 0x9442737e, 0x1447fff5, 0x1602021,
-0xb71021, 0x94437380, 0x96620002, 0x5462fff1,
-0x420c0, 0xb71021, 0x94437382, 0x96620004,
-0x5462ffec, 0x420c0, 0x24060001, 0x30c200ff,
-0x10400400, 0x0, 0x80030d5, 0x0,
-0x97430202, 0x96420000, 0x146203fa, 0x0,
-0x97430204, 0x96420002, 0x146203f6, 0x0,
-0x97430206, 0x96420004, 0x146203f2, 0x0,
-0x92420000, 0x3a030001, 0x30420001, 0x431024,
-0x10400074, 0x2402ffff, 0x8e630000, 0x14620004,
-0x3402ffff, 0x96630004, 0x1062006f, 0x240f0002,
-0x3c020001, 0x571021, 0x904283b2, 0x1440006a,
-0x240f0003, 0x92e204d8, 0x54400068, 0xafaf002c,
-0x3c020001, 0x571021, 0x8c4283b4, 0x28420005,
-0x10400020, 0x3821, 0x3c020001, 0x571021,
-0x8c4283b4, 0x18400016, 0x2821, 0x96660000,
-0x520c0, 0x971021, 0x9442777e, 0x14460009,
-0x971021, 0x94437780, 0x96620002, 0x14620005,
-0x971021, 0x94437782, 0x96620004, 0x50620008,
-0x24070001, 0x3c020001, 0x571021, 0x8c4283b4,
-0x24a50001, 0xa2102a, 0x5440ffee, 0x520c0,
-0x30e200ff, 0x14400044, 0x240f0003, 0x80034c6,
-0x0, 0x2402021, 0xc0022fe, 0x24050006,
-0x3044001f, 0x428c0, 0x2e51021, 0x9442727c,
-0x30424000, 0x144003af, 0xb71021, 0x9443727e,
-0x96620000, 0x1462000b, 0x418c0, 0xb71021,
-0x94437280, 0x96620002, 0x14620006, 0x418c0,
-0xb71021, 0x94437282, 0x96620004, 0x10620027,
-0x418c0, 0x2e31021, 0x9442727c, 0x30428000,
-0x1440039c, 0x2e31021, 0x944b727c, 0x96670000,
-0xb28c0, 0xb71021, 0x9442737e, 0x800313c,
-0x3021, 0x420c0, 0x2e41021, 0x9443737c,
-0x2e41021, 0x944b737c, 0x30638000, 0x14600010,
-0xb28c0, 0xb71021, 0x9442737e, 0x1447fff5,
-0x1602021, 0xb71021, 0x94437380, 0x96620002,
-0x5462fff1, 0x420c0, 0xb71021, 0x94437382,
-0x96620004, 0x5462ffec, 0x420c0, 0x24060001,
-0x30c200ff, 0x1040037b, 0x0, 0x800314f,
-0x240f0003, 0x240f0001, 0xafaf002c, 0x8f420260,
-0x54102b, 0x1040003a, 0x0, 0x8f8300e4,
-0x8f8200e0, 0x10620003, 0x24630008, 0xaf8300e4,
-0xaf8300e8, 0x8ee400c0, 0x8ee500c4, 0x2801821,
-0x1021, 0xa32821, 0xa3302b, 0x822021,
-0x862021, 0xaee400c0, 0xaee500c4, 0x8ee20058,
-0x24420001, 0xaee20058, 0x8ee20058, 0x8ee2007c,
-0x24420001, 0xaee2007c, 0x8ee2007c, 0x8f8200e0,
-0xafa20010, 0x8f8200e4, 0x3c040001, 0x24845878,
-0xafa20014, 0x8fa60018, 0x8fa7001c, 0x3c050006,
-0xc002403, 0x34a5f003, 0x80034cc, 0x0,
-0x8ee25240, 0xafa20010, 0x8ee25244, 0x3c040001,
-0x24845884, 0xafa20014, 0x8ee60e10, 0x8ee70e18,
-0x3c050006, 0xc002403, 0x34a5f002, 0x8ee201c0,
-0x24420001, 0xaee201c0, 0x8ee20000, 0x8ee301c0,
-0x2403ffbf, 0x431024, 0x8003470, 0xaee20000,
-0x96e20468, 0x54102b, 0x10400003, 0x0,
-0x240f0001, 0xa3af0027, 0x12800301, 0x24160007,
-0x24150040, 0x241e0001, 0x240e0012, 0x8ee2724c,
-0x8f430280, 0x24420001, 0x304207ff, 0x106202d3,
-0x0, 0x93a20027, 0x10400014, 0x0,
-0x8ee35240, 0x8ee25244, 0x10620009, 0x26ed5244,
-0x8ee65244, 0x8ee35244, 0x21140, 0x24425248,
-0x2e28021, 0x24630001, 0x80031bf, 0x306b00ff,
-0x92e27248, 0x1440ffca, 0x0, 0x8ee201e0,
-0x24420001, 0xaee201e0, 0x8ee201e0, 0x8ee30e10,
-0x8ee20e18, 0x1062ffc2, 0x26ed0e18, 0x8ee60e18,
-0x8ee30e18, 0x21140, 0x24420e20, 0x2e28021,
-0x24630001, 0x306b01ff, 0x96e2046a, 0x30420010,
-0x10400019, 0x0, 0x9642000c, 0x340f8100,
-0x144f0015, 0x0, 0x3c020001, 0x571021,
-0x904283c0, 0x14400010, 0x0, 0x9642000e,
-0xa6020016, 0x8e420008, 0x8e430004, 0x8e440000,
-0x2694fffc, 0xae42000c, 0xae430008, 0xae440004,
-0x9602000e, 0x26730004, 0x240f0001, 0xa3af0037,
-0x34420200, 0xa602000e, 0x8e020000, 0x8e030004,
-0x3c040001, 0x34843800, 0x306a0007, 0x26a9823,
-0x3641021, 0x262102b, 0x10400005, 0x28aa021,
-0x2641023, 0x3621823, 0x3c020020, 0x439823,
-0x26820007, 0x2404fff8, 0x9603000a, 0x446024,
-0x6a1821, 0x6c102b, 0x10400002, 0x1803821,
-0x603821, 0xae130018, 0x8f880120, 0x24e20007,
-0x443824, 0x27623800, 0x25090020, 0x122102b,
-0x50400001, 0x27693000, 0x8f820128, 0x11220004,
-0x0, 0x8f820124, 0x15220007, 0x1401821,
-0x8ee201a4, 0x8821, 0x24420001, 0xaee201a4,
-0x800324c, 0x8ee201a4, 0x8e040000, 0x8e050004,
-0x1021, 0xad130008, 0xa507000e, 0xad160018,
-0xad06001c, 0xa3302b, 0xa32823, 0x822023,
-0x862023, 0xad040000, 0xad050004, 0x8ee204c0,
-0xad020010, 0xaf890120, 0x92e24e20, 0x14400033,
-0x24110001, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0x8c820000, 0x1456001f, 0x0,
-0x8ee34e30, 0x8ee24e34, 0x1062001b, 0x0,
-0x8c820004, 0x24420001, 0xac820004, 0x8ee24e34,
-0x8ee34e30, 0x24420001, 0x10550007, 0x0,
-0x8ee24e34, 0x24420001, 0x10620005, 0x0,
-0x8003239, 0x0, 0x14600005, 0x0,
-0x8f820128, 0x24420020, 0xaf820128, 0x8f820128,
-0x8c820004, 0x2c420011, 0x50400010, 0xac800000,
-0x800324c, 0x0, 0x8ee24e30, 0x24420001,
-0x50550003, 0x1021, 0x8ee24e30, 0x24420001,
-0xaee24e30, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0xac960000, 0xac9e0004, 0x16200018,
-0x3c050006, 0x8e020018, 0x3c040001, 0x24845890,
-0xafa20010, 0x8e020000, 0x8e030004, 0x34a5f009,
-0x2003021, 0xc002403, 0xafa30014, 0x93a20037,
-0x10400216, 0x340f8100, 0x8e420004, 0x8e430008,
-0x8e44000c, 0xa64f000c, 0xae420000, 0xae430004,
-0xae440008, 0x96020016, 0x8003470, 0xa642000e,
-0x14ec0168, 0x28a1823, 0x960c000a, 0x9603000e,
-0x28a1023, 0xa602000a, 0x34620004, 0xa602000e,
-0x8f880120, 0x27623800, 0x25090020, 0x122102b,
-0x14400002, 0x306affff, 0x27693000, 0x8f820128,
-0x11220004, 0x0, 0x8f820124, 0x15220007,
-0x24040020, 0x8ee201a4, 0x8821, 0x24420001,
-0xaee201a4, 0x80032ca, 0x8ee201a4, 0x8ee5724c,
-0x8ee60490, 0x8ee70494, 0xa504000e, 0x24040004,
-0xad100008, 0xad040018, 0x52940, 0xa01821,
-0x1021, 0xe33821, 0xe3202b, 0xc23021,
-0xc43021, 0xad060000, 0xad070004, 0x8ee2724c,
-0xad02001c, 0x8ee204c4, 0xad020010, 0xaf890120,
-0x92e24e20, 0x14400033, 0x24110001, 0x8ee24e30,
-0x210c0, 0x24425038, 0x2e22021, 0x8c820000,
-0x1456001f, 0x0, 0x8ee34e30, 0x8ee24e34,
-0x1062001b, 0x0, 0x8c820004, 0x24420001,
-0xac820004, 0x8ee24e34, 0x8ee34e30, 0x24420001,
-0x10550007, 0x0, 0x8ee24e34, 0x24420001,
-0x10620005, 0x0, 0x80032b7, 0x0,
-0x14600005, 0x0, 0x8f820128, 0x24420020,
-0xaf820128, 0x8f820128, 0x8c820004, 0x2c420011,
-0x50400010, 0xac800000, 0x80032ca, 0x0,
-0x8ee24e30, 0x24420001, 0x50550003, 0x1021,
-0x8ee24e30, 0x24420001, 0xaee24e30, 0x8ee24e30,
-0x210c0, 0x24425038, 0x2e22021, 0xac960000,
-0xac9e0004, 0x1620000d, 0x0, 0xa60c000a,
-0xa60a000e, 0x8f820100, 0xafa20010, 0x8f820104,
-0x3c040001, 0x2484589c, 0x3c050006, 0xafa20014,
-0x8ee6724c, 0x800343b, 0x34a5f00b, 0x3c010001,
-0x370821, 0xa02083c0, 0xadab0000, 0x8ee201d8,
-0x8ee3724c, 0x2442ffff, 0xaee201d8, 0x8ee201d8,
-0x24630001, 0x306307ff, 0x26e25244, 0x15a20006,
-0xaee3724c, 0x8ee201d0, 0x2442ffff, 0xaee201d0,
-0x80032ef, 0x8ee201d0, 0x8ee201cc, 0x2442ffff,
-0xaee201cc, 0x8ee201cc, 0x8f420240, 0x10400073,
-0x0, 0x8ee20e1c, 0x24420001, 0xaee20e1c,
-0x8f430240, 0x43102b, 0x14400176, 0xa021,
-0x8f830120, 0x27623800, 0x24660020, 0xc2102b,
-0x50400001, 0x27663000, 0x8f820128, 0x10c20004,
-0x0, 0x8f820124, 0x14c20007, 0x0,
-0x8ee201a4, 0x8821, 0x24420001, 0xaee201a4,
-0x800334f, 0x8ee201a4, 0x8ee2724c, 0xac62001c,
-0x8ee404a8, 0x8ee504ac, 0x2462001c, 0xac620008,
-0x24020008, 0xa462000e, 0x24020011, 0xac620018,
-0xac640000, 0xac650004, 0x8ee204c4, 0xac620010,
-0xaf860120, 0x92e24e20, 0x14400033, 0x24110001,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x8c820000, 0x144e001f, 0x0, 0x8ee34e30,
-0x8ee24e34, 0x1062001b, 0x0, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee24e34, 0x8ee34e30,
-0x24420001, 0x10550007, 0x0, 0x8ee24e34,
-0x24420001, 0x10620005, 0x0, 0x800333c,
-0x0, 0x14600005, 0x0, 0x8f820128,
-0x24420020, 0xaf820128, 0x8f820128, 0x8c820004,
-0x2c420011, 0x50400010, 0xac800000, 0x800334f,
-0x0, 0x8ee24e30, 0x24420001, 0x50550003,
-0x1021, 0x8ee24e30, 0x24420001, 0xaee24e30,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0xac8e0000, 0xac9e0004, 0x5620000d, 0x24110001,
-0x8ee2724c, 0x3c040001, 0x248458a8, 0xafa00014,
-0xafa20010, 0x8ee6724c, 0x8f470280, 0x3c050009,
-0x34a5f008, 0xc002403, 0xafae0048, 0x8fae0048,
-0x56200001, 0xaee00e1c, 0x8ee20188, 0x24420001,
-0xaee20188, 0x80033c8, 0x8ee20188, 0x8f830120,
-0x27623800, 0x24660020, 0xc2102b, 0x50400001,
-0x27663000, 0x8f820128, 0x10c20004, 0x0,
-0x8f820124, 0x14c20007, 0x0, 0x8ee201a4,
-0x8821, 0x24420001, 0xaee201a4, 0x80033ba,
-0x8ee201a4, 0x8ee2724c, 0xac62001c, 0x8ee404a8,
-0x8ee504ac, 0x2462001c, 0xac620008, 0x24020008,
-0xa462000e, 0x24020011, 0xac620018, 0xac640000,
-0xac650004, 0x8ee204c4, 0xac620010, 0xaf860120,
-0x92e24e20, 0x14400033, 0x24110001, 0x8ee24e30,
-0x210c0, 0x24425038, 0x2e22021, 0x8c820000,
-0x144e001f, 0x0, 0x8ee34e30, 0x8ee24e34,
-0x1062001b, 0x0, 0x8c820004, 0x24420001,
-0xac820004, 0x8ee24e34, 0x8ee34e30, 0x24420001,
-0x10550007, 0x0, 0x8ee24e34, 0x24420001,
-0x10620005, 0x0, 0x80033a7, 0x0,
-0x14600005, 0x0, 0x8f820128, 0x24420020,
-0xaf820128, 0x8f820128, 0x8c820004, 0x2c420011,
-0x50400010, 0xac800000, 0x80033ba, 0x0,
-0x8ee24e30, 0x24420001, 0x50550003, 0x1021,
-0x8ee24e30, 0x24420001, 0xaee24e30, 0x8ee24e30,
-0x210c0, 0x24425038, 0x2e22021, 0xac8e0000,
-0xac9e0004, 0x1620000d, 0x0, 0x8ee2724c,
-0x3c040001, 0x248458a8, 0xafa00014, 0xafa20010,
-0x8ee6724c, 0x8f470280, 0x3c050009, 0x34a5f008,
-0xc002403, 0xafae0048, 0x8fae0048, 0x8ee20174,
-0x24420001, 0xaee20174, 0x8ee20174, 0x800346e,
-0xa021, 0x960c000a, 0x183102b, 0x54400001,
-0x1801821, 0xa603000a, 0x8f880120, 0x27623800,
-0x25090020, 0x122102b, 0x50400001, 0x27693000,
-0x8f820128, 0x11220004, 0x0, 0x8f820124,
-0x15220007, 0x24040020, 0x8ee201a4, 0x8821,
-0x24420001, 0xaee201a4, 0x800342f, 0x8ee201a4,
-0x8ee5724c, 0x8ee60490, 0x8ee70494, 0xa504000e,
-0x24040004, 0xad100008, 0xad040018, 0x52940,
-0xa01821, 0x1021, 0xe33821, 0xe3202b,
-0xc23021, 0xc43021, 0xad060000, 0xad070004,
-0x8ee2724c, 0xad02001c, 0x8ee204c4, 0xad020010,
-0xaf890120, 0x92e24e20, 0x14400033, 0x24110001,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x8c820000, 0x1456001f, 0x0, 0x8ee34e30,
-0x8ee24e34, 0x1062001b, 0x0, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee24e34, 0x8ee34e30,
-0x24420001, 0x10550007, 0x0, 0x8ee24e34,
-0x24420001, 0x10620005, 0x0, 0x800341c,
-0x0, 0x14600005, 0x0, 0x8f820128,
-0x24420020, 0xaf820128, 0x8f820128, 0x8c820004,
-0x2c420011, 0x50400010, 0xac800000, 0x800342f,
-0x0, 0x8ee24e30, 0x24420001, 0x50550003,
-0x1021, 0x8ee24e30, 0x24420001, 0xaee24e30,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0xac960000, 0xac9e0004, 0x1620001d, 0x0,
-0xa60c000a, 0x8f820100, 0xafa20010, 0x8f820104,
-0x3c040001, 0x2484589c, 0x3c050006, 0xafa20014,
-0x8ee6724c, 0x34a5f00d, 0xc002403, 0x2003821,
-0x93a20037, 0x10400031, 0x340f8100, 0x8e420004,
-0x8e430008, 0x8e44000c, 0xa64f000c, 0xae420000,
-0xae430004, 0xae440008, 0x96020016, 0xa642000e,
-0x9602000e, 0x3042fdff, 0x8003470, 0xa602000e,
-0x8ee201d8, 0x2442ffff, 0xaee201d8, 0x8ee201d8,
-0x8ee201cc, 0x3c04001f, 0x3c010001, 0x370821,
-0xa03e83c0, 0x2442ffff, 0xaee201cc, 0x9603000a,
-0x3484ffff, 0x8ee201cc, 0x6a1821, 0x2639821,
-0x93202b, 0x10800003, 0x3c02fff5, 0x34421000,
-0x2629821, 0xadab0000, 0x8ee2724c, 0x24420001,
-0x304207ff, 0xaee2724c, 0x8f420240, 0x10400004,
-0x283a023, 0x8ee20e1c, 0x24420001, 0xaee20e1c,
-0xa3a00027, 0x1680fd29, 0x0, 0x12800024,
-0x0, 0x3c010001, 0x370821, 0xac3483c4,
-0x3c010001, 0x370821, 0xac3383c8, 0x3c010001,
-0x370821, 0xac3283cc, 0x93a20037, 0x10400008,
-0x0, 0x3c020001, 0x571021, 0x8c4283cc,
-0x24420004, 0x3c010001, 0x370821, 0xac2283cc,
-0x8ee2724c, 0x8f430280, 0x24420001, 0x304207ff,
-0x14620006, 0x0, 0x8ee201c4, 0x24420001,
-0xaee201c4, 0x80034cc, 0x8ee201c4, 0x8ee201bc,
-0x24420001, 0xaee201bc, 0x80034cc, 0x8ee201bc,
-0x97a4001e, 0x2484fffc, 0x801821, 0x8ee400c0,
-0x8ee500c4, 0x1021, 0xa32821, 0xa3302b,
-0x822021, 0x862021, 0xaee400c0, 0xaee500c4,
-0x8faf002c, 0x24020002, 0x11e2000f, 0x29e20003,
-0x14400017, 0x24020003, 0x15e20015, 0x0,
-0x8ee200d0, 0x8ee300d4, 0x24630001, 0x2c640001,
-0x441021, 0xaee200d0, 0xaee300d4, 0x8ee200d0,
-0x80034c6, 0x8ee300d4, 0x8ee200d8, 0x8ee300dc,
-0x24630001, 0x2c640001, 0x441021, 0xaee200d8,
-0xaee300dc, 0x8ee200d8, 0x80034c6, 0x8ee300dc,
-0x8ee200c8, 0x8ee300cc, 0x24630001, 0x2c640001,
-0x441021, 0xaee200c8, 0xaee300cc, 0x8ee200c8,
-0x8ee300cc, 0x8f8300e4, 0x8f8200e0, 0x10620003,
-0x24630008, 0xaf8300e4, 0xaf8300e8, 0x8fbf0070,
-0x8fbe006c, 0x8fb60068, 0x8fb50064, 0x8fb40060,
-0x8fb3005c, 0x8fb20058, 0x8fb10054, 0x8fb00050,
-0x3e00008, 0x27bd0078, 0x27bdffb0, 0xafb50044,
-0xa821, 0xafb00030, 0x8021, 0xafbf004c,
-0xafb60048, 0xafb40040, 0xafb3003c, 0xafb20038,
-0xafb10034, 0x8ee204d4, 0x24140001, 0x30420001,
-0x1440002a, 0xb021, 0x8f8700e0, 0x8f8800c4,
-0x8f8200e8, 0xe22023, 0x2c821000, 0x50400001,
-0x24841000, 0x420c2, 0x801821, 0x8ee400c8,
-0x8ee500cc, 0x1021, 0xa32821, 0xa3302b,
-0x822021, 0x862021, 0xaee400c8, 0xaee500cc,
-0x8f8300c8, 0x3c02000a, 0x3442efff, 0x1032023,
-0x44102b, 0x10400003, 0x3c02000a, 0x3442f000,
-0x822021, 0x801821, 0x8ee400c0, 0x8ee500c4,
-0x1021, 0xa32821, 0xa3302b, 0x822021,
-0x862021, 0xaee400c0, 0xaee500c4, 0xaf8800c8,
-0xaf8700e4, 0x8003850, 0xaf8700e8, 0x3c020001,
-0x571021, 0x904283c0, 0x1040000b, 0x0,
-0x3c130001, 0x2779821, 0x8e7383c4, 0x3c110001,
-0x2378821, 0x8e3183c8, 0x3c120001, 0x2579021,
-0x80036e8, 0x8e5283cc, 0x8f8300e0, 0x8f8200e4,
-0x10430007, 0x4821, 0x8f8200e4, 0x24090001,
-0x8c430000, 0x8c440004, 0xafa30018, 0xafa4001c,
-0x1520000e, 0x3c02ffff, 0x8f8200c4, 0xafa20010,
-0x8f8200c8, 0x3c040001, 0x24845870, 0xafa20014,
-0x8f8600e0, 0x8f8700e4, 0x3c050006, 0xc002403,
-0x34a5f000, 0x8003850, 0x0, 0x8fa3001c,
-0x8fb20018, 0x3073ffff, 0x2673fffc, 0x621024,
-0x10400058, 0x2408821, 0x3c020080, 0x621024,
-0x1040000a, 0x3c040040, 0x8ee2007c, 0x24420001,
-0xaee2007c, 0x8ee2007c, 0x8ee201fc, 0x24420001,
-0xaee201fc, 0x800384a, 0x8ee201fc, 0x3c060004,
-0x3c0b0001, 0x3c0a0002, 0x3c050010, 0x3c090008,
-0x8ee20080, 0x3c080020, 0x34078000, 0x24420001,
-0xaee20080, 0x8ee20080, 0x8fa2001c, 0x441824,
-0x10660021, 0xc3102b, 0x14400007, 0x0,
-0x106b0011, 0x0, 0x106a0015, 0x0,
-0x8003592, 0x42042, 0x10650023, 0xa3102b,
-0x14400005, 0x0, 0x10690019, 0x0,
-0x8003592, 0x42042, 0x10680021, 0x0,
-0x8003592, 0x42042, 0x8ee20034, 0x24420001,
-0xaee20034, 0x8ee20034, 0x8003592, 0x42042,
-0x8ee201ec, 0x24420001, 0xaee201ec, 0x8ee201ec,
-0x8003592, 0x42042, 0x8ee201f0, 0x24420001,
-0xaee201f0, 0x8ee201f0, 0x8003592, 0x42042,
-0x8ee201f4, 0x24420001, 0xaee201f4, 0x8ee201f4,
-0x8003592, 0x42042, 0x8ee20030, 0x24420001,
-0xaee20030, 0x8ee20030, 0x8003592, 0x42042,
-0x8ee201f8, 0x24420001, 0xaee201f8, 0x8ee201f8,
-0x42042, 0x108702b7, 0x0, 0x8003557,
-0x0, 0x3c020001, 0x571021, 0x904283b2,
-0x14400084, 0x24020001, 0x3c030001, 0x771821,
-0x906383b3, 0x1462007f, 0x3c020100, 0x8e430000,
-0x621024, 0x1040006f, 0x2402ffff, 0x14620005,
-0x24100001, 0x96430004, 0x3402ffff, 0x10620075,
-0x0, 0x92e204d8, 0x14400072, 0x0,
-0x3c020001, 0x571021, 0x8c4283b4, 0x28420005,
-0x10400020, 0x3821, 0x3c020001, 0x571021,
-0x8c4283b4, 0x18400016, 0x2821, 0x96260000,
-0x520c0, 0x971021, 0x9442777e, 0x14460009,
-0x971021, 0x94437780, 0x96220002, 0x14620005,
-0x971021, 0x94437782, 0x96220004, 0x50620008,
-0x24070001, 0x3c020001, 0x571021, 0x8c4283b4,
-0x24a50001, 0xa2102a, 0x5440ffee, 0x520c0,
-0x30e200ff, 0x1040027b, 0x0, 0x800361e,
-0x0, 0x2402021, 0xc0022fe, 0x24050006,
-0x3044001f, 0x428c0, 0x2e51021, 0x9442727c,
-0x30424000, 0x1440026f, 0xb71021, 0x9443727e,
-0x96220000, 0x1462000b, 0x418c0, 0xb71021,
-0x94437280, 0x96220002, 0x14620006, 0x418c0,
-0xb71021, 0x94437282, 0x96220004, 0x10620035,
-0x418c0, 0x2e31021, 0x9442727c, 0x30428000,
-0x1440025c, 0x2e31021, 0x9448727c, 0x96270000,
-0x828c0, 0xb71021, 0x9442737e, 0x8003600,
-0x3021, 0x420c0, 0x2e41021, 0x9443737c,
-0x2e41021, 0x9448737c, 0x30638000, 0x14600010,
-0x828c0, 0xb71021, 0x9442737e, 0x1447fff5,
-0x1002021, 0xb71021, 0x94437380, 0x96220002,
-0x5462fff1, 0x420c0, 0xb71021, 0x94437382,
-0x96220004, 0x5462ffec, 0x420c0, 0x24060001,
-0x30c200ff, 0x1040023b, 0x0, 0x800361e,
-0x0, 0x97430202, 0x96420000, 0x14620235,
-0x0, 0x97430204, 0x96420002, 0x14620231,
-0x0, 0x97430206, 0x96420004, 0x1462022d,
-0x0, 0x92420000, 0x3a030001, 0x30420001,
-0x431024, 0x10400074, 0x2402ffff, 0x8e230000,
-0x14620004, 0x3402ffff, 0x96230004, 0x1062006f,
-0x24140002, 0x3c020001, 0x571021, 0x904283b2,
-0x1440006a, 0x24140003, 0x92e204d8, 0x14400067,
-0x0, 0x3c020001, 0x571021, 0x8c4283b4,
-0x28420005, 0x10400020, 0x3821, 0x3c020001,
-0x571021, 0x8c4283b4, 0x18400016, 0x2821,
-0x96260000, 0x520c0, 0x971021, 0x9442777e,
-0x14460009, 0x971021, 0x94437780, 0x96220002,
-0x14620005, 0x971021, 0x94437782, 0x96220004,
-0x50620008, 0x24070001, 0x3c020001, 0x571021,
-0x8c4283b4, 0x24a50001, 0xa2102a, 0x5440ffee,
-0x520c0, 0x30e200ff, 0x14400044, 0x24140003,
-0x800384a, 0x0, 0x2402021, 0xc0022fe,
-0x24050006, 0x3044001f, 0x428c0, 0x2e51021,
-0x9442727c, 0x30424000, 0x144001ea, 0xb71021,
-0x9443727e, 0x96220000, 0x1462000b, 0x418c0,
-0xb71021, 0x94437280, 0x96220002, 0x14620006,
-0x418c0, 0xb71021, 0x94437282, 0x96220004,
-0x10620027, 0x418c0, 0x2e31021, 0x9442727c,
-0x30428000, 0x144001d7, 0x2e31021, 0x9448727c,
-0x96270000, 0x828c0, 0xb71021, 0x9442737e,
-0x8003685, 0x3021, 0x420c0, 0x2e41021,
-0x9443737c, 0x2e41021, 0x9448737c, 0x30638000,
-0x14600010, 0x828c0, 0xb71021, 0x9442737e,
-0x1447fff5, 0x1002021, 0xb71021, 0x94437380,
-0x96220002, 0x5462fff1, 0x420c0, 0xb71021,
-0x94437382, 0x96220004, 0x5462ffec, 0x420c0,
-0x24060001, 0x30c200ff, 0x104001b6, 0x0,
-0x8003698, 0x24140003, 0x24140001, 0x8f420260,
-0x53102b, 0x10400049, 0x0, 0x8f8300e4,
-0x8f8200e0, 0x10620003, 0x24630008, 0xaf8300e4,
-0xaf8300e8, 0x8ee400c0, 0x8ee500c4, 0x2601821,
-0x1021, 0xa32821, 0xa3302b, 0x822021,
-0x862021, 0xaee400c0, 0xaee500c4, 0x8ee20058,
-0x24420001, 0xaee20058, 0x8ee20058, 0x8ee2007c,
-0x24420001, 0xaee2007c, 0x8ee2007c, 0x8f8200e0,
-0xafa20010, 0x8f8200e4, 0x3c040001, 0x24845878,
-0xafa20014, 0x8fa60018, 0x8fa7001c, 0x3c050006,
-0xc002403, 0x34a5f003, 0x8003850, 0x0,
-0x8ee25240, 0xafa20010, 0x8ee25244, 0x3c040001,
-0x24845884, 0xafa20014, 0x8ee60e10, 0x8ee70e18,
-0xc002403, 0x34a5f002, 0x8ee201c0, 0x24420001,
-0xaee201c0, 0x8ee20000, 0x8ee301c0, 0x2403ffbf,
-0x431024, 0x80037f8, 0xaee20000, 0x8ee25240,
-0xafa20010, 0x8ee25244, 0x3c040001, 0x24845884,
-0xafa20014, 0x8ee60e10, 0x8ee70e18, 0x3c050006,
-0xc002403, 0x34a5f002, 0x8ee201c0, 0x24420001,
-0xaee201c0, 0x80037f8, 0x8ee201c0, 0x96e20468,
-0x53102b, 0x54400001, 0x3c158000, 0x12600131,
-0x3c0c001f, 0x358cffff, 0x8ee2724c, 0x8f430280,
-0x24420001, 0x304207ff, 0x10620108, 0x0,
-0x12a00014, 0x0, 0x8ee35240, 0x8ee25244,
-0x10620009, 0x26ee5244, 0x8eeb5244, 0x8ee35244,
-0x21140, 0x24425248, 0x2e28021, 0x24630001,
-0x8003712, 0x306800ff, 0x92e27248, 0x1440ffc0,
-0x3c050006, 0x8ee201e0, 0x24420001, 0xaee201e0,
-0x8ee201e0, 0x8ee30e10, 0x8ee20e18, 0x1062ffcb,
-0x26ee0e18, 0x8eeb0e18, 0xa821, 0x8ee30e18,
-0x21140, 0x24420e20, 0x2e28021, 0x24630001,
-0x306801ff, 0x96e2046a, 0x30420010, 0x10400017,
-0x34028100, 0x9643000c, 0x14620014, 0x0,
-0x3c020001, 0x571021, 0x904283c0, 0x1440000f,
-0x0, 0x9642000e, 0xa6020016, 0x8e420008,
-0x8e430004, 0x8e440000, 0x2673fffc, 0xae42000c,
-0xae430008, 0xae440004, 0x9602000e, 0x26310004,
-0x24160001, 0x34420200, 0xa602000e, 0x9603000a,
-0x2605021, 0x73102b, 0x10400002, 0x2606821,
-0x605021, 0x2d42003d, 0x1040002a, 0x3821,
-0x9623000c, 0x24020800, 0x54620027, 0xae110018,
-0x3c020001, 0x571021, 0x904283c0, 0x54400022,
-0xae110018, 0x26220017, 0x182102b, 0x10400013,
-0x0, 0x3c02fff5, 0x511021, 0x90421017,
-0x38430006, 0x2c630001, 0x38420011, 0x2c420001,
-0x621825, 0x10600013, 0x26220010, 0x182102b,
-0x1040000e, 0x0, 0x3c07fff5, 0xf13821,
-0x94e71010, 0x800375e, 0x24e7000e, 0x92220017,
-0x38430006, 0x2c630001, 0x38420011, 0x2c420001,
-0x621825, 0x50600004, 0xae110018, 0x96270010,
-0x24e7000e, 0xae110018, 0x3c020001, 0x571021,
-0x904283c0, 0x2102b, 0x14e00002, 0x24ec0,
-0x1403821, 0x8f830120, 0x27623800, 0x24660020,
-0xc2102b, 0x50400001, 0x27663000, 0x8f820128,
-0x10c20004, 0x0, 0x8f820124, 0x14c20007,
-0x2402000b, 0x8ee201a4, 0x4821, 0x24420001,
-0xaee201a4, 0x80037bf, 0x8ee201a4, 0x8e040000,
-0x8e050004, 0xac620018, 0x1751025, 0x491025,
-0xac710008, 0xa467000e, 0xac62001c, 0xac640000,
-0xac650004, 0x8ee204c0, 0xac620010, 0xaf860120,
-0x92e24e20, 0x14400038, 0x24090001, 0x8ee24e30,
-0x210c0, 0x24425038, 0x2e22021, 0x8c830000,
-0x24020007, 0x14620020, 0x0, 0x8ee34e30,
-0x8ee24e34, 0x1062001c, 0x0, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee34e34, 0x8ee54e30,
-0x24020040, 0x24630001, 0x10620007, 0x0,
-0x8ee24e34, 0x24420001, 0x10a20005, 0x0,
-0x80037a9, 0x0, 0x14a00005, 0x0,
-0x8f820128, 0x24420020, 0xaf820128, 0x8f820128,
-0x8c820004, 0x2c420011, 0x50400013, 0xac800000,
-0x80037bf, 0x0, 0x8ee24e30, 0x24030040,
-0x24420001, 0x50430003, 0x1021, 0x8ee24e30,
-0x24420001, 0xaee24e30, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x24020007, 0xac820000,
-0x24020001, 0xac820004, 0x15200018, 0x3c050006,
-0x8e020018, 0x3c040001, 0x24845890, 0xafa20010,
-0x8e020000, 0x8e030004, 0x34a5f009, 0x2003021,
-0xc002403, 0xafa30014, 0x32c200ff, 0x1040002b,
-0x34028100, 0x8e430004, 0x8e440008, 0x8e45000c,
-0xa642000c, 0xae430000, 0xae440004, 0xae450008,
-0x96020016, 0x80037f8, 0xa642000e, 0x154d000a,
-0x0, 0x9602000e, 0xa613000a, 0x34420004,
-0xa602000e, 0x3c010001, 0x370821, 0xa02083c0,
-0x80037f6, 0x9821, 0x9604000a, 0x93102b,
-0x10400002, 0x2601821, 0x801821, 0x24020001,
-0xa603000a, 0x3c010001, 0x370821, 0xa02283c0,
-0x9604000a, 0x2248821, 0x191102b, 0x10400003,
-0x3c02fff5, 0x34421000, 0x2228821, 0x2649823,
-0xa821, 0x1660fef4, 0xadc80000, 0x12600021,
-0x32c200ff, 0x3c010001, 0x370821, 0xac3383c4,
-0x3c010001, 0x370821, 0xac3183c8, 0x3c010001,
-0x370821, 0x10400008, 0xac3283cc, 0x3c020001,
-0x571021, 0x8c4283cc, 0x24420004, 0x3c010001,
-0x370821, 0xac2283cc, 0x8ee2724c, 0x8f430280,
-0x24420001, 0x14620006, 0x0, 0x8ee201c4,
-0x24420001, 0xaee201c4, 0x8003850, 0x8ee201c4,
-0x8ee201bc, 0x24420001, 0xaee201bc, 0x8003850,
-0x8ee201bc, 0x97a4001e, 0x2484fffc, 0x801821,
-0x8ee400c0, 0x8ee500c4, 0x1021, 0xa32821,
-0xa3302b, 0x822021, 0x862021, 0x24020002,
-0xaee400c0, 0xaee500c4, 0x1282000f, 0x2a820003,
-0x14400017, 0x24020003, 0x16820015, 0x0,
-0x8ee200d0, 0x8ee300d4, 0x24630001, 0x2c640001,
-0x441021, 0xaee200d0, 0xaee300d4, 0x8ee200d0,
-0x800384a, 0x8ee300d4, 0x8ee200d8, 0x8ee300dc,
-0x24630001, 0x2c640001, 0x441021, 0xaee200d8,
-0xaee300dc, 0x8ee200d8, 0x800384a, 0x8ee300dc,
-0x8ee200c8, 0x8ee300cc, 0x24630001, 0x2c640001,
-0x441021, 0xaee200c8, 0xaee300cc, 0x8ee200c8,
-0x8ee300cc, 0x8f8300e4, 0x8f8200e0, 0x10620003,
-0x24630008, 0xaf8300e4, 0xaf8300e8, 0x8fbf004c,
-0x8fb60048, 0x8fb50044, 0x8fb40040, 0x8fb3003c,
-0x8fb20038, 0x8fb10034, 0x8fb00030, 0x3e00008,
-0x27bd0050, 0x27bdff90, 0xafb60060, 0xb021,
-0xafbf0068, 0xafbe0064, 0xafb5005c, 0xafb40058,
-0xafb30054, 0xafb20050, 0xafb1004c, 0xafb00048,
-0x8ee204d4, 0x8821, 0x24150001, 0x30420001,
-0x1440002a, 0xa3a0002f, 0x8f8700e0, 0x8f8800c4,
-0x8f8200e8, 0xe22023, 0x2c821000, 0x50400001,
-0x24841000, 0x420c2, 0x801821, 0x8ee400c8,
-0x8ee500cc, 0x1021, 0xa32821, 0xa3302b,
-0x822021, 0x862021, 0xaee400c8, 0xaee500cc,
-0x8f8300c8, 0x3c02000a, 0x3442efff, 0x1032023,
-0x44102b, 0x10400003, 0x3c02000a, 0x3442f000,
-0x822021, 0x801821, 0x8ee400c0, 0x8ee500c4,
-0x1021, 0xa32821, 0xa3302b, 0x822021,
-0x862021, 0xaee400c0, 0xaee500c4, 0xaf8800c8,
-0xaf8700e4, 0x8003c5b, 0xaf8700e8, 0x3c020001,
-0x571021, 0x904283c0, 0x1040000b, 0x0,
-0x3c130001, 0x2779821, 0x8e7383c4, 0x3c100001,
-0x2178021, 0x8e1083c8, 0x3c120001, 0x2579021,
-0x8003a59, 0x8e5283cc, 0x8f8300e0, 0x8f8200e4,
-0x10430007, 0x3821, 0x8f8200e4, 0x24070001,
-0x8c430000, 0x8c440004, 0xafa30018, 0xafa4001c,
-0x14e0000e, 0x3c02ffff, 0x8f8200c4, 0xafa20010,
-0x8f8200c8, 0x3c040001, 0x248458b4, 0xafa20014,
-0x8f8600e0, 0x8f8700e4, 0x3c050006, 0xc002403,
-0x34a5f200, 0x8003c5b, 0x0, 0x8fa3001c,
-0x8fb20018, 0x3073ffff, 0x2673fffc, 0x621024,
-0x10400058, 0x2408021, 0x3c020080, 0x621024,
-0x1040000a, 0x3c040040, 0x8ee2007c, 0x24420001,
-0xaee2007c, 0x8ee2007c, 0x8ee201fc, 0x24420001,
-0xaee201fc, 0x8003c55, 0x8ee201fc, 0x3c060004,
-0x3c0b0001, 0x3c0a0002, 0x3c050010, 0x3c090008,
-0x8ee20080, 0x3c080020, 0x34078000, 0x24420001,
-0xaee20080, 0x8ee20080, 0x8fa2001c, 0x441824,
-0x10660021, 0xc3102b, 0x14400007, 0x0,
-0x106b0011, 0x0, 0x106a0015, 0x0,
-0x8003916, 0x42042, 0x10650023, 0xa3102b,
-0x14400005, 0x0, 0x10690019, 0x0,
-0x8003916, 0x42042, 0x10680021, 0x0,
-0x8003916, 0x42042, 0x8ee20034, 0x24420001,
-0xaee20034, 0x8ee20034, 0x8003916, 0x42042,
-0x8ee201ec, 0x24420001, 0xaee201ec, 0x8ee201ec,
-0x8003916, 0x42042, 0x8ee201f0, 0x24420001,
-0xaee201f0, 0x8ee201f0, 0x8003916, 0x42042,
-0x8ee201f4, 0x24420001, 0xaee201f4, 0x8ee201f4,
-0x8003916, 0x42042, 0x8ee20030, 0x24420001,
-0xaee20030, 0x8ee20030, 0x8003916, 0x42042,
-0x8ee201f8, 0x24420001, 0xaee201f8, 0x8ee201f8,
-0x42042, 0x1087033e, 0x0, 0x80038db,
-0x0, 0x3c020001, 0x571021, 0x904283b2,
-0x14400084, 0x24020001, 0x3c030001, 0x771821,
-0x906383b3, 0x1462007f, 0x3c020100, 0x8e430000,
-0x621024, 0x1040006f, 0x2402ffff, 0x14620005,
-0x24110001, 0x96430004, 0x3402ffff, 0x10620075,
-0x0, 0x92e204d8, 0x14400072, 0x0,
-0x3c020001, 0x571021, 0x8c4283b4, 0x28420005,
-0x10400020, 0x3821, 0x3c020001, 0x571021,
-0x8c4283b4, 0x18400016, 0x2821, 0x96060000,
-0x520c0, 0x971021, 0x9442777e, 0x14460009,
-0x971021, 0x94437780, 0x96020002, 0x14620005,
-0x971021, 0x94437782, 0x96020004, 0x50620008,
-0x24070001, 0x3c020001, 0x571021, 0x8c4283b4,
-0x24a50001, 0xa2102a, 0x5440ffee, 0x520c0,
-0x30e200ff, 0x10400302, 0x0, 0x80039a2,
-0x0, 0x2402021, 0xc0022fe, 0x24050006,
-0x3044001f, 0x428c0, 0x2e51021, 0x9442727c,
-0x30424000, 0x144002f6, 0xb71021, 0x9443727e,
-0x96020000, 0x1462000b, 0x418c0, 0xb71021,
-0x94437280, 0x96020002, 0x14620006, 0x418c0,
-0xb71021, 0x94437282, 0x96020004, 0x10620035,
-0x418c0, 0x2e31021, 0x9442727c, 0x30428000,
-0x144002e3, 0x2e31021, 0x944d727c, 0x96070000,
-0xd28c0, 0xb71021, 0x9442737e, 0x8003984,
-0x3021, 0x420c0, 0x2e41021, 0x9443737c,
-0x2e41021, 0x944d737c, 0x30638000, 0x14600010,
-0xd28c0, 0xb71021, 0x9442737e, 0x1447fff5,
-0x1a02021, 0xb71021, 0x94437380, 0x96020002,
-0x5462fff1, 0x420c0, 0xb71021, 0x94437382,
-0x96020004, 0x5462ffec, 0x420c0, 0x24060001,
-0x30c200ff, 0x104002c2, 0x0, 0x80039a2,
-0x0, 0x97430202, 0x96420000, 0x146202bc,
-0x0, 0x97430204, 0x96420002, 0x146202b8,
-0x0, 0x97430206, 0x96420004, 0x146202b4,
-0x0, 0x92420000, 0x3a230001, 0x30420001,
-0x431024, 0x10400074, 0x2402ffff, 0x8e030000,
-0x14620004, 0x3402ffff, 0x96030004, 0x1062006f,
-0x24150002, 0x3c020001, 0x571021, 0x904283b2,
-0x1440006a, 0x24150003, 0x92e204d8, 0x14400067,
-0x0, 0x3c020001, 0x571021, 0x8c4283b4,
-0x28420005, 0x10400020, 0x3821, 0x3c020001,
-0x571021, 0x8c4283b4, 0x18400016, 0x2821,
-0x96060000, 0x520c0, 0x971021, 0x9442777e,
-0x14460009, 0x971021, 0x94437780, 0x96020002,
-0x14620005, 0x971021, 0x94437782, 0x96020004,
-0x50620008, 0x24070001, 0x3c020001, 0x571021,
-0x8c4283b4, 0x24a50001, 0xa2102a, 0x5440ffee,
-0x520c0, 0x30e200ff, 0x14400044, 0x24150003,
-0x8003c55, 0x0, 0x2402021, 0xc0022fe,
-0x24050006, 0x3044001f, 0x428c0, 0x2e51021,
-0x9442727c, 0x30424000, 0x14400271, 0xb71021,
-0x9443727e, 0x96020000, 0x1462000b, 0x418c0,
-0xb71021, 0x94437280, 0x96020002, 0x14620006,
-0x418c0, 0xb71021, 0x94437282, 0x96020004,
-0x10620027, 0x418c0, 0x2e31021, 0x9442727c,
-0x30428000, 0x1440025e, 0x2e31021, 0x944d727c,
-0x96070000, 0xd28c0, 0xb71021, 0x9442737e,
-0x8003a09, 0x3021, 0x420c0, 0x2e41021,
-0x9443737c, 0x2e41021, 0x944d737c, 0x30638000,
-0x14600010, 0xd28c0, 0xb71021, 0x9442737e,
-0x1447fff5, 0x1a02021, 0xb71021, 0x94437380,
-0x96020002, 0x5462fff1, 0x420c0, 0xb71021,
-0x94437382, 0x96020004, 0x5462ffec, 0x420c0,
-0x24060001, 0x30c200ff, 0x1040023d, 0x0,
-0x8003a1c, 0x24150003, 0x24150001, 0x8f420260,
-0x53102b, 0x10400036, 0x0, 0x8f8300e4,
-0x8f8200e0, 0x10620003, 0x24630008, 0xaf8300e4,
-0xaf8300e8, 0x8ee400c0, 0x8ee500c4, 0x2601821,
-0x1021, 0xa32821, 0xa3302b, 0x822021,
-0x862021, 0xaee400c0, 0xaee500c4, 0x8ee20058,
-0x24420001, 0xaee20058, 0x8ee20058, 0x8ee2007c,
-0x24420001, 0xaee2007c, 0x8ee2007c, 0x8f8200e0,
-0xafa20010, 0x8f8200e4, 0x3c040001, 0x248458c0,
-0xafa20014, 0x8fa60018, 0x8fa7001c, 0x3c050006,
-0xc002403, 0x34a5f203, 0x8003c5b, 0x0,
-0x8ee25240, 0xafa20010, 0x8ee25244, 0x3c040001,
-0x248458cc, 0xafa20014, 0x8ee60e10, 0x8ee70e18,
-0x3c050006, 0xc002403, 0x34a5f202, 0x8ee201c0,
-0x24420001, 0xaee201c0, 0x8003c02, 0x8ee201c0,
-0x96e20468, 0x53102b, 0x54400001, 0x3c168000,
-0x126001cb, 0x3c0e001f, 0x35ceffff, 0x3c0ffff5,
-0x35ef1000, 0x241e0040, 0x8ee2724c, 0x8f430280,
-0x24420001, 0x304207ff, 0x1062019e, 0x0,
-0x12c00012, 0x0, 0x8ee35240, 0x8ee25244,
-0x1062000a, 0x26f85244, 0x8ef45244, 0xafb80024,
-0x8ee35244, 0x21140, 0x24425248, 0x2e28821,
-0x24630001, 0x8003a85, 0x306d00ff, 0x8ee201e0,
-0x24420001, 0xaee201e0, 0x8ee201e0, 0x8ee30e10,
-0x8ee20e18, 0x1062ffca, 0x26f80e18, 0x8ef40e18,
-0xb021, 0xafb80024, 0x8ee30e18, 0x21140,
-0x24420e20, 0x2e28821, 0x24630001, 0x306d01ff,
-0x96e2046a, 0x30420010, 0x10400018, 0x34028100,
-0x9643000c, 0x14620015, 0x0, 0x3c020001,
-0x571021, 0x904283c0, 0x14400010, 0x0,
-0x9642000e, 0xa6220016, 0x8e420008, 0x8e430004,
-0x8e440000, 0x2673fffc, 0xae42000c, 0xae430008,
-0xae440004, 0x9622000e, 0x26100004, 0x24180001,
-0xa3b8002f, 0x34420200, 0xa622000e, 0x8e220000,
-0x8e230004, 0x3c040001, 0x34843800, 0x2003021,
-0x306a0007, 0x20a8023, 0x3641021, 0x202102b,
-0x10400005, 0x26a9821, 0x2041023, 0x3621823,
-0x3c020020, 0x438023, 0x26620007, 0x9623000a,
-0x2418fff8, 0x58c824, 0x6a1821, 0x79102b,
-0x10400002, 0x3206021, 0x606021, 0x1801821,
-0x24620007, 0x2418fff8, 0x586024, 0x26c102b,
-0x14400004, 0x1932823, 0x1832823, 0x8003ac3,
-0xc31021, 0xd31021, 0x4a2023, 0x1c4102b,
-0x54400001, 0x8f2021, 0x25420040, 0x4c102b,
-0x14400035, 0x5821, 0x94c3000c, 0x24020800,
-0x54620032, 0xae260018, 0x3c020001, 0x571021,
-0x904283c0, 0x5440002d, 0xae260018, 0x24c20017,
-0x1c2102b, 0x10400013, 0x0, 0x3c02fff5,
-0x461021, 0x90421017, 0x38430006, 0x2c630001,
-0x38420011, 0x2c420001, 0x621825, 0x10600014,
-0x24c20010, 0x1c2102b, 0x1040000e, 0x0,
-0x3c0bfff5, 0x1665821, 0x956b1010, 0x8003af4,
-0x2562000e, 0x90c20017, 0x38430006, 0x2c630001,
-0x38420011, 0x2c420001, 0x621825, 0x10600005,
-0x1601821, 0x94cb0010, 0x2562000e, 0x4a5821,
-0x1601821, 0x24620007, 0x2418fff8, 0x585824,
-0xc31021, 0x4a2023, 0x1c4102b, 0x10400002,
-0x1632823, 0x8f2021, 0xae260018, 0x3c020001,
-0x571021, 0x904283c0, 0x2102b, 0x216c0,
-0x15600002, 0xafa20044, 0x1805821, 0x30820001,
-0x10400007, 0x4021, 0x90880000, 0x24840001,
-0x1c4102b, 0x10400002, 0x24a5ffff, 0x8f2021,
-0x50a00012, 0x81c02, 0x2ca20002, 0x54400009,
-0x24a5ffff, 0x94820000, 0x24840002, 0x1024021,
-0x1c4102b, 0x10400006, 0x24a5fffe, 0x8003b21,
-0x8f2021, 0x90820000, 0x21200, 0x1024021,
-0x14a0fff2, 0x2ca20002, 0x81c02, 0x3102ffff,
-0x624021, 0x3108ffff, 0x1402821, 0x11400011,
-0x2002021, 0x2ca20002, 0x54400009, 0x24a5ffff,
-0x94820000, 0x24840002, 0x1024021, 0x1c4102b,
-0x10400006, 0x24a5fffe, 0x8003b38, 0x8f2021,
-0x90820000, 0x21200, 0x1024021, 0x14a0fff2,
-0x2ca20002, 0x81c02, 0x3102ffff, 0x624021,
-0x81c02, 0x3102ffff, 0x8f890120, 0x624021,
-0x27623800, 0x25230020, 0x62102b, 0x14400002,
-0x3108ffff, 0x27633000, 0x8f820128, 0x10620004,
-0x0, 0x8f820124, 0x14620007, 0x1402821,
-0x8ee201a4, 0x3821, 0x24420001, 0xaee201a4,
-0x8003bc9, 0x8ee201a4, 0x8e260000, 0x8e270004,
-0x81400, 0x3448000b, 0xad300008, 0xa52b000e,
-0xad280018, 0x8fb80044, 0x2021, 0x2961025,
-0x581025, 0xad22001c, 0xe5102b, 0xe53823,
-0xc43023, 0xc23023, 0xad260000, 0xad270004,
-0x8ee204c0, 0xad220010, 0xaf830120, 0x92e24e20,
-0x1440005f, 0x24070001, 0x2502ffee, 0x2c420002,
-0x14400003, 0x24020011, 0x15020024, 0x0,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x8c830000, 0x24020012, 0x1462000f, 0x0,
-0x8ee34e30, 0x8ee24e34, 0x1062000b, 0x0,
-0x8c820004, 0x24420001, 0xac820004, 0x8ee24e34,
-0x8ee34e30, 0x24420001, 0x105e002a, 0x0,
-0x8003ba8, 0x0, 0x8ee24e30, 0x24420001,
-0x505e0003, 0x1021, 0x8ee24e30, 0x24420001,
-0xaee24e30, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0x8003bc6, 0x24020012, 0x8ee24e30,
-0x210c0, 0x24425038, 0x2e22021, 0x8c830000,
-0x24020007, 0x1462001f, 0x0, 0x8ee34e30,
-0x8ee24e34, 0x1062001b, 0x0, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee24e34, 0x8ee34e30,
-0x24420001, 0x105e0007, 0x0, 0x8ee24e34,
-0x24420001, 0x10620005, 0x0, 0x8003bb4,
-0x0, 0x14600005, 0x0, 0x8f820128,
-0x24420020, 0xaf820128, 0x8f820128, 0x8c820004,
-0x2c420011, 0x50400012, 0xac800000, 0x8003bc9,
-0x0, 0x8ee24e30, 0x24420001, 0x505e0003,
-0x1021, 0x8ee24e30, 0x24420001, 0xaee24e30,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x24020007, 0xac820000, 0x24020001, 0xac820004,
-0x14e00019, 0x3c050006, 0x3c040001, 0x24845890,
-0x8e220018, 0x34a5f209, 0xafa20010, 0x8e220000,
-0x8e230004, 0x2203021, 0x1603821, 0xc002403,
-0xafa30014, 0x93a2002f, 0x1040002a, 0x34028100,
-0x8e430004, 0x8e440008, 0x8e45000c, 0xa642000c,
-0xae430000, 0xae440004, 0xae450008, 0x96220016,
-0x8003c02, 0xa642000e, 0x1599000a, 0x26a1823,
-0x9622000e, 0xa623000a, 0x34420004, 0xa622000e,
-0x3c010001, 0x370821, 0xa02083c0, 0x8003bff,
-0x9821, 0x9624000a, 0x83102b, 0x54400001,
-0x801821, 0x24020001, 0xa623000a, 0x3c010001,
-0x370821, 0xa02283c0, 0x9622000a, 0x4a1821,
-0x2038021, 0x1d0102b, 0x54400001, 0x20f8021,
-0x2639823, 0xb021, 0x8fb80024, 0x1660fe5e,
-0xaf0d0000, 0x12600022, 0x0, 0x3c010001,
-0x370821, 0xac3383c4, 0x3c010001, 0x370821,
-0xac3083c8, 0x3c010001, 0x370821, 0xac3283cc,
-0x93a2002f, 0x10400008, 0x0, 0x3c020001,
-0x571021, 0x8c4283cc, 0x24420004, 0x3c010001,
-0x370821, 0xac2283cc, 0x8f430280, 0x8ee2724c,
-0x14620006, 0x0, 0x8ee201c4, 0x24420001,
-0xaee201c4, 0x8003c5b, 0x8ee201c4, 0x8ee201bc,
-0x24420001, 0xaee201bc, 0x8003c5b, 0x8ee201bc,
-0x97a4001e, 0x2484fffc, 0x801821, 0x8ee400c0,
-0x8ee500c4, 0x1021, 0xa32821, 0xa3302b,
-0x822021, 0x862021, 0x24020002, 0xaee400c0,
-0xaee500c4, 0x12a2000f, 0x2aa20003, 0x14400017,
-0x24020003, 0x16a20015, 0x0, 0x8ee200d0,
-0x8ee300d4, 0x24630001, 0x2c640001, 0x441021,
-0xaee200d0, 0xaee300d4, 0x8ee200d0, 0x8003c55,
-0x8ee300d4, 0x8ee200d8, 0x8ee300dc, 0x24630001,
-0x2c640001, 0x441021, 0xaee200d8, 0xaee300dc,
-0x8ee200d8, 0x8003c55, 0x8ee300dc, 0x8ee200c8,
-0x8ee300cc, 0x24630001, 0x2c640001, 0x441021,
-0xaee200c8, 0xaee300cc, 0x8ee200c8, 0x8ee300cc,
-0x8f8300e4, 0x8f8200e0, 0x10620003, 0x24630008,
-0xaf8300e4, 0xaf8300e8, 0x8fbf0068, 0x8fbe0064,
-0x8fb60060, 0x8fb5005c, 0x8fb40058, 0x8fb30054,
-0x8fb20050, 0x8fb1004c, 0x8fb00048, 0x3e00008,
-0x27bd0070, 0x27bdffe0, 0xafbf0018, 0x8ee30e14,
-0x8ee20e0c, 0x10620074, 0x0, 0x8ee30e0c,
-0x8ee20e14, 0x622023, 0x4820001, 0x24840200,
-0x8ee30e18, 0x8ee20e14, 0x43102b, 0x14400004,
-0x24020200, 0x8ee30e14, 0x8003c7d, 0x431823,
-0x8ee20e18, 0x8ee30e14, 0x431023, 0x2443ffff,
-0x804821, 0x69102a, 0x54400001, 0x604821,
-0x8f870100, 0x27623000, 0x24e80020, 0x102102b,
-0x50400001, 0x27682800, 0x8f820108, 0x11020004,
-0x0, 0x8f820104, 0x15020007, 0x1021,
-0x8ee201a8, 0x2021, 0x24420001, 0xaee201a8,
-0x8003cbf, 0x8ee201a8, 0x8ee40e14, 0x42140,
-0x801821, 0x8ee40460, 0x8ee50464, 0xa32821,
-0xa3302b, 0x822021, 0x862021, 0xace40000,
-0xace50004, 0x8ee30e14, 0x91140, 0xa4e2000e,
-0x24020002, 0xace20018, 0x31940, 0x24630e20,
-0x2e31021, 0xace20008, 0x8ee20e14, 0xace2001c,
-0x8ee204cc, 0xace20010, 0xaf880100, 0x92e204ec,
-0x14400011, 0x24040001, 0x8ee24e28, 0x24030040,
-0x24420001, 0x50430003, 0x1021, 0x8ee24e28,
-0x24420001, 0xaee24e28, 0x8ee24e28, 0x210c0,
-0x24424e38, 0x2e21821, 0x24020002, 0xac620000,
-0x24020001, 0xac620004, 0x1480000e, 0x24030040,
-0x8ee20e14, 0xafa20010, 0x8ee20e18, 0x3c050007,
-0xafa20014, 0x8ee60e0c, 0x8ee70e10, 0x3c040001,
-0x248458d4, 0xc002403, 0x34a5f001, 0x8003cdd,
-0x0, 0x8ee20500, 0x24420001, 0x50430003,
-0x1021, 0x8ee20500, 0x24420001, 0xaee20500,
-0x8ee20500, 0x21080, 0x571021, 0xac490508,
-0x8ee20e14, 0x491021, 0x304201ff, 0xaee20e14,
-0x8ee30e14, 0x8ee20e0c, 0x14620005, 0x0,
-0x8f820060, 0x2403fdff, 0x431024, 0xaf820060,
-0x8fbf0018, 0x3e00008, 0x27bd0020, 0x27bdffe0,
-0xafbf0018, 0x8ee3523c, 0x8ee25238, 0x10620074,
-0x0, 0x8ee35238, 0x8ee2523c, 0x622023,
-0x4820001, 0x24840100, 0x8ee35244, 0x8ee2523c,
-0x43102b, 0x14400004, 0x24020100, 0x8ee3523c,
-0x8003cff, 0x431823, 0x8ee25244, 0x8ee3523c,
-0x431023, 0x2443ffff, 0x804821, 0x69102a,
-0x54400001, 0x604821, 0x8f870100, 0x27623000,
-0x24e80020, 0x102102b, 0x50400001, 0x27682800,
-0x8f820108, 0x11020004, 0x0, 0x8f820104,
-0x15020007, 0x1021, 0x8ee201a8, 0x2021,
-0x24420001, 0xaee201a8, 0x8003d41, 0x8ee201a8,
-0x8ee4523c, 0x42140, 0x801821, 0x8ee40470,
-0x8ee50474, 0xa32821, 0xa3302b, 0x822021,
-0x862021, 0xace40000, 0xace50004, 0x8ee3523c,
-0x91140, 0xa4e2000e, 0x24020003, 0xace20018,
-0x31940, 0x24635248, 0x2e31021, 0xace20008,
-0x8ee2523c, 0xace2001c, 0x8ee204cc, 0xace20010,
-0xaf880100, 0x92e204ec, 0x14400011, 0x24040001,
-0x8ee24e28, 0x24030040, 0x24420001, 0x50430003,
-0x1021, 0x8ee24e28, 0x24420001, 0xaee24e28,
-0x8ee24e28, 0x210c0, 0x24424e38, 0x2e21821,
-0x24020003, 0xac620000, 0x24020001, 0xac620004,
-0x1480000e, 0x24030040, 0x8ee2523c, 0xafa20010,
-0x8ee25244, 0x3c050007, 0xafa20014, 0x8ee65238,
-0x8ee75240, 0x3c040001, 0x248458e0, 0xc002403,
-0x34a5f010, 0x8003d5f, 0x0, 0x8ee20500,
-0x24420001, 0x50430003, 0x1021, 0x8ee20500,
-0x24420001, 0xaee20500, 0x8ee20500, 0x21080,
-0x571021, 0xac490508, 0x8ee2523c, 0x491021,
-0x304200ff, 0xaee2523c, 0x8ee3523c, 0x8ee25238,
-0x14620005, 0x0, 0x8f820060, 0x2403feff,
-0x431024, 0xaf820060, 0x8fbf0018, 0x3e00008,
-0x27bd0020, 0x8f820120, 0x8ee34e34, 0x8f820124,
-0x8f860128, 0x24020040, 0x24630001, 0x50620003,
-0x1021, 0x8ee24e34, 0x24420001, 0xaee24e34,
-0x8ee24e34, 0x8ee44e34, 0x8ee34e30, 0x210c0,
-0x24425038, 0x14830007, 0x2e22821, 0x8f820128,
-0x24420020, 0xaf820128, 0x8f820128, 0x8003d92,
-0xaca00000, 0x8ee24e34, 0x24030040, 0x24420001,
-0x50430003, 0x1021, 0x8ee24e34, 0x24420001,
-0x210c0, 0x24425038, 0x2e22821, 0x8ca20004,
-0x8f830128, 0x21140, 0x621821, 0xaf830128,
-0xaca00000, 0x8cc20018, 0x2443fffe, 0x2c620012,
-0x10400008, 0x31080, 0x3c010001, 0x220821,
-0x8c2258f0, 0x400008, 0x0, 0x24020001,
-0xaee24e24, 0x3e00008, 0x0, 0x27bdffc8,
-0xafbf0030, 0xafb5002c, 0xafb40028, 0xafb30024,
-0xafb20020, 0xafb1001c, 0xafb00018, 0x8f830128,
-0x8f820124, 0x106202b0, 0x9821, 0x3c11001f,
-0x3631ffff, 0x3c12fff5, 0x36521000, 0x24150012,
-0x24140040, 0x8f8c0128, 0x8f820128, 0x24420020,
-0xaf820128, 0x9182001b, 0x8f830128, 0x2443fffe,
-0x2c620012, 0x1040029c, 0x31080, 0x3c010001,
-0x220821, 0x8c225948, 0x400008, 0x0,
-0x8f420218, 0x30420100, 0x10400007, 0x0,
-0x95830016, 0x95820018, 0x621823, 0x31402,
-0x431021, 0xa5820016, 0x8d82001c, 0x3c038000,
-0x3044ffff, 0x436824, 0x3c030800, 0x431824,
-0x11a00004, 0xad84001c, 0x41140, 0x8003dd8,
-0x24425248, 0x41140, 0x24420e20, 0x2e25821,
-0x9562000e, 0x3042fffc, 0x10600004, 0xa562000e,
-0x95840016, 0x8003ec0, 0x0, 0x8d690018,
-0x4021, 0x952a0000, 0x25290002, 0x95270000,
-0x25290002, 0x95260000, 0x25290002, 0x95250000,
-0x25290002, 0x95240000, 0x25290002, 0x95230000,
-0x25290002, 0x95220000, 0x25290002, 0x1475021,
-0x1465021, 0x1455021, 0x1445021, 0x1435021,
-0x1425021, 0xa1c02, 0x3142ffff, 0x625021,
-0xa1c02, 0x3142ffff, 0x625021, 0x96e2046a,
-0x314effff, 0x30420002, 0x10400044, 0x5021,
-0x25220014, 0x222102b, 0x10400014, 0x1201821,
-0x2405000a, 0x2021, 0x223102b, 0x54400001,
-0x721821, 0x94620000, 0x24630002, 0x24a5ffff,
-0x14a0fff9, 0x822021, 0x41c02, 0x3082ffff,
-0x622021, 0x41402, 0x3083ffff, 0x431021,
-0x3042ffff, 0x8003e33, 0x1425021, 0x952a0000,
-0x25290002, 0x95280000, 0x25290002, 0x95270000,
-0x25290002, 0x95260000, 0x25290002, 0x95250000,
-0x25290002, 0x95230000, 0x25290002, 0x95220000,
-0x25290002, 0x95240000, 0x25290002, 0x1485021,
-0x1475021, 0x1465021, 0x1455021, 0x1435021,
-0x1425021, 0x95220000, 0x95230002, 0x1445021,
-0x1425021, 0x1435021, 0xa1c02, 0x3142ffff,
-0x625021, 0xa1c02, 0x3142ffff, 0x625021,
-0x3148ffff, 0x51000001, 0x3408ffff, 0x8d620018,
-0x9443000c, 0x24020800, 0x54620005, 0xa5680010,
-0x9562000e, 0x34420002, 0xa562000e, 0xa5680010,
-0x96e2046a, 0x2821, 0x30420008, 0x14400056,
-0x3021, 0x8d630018, 0x24620024, 0x222102b,
-0x10400034, 0x24690010, 0x229102b, 0x54400001,
-0x1324821, 0x95250000, 0x24690014, 0x229102b,
-0x10400002, 0x24a5ffec, 0x1324821, 0x95220000,
-0x30420fff, 0x14400003, 0x25290002, 0x8003e60,
-0x24130001, 0x9821, 0xa03021, 0x229102b,
-0x54400001, 0x1324821, 0x91220001, 0x25290002,
-0xa22821, 0x229102b, 0x54400001, 0x1324821,
-0x25290002, 0x229102b, 0x54400001, 0x1324821,
-0x95220000, 0x25290002, 0xa22821, 0x229102b,
-0x54400001, 0x1324821, 0x95220000, 0x25290002,
-0xa22821, 0x229102b, 0x54400001, 0x1324821,
-0x95220000, 0x25290002, 0xa22821, 0x229102b,
-0x54400001, 0x1324821, 0x95220000, 0x8003e99,
-0xa22821, 0x94650010, 0x94620014, 0x24690016,
-0x30420fff, 0x14400003, 0x24a5ffec, 0x8003e8c,
-0x24130001, 0x9821, 0xa03021, 0x91230001,
-0x25290004, 0x95220000, 0x25290002, 0x95240000,
-0x25290002, 0xa32821, 0xa22821, 0x95220000,
-0x95230002, 0xa42821, 0xa22821, 0xa32821,
-0x51c02, 0x30a2ffff, 0x622821, 0x51c02,
-0x30a2ffff, 0x622821, 0x96e2046a, 0x30420001,
-0x1040001e, 0x2021, 0x95820016, 0x4e2023,
-0x41402, 0x822021, 0x326200ff, 0x50400002,
-0x862021, 0x852021, 0x41402, 0x822021,
-0x3084ffff, 0x50800001, 0x3404ffff, 0x8d620018,
-0x24430017, 0x223102b, 0x54400001, 0x721821,
-0x90620000, 0x38430011, 0x2c630001, 0x38420006,
-0x2c420001, 0x621825, 0x10600004, 0x0,
-0x9562000e, 0x34420001, 0xa562000e, 0x9562000e,
-0x240a0002, 0x30420004, 0x10400002, 0xa5640012,
-0x240a0004, 0x8f880120, 0x27623800, 0x25090020,
-0x122102b, 0x50400001, 0x27693000, 0x8f820128,
-0x11220004, 0x0, 0x8f820124, 0x15220007,
-0x24040020, 0x8ee201a4, 0x8021, 0x24420001,
-0xaee201a4, 0x8003f4f, 0x8ee201a4, 0x8ee5724c,
-0x8ee60490, 0x8ee70494, 0xad0b0008, 0xa504000e,
-0xad0a0018, 0x52940, 0xa01821, 0x1021,
-0xe33821, 0xe3202b, 0xc23021, 0xc43021,
-0xad060000, 0xad070004, 0x8ee2724c, 0x4d1025,
-0xad02001c, 0x8ee204c4, 0xad020010, 0xaf890120,
-0x92e24e20, 0x14400060, 0x24100001, 0x2543ffee,
-0x2c630002, 0x39420011, 0x2c420001, 0x621825,
-0x10600024, 0x0, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x8c820000, 0x1455000f,
-0x0, 0x8ee34e30, 0x8ee24e34, 0x1062000b,
-0x0, 0x8c820004, 0x24420001, 0xac820004,
-0x8ee24e34, 0x8ee34e30, 0x24420001, 0x1054002b,
-0x0, 0x8003f2e, 0x0, 0x8ee24e30,
-0x24420001, 0x50540003, 0x1021, 0x8ee24e30,
-0x24420001, 0xaee24e30, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x24020001, 0x8003f4e,
-0xac950000, 0x8ee24e30, 0x210c0, 0x24425038,
-0x2e22021, 0x8c830000, 0x24020007, 0x1462001f,
-0x0, 0x8ee34e30, 0x8ee24e34, 0x1062001b,
-0x0, 0x8c820004, 0x24420001, 0xac820004,
-0x8ee24e34, 0x8ee34e30, 0x24420001, 0x10540007,
-0x0, 0x8ee24e34, 0x24420001, 0x10620005,
-0x0, 0x8003f3a, 0x0, 0x14600005,
-0x0, 0x8f820128, 0x24420020, 0xaf820128,
-0x8f820128, 0x8c820004, 0x2c420011, 0x50400012,
-0xac800000, 0x8003f4f, 0x0, 0x8ee24e30,
-0x24420001, 0x50540003, 0x1021, 0x8ee24e30,
-0x24420001, 0xaee24e30, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x24020007, 0xac820000,
-0x24020001, 0xac820004, 0x1600000d, 0x0,
-0x8f820120, 0x3c040001, 0x24845938, 0xafa00014,
-0xafa20010, 0x8d86001c, 0x8f870124, 0x3c050008,
-0xc002403, 0x34a50001, 0x8004057, 0x0,
-0x8ee2724c, 0x24420001, 0x304207ff, 0x11a00006,
-0xaee2724c, 0x8ee201d0, 0x2442ffff, 0xaee201d0,
-0x8003f6b, 0x8ee201d0, 0x8ee201cc, 0x2442ffff,
-0xaee201cc, 0x8ee201cc, 0x8ee201d8, 0x2442ffff,
-0xaee201d8, 0x8004057, 0x8ee201d8, 0x8f420240,
-0x104000e5, 0x0, 0x8ee20e1c, 0x24420001,
-0x8004057, 0xaee20e1c, 0x9582001e, 0xad82001c,
-0x8f420240, 0x10400072, 0x0, 0x8ee20e1c,
-0x24420001, 0xaee20e1c, 0x8f430240, 0x43102b,
-0x144000d5, 0x0, 0x8f830120, 0x27623800,
-0x24660020, 0xc2102b, 0x50400001, 0x27663000,
-0x8f820128, 0x10c20004, 0x0, 0x8f820124,
-0x14c20007, 0x0, 0x8ee201a4, 0x8021,
-0x24420001, 0xaee201a4, 0x8003fda, 0x8ee201a4,
-0x8ee2724c, 0xac62001c, 0x8ee404a8, 0x8ee504ac,
-0x2462001c, 0xac620008, 0x24020008, 0xa462000e,
-0x24020011, 0xac620018, 0xac640000, 0xac650004,
-0x8ee204c4, 0xac620010, 0xaf860120, 0x92e24e20,
-0x14400034, 0x24100001, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x8c820000, 0x1455001f,
-0x0, 0x8ee34e30, 0x8ee24e34, 0x1062001b,
-0x0, 0x8c820004, 0x24420001, 0xac820004,
-0x8ee24e34, 0x8ee34e30, 0x24420001, 0x10540007,
-0x0, 0x8ee24e34, 0x24420001, 0x10620005,
-0x0, 0x8003fc6, 0x0, 0x14600005,
-0x0, 0x8f820128, 0x24420020, 0xaf820128,
-0x8f820128, 0x8c820004, 0x2c420011, 0x50400011,
-0xac800000, 0x8003fda, 0x0, 0x8ee24e30,
-0x24420001, 0x50540003, 0x1021, 0x8ee24e30,
-0x24420001, 0xaee24e30, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x24020001, 0xac950000,
-0xac820004, 0x5600000b, 0x24100001, 0x8ee2724c,
-0x3c040001, 0x248458a8, 0xafa00014, 0xafa20010,
-0x8ee6724c, 0x8f470280, 0x3c050009, 0xc002403,
-0x34a5f008, 0x56000001, 0xaee00e1c, 0x8ee20188,
-0x24420001, 0xaee20188, 0x8004050, 0x8ee20188,
-0x8f830120, 0x27623800, 0x24660020, 0xc2102b,
-0x50400001, 0x27663000, 0x8f820128, 0x10c20004,
-0x0, 0x8f820124, 0x14c20007, 0x0,
-0x8ee201a4, 0x8021, 0x24420001, 0xaee201a4,
-0x8004044, 0x8ee201a4, 0x8ee2724c, 0xac62001c,
-0x8ee404a8, 0x8ee504ac, 0x2462001c, 0xac620008,
-0x24020008, 0xa462000e, 0x24020011, 0xac620018,
-0xac640000, 0xac650004, 0x8ee204c4, 0xac620010,
-0xaf860120, 0x92e24e20, 0x14400034, 0x24100001,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x8c820000, 0x1455001f, 0x0, 0x8ee34e30,
-0x8ee24e34, 0x1062001b, 0x0, 0x8c820004,
-0x24420001, 0xac820004, 0x8ee24e34, 0x8ee34e30,
-0x24420001, 0x10540007, 0x0, 0x8ee24e34,
-0x24420001, 0x10620005, 0x0, 0x8004030,
-0x0, 0x14600005, 0x0, 0x8f820128,
-0x24420020, 0xaf820128, 0x8f820128, 0x8c820004,
-0x2c420011, 0x50400011, 0xac800000, 0x8004044,
-0x0, 0x8ee24e30, 0x24420001, 0x50540003,
-0x1021, 0x8ee24e30, 0x24420001, 0xaee24e30,
-0x8ee24e30, 0x210c0, 0x24425038, 0x2e22021,
-0x24020001, 0xac950000, 0xac820004, 0x1600000b,
-0x0, 0x8ee2724c, 0x3c040001, 0x248458a8,
-0xafa00014, 0xafa20010, 0x8ee6724c, 0x8f470280,
-0x3c050009, 0xc002403, 0x34a5f008, 0x8ee20174,
-0x24420001, 0xaee20174, 0x8004057, 0x8ee20174,
-0x24020001, 0xaee24e24, 0x8f830128, 0x8f820124,
-0x1462fd58, 0x0, 0x8fbf0030, 0x8fb5002c,
-0x8fb40028, 0x8fb30024, 0x8fb20020, 0x8fb1001c,
-0x8fb00018, 0x3e00008, 0x27bd0038, 0x27bdffe8,
-0x27840208, 0x27450200, 0x24060008, 0xafbf0014,
-0xc00249a, 0xafb00010, 0x2021, 0x24100001,
-0x2402241f, 0xaf900210, 0xaf900200, 0xaf800204,
-0xaf820214, 0x8f460248, 0x24030004, 0x3c020040,
-0x3c010001, 0xac235cc4, 0x3c010001, 0xac235cc8,
-0x3c010001, 0xac205d9c, 0x3c010001, 0xac225cc0,
-0x3c010001, 0xac235cc8, 0xc005108, 0x24050004,
-0xc004822, 0x0, 0x8ee20000, 0x3c03feff,
-0x3463fffd, 0x431024, 0xaee20000, 0x3c023c00,
-0xaf82021c, 0x3c010001, 0x370821, 0xac3083ac,
-0x8fbf0014, 0x8fb00010, 0x3e00008, 0x27bd0018,
-0x27bdffe0, 0x3c050008, 0x34a50400, 0xafbf0018,
-0xafa00010, 0xafa00014, 0x8f860200, 0x3c040001,
-0x248459f0, 0xc002403, 0x3821, 0x8ee20280,
-0x24420001, 0xaee20280, 0x8ee20280, 0x8f830200,
-0x3c023f00, 0x621824, 0x8fbf0018, 0x3c020400,
-0x3e00008, 0x27bd0020, 0x27bdffd8, 0xafbf0020,
-0xafb1001c, 0xafb00018, 0x8f900220, 0x8ee20214,
-0x3821, 0x24420001, 0xaee20214, 0x8ee20214,
-0x3c020300, 0x2021024, 0x10400027, 0x3c110400,
-0xc00429b, 0x0, 0x3c020100, 0x2021024,
-0x10400007, 0x0, 0x8ee20218, 0x24420001,
-0xaee20218, 0x8ee20218, 0x80040c6, 0x3c03fdff,
-0x8ee2021c, 0x24420001, 0xaee2021c, 0x8ee2021c,
-0x3c03fdff, 0x3463ffff, 0x3c0808ff, 0x3508ffff,
-0x8ee20000, 0x3c040001, 0x248459fc, 0x3c050008,
-0x2003021, 0x431024, 0xaee20000, 0x8f820220,
-0x3821, 0x3c030300, 0x481024, 0x431025,
-0xaf820220, 0xafa00010, 0xc002403, 0xafa00014,
-0x8004296, 0x0, 0x2111024, 0x1040001f,
-0x3c024000, 0x8f830224, 0x24021402, 0x1462000b,
-0x3c03fdff, 0x3c040001, 0x24845a08, 0x3c050008,
-0xafa00010, 0xafa00014, 0x8f860224, 0x34a5ffff,
-0xc002403, 0x3821, 0x3c03fdff, 0x8ee20000,
-0x3463ffff, 0x2002021, 0x431024, 0xc004e54,
-0xaee20000, 0x8ee20220, 0x24420001, 0xaee20220,
-0x8ee20220, 0x8f820220, 0x3c0308ff, 0x3463ffff,
-0x431024, 0x8004295, 0x511025, 0x2021024,
-0x10400142, 0x0, 0x8ee2022c, 0x24420001,
-0xaee2022c, 0x8ee2022c, 0x8f820220, 0x3c0308ff,
-0x3463ffff, 0x431024, 0x34420004, 0xaf820220,
-0x8f830054, 0x8f820054, 0x800410e, 0x24630002,
-0x8f820054, 0x621023, 0x2c420003, 0x1440fffc,
-0x0, 0x8f8600e0, 0x8f8400e4, 0x30c20007,
-0x10400012, 0x0, 0x8f8300e4, 0x2402fff8,
-0xc21024, 0x1043000d, 0x0, 0x8f820054,
-0x8f8300e0, 0x14c30009, 0x24440050, 0x8f820054,
-0x821023, 0x2c420051, 0x10400004, 0x0,
-0x8f8200e0, 0x10c2fff9, 0x0, 0x8f820220,
-0x3c0308ff, 0x3463fffd, 0x431024, 0xaf820220,
-0x8f8600e0, 0x30c20007, 0x10400003, 0x2402fff8,
-0xc23024, 0xaf8600e0, 0x8f8300c4, 0x3c02001f,
-0x3442ffff, 0x24680008, 0x48102b, 0x10400003,
-0x3c02fff5, 0x34421000, 0x1024021, 0x8f8b00c8,
-0x8f850120, 0x8f840124, 0x8004145, 0x6021,
-0x27623800, 0x82102b, 0x50400001, 0x27643000,
-0x10a40010, 0x318200ff, 0x8c820018, 0x38430007,
-0x2c630001, 0x3842000b, 0x2c420001, 0x621825,
-0x5060fff3, 0x24840020, 0x8ee20240, 0x240c0001,
-0x24420001, 0xaee20240, 0x8ee20240, 0x8c8b0008,
-0x318200ff, 0x14400065, 0x0, 0x3c020001,
-0x571021, 0x904283c0, 0x14400060, 0x0,
-0x8f8400e4, 0xc41023, 0x218c3, 0x4620001,
-0x24630200, 0x8f8900c4, 0x10600005, 0x24020001,
-0x10620009, 0x0, 0x8004187, 0x0,
-0x8ee20230, 0x1205821, 0x24420001, 0xaee20230,
-0x80041bc, 0x8ee20230, 0x8ee20234, 0x3c05000a,
-0x24420001, 0xaee20234, 0x8c8b0000, 0x34a5f000,
-0x8ee20234, 0x12b1823, 0xa3102b, 0x54400001,
-0x651821, 0x2c62233f, 0x14400040, 0x0,
-0x8f8200e8, 0x24420008, 0xaf8200e8, 0x8f8200e8,
-0x8f8200e4, 0x1205821, 0x24420008, 0xaf8200e4,
-0x80041bc, 0x8f8200e4, 0x8ee20238, 0x3c03000a,
-0x24420001, 0xaee20238, 0x8c840000, 0x3463f000,
-0x8ee20238, 0x883823, 0x67102b, 0x54400001,
-0xe33821, 0x3c020003, 0x34420d40, 0x47102b,
-0x10400003, 0x0, 0x80041bc, 0x805821,
-0x8f8200e4, 0x24440008, 0xaf8400e4, 0x8f8400e4,
-0x10860018, 0x3c05000a, 0x34a5f000, 0x3c0a0003,
-0x354a0d40, 0x8ee2007c, 0x24420001, 0xaee2007c,
-0x8c830000, 0x8ee2007c, 0x683823, 0xa7102b,
-0x54400001, 0xe53821, 0x147102b, 0x54400007,
-0x605821, 0x8f8200e4, 0x24440008, 0xaf8400e4,
-0x8f8400e4, 0x1486ffef, 0x0, 0x14860005,
-0x0, 0x1205821, 0xaf8600e4, 0x80041bc,
-0xaf8600e8, 0xaf8400e4, 0xaf8400e8, 0x8f8200c8,
-0x3c03000a, 0x3463f000, 0x483823, 0x67102b,
-0x54400001, 0xe33821, 0x3c020003, 0x34420d3f,
-0x47102b, 0x54400007, 0x6021, 0x1683823,
-0x67102b, 0x54400003, 0xe33821, 0x80041cf,
-0x3c020003, 0x3c020003, 0x34420d3f, 0x47102b,
-0x14400016, 0x318200ff, 0x14400006, 0x0,
-0x3c020001, 0x571021, 0x904283c0, 0x1040000f,
-0x0, 0x8ee2023c, 0x3c04fdff, 0x8ee30000,
-0x3484ffff, 0x24420001, 0xaee2023c, 0x8ee2023c,
-0x24020001, 0x641824, 0x3c010001, 0x370821,
-0xa02283b8, 0x800422c, 0xaee30000, 0xaf8b00c8,
-0x8f8300c8, 0x8f8200c4, 0x3c04000a, 0x3484f000,
-0x623823, 0x87102b, 0x54400001, 0xe43821,
-0x3c020003, 0x34420d40, 0x47102b, 0x2ce30001,
-0x431025, 0x10400008, 0x0, 0x8f820220,
-0x3c0308ff, 0x3463ffff, 0x431024, 0x3c034000,
-0x431025, 0xaf820220, 0x8f8600e0, 0x8f8400e4,
-0x10c4002a, 0x0, 0x8ee2007c, 0x24420001,
-0xaee2007c, 0x8ee2007c, 0x24c2fff8, 0xaf8200e0,
-0x3c020001, 0x8c427e30, 0x3c030008, 0x8f8600e0,
-0x431024, 0x1040001d, 0x0, 0x10c4001b,
-0x240dfff8, 0x3c0a000a, 0x354af000, 0x3c0c0080,
-0x24850008, 0x27622800, 0x50a20001, 0x27651800,
-0x8c880004, 0x8c820000, 0x8ca90000, 0x3103ffff,
-0x431021, 0x4d1024, 0x24430010, 0x6b102b,
-0x54400001, 0x6a1821, 0x12b102b, 0x54400001,
-0x12a4821, 0x10690002, 0x10c1025, 0xac820004,
-0xa02021, 0x14c4ffeb, 0x24850008, 0x8f820220,
-0x3c0308ff, 0x3463ffff, 0x431024, 0x34420002,
-0xaf820220, 0x8f830054, 0x8f820054, 0x8004237,
-0x24630001, 0x8f820054, 0x621023, 0x2c420002,
-0x1440fffc, 0x0, 0x8f820220, 0x3c0308ff,
-0x3463fffb, 0x431024, 0xaf820220, 0x6010055,
-0x0, 0x8ee20228, 0x24420001, 0xaee20228,
-0x8ee20228, 0x8f820220, 0x3c0308ff, 0x3463ffff,
-0x431024, 0x34420004, 0xaf820220, 0x8f830054,
-0x8f820054, 0x8004251, 0x24630002, 0x8f820054,
-0x621023, 0x2c420003, 0x1440fffc, 0x0,
-0x8f8600e0, 0x30c20007, 0x10400012, 0x0,
-0x8f8300e4, 0x2402fff8, 0xc21024, 0x1043000d,
-0x0, 0x8f820054, 0x8f8300e0, 0x14c30009,
-0x24440032, 0x8f820054, 0x821023, 0x2c420033,
-0x10400004, 0x0, 0x8f8200e0, 0x10c2fff9,
-0x0, 0x8f820220, 0x3c0308ff, 0x3463fffd,
-0x431024, 0xaf820220, 0x8f8600e0, 0x30c20007,
-0x10400003, 0x2402fff8, 0xc23024, 0xaf8600e0,
-0x240301f5, 0x8f8200e8, 0x673823, 0x718c0,
-0x431021, 0xaf8200e8, 0x8f8200e8, 0xaf8200e4,
-0x8ee2007c, 0x3c0408ff, 0x3484ffff, 0x471021,
-0xaee2007c, 0x8f820220, 0x3c038000, 0x34630002,
-0x441024, 0x431025, 0xaf820220, 0x8f830054,
-0x8f820054, 0x800428d, 0x24630001, 0x8f820054,
-0x621023, 0x2c420002, 0x1440fffc, 0x0,
-0x8f820220, 0x3c0308ff, 0x3463fffb, 0x431024,
-0xaf820220, 0x8fbf0020, 0x8fb1001c, 0x8fb00018,
-0x3e00008, 0x27bd0028, 0x3c020001, 0x8c425cd8,
-0x27bdffd8, 0x10400012, 0xafbf0020, 0x3c040001,
-0x24845a14, 0x3c050008, 0x24020001, 0x3c010001,
-0x370821, 0xac2283ac, 0xafa00010, 0xafa00014,
-0x8f860220, 0x34a50498, 0x3c010001, 0xac205cd8,
-0x3c010001, 0xac225ccc, 0xc002403, 0x3821,
-0x8f420268, 0x3c037fff, 0x3463ffff, 0x431024,
-0xaf420268, 0x8ee204d0, 0x8ee404d4, 0x2403fffe,
-0x431024, 0x30840002, 0x1080011e, 0xaee204d0,
-0x8ee204d4, 0x2403fffd, 0x431024, 0xaee204d4,
-0x8f820044, 0x3c030600, 0x34632000, 0x34420020,
-0xaf820044, 0xafa30018, 0x8ee20608, 0x8f430228,
-0x24420001, 0x304a00ff, 0x514300fe, 0xafa00010,
-0x8ee20608, 0x210c0, 0x571021, 0x8fa30018,
-0x8fa4001c, 0xac43060c, 0xac440610, 0x8f830054,
-0x8f820054, 0x24690032, 0x1221023, 0x2c420033,
-0x1040006a, 0x5821, 0x24180008, 0x240f000d,
-0x240d0007, 0x240c0040, 0x240e0001, 0x8f870120,
-0x27623800, 0x24e80020, 0x102102b, 0x50400001,
-0x27683000, 0x8f820128, 0x11020004, 0x0,
-0x8f820124, 0x15020007, 0x1021, 0x8ee201a4,
-0x2821, 0x24420001, 0xaee201a4, 0x800433d,
-0x8ee201a4, 0x8ee40608, 0x420c0, 0x801821,
-0x8ee40430, 0x8ee50434, 0xa32821, 0xa3302b,
-0x822021, 0x862021, 0xace40000, 0xace50004,
-0x8ee20608, 0xa4f8000e, 0xacef0018, 0xacea001c,
-0x210c0, 0x2442060c, 0x2e21021, 0xace20008,
-0x8ee204c4, 0xace20010, 0xaf880120, 0x92e24e20,
-0x14400033, 0x24050001, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x8c820000, 0x144d001f,
-0x0, 0x8ee34e30, 0x8ee24e34, 0x1062001b,
-0x0, 0x8c820004, 0x24420001, 0xac820004,
-0x8ee24e34, 0x8ee34e30, 0x24420001, 0x104c0007,
-0x0, 0x8ee24e34, 0x24420001, 0x10620005,
-0x0, 0x800432a, 0x0, 0x14600005,
-0x0, 0x8f820128, 0x24420020, 0xaf820128,
-0x8f820128, 0x8c820004, 0x2c420011, 0x50400010,
-0xac800000, 0x800433d, 0x0, 0x8ee24e30,
-0x24420001, 0x504c0003, 0x1021, 0x8ee24e30,
-0x24420001, 0xaee24e30, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0xac8d0000, 0xac8e0004,
-0x54a00006, 0x240b0001, 0x8f820054, 0x1221023,
-0x2c420033, 0x1440ff9d, 0x0, 0x316300ff,
-0x24020001, 0x54620079, 0xafa00010, 0xaeea0608,
-0x8f830054, 0x8f820054, 0x24690032, 0x1221023,
-0x2c420033, 0x10400061, 0x5821, 0x240d0008,
-0x240c0011, 0x24080012, 0x24070040, 0x240a0001,
-0x8f830120, 0x27623800, 0x24660020, 0xc2102b,
-0x50400001, 0x27663000, 0x8f820128, 0x10c20004,
-0x0, 0x8f820124, 0x14c20007, 0x0,
-0x8ee201a4, 0x2821, 0x24420001, 0xaee201a4,
-0x80043a9, 0x8ee201a4, 0x8ee20608, 0xac62001c,
-0x8ee404a0, 0x8ee504a4, 0x2462001c, 0xac620008,
-0xa46d000e, 0xac6c0018, 0xac640000, 0xac650004,
-0x8ee204c4, 0xac620010, 0xaf860120, 0x92e24e20,
-0x14400033, 0x24050001, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0x8c820000, 0x1448001f,
-0x0, 0x8ee34e30, 0x8ee24e34, 0x1062001b,
-0x0, 0x8c820004, 0x24420001, 0xac820004,
-0x8ee24e34, 0x8ee34e30, 0x24420001, 0x10470007,
-0x0, 0x8ee24e34, 0x24420001, 0x10620005,
-0x0, 0x8004396, 0x0, 0x14600005,
-0x0, 0x8f820128, 0x24420020, 0xaf820128,
-0x8f820128, 0x8c820004, 0x2c420011, 0x50400010,
-0xac800000, 0x80043a9, 0x0, 0x8ee24e30,
-0x24420001, 0x50470003, 0x1021, 0x8ee24e30,
-0x24420001, 0xaee24e30, 0x8ee24e30, 0x210c0,
-0x24425038, 0x2e22021, 0xac880000, 0xac8a0004,
-0x54a00006, 0x240b0001, 0x8f820054, 0x1221023,
-0x2c420033, 0x1440ffa6, 0x0, 0x316300ff,
-0x24020001, 0x54620003, 0xafa00010, 0x80043d6,
-0x0, 0x3c040001, 0x24845a20, 0xafa00014,
-0x8f860120, 0x8f870124, 0x3c050009, 0xc002403,
-0x34a5f011, 0x80043d6, 0x0, 0x3c040001,
-0x24845a2c, 0xafa00014, 0x8f860120, 0x8f870124,
-0x3c050009, 0xc002403, 0x34a5f010, 0x80043d6,
-0x0, 0x3c040001, 0x24845a38, 0xafa00014,
-0x8ee60608, 0x8f470228, 0x3c050009, 0xc002403,
-0x34a5f00f, 0x8ee201ac, 0x24420001, 0xaee201ac,
-0x8ee201ac, 0x8ee2015c, 0x24420001, 0xaee2015c,
-0x8ee2015c, 0x8fbf0020, 0x3e00008, 0x27bd0028,
-0x3c020001, 0x8c425cd8, 0x27bdffe0, 0x1440000d,
-0xafbf0018, 0x3c040001, 0x24845a44, 0x3c050008,
-0xafa00010, 0xafa00014, 0x8f860220, 0x34a50499,
-0x24020001, 0x3c010001, 0xac225cd8, 0xc002403,
-0x3821, 0x8ee204d0, 0x3c030001, 0x771821,
-0x946383b2, 0x34420001, 0x10600007, 0xaee204d0,
-0x8f820220, 0x3c0308ff, 0x3463ffff, 0x431024,
-0x34420008, 0xaf820220, 0x2021, 0xc0052a2,
-0x24050004, 0xaf420268, 0x8fbf0018, 0x3e00008,
-0x27bd0020, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x3c120001,
-0x26521200, 0x3c140001, 0x8e945c50, 0x3c100001,
-0x26101120, 0x3c15c000, 0x36b50060, 0x8e8a0000,
-0x8eb30000, 0x26a400b, 0x248000a, 0x200f821,
-0x0, 0xd, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x80014d6,
-0x0, 0x80014d8, 0x3c0a0001, 0x80014d8,
-0x3c0a0002, 0x80014d8, 0x0, 0x80024a6,
-0x0, 0x80014d8, 0x3c0a0003, 0x80014d8,
-0x3c0a0004, 0x8002f8c, 0x0, 0x80014d8,
-0x3c0a0005, 0x8003ce8, 0x0, 0x8003c66,
-0x0, 0x80014d8, 0x3c0a0006, 0x80014d8,
-0x3c0a0007, 0x80014d8, 0x0, 0x80014d8,
-0x0, 0x80014d8, 0x0, 0x8002a75,
-0x0, 0x80014d8, 0x3c0a000b, 0x80014d8,
-0x3c0a000c, 0x80014d8, 0x3c0a000d, 0x800237a,
-0x0, 0x8002339, 0x0, 0x80014d8,
-0x3c0a000e, 0x8001b3c, 0x0, 0x80024a4,
-0x0, 0x80014d8, 0x3c0a000f, 0x80040a7,
-0x0, 0x8004091, 0x0, 0x80014d8,
-0x3c0a0010, 0x80014ee, 0x0, 0x80014d8,
-0x3c0a0011, 0x80014d8, 0x3c0a0012, 0x80014d8,
-0x3c0a0013, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x3c030001,
-0x34633800, 0x24050080, 0x2404001f, 0x2406ffff,
-0x24020001, 0xaf80021c, 0xaf820200, 0xaf820220,
-0x3631021, 0xaf8200c0, 0x3631021, 0xaf8200c4,
-0x3631021, 0xaf8200c8, 0x27623800, 0xaf8200d0,
-0x27623800, 0xaf8200d4, 0x27623800, 0xaf8200d8,
-0x27621800, 0xaf8200e0, 0x27621800, 0xaf8200e4,
-0x27621800, 0xaf8200e8, 0x27621000, 0xaf8200f0,
-0x27621000, 0xaf8200f4, 0x27621000, 0xaf8200f8,
-0xaca00000, 0x2484ffff, 0x1486fffd, 0x24a50004,
-0x8f830040, 0x3c02f000, 0x621824, 0x3c025000,
-0x1062000c, 0x43102b, 0x14400006, 0x3c026000,
-0x3c024000, 0x10620008, 0x24020800, 0x8004539,
-0x0, 0x10620004, 0x24020800, 0x8004539,
-0x0, 0x24020700, 0x3c010001, 0xac225cdc,
-0x3e00008, 0x0, 0x27bdffd8, 0xafbf0024,
-0xafb00020, 0x8f830054, 0x8f820054, 0x3c010001,
-0xac205cc4, 0x8004545, 0x24630064, 0x8f820054,
-0x621023, 0x2c420065, 0x1440fffc, 0x0,
-0xc004d71, 0x0, 0x24040001, 0x2821,
-0x27a60018, 0x34028000, 0xc00498e, 0xa7a20018,
-0x8f830054, 0x8f820054, 0x8004556, 0x24630064,
-0x8f820054, 0x621023, 0x2c420065, 0x1440fffc,
-0x24040001, 0x24050001, 0xc00494c, 0x27a60018,
-0x8f830054, 0x8f820054, 0x8004562, 0x24630064,
-0x8f820054, 0x621023, 0x2c420065, 0x1440fffc,
-0x24040001, 0x24050001, 0xc00494c, 0x27a60018,
-0x8f830054, 0x8f820054, 0x800456e, 0x24630064,
-0x8f820054, 0x621023, 0x2c420065, 0x1440fffc,
-0x24040001, 0x3c060001, 0x24c65da0, 0xc00494c,
-0x24050002, 0x8f830054, 0x8f820054, 0x800457b,
-0x24630064, 0x8f820054, 0x621023, 0x2c420065,
-0x1440fffc, 0x24040001, 0x24050003, 0x3c100001,
-0x26105da2, 0xc00494c, 0x2003021, 0x97a60018,
-0x3c070001, 0x94e75da0, 0x3c040001, 0x24845ab0,
-0xafa00014, 0x96020000, 0x3c05000d, 0x34a50100,
-0xc002403, 0xafa20010, 0x97a20018, 0x1040004c,
-0x24036040, 0x96020000, 0x3042fff0, 0x1443000a,
-0x24020020, 0x3c030001, 0x94635da0, 0x54620009,
-0x24027830, 0x24020003, 0x3c010001, 0xac225cc4,
-0x80045ac, 0x24020005, 0x3c030001, 0x94635da0,
-0x24027830, 0x1462000f, 0x24030010, 0x3c020001,
-0x94425da2, 0x3042fff0, 0x1443000a, 0x24020003,
-0x3c010001, 0xac225cc4, 0x24020006, 0x3c010001,
-0xac225db0, 0x3c010001, 0xac225dbc, 0x80045e6,
-0x3c09fff0, 0x3c020001, 0x8c425cc4, 0x3c030001,
-0x94635da0, 0x34420001, 0x3c010001, 0xac225cc4,
-0x24020015, 0x1462000f, 0x0, 0x3c020001,
-0x94425da2, 0x3042fff0, 0x3843f420, 0x2c630001,
-0x3842f430, 0x2c420001, 0x621825, 0x10600005,
-0x24020003, 0x3c010001, 0xac225dbc, 0x80045e6,
-0x3c09fff0, 0x3c030001, 0x94635da0, 0x24027810,
-0x1462000b, 0x24020002, 0x3c020001, 0x94425da2,
-0x3042fff0, 0x14400006, 0x24020002, 0x24020004,
-0x3c010001, 0xac225dbc, 0x80045e6, 0x3c09fff0,
-0x3c010001, 0xac225dbc, 0x80045e6, 0x3c09fff0,
-0x3c020001, 0x8c425cc4, 0x24030001, 0x3c010001,
-0xac235dbc, 0x34420004, 0x3c010001, 0xac225cc4,
-0x3c09fff0, 0x3529bdc0, 0x3c060001, 0x8cc65cc4,
-0x3c040001, 0x24845ab0, 0x24020001, 0x3c010001,
-0xac225ccc, 0x8f820054, 0x3c070001, 0x8ce75dbc,
-0x3c030001, 0x94635da0, 0x3c080001, 0x95085da2,
-0x3c05000d, 0x34a50100, 0x3c010001, 0xac205cc8,
-0x491021, 0x3c010001, 0xac225dac, 0xafa30010,
-0xc002403, 0xafa80014, 0x8fbf0024, 0x8fb00020,
-0x3e00008, 0x27bd0028, 0x27bdffe8, 0x3c050001,
-0x8ca55cc8, 0x24060004, 0x24020001, 0x14a20014,
-0xafbf0010, 0x3c020001, 0x8c427e3c, 0x30428000,
-0x10400005, 0x3c04000f, 0x3c030001, 0x8c635dbc,
-0x8004617, 0x34844240, 0x3c040004, 0x3c030001,
-0x8c635dbc, 0x348493e0, 0x24020005, 0x14620016,
-0x0, 0x3c04003d, 0x800462f, 0x34840900,
-0x3c020001, 0x8c427e38, 0x30428000, 0x10400005,
-0x3c04001e, 0x3c030001, 0x8c635dbc, 0x800462a,
-0x34848480, 0x3c04000f, 0x3c030001, 0x8c635dbc,
-0x34844240, 0x24020005, 0x14620003, 0x0,
-0x3c04007a, 0x34841200, 0x3c020001, 0x8c425dac,
-0x8f830054, 0x441021, 0x431023, 0x44102b,
-0x14400037, 0x0, 0x3c020001, 0x8c425cd0,
-0x14400033, 0x0, 0x3c010001, 0x10c00025,
-0xac205ce0, 0x3c090001, 0x8d295cc4, 0x24070001,
-0x3c044000, 0x3c080001, 0x25087e3c, 0x250afffc,
-0x52842, 0x14a00002, 0x24c6ffff, 0x24050008,
-0xa91024, 0x10400010, 0x0, 0x14a70008,
-0x0, 0x8d020000, 0x441024, 0x1040000a,
-0x0, 0x3c010001, 0x800465b, 0xac255ce0,
-0x8d420000, 0x441024, 0x10400003, 0x0,
-0x3c010001, 0xac275ce0, 0x3c020001, 0x8c425ce0,
-0x6182b, 0x2c420001, 0x431024, 0x5440ffe5,
-0x52842, 0x8f820054, 0x3c030001, 0x8c635ce0,
-0x3c010001, 0xac225dac, 0x1060002a, 0x24020001,
-0x3c010001, 0xac255cc8, 0x3c010001, 0xac225ccc,
-0x3c020001, 0x8c425ce0, 0x10400022, 0x0,
-0x3c020001, 0x8c425ccc, 0x1040000a, 0x24020001,
-0x3c010001, 0xac205ccc, 0x3c010001, 0x370821,
-0xac2283ac, 0x3c010001, 0xac205d4c, 0x3c010001,
-0xac225d04, 0x3c030001, 0x771821, 0x8c6383ac,
-0x24020008, 0x10620005, 0x24020001, 0xc004695,
-0x0, 0x8004692, 0x0, 0x3c030001,
-0x8c635cc8, 0x10620007, 0x2402000e, 0x3c030001,
-0x8c637dd0, 0x10620003, 0x0, 0xc004e54,
-0x8f840220, 0x8fbf0010, 0x3e00008, 0x27bd0018,
-0x27bdffe0, 0x3c02fdff, 0xafbf0018, 0x8ee30000,
-0x3c050001, 0x8ca55cc8, 0x3c040001, 0x8c845cf0,
-0x3442ffff, 0x621824, 0x14a40008, 0xaee30000,
-0x3c030001, 0x771821, 0x8c6383ac, 0x3c020001,
-0x8c425cf4, 0x10620008, 0x0, 0x3c020001,
-0x571021, 0x8c4283ac, 0x3c010001, 0xac255cf0,
-0x3c010001, 0xac225cf4, 0x3c030001, 0x8c635cc8,
-0x24020002, 0x10620169, 0x2c620003, 0x10400005,
-0x24020001, 0x10620008, 0x0, 0x800481c,
-0x0, 0x24020004, 0x106200b1, 0x24020001,
-0x800481d, 0x0, 0x3c020001, 0x571021,
-0x8c4283ac, 0x2443ffff, 0x2c620008, 0x1040015a,
-0x31080, 0x3c010001, 0x220821, 0x8c225ac8,
-0x400008, 0x0, 0x3c030001, 0x8c635dbc,
-0x24020005, 0x14620014, 0x0, 0x3c020001,
-0x8c425cd4, 0x1040000a, 0x24020003, 0xc004822,
-0x0, 0x24020002, 0x3c010001, 0x370821,
-0xac2283ac, 0x3c010001, 0x80046e0, 0xac205cd4,
-0x3c010001, 0x370821, 0xac2283ac, 0x3c010001,
-0x800481f, 0xac205c60, 0xc004822, 0x0,
-0x3c020001, 0x8c425cd4, 0x3c010001, 0xac205c60,
-0x104000dd, 0x24020002, 0x3c010001, 0x370821,
-0xac2283ac, 0x3c010001, 0x800481f, 0xac205cd4,
-0x3c030001, 0x8c635dbc, 0x24020005, 0x14620003,
-0x24020001, 0x3c010001, 0xac225d00, 0xc0049cf,
-0x0, 0x3c030001, 0x8c635d00, 0x800478e,
-0x24020011, 0x3c050001, 0x8ca55cc8, 0x3c060001,
-0x8cc67e3c, 0xc005108, 0x2021, 0x24020005,
-0x3c010001, 0xac205cd4, 0x3c010001, 0x370821,
-0x800481f, 0xac2283ac, 0x3c040001, 0x24845abc,
-0x3c05000f, 0x34a50100, 0x3021, 0x3821,
-0xafa00010, 0xc002403, 0xafa00014, 0x800481f,
-0x0, 0x8f820220, 0x3c03f700, 0x431025,
-0x80047b7, 0xaf820220, 0x8f820220, 0x3c030004,
-0x431024, 0x144000a9, 0x24020007, 0x8f830054,
-0x3c020001, 0x8c425da4, 0x2463d8f0, 0x431023,
-0x2c422710, 0x144000f8, 0x24020001, 0x800481d,
-0x0, 0x3c050001, 0x8ca55cc8, 0xc0052a2,
-0x2021, 0xc005386, 0x2021, 0x3c030001,
-0x8c637e34, 0x46100ea, 0x24020001, 0x3c020008,
-0x621024, 0x10400006, 0x0, 0x8f820214,
-0x3c03ffff, 0x431024, 0x8004741, 0x3442251f,
-0x8f820214, 0x3c03ffff, 0x431024, 0x3442241f,
-0xaf820214, 0x8ee20000, 0x3c030200, 0x431025,
-0xaee20000, 0x8f820220, 0x2403fffb, 0x431024,
-0xaf820220, 0x8f820220, 0x34420002, 0xaf820220,
-0x24020008, 0x3c010001, 0x370821, 0xac2283ac,
-0x8f820220, 0x3c030004, 0x431024, 0x14400005,
-0x0, 0x8f820220, 0x3c03f700, 0x431025,
-0xaf820220, 0x3c030001, 0x8c635dbc, 0x24020005,
-0x1462000a, 0x0, 0x3c020001, 0x94425da2,
-0x24429fbc, 0x2c420004, 0x10400004, 0x24040018,
-0x24050002, 0xc004d93, 0x24060020, 0xc0043dd,
-0x0, 0x3c010001, 0x800481f, 0xac205d50,
-0x3c020001, 0x571021, 0x8c4283ac, 0x2443ffff,
-0x2c620008, 0x104000ac, 0x31080, 0x3c010001,
-0x220821, 0x8c225ae8, 0x400008, 0x0,
-0xc00429b, 0x0, 0x3c010001, 0xac205ccc,
-0xaf800204, 0x3c010001, 0xc004822, 0xac207e20,
-0x24020001, 0x3c010001, 0xac225ce4, 0x24020002,
-0x3c010001, 0x370821, 0x800481f, 0xac2283ac,
-0xc00489f, 0x0, 0x3c030001, 0x8c635ce4,
-0x24020009, 0x14620090, 0x24020003, 0x3c010001,
-0x370821, 0x800481f, 0xac2283ac, 0x3c020001,
-0x8c427e38, 0x30424000, 0x10400005, 0x0,
-0x8f820044, 0x3c03ffff, 0x800479f, 0x34637fff,
-0x8f820044, 0x2403ff7f, 0x431024, 0xaf820044,
-0x8f830054, 0x80047b9, 0x24020004, 0x8f830054,
-0x3c020001, 0x8c425da4, 0x2463d8f0, 0x431023,
-0x2c422710, 0x14400074, 0x24020005, 0x3c010001,
-0x370821, 0x800481f, 0xac2283ac, 0x8f820220,
-0x3c03f700, 0x431025, 0xaf820220, 0xaf800204,
-0x3c010001, 0xac207e20, 0x8f830054, 0x24020006,
-0x3c010001, 0x370821, 0xac2283ac, 0x3c010001,
-0x800481f, 0xac235da4, 0x8f830054, 0x3c020001,
-0x8c425da4, 0x2463fff6, 0x431023, 0x2c42000a,
-0x14400059, 0x0, 0x24020007, 0x3c010001,
-0x370821, 0x800481f, 0xac2283ac, 0x8f820220,
-0x3c04f700, 0x441025, 0xaf820220, 0x8f820220,
-0x3c030300, 0x431024, 0x14400005, 0x1821,
-0x8f820220, 0x24030001, 0x441025, 0xaf820220,
-0x10600043, 0x24020001, 0x8f820214, 0x3c03ffff,
-0x3c040001, 0x8c845d98, 0x431024, 0x3442251f,
-0xaf820214, 0x24020008, 0x3c010001, 0x370821,
-0x1080000b, 0xac2283ac, 0x3c020001, 0x8c425d74,
-0x14400007, 0x24020001, 0x3c010001, 0xac227dd0,
-0xc004e54, 0x8f840220, 0x800480c, 0x0,
-0x8f820220, 0x3c030008, 0x431024, 0x14400017,
-0x2402000e, 0x3c010001, 0xac227dd0, 0x8ee20000,
-0x2021, 0x3c030200, 0x431025, 0xc005386,
-0xaee20000, 0x8f820220, 0x2403fffb, 0x431024,
-0xaf820220, 0x8f820220, 0x34420002, 0xc0043dd,
-0xaf820220, 0x3c050001, 0x8ca55cc8, 0xc0052a2,
-0x2021, 0x800481f, 0x0, 0x3c020001,
-0x8c425d74, 0x10400010, 0x0, 0x3c020001,
-0x8c425d70, 0x2442ffff, 0x3c010001, 0xac225d70,
-0x14400009, 0x24020002, 0x3c010001, 0xac205d74,
-0x3c010001, 0x800481f, 0xac225d70, 0x24020001,
-0x3c010001, 0xac225ccc, 0x8fbf0018, 0x3e00008,
-0x27bd0020, 0x8f820200, 0x8f820220, 0x8f820220,
-0x34420004, 0xaf820220, 0x8f820200, 0x3c060001,
-0x8cc65cc8, 0x34420004, 0xaf820200, 0x24020002,
-0x10c2003a, 0x2cc20003, 0x10400005, 0x24020001,
-0x10c20008, 0x0, 0x8004868, 0x0,
-0x24020004, 0x10c20013, 0x24020001, 0x8004868,
-0x0, 0x3c030001, 0x8c635cb8, 0x3c020001,
-0x8c425cc0, 0x3c040001, 0x8c845cdc, 0x3c050001,
-0x8ca55cbc, 0xaf860200, 0xaf860220, 0x34630022,
-0x441025, 0x451025, 0x34420002, 0x8004867,
-0xaf830200, 0x3c030001, 0x8c635d98, 0xaf820200,
-0x10600009, 0xaf820220, 0x3c020001, 0x8c425d74,
-0x14400005, 0x3c033f00, 0x3c020001, 0x8c425cb0,
-0x800485b, 0x346300e0, 0x3c020001, 0x8c425cb0,
-0x3c033f00, 0x346300e2, 0x431025, 0xaf820200,
-0x3c030001, 0x8c635cb4, 0x3c04f700, 0x3c020001,
-0x8c425cc0, 0x3c050001, 0x8ca55cdc, 0x641825,
-0x431025, 0x451025, 0xaf820220, 0x3e00008,
-0x0, 0x8f820220, 0x3c030001, 0x8c635cc8,
-0x34420004, 0xaf820220, 0x24020001, 0x1062000f,
-0x0, 0x8f830054, 0x8f820054, 0x24630002,
-0x621023, 0x2c420003, 0x10400011, 0x0,
-0x8f820054, 0x621023, 0x2c420003, 0x1040000c,
-0x0, 0x8004879, 0x0, 0x8f830054,
-0x8f820054, 0x8004885, 0x24630007, 0x8f820054,
-0x621023, 0x2c420008, 0x1440fffc, 0x0,
-0x8f8400e0, 0x30820007, 0x1040000d, 0x0,
-0x8f820054, 0x8f8300e0, 0x14830009, 0x24450032,
-0x8f820054, 0xa21023, 0x2c420033, 0x10400004,
-0x0, 0x8f8200e0, 0x1082fff9, 0x0,
-0x8f820220, 0x2403fffd, 0x431024, 0xaf820220,
-0x3e00008, 0x0, 0x3c030001, 0x8c635ce4,
-0x3c020001, 0x8c425ce8, 0x50620004, 0x2463ffff,
-0x3c010001, 0xac235ce8, 0x2463ffff, 0x2c620009,
-0x1040009d, 0x31080, 0x3c010001, 0x220821,
-0x8c225b08, 0x400008, 0x0, 0x8f820044,
-0x34428080, 0xaf820044, 0x8f830054, 0x8004938,
-0x24020002, 0x8f830054, 0x3c020001, 0x8c425da8,
-0x2463d8f0, 0x431023, 0x2c422710, 0x1440008a,
-0x24020003, 0x8004945, 0x0, 0x8f820044,
-0x3c03ffff, 0x34637fff, 0x431024, 0xaf820044,
-0x8f830054, 0x8004938, 0x24020004, 0x8f830054,
-0x3c020001, 0x8c425da8, 0x2463fff6, 0x431023,
-0x2c42000a, 0x14400078, 0x24020005, 0x8004945,
-0x0, 0x8f820220, 0x3c03f700, 0x431025,
-0xaf820220, 0x8f820220, 0x2403fffb, 0x431024,
-0xaf820220, 0x8f820220, 0x34420002, 0xaf820220,
-0x3c023f00, 0x344200e0, 0xaf820200, 0x8f820200,
-0x2403fffd, 0x431024, 0xaf820200, 0x24040001,
-0x3405ffff, 0xaf840204, 0x8f830054, 0x8f820054,
-0x80048ec, 0x24630001, 0x8f820054, 0x621023,
-0x2c420002, 0x1440fffc, 0x0, 0x8f820224,
-0x42040, 0xa4102b, 0x1040fff2, 0x0,
-0x8f820220, 0x3c03f700, 0x431025, 0xaf820220,
-0x8f820214, 0x3c03ffff, 0x431024, 0x3442251f,
-0xaf820214, 0x8f820220, 0x2403fffb, 0x431024,
-0xaf820220, 0x8f820220, 0x3c04f700, 0x34840008,
-0x34420002, 0xaf820220, 0x8f820220, 0x3c033f00,
-0x346300e2, 0x441025, 0xaf820220, 0xaf830200,
-0x8f8400f0, 0x276217f8, 0x14820002, 0x24850008,
-0x27651000, 0x8f8200f4, 0x10a20007, 0x3c038000,
-0x34630040, 0x3c020001, 0x24425c70, 0xac820000,
-0xac830004, 0xaf8500f0, 0x8f830054, 0x8004938,
-0x24020006, 0x8f830054, 0x3c020001, 0x8c425da8,
-0x2463fff6, 0x431023, 0x2c42000a, 0x14400022,
-0x24020007, 0x8004945, 0x0, 0x8f8200e0,
-0xaf8200e4, 0x8f8200e0, 0xaf8200e8, 0x8f820220,
-0x34420004, 0xaf820220, 0x8f820220, 0x2403fff7,
-0x431024, 0xaf820220, 0x8f820044, 0x34428080,
-0xaf820044, 0x8f830054, 0x24020008, 0x3c010001,
-0xac225ce4, 0x3c010001, 0x8004947, 0xac235da8,
-0x8f830054, 0x3c020001, 0x8c425da8, 0x2463d8f0,
-0x431023, 0x2c422710, 0x14400003, 0x24020009,
-0x3c010001, 0xac225ce4, 0x3e00008, 0x0,
-0x0, 0x0, 0x0, 0x27bdffd8,
-0xafb20018, 0x809021, 0xafb3001c, 0xa09821,
-0xafb10014, 0xc08821, 0xafb00010, 0x8021,
-0xafbf0020, 0xa6200000, 0xc004d4b, 0x24040001,
-0x26100001, 0x2e020020, 0x1440fffb, 0x0,
-0xc004d4b, 0x2021, 0xc004d4b, 0x24040001,
-0xc004d4b, 0x24040001, 0xc004d4b, 0x2021,
-0x24100010, 0x2501024, 0x10400002, 0x2021,
-0x24040001, 0xc004d4b, 0x108042, 0x1600fffa,
-0x2501024, 0x24100010, 0x2701024, 0x10400002,
-0x2021, 0x24040001, 0xc004d4b, 0x108042,
-0x1600fffa, 0x2701024, 0xc004d71, 0x34108000,
-0xc004d71, 0x0, 0xc004d2b, 0x0,
-0x50400005, 0x108042, 0x96220000, 0x501025,
-0xa6220000, 0x108042, 0x1600fff7, 0x0,
-0xc004d71, 0x0, 0x8fbf0020, 0x8fb3001c,
-0x8fb20018, 0x8fb10014, 0x8fb00010, 0x3e00008,
-0x27bd0028, 0x27bdffd8, 0xafb10014, 0x808821,
-0xafb20018, 0xa09021, 0xafb3001c, 0xc09821,
-0xafb00010, 0x8021, 0xafbf0020, 0xc004d4b,
-0x24040001, 0x26100001, 0x2e020020, 0x1440fffb,
-0x0, 0xc004d4b, 0x2021, 0xc004d4b,
-0x24040001, 0xc004d4b, 0x2021, 0xc004d4b,
-0x24040001, 0x24100010, 0x2301024, 0x10400002,
-0x2021, 0x24040001, 0xc004d4b, 0x108042,
-0x1600fffa, 0x2301024, 0x24100010, 0x2501024,
-0x10400002, 0x2021, 0x24040001, 0xc004d4b,
-0x108042, 0x1600fffa, 0x2501024, 0xc004d4b,
-0x24040001, 0xc004d4b, 0x2021, 0x34108000,
-0x96620000, 0x501024, 0x10400002, 0x2021,
-0x24040001, 0xc004d4b, 0x108042, 0x1600fff8,
-0x0, 0xc004d71, 0x0, 0x8fbf0020,
-0x8fb3001c, 0x8fb20018, 0x8fb10014, 0x8fb00010,
-0x3e00008, 0x27bd0028, 0x3c030001, 0x8c635d00,
-0x3c020001, 0x8c425d48, 0x27bdffd8, 0xafbf0020,
-0xafb1001c, 0x10620003, 0xafb00018, 0x3c010001,
-0xac235d48, 0x2463ffff, 0x2c620013, 0x10400349,
-0x31080, 0x3c010001, 0x220821, 0x8c225b30,
-0x400008, 0x0, 0xc004d71, 0x8021,
-0x34028000, 0xa7a20010, 0x27b10010, 0xc004d4b,
-0x24040001, 0x26100001, 0x2e020020, 0x1440fffb,
-0x0, 0xc004d4b, 0x2021, 0xc004d4b,
-0x24040001, 0xc004d4b, 0x2021, 0xc004d4b,
-0x24040001, 0x24100010, 0x32020001, 0x10400002,
-0x2021, 0x24040001, 0xc004d4b, 0x108042,
-0x1600fffa, 0x32020001, 0x24100010, 0xc004d4b,
-0x2021, 0x108042, 0x1600fffc, 0x0,
-0xc004d4b, 0x24040001, 0xc004d4b, 0x2021,
-0x34108000, 0x96220000, 0x501024, 0x10400002,
-0x2021, 0x24040001, 0xc004d4b, 0x108042,
-0x1600fff8, 0x0, 0xc004d71, 0x0,
-0x8004d24, 0x24020002, 0x27b10010, 0xa7a00010,
-0x8021, 0xc004d4b, 0x24040001, 0x26100001,
-0x2e020020, 0x1440fffb, 0x0, 0xc004d4b,
-0x2021, 0xc004d4b, 0x24040001, 0xc004d4b,
-0x24040001, 0xc004d4b, 0x2021, 0x24100010,
-0x32020001, 0x10400002, 0x2021, 0x24040001,
-0xc004d4b, 0x108042, 0x1600fffa, 0x32020001,
-0x24100010, 0xc004d4b, 0x2021, 0x108042,
-0x1600fffc, 0x0, 0xc004d71, 0x34108000,
-0xc004d71, 0x0, 0xc004d2b, 0x0,
-0x50400005, 0x108042, 0x96220000, 0x501025,
-0xa6220000, 0x108042, 0x1600fff7, 0x0,
-0xc004d71, 0x0, 0x97a20010, 0x30428000,
-0x144002dc, 0x24020003, 0x8004d24, 0x0,
-0x24021200, 0xa7a20010, 0x27b10010, 0x8021,
-0xc004d4b, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d4b, 0x2021,
-0xc004d4b, 0x24040001, 0xc004d4b, 0x2021,
-0xc004d4b, 0x24040001, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d4b,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0xc004d4b, 0x2021, 0x108042, 0x1600fffc,
-0x0, 0xc004d4b, 0x24040001, 0xc004d4b,
-0x2021, 0x34108000, 0x96220000, 0x501024,
-0x10400002, 0x2021, 0x24040001, 0xc004d4b,
-0x108042, 0x1600fff8, 0x0, 0xc004d71,
-0x0, 0x8f830054, 0x8004d16, 0x24020004,
-0x8f830054, 0x3c020001, 0x8c425db8, 0x2463ff9c,
-0x431023, 0x2c420064, 0x1440029e, 0x24020002,
-0x3c030001, 0x8c635dbc, 0x10620297, 0x2c620003,
-0x14400296, 0x24020011, 0x24020003, 0x10620005,
-0x24020004, 0x10620291, 0x2402000f, 0x8004d24,
-0x24020011, 0x8004d24, 0x24020005, 0x24020014,
-0xa7a20010, 0x27b10010, 0x8021, 0xc004d4b,
-0x24040001, 0x26100001, 0x2e020020, 0x1440fffb,
-0x0, 0xc004d4b, 0x2021, 0xc004d4b,
-0x24040001, 0xc004d4b, 0x2021, 0xc004d4b,
-0x24040001, 0x24100010, 0x32020001, 0x10400002,
-0x2021, 0x24040001, 0xc004d4b, 0x108042,
-0x1600fffa, 0x32020001, 0x24100010, 0x32020012,
-0x10400002, 0x2021, 0x24040001, 0xc004d4b,
-0x108042, 0x1600fffa, 0x32020012, 0xc004d4b,
-0x24040001, 0xc004d4b, 0x2021, 0x34108000,
-0x96220000, 0x501024, 0x10400002, 0x2021,
-0x24040001, 0xc004d4b, 0x108042, 0x1600fff8,
-0x0, 0xc004d71, 0x0, 0x8f830054,
-0x8004d16, 0x24020006, 0x8f830054, 0x3c020001,
-0x8c425db8, 0x2463ff9c, 0x431023, 0x2c420064,
-0x14400250, 0x24020007, 0x8004d24, 0x0,
-0x24020006, 0xa7a20010, 0x27b10010, 0x8021,
-0xc004d4b, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d4b, 0x2021,
-0xc004d4b, 0x24040001, 0xc004d4b, 0x2021,
-0xc004d4b, 0x24040001, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d4b,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020013, 0x10400002, 0x2021, 0x24040001,
-0xc004d4b, 0x108042, 0x1600fffa, 0x32020013,
-0xc004d4b, 0x24040001, 0xc004d4b, 0x2021,
-0x34108000, 0x96220000, 0x501024, 0x10400002,
-0x2021, 0x24040001, 0xc004d4b, 0x108042,
-0x1600fff8, 0x0, 0xc004d71, 0x0,
-0x8f830054, 0x8004d16, 0x24020008, 0x8f830054,
-0x3c020001, 0x8c425db8, 0x2463ff9c, 0x431023,
-0x2c420064, 0x1440020f, 0x24020009, 0x8004d24,
-0x0, 0x27b10010, 0xa7a00010, 0x8021,
-0xc004d4b, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d4b, 0x2021,
-0xc004d4b, 0x24040001, 0xc004d4b, 0x24040001,
-0xc004d4b, 0x2021, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d4b,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020018, 0x10400002, 0x2021, 0x24040001,
-0xc004d4b, 0x108042, 0x1600fffa, 0x32020018,
-0xc004d71, 0x34108000, 0xc004d71, 0x0,
-0xc004d2b, 0x0, 0x50400005, 0x108042,
-0x96220000, 0x501025, 0xa6220000, 0x108042,
-0x1600fff7, 0x0, 0xc004d71, 0x8021,
-0x97a20010, 0x27b10010, 0x34420001, 0xa7a20010,
-0xc004d4b, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d4b, 0x2021,
-0xc004d4b, 0x24040001, 0xc004d4b, 0x2021,
-0xc004d4b, 0x24040001, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d4b,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020018, 0x10400002, 0x2021, 0x24040001,
-0xc004d4b, 0x108042, 0x1600fffa, 0x32020018,
-0xc004d4b, 0x24040001, 0xc004d4b, 0x2021,
-0x34108000, 0x96220000, 0x501024, 0x10400002,
-0x2021, 0x24040001, 0xc004d4b, 0x108042,
-0x1600fff8, 0x0, 0xc004d71, 0x0,
-0x8f830054, 0x8004d16, 0x2402000a, 0x8f830054,
-0x3c020001, 0x8c425db8, 0x2463ff9c, 0x431023,
-0x2c420064, 0x1440019b, 0x2402000b, 0x8004d24,
-0x0, 0x27b10010, 0xa7a00010, 0x8021,
-0xc004d4b, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d4b, 0x2021,
-0xc004d4b, 0x24040001, 0xc004d4b, 0x24040001,
-0xc004d4b, 0x2021, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d4b,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020017, 0x10400002, 0x2021, 0x24040001,
-0xc004d4b, 0x108042, 0x1600fffa, 0x32020017,
-0xc004d71, 0x34108000, 0xc004d71, 0x0,
-0xc004d2b, 0x0, 0x50400005, 0x108042,
-0x96220000, 0x501025, 0xa6220000, 0x108042,
-0x1600fff7, 0x0, 0xc004d71, 0x8021,
-0x97a20010, 0x27b10010, 0x34420700, 0xa7a20010,
-0xc004d4b, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d4b, 0x2021,
-0xc004d4b, 0x24040001, 0xc004d4b, 0x2021,
-0xc004d4b, 0x24040001, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d4b,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020017, 0x10400002, 0x2021, 0x24040001,
-0xc004d4b, 0x108042, 0x1600fffa, 0x32020017,
-0xc004d4b, 0x24040001, 0xc004d4b, 0x2021,
-0x34108000, 0x96220000, 0x501024, 0x10400002,
-0x2021, 0x24040001, 0xc004d4b, 0x108042,
-0x1600fff8, 0x0, 0xc004d71, 0x0,
-0x8f830054, 0x8004d16, 0x2402000c, 0x8f830054,
-0x3c020001, 0x8c425db8, 0x2463ff9c, 0x431023,
-0x2c420064, 0x14400127, 0x24020012, 0x8004d24,
-0x0, 0x27b10010, 0xa7a00010, 0x8021,
-0xc004d4b, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d4b, 0x2021,
-0xc004d4b, 0x24040001, 0xc004d4b, 0x24040001,
-0xc004d4b, 0x2021, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d4b,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020014, 0x10400002, 0x2021, 0x24040001,
-0xc004d4b, 0x108042, 0x1600fffa, 0x32020014,
-0xc004d71, 0x34108000, 0xc004d71, 0x0,
-0xc004d2b, 0x0, 0x50400005, 0x108042,
-0x96220000, 0x501025, 0xa6220000, 0x108042,
-0x1600fff7, 0x0, 0xc004d71, 0x8021,
-0x97a20010, 0x27b10010, 0x34420010, 0xa7a20010,
-0xc004d4b, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d4b, 0x2021,
-0xc004d4b, 0x24040001, 0xc004d4b, 0x2021,
-0xc004d4b, 0x24040001, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d4b,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020014, 0x10400002, 0x2021, 0x24040001,
-0xc004d4b, 0x108042, 0x1600fffa, 0x32020014,
-0xc004d4b, 0x24040001, 0xc004d4b, 0x2021,
-0x34108000, 0x96220000, 0x501024, 0x10400002,
-0x2021, 0x24040001, 0xc004d4b, 0x108042,
-0x1600fff8, 0x0, 0xc004d71, 0x0,
-0x8f830054, 0x8004d16, 0x24020013, 0x8f830054,
-0x3c020001, 0x8c425db8, 0x2463ff9c, 0x431023,
-0x2c420064, 0x144000b3, 0x2402000d, 0x8004d24,
-0x0, 0x27b10010, 0xa7a00010, 0x8021,
-0xc004d4b, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d4b, 0x2021,
-0xc004d4b, 0x24040001, 0xc004d4b, 0x24040001,
-0xc004d4b, 0x2021, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d4b,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020018, 0x10400002, 0x2021, 0x24040001,
-0xc004d4b, 0x108042, 0x1600fffa, 0x32020018,
-0xc004d71, 0x34108000, 0xc004d71, 0x0,
-0xc004d2b, 0x0, 0x50400005, 0x108042,
-0x96220000, 0x501025, 0xa6220000, 0x108042,
-0x1600fff7, 0x0, 0xc004d71, 0x8021,
-0x97a20010, 0x27b10010, 0x3042fffe, 0xa7a20010,
-0xc004d4b, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d4b, 0x2021,
-0xc004d4b, 0x24040001, 0xc004d4b, 0x2021,
-0xc004d4b, 0x24040001, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d4b,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020018, 0x10400002, 0x2021, 0x24040001,
-0xc004d4b, 0x108042, 0x1600fffa, 0x32020018,
-0xc004d4b, 0x24040001, 0xc004d4b, 0x2021,
-0x34108000, 0x96220000, 0x501024, 0x10400002,
-0x2021, 0x24040001, 0xc004d4b, 0x108042,
-0x1600fff8, 0x0, 0xc004d71, 0x0,
-0x8f830054, 0x8004d16, 0x2402000e, 0x24020840,
-0xa7a20010, 0x27b10010, 0x8021, 0xc004d4b,
-0x24040001, 0x26100001, 0x2e020020, 0x1440fffb,
-0x0, 0xc004d4b, 0x2021, 0xc004d4b,
-0x24040001, 0xc004d4b, 0x2021, 0xc004d4b,
-0x24040001, 0x24100010, 0x32020001, 0x10400002,
-0x2021, 0x24040001, 0xc004d4b, 0x108042,
-0x1600fffa, 0x32020001, 0x24100010, 0x32020013,
-0x10400002, 0x2021, 0x24040001, 0xc004d4b,
-0x108042, 0x1600fffa, 0x32020013, 0xc004d4b,
-0x24040001, 0xc004d4b, 0x2021, 0x34108000,
-0x96220000, 0x501024, 0x10400002, 0x2021,
-0x24040001, 0xc004d4b, 0x108042, 0x1600fff8,
-0x0, 0xc004d71, 0x0, 0x8f830054,
-0x24020010, 0x3c010001, 0xac225d00, 0x3c010001,
-0x8004d26, 0xac235db8, 0x8f830054, 0x3c020001,
-0x8c425db8, 0x2463ff9c, 0x431023, 0x2c420064,
-0x14400004, 0x0, 0x24020011, 0x3c010001,
-0xac225d00, 0x8fbf0020, 0x8fb1001c, 0x8fb00018,
-0x3e00008, 0x27bd0028, 0x8f850044, 0x8f820044,
-0x3c030001, 0x431025, 0x3c030008, 0xaf820044,
-0x8f840054, 0x8f820054, 0xa32824, 0x8004d37,
-0x24840001, 0x8f820054, 0x821023, 0x2c420002,
-0x1440fffc, 0x0, 0x8f820044, 0x3c03fffe,
-0x3463ffff, 0x431024, 0xaf820044, 0x8f830054,
-0x8f820054, 0x8004d45, 0x24630001, 0x8f820054,
-0x621023, 0x2c420002, 0x1440fffc, 0x0,
-0x3e00008, 0xa01021, 0x8f830044, 0x3c02fff0,
-0x3442ffff, 0x42480, 0x621824, 0x3c020002,
-0x822025, 0x641825, 0xaf830044, 0x8f820044,
-0x3c03fffe, 0x3463ffff, 0x431024, 0xaf820044,
-0x8f830054, 0x8f820054, 0x8004d5e, 0x24630001,
-0x8f820054, 0x621023, 0x2c420002, 0x1440fffc,
-0x0, 0x8f820044, 0x3c030001, 0x431025,
-0xaf820044, 0x8f830054, 0x8f820054, 0x8004d6b,
-0x24630001, 0x8f820054, 0x621023, 0x2c420002,
-0x1440fffc, 0x0, 0x3e00008, 0x0,
-0x8f820044, 0x3c03fff0, 0x3463ffff, 0x431024,
-0xaf820044, 0x8f820044, 0x3c030001, 0x431025,
-0xaf820044, 0x8f830054, 0x8f820054, 0x8004d7f,
-0x24630001, 0x8f820054, 0x621023, 0x2c420002,
-0x1440fffc, 0x0, 0x8f820044, 0x3c03fffe,
-0x3463ffff, 0x431024, 0xaf820044, 0x8f830054,
-0x8f820054, 0x8004d8d, 0x24630001, 0x8f820054,
-0x621023, 0x2c420002, 0x1440fffc, 0x0,
-0x3e00008, 0x0, 0x27bdffc8, 0xafb30024,
-0x809821, 0xafb5002c, 0xa0a821, 0xafb20020,
-0xc09021, 0x32a2ffff, 0xafbf0030, 0xafb40028,
-0xafb1001c, 0xafb00018, 0x14400034, 0xa7b20010,
-0x3271ffff, 0x27b20010, 0x8021, 0xc004d4b,
-0x24040001, 0x26100001, 0x2e020020, 0x1440fffb,
-0x0, 0xc004d4b, 0x2021, 0xc004d4b,
-0x24040001, 0xc004d4b, 0x2021, 0xc004d4b,
-0x24040001, 0x24100010, 0x32020001, 0x10400002,
-0x2021, 0x24040001, 0xc004d4b, 0x108042,
-0x1600fffa, 0x32020001, 0x24100010, 0x2301024,
-0x10400002, 0x2021, 0x24040001, 0xc004d4b,
-0x108042, 0x1600fffa, 0x2301024, 0xc004d4b,
-0x24040001, 0xc004d4b, 0x2021, 0x34108000,
-0x96420000, 0x501024, 0x10400002, 0x2021,
-0x24040001, 0xc004d4b, 0x108042, 0x12000075,
-0x0, 0x8004dc9, 0x0, 0x3274ffff,
-0x27b10010, 0xa7a00010, 0x8021, 0xc004d4b,
-0x24040001, 0x26100001, 0x2e020020, 0x1440fffb,
-0x0, 0xc004d4b, 0x2021, 0xc004d4b,
-0x24040001, 0xc004d4b, 0x24040001, 0xc004d4b,
-0x2021, 0x24100010, 0x32020001, 0x10400002,
-0x2021, 0x24040001, 0xc004d4b, 0x108042,
-0x1600fffa, 0x32020001, 0x24100010, 0x2901024,
-0x10400002, 0x2021, 0x24040001, 0xc004d4b,
-0x108042, 0x1600fffa, 0x2901024, 0xc004d71,
-0x34108000, 0xc004d71, 0x0, 0xc004d2b,
-0x0, 0x50400005, 0x108042, 0x96220000,
-0x501025, 0xa6220000, 0x108042, 0x1600fff7,
-0x0, 0xc004d71, 0x0, 0x32a5ffff,
-0x24020001, 0x54a20004, 0x24020002, 0x97a20010,
-0x8004e14, 0x521025, 0x14a20006, 0x3271ffff,
-0x97a20010, 0x121827, 0x431024, 0xa7a20010,
-0x3271ffff, 0x27b20010, 0x8021, 0xc004d4b,
-0x24040001, 0x26100001, 0x2e020020, 0x1440fffb,
-0x0, 0xc004d4b, 0x2021, 0xc004d4b,
-0x24040001, 0xc004d4b, 0x2021, 0xc004d4b,
-0x24040001, 0x24100010, 0x32020001, 0x10400002,
-0x2021, 0x24040001, 0xc004d4b, 0x108042,
-0x1600fffa, 0x32020001, 0x24100010, 0x2301024,
-0x10400002, 0x2021, 0x24040001, 0xc004d4b,
-0x108042, 0x1600fffa, 0x2301024, 0xc004d4b,
-0x24040001, 0xc004d4b, 0x2021, 0x34108000,
-0x96420000, 0x501024, 0x10400002, 0x2021,
-0x24040001, 0xc004d4b, 0x108042, 0x1600fff8,
-0x0, 0xc004d71, 0x0, 0x8fbf0030,
-0x8fb5002c, 0x8fb40028, 0x8fb30024, 0x8fb20020,
-0x8fb1001c, 0x8fb00018, 0x3e00008, 0x27bd0038,
-0x0, 0x0, 0x0, 0x27bdffe8,
-0xafbf0010, 0x3c030001, 0x771821, 0x8c6383ac,
-0x24020008, 0x1462022c, 0x803021, 0x3c020001,
-0x8c425d98, 0x14400033, 0x0, 0x8f850224,
-0x38a30020, 0x2c630001, 0x38a20010, 0x2c420001,
-0x621825, 0x1460000d, 0x38a30030, 0x2c630001,
-0x38a20400, 0x2c420001, 0x621825, 0x14600007,
-0x38a30402, 0x2c630001, 0x38a20404, 0x2c420001,
-0x621825, 0x10600005, 0x0, 0xc00429b,
-0x0, 0x8004e8d, 0x2402000e, 0xc0043dd,
-0x0, 0x3c050001, 0x8ca55cc8, 0xc0052a2,
-0x2021, 0x3c030001, 0x8c635cc8, 0x24020004,
-0x14620005, 0x2403fffb, 0x3c020001, 0x8c425cc4,
-0x8004e89, 0x2403fff7, 0x3c020001, 0x8c425cc4,
-0x431024, 0x3c010001, 0xac225cc4, 0x2402000e,
-0x3c010001, 0xc00429b, 0xac227dd0, 0x8005087,
-0x0, 0x8f820220, 0x3c030400, 0x431024,
-0x10400027, 0x2403ffbf, 0x8f850224, 0x3c020001,
-0x8c427ddc, 0xa32024, 0x431024, 0x1482000c,
-0x0, 0x3c020001, 0x8c427de0, 0x24420001,
-0x3c010001, 0xac227de0, 0x2c420002, 0x14400008,
-0x24020001, 0x3c010001, 0x8004ead, 0xac227e00,
-0x3c010001, 0xac207de0, 0x3c010001, 0xac207e00,
-0x3c020001, 0x8c427e00, 0x10400006, 0x30a20040,
-0x10400004, 0x24020001, 0x3c010001, 0x8004eb8,
-0xac227e04, 0x3c010001, 0xac207e04, 0x3c010001,
-0xac257ddc, 0x3c010001, 0x8004ec8, 0xac207e10,
-0x24020001, 0x3c010001, 0xac227e10, 0x3c010001,
-0xac207e00, 0x3c010001, 0xac207de0, 0x3c010001,
-0xac207e04, 0x3c010001, 0xac207ddc, 0x3c030001,
-0x8c637dd0, 0x3c020001, 0x8c427dd4, 0x10620003,
-0x3c020200, 0x3c010001, 0xac237dd4, 0xc21024,
-0x10400007, 0x2463ffff, 0x8f820220, 0x24030001,
-0x3c010001, 0xac235ccc, 0x8005085, 0x3c03f700,
-0x2c62000e, 0x104001a8, 0x31080, 0x3c010001,
-0x220821, 0x8c225b80, 0x400008, 0x0,
-0x3c010001, 0xac207e00, 0x3c010001, 0xac207de0,
-0x3c010001, 0xac207ddc, 0x3c010001, 0xac207e04,
-0x3c010001, 0xac207df8, 0x3c010001, 0xac207df0,
-0xc00486a, 0xaf800224, 0x24020002, 0x3c010001,
-0xac227dd0, 0x3c020001, 0x8c427e10, 0x14400056,
-0x3c03fdff, 0x8ee20000, 0x3463ffff, 0x431024,
-0xc00429b, 0xaee20000, 0xaf800204, 0x8f820200,
-0x2403fffd, 0x431024, 0xaf820200, 0x3c010001,
-0xac207e20, 0x8f830054, 0x3c020001, 0x8c427df8,
-0x24040001, 0x3c010001, 0xac247e0c, 0x24420001,
-0x3c010001, 0xac227df8, 0x2c420004, 0x3c010001,
-0xac237df4, 0x14400006, 0x24020003, 0x3c010001,
-0xac245ccc, 0x3c010001, 0x8005083, 0xac207df8,
-0x3c010001, 0x8005083, 0xac227dd0, 0x8f830054,
-0x3c020001, 0x8c427df4, 0x2463d8f0, 0x431023,
-0x2c422710, 0x14400003, 0x24020004, 0x3c010001,
-0xac227dd0, 0x3c020001, 0x8c427e10, 0x14400026,
-0x3c03fdff, 0x8ee20000, 0x3463ffff, 0x431024,
-0x8005083, 0xaee20000, 0x3c040001, 0x8c845d9c,
-0x3c010001, 0xc00508a, 0xac207de8, 0x3c020001,
-0x8c427e1c, 0xaf820204, 0x3c020001, 0x8c427e10,
-0x14400015, 0x3c03fdff, 0x8ee20000, 0x3463ffff,
-0x431024, 0xaee20000, 0x8f820204, 0x30420030,
-0x1440013c, 0x24020002, 0x3c030001, 0x8c637e1c,
-0x24020005, 0x3c010001, 0xac227dd0, 0x3c010001,
-0x8005083, 0xac237e20, 0x3c020001, 0x8c427e10,
-0x10400010, 0x3c03fdff, 0x3c020001, 0x8c425d6c,
-0x24420001, 0x3c010001, 0xac225d6c, 0x2c420002,
-0x14400131, 0x24020001, 0x3c010001, 0xac225d74,
-0x3c010001, 0xac205d6c, 0x3c010001, 0x8005083,
-0xac225ccc, 0x8ee20000, 0x3463ffff, 0x431024,
-0xaee20000, 0x3c020001, 0x8c427e00, 0x10400122,
-0x0, 0x3c020001, 0x8c427ddc, 0x1040011e,
-0x0, 0x3c010001, 0xac227e08, 0x24020003,
-0x3c010001, 0xac227de0, 0x8005024, 0x24020006,
-0x3c010001, 0xac207de8, 0x8f820204, 0x34420040,
-0xaf820204, 0x3c020001, 0x8c427e20, 0x24030007,
-0x3c010001, 0xac237dd0, 0x34420040, 0x3c010001,
-0xac227e20, 0x3c020001, 0x8c427e00, 0x10400005,
-0x0, 0x3c020001, 0x8c427ddc, 0x104000f9,
-0x24020002, 0x3c050001, 0x24a57de0, 0x8ca20000,
-0x2c424e21, 0x104000f3, 0x24020002, 0x3c020001,
-0x8c427e04, 0x104000f8, 0x2404ffbf, 0x3c020001,
-0x8c427ddc, 0x3c030001, 0x8c637e08, 0x441024,
-0x641824, 0x10430004, 0x24020001, 0x3c010001,
-0x8005083, 0xac227dd0, 0x24020003, 0xaca20000,
-0x24020008, 0x3c010001, 0xac227dd0, 0x3c020001,
-0x8c427e0c, 0x1040000c, 0x24020001, 0x3c040001,
-0xc005097, 0x8c847ddc, 0x3c020001, 0x8c427e28,
-0x14400005, 0x24020001, 0x3c020001, 0x8c427e24,
-0x10400006, 0x24020001, 0x3c010001, 0xac225ccc,
-0x3c010001, 0x8005083, 0xac207df8, 0x3c020001,
-0x8c427df0, 0x3c030001, 0x8c637ddc, 0x2c420001,
-0x210c0, 0x30630008, 0x3c010001, 0xac227df0,
-0x3c010001, 0xac237dec, 0x8f830054, 0x24020009,
-0x3c010001, 0xac227dd0, 0x3c010001, 0x8005083,
-0xac237df4, 0x8f830054, 0x3c020001, 0x8c427df4,
-0x2463d8f0, 0x431023, 0x2c422710, 0x144000a8,
-0x0, 0x3c020001, 0x8c427e00, 0x10400005,
-0x0, 0x3c020001, 0x8c427ddc, 0x104000a9,
-0x24020002, 0x3c030001, 0x24637de0, 0x8c620000,
-0x2c424e21, 0x104000a3, 0x24020002, 0x3c020001,
-0x8c427e0c, 0x1040000e, 0x0, 0x3c020001,
-0x8c427ddc, 0x3c010001, 0xac207e0c, 0x30420080,
-0x1040002f, 0x2402000c, 0x8f820204, 0x30420080,
-0x1440000c, 0x24020003, 0x8005011, 0x2402000c,
-0x3c020001, 0x8c427ddc, 0x30420080, 0x14400005,
-0x24020003, 0x8f820204, 0x30420080, 0x1040001f,
-0x24020003, 0xac620000, 0x2402000a, 0x3c010001,
-0xac227dd0, 0x3c040001, 0x24847e18, 0x8c820000,
-0x3c030001, 0x8c637df0, 0x431025, 0xaf820204,
-0x8c830000, 0x3c040001, 0x8c847df0, 0x2402000b,
-0x3c010001, 0xac227dd0, 0x641825, 0x3c010001,
-0xac237e20, 0x3c050001, 0x24a57de0, 0x8ca20000,
-0x2c424e21, 0x1040006f, 0x24020002, 0x3c020001,
-0x8c427e10, 0x10400005, 0x0, 0x2402000c,
-0x3c010001, 0x8005083, 0xac227dd0, 0x3c020001,
-0x8c427e00, 0x1040006c, 0x0, 0x3c040001,
-0x8c847ddc, 0x1080005e, 0x30820008, 0x3c030001,
-0x8c637dec, 0x10620064, 0x24020003, 0x3c010001,
-0xac247e08, 0xaca20000, 0x24020006, 0x3c010001,
-0x8005083, 0xac227dd0, 0x8f820200, 0x34420002,
-0xaf820200, 0x8f830054, 0x2402000d, 0x3c010001,
-0xac227dd0, 0x3c010001, 0xac237df4, 0x8f830054,
-0x3c020001, 0x8c427df4, 0x2463d8f0, 0x431023,
-0x2c422710, 0x1440003a, 0x0, 0x3c020001,
-0x8c427e10, 0x10400029, 0x2402000e, 0x3c030001,
-0x8c637e24, 0x3c010001, 0x14600015, 0xac227dd0,
-0xc0043dd, 0x0, 0x3c050001, 0x8ca55cc8,
-0xc0052a2, 0x2021, 0x3c030001, 0x8c635cc8,
-0x24020004, 0x14620005, 0x2403fffb, 0x3c020001,
-0x8c425cc4, 0x8005052, 0x2403fff7, 0x3c020001,
-0x8c425cc4, 0x431024, 0x3c010001, 0xac225cc4,
-0x8ee20000, 0x3c030200, 0x431025, 0xaee20000,
-0x8f820224, 0x3c010001, 0xac227e2c, 0x8f820220,
-0x2403fffb, 0x431024, 0xaf820220, 0x8f820220,
-0x34420002, 0x8005083, 0xaf820220, 0x3c020001,
-0x8c427e00, 0x10400005, 0x0, 0x3c020001,
-0x8c427ddc, 0x1040000f, 0x24020002, 0x3c020001,
-0x8c427de0, 0x2c424e21, 0x1040000a, 0x24020002,
-0x3c020001, 0x8c427e00, 0x1040000f, 0x0,
-0x3c020001, 0x8c427ddc, 0x1440000b, 0x0,
-0x24020002, 0x3c010001, 0x8005083, 0xac227dd0,
-0x3c020001, 0x8c427e00, 0x10400003, 0x0,
-0xc00429b, 0x0, 0x8f820220, 0x3c03f700,
-0x431025, 0xaf820220, 0x8fbf0010, 0x3e00008,
-0x27bd0018, 0x3c030001, 0x24637e28, 0x8c620000,
-0x10400005, 0x34422000, 0x3c010001, 0xac227e1c,
-0x8005095, 0xac600000, 0x3c010001, 0xac247e1c,
-0x3e00008, 0x0, 0x27bdffe0, 0x30820030,
-0xafbf0018, 0x3c010001, 0xac227e24, 0x14400067,
-0x3c02ffff, 0x34421f0e, 0x821024, 0x14400061,
-0x24020030, 0x30822000, 0x1040005d, 0x30838000,
-0x31a02, 0x30820001, 0x21200, 0x3c040001,
-0x8c845d9c, 0x621825, 0x331c2, 0x3c030001,
-0x24635d78, 0x30828000, 0x21202, 0x30840001,
-0x42200, 0x441025, 0x239c2, 0x61080,
-0x431021, 0x471021, 0x90430000, 0x24020001,
-0x10620025, 0x0, 0x10600007, 0x24020002,
-0x10620013, 0x24020003, 0x1062002c, 0x3c05000f,
-0x80050f9, 0x0, 0x8f820200, 0x2403feff,
-0x431024, 0xaf820200, 0x8f820220, 0x3c03fffe,
-0x3463ffff, 0x431024, 0xaf820220, 0x3c010001,
-0xac207e44, 0x3c010001, 0x8005104, 0xac207e4c,
-0x8f820200, 0x34420100, 0xaf820200, 0x8f820220,
-0x3c03fffe, 0x3463ffff, 0x431024, 0xaf820220,
-0x24020100, 0x3c010001, 0xac227e44, 0x3c010001,
-0x8005104, 0xac207e4c, 0x8f820200, 0x2403feff,
-0x431024, 0xaf820200, 0x8f820220, 0x3c030001,
-0x431025, 0xaf820220, 0x3c010001, 0xac207e44,
-0x3c010001, 0x8005104, 0xac237e4c, 0x8f820200,
-0x34420100, 0xaf820200, 0x8f820220, 0x3c030001,
-0x431025, 0xaf820220, 0x24020100, 0x3c010001,
-0xac227e44, 0x3c010001, 0x8005104, 0xac237e4c,
-0x34a5ffff, 0x3c040001, 0x24845bb8, 0xafa30010,
-0xc002403, 0xafa00014, 0x8005104, 0x0,
-0x24020030, 0x3c010001, 0xac227e28, 0x8fbf0018,
-0x3e00008, 0x27bd0020, 0x0, 0x27bdffc8,
-0xafb20028, 0x809021, 0xafb3002c, 0xa09821,
-0xafb00020, 0xc08021, 0x3c040001, 0x24845bd0,
-0x3c050009, 0x3c020001, 0x8c425cc8, 0x34a59001,
-0x2403021, 0x2603821, 0xafbf0030, 0xafb10024,
-0xa7a0001a, 0xafb00014, 0xc002403, 0xafa20010,
-0x24020002, 0x12620083, 0x2e620003, 0x10400005,
-0x24020001, 0x1262000a, 0x0, 0x800529b,
-0x0, 0x24020004, 0x126200fa, 0x24020008,
-0x126200f9, 0x3c02ffec, 0x800529b, 0x0,
-0x3c020001, 0x8c425cc4, 0x30420002, 0x14400004,
-0x128940, 0x3c02fffb, 0x3442ffff, 0x2028024,
-0x3c010001, 0x310821, 0xac307e3c, 0x3c024000,
-0x2021024, 0x1040004e, 0x1023c2, 0x30840030,
-0x101382, 0x3042001c, 0x3c030001, 0x24635d08,
-0x431021, 0x823821, 0x3c020020, 0x2021024,
-0x10400006, 0x24020100, 0x3c010001, 0x310821,
-0xac227e40, 0x8005150, 0x3c020080, 0x3c010001,
-0x310821, 0xac207e40, 0x3c020080, 0x2021024,
-0x10400006, 0x121940, 0x3c020001, 0x3c010001,
-0x230821, 0x800515c, 0xac227e48, 0x121140,
-0x3c010001, 0x220821, 0xac207e48, 0x94e40000,
-0x3c030001, 0x8c635dbc, 0x24020005, 0x10620010,
-0xa7a40018, 0x32024000, 0x10400002, 0x34824000,
-0xa7a20018, 0x24040001, 0x94e20002, 0x24050004,
-0x24e60002, 0x34420001, 0xc00498e, 0xa4e20002,
-0x24040001, 0x2821, 0xc00498e, 0x27a60018,
-0x3c020001, 0x8c425cc8, 0x24110001, 0x3c010001,
-0xac315cd4, 0x14530004, 0x32028000, 0xc00429b,
-0x0, 0x32028000, 0x1040011f, 0x0,
-0xc00429b, 0x0, 0x3c030001, 0x8c635dbc,
-0x24020005, 0x10620118, 0x24020002, 0x3c010001,
-0xac315ccc, 0x3c010001, 0x800529b, 0xac225cc8,
-0x24040001, 0x24050004, 0x27b0001a, 0xc00498e,
-0x2003021, 0x24040001, 0x2821, 0xc00498e,
-0x2003021, 0x3c020001, 0x511021, 0x8c427e34,
-0x3c040001, 0x8c845cc8, 0x3c03bfff, 0x3463ffff,
-0x3c010001, 0xac335cd4, 0x431024, 0x3c010001,
-0x310821, 0x109300fa, 0xac227e34, 0x800529b,
-0x0, 0x3c022000, 0x2021024, 0x10400005,
-0x24020001, 0x3c010001, 0xac225d98, 0x80051ad,
-0x128940, 0x3c010001, 0xac205d98, 0x128940,
-0x3c010001, 0x310821, 0xac307e38, 0x3c024000,
-0x2021024, 0x14400016, 0x0, 0x3c020001,
-0x8c425d98, 0x10400008, 0x24040004, 0x24050001,
-0xc004d93, 0x24062000, 0x24020001, 0x3c010001,
-0x370821, 0xac2283ac, 0x3c020001, 0x511021,
-0x8c427e30, 0x3c03bfff, 0x3463ffff, 0x431024,
-0x3c010001, 0x310821, 0x8005299, 0xac227e30,
-0x3c020001, 0x8c425d98, 0x10400028, 0x3c0300a0,
-0x2031024, 0x5443000d, 0x3c020020, 0x3c020001,
-0x8c425d9c, 0x24030100, 0x3c010001, 0x310821,
-0xac237e44, 0x3c030001, 0x3c010001, 0x310821,
-0xac237e4c, 0x80051f0, 0x34420400, 0x2021024,
-0x10400008, 0x24030100, 0x3c020001, 0x8c425d9c,
-0x3c010001, 0x310821, 0xac237e44, 0x80051f0,
-0x34420800, 0x3c020080, 0x2021024, 0x1040002e,
-0x3c030001, 0x3c020001, 0x8c425d9c, 0x3c010001,
-0x310821, 0xac237e4c, 0x34420c00, 0x3c010001,
-0xac225d9c, 0x8005218, 0x24040001, 0x3c020020,
-0x2021024, 0x10400006, 0x24020100, 0x3c010001,
-0x310821, 0xac227e44, 0x8005201, 0x3c020080,
-0x3c010001, 0x310821, 0xac207e44, 0x3c020080,
-0x2021024, 0x10400007, 0x121940, 0x3c020001,
-0x3c010001, 0x230821, 0xac227e4c, 0x800520f,
-0x24040001, 0x121140, 0x3c010001, 0x220821,
-0xac207e4c, 0x24040001, 0x2821, 0x27b0001e,
-0xc00494c, 0x2003021, 0x24040001, 0x2821,
-0xc00494c, 0x2003021, 0x24040001, 0x24050001,
-0x27b0001c, 0xc00494c, 0x2003021, 0x24040001,
-0x24050001, 0xc00494c, 0x2003021, 0x8005299,
-0x0, 0x3c02ffec, 0x3442ffff, 0x2028024,
-0x3c020008, 0x2028025, 0x121140, 0x3c010001,
-0x220821, 0xac307e38, 0x3c022000, 0x2021024,
-0x10400009, 0x0, 0x3c020001, 0x8c425d74,
-0x14400005, 0x24020001, 0x3c010001, 0xac225d98,
-0x800523a, 0x3c024000, 0x3c010001, 0xac205d98,
-0x3c024000, 0x2021024, 0x1440001e, 0x0,
-0x3c020001, 0x8c425d98, 0x3c010001, 0xac205ce0,
-0x10400007, 0x24022020, 0x3c010001, 0xac225d9c,
-0x24020001, 0x3c010001, 0x370821, 0xac2283ac,
-0x3c04bfff, 0x121940, 0x3c020001, 0x431021,
-0x8c427e30, 0x3c050001, 0x8ca55cc8, 0x3484ffff,
-0x441024, 0x3c010001, 0x230821, 0xac227e30,
-0x24020001, 0x10a20044, 0x0, 0x8005299,
-0x0, 0x3c020001, 0x8c425d98, 0x1040001c,
-0x24022000, 0x3c010001, 0xac225d9c, 0x3c0300a0,
-0x2031024, 0x14430005, 0x121140, 0x3402a000,
-0x3c010001, 0x8005294, 0xac225d9c, 0x3c030001,
-0x621821, 0x8c637e38, 0x3c020020, 0x621024,
-0x10400004, 0x24022001, 0x3c010001, 0x8005294,
-0xac225d9c, 0x3c020080, 0x621024, 0x1040001f,
-0x3402a001, 0x3c010001, 0x8005294, 0xac225d9c,
-0x3c020020, 0x2021024, 0x10400007, 0x121940,
-0x24020100, 0x3c010001, 0x230821, 0xac227e44,
-0x8005288, 0x3c020080, 0x121140, 0x3c010001,
-0x220821, 0xac207e44, 0x3c020080, 0x2021024,
-0x10400006, 0x121940, 0x3c020001, 0x3c010001,
-0x230821, 0x8005294, 0xac227e4c, 0x121140,
-0x3c010001, 0x220821, 0xac207e4c, 0x3c030001,
-0x8c635cc8, 0x24020001, 0x10620003, 0x0,
-0xc00429b, 0x0, 0x8fbf0030, 0x8fb3002c,
-0x8fb20028, 0x8fb10024, 0x8fb00020, 0x3e00008,
-0x27bd0038, 0x27bdffd8, 0xafb20020, 0x809021,
-0xafb1001c, 0x8821, 0x24020002, 0xafbf0024,
-0xafb00018, 0xa7a00012, 0x10a200d3, 0xa7a00010,
-0x2ca20003, 0x10400005, 0x24020001, 0x10a2000a,
-0x128140, 0x8005380, 0x2201021, 0x24020004,
-0x10a2007d, 0x24020008, 0x10a2007c, 0x122940,
-0x8005380, 0x2201021, 0x3c030001, 0x701821,
-0x8c637e3c, 0x3c024000, 0x621024, 0x14400009,
-0x24040001, 0x3c027fff, 0x3442ffff, 0x628824,
-0x3c010001, 0x300821, 0xac317e34, 0x8005380,
-0x2201021, 0x24050001, 0xc00494c, 0x27a60010,
-0x24040001, 0x24050001, 0xc00494c, 0x27a60010,
-0x97a20010, 0x30420004, 0x10400034, 0x3c114000,
-0x3c020001, 0x8c425dbc, 0x2443ffff, 0x2c620006,
-0x10400034, 0x31080, 0x3c010001, 0x220821,
-0x8c225be0, 0x400008, 0x0, 0x24040001,
-0x24050011, 0x27b00012, 0xc00494c, 0x2003021,
-0x24040001, 0x24050011, 0xc00494c, 0x2003021,
-0x97a50012, 0x30a24000, 0x10400002, 0x3c040010,
-0x3c040008, 0x3c030001, 0x8005301, 0x30a28000,
-0x24040001, 0x24050014, 0x27b00012, 0xc00494c,
-0x2003021, 0x24040001, 0x24050014, 0xc00494c,
-0x2003021, 0x97a50012, 0x30a21000, 0x10400002,
-0x3c040010, 0x3c040008, 0x3c030001, 0x30a20800,
-0x54400001, 0x3c030002, 0x3c028000, 0x2221025,
-0x641825, 0x800530e, 0x438825, 0x3c110001,
-0x2308821, 0x8e317e3c, 0x3c027fff, 0x3442ffff,
-0x2228824, 0x3c020001, 0x8c425cd8, 0x1040001d,
-0x121140, 0x3c020001, 0x8c425d98, 0x10400002,
-0x3c022000, 0x2228825, 0x121140, 0x3c010001,
-0x220821, 0x8c227e40, 0x10400003, 0x3c020020,
-0x8005322, 0x2228825, 0x3c02ffdf, 0x3442ffff,
-0x2228824, 0x121140, 0x3c010001, 0x220821,
-0x8c227e48, 0x10400003, 0x3c020080, 0x800532d,
-0x2228825, 0x3c02ff7f, 0x3442ffff, 0x2228824,
-0x121140, 0x3c010001, 0x220821, 0xac317e34,
-0x8005380, 0x2201021, 0x122940, 0x3c030001,
-0x651821, 0x8c637e38, 0x3c024000, 0x621024,
-0x14400008, 0x3c027fff, 0x3442ffff, 0x628824,
-0x3c010001, 0x250821, 0xac317e30, 0x8005380,
-0x2201021, 0x3c020001, 0x8c425cd8, 0x10400033,
-0x3c11c00c, 0x3c020001, 0x8c425d74, 0x3c04c00c,
-0x34842000, 0x3c030001, 0x8c635d98, 0x2102b,
-0x21023, 0x441024, 0x10600003, 0x518825,
-0x3c022000, 0x2228825, 0x3c020001, 0x451021,
-0x8c427e44, 0x10400003, 0x3c020020, 0x800535d,
-0x2228825, 0x3c02ffdf, 0x3442ffff, 0x2228824,
-0x121140, 0x3c010001, 0x220821, 0x8c227e4c,
-0x10400003, 0x3c020080, 0x8005368, 0x2228825,
-0x3c02ff7f, 0x3442ffff, 0x2228824, 0x3c020001,
-0x8c425d60, 0x10400002, 0x3c020800, 0x2228825,
-0x3c020001, 0x8c425d64, 0x10400002, 0x3c020400,
-0x2228825, 0x3c020001, 0x8c425d68, 0x10400006,
-0x3c020100, 0x800537b, 0x2228825, 0x3c027fff,
-0x3442ffff, 0x628824, 0x121140, 0x3c010001,
-0x220821, 0xac317e30, 0x2201021, 0x8fbf0024,
-0x8fb20020, 0x8fb1001c, 0x8fb00018, 0x3e00008,
-0x27bd0028, 0x27bdffd8, 0xafb40020, 0x80a021,
-0xafbf0024, 0xafb3001c, 0xafb20018, 0xafb10014,
-0xafb00010, 0x8f900200, 0x3c030001, 0x8c635cc8,
-0x8f930220, 0x24020002, 0x10620063, 0x2c620003,
-0x10400005, 0x24020001, 0x1062000a, 0x141940,
-0x8005448, 0x0, 0x24020004, 0x1062005a,
-0x24020008, 0x10620059, 0x149140, 0x8005448,
-0x0, 0x3c040001, 0x832021, 0x8c847e3c,
-0x3c110001, 0x2238821, 0x8e317e34, 0x3c024000,
-0x821024, 0x1040003e, 0x3c020008, 0x2221024,
-0x10400020, 0x36100002, 0x3c020001, 0x431021,
-0x8c427e40, 0x10400005, 0x36100020, 0x36100100,
-0x3c020020, 0x80053bd, 0x2228825, 0x2402feff,
-0x2028024, 0x3c02ffdf, 0x3442ffff, 0x2228824,
-0x141140, 0x3c010001, 0x220821, 0x8c227e48,
-0x10400005, 0x3c020001, 0x2629825, 0x3c020080,
-0x80053dc, 0x2228825, 0x3c02fffe, 0x3442ffff,
-0x2629824, 0x3c02ff7f, 0x3442ffff, 0x80053dc,
-0x2228824, 0x2402fedf, 0x2028024, 0x3c02fffe,
-0x3442ffff, 0x2629824, 0x3c02ff5f, 0x3442ffff,
-0x2228824, 0x3c010001, 0x230821, 0xac207e40,
-0x3c010001, 0x230821, 0xac207e48, 0xc00486a,
-0x0, 0xaf900200, 0xaf930220, 0x8f820220,
-0x2403fffb, 0x431024, 0xaf820220, 0x8f820220,
-0x34420002, 0xaf820220, 0x80053f3, 0x141140,
-0x8f820200, 0x2403fffd, 0x431024, 0xc00486a,
-0xaf820200, 0x3c02bfff, 0x3442ffff, 0xc00429b,
-0x2228824, 0x141140, 0x3c010001, 0x220821,
-0x8005448, 0xac317e34, 0x149140, 0x3c040001,
-0x922021, 0x8c847e38, 0x3c110001, 0x2328821,
-0x8e317e30, 0x3c024000, 0x821024, 0x14400011,
-0x0, 0x3c020001, 0x8c425d98, 0x14400006,
-0x3c02bfff, 0x8f820200, 0x34420002, 0xc00486a,
-0xaf820200, 0x3c02bfff, 0x3442ffff, 0xc00429b,
-0x2228824, 0x3c010001, 0x320821, 0x8005448,
-0xac317e30, 0x3c020001, 0x8c425d98, 0x10400005,
-0x3c020020, 0x3c020001, 0x8c425d74, 0x1040002b,
-0x3c020020, 0x821024, 0x10400007, 0x36100020,
-0x24020100, 0x3c010001, 0x320821, 0xac227e44,
-0x8005428, 0x36100100, 0x3c010001, 0x320821,
-0xac207e44, 0x2402feff, 0x2028024, 0x3c020080,
-0x821024, 0x10400007, 0x141940, 0x3c020001,
-0x3c010001, 0x230821, 0xac227e4c, 0x8005439,
-0x2629825, 0x141140, 0x3c010001, 0x220821,
-0xac207e4c, 0x3c02fffe, 0x3442ffff, 0x2629824,
-0xc00486a, 0x0, 0xaf900200, 0xaf930220,
-0x8f820220, 0x2403fffb, 0x431024, 0xaf820220,
-0x8f820220, 0x34420002, 0xaf820220, 0x141140,
-0x3c010001, 0x220821, 0xac317e30, 0x8fbf0024,
-0x8fb40020, 0x8fb3001c, 0x8fb20018, 0x8fb10014,
-0x8fb00010, 0x3e00008, 0x27bd0028, 0x0 };
-static u32 tigonFwRodata[(MAX_RODATA_LEN/4) + 1] __devinitdata = {
-0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x772f636f, 0x6d6d6f6e,
-0x2f66776d, 0x61696e2e, 0x632c7620, 0x312e312e,
-0x322e3131, 0x20313939, 0x382f3034, 0x2f323720,
-0x32323a31, 0x333a3432, 0x20736875, 0x616e6720,
-0x45787020, 0x24000000, 0x7468655f, 0x4441574e,
-0x0, 0x53544143, 0x4b5f3120, 0x0,
-0x42616453, 0x6e64526e, 0x67000000, 0x3f456e71,
-0x45767400, 0x3f6e6f51, 0x64457650, 0x0,
-0x6576526e, 0x6746756c, 0x6c000000, 0x496c6c43,
-0x6f6e6652, 0x78000000, 0x53656e64, 0x436b5375,
-0x6d000000, 0x52656376, 0x566c616e, 0x0,
-0x0, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x772f636f, 0x6d6d6f6e,
-0x2f74696d, 0x65722e63, 0x2c762031, 0x2e312e32,
-0x2e382031, 0x3939382f, 0x30372f33, 0x31203137,
-0x3a35383a, 0x34352073, 0x6875616e, 0x67204578,
-0x70202400, 0x542d446d, 0x61526431, 0x0,
-0x542d446d, 0x61424200, 0x542d446d, 0x61320000,
-0x3f6e6f51, 0x64547845, 0x0, 0x3f6e6f51,
-0x64527845, 0x0, 0x656e714d, 0x45765046,
-0x61696c00, 0x656e714d, 0x45764661, 0x696c0000,
-0x6661696c, 0x456e454d, 0x0, 0x3f456e71,
-0x45767400, 0x3f6e6f51, 0x64457650, 0x0,
-0x6576526e, 0x6746756c, 0x6c000000, 0x0,
-0x0, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x772f636f, 0x6d6d6f6e,
-0x2f636f6d, 0x6d616e64, 0x2e632c76, 0x20312e31,
-0x2e322e31, 0x30203139, 0x39382f31, 0x312f3138,
-0x2031373a, 0x31313a31, 0x38207368, 0x75616e67,
-0x20457870, 0x20240000, 0x3f4d626f, 0x78457674,
-0x0, 0x4e4f636f, 0x6d616e64, 0x0,
-0x68737465, 0x5f455252, 0x0, 0x412d4572,
-0x72427563, 0x0, 0x4552524f, 0x522d4164,
-0x64000000, 0x656e714d, 0x45765046, 0x61696c00,
-0x656e714d, 0x45764661, 0x696c0000, 0x6661696c,
-0x456e454d, 0x0, 0x442d4572, 0x724c6173,
-0x74000000, 0x442d4572, 0x72320000, 0x6d437374,
-0x4d644552, 0x52000000, 0x70726f6d, 0x4d644552,
-0x52000000, 0x46696c74, 0x4d644552, 0x52000000,
-0x636d645f, 0x45525200, 0x3f456e71, 0x45767400,
-0x3f6e6f51, 0x64457650, 0x0, 0x6576526e,
-0x6746756c, 0x6c000000, 0x0, 0x6ea0,
-0x7fbc, 0x6e38, 0x8734, 0x82b0,
-0x8780, 0x8780, 0x6f54, 0x7694,
-0x7f0c, 0x80a8, 0x8074, 0x8780,
-0x7e70, 0x80cc, 0x6e64, 0x81cc,
-0x0, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x772f636f, 0x6d6d6f6e,
-0x2f646d61, 0x2e632c76, 0x20312e31, 0x2e322e33,
-0x20313939, 0x382f3034, 0x2f323720, 0x32323a31,
-0x333a3431, 0x20736875, 0x616e6720, 0x45787020,
-0x24000000, 0x646d6172, 0x6441544e, 0x0,
-0x646d6177, 0x7241544e, 0x0, 0x0,
-0x0, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x772f636f, 0x6d6d6f6e,
-0x2f747261, 0x63652e63, 0x2c762031, 0x2e312e32,
-0x2e322031, 0x3939382f, 0x30342f32, 0x37203232,
-0x3a31333a, 0x35302073, 0x6875616e, 0x67204578,
-0x70202400, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x772f636f, 0x6d6d6f6e,
-0x2f646174, 0x612e632c, 0x7620312e, 0x312e322e,
-0x32203139, 0x39382f30, 0x342f3237, 0x2032323a,
-0x31333a34, 0x30207368, 0x75616e67, 0x20457870,
-0x20240000, 0x46575f56, 0x45525349, 0x4f4e3a20,
-0x23312046, 0x72692041, 0x70722037, 0x2031373a,
-0x35353a34, 0x38205044, 0x54203230, 0x30300000,
-0x46575f43, 0x4f4d5049, 0x4c455f54, 0x494d453a,
-0x2031373a, 0x35353a34, 0x38000000, 0x46575f43,
-0x4f4d5049, 0x4c455f42, 0x593a2064, 0x65767263,
-0x73000000, 0x46575f43, 0x4f4d5049, 0x4c455f48,
-0x4f53543a, 0x20636f6d, 0x70757465, 0x0,
-0x46575f43, 0x4f4d5049, 0x4c455f44, 0x4f4d4149,
-0x4e3a2065, 0x6e672e61, 0x6374656f, 0x6e2e636f,
-0x6d000000, 0x46575f43, 0x4f4d5049, 0x4c45523a,
-0x20676363, 0x20766572, 0x73696f6e, 0x20322e37,
-0x2e320000, 0x0, 0x0, 0x0,
-0x0, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x772f636f, 0x6d6d6f6e,
-0x2f6d656d, 0x2e632c76, 0x20312e31, 0x2e322e32,
-0x20313939, 0x382f3034, 0x2f323720, 0x32323a31,
-0x333a3434, 0x20736875, 0x616e6720, 0x45787020,
-0x24000000, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x772f636f, 0x6d6d6f6e,
-0x2f73656e, 0x642e632c, 0x7620312e, 0x312e322e,
-0x31312031, 0x3939382f, 0x31322f32, 0x32203137,
-0x3a31373a, 0x35352073, 0x6875616e, 0x67204578,
-0x70202400, 0x736e6464, 0x654e6f51, 0x20000000,
-0x6e6f454e, 0x515f5458, 0x0, 0x736e6464,
-0x744e6f51, 0x20000000, 0x3f6e6f51, 0x64547845,
-0x0, 0x756e6b72, 0x64747970, 0x65000000,
-0x0, 0xaccc, 0xaccc, 0xad9c,
-0xaab0, 0xaab0, 0xad9c, 0xad9c,
-0xad9c, 0xad9c, 0xad9c, 0xad9c,
-0xad9c, 0xad9c, 0xad9c, 0xad9c,
-0xad9c, 0xad9c, 0xad9c, 0xad7c,
-0x0, 0xbca8, 0xbca8, 0xbd70,
-0xae4c, 0xb058, 0xbd70, 0xbd70,
-0xbd70, 0xbd70, 0xbd70, 0xbd70,
-0xbd70, 0xbd70, 0xbd70, 0xbd70,
-0xbd70, 0xbd70, 0xbd70, 0xbd54,
-0xb040, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x772f636f, 0x6d6d6f6e,
-0x2f726563, 0x762e632c, 0x7620312e, 0x312e322e,
-0x31392031, 0x3939382f, 0x30372f32, 0x34203231,
-0x3a33303a, 0x30352073, 0x6875616e, 0x67204578,
-0x70202400, 0x706b5278, 0x45525200, 0x66726d32,
-0x4c617267, 0x65000000, 0x72784e6f, 0x52784264,
-0x0, 0x72785144, 0x6d614446, 0x0,
-0x72785144, 0x6d614246, 0x0, 0x3f6e6f51,
-0x64527845, 0x0, 0x706b5278, 0x45525273,
-0x0, 0x66726d32, 0x4c726753, 0x0,
-0x72784e6f, 0x42645300, 0x3f724264, 0x446d6146,
-0x0, 0x3f724a42, 0x64446d46, 0x0,
-0x0, 0xf678, 0xf678, 0xf678,
-0xf678, 0xf678, 0xf678, 0xf678,
-0xf678, 0xf678, 0xf678, 0xf678,
-0xf678, 0xf678, 0xf678, 0xf678,
-0xf670, 0xf670, 0xf670, 0x572d444d,
-0x41456e46, 0x0, 0x0, 0xfdc0,
-0x1015c, 0xfddc, 0x1015c, 0x1015c,
-0x1015c, 0x1015c, 0x1015c, 0x1015c,
-0xf704, 0x1015c, 0x1015c, 0x1015c,
-0x1015c, 0x1015c, 0x10154, 0x10154,
-0x10154, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x772f636f, 0x6d6d6f6e,
-0x2f6d6163, 0x2e632c76, 0x20312e31, 0x2e322e31,
-0x32203139, 0x39382f30, 0x342f3237, 0x2032323a,
-0x31333a34, 0x32207368, 0x75616e67, 0x20457870,
-0x20240000, 0x6d616374, 0x7841544e, 0x0,
-0x4e745379, 0x6e264c6b, 0x0, 0x72656d61,
-0x73737274, 0x0, 0x6c696e6b, 0x444f574e,
-0x0, 0x656e714d, 0x45765046, 0x61696c00,
-0x656e714d, 0x45764661, 0x696c0000, 0x6661696c,
-0x456e454d, 0x0, 0x6c696e6b, 0x55500000,
-0x0, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x772f636f, 0x6d6d6f6e,
-0x2f636b73, 0x756d2e63, 0x2c762031, 0x2e312e32,
-0x2e322031, 0x3939382f, 0x30342f32, 0x37203232,
-0x3a31333a, 0x33392073, 0x6875616e, 0x67204578,
-0x70202400, 0x50726f62, 0x65506879, 0x0,
-0x6c6e6b41, 0x53535254, 0x0, 0x11b2c,
-0x11bc4, 0x11bf8, 0x11c2c, 0x11c58,
-0x11c6c, 0x11ca8, 0x1207c, 0x11de4,
-0x11e24, 0x11e50, 0x11e90, 0x11ec0,
-0x11efc, 0x11f30, 0x1207c, 0x122c0,
-0x122d8, 0x12300, 0x12320, 0x12348,
-0x12478, 0x124a0, 0x124f4, 0x1251c,
-0x0, 0x1278c, 0x1285c, 0x12934,
-0x12a04, 0x12a60, 0x12b3c, 0x12b64,
-0x12c40, 0x12c68, 0x12e10, 0x12e38,
-0x12fe0, 0x131d8, 0x1346c, 0x13380,
-0x1346c, 0x13498, 0x13008, 0x131b0,
-0x0, 0x13b84, 0x13bc8, 0x13c60,
-0x13cac, 0x13d1c, 0x13db4, 0x13de8,
-0x13e70, 0x13f08, 0x13fd8, 0x14018,
-0x1409c, 0x140c0, 0x141f4, 0x646f4261,
-0x73655067, 0x0, 0x0, 0x0,
-0x0, 0x73746d61, 0x634c4e4b, 0x0,
-0x0, 0x14c38, 0x14c38, 0x14b80,
-0x14bc4, 0x14c38, 0x14c38, 0x0,
-0x0, 0x0 };
-static u32 tigonFwData[(MAX_DATA_LEN/4) + 1] __devinitdata = {
-0x416c7465,
-0x6f6e2041, 0x63654e49, 0x43205600, 0x416c7465,
-0x6f6e2041, 0x63654e49, 0x43205600, 0x42424242,
-0x0, 0x0, 0x0, 0x135418,
-0x13e7fc, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x60cf00,
-0x60, 0xcf000000, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x3, 0x0,
-0x1, 0x0, 0x0, 0x0,
-0x1, 0x0, 0x1, 0x0,
-0x0, 0x0, 0x0, 0x1,
-0x1, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x1000000, 0x21000000,
-0x12000140, 0x0, 0x0, 0x20000000,
-0x120000a0, 0x0, 0x12000060, 0x12000180,
-0x120001e0, 0x0, 0x0, 0x0,
-0x1, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x2,
-0x0, 0x0, 0x30001, 0x1,
-0x30201, 0x0, 0x0, 0x0 };
-#endif
-/* Generated by genfw.c */
-#define tigon2FwReleaseMajor 0xc
-#define tigon2FwReleaseMinor 0x4
-#define tigon2FwReleaseFix 0xb
-#define tigon2FwStartAddr 0x00004000
-#define tigon2FwTextAddr 0x00004000
-#define tigon2FwTextLen 0x11bc0
-#define tigon2FwRodataAddr 0x00015bc0
-#define tigon2FwRodataLen 0x10d0
-#define tigon2FwDataAddr 0x00016cc0
-#define tigon2FwDataLen 0x1c0
-#define tigon2FwSbssAddr 0x00016e80
-#define tigon2FwSbssLen 0xcc
-#define tigon2FwBssAddr 0x00016f50
-#define tigon2FwBssLen 0x20c0
-static u32 tigon2FwText[(MAX_TEXT_LEN/4) + 1] __devinitdata = {
-0x0,
-0x10000003, 0x0, 0xd, 0xd,
-0x3c1d0001, 0x8fbd6d20, 0x3a0f021, 0x3c100000,
-0x26104000, 0xc0010c0, 0x0, 0xd,
-0x3c1d0001, 0x8fbd6d24, 0x3a0f021, 0x3c100000,
-0x26104000, 0xc0017e0, 0x0, 0xd,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x2000008,
-0x0, 0x800172f, 0x3c0a0001, 0x800172f,
-0x3c0a0002, 0x800172f, 0x0, 0x8002cac,
-0x0, 0x8002c4f, 0x0, 0x800172f,
-0x3c0a0004, 0x800328a, 0x0, 0x8001a52,
-0x0, 0x800394d, 0x0, 0x80038f4,
-0x0, 0x800172f, 0x3c0a0006, 0x80039bb,
-0x3c0a0007, 0x800172f, 0x3c0a0008, 0x800172f,
-0x3c0a0009, 0x8003a13, 0x0, 0x8002ea6,
-0x0, 0x800172f, 0x3c0a000b, 0x800172f,
-0x3c0a000c, 0x800172f, 0x3c0a000d, 0x80028fb,
-0x0, 0x8002890, 0x0, 0x800172f,
-0x3c0a000e, 0x800208c, 0x0, 0x8001964,
-0x0, 0x8001a04, 0x0, 0x8003ca6,
-0x0, 0x8003c94, 0x0, 0x800172f,
-0x0, 0x800191a, 0x0, 0x800172f,
-0x0, 0x800172f, 0x3c0a0013, 0x800172f,
-0x3c0a0014, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x27bdffe0,
-0x3c1cc000, 0xafbf001c, 0xafb00018, 0x8f820140,
-0x24030003, 0xaf8300ec, 0x34420004, 0xc002b20,
-0xaf820140, 0x3c0100c0, 0xc001763, 0xac203ffc,
-0x401821, 0x3c020010, 0x3c010001, 0xac236e9c,
-0x10620011, 0x43102b, 0x14400002, 0x3c020020,
-0x3c020008, 0x1062000c, 0x24050100, 0x3c060001,
-0x8cc66e9c, 0x3c040001, 0x24845c74, 0x3821,
-0xafa00010, 0xc002b3b, 0xafa00014, 0x3c020020,
-0x3c010001, 0xac226e9c, 0x24020008, 0x3c010001,
-0xac226eb4, 0x2402001f, 0x3c010001, 0xac226ec4,
-0x24020016, 0x3c010001, 0xac226e98, 0x3c05fffe,
-0x34a56f08, 0x3c020001, 0x8c426e9c, 0x3c030002,
-0x24639010, 0x3c040001, 0x8c846cc4, 0x431023,
-0x14800002, 0x458021, 0x2610fa38, 0x2402f000,
-0x2028024, 0xc001785, 0x2002021, 0x2022823,
-0x3c040020, 0x821823, 0x651823, 0x247bb000,
-0x3c03fffe, 0x3463bf08, 0x363b821, 0x3c0600bf,
-0x34c6f000, 0x3c070001, 0x8ce76cc0, 0x3c0300bf,
-0x3463e000, 0x852023, 0x3c010001, 0xac246ea8,
-0x822023, 0x3c010001, 0xac256e90, 0x52842,
-0x3c010001, 0xac226e84, 0x27620ffc, 0x3c010001,
-0xac226d20, 0x27621ffc, 0xdb3023, 0x7b1823,
-0x3c010001, 0xac246e88, 0x3c010001, 0xac256eac,
-0x3c010001, 0xac226d24, 0xaf860150, 0x10e00011,
-0xaf830250, 0x3c1d0001, 0x8fbd6ccc, 0x3a0f021,
-0xc001749, 0x0, 0x3c020001, 0x8c426cd0,
-0x3c030001, 0x8c636cd4, 0x2442fe00, 0x24630200,
-0x3c010001, 0xac226cd0, 0x3c010001, 0x10000004,
-0xac236cd4, 0x3c1d0001, 0x8fbd6d20, 0x3a0f021,
-0x3c020001, 0x8c426cc4, 0x1040000d, 0x26fafa38,
-0x3c020001, 0x8c426cd0, 0x3c030001, 0x8c636cd4,
-0x3c1a0001, 0x8f5a6cd4, 0x2442fa38, 0x246305c8,
-0x3c010001, 0xac226cd0, 0x3c010001, 0xac236cd4,
-0x3c020001, 0x8c426cc8, 0x14400003, 0x0,
-0x3c010001, 0xac206cd0, 0xc001151, 0x0,
-0x8fbf001c, 0x8fb00018, 0x3e00008, 0x27bd0020,
-0x3c020001, 0x8c426cd0, 0x3c030001, 0x8c636cd4,
-0x27bdff98, 0xafb00048, 0x3c100001, 0x8e1066b8,
-0xafb20050, 0x3c120000, 0x26524100, 0xafbf0060,
-0xafbe005c, 0xafb50058, 0xafb30054, 0xafb1004c,
-0xafa20034, 0xafa30030, 0xafa00010, 0xafa00014,
-0x8f860040, 0x3c040001, 0x24845c80, 0x24050200,
-0x3c010001, 0xac326e80, 0xc002b3b, 0x2003821,
-0x8f830040, 0x3c02f000, 0x621824, 0x3c026000,
-0x1062000b, 0xa3a0003f, 0x240e0001, 0x3c040001,
-0x24845c88, 0xa3ae003f, 0xafa00010, 0xafa00014,
-0x8f860040, 0x24050300, 0xc002b3b, 0x2003821,
-0x8f820240, 0x3c030001, 0x431025, 0xaf820240,
-0xaf800048, 0x8f820048, 0x14400005, 0x0,
-0xaf800048, 0x8f820048, 0x10400004, 0x0,
-0xaf800048, 0x10000003, 0x2e02021, 0xaf80004c,
-0x2e02021, 0x3c050001, 0xc002ba8, 0x34a540f8,
-0x3402021, 0xc002ba8, 0x240505c8, 0x3c020001,
-0x8c426ea8, 0x3c0d0001, 0x8dad6e88, 0x3c030001,
-0x8c636e84, 0x3c080001, 0x8d086e90, 0x3c090001,
-0x8d296eac, 0x3c0a0001, 0x8d4a6eb4, 0x3c0b0001,
-0x8d6b6ec4, 0x3c0c0001, 0x8d8c6e98, 0x3c040001,
-0x24845c94, 0x24050400, 0xaf42013c, 0x8f42013c,
-0x24060001, 0x24070001, 0xaf400000, 0xaf4d0138,
-0xaf430144, 0xaf480148, 0xaf49014c, 0xaf4a0150,
-0xaf4b0154, 0xaf4c0158, 0x2442ff80, 0xaf420140,
-0x24020001, 0xafa20010, 0xc002b3b, 0xafa00014,
-0x8f420138, 0xafa20010, 0x8f42013c, 0xafa20014,
-0x8f460144, 0x8f470148, 0x3c040001, 0x24845ca0,
-0xc002b3b, 0x24050500, 0xafb70010, 0xafba0014,
-0x8f46014c, 0x8f470150, 0x3c040001, 0x24845cac,
-0xc002b3b, 0x24050600, 0x3c020001, 0x8c426e9c,
-0x3603821, 0x3c060002, 0x24c69010, 0x2448ffff,
-0x1061824, 0xe81024, 0x43102b, 0x10400006,
-0x24050900, 0x3c040001, 0x24845cb8, 0xafa80010,
-0xc002b3b, 0xafa00014, 0x8f82000c, 0xafa20010,
-0x8f82003c, 0xafa20014, 0x8f860000, 0x8f870004,
-0x3c040001, 0x24845cc4, 0xc002b3b, 0x24051000,
-0x8c020220, 0x8c030224, 0x8c060218, 0x8c07021c,
-0x3c040001, 0x24845ccc, 0x24051100, 0xafa20010,
-0xc002b3b, 0xafa30014, 0xaf800054, 0xaf80011c,
-0x8c020218, 0x30420002, 0x10400009, 0x0,
-0x8c020220, 0x3c030002, 0x34630004, 0x431025,
-0xaf42000c, 0x8c02021c, 0x10000008, 0x34420004,
-0x8c020220, 0x3c030002, 0x34630006, 0x431025,
-0xaf42000c, 0x8c02021c, 0x34420006, 0xaf420014,
-0x8c020218, 0x30420010, 0x1040000a, 0x0,
-0x8c02021c, 0x34420004, 0xaf420010, 0x8c020220,
-0x3c03000a, 0x34630004, 0x431025, 0x10000009,
-0xaf420008, 0x8c020220, 0x3c03000a, 0x34630006,
-0x431025, 0xaf420008, 0x8c02021c, 0x34420006,
-0xaf420010, 0x24020001, 0xaf8200a0, 0xaf8200b0,
-0x8f830054, 0x8f820054, 0xaf8000d0, 0xaf8000c0,
-0x10000002, 0x24630064, 0x8f820054, 0x621023,
-0x2c420065, 0x1440fffc, 0x0, 0x8c040208,
-0x8c05020c, 0x26e20028, 0xaee20020, 0x24020490,
-0xaee20010, 0xaee40008, 0xaee5000c, 0x26e40008,
-0x8c820000, 0x8c830004, 0xaf820090, 0xaf830094,
-0x8c820018, 0xaf8200b4, 0x9482000a, 0xaf82009c,
-0x8f420014, 0xaf8200b0, 0x8f8200b0, 0x30420004,
-0x1440fffd, 0x0, 0x8f8200b0, 0x3c03ef00,
-0x431024, 0x10400021, 0x0, 0x8f8200b4,
-0xafa20010, 0x8f820090, 0x8f830094, 0x3c040001,
-0x24845cd4, 0xafa30014, 0x8f8600b0, 0x8f87009c,
-0x3c050001, 0xc002b3b, 0x34a5200d, 0x3c040001,
-0x24845ce0, 0x240203c0, 0xafa20010, 0xafa00014,
-0x8f860144, 0x3c070001, 0x24e75ce8, 0xc002b3b,
-0x3405dead, 0x8f82011c, 0x34420002, 0xaf82011c,
-0x8f820220, 0x34420004, 0xaf820220, 0x8f820140,
-0x3c030001, 0x431025, 0xaf820140, 0x96e20472,
-0x96e60452, 0x96e70462, 0xafa20010, 0x96e20482,
-0x3c040001, 0x24845d14, 0x24051200, 0xc002b3b,
-0xafa20014, 0x96f00452, 0x32020001, 0x10400002,
-0xb021, 0x24160001, 0x32020002, 0x54400001,
-0x36d60002, 0x32020008, 0x54400001, 0x36d60004,
-0x32020010, 0x54400001, 0x36d60008, 0x32020020,
-0x54400001, 0x36d60010, 0x32020040, 0x54400001,
-0x36d60020, 0x32020080, 0x54400001, 0x36d60040,
-0x96e60482, 0x30c20200, 0x54400001, 0x36d64000,
-0x96e30472, 0x30620200, 0x10400003, 0x30620100,
-0x10000003, 0x36d62000, 0x54400001, 0x36d61000,
-0x96f00462, 0x32c24000, 0x14400004, 0x3207009b,
-0x30c2009b, 0x14e20007, 0x240e0001, 0x32c22000,
-0x1440000d, 0x32020001, 0x3062009b, 0x10e20009,
-0x240e0001, 0x3c040001, 0x24845d20, 0x24051300,
-0x2003821, 0xa3ae003f, 0xafa30010, 0xc002b3b,
-0xafa00014, 0x32020001, 0x54400001, 0x36d60080,
-0x32020002, 0x54400001, 0x36d60100, 0x32020008,
-0x54400001, 0x36d60200, 0x32020010, 0x54400001,
-0x36d60400, 0x32020080, 0x54400001, 0x36d60800,
-0x8c020218, 0x30420200, 0x10400002, 0x3c020008,
-0x2c2b025, 0x8c020218, 0x30420800, 0x10400002,
-0x3c020080, 0x2c2b025, 0x8c020218, 0x30420400,
-0x10400002, 0x3c020100, 0x2c2b025, 0x8c020218,
-0x30420100, 0x10400002, 0x3c020200, 0x2c2b025,
-0x8c020218, 0x30420080, 0x10400002, 0x3c020400,
-0x2c2b025, 0x8c020218, 0x30422000, 0x10400002,
-0x3c020010, 0x2c2b025, 0x8c020218, 0x30424000,
-0x10400002, 0x3c020020, 0x2c2b025, 0x8c020218,
-0x30421000, 0x10400002, 0x3c020040, 0x2c2b025,
-0x8ee20498, 0x8ee3049c, 0xaf420160, 0xaf430164,
-0x8ee204a0, 0x8ee304a4, 0xaf420168, 0xaf43016c,
-0x8ee204a8, 0x8ee304ac, 0xaf420170, 0xaf430174,
-0x8ee20428, 0x8ee3042c, 0xaf420178, 0xaf43017c,
-0x8ee20448, 0x8ee3044c, 0xaf420180, 0xaf430184,
-0x8ee20458, 0x8ee3045c, 0xaf420188, 0xaf43018c,
-0x8ee20468, 0x8ee3046c, 0xaf420190, 0xaf430194,
-0x8ee20478, 0x8ee3047c, 0xaf420198, 0xaf43019c,
-0x8ee20488, 0x8ee3048c, 0xaf4201a0, 0xaf4301a4,
-0x8ee204b0, 0x8ee304b4, 0x24040080, 0xaf4201a8,
-0xaf4301ac, 0xc002ba8, 0x24050080, 0x8c02025c,
-0x27440224, 0xaf4201f0, 0x8c020260, 0x24050200,
-0x24060008, 0xc002bbf, 0xaf4201f8, 0x3c043b9a,
-0x3484ca00, 0x3821, 0x24020006, 0x24030002,
-0xaf4201f4, 0x240203e8, 0xaf430204, 0xaf430200,
-0xaf4401fc, 0xaf420294, 0x24020001, 0xaf430290,
-0xaf42029c, 0x3c030001, 0x671821, 0x90636cd8,
-0x3471021, 0x24e70001, 0xa043022c, 0x2ce2000f,
-0x1440fff8, 0x3471821, 0x24e70001, 0x3c080001,
-0x350840f8, 0x8f820040, 0x3c040001, 0x24845d2c,
-0x24051400, 0x21702, 0x24420030, 0xa062022c,
-0x3471021, 0xa040022c, 0x8c070218, 0x2c03021,
-0x240205c8, 0xafa20010, 0xc002b3b, 0xafa80014,
-0x3c040001, 0x24845d38, 0x3c050000, 0x24a55c80,
-0x24060010, 0x27b10030, 0x2203821, 0x27b30034,
-0xc0017a3, 0xafb30010, 0x3c030001, 0x8c636cc8,
-0x1060000a, 0x408021, 0x8fa30030, 0x2405ff00,
-0x8fa20034, 0x246400ff, 0x852024, 0x831823,
-0x431023, 0xafa20034, 0xafa40030, 0x3c040001,
-0x24845d44, 0x3c050000, 0x24a54100, 0x24060108,
-0x2203821, 0xc0017a3, 0xafb30010, 0x409021,
-0x32c20003, 0x3c010001, 0xac326e80, 0x10400045,
-0x2203821, 0x8f820050, 0x3c030010, 0x431024,
-0x10400016, 0x0, 0x8c020218, 0x30420040,
-0x1040000f, 0x24020001, 0x8f820050, 0x8c030218,
-0x240e0001, 0x3c040001, 0x24845d50, 0xa3ae003f,
-0xafa20010, 0xafa30014, 0x8f870040, 0x24051500,
-0xc002b3b, 0x2c03021, 0x10000004, 0x0,
-0x3c010001, 0x370821, 0xa02240f4, 0x3c040001,
-0x24845d5c, 0x3c050001, 0x24a55b40, 0x3c060001,
-0x24c65bac, 0xc53023, 0x8f420010, 0x27b30030,
-0x2603821, 0x27b10034, 0x34420a00, 0xaf420010,
-0xc0017a3, 0xafb10010, 0x3c040001, 0x24845d70,
-0x3c050001, 0x24a5b714, 0x3c060001, 0x24c6ba90,
-0xc53023, 0x2603821, 0xaf420108, 0xc0017a3,
-0xafb10010, 0x3c040001, 0x24845d8c, 0x3c050001,
-0x24a5be58, 0x3c060001, 0x24c6c900, 0xc53023,
-0x2603821, 0x3c010001, 0xac226ef4, 0xc0017a3,
-0xafb10010, 0x3c040001, 0x24845da4, 0x10000024,
-0x24051600, 0x3c040001, 0x24845dac, 0x3c050001,
-0x24a5a10c, 0x3c060001, 0x24c6a238, 0xc53023,
-0xc0017a3, 0xafb30010, 0x3c040001, 0x24845dbc,
-0x3c050001, 0x24a5b2b0, 0x3c060001, 0x24c6b70c,
-0xc53023, 0x2203821, 0xaf420108, 0xc0017a3,
-0xafb30010, 0x3c040001, 0x24845dd0, 0x3c050001,
-0x24a5ba98, 0x3c060001, 0x24c6be50, 0xc53023,
-0x2203821, 0x3c010001, 0xac226ef4, 0xc0017a3,
-0xafb30010, 0x3c040001, 0x24845de4, 0x24051650,
-0x2c03021, 0x3821, 0x3c010001, 0xac226ef8,
-0xafa00010, 0xc002b3b, 0xafa00014, 0x32c20020,
-0x10400021, 0x27a70030, 0x3c040001, 0x24845df0,
-0x3c050001, 0x24a5b13c, 0x3c060001, 0x24c6b2a8,
-0xc53023, 0x24022000, 0xaf42001c, 0x27a20034,
-0xc0017a3, 0xafa20010, 0x21900, 0x31982,
-0x3c040800, 0x641825, 0xae430028, 0x24030010,
-0xaf43003c, 0x96e30450, 0xaf430040, 0x8f430040,
-0x3c040001, 0x24845e04, 0xafa00014, 0xafa30010,
-0x8f47001c, 0x24051660, 0x3c010001, 0xac226ef0,
-0x10000025, 0x32c60020, 0x8ee20448, 0x8ee3044c,
-0xaf43001c, 0x8f42001c, 0x2442e000, 0x2c422001,
-0x1440000a, 0x240e0001, 0x3c040001, 0x24845e10,
-0xa3ae003f, 0xafa00010, 0xafa00014, 0x8f46001c,
-0x24051700, 0xc002b3b, 0x3821, 0x3c020000,
-0x24425cbc, 0x21100, 0x21182, 0x3c030800,
-0x431025, 0xae420028, 0x24020008, 0xaf42003c,
-0x96e20450, 0xaf420040, 0x8f420040, 0x3c040001,
-0x24845e1c, 0xafa00014, 0xafa20010, 0x8f47001c,
-0x24051800, 0x32c60020, 0xc002b3b, 0x0,
-0x3c050fff, 0x3c030001, 0x8c636ef4, 0x34a5ffff,
-0x2403021, 0x3c020001, 0x8c426ef8, 0x3c040800,
-0x651824, 0x31882, 0x641825, 0x451024,
-0x21082, 0x441025, 0xacc20080, 0x32c20180,
-0x10400056, 0xacc30020, 0x8f82005c, 0x3c030080,
-0x431024, 0x1040000d, 0x0, 0x8f820050,
-0xafa20010, 0x8f82005c, 0x240e0001, 0x3c040001,
-0x24845e28, 0xa3ae003f, 0xafa20014, 0x8f870040,
-0x24051900, 0xc002b3b, 0x2c03021, 0x8f820050,
-0x3c030010, 0x431024, 0x10400016, 0x0,
-0x8c020218, 0x30420040, 0x1040000f, 0x24020001,
-0x8f820050, 0x8c030218, 0x240e0001, 0x3c040001,
-0x24845d50, 0xa3ae003f, 0xafa20010, 0xafa30014,
-0x8f870040, 0x24052000, 0xc002b3b, 0x2c03021,
-0x10000004, 0x0, 0x3c010001, 0x370821,
-0xa02240f4, 0x3c040001, 0x24845e34, 0x3c050001,
-0x24a55ac0, 0x3c060001, 0x24c65b38, 0xc53023,
-0x8f420008, 0x27b30030, 0x2603821, 0x27b10034,
-0x34420e00, 0xaf420008, 0xc0017a3, 0xafb10010,
-0x3c040001, 0x24845e4c, 0x3c050001, 0x24a5d8b4,
-0x3c060001, 0x24c6e3c8, 0xc53023, 0x2603821,
-0xaf42010c, 0xc0017a3, 0xafb10010, 0x3c040001,
-0x24845e64, 0x3c050001, 0x24a5e9ac, 0x3c060001,
-0x24c6f0f0, 0xc53023, 0x2603821, 0x3c010001,
-0xac226f04, 0xc0017a3, 0xafb10010, 0x3c040001,
-0x24845e7c, 0x10000027, 0x24052100, 0x3c040001,
-0x24845e84, 0x3c050001, 0x24a59fc8, 0x3c060001,
-0x24c6a104, 0xc53023, 0x27b10030, 0x2203821,
-0x27b30034, 0xc0017a3, 0xafb30010, 0x3c040001,
-0x24845e94, 0x3c050001, 0x24a5cad4, 0x3c060001,
-0x24c6d8ac, 0xc53023, 0x2203821, 0xaf42010c,
-0xc0017a3, 0xafb30010, 0x3c040001, 0x24845ea4,
-0x3c050001, 0x24a5e84c, 0x3c060001, 0x24c6e9a4,
-0xc53023, 0x2203821, 0x3c010001, 0xac226f04,
-0xc0017a3, 0xafb30010, 0x3c040001, 0x24845eb8,
-0x24052150, 0x2c03021, 0x3821, 0x3c010001,
-0xac226f10, 0xafa00010, 0xc002b3b, 0xafa00014,
-0x3c110fff, 0x3c030001, 0x8c636f04, 0x3631ffff,
-0x2409821, 0x3c020001, 0x8c426f10, 0x3c0e0800,
-0x711824, 0x31882, 0x6e1825, 0x511024,
-0x21082, 0x4e1025, 0xae630038, 0xae620078,
-0x8c020218, 0x30420040, 0x14400004, 0x24020001,
-0x3c010001, 0x370821, 0xa02240f4, 0x3c040001,
-0x24845ec4, 0x3c050001, 0x24a5e3d0, 0x3c060001,
-0x24c6e52c, 0xc53023, 0x27be0030, 0x3c03821,
-0x27b50034, 0xc0017a3, 0xafb50010, 0x3c010001,
-0xac226efc, 0x511024, 0x21082, 0x3c0e0800,
-0x4e1025, 0xae620050, 0x32c22000, 0x10400006,
-0x3c03821, 0x3c020000, 0x24425cbc, 0x2221024,
-0x1000000f, 0x21082, 0x3c040001, 0x24845ed8,
-0x3c050001, 0x24a5e534, 0x3c060001, 0x24c6e6e4,
-0xc53023, 0xc0017a3, 0xafb50010, 0x3c010001,
-0xac226f14, 0x511024, 0x21082, 0x3c0e0800,
-0x4e1025, 0xae620048, 0x32c24000, 0x10400005,
-0x27a70030, 0x3c020000, 0x24425cbc, 0x1000000e,
-0x21100, 0x3c040001, 0x24845ef0, 0x3c050001,
-0x24a5e6ec, 0x3c060001, 0x24c6e844, 0xc53023,
-0x27a20034, 0xc0017a3, 0xafa20010, 0x3c010001,
-0xac226f08, 0x21100, 0x21182, 0x3c030800,
-0x431025, 0xae420060, 0x3c040001, 0x24845f08,
-0x3c050001, 0x24a58230, 0x3c060001, 0x24c68650,
-0xc53023, 0x27b10030, 0x2203821, 0x27b30034,
-0xc0017a3, 0xafb30010, 0x3c0e0fff, 0x35ceffff,
-0x3c040001, 0x24845f14, 0x3c050000, 0x24a56468,
-0x3c060000, 0x24c66588, 0xc53023, 0x2203821,
-0x240f021, 0x3c010001, 0xac226edc, 0x4e1024,
-0x21082, 0x3c150800, 0x551025, 0xafae0044,
-0xafc200b8, 0xc0017a3, 0xafb30010, 0x3c040001,
-0x24845f20, 0x3c050000, 0x24a56590, 0x3c060000,
-0x24c66808, 0x8fae0044, 0xc53023, 0x2203821,
-0x3c010001, 0xac226ed0, 0x4e1024, 0x21082,
-0x551025, 0xafc200e8, 0xc0017a3, 0xafb30010,
-0x3c040001, 0x24845f38, 0x3c050000, 0x24a56810,
-0x3c060000, 0x24c66940, 0x8fae0044, 0xc53023,
-0x2203821, 0x3c010001, 0xac226ec8, 0x4e1024,
-0x21082, 0x551025, 0xafc200c0, 0xc0017a3,
-0xafb30010, 0x3c040001, 0x24845f50, 0x3c050001,
-0x24a5fad0, 0x3c060001, 0x24c6fba8, 0x8fae0044,
-0xc53023, 0x2203821, 0x3c010001, 0xac226ed4,
-0x4e1024, 0x21082, 0x551025, 0xafc200c8,
-0xc0017a3, 0xafb30010, 0x3c040001, 0x24845f5c,
-0x3c050001, 0x24a5c93c, 0x3c060001, 0x24c6ca20,
-0xc53023, 0x2203821, 0xaf420110, 0xc0017a3,
-0xafb30010, 0x3c040001, 0x24845f6c, 0x3c050001,
-0x24a5c910, 0x3c060001, 0x24c6c934, 0xc53023,
-0x2203821, 0xaf420124, 0xc0017a3, 0xafb30010,
-0x3c040001, 0x24845f7c, 0x3c050001, 0x24a55a80,
-0x3c060001, 0x24c65aac, 0xc53023, 0x2203821,
-0xaf420120, 0xaf420114, 0xc0017a3, 0xafb30010,
-0x3c040001, 0x24845f88, 0x3c050001, 0x24a5f298,
-0x3c060001, 0x24c6f6b4, 0xc53023, 0x2203821,
-0xaf420118, 0xc0017a3, 0xafb30010, 0x8fae0044,
-0x3c010001, 0xac226f18, 0x4e1024, 0x21082,
-0x551025, 0xc003fc3, 0xafc200d0, 0xc003c40,
-0x0, 0xc0027a8, 0x0, 0xac000228,
-0xac00022c, 0x96e20450, 0x2442ffff, 0xaf420038,
-0x96e20460, 0xaf420080, 0x32c24000, 0x14400003,
-0x0, 0x96e20480, 0xaf420084, 0x96e70490,
-0x50e00001, 0x24070800, 0x24e2ffff, 0xaf420088,
-0xaf42007c, 0x24020800, 0x10e2000f, 0x32c24000,
-0x10400003, 0x24020400, 0x10e2000b, 0x0,
-0x240e0001, 0x3c040001, 0x24845f98, 0xa3ae003f,
-0x96e60490, 0x24052170, 0x2c03821, 0xafa00010,
-0xc002b3b, 0xafa00014, 0x8f430138, 0x8f440138,
-0x24020001, 0xa34205c2, 0xaf430094, 0xaf440098,
-0xafa00010, 0xafa00014, 0x8f460080, 0x8f470084,
-0x3c040001, 0x24845fa4, 0xc002b3b, 0x24052200,
-0xc0024a4, 0x3c110800, 0x3c1433d8, 0x3694cb58,
-0x3c020800, 0x34420080, 0x3c040001, 0x24845fb0,
-0x3c050000, 0x24a55d00, 0x3c060000, 0x24c65d1c,
-0xc53023, 0x27a70030, 0xaf820060, 0x2402ffff,
-0xaf820064, 0x27a20034, 0xc0017a3, 0xafa20010,
-0x3c010001, 0xac226eb8, 0x21100, 0x21182,
-0x511025, 0xc0018fc, 0xae420000, 0x8f820240,
-0x3c030001, 0x431025, 0xaf820240, 0x3c020000,
-0x24424034, 0xaf820244, 0xaf800240, 0x8f820060,
-0x511024, 0x14400005, 0x3c030800, 0x8f820060,
-0x431024, 0x1040fffd, 0x0, 0xc003c4d,
-0x8821, 0x3c020100, 0xafa20020, 0x8f530018,
-0x240200ff, 0x56620001, 0x26710001, 0x8c020228,
-0x1622000e, 0x1330c0, 0x8f42033c, 0x24420001,
-0xaf42033c, 0x8f42033c, 0x8c020228, 0x3c040001,
-0x24845c24, 0x3c050009, 0xafa00014, 0xafa20010,
-0x8fa60020, 0x1000003f, 0x34a50100, 0xd71021,
-0x8fa30020, 0x8fa40024, 0xac4304c0, 0xac4404c4,
-0xc01821, 0x8f440178, 0x8f45017c, 0x1021,
-0x24070004, 0xafa70010, 0xafb10014, 0x8f48000c,
-0x24c604c0, 0x2e63021, 0xafa80018, 0x8f48010c,
-0x24070008, 0xa32821, 0xa3482b, 0x822021,
-0x100f809, 0x892021, 0x1440000b, 0x24070008,
-0x8f820120, 0xafa20010, 0x8f820124, 0x3c040001,
-0x24845c2c, 0x3c050009, 0xafa20014, 0x8fa60020,
-0x1000001c, 0x34a50200, 0x8f440160, 0x8f450164,
-0x8f43000c, 0xaf510018, 0x8f860120, 0x24020010,
-0xafa20010, 0xafb10014, 0xafa30018, 0x8f42010c,
-0x40f809, 0x24c6001c, 0x14400010, 0x0,
-0x8f420340, 0x24420001, 0xaf420340, 0x8f420340,
-0x8f820120, 0xafa20010, 0x8f820124, 0x3c040001,
-0x24845c34, 0x3c050009, 0xafa20014, 0x8fa60020,
-0x34a50300, 0xc002b3b, 0x2603821, 0x8f4202e4,
-0x24420001, 0xaf4202e4, 0x8f4202e4, 0x93a2003f,
-0x10400069, 0x3c020700, 0x34423000, 0xafa20028,
-0x8f530018, 0x240200ff, 0x12620002, 0x8821,
-0x26710001, 0x8c020228, 0x1622000e, 0x1330c0,
-0x8f42033c, 0x24420001, 0xaf42033c, 0x8f42033c,
-0x8c020228, 0x3c040001, 0x24845c24, 0x3c050009,
-0xafa00014, 0xafa20010, 0x8fa60028, 0x1000003f,
-0x34a50100, 0xd71021, 0x8fa30028, 0x8fa4002c,
-0xac4304c0, 0xac4404c4, 0xc01821, 0x8f440178,
-0x8f45017c, 0x1021, 0x24070004, 0xafa70010,
-0xafb10014, 0x8f48000c, 0x24c604c0, 0x2e63021,
-0xafa80018, 0x8f48010c, 0x24070008, 0xa32821,
-0xa3482b, 0x822021, 0x100f809, 0x892021,
-0x1440000b, 0x24070008, 0x8f820120, 0xafa20010,
-0x8f820124, 0x3c040001, 0x24845c2c, 0x3c050009,
-0xafa20014, 0x8fa60028, 0x1000001c, 0x34a50200,
-0x8f440160, 0x8f450164, 0x8f43000c, 0xaf510018,
-0x8f860120, 0x24020010, 0xafa20010, 0xafb10014,
-0xafa30018, 0x8f42010c, 0x40f809, 0x24c6001c,
-0x14400010, 0x0, 0x8f420340, 0x24420001,
-0xaf420340, 0x8f420340, 0x8f820120, 0xafa20010,
-0x8f820124, 0x3c040001, 0x24845c34, 0x3c050009,
-0xafa20014, 0x8fa60028, 0x34a50300, 0xc002b3b,
-0x2603821, 0x8f4202f0, 0x24420001, 0xaf4202f0,
-0x8f4202f0, 0x3c040001, 0x24845fc0, 0xafa00010,
-0xafa00014, 0x8fa60028, 0x24052300, 0xc002b3b,
-0x3821, 0x10000004, 0x0, 0x8c020264,
-0x10400005, 0x0, 0x8f8200a0, 0x30420004,
-0x1440fffa, 0x0, 0x8f820044, 0x34420004,
-0xaf820044, 0x8f420308, 0x24420001, 0xaf420308,
-0x8f420308, 0x8f8200d8, 0x8f8300d4, 0x431023,
-0x2442ff80, 0xaf420090, 0x8f420090, 0x2842ff81,
-0x10400006, 0x24020001, 0x8f420090, 0x8f430144,
-0x431021, 0xaf420090, 0x24020001, 0xaf42008c,
-0x32c20008, 0x10400006, 0x0, 0x8f820214,
-0x3c038100, 0x3042ffff, 0x431025, 0xaf820214,
-0x3c030001, 0x8c636d94, 0x30620002, 0x10400009,
-0x30620001, 0x3c040001, 0x24845fcc, 0x3c050000,
-0x24a56d50, 0x3c060000, 0x24c671c8, 0x10000012,
-0xc53023, 0x10400009, 0x0, 0x3c040001,
-0x24845fdc, 0x3c050000, 0x24a571d0, 0x3c060000,
-0x24c67678, 0x10000008, 0xc53023, 0x3c040001,
-0x24845fec, 0x3c050000, 0x24a56948, 0x3c060000,
-0x24c66d48, 0xc53023, 0x27a70030, 0x27a20034,
-0xc0017a3, 0xafa20010, 0x3c010001, 0xac226ecc,
-0x3c020001, 0x8c426ecc, 0x3c030800, 0x21100,
-0x21182, 0x431025, 0xae420040, 0x8f8200a0,
-0xafa20010, 0x8f8200b0, 0xafa20014, 0x8f86005c,
-0x8f87011c, 0x3c040001, 0x24845ffc, 0x3c010001,
-0xac366ea4, 0x3c010001, 0xac206e94, 0x3c010001,
-0xac3c6e8c, 0x3c010001, 0xac3b6ebc, 0x3c010001,
-0xac376ec0, 0x3c010001, 0xac3a6ea0, 0xc002b3b,
-0x24052400, 0x8f820200, 0xafa20010, 0x8f820220,
-0xafa20014, 0x8f860044, 0x8f870050, 0x3c040001,
-0x24846008, 0xc002b3b, 0x24052500, 0x8f830060,
-0x74100b, 0x242000a, 0x200f821, 0x0,
-0xd, 0x8fbf0060, 0x8fbe005c, 0x8fb50058,
-0x8fb30054, 0x8fb20050, 0x8fb1004c, 0x8fb00048,
-0x3e00008, 0x27bd0068, 0x27bdffe0, 0x3c040001,
-0x24846014, 0x24052600, 0x3021, 0x3821,
-0xafbf0018, 0xafa00010, 0xc002b3b, 0xafa00014,
-0x8fbf0018, 0x3e00008, 0x27bd0020, 0x3e00008,
-0x0, 0x3e00008, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x3e00008, 0x0, 0x3e00008, 0x0,
-0x27bdfde0, 0x27a50018, 0x3c04dead, 0x3484beef,
-0xafbf0218, 0x8f820150, 0x3c03001f, 0x3463ffff,
-0xafa40018, 0xa22823, 0xa32824, 0x8ca20000,
-0x1044000a, 0x0, 0xafa50010, 0x8ca20000,
-0xafa20014, 0x8f860150, 0x8f870250, 0x3c040001,
-0x2484601c, 0xc002b3b, 0x24052700, 0x8fbf0218,
-0x3e00008, 0x27bd0220, 0x27bdffe0, 0x3c06abba,
-0x34c6babe, 0xafb00018, 0x3c100004, 0x3c07007f,
-0x34e7ffff, 0xafbf001c, 0x102840, 0x8e040000,
-0x8ca30000, 0xaca00000, 0xae060000, 0x8ca20000,
-0xaca30000, 0x10460005, 0xae040000, 0xa08021,
-0xf0102b, 0x1040fff5, 0x102840, 0x3c040001,
-0x24846028, 0x24052800, 0x2003021, 0x3821,
-0xafa00010, 0xc002b3b, 0xafa00014, 0x2001021,
-0x8fbf001c, 0x8fb00018, 0x3e00008, 0x27bd0020,
-0x8c020224, 0x3047003f, 0x10e00010, 0x803021,
-0x2821, 0x24030020, 0xe31024, 0x10400002,
-0x63042, 0xa62821, 0x31842, 0x1460fffb,
-0xe31024, 0x2402f000, 0xa22824, 0x3402ffff,
-0x45102b, 0x14400003, 0x3c020001, 0x10000008,
-0x3c020001, 0x3442ffff, 0x851823, 0x43102b,
-0x14400003, 0xa01021, 0x3c02fffe, 0x821021,
-0x3e00008, 0x0, 0x27bdffd0, 0xafb50028,
-0x8fb50040, 0xafb20020, 0xa09021, 0xafb1001c,
-0x24c60003, 0xafbf002c, 0xafb30024, 0xafb00018,
-0x8ea20000, 0x2403fffc, 0xc38024, 0x50102b,
-0x1440001b, 0xe08821, 0x8e330000, 0xafb00010,
-0x8ea20000, 0xafa20014, 0x8e270000, 0x24053000,
-0xc002b3b, 0x2403021, 0x8e230000, 0x702021,
-0x64102b, 0x10400007, 0x2402821, 0x8ca20000,
-0xac620000, 0x24630004, 0x64102b, 0x1440fffb,
-0x24a50004, 0x8ea20000, 0x501023, 0xaea20000,
-0x8e220000, 0x501021, 0x1000000b, 0xae220000,
-0x2402002d, 0xa0820000, 0xafb00010, 0x8ea20000,
-0x2409821, 0xafa20014, 0x8e270000, 0x24053100,
-0xc002b3b, 0x2603021, 0x2601021, 0x8fbf002c,
-0x8fb50028, 0x8fb30024, 0x8fb20020, 0x8fb1001c,
-0x8fb00018, 0x3e00008, 0x27bd0030, 0x27bdffe8,
-0x3c1cc000, 0x3c05fffe, 0x3c030001, 0x8c636e84,
-0x3c040001, 0x8c846e90, 0x34a5bf08, 0x24021ffc,
-0x3c010001, 0xac226cd0, 0x3c0200c0, 0x3c010001,
-0xac226cd4, 0x3c020020, 0xafbf0010, 0x3c0100c0,
-0xac201ffc, 0x431023, 0x441023, 0x245bb000,
-0x365b821, 0x3c1d0001, 0x8fbd6ccc, 0x3a0f021,
-0x3c0400c0, 0x34840200, 0x3c1a00c0, 0x3c0300c0,
-0x346307c8, 0x24021dfc, 0x3c010001, 0xac226cd0,
-0x24021834, 0x3c010001, 0xac246cd4, 0x3c010001,
-0xac226cd0, 0x3c010001, 0xac236cd4, 0xc00180d,
-0x375a0200, 0x8fbf0010, 0x3e00008, 0x27bd0018,
-0x27bdffc8, 0x3c040001, 0x24846034, 0x24053200,
-0x3c020001, 0x8c426cd0, 0x3c030001, 0x8c636cd4,
-0x3021, 0x3603821, 0xafbf0030, 0xafb3002c,
-0xafb20028, 0xafb10024, 0xafb00020, 0xafa2001c,
-0xafa30018, 0xafb70010, 0xc002b3b, 0xafba0014,
-0xc001916, 0x0, 0x8f820240, 0x34420004,
-0xaf820240, 0x24020001, 0xaf420000, 0x3c020001,
-0x571021, 0x904240f4, 0x10400092, 0x2403fffc,
-0x3c100001, 0x2610ac73, 0x3c120001, 0x2652a84c,
-0x2121023, 0x438024, 0x8fa3001c, 0x3c040001,
-0x24846040, 0x70102b, 0x1440001a, 0x27b30018,
-0x8fb10018, 0x24053000, 0x2403021, 0xafb00010,
-0xafa30014, 0xc002b3b, 0x2203821, 0x8fa30018,
-0x702021, 0x64102b, 0x10400007, 0x2403021,
-0x8cc20000, 0xac620000, 0x24630004, 0x64102b,
-0x1440fffb, 0x24c60004, 0x8fa2001c, 0x501023,
-0xafa2001c, 0x8e620000, 0x501021, 0x1000000a,
-0xae620000, 0x2408821, 0x24053100, 0xafb00010,
-0xafa30014, 0x8fa70018, 0x2203021, 0x2402002d,
-0xc002b3b, 0xa0820000, 0x24070020, 0x8fa3001c,
-0x3c040001, 0x2484605c, 0x24120020, 0x3c010001,
-0xac316eb0, 0x2c620020, 0x1440001d, 0x27b10018,
-0x8fb00018, 0x24053000, 0x3c060001, 0x24c66f50,
-0xafa70010, 0xafa30014, 0xc002b3b, 0x2003821,
-0x8fa30018, 0x3c040001, 0x24846f50, 0x24650020,
-0x65102b, 0x10400007, 0x0, 0x8c820000,
-0xac620000, 0x24630004, 0x65102b, 0x1440fffb,
-0x24840004, 0x8fa2001c, 0x521023, 0xafa2001c,
-0x8e220000, 0x521021, 0x1000000b, 0xae220000,
-0x3c100001, 0x26106f50, 0x24053100, 0xafa70010,
-0xafa30014, 0x8fa70018, 0x2003021, 0x2402002d,
-0xc002b3b, 0xa0820000, 0x24070020, 0x3c040001,
-0x24846070, 0x8fa3001c, 0x24120020, 0x3c010001,
-0xac306ee4, 0x2c620020, 0x1440001d, 0x27b10018,
-0x8fb00018, 0x24053000, 0x3c060001, 0x24c66f70,
-0xafa70010, 0xafa30014, 0xc002b3b, 0x2003821,
-0x8fa30018, 0x3c040001, 0x24846f70, 0x24650020,
-0x65102b, 0x10400007, 0x0, 0x8c820000,
-0xac620000, 0x24630004, 0x65102b, 0x1440fffb,
-0x24840004, 0x8fa2001c, 0x521023, 0xafa2001c,
-0x8e220000, 0x521021, 0x1000000b, 0xae220000,
-0x3c100001, 0x26106f70, 0x24053100, 0xafa70010,
-0xafa30014, 0x8fa70018, 0x2003021, 0x2402002d,
-0xc002b3b, 0xa0820000, 0x3c010001, 0x10000031,
-0xac306ee0, 0x3c100001, 0x2610821f, 0x3c120001,
-0x2652809c, 0x2121023, 0x438024, 0x8fa3001c,
-0x3c040001, 0x24846084, 0x70102b, 0x1440001a,
-0x27b30018, 0x8fb10018, 0x24053000, 0x2403021,
-0xafb00010, 0xafa30014, 0xc002b3b, 0x2203821,
-0x8fa30018, 0x702021, 0x64102b, 0x10400007,
-0x2403021, 0x8cc20000, 0xac620000, 0x24630004,
-0x64102b, 0x1440fffb, 0x24c60004, 0x8fa2001c,
-0x501023, 0xafa2001c, 0x8e620000, 0x501021,
-0x1000000a, 0xae620000, 0x2408821, 0x24053100,
-0xafb00010, 0xafa30014, 0x8fa70018, 0x2203021,
-0x2402002d, 0xc002b3b, 0xa0820000, 0x3c010001,
-0xac316eb0, 0x3c030001, 0x8c636eb0, 0x24020400,
-0x60f809, 0xaf820070, 0x8fbf0030, 0x8fb3002c,
-0x8fb20028, 0x8fb10024, 0x8fb00020, 0x3e00008,
-0x27bd0038, 0x0, 0x0, 0x8f820040,
-0x3c03f000, 0x431024, 0x3c036000, 0x14430006,
-0x0, 0x8f820050, 0x2403ff80, 0x431024,
-0x34420055, 0xaf820050, 0x8f820054, 0x244203e8,
-0xaf820058, 0x240201f4, 0xaf4200e0, 0x24020004,
-0xaf4200e8, 0x24020002, 0xaf4001b0, 0xaf4000e4,
-0xaf4200dc, 0xaf4000d8, 0xaf4000d4, 0x3e00008,
-0xaf4000d0, 0x8f820054, 0x24420005, 0x3e00008,
-0xaf820078, 0x27bdffe8, 0xafbf0010, 0x8f820054,
-0x244203e8, 0xaf820058, 0x3c020800, 0x2c21024,
-0x10400004, 0x3c02f7ff, 0x3442ffff, 0x2c2b024,
-0x36940040, 0x3c020001, 0x8c426da8, 0x10400017,
-0x3c020200, 0x3c030001, 0x8c636f1c, 0x10600016,
-0x282a025, 0x3c020001, 0x8c426e44, 0x14400012,
-0x3c020200, 0x3c020001, 0x8c426d94, 0x30420003,
-0x1440000d, 0x3c020200, 0x8f830224, 0x3c020002,
-0x8c428fec, 0x10620008, 0x3c020200, 0xc003daf,
-0x0, 0x10000004, 0x3c020200, 0xc004196,
-0x0, 0x3c020200, 0x2c21024, 0x10400003,
-0x0, 0xc001f4b, 0x0, 0x8f4200d8,
-0x8f4300dc, 0x24420001, 0xaf4200d8, 0x43102b,
-0x14400003, 0x0, 0xaf4000d8, 0x36940080,
-0x8c030238, 0x1060000c, 0x0, 0x8f4201b0,
-0x244203e8, 0xaf4201b0, 0x43102b, 0x14400006,
-0x0, 0x934205c5, 0x14400003, 0x0,
-0xc001da0, 0x0, 0x8fbf0010, 0x3e00008,
-0x27bd0018, 0x3e00008, 0x0, 0x27bdffd8,
-0xafbf0020, 0x8f43002c, 0x8f420038, 0x10620059,
-0x0, 0x3c020001, 0x571021, 0x904240f0,
-0x10400026, 0x24070008, 0x8f440170, 0x8f450174,
-0x8f48000c, 0x8f860120, 0x24020020, 0xafa20010,
-0xafa30014, 0xafa80018, 0x8f42010c, 0x40f809,
-0x24c6001c, 0x14400011, 0x24020001, 0x3c010001,
-0x370821, 0xa02240f0, 0x8f820124, 0xafa20010,
-0x8f820128, 0x3c040001, 0x24846128, 0xafa20014,
-0x8f46002c, 0x8f870120, 0x3c050009, 0xc002b3b,
-0x34a50900, 0x1000005c, 0x0, 0x8f420300,
-0x24420001, 0xaf420300, 0x8f420300, 0x8f42002c,
-0xa34005c1, 0x10000027, 0xaf420038, 0x8f440170,
-0x8f450174, 0x8f43002c, 0x8f48000c, 0x8f860120,
-0x24020080, 0xafa20010, 0xafa30014, 0xafa80018,
-0x8f42010c, 0x40f809, 0x24c6001c, 0x14400011,
-0x24020001, 0x3c010001, 0x370821, 0xa02240f1,
-0x8f820124, 0xafa20010, 0x8f820128, 0x3c040001,
-0x24846134, 0xafa20014, 0x8f46002c, 0x8f870120,
-0x3c050009, 0xc002b3b, 0x34a51100, 0x10000036,
-0x0, 0x8f420300, 0x8f43002c, 0x24420001,
-0xaf420300, 0x8f420300, 0x24020001, 0xa34205c1,
-0xaf430038, 0x3c010001, 0x370821, 0xa02040f1,
-0x3c010001, 0x370821, 0xa02040f0, 0x10000026,
-0xaf400034, 0x934205c1, 0x1040001d, 0x0,
-0xa34005c1, 0x8f820040, 0x30420001, 0x14400008,
-0x2021, 0x8c030104, 0x24020001, 0x50620005,
-0x24040001, 0x8c020264, 0x10400003, 0x801021,
-0x24040001, 0x801021, 0x10400006, 0x0,
-0x8f42030c, 0x24420001, 0xaf42030c, 0x10000008,
-0x8f42030c, 0x8f820044, 0x34420004, 0xaf820044,
-0x8f420308, 0x24420001, 0xaf420308, 0x8f420308,
-0x3c010001, 0x370821, 0xa02040f0, 0x3c010001,
-0x370821, 0xa02040f1, 0x8f420000, 0x10400007,
-0x0, 0xaf80004c, 0x8f82004c, 0x1040fffd,
-0x0, 0x10000005, 0x0, 0xaf800048,
-0x8f820048, 0x1040fffd, 0x0, 0x8f820060,
-0x3c03ff7f, 0x3463ffff, 0x431024, 0xaf820060,
-0x8f420000, 0x10400003, 0x0, 0x10000002,
-0xaf80004c, 0xaf800048, 0x8fbf0020, 0x3e00008,
-0x27bd0028, 0x3e00008, 0x0, 0x27bdffd8,
-0xafbf0020, 0x8f430044, 0x8f42007c, 0x10620029,
-0x24070008, 0x8f440168, 0x8f45016c, 0x8f48000c,
-0x8f860120, 0x24020040, 0xafa20010, 0xafa30014,
-0xafa80018, 0x8f42010c, 0x40f809, 0x24c6001c,
-0x14400011, 0x24020001, 0x3c010001, 0x370821,
-0xa02240f2, 0x8f820124, 0xafa20010, 0x8f820128,
-0x3c040001, 0x2484613c, 0xafa20014, 0x8f460044,
-0x8f870120, 0x3c050009, 0xc002b3b, 0x34a51300,
-0x1000000f, 0x0, 0x8f420304, 0x24420001,
-0xaf420304, 0x8f420304, 0x8f420044, 0xaf42007c,
-0x3c010001, 0x370821, 0xa02040f2, 0x10000004,
-0xaf400078, 0x3c010001, 0x370821, 0xa02040f2,
-0x8f420000, 0x10400007, 0x0, 0xaf80004c,
-0x8f82004c, 0x1040fffd, 0x0, 0x10000005,
-0x0, 0xaf800048, 0x8f820048, 0x1040fffd,
-0x0, 0x8f820060, 0x3c03feff, 0x3463ffff,
-0x431024, 0xaf820060, 0x8f420000, 0x10400003,
-0x0, 0x10000002, 0xaf80004c, 0xaf800048,
-0x8fbf0020, 0x3e00008, 0x27bd0028, 0x3e00008,
-0x0, 0x3c020001, 0x8c426da8, 0x27bdffa8,
-0xafbf0050, 0xafbe004c, 0xafb50048, 0xafb30044,
-0xafb20040, 0xafb1003c, 0xafb00038, 0x104000d5,
-0x8f900044, 0x8f4200d0, 0x24430001, 0x2842000b,
-0x144000e4, 0xaf4300d0, 0x8f420004, 0x30420002,
-0x1440009c, 0xaf4000d0, 0x8f420004, 0x3c030001,
-0x8c636d98, 0x34420002, 0xaf420004, 0x24020001,
-0x14620003, 0x3c020600, 0x10000002, 0x34423000,
-0x34421000, 0xafa20020, 0x8f4a0018, 0xafaa0034,
-0x27aa0020, 0xafaa002c, 0x8faa0034, 0x240200ff,
-0x11420002, 0x1821, 0x25430001, 0x8c020228,
-0x609821, 0x1662000e, 0x3c050009, 0x8f42033c,
-0x24420001, 0xaf42033c, 0x8f42033c, 0x8c020228,
-0x8fa70034, 0x3c040001, 0x2484610c, 0xafa00014,
-0xafa20010, 0x8fa60020, 0x10000070, 0x34a50500,
-0x8faa0034, 0xa38c0, 0xf71021, 0x8fa30020,
-0x8fa40024, 0xac4304c0, 0xac4404c4, 0x8f830054,
-0x8f820054, 0x247103e8, 0x2221023, 0x2c4203e9,
-0x1040001b, 0xa821, 0xe09021, 0x265e04c0,
-0x8f440178, 0x8f45017c, 0x2401821, 0x240a0004,
-0xafaa0010, 0xafb30014, 0x8f48000c, 0x1021,
-0x2fe3021, 0xafa80018, 0x8f48010c, 0x24070008,
-0xa32821, 0xa3482b, 0x822021, 0x100f809,
-0x892021, 0x54400006, 0x24150001, 0x8f820054,
-0x2221023, 0x2c4203e9, 0x1440ffe9, 0x0,
-0x32a200ff, 0x54400018, 0xaf530018, 0x8f420378,
-0x24420001, 0xaf420378, 0x8f420378, 0x8f820120,
-0x8faa002c, 0x8fa70034, 0xafa20010, 0x8f820124,
-0x3c040001, 0x24846118, 0xafa20014, 0x8d460000,
-0x3c050009, 0x10000035, 0x34a50600, 0x8f420308,
-0x24150001, 0x24420001, 0xaf420308, 0x8f420308,
-0x1000001e, 0x32a200ff, 0x8f830054, 0x8f820054,
-0x247103e8, 0x2221023, 0x2c4203e9, 0x10400016,
-0xa821, 0x3c1e0020, 0x24120010, 0x8f42000c,
-0x8f440160, 0x8f450164, 0x8f860120, 0xafb20010,
-0xafb30014, 0x5e1025, 0xafa20018, 0x8f42010c,
-0x24070008, 0x40f809, 0x24c6001c, 0x1440ffe3,
-0x0, 0x8f820054, 0x2221023, 0x2c4203e9,
-0x1440ffee, 0x0, 0x32a200ff, 0x14400011,
-0x3c050009, 0x8f420378, 0x24420001, 0xaf420378,
-0x8f420378, 0x8f820120, 0x8faa002c, 0x8fa70034,
-0xafa20010, 0x8f820124, 0x3c040001, 0x24846120,
-0xafa20014, 0x8d460000, 0x34a50700, 0xc002b3b,
-0x0, 0x8f4202ec, 0x24420001, 0xaf4202ec,
-0x8f4202ec, 0x8f420004, 0x30420001, 0x50400029,
-0x36100040, 0x3c020400, 0x2c21024, 0x10400013,
-0x2404ffdf, 0x8f420250, 0x8f430254, 0x8f4401b4,
-0x14640006, 0x36100040, 0x8f420270, 0x8f430274,
-0x8f4401b8, 0x10640007, 0x2402ffdf, 0x8f420250,
-0x8f430254, 0x8f440270, 0x8f450274, 0x10000012,
-0x3a100020, 0x1000002b, 0x2028024, 0x8f420250,
-0x8f430254, 0x8f4501b4, 0x14650006, 0x2048024,
-0x8f420270, 0x8f430274, 0x8f4401b8, 0x50640021,
-0x36100040, 0x8f420250, 0x8f430254, 0x8f440270,
-0x8f450274, 0x3a100040, 0xaf4301b4, 0x10000019,
-0xaf4501b8, 0x8f4200d4, 0x24430001, 0x10000011,
-0x28420033, 0x8f420004, 0x30420001, 0x10400009,
-0x3c020400, 0x2c21024, 0x10400004, 0x2402ffdf,
-0x2028024, 0x1000000b, 0x36100040, 0x10000009,
-0x36100060, 0x8f4200d4, 0x36100040, 0x24430001,
-0x284201f5, 0x14400003, 0xaf4300d4, 0xaf4000d4,
-0x3a100020, 0xaf900044, 0x2402ff7f, 0x282a024,
-0x8fbf0050, 0x8fbe004c, 0x8fb50048, 0x8fb30044,
-0x8fb20040, 0x8fb1003c, 0x8fb00038, 0x3e00008,
-0x27bd0058, 0x3e00008, 0x0, 0x3c020001,
-0x8c426da8, 0x27bdffb0, 0xafbf0048, 0xafbe0044,
-0xafb50040, 0xafb3003c, 0xafb20038, 0xafb10034,
-0x104000c7, 0xafb00030, 0x8f4200d0, 0x24430001,
-0x2842000b, 0x144000da, 0xaf4300d0, 0x8f420004,
-0x30420002, 0x14400097, 0xaf4000d0, 0x8f420004,
-0x3c030001, 0x8c636d98, 0x34420002, 0xaf420004,
-0x24020001, 0x14620003, 0x3c020600, 0x10000002,
-0x34423000, 0x34421000, 0xafa20020, 0x1821,
-0x8f5e0018, 0x27aa0020, 0x240200ff, 0x13c20002,
-0xafaa002c, 0x27c30001, 0x8c020228, 0x609021,
-0x1642000e, 0x1e38c0, 0x8f42033c, 0x24420001,
-0xaf42033c, 0x8f42033c, 0x8c020228, 0x3c040001,
-0x2484610c, 0x3c050009, 0xafa00014, 0xafa20010,
-0x8fa60020, 0x1000006d, 0x34a50500, 0xf71021,
-0x8fa30020, 0x8fa40024, 0xac4304c0, 0xac4404c4,
-0x8f830054, 0x8f820054, 0x247003e8, 0x2021023,
-0x2c4203e9, 0x1040001b, 0x9821, 0xe08821,
-0x263504c0, 0x8f440178, 0x8f45017c, 0x2201821,
-0x240a0004, 0xafaa0010, 0xafb20014, 0x8f48000c,
-0x1021, 0x2f53021, 0xafa80018, 0x8f48010c,
-0x24070008, 0xa32821, 0xa3482b, 0x822021,
-0x100f809, 0x892021, 0x54400006, 0x24130001,
-0x8f820054, 0x2021023, 0x2c4203e9, 0x1440ffe9,
-0x0, 0x326200ff, 0x54400017, 0xaf520018,
-0x8f420378, 0x24420001, 0xaf420378, 0x8f420378,
-0x8f820120, 0x8faa002c, 0xafa20010, 0x8f820124,
-0x3c040001, 0x24846118, 0x3c050009, 0xafa20014,
-0x8d460000, 0x10000035, 0x34a50600, 0x8f420308,
-0x24130001, 0x24420001, 0xaf420308, 0x8f420308,
-0x1000001e, 0x326200ff, 0x8f830054, 0x8f820054,
-0x247003e8, 0x2021023, 0x2c4203e9, 0x10400016,
-0x9821, 0x3c150020, 0x24110010, 0x8f42000c,
-0x8f440160, 0x8f450164, 0x8f860120, 0xafb10010,
-0xafb20014, 0x551025, 0xafa20018, 0x8f42010c,
-0x24070008, 0x40f809, 0x24c6001c, 0x1440ffe3,
-0x0, 0x8f820054, 0x2021023, 0x2c4203e9,
-0x1440ffee, 0x0, 0x326200ff, 0x14400011,
-0x0, 0x8f420378, 0x24420001, 0xaf420378,
-0x8f420378, 0x8f820120, 0x8faa002c, 0xafa20010,
-0x8f820124, 0x3c040001, 0x24846120, 0x3c050009,
-0xafa20014, 0x8d460000, 0x34a50700, 0xc002b3b,
-0x3c03821, 0x8f4202ec, 0x24420001, 0xaf4202ec,
-0x8f4202ec, 0x8f420004, 0x30420001, 0x10400018,
-0x24040001, 0x8f420250, 0x8f430254, 0x8f4501b4,
-0x3c010001, 0x14650006, 0xa0246cf1, 0x8f420270,
-0x8f430274, 0x8f4401b8, 0x10640021, 0x0,
-0x8f420250, 0x8f430254, 0x3c040001, 0x90846cf0,
-0x8f460270, 0x8f470274, 0x38840001, 0xaf4301b4,
-0xaf4701b8, 0x3c010001, 0x10000025, 0xa0246cf0,
-0x8f4200d4, 0x3c010001, 0xa0206cf0, 0x24430001,
-0x28420033, 0x1440001e, 0xaf4300d4, 0x3c020001,
-0x90426cf1, 0xaf4000d4, 0x10000017, 0x38420001,
-0x8f420004, 0x30420001, 0x10400008, 0x0,
-0xc00565a, 0x2021, 0x3c010001, 0xa0206cf1,
-0x3c010001, 0x1000000e, 0xa0206cf0, 0x8f4200d4,
-0x3c010001, 0xa0206cf0, 0x24430001, 0x284201f5,
-0x14400007, 0xaf4300d4, 0x3c020001, 0x90426cf1,
-0xaf4000d4, 0x421026, 0x3c010001, 0xa0226cf1,
-0x3c030001, 0x8c636d98, 0x24020002, 0x1462000c,
-0x3c030002, 0x3c030001, 0x90636cf1, 0x24020001,
-0x5462001f, 0x2021, 0x3c020001, 0x90426cf0,
-0x1443001b, 0x24040005, 0x10000019, 0x24040006,
-0x3c020002, 0x8c428ff4, 0x431024, 0x1040000b,
-0x24020001, 0x3c030001, 0x90636cf1, 0x54620010,
-0x2021, 0x3c020001, 0x90426cf0, 0x1443000c,
-0x24040003, 0x1000000a, 0x24040004, 0x3c030001,
-0x90636cf1, 0x14620006, 0x2021, 0x3c020001,
-0x90426cf0, 0x24040001, 0x50440001, 0x24040002,
-0xc00565a, 0x0, 0x2402ff7f, 0x282a024,
-0x8fbf0048, 0x8fbe0044, 0x8fb50040, 0x8fb3003c,
-0x8fb20038, 0x8fb10034, 0x8fb00030, 0x3e00008,
-0x27bd0050, 0x3e00008, 0x0, 0x3c020001,
-0x8c426da8, 0x27bdffb0, 0xafbf0048, 0xafbe0044,
-0xafb50040, 0xafb3003c, 0xafb20038, 0xafb10034,
-0x104000de, 0xafb00030, 0x8f4200d0, 0x3c040001,
-0x8c846d98, 0x24430001, 0x2842000b, 0xaf4400e8,
-0x144000fe, 0xaf4300d0, 0x8f420004, 0x30420002,
-0x14400095, 0xaf4000d0, 0x8f420004, 0x34420002,
-0xaf420004, 0x24020001, 0x14820003, 0x3c020600,
-0x10000002, 0x34423000, 0x34421000, 0xafa20020,
-0x1821, 0x8f5e0018, 0x27aa0020, 0x240200ff,
-0x13c20002, 0xafaa002c, 0x27c30001, 0x8c020228,
-0x609021, 0x1642000e, 0x1e38c0, 0x8f42033c,
-0x24420001, 0xaf42033c, 0x8f42033c, 0x8c020228,
-0x3c040001, 0x2484610c, 0x3c050009, 0xafa00014,
-0xafa20010, 0x8fa60020, 0x1000006d, 0x34a50500,
-0xf71021, 0x8fa30020, 0x8fa40024, 0xac4304c0,
-0xac4404c4, 0x8f830054, 0x8f820054, 0x247003e8,
-0x2021023, 0x2c4203e9, 0x1040001b, 0x9821,
-0xe08821, 0x263504c0, 0x8f440178, 0x8f45017c,
-0x2201821, 0x240a0004, 0xafaa0010, 0xafb20014,
-0x8f48000c, 0x1021, 0x2f53021, 0xafa80018,
-0x8f48010c, 0x24070008, 0xa32821, 0xa3482b,
-0x822021, 0x100f809, 0x892021, 0x54400006,
-0x24130001, 0x8f820054, 0x2021023, 0x2c4203e9,
-0x1440ffe9, 0x0, 0x326200ff, 0x54400017,
-0xaf520018, 0x8f420378, 0x24420001, 0xaf420378,
-0x8f420378, 0x8f820120, 0x8faa002c, 0xafa20010,
-0x8f820124, 0x3c040001, 0x24846118, 0x3c050009,
-0xafa20014, 0x8d460000, 0x10000035, 0x34a50600,
-0x8f420308, 0x24130001, 0x24420001, 0xaf420308,
-0x8f420308, 0x1000001e, 0x326200ff, 0x8f830054,
-0x8f820054, 0x247003e8, 0x2021023, 0x2c4203e9,
-0x10400016, 0x9821, 0x3c150020, 0x24110010,
-0x8f42000c, 0x8f440160, 0x8f450164, 0x8f860120,
-0xafb10010, 0xafb20014, 0x551025, 0xafa20018,
-0x8f42010c, 0x24070008, 0x40f809, 0x24c6001c,
-0x1440ffe3, 0x0, 0x8f820054, 0x2021023,
-0x2c4203e9, 0x1440ffee, 0x0, 0x326200ff,
-0x14400011, 0x0, 0x8f420378, 0x24420001,
-0xaf420378, 0x8f420378, 0x8f820120, 0x8faa002c,
-0xafa20010, 0x8f820124, 0x3c040001, 0x24846120,
-0x3c050009, 0xafa20014, 0x8d460000, 0x34a50700,
-0xc002b3b, 0x3c03821, 0x8f4202ec, 0x24420001,
-0xaf4202ec, 0x8f4202ec, 0x8f420004, 0x30420001,
-0x10400033, 0x3c020400, 0x2c21024, 0x10400017,
-0x0, 0x934205c0, 0x8f440250, 0x8f450254,
-0x8f4301b4, 0x34420020, 0x14a30006, 0xa34205c0,
-0x8f420270, 0x8f430274, 0x8f4401b8, 0x10640008,
-0x0, 0x8f420250, 0x8f430254, 0x934405c0,
-0x8f460270, 0x8f470274, 0x10000016, 0x38840040,
-0x934205c0, 0x10000048, 0x304200bf, 0x934205c0,
-0x8f440250, 0x8f450254, 0x8f4301b4, 0x304200bf,
-0x14a30006, 0xa34205c0, 0x8f420270, 0x8f430274,
-0x8f4401b8, 0x1064000b, 0x0, 0x8f420250,
-0x8f430254, 0x934405c0, 0x8f460270, 0x8f470274,
-0x38840020, 0xaf4301b4, 0xaf4701b8, 0x10000033,
-0xa34405c0, 0x934205c0, 0x1000002f, 0x34420020,
-0x934205c0, 0x8f4300d4, 0x34420020, 0xa34205c0,
-0x24620001, 0x10000023, 0x28630033, 0x8f4200e4,
-0x8f4300e0, 0x24420001, 0xaf4200e4, 0x43102a,
-0x14400006, 0x24030001, 0x8f4200e8, 0x14430002,
-0xaf4000e4, 0x24030004, 0xaf4300e8, 0x8f420004,
-0x30420001, 0x1040000d, 0x3c020400, 0x2c21024,
-0x10400007, 0x0, 0x934205c0, 0x34420040,
-0xa34205c0, 0x934205c0, 0x1000000f, 0x304200df,
-0x934205c0, 0x1000000c, 0x34420060, 0x934205c0,
-0x8f4300d4, 0x34420020, 0xa34205c0, 0x24620001,
-0x286300fb, 0x14600005, 0xaf4200d4, 0x934205c0,
-0xaf4000d4, 0x38420040, 0xa34205c0, 0x934205c0,
-0x8f4300e8, 0x3042007f, 0xa34205c0, 0x24020001,
-0x14620005, 0x0, 0x934405c0, 0x42102,
-0x10000003, 0x348400f0, 0x934405c0, 0x3484000f,
-0xc005640, 0x0, 0x2402ff7f, 0x282a024,
-0x8fbf0048, 0x8fbe0044, 0x8fb50040, 0x8fb3003c,
-0x8fb20038, 0x8fb10034, 0x8fb00030, 0x3e00008,
-0x27bd0050, 0x3e00008, 0x0, 0x27bdffb0,
-0x274401c0, 0x26e30028, 0x24650400, 0x65102b,
-0xafbf0048, 0xafbe0044, 0xafb50040, 0xafb3003c,
-0xafb20038, 0xafb10034, 0x10400007, 0xafb00030,
-0x8c820000, 0xac620000, 0x24630004, 0x65102b,
-0x1440fffb, 0x24840004, 0x8c020080, 0xaee20044,
-0x8c0200c0, 0xaee20040, 0x8c020084, 0xaee20030,
-0x8c020084, 0xaee2023c, 0x8c020088, 0xaee20240,
-0x8c02008c, 0xaee20244, 0x8c020090, 0xaee20248,
-0x8c020094, 0xaee2024c, 0x8c020098, 0xaee20250,
-0x8c02009c, 0xaee20254, 0x8c0200a0, 0xaee20258,
-0x8c0200a4, 0xaee2025c, 0x8c0200a8, 0xaee20260,
-0x8c0200ac, 0xaee20264, 0x8c0200b0, 0xaee20268,
-0x8c0200b4, 0xaee2026c, 0x8c0200b8, 0xaee20270,
-0x8c0200bc, 0x24040001, 0xaee20274, 0xaee00034,
-0x41080, 0x571021, 0x8ee30034, 0x8c42023c,
-0x24840001, 0x621821, 0x2c82000f, 0xaee30034,
-0x1440fff8, 0x41080, 0x8c0200cc, 0xaee20048,
-0x8c0200d0, 0xaee2004c, 0x8c0200e0, 0xaee201f8,
-0x8c0200e4, 0xaee201fc, 0x8c0200e8, 0xaee20200,
-0x8c0200ec, 0xaee20204, 0x8c0200f0, 0xaee20208,
-0x8ee400c0, 0x8ee500c4, 0x8c0200fc, 0x45102b,
-0x1040000b, 0x0, 0x8ee200c0, 0x8ee300c4,
-0x24040001, 0x24050000, 0x651821, 0x65302b,
-0x441021, 0x461021, 0xaee200c0, 0xaee300c4,
-0x8c0200fc, 0x8ee400c0, 0x8ee500c4, 0x2408ffff,
-0x24090000, 0x401821, 0x1021, 0x882024,
-0xa92824, 0x822025, 0xa32825, 0xaee400c0,
-0xaee500c4, 0x8ee400d0, 0x8ee500d4, 0x8c0200f4,
-0x45102b, 0x1040000b, 0x0, 0x8ee200d0,
-0x8ee300d4, 0x24040001, 0x24050000, 0x651821,
-0x65302b, 0x441021, 0x461021, 0xaee200d0,
-0xaee300d4, 0x8c0200f4, 0x8ee400d0, 0x8ee500d4,
-0x401821, 0x1021, 0x882024, 0xa92824,
-0x822025, 0xa32825, 0xaee400d0, 0xaee500d4,
-0x8ee400c8, 0x8ee500cc, 0x8c0200f8, 0x45102b,
-0x1040000b, 0x0, 0x8ee200c8, 0x8ee300cc,
-0x24040001, 0x24050000, 0x651821, 0x65302b,
-0x441021, 0x461021, 0xaee200c8, 0xaee300cc,
-0x8c0200f8, 0x8ee400c8, 0x8ee500cc, 0x401821,
-0x1021, 0x882024, 0xa92824, 0x822025,
-0xa32825, 0x24020008, 0xaee400c8, 0xaee500cc,
-0xafa20010, 0xafa00014, 0x8f42000c, 0x8c040208,
-0x8c05020c, 0xafa20018, 0x8f42010c, 0x26e60028,
-0x40f809, 0x24070400, 0x104000f0, 0x3c020400,
-0xafa20020, 0x934205c6, 0x10400089, 0x1821,
-0x8f5e0018, 0x27aa0020, 0x240200ff, 0x13c20002,
-0xafaa002c, 0x27c30001, 0x8c020228, 0x609021,
-0x1642000e, 0x1e38c0, 0x8f42033c, 0x24420001,
-0xaf42033c, 0x8f42033c, 0x8c020228, 0x3c040001,
-0x2484610c, 0x3c050009, 0xafa00014, 0xafa20010,
-0x8fa60020, 0x1000006b, 0x34a50500, 0xf71021,
-0x8fa30020, 0x8fa40024, 0xac4304c0, 0xac4404c4,
-0x8f830054, 0x8f820054, 0x247003e8, 0x2021023,
-0x2c4203e9, 0x1040001b, 0x9821, 0xe08821,
-0x263504c0, 0x8f440178, 0x8f45017c, 0x2201821,
-0x240a0004, 0xafaa0010, 0xafb20014, 0x8f48000c,
-0x1021, 0x2f53021, 0xafa80018, 0x8f48010c,
-0x24070008, 0xa32821, 0xa3482b, 0x822021,
-0x100f809, 0x892021, 0x54400006, 0x24130001,
-0x8f820054, 0x2021023, 0x2c4203e9, 0x1440ffe9,
-0x0, 0x326200ff, 0x54400017, 0xaf520018,
-0x8f420378, 0x24420001, 0xaf420378, 0x8f420378,
-0x8f820120, 0x8faa002c, 0xafa20010, 0x8f820124,
-0x3c040001, 0x24846118, 0x3c050009, 0xafa20014,
-0x8d460000, 0x10000033, 0x34a50600, 0x8f420308,
-0x24130001, 0x24420001, 0xaf420308, 0x8f420308,
-0x1000001c, 0x326200ff, 0x8f830054, 0x8f820054,
-0x247003e8, 0x2021023, 0x2c4203e9, 0x10400014,
-0x9821, 0x24110010, 0x8f42000c, 0x8f440160,
-0x8f450164, 0x8f860120, 0xafb10010, 0xafb20014,
-0xafa20018, 0x8f42010c, 0x24070008, 0x40f809,
-0x24c6001c, 0x1440ffe5, 0x0, 0x8f820054,
-0x2021023, 0x2c4203e9, 0x1440ffef, 0x0,
-0x326200ff, 0x54400012, 0x24020001, 0x8f420378,
-0x24420001, 0xaf420378, 0x8f420378, 0x8f820120,
-0x8faa002c, 0xafa20010, 0x8f820124, 0x3c040001,
-0x24846120, 0x3c050009, 0xafa20014, 0x8d460000,
-0x34a50700, 0xc002b3b, 0x3c03821, 0x1021,
-0x1440005b, 0x24020001, 0x10000065, 0x0,
-0x8f510018, 0x240200ff, 0x12220002, 0x8021,
-0x26300001, 0x8c020228, 0x1602000e, 0x1130c0,
-0x8f42033c, 0x24420001, 0xaf42033c, 0x8f42033c,
-0x8c020228, 0x3c040001, 0x248460f4, 0x3c050009,
-0xafa00014, 0xafa20010, 0x8fa60020, 0x1000003f,
-0x34a50100, 0xd71021, 0x8fa30020, 0x8fa40024,
-0xac4304c0, 0xac4404c4, 0xc01821, 0x8f440178,
-0x8f45017c, 0x1021, 0x24070004, 0xafa70010,
-0xafb00014, 0x8f48000c, 0x24c604c0, 0x2e63021,
-0xafa80018, 0x8f48010c, 0x24070008, 0xa32821,
-0xa3482b, 0x822021, 0x100f809, 0x892021,
-0x1440000b, 0x24070008, 0x8f820120, 0xafa20010,
-0x8f820124, 0x3c040001, 0x248460fc, 0x3c050009,
-0xafa20014, 0x8fa60020, 0x1000001c, 0x34a50200,
-0x8f440160, 0x8f450164, 0x8f43000c, 0xaf500018,
-0x8f860120, 0x24020010, 0xafa20010, 0xafb00014,
-0xafa30018, 0x8f42010c, 0x40f809, 0x24c6001c,
-0x54400011, 0x24020001, 0x8f420340, 0x24420001,
-0xaf420340, 0x8f420340, 0x8f820120, 0xafa20010,
-0x8f820124, 0x3c040001, 0x24846104, 0x3c050009,
-0xafa20014, 0x8fa60020, 0x34a50300, 0xc002b3b,
-0x2203821, 0x1021, 0x1040000d, 0x24020001,
-0x8f4202e8, 0xa34005c6, 0xaf4001b0, 0x24420001,
-0xaf4202e8, 0x8f4202e8, 0x8ee20150, 0x24420001,
-0xaee20150, 0x10000003, 0x8ee20150, 0x24020001,
-0xa34205c6, 0x8fbf0048, 0x8fbe0044, 0x8fb50040,
-0x8fb3003c, 0x8fb20038, 0x8fb10034, 0x8fb00030,
-0x3e00008, 0x27bd0050, 0x27bdffd8, 0xafbf0020,
-0x8f8200b0, 0x30420004, 0x10400068, 0x0,
-0x8f430128, 0x8f820104, 0x14620005, 0x0,
-0x8f430130, 0x8f8200b4, 0x10620006, 0x0,
-0x8f820104, 0xaf420128, 0x8f8200b4, 0x1000005b,
-0xaf420130, 0x8f8200b0, 0x3c030080, 0x431024,
-0x1040000d, 0x0, 0x8f82011c, 0x34420002,
-0xaf82011c, 0x8f8200b0, 0x2403fffb, 0x431024,
-0xaf8200b0, 0x8f82011c, 0x2403fffd, 0x431024,
-0x1000004a, 0xaf82011c, 0x8f430128, 0x8f820104,
-0x14620005, 0x0, 0x8f430130, 0x8f8200b4,
-0x10620010, 0x0, 0x8f820104, 0xaf420128,
-0x8f8200b4, 0x8f430128, 0xaf420130, 0xafa30010,
-0x8f420130, 0x3c040001, 0x24846144, 0xafa20014,
-0x8f86011c, 0x8f8700b0, 0x3c050005, 0x10000031,
-0x34a50900, 0x8f420128, 0xafa20010, 0x8f420130,
-0x3c040001, 0x24846150, 0xafa20014, 0x8f86011c,
-0x8f8700b0, 0x3c050005, 0xc002b3b, 0x34a51000,
-0x8f82011c, 0x34420002, 0xaf82011c, 0x8f830104,
-0x8f8200b0, 0x34420001, 0xaf8200b0, 0x24020008,
-0xaf830104, 0xafa20010, 0xafa00014, 0x8f42000c,
-0x8c040208, 0x8c05020c, 0xafa20018, 0x8f42010c,
-0x26e60028, 0x40f809, 0x24070400, 0x8f82011c,
-0x2403fffd, 0x431024, 0xaf82011c, 0x8ee201dc,
-0x24420001, 0xaee201dc, 0x8ee201dc, 0x8f420128,
-0xafa20010, 0x8f420130, 0x3c040001, 0x2484615c,
-0xafa20014, 0x8f86011c, 0x8f8700b0, 0x3c050005,
-0x34a51100, 0xc002b3b, 0x0, 0x8f8200a0,
-0x30420004, 0x10400069, 0x0, 0x8f43012c,
-0x8f820124, 0x14620005, 0x0, 0x8f430134,
-0x8f8200a4, 0x10620006, 0x0, 0x8f820124,
-0xaf42012c, 0x8f8200a4, 0x1000005c, 0xaf420134,
-0x8f8200a0, 0x3c030080, 0x431024, 0x1040000d,
-0x0, 0x8f82011c, 0x34420002, 0xaf82011c,
-0x8f8200a0, 0x2403fffb, 0x431024, 0xaf8200a0,
-0x8f82011c, 0x2403fffd, 0x431024, 0x1000004b,
-0xaf82011c, 0x8f43012c, 0x8f820124, 0x14620005,
-0x0, 0x8f430134, 0x8f8200a4, 0x10620010,
-0x0, 0x8f820124, 0xaf42012c, 0x8f8200a4,
-0x8f43012c, 0xaf420134, 0xafa30010, 0x8f420134,
-0x3c040001, 0x24846168, 0xafa20014, 0x8f86011c,
-0x8f8700a0, 0x3c050005, 0x10000032, 0x34a51200,
-0x8f42012c, 0xafa20010, 0x8f420134, 0x3c040001,
-0x24846174, 0xafa20014, 0x8f86011c, 0x8f8700a0,
-0x3c050005, 0xc002b3b, 0x34a51300, 0x8f82011c,
-0x34420002, 0xaf82011c, 0x8f830124, 0x8f8200a0,
-0x34420001, 0xaf8200a0, 0x24020080, 0xaf830124,
-0xafa20010, 0xafa00014, 0x8f420014, 0x8c040208,
-0x8c05020c, 0xafa20018, 0x8f420108, 0x3c060001,
-0x24c66ed8, 0x40f809, 0x24070004, 0x8f82011c,
-0x2403fffd, 0x431024, 0xaf82011c, 0x8ee201dc,
-0x24420001, 0xaee201dc, 0x8ee201dc, 0x8f42012c,
-0xafa20010, 0x8f420134, 0x3c040001, 0x24846180,
-0xafa20014, 0x8f86011c, 0x8f8700a0, 0x3c050005,
-0x34a51400, 0xc002b3b, 0x0, 0x8fbf0020,
-0x3e00008, 0x27bd0028, 0x3c081000, 0x24070001,
-0x3c060080, 0x3c050100, 0x8f820070, 0x481024,
-0x1040fffd, 0x0, 0x8f820054, 0x24420005,
-0xaf820078, 0x8c040234, 0x10800016, 0x1821,
-0x3c020001, 0x571021, 0x8c4240e8, 0x24420005,
-0x3c010001, 0x370821, 0xac2240e8, 0x3c020001,
-0x571021, 0x8c4240e8, 0x44102b, 0x14400009,
-0x0, 0x3c030080, 0x3c010001, 0x370821,
-0xac2040e8, 0x3c010001, 0x370821, 0x1000000b,
-0xa02740f0, 0x3c020001, 0x571021, 0x904240f0,
-0x54400006, 0x661825, 0x3c020001, 0x571021,
-0x904240f1, 0x54400001, 0x661825, 0x8c040230,
-0x10800013, 0x0, 0x3c020001, 0x571021,
-0x8c4240ec, 0x24420005, 0x3c010001, 0x370821,
-0xac2240ec, 0x3c020001, 0x571021, 0x8c4240ec,
-0x44102b, 0x14400006, 0x0, 0x3c010001,
-0x370821, 0xac2040ec, 0x10000006, 0x651825,
-0x3c020001, 0x571021, 0x904240f2, 0x54400001,
-0x651825, 0x1060ffbc, 0x0, 0x8f420000,
-0x10400007, 0x0, 0xaf80004c, 0x8f82004c,
-0x1040fffd, 0x0, 0x10000005, 0x0,
-0xaf800048, 0x8f820048, 0x1040fffd, 0x0,
-0x8f820060, 0x431025, 0xaf820060, 0x8f420000,
-0x10400003, 0x0, 0x1000ffa7, 0xaf80004c,
-0x1000ffa5, 0xaf800048, 0x3e00008, 0x0,
-0x0, 0x0, 0x0, 0x27bdffe0,
-0xafbf0018, 0x8f860064, 0x30c20004, 0x10400025,
-0x24040004, 0x8c020114, 0xaf420020, 0xaf840064,
-0x8f4202fc, 0x24420001, 0xaf4202fc, 0x8f4202fc,
-0x8f820064, 0x30420004, 0x14400005, 0x0,
-0x8c030114, 0x8f420020, 0x1462fff2, 0x0,
-0x8f420000, 0x10400007, 0x8f43003c, 0xaf80004c,
-0x8f82004c, 0x1040fffd, 0x0, 0x10000005,
-0x0, 0xaf800048, 0x8f820048, 0x1040fffd,
-0x0, 0x8f820060, 0x431025, 0xaf820060,
-0x8f420000, 0x10400073, 0x0, 0x1000006f,
-0x0, 0x30c20008, 0x10400020, 0x24040008,
-0x8c02011c, 0xaf420048, 0xaf840064, 0x8f4202a8,
-0x24420001, 0xaf4202a8, 0x8f4202a8, 0x8f820064,
-0x30420008, 0x14400005, 0x0, 0x8c03011c,
-0x8f420048, 0x1462fff2, 0x0, 0x8f420000,
-0x10400007, 0x0, 0xaf80004c, 0x8f82004c,
-0x1040fffd, 0x0, 0x10000005, 0x0,
-0xaf800048, 0x8f820048, 0x1040fffd, 0x0,
-0x8f820060, 0x1000ffd9, 0x34420200, 0x30c20020,
-0x10400023, 0x24040020, 0x8c02012c, 0xaf420068,
-0xaf840064, 0x8f4202d8, 0x24420001, 0xaf4202d8,
-0x8f4202d8, 0x8f820064, 0x30420020, 0x14400005,
-0x32c24000, 0x8c03012c, 0x8f420068, 0x1462fff2,
-0x32c24000, 0x14400002, 0x3c020001, 0x2c2b025,
-0x8f420000, 0x10400007, 0x0, 0xaf80004c,
-0x8f82004c, 0x1040fffd, 0x0, 0x10000005,
-0x0, 0xaf800048, 0x8f820048, 0x1040fffd,
-0x0, 0x8f820060, 0x1000ffb4, 0x34420800,
-0x30c20010, 0x10400029, 0x24040010, 0x8c020124,
-0xaf420058, 0xaf840064, 0x8f4202d4, 0x24420001,
-0xaf4202d4, 0x8f4202d4, 0x8f820064, 0x30420010,
-0x14400005, 0x32c22000, 0x8c030124, 0x8f420058,
-0x1462fff2, 0x32c22000, 0x50400001, 0x36d68000,
-0x8f420000, 0x10400007, 0x0, 0xaf80004c,
-0x8f82004c, 0x1040fffd, 0x0, 0x10000005,
-0x0, 0xaf800048, 0x8f820048, 0x1040fffd,
-0x0, 0x8f820060, 0x34420100, 0xaf820060,
-0x8f420000, 0x10400003, 0x0, 0x1000006c,
-0xaf80004c, 0x1000006a, 0xaf800048, 0x30c20001,
-0x10400004, 0x24020001, 0xaf820064, 0x10000064,
-0x0, 0x30c20002, 0x1440000b, 0x3c050003,
-0x3c040001, 0x24846244, 0x34a50500, 0x3821,
-0xafa00010, 0xc002b3b, 0xafa00014, 0x2402ffc0,
-0x10000057, 0xaf820064, 0x8c05022c, 0x8c02010c,
-0x10a20048, 0x51080, 0x8c460300, 0x24a20001,
-0x3045003f, 0x24020003, 0xac05022c, 0x61e02,
-0x10620005, 0x24020010, 0x1062001d, 0x30c20fff,
-0x10000039, 0x0, 0x8f4302a8, 0x8f440000,
-0x30c20fff, 0xaf420048, 0x24630001, 0xaf4302a8,
-0x10800007, 0x8f4202a8, 0xaf80004c, 0x8f82004c,
-0x1040fffd, 0x0, 0x10000005, 0x0,
-0xaf800048, 0x8f820048, 0x1040fffd, 0x0,
-0x8f820060, 0x34420200, 0xaf820060, 0x8f420000,
-0x1040001f, 0x0, 0x1000001b, 0x0,
-0xaf420058, 0x32c22000, 0x50400001, 0x36d68000,
-0x8f4202d4, 0x8f430000, 0x24420001, 0xaf4202d4,
-0x10600007, 0x8f4202d4, 0xaf80004c, 0x8f82004c,
-0x1040fffd, 0x0, 0x10000005, 0x0,
-0xaf800048, 0x8f820048, 0x1040fffd, 0x0,
-0x8f820060, 0x34420100, 0xaf820060, 0x8f420000,
-0x10400003, 0x0, 0x10000006, 0xaf80004c,
-0x10000004, 0xaf800048, 0xc002196, 0xc02021,
-0x402821, 0x8c02010c, 0x14a20002, 0x24020002,
-0xaf820064, 0x8f820064, 0x30420002, 0x14400004,
-0x0, 0x8c02010c, 0x14a2ffac, 0x0,
-0x8fbf0018, 0x3e00008, 0x27bd0020, 0x3e00008,
-0x0, 0x27bdffa0, 0xafb00040, 0x808021,
-0x101602, 0x2442ffff, 0x304300ff, 0x2c620013,
-0xafbf0058, 0xafbe0054, 0xafb50050, 0xafb3004c,
-0xafb20048, 0xafb10044, 0x104001f3, 0xafa50034,
-0x31080, 0x3c010001, 0x220821, 0x8c226288,
-0x400008, 0x0, 0x101302, 0x30440fff,
-0x24020001, 0x10820005, 0x24020002, 0x1082000c,
-0x2402fffe, 0x10000024, 0x3c050003, 0x8f430004,
-0x3c020001, 0x8c426f04, 0xaf440200, 0xaf440204,
-0x3c040001, 0x8c846e80, 0x10000009, 0x34630001,
-0x8f430004, 0xaf440200, 0xaf440204, 0x3c040001,
-0x8c846e80, 0x621824, 0x3c020001, 0x2442ca28,
-0x21100, 0x21182, 0xaf430004, 0x3c030800,
-0x431025, 0xac820038, 0x8f840054, 0x41442,
-0x41c82, 0x431021, 0x41cc2, 0x431023,
-0x41d02, 0x431021, 0x41d42, 0x431023,
-0x10000009, 0xaf420208, 0x3c040001, 0x24846250,
-0x34a51000, 0x2003021, 0x3821, 0xafa00010,
-0xc002b3b, 0xafa00014, 0x8f4202a0, 0x24420001,
-0xaf4202a0, 0x1000021f, 0x8f4202a0, 0x27b00028,
-0x2002021, 0x24050210, 0xc002bbf, 0x24060008,
-0xc002518, 0x2002021, 0x10000216, 0x0,
-0x8faa0034, 0x27a40028, 0xa1880, 0x25420001,
-0x3042003f, 0xafa20034, 0x8c650300, 0x8faa0034,
-0x21080, 0x8c430300, 0x25420001, 0x3042003f,
-0xafa20034, 0xac02022c, 0xafa50028, 0xc002518,
-0xafa3002c, 0x10000203, 0x0, 0x27b00028,
-0x2002021, 0x24050210, 0xc002bbf, 0x24060008,
-0xc002657, 0x2002021, 0x100001fa, 0x0,
-0x8faa0034, 0x27a40028, 0xa1880, 0x25420001,
-0x3042003f, 0xafa20034, 0x8c650300, 0x8faa0034,
-0x21080, 0x8c430300, 0x25420001, 0x3042003f,
-0xafa20034, 0xac02022c, 0xafa50028, 0xc002657,
-0xafa3002c, 0x100001e7, 0x0, 0x101302,
-0x30430fff, 0x24020001, 0x10620005, 0x24020002,
-0x1062001e, 0x3c020002, 0x10000033, 0x3c050003,
-0x3c030002, 0x2c31024, 0x54400037, 0x2c3b025,
-0x8f820228, 0x3c010001, 0x370821, 0xac2238d8,
-0x8f82022c, 0x3c010001, 0x370821, 0xac2238dc,
-0x8f820230, 0x3c010001, 0x370821, 0xac2238e0,
-0x8f820234, 0x3c010001, 0x370821, 0xac2238e4,
-0x2402ffff, 0xaf820228, 0xaf82022c, 0xaf820230,
-0xaf820234, 0x10000020, 0x2c3b025, 0x2c21024,
-0x10400012, 0x3c02fffd, 0x3c020001, 0x571021,
-0x8c4238d8, 0xaf820228, 0x3c020001, 0x571021,
-0x8c4238dc, 0xaf82022c, 0x3c020001, 0x571021,
-0x8c4238e0, 0xaf820230, 0x3c020001, 0x571021,
-0x8c4238e4, 0xaf820234, 0x3c02fffd, 0x3442ffff,
-0x10000009, 0x2c2b024, 0x3c040001, 0x2484625c,
-0x34a51100, 0x2003021, 0x3821, 0xafa00010,
-0xc002b3b, 0xafa00014, 0x8f4202cc, 0x24420001,
-0xaf4202cc, 0x1000019f, 0x8f4202cc, 0x101302,
-0x30450fff, 0x24020001, 0x10a20005, 0x24020002,
-0x10a2000d, 0x3c0408ff, 0x10000014, 0x3c050003,
-0x3c0208ff, 0x3442ffff, 0x8f830220, 0x3c040004,
-0x2c4b025, 0x621824, 0x34630008, 0xaf830220,
-0x10000012, 0xaf450298, 0x3484fff7, 0x3c03fffb,
-0x8f820220, 0x3463ffff, 0x2c3b024, 0x441024,
-0xaf820220, 0x10000009, 0xaf450298, 0x3c040001,
-0x24846268, 0x34a51200, 0x2003021, 0x3821,
-0xafa00010, 0xc002b3b, 0xafa00014, 0x8f4202bc,
-0x24420001, 0xaf4202bc, 0x10000176, 0x8f4202bc,
-0x27840208, 0x24050200, 0xc002bbf, 0x24060008,
-0x27440224, 0x24050200, 0xc002bbf, 0x24060008,
-0x8f4202c4, 0x24420001, 0xaf4202c4, 0x10000169,
-0x8f4202c4, 0x101302, 0x30430fff, 0x24020001,
-0x10620011, 0x28620002, 0x50400005, 0x24020002,
-0x10600007, 0x0, 0x10000017, 0x0,
-0x1062000f, 0x0, 0x10000013, 0x0,
-0x8c060248, 0x2021, 0xc005104, 0x24050004,
-0x10000007, 0x0, 0x8c060248, 0x2021,
-0xc005104, 0x24050004, 0x10000010, 0x0,
-0x8c06024c, 0x2021, 0xc005104, 0x24050001,
-0x1000000a, 0x0, 0x3c040001, 0x24846274,
-0x3c050003, 0x34a51300, 0x2003021, 0x3821,
-0xafa00010, 0xc002b3b, 0xafa00014, 0x8f4202c0,
-0x24420001, 0xaf4202c0, 0x1000013a, 0x8f4202c0,
-0xc002426, 0x0, 0x10000136, 0x0,
-0x24020001, 0xa34205c5, 0x24100100, 0x8f4401a8,
-0x8f4501ac, 0xafb00010, 0xafa00014, 0x8f420014,
-0xafa20018, 0x8f420108, 0x26e60028, 0x40f809,
-0x24070400, 0x1040fff5, 0x0, 0x10000125,
-0x0, 0x3c03ffff, 0x34637fff, 0x8f420368,
-0x8f440360, 0x2c3b024, 0x1821, 0xaf400058,
-0xaf40005c, 0xaf400060, 0xaf400064, 0x441023,
-0xaf420368, 0x3c020900, 0xaf400360, 0xafa20020,
-0x8f5e0018, 0x27aa0020, 0x240200ff, 0x13c20002,
-0xafaa003c, 0x27c30001, 0x8c020228, 0x609021,
-0x1642000e, 0x1e38c0, 0x8f42033c, 0x24420001,
-0xaf42033c, 0x8f42033c, 0x8c020228, 0x3c040001,
-0x2484620c, 0x3c050009, 0xafa00014, 0xafa20010,
-0x8fa60020, 0x1000006b, 0x34a50500, 0xf71021,
-0x8fa30020, 0x8fa40024, 0xac4304c0, 0xac4404c4,
-0x8f830054, 0x8f820054, 0x247003e8, 0x2021023,
-0x2c4203e9, 0x1040001b, 0x9821, 0xe08821,
-0x263504c0, 0x8f440178, 0x8f45017c, 0x2201821,
-0x240a0004, 0xafaa0010, 0xafb20014, 0x8f48000c,
-0x1021, 0x2f53021, 0xafa80018, 0x8f48010c,
-0x24070008, 0xa32821, 0xa3482b, 0x822021,
-0x100f809, 0x892021, 0x54400006, 0x24130001,
-0x8f820054, 0x2021023, 0x2c4203e9, 0x1440ffe9,
-0x0, 0x326200ff, 0x54400017, 0xaf520018,
-0x8f420378, 0x24420001, 0xaf420378, 0x8f420378,
-0x8f820120, 0x8faa003c, 0xafa20010, 0x8f820124,
-0x3c040001, 0x24846218, 0x3c050009, 0xafa20014,
-0x8d460000, 0x10000033, 0x34a50600, 0x8f420308,
-0x24130001, 0x24420001, 0xaf420308, 0x8f420308,
-0x1000001c, 0x326200ff, 0x8f830054, 0x8f820054,
-0x247003e8, 0x2021023, 0x2c4203e9, 0x10400014,
-0x9821, 0x24110010, 0x8f42000c, 0x8f440160,
-0x8f450164, 0x8f860120, 0xafb10010, 0xafb20014,
-0xafa20018, 0x8f42010c, 0x24070008, 0x40f809,
-0x24c6001c, 0x1440ffe5, 0x0, 0x8f820054,
-0x2021023, 0x2c4203e9, 0x1440ffef, 0x0,
-0x326200ff, 0x14400011, 0x0, 0x8f420378,
-0x24420001, 0xaf420378, 0x8f420378, 0x8f820120,
-0x8faa003c, 0xafa20010, 0x8f820124, 0x3c040001,
-0x24846220, 0x3c050009, 0xafa20014, 0x8d460000,
-0x34a50700, 0xc002b3b, 0x3c03821, 0x8f4202b0,
-0x24420001, 0xaf4202b0, 0x8f4202b0, 0x8f4202f8,
-0x24420001, 0xaf4202f8, 0x1000008a, 0x8f4202f8,
-0x8c02025c, 0x27440224, 0xaf4201f0, 0x8c020260,
-0x24050200, 0x24060008, 0xc002bbf, 0xaf4201f8,
-0x8f820220, 0x30420008, 0x14400002, 0x24020001,
-0x24020002, 0xaf420298, 0x8f4202ac, 0x24420001,
-0xaf4202ac, 0x10000077, 0x8f4202ac, 0x3c0200ff,
-0x3442ffff, 0x2021824, 0x32c20180, 0x14400006,
-0x3402fffb, 0x43102b, 0x14400003, 0x0,
-0x1000006c, 0xaf4300bc, 0x3c040001, 0x24846280,
-0x3c050003, 0x34a51500, 0x2003021, 0x3821,
-0xafa00010, 0xc002b3b, 0xafa00014, 0x3c020700,
-0x34421000, 0x101e02, 0x621825, 0xafa30020,
-0x8f510018, 0x240200ff, 0x12220002, 0x8021,
-0x26300001, 0x8c020228, 0x1602000e, 0x1130c0,
-0x8f42033c, 0x24420001, 0xaf42033c, 0x8f42033c,
-0x8c020228, 0x3c040001, 0x248461f4, 0x3c050009,
-0xafa00014, 0xafa20010, 0x8fa60020, 0x1000003f,
-0x34a50100, 0xd71021, 0x8fa30020, 0x8fa40024,
-0xac4304c0, 0xac4404c4, 0xc01821, 0x8f440178,
-0x8f45017c, 0x1021, 0x24070004, 0xafa70010,
-0xafb00014, 0x8f48000c, 0x24c604c0, 0x2e63021,
-0xafa80018, 0x8f48010c, 0x24070008, 0xa32821,
-0xa3482b, 0x822021, 0x100f809, 0x892021,
-0x1440000b, 0x24070008, 0x8f820120, 0xafa20010,
-0x8f820124, 0x3c040001, 0x248461fc, 0x3c050009,
-0xafa20014, 0x8fa60020, 0x1000001c, 0x34a50200,
-0x8f440160, 0x8f450164, 0x8f43000c, 0xaf500018,
-0x8f860120, 0x24020010, 0xafa20010, 0xafb00014,
-0xafa30018, 0x8f42010c, 0x40f809, 0x24c6001c,
-0x14400010, 0x0, 0x8f420340, 0x24420001,
-0xaf420340, 0x8f420340, 0x8f820120, 0xafa20010,
-0x8f820124, 0x3c040001, 0x24846204, 0x3c050009,
-0xafa20014, 0x8fa60020, 0x34a50300, 0xc002b3b,
-0x2203821, 0x8f4202e0, 0x24420001, 0xaf4202e0,
-0x8f4202e0, 0x8f4202f0, 0x24420001, 0xaf4202f0,
-0x8f4202f0, 0x8fa20034, 0x8fbf0058, 0x8fbe0054,
-0x8fb50050, 0x8fb3004c, 0x8fb20048, 0x8fb10044,
-0x8fb00040, 0x3e00008, 0x27bd0060, 0x27bdfff8,
-0x2408ffff, 0x10a00014, 0x4821, 0x3c0aedb8,
-0x354a8320, 0x90870000, 0x24840001, 0x3021,
-0x1071026, 0x30420001, 0x10400002, 0x81842,
-0x6a1826, 0x604021, 0x24c60001, 0x2cc20008,
-0x1440fff7, 0x73842, 0x25290001, 0x125102b,
-0x1440fff0, 0x0, 0x1001021, 0x3e00008,
-0x27bd0008, 0x27bdffb0, 0xafbf0048, 0xafbe0044,
-0xafb50040, 0xafb3003c, 0xafb20038, 0xafb10034,
-0xafb00030, 0x8f870220, 0xafa70024, 0x8f870200,
-0xafa7002c, 0x8f820220, 0x3c0308ff, 0x3463ffff,
-0x431024, 0x34420004, 0xaf820220, 0x8f820200,
-0x3c03c0ff, 0x3463ffff, 0x431024, 0x34420004,
-0xaf820200, 0x8f530358, 0x8f55035c, 0x8f5e0360,
-0x8f470364, 0xafa70014, 0x8f470368, 0xafa7001c,
-0x8f4202d0, 0x274401c0, 0x24420001, 0xaf4202d0,
-0x8f5002d0, 0x8f510204, 0x8f520200, 0xc002ba8,
-0x24050400, 0xaf530358, 0xaf55035c, 0xaf5e0360,
-0x8fa70014, 0xaf470364, 0x8fa7001c, 0xaf470368,
-0xaf5002d0, 0xaf510204, 0xaf520200, 0x8c02025c,
-0x27440224, 0xaf4201f0, 0x8c020260, 0x24050200,
-0x24060008, 0xaf4201f8, 0x24020006, 0xc002bbf,
-0xaf4201f4, 0x3c023b9a, 0x3442ca00, 0xaf4201fc,
-0x240203e8, 0x24040002, 0x24030001, 0xaf420294,
-0xaf440290, 0xaf43029c, 0x8f820220, 0x30420008,
-0x10400004, 0x0, 0xaf430298, 0x10000003,
-0x3021, 0xaf440298, 0x3021, 0x3c030001,
-0x661821, 0x90636d00, 0x3461021, 0x24c60001,
-0xa043022c, 0x2cc2000f, 0x1440fff8, 0x3461821,
-0x24c60001, 0x8f820040, 0x24040080, 0x24050080,
-0x21702, 0x24420030, 0xa062022c, 0x3461021,
-0xc002ba8, 0xa040022c, 0x8fa70024, 0x30e20004,
-0x14400006, 0x0, 0x8f820220, 0x3c0308ff,
-0x3463fffb, 0x431024, 0xaf820220, 0x8fa7002c,
-0x30e20004, 0x14400006, 0x0, 0x8f820200,
-0x3c03c0ff, 0x3463fffb, 0x431024, 0xaf820200,
-0x8fbf0048, 0x8fbe0044, 0x8fb50040, 0x8fb3003c,
-0x8fb20038, 0x8fb10034, 0x8fb00030, 0x3e00008,
-0x27bd0050, 0x0, 0x0, 0xaf400104,
-0x24040001, 0x410c0, 0x2e21821, 0x24820001,
-0x3c010001, 0x230821, 0xa42234d0, 0x402021,
-0x2c820080, 0x1440fff8, 0x410c0, 0x24020001,
-0x3c010001, 0x370821, 0xa42038d0, 0xaf420100,
-0xaf800228, 0xaf80022c, 0xaf800230, 0xaf800234,
-0x3e00008, 0x0, 0x27bdffe8, 0xafbf0014,
-0xafb00010, 0x8f420104, 0x28420005, 0x10400026,
-0x808021, 0x3c020001, 0x8f430104, 0x344230d0,
-0x2e22021, 0x318c0, 0x621821, 0x2e31821,
-0x83102b, 0x10400015, 0x1021, 0x96070000,
-0x24840006, 0x24660006, 0x9482fffc, 0x14470009,
-0x2821, 0x9483fffe, 0x96020002, 0x14620006,
-0xa01021, 0x94820000, 0x96030004, 0x431026,
-0x2c450001, 0xa01021, 0x14400009, 0x24840008,
-0x86102b, 0x1440fff0, 0x1021, 0x304200ff,
-0x14400030, 0x24020001, 0x1000002e, 0x1021,
-0x1000fffa, 0x24020001, 0x2002021, 0xc00240c,
-0x24050006, 0x3042007f, 0x218c0, 0x2e31021,
-0x3c010001, 0x220821, 0x942230d0, 0x1040fff2,
-0x2e31021, 0x3c060001, 0xc23021, 0x94c630d0,
-0x10c0ffed, 0x3c080001, 0x350834d2, 0x96070000,
-0x610c0, 0x572021, 0x882021, 0x94820000,
-0x14470009, 0x2821, 0x94830002, 0x96020002,
-0x14620006, 0xa01021, 0x94820004, 0x96030004,
-0x431026, 0x2c450001, 0xa01021, 0x14400007,
-0x610c0, 0x2e21021, 0x3c060001, 0xc23021,
-0x94c634d0, 0x14c0ffeb, 0x610c0, 0x10c0ffd2,
-0x24020001, 0x8fbf0014, 0x8fb00010, 0x3e00008,
-0x27bd0018, 0x3e00008, 0x0, 0x27bdffb0,
-0x801021, 0xafb00030, 0x24500002, 0x2002021,
-0x24050006, 0xafb10034, 0x408821, 0xafbf0048,
-0xafbe0044, 0xafb50040, 0xafb3003c, 0xc00240c,
-0xafb20038, 0x3047007f, 0x710c0, 0x2e21021,
-0x3c050001, 0xa22821, 0x94a530d0, 0x50a0001c,
-0xa03021, 0x3c090001, 0x352934d2, 0x96280002,
-0x510c0, 0x572021, 0x892021, 0x94820000,
-0x14480009, 0x3021, 0x94830002, 0x96020002,
-0x14620006, 0xc01021, 0x94820004, 0x96030004,
-0x431026, 0x2c460001, 0xc01021, 0x14400007,
-0x510c0, 0x2e21021, 0x3c050001, 0xa22821,
-0x94a534d0, 0x14a0ffeb, 0x510c0, 0xa03021,
-0x10c00014, 0x610c0, 0x571821, 0x3c010001,
-0x230821, 0x8c2334d0, 0x571021, 0xafa30010,
-0x3c010001, 0x220821, 0x8c2234d4, 0x3c040001,
-0x24846394, 0xafa20014, 0x8e260000, 0x8e270004,
-0x3c050004, 0xc002b3b, 0x34a50400, 0x10000063,
-0x3c020800, 0x8f450100, 0x10a00006, 0x510c0,
-0x2e21021, 0x3c010001, 0x220821, 0x942234d0,
-0xaf420100, 0xa03021, 0x14c00011, 0x628c0,
-0x710c0, 0x2e21021, 0xafa70010, 0x3c010001,
-0x220821, 0x942230d0, 0x3c040001, 0x248463a0,
-0xafa20014, 0x8e260000, 0x8e270004, 0x3c050004,
-0xc002b3b, 0x34a50500, 0x10000048, 0x3c020800,
-0xb71821, 0x3c020001, 0x96040000, 0x344234d2,
-0x621821, 0xa4640000, 0x8e020002, 0x720c0,
-0xac620002, 0x2e41021, 0x3c030001, 0x621821,
-0x946330d0, 0x2e51021, 0x3c010001, 0x220821,
-0xa42334d0, 0x2e41021, 0x3c010001, 0x220821,
-0xa42630d0, 0x8f420104, 0x24420001, 0x28420080,
-0x1040000f, 0x3c020002, 0x8f420104, 0x3c040001,
-0x348430d2, 0x96030000, 0x210c0, 0x571021,
-0x441021, 0xa4430000, 0x8e030002, 0xac430002,
-0x8f420104, 0x24420001, 0xaf420104, 0x3c020002,
-0x2c21024, 0x10400011, 0x72142, 0x3c030001,
-0x346338d8, 0x24020003, 0x441023, 0x21080,
-0x572021, 0x832021, 0x571021, 0x431021,
-0x30e5001f, 0x8c430000, 0x24020001, 0xa21004,
-0x621825, 0x1000000c, 0xac830000, 0x24020003,
-0x441023, 0x21080, 0x5c2821, 0x5c1021,
-0x30e4001f, 0x8c430228, 0x24020001, 0x821004,
-0x621825, 0xaca30228, 0x3c020800, 0x34421000,
-0x1821, 0xafa20020, 0x8f5e0018, 0x27aa0020,
-0x240200ff, 0x13c20002, 0xafaa002c, 0x27c30001,
-0x8c020228, 0x609021, 0x1642000e, 0x1e38c0,
-0x8f42033c, 0x24420001, 0xaf42033c, 0x8f42033c,
-0x8c020228, 0x3c040001, 0x2484635c, 0x3c050009,
-0xafa00014, 0xafa20010, 0x8fa60020, 0x1000006b,
-0x34a50500, 0xf71021, 0x8fa30020, 0x8fa40024,
-0xac4304c0, 0xac4404c4, 0x8f830054, 0x8f820054,
-0x247003e8, 0x2021023, 0x2c4203e9, 0x1040001b,
-0x9821, 0xe08821, 0x263504c0, 0x8f440178,
-0x8f45017c, 0x2201821, 0x240a0004, 0xafaa0010,
-0xafb20014, 0x8f48000c, 0x1021, 0x2f53021,
-0xafa80018, 0x8f48010c, 0x24070008, 0xa32821,
-0xa3482b, 0x822021, 0x100f809, 0x892021,
-0x54400006, 0x24130001, 0x8f820054, 0x2021023,
-0x2c4203e9, 0x1440ffe9, 0x0, 0x326200ff,
-0x54400017, 0xaf520018, 0x8f420378, 0x24420001,
-0xaf420378, 0x8f420378, 0x8f820120, 0x8faa002c,
-0xafa20010, 0x8f820124, 0x3c040001, 0x24846368,
-0x3c050009, 0xafa20014, 0x8d460000, 0x10000033,
-0x34a50600, 0x8f420308, 0x24130001, 0x24420001,
-0xaf420308, 0x8f420308, 0x1000001c, 0x326200ff,
-0x8f830054, 0x8f820054, 0x247003e8, 0x2021023,
-0x2c4203e9, 0x10400014, 0x9821, 0x24110010,
-0x8f42000c, 0x8f440160, 0x8f450164, 0x8f860120,
-0xafb10010, 0xafb20014, 0xafa20018, 0x8f42010c,
-0x24070008, 0x40f809, 0x24c6001c, 0x1440ffe5,
-0x0, 0x8f820054, 0x2021023, 0x2c4203e9,
-0x1440ffef, 0x0, 0x326200ff, 0x14400011,
-0x0, 0x8f420378, 0x24420001, 0xaf420378,
-0x8f420378, 0x8f820120, 0x8faa002c, 0xafa20010,
-0x8f820124, 0x3c040001, 0x24846370, 0x3c050009,
-0xafa20014, 0x8d460000, 0x34a50700, 0xc002b3b,
-0x3c03821, 0x8f4202b4, 0x24420001, 0xaf4202b4,
-0x8f4202b4, 0x8f4202f4, 0x24420001, 0xaf4202f4,
-0x8f4202f4, 0x8fbf0048, 0x8fbe0044, 0x8fb50040,
-0x8fb3003c, 0x8fb20038, 0x8fb10034, 0x8fb00030,
-0x3e00008, 0x27bd0050, 0x27bdffa0, 0x801021,
-0xafb00040, 0x24500002, 0x2002021, 0x24050006,
-0xafb10044, 0x408821, 0xafbf0058, 0xafbe0054,
-0xafb50050, 0xafb3004c, 0xc00240c, 0xafb20048,
-0x3048007f, 0x810c0, 0x2e21021, 0x3c060001,
-0xc23021, 0x94c630d0, 0x10c0001c, 0x3821,
-0x3c0a0001, 0x354a34d2, 0x96290002, 0x610c0,
-0x572021, 0x8a2021, 0x94820000, 0x14490009,
-0x2821, 0x94830002, 0x96020002, 0x14620006,
-0xa01021, 0x94820004, 0x96030004, 0x431026,
-0x2c450001, 0xa01021, 0x14400008, 0x610c0,
-0xc03821, 0x2e21021, 0x3c060001, 0xc23021,
-0x94c634d0, 0x14c0ffea, 0x610c0, 0x14c00011,
-0xafa70028, 0x810c0, 0x2e21021, 0xafa80010,
-0x3c010001, 0x220821, 0x942230d0, 0x3c040001,
-0x248463ac, 0xafa20014, 0x8e260000, 0x8e270004,
-0x3c050004, 0xc002b3b, 0x34a50900, 0x10000075,
-0x3c020800, 0x10e0000c, 0x610c0, 0x2e21021,
-0x3c030001, 0x621821, 0x946334d0, 0x710c0,
-0x2e21021, 0x3c010001, 0x220821, 0xa42334d0,
-0x1000000b, 0x3c040001, 0x2e21021, 0x3c030001,
-0x621821, 0x946334d0, 0x810c0, 0x2e21021,
-0x3c010001, 0x220821, 0xa42330d0, 0x3c040001,
-0x348430d0, 0x8f430100, 0x610c0, 0x2e21021,
-0x3c010001, 0x220821, 0xa42334d0, 0x8f420104,
-0x2e43821, 0x2821, 0x18400029, 0xaf460100,
-0x24e60006, 0x94c3fffc, 0x96020000, 0x14620009,
-0x2021, 0x94c3fffe, 0x96020002, 0x14620006,
-0x801021, 0x94c20000, 0x96030004, 0x431026,
-0x2c440001, 0x801021, 0x50400014, 0x24a50001,
-0x8f420104, 0x2442ffff, 0xa2102a, 0x1040000b,
-0x24e40004, 0x94820006, 0x8c830008, 0xa482fffe,
-0xac830000, 0x8f420104, 0x24a50001, 0x2442ffff,
-0xa2102a, 0x1440fff7, 0x24840008, 0x8f420104,
-0x2442ffff, 0x10000006, 0xaf420104, 0x8f420104,
-0x24c60008, 0xa2102a, 0x1440ffda, 0x24e70008,
-0x810c0, 0x2e21021, 0x3c010001, 0x220821,
-0x942230d0, 0x14400023, 0x3c020800, 0x3c020002,
-0x2c21024, 0x10400012, 0x82142, 0x3c030001,
-0x346338d8, 0x24020003, 0x441023, 0x21080,
-0x572021, 0x832021, 0x571021, 0x431021,
-0x3105001f, 0x24030001, 0x8c420000, 0xa31804,
-0x31827, 0x431024, 0x1000000d, 0xac820000,
-0x24020003, 0x441023, 0x21080, 0x5c2821,
-0x5c1021, 0x3104001f, 0x24030001, 0x8c420228,
-0x831804, 0x31827, 0x431024, 0xaca20228,
-0x3c020800, 0x34422000, 0x1821, 0xafa20020,
-0x8f5e0018, 0x27ab0020, 0x240200ff, 0x13c20002,
-0xafab0034, 0x27c30001, 0x8c020228, 0x609021,
-0x1642000e, 0x1e38c0, 0x8f42033c, 0x24420001,
-0xaf42033c, 0x8f42033c, 0x8c020228, 0x3c040001,
-0x2484635c, 0x3c050009, 0xafa00014, 0xafa20010,
-0x8fa60020, 0x1000006b, 0x34a50500, 0xf71021,
-0x8fa30020, 0x8fa40024, 0xac4304c0, 0xac4404c4,
-0x8f830054, 0x8f820054, 0x247003e8, 0x2021023,
-0x2c4203e9, 0x1040001b, 0x9821, 0xe08821,
-0x263504c0, 0x8f440178, 0x8f45017c, 0x2201821,
-0x240b0004, 0xafab0010, 0xafb20014, 0x8f48000c,
-0x1021, 0x2f53021, 0xafa80018, 0x8f48010c,
-0x24070008, 0xa32821, 0xa3482b, 0x822021,
-0x100f809, 0x892021, 0x54400006, 0x24130001,
-0x8f820054, 0x2021023, 0x2c4203e9, 0x1440ffe9,
-0x0, 0x326200ff, 0x54400017, 0xaf520018,
-0x8f420378, 0x24420001, 0xaf420378, 0x8f420378,
-0x8f820120, 0x8fab0034, 0xafa20010, 0x8f820124,
-0x3c040001, 0x24846368, 0x3c050009, 0xafa20014,
-0x8d660000, 0x10000033, 0x34a50600, 0x8f420308,
-0x24130001, 0x24420001, 0xaf420308, 0x8f420308,
-0x1000001c, 0x326200ff, 0x8f830054, 0x8f820054,
-0x247003e8, 0x2021023, 0x2c4203e9, 0x10400014,
-0x9821, 0x24110010, 0x8f42000c, 0x8f440160,
-0x8f450164, 0x8f860120, 0xafb10010, 0xafb20014,
-0xafa20018, 0x8f42010c, 0x24070008, 0x40f809,
-0x24c6001c, 0x1440ffe5, 0x0, 0x8f820054,
-0x2021023, 0x2c4203e9, 0x1440ffef, 0x0,
-0x326200ff, 0x14400011, 0x0, 0x8f420378,
-0x24420001, 0xaf420378, 0x8f420378, 0x8f820120,
-0x8fab0034, 0xafa20010, 0x8f820124, 0x3c040001,
-0x24846370, 0x3c050009, 0xafa20014, 0x8d660000,
-0x34a50700, 0xc002b3b, 0x3c03821, 0x8f4202b8,
-0x24420001, 0xaf4202b8, 0x8f4202b8, 0x8f4202f4,
-0x24420001, 0xaf4202f4, 0x8f4202f4, 0x8fbf0058,
-0x8fbe0054, 0x8fb50050, 0x8fb3004c, 0x8fb20048,
-0x8fb10044, 0x8fb00040, 0x3e00008, 0x27bd0060,
-0x0, 0x0, 0x0, 0x27bdffe0,
-0x27644000, 0xafbf0018, 0xc002ba8, 0x24051000,
-0x3c030001, 0x34632cc0, 0x3c040001, 0x34842ec8,
-0x24020020, 0xaf82011c, 0x2e31021, 0xaf800100,
-0xaf800104, 0xaf800108, 0xaf800110, 0xaf800114,
-0xaf800118, 0xaf800120, 0xaf800124, 0xaf800128,
-0xaf800130, 0xaf800134, 0xaf800138, 0xaf4200ec,
-0x2e31021, 0xaf4200f0, 0x2e41021, 0xaf4200f4,
-0x2e41021, 0xaf4200f8, 0x3c020001, 0x571021,
-0x904240f4, 0x1440001c, 0x3c050001, 0x8f82011c,
-0x3c040001, 0x24846470, 0x3c050001, 0x34420001,
-0xaf82011c, 0xafa00010, 0xafa00014, 0x8f86011c,
-0x34a50100, 0xc002b3b, 0x3821, 0x8c020218,
-0x30420040, 0x10400014, 0x0, 0x8f82011c,
-0x3c040001, 0x2484647c, 0x3c050001, 0x34420004,
-0xaf82011c, 0xafa00010, 0xafa00014, 0x8f86011c,
-0x10000007, 0x34a50200, 0x3c040001, 0x24846484,
-0xafa00010, 0xafa00014, 0x8f86011c, 0x34a50300,
-0xc002b3b, 0x3821, 0x8fbf0018, 0x3e00008,
-0x27bd0020, 0x8fa90010, 0x8f83012c, 0x8faa0014,
-0x8fab0018, 0x1060000a, 0x27624fe0, 0x14620002,
-0x24680020, 0x27684800, 0x8f820128, 0x11020004,
-0x0, 0x8f820124, 0x15020007, 0x0,
-0x8f430334, 0x1021, 0x24630001, 0xaf430334,
-0x10000039, 0x8f430334, 0xac640000, 0xac650004,
-0xac660008, 0xa467000e, 0xac690018, 0xac6a001c,
-0xac6b0010, 0xac620014, 0xaf880120, 0x8f4200fc,
-0x8f4400f4, 0x2442ffff, 0xaf4200fc, 0x8c820000,
-0x10490005, 0x3042ff8f, 0x10400019, 0x3122ff8f,
-0x10400018, 0x3c020001, 0x8c830004, 0x2c620010,
-0x10400013, 0x3c020001, 0x24630001, 0xac830004,
-0x8f4300f8, 0x344230c8, 0x2e21021, 0x54620004,
-0x24620008, 0x3c020001, 0x34422ec8, 0x2e21021,
-0x14440015, 0x24020001, 0x8f820128, 0x24420020,
-0xaf820128, 0x8f820128, 0x1000000f, 0x24020001,
-0x3c020001, 0x344230c8, 0x2e21021, 0x54820004,
-0x24820008, 0x3c020001, 0x34422ec8, 0x2e21021,
-0x402021, 0x24020001, 0xaf4400f4, 0xac890000,
-0xac820004, 0x24020001, 0x3e00008, 0x0,
-0x3e00008, 0x0, 0x8fa90010, 0x8f83010c,
-0x8faa0014, 0x8fab0018, 0x1060000a, 0x276247e0,
-0x14620002, 0x24680020, 0x27684000, 0x8f820108,
-0x11020004, 0x0, 0x8f820104, 0x15020007,
-0x0, 0x8f430338, 0x1021, 0x24630001,
-0xaf430338, 0x10000035, 0x8f430338, 0xac640000,
-0xac650004, 0xac660008, 0xa467000e, 0xac690018,
-0xac6a001c, 0xac6b0010, 0xac620014, 0xaf880100,
-0x8f4400ec, 0x8c820000, 0x30420006, 0x10400019,
-0x31220006, 0x10400018, 0x3c020001, 0x8c830004,
-0x2c620010, 0x10400013, 0x3c020001, 0x24630001,
-0xac830004, 0x8f4300f0, 0x34422ec0, 0x2e21021,
-0x54620004, 0x24620008, 0x3c020001, 0x34422cc0,
-0x2e21021, 0x14440015, 0x24020001, 0x8f820108,
-0x24420020, 0xaf820108, 0x8f820108, 0x1000000f,
-0x24020001, 0x3c020001, 0x34422ec0, 0x2e21021,
-0x54820004, 0x24820008, 0x3c020001, 0x34422cc0,
-0x2e21021, 0x402021, 0x24020001, 0xaf4400ec,
-0xac890000, 0xac820004, 0x24020001, 0x3e00008,
-0x0, 0x3e00008, 0x0, 0x27bdffd8,
-0x3c040001, 0x2484648c, 0x3c050001, 0xafbf0024,
-0xafb20020, 0xafb1001c, 0xafb00018, 0x8f900104,
-0x8f9100b0, 0x8f92011c, 0x34a52500, 0x8f820100,
-0x2403021, 0x2203821, 0xafa20010, 0xc002b3b,
-0xafb00014, 0x8e020008, 0xafa20010, 0x8e02000c,
-0x3c040001, 0x24846498, 0xafa20014, 0x8e060000,
-0x8e070004, 0x3c050001, 0xc002b3b, 0x34a52510,
-0x8e020018, 0xafa20010, 0x8e02001c, 0x3c040001,
-0x248464a4, 0xafa20014, 0x8e060010, 0x8e070014,
-0x3c050001, 0xc002b3b, 0x34a52520, 0x3c027f00,
-0x2221024, 0x3c030800, 0x54430016, 0x3c030200,
-0x8f82009c, 0x3042ffff, 0x14400012, 0x3c030200,
-0x3c040001, 0x248464b0, 0x3c050002, 0x34a5f030,
-0x3021, 0x3821, 0x36420002, 0xaf82011c,
-0x36220001, 0xaf8200b0, 0xaf900104, 0xaf92011c,
-0xafa00010, 0xc002b3b, 0xafa00014, 0x10000024,
-0x0, 0x2c31024, 0x1040000d, 0x2231024,
-0x1040000b, 0x36420002, 0xaf82011c, 0x36220001,
-0xaf8200b0, 0xaf900104, 0xaf92011c, 0x8f420330,
-0x24420001, 0xaf420330, 0x10000015, 0x8f420330,
-0x3c040001, 0x248464b8, 0x240202a9, 0xafa20010,
-0xafa00014, 0x8f860144, 0x3c070001, 0x24e764c0,
-0xc002b3b, 0x3405dead, 0x8f82011c, 0x34420002,
-0xaf82011c, 0x8f820220, 0x34420004, 0xaf820220,
-0x8f820140, 0x3c030001, 0x431025, 0xaf820140,
-0x8fbf0024, 0x8fb20020, 0x8fb1001c, 0x8fb00018,
-0x3e00008, 0x27bd0028, 0x27bdffd8, 0x3c040001,
-0x248464e8, 0x3c050001, 0xafbf0024, 0xafb20020,
-0xafb1001c, 0xafb00018, 0x8f900124, 0x8f9100a0,
-0x8f92011c, 0x34a52600, 0x8f820120, 0x2403021,
-0x2203821, 0xafa20010, 0xc002b3b, 0xafb00014,
-0x8e020008, 0xafa20010, 0x8e02000c, 0x3c040001,
-0x248464f4, 0xafa20014, 0x8e060000, 0x8e070004,
-0x3c050001, 0xc002b3b, 0x34a52610, 0x8e020018,
-0xafa20010, 0x8e02001c, 0x3c040001, 0x24846500,
-0xafa20014, 0x8e060010, 0x8e070014, 0x3c050001,
-0xc002b3b, 0x34a52620, 0x3c027f00, 0x2221024,
-0x3c030800, 0x54430016, 0x3c030200, 0x8f8200ac,
-0x3042ffff, 0x14400012, 0x3c030200, 0x3c040001,
-0x2484650c, 0x3c050001, 0x34a5f030, 0x3021,
-0x3821, 0x36420002, 0xaf82011c, 0x36220001,
-0xaf8200a0, 0xaf900124, 0xaf92011c, 0xafa00010,
-0xc002b3b, 0xafa00014, 0x10000024, 0x0,
-0x2c31024, 0x1040000d, 0x2231024, 0x1040000b,
-0x36420002, 0xaf82011c, 0x36220001, 0xaf8200a0,
-0xaf900124, 0xaf92011c, 0x8f42032c, 0x24420001,
-0xaf42032c, 0x10000015, 0x8f42032c, 0x3c040001,
-0x248464b8, 0x240202e2, 0xafa20010, 0xafa00014,
-0x8f860144, 0x3c070001, 0x24e764c0, 0xc002b3b,
-0x3405dead, 0x8f82011c, 0x34420002, 0xaf82011c,
-0x8f820220, 0x34420004, 0xaf820220, 0x8f820140,
-0x3c030001, 0x431025, 0xaf820140, 0x8fbf0024,
-0x8fb20020, 0x8fb1001c, 0x8fb00018, 0x3e00008,
-0x27bd0028, 0x6021, 0x5021, 0x3021,
-0x2821, 0x6821, 0x4821, 0x7821,
-0x7021, 0x8f880124, 0x8f870104, 0x1580002e,
-0x8f8b011c, 0x11a00014, 0x31620800, 0x8f820120,
-0x10460029, 0x0, 0x3c040001, 0x8c846ee4,
-0x8cc20000, 0x8cc30004, 0xac820000, 0xac830004,
-0x8cc20008, 0xac820008, 0x94c2000e, 0xa482000e,
-0x8cc20010, 0x240c0001, 0xac820010, 0x8cc20014,
-0x10000012, 0x24c60020, 0x10400017, 0x0,
-0x3c040001, 0x8c846ee4, 0x8d020000, 0x8d030004,
-0xac820000, 0xac830004, 0x8d020008, 0xac820008,
-0x9502000e, 0xa482000e, 0x8d020010, 0x25060020,
-0xac820010, 0x8d020014, 0x240c0001, 0xc01821,
-0xac820014, 0x27624fe0, 0x43102b, 0x54400001,
-0x27634800, 0x603021, 0x1540002f, 0x31620100,
-0x11200014, 0x31628000, 0x8f820100, 0x1045002a,
-0x31620100, 0x3c040001, 0x8c846ee0, 0x8ca20000,
-0x8ca30004, 0xac820000, 0xac830004, 0x8ca20008,
-0xac820008, 0x94a2000e, 0xa482000e, 0x8ca20010,
-0x240a0001, 0xac820010, 0x8ca20014, 0x10000012,
-0x24a50020, 0x10400018, 0x31620100, 0x3c040001,
-0x8c846ee0, 0x8ce20000, 0x8ce30004, 0xac820000,
-0xac830004, 0x8ce20008, 0xac820008, 0x94e2000e,
-0xa482000e, 0x8ce20010, 0x24e50020, 0xac820010,
-0x8ce20014, 0x240a0001, 0xa01821, 0xac820014,
-0x276247e0, 0x43102b, 0x54400001, 0x27634000,
-0x602821, 0x31620100, 0x5440001d, 0x31621000,
-0x11a00009, 0x31a20800, 0x10400004, 0x25020020,
-0x8f8200a8, 0xa5e20000, 0x25020020, 0xaf820124,
-0x8f880124, 0x6821, 0x11800011, 0x31621000,
-0x3c040001, 0x8c846ee4, 0x8c820000, 0x8c830004,
-0xaf820080, 0xaf830084, 0x8c820008, 0xaf8200a4,
-0x9482000e, 0xaf8200ac, 0x8c820010, 0x6021,
-0xaf8200a0, 0x8c8d0010, 0x8c8f0014, 0x31621000,
-0x1440ff82, 0x0, 0x1120000f, 0x31220800,
-0x10400004, 0x3c020002, 0x8f8200b8, 0xa5c20000,
-0x3c020002, 0x1221024, 0x10400004, 0x24e20020,
-0x8f8200b4, 0xaf8200d4, 0x24e20020, 0xaf820104,
-0x8f870104, 0x4821, 0x1140ff70, 0x0,
-0x3c040001, 0x8c846ee0, 0x8c820000, 0x8c830004,
-0xaf820090, 0xaf830094, 0x8c820008, 0xaf8200b4,
-0x9482000e, 0xaf82009c, 0x8c820010, 0x5021,
-0xaf8200b0, 0x8c890010, 0x1000ff60, 0x8c8e0014,
-0x3e00008, 0x0, 0x6021, 0x5821,
-0x3021, 0x2821, 0x6821, 0x5021,
-0x7821, 0x7021, 0x8f880124, 0x8f870104,
-0x3c180100, 0x1580002e, 0x8f89011c, 0x11a00014,
-0x31220800, 0x8f820120, 0x10460029, 0x0,
-0x3c040001, 0x8c846ee4, 0x8cc20000, 0x8cc30004,
-0xac820000, 0xac830004, 0x8cc20008, 0xac820008,
-0x94c2000e, 0xa482000e, 0x8cc20010, 0x240c0001,
-0xac820010, 0x8cc20014, 0x10000012, 0x24c60020,
-0x10400017, 0x0, 0x3c040001, 0x8c846ee4,
-0x8d020000, 0x8d030004, 0xac820000, 0xac830004,
-0x8d020008, 0xac820008, 0x9502000e, 0xa482000e,
-0x8d020010, 0x25060020, 0xac820010, 0x8d020014,
-0x240c0001, 0xc01821, 0xac820014, 0x27624fe0,
-0x43102b, 0x54400001, 0x27634800, 0x603021,
-0x1560002f, 0x31220100, 0x11400014, 0x31228000,
-0x8f820100, 0x1045002a, 0x31220100, 0x3c040001,
-0x8c846ee0, 0x8ca20000, 0x8ca30004, 0xac820000,
-0xac830004, 0x8ca20008, 0xac820008, 0x94a2000e,
-0xa482000e, 0x8ca20010, 0x240b0001, 0xac820010,
-0x8ca20014, 0x10000012, 0x24a50020, 0x10400018,
-0x31220100, 0x3c040001, 0x8c846ee0, 0x8ce20000,
-0x8ce30004, 0xac820000, 0xac830004, 0x8ce20008,
-0xac820008, 0x94e2000e, 0xa482000e, 0x8ce20010,
-0x24e50020, 0xac820010, 0x8ce20014, 0x240b0001,
-0xa01821, 0xac820014, 0x276247e0, 0x43102b,
-0x54400001, 0x27634000, 0x602821, 0x31220100,
-0x5440001d, 0x31221000, 0x11a00009, 0x31a20800,
-0x10400004, 0x25020020, 0x8f8200a8, 0xa5e20000,
-0x25020020, 0xaf820124, 0x8f880124, 0x6821,
-0x11800011, 0x31221000, 0x3c040001, 0x8c846ee4,
-0x8c820000, 0x8c830004, 0xaf820080, 0xaf830084,
-0x8c820008, 0xaf8200a4, 0x9482000e, 0xaf8200ac,
-0x8c820010, 0x6021, 0xaf8200a0, 0x8c8d0010,
-0x8c8f0014, 0x31221000, 0x14400022, 0x0,
-0x1140000f, 0x31420800, 0x10400004, 0x3c020002,
-0x8f8200b8, 0xa5c20000, 0x3c020002, 0x1421024,
-0x10400004, 0x24e20020, 0x8f8200b4, 0xaf8200d4,
-0x24e20020, 0xaf820104, 0x8f870104, 0x5021,
-0x11600010, 0x0, 0x3c040001, 0x8c846ee0,
-0x8c820000, 0x8c830004, 0xaf820090, 0xaf830094,
-0x8c820008, 0xaf8200b4, 0x9482000e, 0xaf82009c,
-0x8c820010, 0x5821, 0xaf8200b0, 0x8c8a0010,
-0x8c8e0014, 0x8f820070, 0x3c031000, 0x431024,
-0x1040ff5c, 0x0, 0x8f820054, 0x24420005,
-0xaf820078, 0x8c040234, 0x10800016, 0x1821,
-0x3c020001, 0x571021, 0x8c4240e8, 0x24420005,
-0x3c010001, 0x370821, 0xac2240e8, 0x3c020001,
-0x571021, 0x8c4240e8, 0x44102b, 0x14400009,
-0x24020001, 0x3c030080, 0x3c010001, 0x370821,
-0xac2040e8, 0x3c010001, 0x370821, 0x1000000c,
-0xa02240f0, 0x3c020001, 0x571021, 0x904240f0,
-0x14400006, 0x3c020080, 0x3c020001, 0x571021,
-0x904240f1, 0x10400002, 0x3c020080, 0x621825,
-0x8c040230, 0x10800013, 0x0, 0x3c020001,
-0x571021, 0x8c4240ec, 0x24420005, 0x3c010001,
-0x370821, 0xac2240ec, 0x3c020001, 0x571021,
-0x8c4240ec, 0x44102b, 0x14400006, 0x0,
-0x3c010001, 0x370821, 0xac2040ec, 0x10000006,
-0x781825, 0x3c020001, 0x571021, 0x904240f2,
-0x54400001, 0x781825, 0x1060ff1a, 0x0,
-0x8f420000, 0x10400007, 0x0, 0xaf80004c,
-0x8f82004c, 0x1040fffd, 0x0, 0x10000005,
-0x0, 0xaf800048, 0x8f820048, 0x1040fffd,
-0x0, 0x8f820060, 0x431025, 0xaf820060,
-0x8f420000, 0x10400003, 0x0, 0x1000ff05,
-0xaf80004c, 0x1000ff03, 0xaf800048, 0x3e00008,
-0x0, 0x0, 0x0, 0x3c020001,
-0x8c426d28, 0x27bdffe8, 0xafbf0014, 0x14400012,
-0xafb00010, 0x3c100001, 0x26106f90, 0x2002021,
-0xc002ba8, 0x24052000, 0x26021fe0, 0x3c010001,
-0xac226eec, 0x3c010001, 0xac226ee8, 0xac020250,
-0x24022000, 0xac100254, 0xac020258, 0x24020001,
-0x3c010001, 0xac226d28, 0x8fbf0014, 0x8fb00010,
-0x3e00008, 0x27bd0018, 0x3c090001, 0x8d296eec,
-0x8c820000, 0x8fa30010, 0x8fa80014, 0xad220000,
-0x8c820004, 0xad250008, 0xad220004, 0x8f820054,
-0xad260010, 0xad270014, 0xad230018, 0xad28001c,
-0xad22000c, 0x2529ffe0, 0x3c020001, 0x24426f90,
-0x122102b, 0x10400003, 0x0, 0x3c090001,
-0x8d296ee8, 0x3c020001, 0x8c426d10, 0xad220000,
-0x3c020001, 0x8c426d10, 0x3c010001, 0xac296eec,
-0xad220004, 0xac090250, 0x3e00008, 0x0,
-0x27bdffd0, 0xafb00010, 0x3c100001, 0x8e106eec,
-0x3c020001, 0x8c426d10, 0xafb10014, 0x808821,
-0xafbe0024, 0x8fbe0040, 0x8fa40048, 0xafb20018,
-0xa09021, 0xafbf0028, 0xafb50020, 0xafb3001c,
-0xae020000, 0x3c020001, 0x8c426d10, 0xc09821,
-0xe0a821, 0x10800006, 0xae020004, 0x26050008,
-0xc002bb3, 0x24060018, 0x10000005, 0x2610ffe0,
-0x26040008, 0xc002ba8, 0x24050018, 0x2610ffe0,
-0x3c030001, 0x24636f90, 0x203102b, 0x10400003,
-0x0, 0x3c100001, 0x8e106ee8, 0x8e220000,
-0xae020000, 0x8e220004, 0xae120008, 0xae020004,
-0x8f820054, 0xae130010, 0xae150014, 0xae1e0018,
-0x8fa80044, 0xae08001c, 0xae02000c, 0x2610ffe0,
-0x203102b, 0x10400003, 0x0, 0x3c100001,
-0x8e106ee8, 0x3c020001, 0x8c426d10, 0xae020000,
-0x3c020001, 0x8c426d10, 0x3c010001, 0xac306eec,
-0xae020004, 0xac100250, 0x8fbf0028, 0x8fbe0024,
-0x8fb50020, 0x8fb3001c, 0x8fb20018, 0x8fb10014,
-0x8fb00010, 0x3e00008, 0x27bd0030, 0x851821,
-0x83102b, 0x10400006, 0x0, 0xac800000,
-0x24840004, 0x83102b, 0x5440fffd, 0xac800000,
-0x3e00008, 0x0, 0xa61821, 0xa3102b,
-0x10400007, 0x0, 0x8c820000, 0xaca20000,
-0x24a50004, 0xa3102b, 0x1440fffb, 0x24840004,
-0x3e00008, 0x0, 0x861821, 0x83102b,
-0x10400007, 0x0, 0x8ca20000, 0xac820000,
-0x24840004, 0x83102b, 0x1440fffb, 0x24a50004,
-0x3e00008, 0x0, 0x63080, 0x861821,
-0x83102b, 0x10400006, 0x0, 0xac850000,
-0x24840004, 0x83102b, 0x5440fffd, 0xac850000,
-0x3e00008, 0x0, 0x0, 0x26e50028,
-0xa03021, 0x274301c0, 0x8f4d0358, 0x8f47035c,
-0x8f480360, 0x8f490364, 0x8f4a0368, 0x8f4b0204,
-0x8f4c0200, 0x24640400, 0x64102b, 0x10400008,
-0x3c0208ff, 0x8cc20000, 0xac620000, 0x24630004,
-0x64102b, 0x1440fffb, 0x24c60004, 0x3c0208ff,
-0x3442ffff, 0x3c03c0ff, 0xaf4d0358, 0xaf47035c,
-0xaf480360, 0xaf490364, 0xaf4a0368, 0xaf4b0204,
-0xaf4c0200, 0x8f840220, 0x3463ffff, 0x8f860200,
-0x821024, 0x34420004, 0xc31824, 0x34630004,
-0xaf820220, 0xaf830200, 0x8ca20214, 0xac020084,
-0x8ca20218, 0xac020088, 0x8ca2021c, 0xac02008c,
-0x8ca20220, 0xac020090, 0x8ca20224, 0xac020094,
-0x8ca20228, 0xac020098, 0x8ca2022c, 0xac02009c,
-0x8ca20230, 0xac0200a0, 0x8ca20234, 0xac0200a4,
-0x8ca20238, 0xac0200a8, 0x8ca2023c, 0xac0200ac,
-0x8ca20240, 0xac0200b0, 0x8ca20244, 0xac0200b4,
-0x8ca20248, 0xac0200b8, 0x8ca2024c, 0xac0200bc,
-0x8ca2001c, 0xac020080, 0x8ca20018, 0xac0200c0,
-0x8ca20020, 0xac0200cc, 0x8ca20024, 0xac0200d0,
-0x8ca201d0, 0xac0200e0, 0x8ca201d4, 0xac0200e4,
-0x8ca201d8, 0xac0200e8, 0x8ca201dc, 0xac0200ec,
-0x8ca201e0, 0xac0200f0, 0x8ca20098, 0x8ca3009c,
-0xac0300fc, 0x8ca200a8, 0x8ca300ac, 0xac0300f4,
-0x8ca200a0, 0x8ca300a4, 0x30840004, 0xac0300f8,
-0x14800007, 0x30c20004, 0x8f820220, 0x3c0308ff,
-0x3463fffb, 0x431024, 0xaf820220, 0x30c20004,
-0x14400006, 0x0, 0x8f820200, 0x3c03c0ff,
-0x3463fffb, 0x431024, 0xaf820200, 0x8f4202dc,
-0xa34005c5, 0x24420001, 0xaf4202dc, 0x8f4202dc,
-0x3e00008, 0x0, 0x27bdffd8, 0xafbf0024,
-0xafb00020, 0x8f430024, 0x8f420020, 0x10620038,
-0x0, 0x8f430020, 0x8f420024, 0x622023,
-0x4810003, 0x0, 0x8f420040, 0x822021,
-0x8f430030, 0x8f420024, 0x43102b, 0x14400005,
-0x0, 0x8f430040, 0x8f420024, 0x10000005,
-0x621023, 0x8f420030, 0x8f430024, 0x431023,
-0x2442ffff, 0x406021, 0x8c102a, 0x54400001,
-0x806021, 0x8f4a0024, 0x8f490040, 0x8f480024,
-0x8f440180, 0x8f450184, 0x8f460024, 0x8f4b001c,
-0x24070001, 0xafa70010, 0x84100, 0x1001821,
-0x14c5021, 0x2529ffff, 0x1498024, 0xafb00014,
-0x8f470014, 0x1021, 0x63100, 0xafa70018,
-0xa32821, 0xa3382b, 0x822021, 0x872021,
-0x8f420108, 0x1663021, 0x40f809, 0xc3900,
-0x54400001, 0xaf500024, 0x8f430024, 0x8f420020,
-0x14620018, 0x0, 0x8f420000, 0x10400007,
-0x0, 0xaf80004c, 0x8f82004c, 0x1040fffd,
-0x0, 0x10000005, 0x0, 0xaf800048,
-0x8f820048, 0x1040fffd, 0x0, 0x8f820060,
-0x2403ffef, 0x431024, 0xaf820060, 0x8f420000,
-0x10400003, 0x0, 0x10000002, 0xaf80004c,
-0xaf800048, 0x8fbf0024, 0x8fb00020, 0x3e00008,
-0x27bd0028, 0x3e00008, 0x0, 0x27bdffc0,
-0x32c20020, 0xafbf0038, 0xafb30034, 0xafb20030,
-0xafb1002c, 0x10400004, 0xafb00028, 0x8f530028,
-0x10000002, 0x0, 0x8f530020, 0x8f420030,
-0x105300eb, 0x21100, 0x8f43001c, 0x628021,
-0x8e040000, 0x8e050004, 0x96120008, 0x8f420090,
-0x9611000a, 0x3246ffff, 0x46102a, 0x10400017,
-0x0, 0x8f8200d8, 0x8f430098, 0x431023,
-0x2442dcbe, 0xaf420090, 0x8f420090, 0x2842dcbf,
-0x10400005, 0x0, 0x8f420090, 0x8f430144,
-0x431021, 0xaf420090, 0x8f420090, 0x46102a,
-0x10400006, 0x0, 0x8f420348, 0x24420001,
-0xaf420348, 0x100000e1, 0x8f420348, 0x8f8200fc,
-0x14400006, 0x0, 0x8f420344, 0x24420001,
-0xaf420344, 0x100000d9, 0x8f420344, 0x934205c2,
-0x1040000b, 0x32c20008, 0x10400008, 0x32220200,
-0x10400006, 0x3c034000, 0x9602000e, 0xaf4300ac,
-0x21400, 0x10000002, 0xaf4200b0, 0xaf4000ac,
-0x32220004, 0x1040007f, 0x32220800, 0x10400003,
-0x3247ffff, 0x10000002, 0x24020020, 0x24020004,
-0xafa20010, 0x8f420030, 0xafa20014, 0x8f420010,
-0x3c030002, 0x431025, 0xafa20018, 0x8f460098,
-0x8f420108, 0x40f809, 0x0, 0x104000b7,
-0x0, 0x8f42009c, 0x8f430094, 0x2421021,
-0xaf42009c, 0xae03000c, 0x8f4200ac, 0x10400008,
-0x3c034000, 0x8f420094, 0x431025, 0xafa20020,
-0x8f42009c, 0x8f4300b0, 0x10000004, 0x431025,
-0x8f420094, 0xafa20020, 0x8f42009c, 0xafa20024,
-0x8f8200fc, 0x8fa30020, 0x8fa40024, 0xac430000,
-0xac440004, 0x24420008, 0xaf8200f0, 0x8f42009c,
-0x8f440270, 0x8f450274, 0x401821, 0x1021,
-0xa32821, 0xa3302b, 0x822021, 0x862021,
-0x32230060, 0x24020040, 0xaf440270, 0xaf450274,
-0x10620017, 0x2c620041, 0x10400005, 0x24020020,
-0x10620008, 0x24020001, 0x10000026, 0x0,
-0x24020060, 0x10620019, 0x24020001, 0x10000021,
-0x0, 0x8f420278, 0x8f43027c, 0x24630001,
-0x2c640001, 0x441021, 0xaf420278, 0xaf43027c,
-0x8f420278, 0x8f43027c, 0x10000016, 0x24020001,
-0x8f420280, 0x8f430284, 0x24630001, 0x2c640001,
-0x441021, 0xaf420280, 0xaf430284, 0x8f420280,
-0x8f430284, 0x1000000b, 0x24020001, 0x8f420288,
-0x8f43028c, 0x24630001, 0x2c640001, 0x441021,
-0xaf420288, 0xaf43028c, 0x8f420288, 0x8f43028c,
-0x24020001, 0xa34205c2, 0x8f420098, 0x3244ffff,
-0x2406fff8, 0x8f45013c, 0x441021, 0x24420007,
-0x461024, 0x24840007, 0xaf420094, 0x8f420090,
-0x8f430094, 0x862024, 0x441023, 0x65182b,
-0x14600005, 0xaf420090, 0x8f420094, 0x8f430144,
-0x431023, 0xaf420094, 0x8f420094, 0x10000023,
-0xaf40009c, 0x3247ffff, 0x50e00022, 0x32c20020,
-0x14400002, 0x24020010, 0x24020002, 0xafa20010,
-0x8f420030, 0xafa20014, 0x8f420010, 0xafa20018,
-0x8f460098, 0x8f420108, 0x40f809, 0x0,
-0x1040003a, 0x3245ffff, 0x8f420098, 0x8f430090,
-0x8f46013c, 0x451021, 0xaf420098, 0x8f42009c,
-0x8f440098, 0xa34005c2, 0x651823, 0xaf430090,
-0x451021, 0x86202b, 0x14800005, 0xaf42009c,
-0x8f420098, 0x8f430144, 0x431023, 0xaf420098,
-0x32c20020, 0x10400005, 0x0, 0x8f420358,
-0x2442ffff, 0xaf420358, 0x8f420358, 0x8f420030,
-0x8f430040, 0x24420001, 0x2463ffff, 0x431024,
-0xaf420030, 0x8f420030, 0x14530018, 0x0,
-0x8f420000, 0x10400007, 0x0, 0xaf80004c,
-0x8f82004c, 0x1040fffd, 0x0, 0x10000005,
-0x0, 0xaf800048, 0x8f820048, 0x1040fffd,
-0x0, 0x8f820060, 0x2403fff7, 0x431024,
-0xaf820060, 0x8f420000, 0x10400003, 0x0,
-0x10000002, 0xaf80004c, 0xaf800048, 0x8fbf0038,
-0x8fb30034, 0x8fb20030, 0x8fb1002c, 0x8fb00028,
-0x3e00008, 0x27bd0040, 0x3e00008, 0x0,
-0x27bdffd0, 0x32c20020, 0xafbf002c, 0xafb20028,
-0xafb10024, 0x10400004, 0xafb00020, 0x8f520028,
-0x10000002, 0x0, 0x8f520020, 0x8f420030,
-0x105200b5, 0x21100, 0x8f43001c, 0x628021,
-0x8e040000, 0x8e050004, 0x96110008, 0x8f420090,
-0x9607000a, 0x3226ffff, 0x46102a, 0x10400017,
-0x0, 0x8f8200d8, 0x8f430098, 0x431023,
-0x2442dc46, 0xaf420090, 0x8f420090, 0x2842dc47,
-0x10400005, 0x0, 0x8f420090, 0x8f430144,
-0x431021, 0xaf420090, 0x8f420090, 0x46102a,
-0x10400006, 0x0, 0x8f420348, 0x24420001,
-0xaf420348, 0x100000ab, 0x8f420348, 0x8f8600fc,
-0x10c0000c, 0x0, 0x8f8200f4, 0x2403fff8,
-0x431024, 0x461023, 0x218c3, 0x58600001,
-0x24630100, 0x8f42008c, 0x43102b, 0x14400006,
-0x712c2, 0x8f420344, 0x24420001, 0xaf420344,
-0x10000098, 0x8f420344, 0x934305c2, 0x1060000f,
-0x30460001, 0x8f420010, 0x34480400, 0x32c20008,
-0x10400008, 0x30e20200, 0x10400006, 0x3c034000,
-0x9602000e, 0xaf4300ac, 0x21400, 0x10000004,
-0xaf4200b0, 0x10000002, 0xaf4000ac, 0x8f480010,
-0x30e20004, 0x10400045, 0x3227ffff, 0x8f4900ac,
-0x11200005, 0x30c200ff, 0x14400006, 0x24020040,
-0x10000004, 0x24020008, 0x14400002, 0x24020020,
-0x24020004, 0xafa20010, 0x8f430030, 0x11200004,
-0xafa30014, 0x8f4200b0, 0x621025, 0xafa20014,
-0x3c020002, 0x1021025, 0xafa20018, 0x8f460098,
-0x8f420108, 0x40f809, 0x0, 0x10400069,
-0x3224ffff, 0x8f42008c, 0x8f430094, 0x24420001,
-0xaf42008c, 0x24020001, 0xae03000c, 0xa34205c2,
-0x8f420098, 0x2406fff8, 0x8f45013c, 0x441021,
-0x24420007, 0x461024, 0x24840007, 0xaf420094,
-0x8f420090, 0x8f430094, 0x862024, 0x441023,
-0x65182b, 0x14600005, 0xaf420090, 0x8f420094,
-0x8f430144, 0x431023, 0xaf420094, 0x8f430094,
-0x8f420140, 0x43102b, 0x10400009, 0x0,
-0x8f43013c, 0x8f440094, 0x8f420090, 0x8f450138,
-0x641823, 0x431023, 0xaf420090, 0xaf450094,
-0x8f420094, 0x1000001f, 0xaf420098, 0x10e0001d,
-0x30c200ff, 0x14400002, 0x24020010, 0x24020002,
-0xafa20010, 0x8f420030, 0xafa80018, 0xafa20014,
-0x8f460098, 0x8f420108, 0x40f809, 0x0,
-0x10400030, 0x3225ffff, 0x8f420098, 0x8f44013c,
-0x451021, 0xaf420098, 0x8f420090, 0x8f430098,
-0xa34005c2, 0x451023, 0x64182b, 0x14600005,
-0xaf420090, 0x8f420098, 0x8f430144, 0x431023,
-0xaf420098, 0x8f420030, 0x8f430040, 0x24420001,
-0x2463ffff, 0x431024, 0xaf420030, 0x8f420030,
-0x14520018, 0x0, 0x8f420000, 0x10400007,
-0x0, 0xaf80004c, 0x8f82004c, 0x1040fffd,
-0x0, 0x10000005, 0x0, 0xaf800048,
-0x8f820048, 0x1040fffd, 0x0, 0x8f820060,
-0x2403fff7, 0x431024, 0xaf820060, 0x8f420000,
-0x10400003, 0x0, 0x10000002, 0xaf80004c,
-0xaf800048, 0x8fbf002c, 0x8fb20028, 0x8fb10024,
-0x8fb00020, 0x3e00008, 0x27bd0030, 0x3e00008,
-0x0, 0x27bdffd8, 0x3c020001, 0x34422ec0,
-0xafbf0020, 0x8f4300f0, 0x8f840108, 0x2e21021,
-0x54620004, 0x24620008, 0x3c020001, 0x34422cc0,
-0x2e21021, 0x401821, 0xaf4300f0, 0xac600000,
-0x8f4200ec, 0x8c660004, 0x14620004, 0x3c020001,
-0x24820020, 0x1000000f, 0xaf820108, 0x8f4300f0,
-0x34422ec0, 0x2e21021, 0x54620004, 0x24620008,
-0x3c020001, 0x34422cc0, 0x2e21021, 0x401821,
-0x8c620004, 0x21140, 0x821021, 0xaf820108,
-0xac600000, 0x8c850018, 0x30a20036, 0x1040006c,
-0x30a20001, 0x8c82001c, 0x8f430040, 0x8f440034,
-0x24420001, 0x2463ffff, 0x431024, 0x862021,
-0xaf42002c, 0x30a20030, 0x14400006, 0xaf440034,
-0x8f420034, 0x8c03023c, 0x43102b, 0x144000b4,
-0x0, 0x32c20010, 0x10400028, 0x24070008,
-0x8f440170, 0x8f450174, 0x8f43002c, 0x8f48000c,
-0x8f860120, 0x24020080, 0xafa20010, 0xafa30014,
-0xafa80018, 0x8f42010c, 0x40f809, 0x24c6001c,
-0x14400011, 0x24020001, 0x3c010001, 0x370821,
-0xa02240f1, 0x8f820124, 0xafa20010, 0x8f820128,
-0x3c040001, 0x248467c4, 0xafa20014, 0x8f46002c,
-0x8f870120, 0x3c050009, 0xc002b3b, 0x34a51100,
-0x10000036, 0x0, 0x8f420300, 0x8f43002c,
-0x24420001, 0xaf420300, 0x8f420300, 0x24020001,
-0xa34205c1, 0x10000026, 0xaf430038, 0x8f440170,
-0x8f450174, 0x8f43002c, 0x8f48000c, 0x8f860120,
-0x24020020, 0xafa20010, 0xafa30014, 0xafa80018,
-0x8f42010c, 0x40f809, 0x24c6001c, 0x14400011,
-0x24020001, 0x3c010001, 0x370821, 0xa02240f0,
-0x8f820124, 0xafa20010, 0x8f820128, 0x3c040001,
-0x248467b8, 0xafa20014, 0x8f46002c, 0x8f870120,
-0x3c050009, 0xc002b3b, 0x34a50900, 0x1000000f,
-0x0, 0x8f420300, 0x24420001, 0xaf420300,
-0x8f420300, 0x8f42002c, 0xa34005c1, 0xaf420038,
-0x3c010001, 0x370821, 0xa02040f1, 0x3c010001,
-0x370821, 0xa02040f0, 0xaf400034, 0x8f420314,
-0x24420001, 0xaf420314, 0x10000059, 0x8f420314,
-0x10400022, 0x30a27000, 0x8c85001c, 0x8f420028,
-0xa22023, 0x4810003, 0x0, 0x8f420040,
-0x822021, 0x8f420358, 0x8f430000, 0xaf450028,
-0x441021, 0x10600007, 0xaf420358, 0xaf80004c,
-0x8f82004c, 0x1040fffd, 0x0, 0x10000005,
-0x0, 0xaf800048, 0x8f820048, 0x1040fffd,
-0x0, 0x8f820060, 0x34420008, 0xaf820060,
-0x8f420000, 0x10400003, 0x0, 0x10000038,
-0xaf80004c, 0x10000036, 0xaf800048, 0x1040002f,
-0x30a21000, 0x1040000c, 0x30a24000, 0x8c83001c,
-0x8f420050, 0x622023, 0x4820001, 0x24840200,
-0x8f42035c, 0x441021, 0xaf42035c, 0x8f420368,
-0x1000001a, 0xaf430050, 0x1040000c, 0x32c28000,
-0x8c83001c, 0x8f420070, 0x622023, 0x4820001,
-0x24840400, 0x8f420364, 0x441021, 0xaf420364,
-0x8f420368, 0x1000000d, 0xaf430070, 0x1040000e,
-0x3c020800, 0x8c83001c, 0x8f420060, 0x622023,
-0x4820001, 0x24840100, 0x8f420360, 0x441021,
-0xaf420360, 0x8f420368, 0xaf430060, 0x441021,
-0xaf420368, 0x3c020800, 0x2c21024, 0x50400008,
-0x36940040, 0x10000006, 0x0, 0x30a20100,
-0x10400003, 0x0, 0xc002bd8, 0x0,
-0x8fbf0020, 0x3e00008, 0x27bd0028, 0x3e00008,
-0x0, 0x27bdffa8, 0xafbf0050, 0xafbe004c,
-0xafb50048, 0xafb30044, 0xafb20040, 0xafb1003c,
-0xafb00038, 0x8f910108, 0x26220020, 0xaf820108,
-0x8e320018, 0xa821, 0x32420024, 0x104001ba,
-0xf021, 0x8e26001c, 0x8f43001c, 0x61100,
-0x621821, 0x8c70000c, 0x9604000c, 0x962d0016,
-0x9473000a, 0x2c8305dd, 0x38828870, 0x2c420001,
-0x621825, 0x10600015, 0x2821, 0x32c20040,
-0x10400015, 0x24020800, 0x96030014, 0x14620012,
-0x3402aaaa, 0x9603000e, 0x14620007, 0x2021,
-0x96030010, 0x24020300, 0x14620004, 0x801021,
-0x96020012, 0x2c440001, 0x801021, 0x54400006,
-0x24050016, 0x10000004, 0x0, 0x24020800,
-0x50820001, 0x2405000e, 0x934205c3, 0x14400008,
-0x5821, 0x240b0001, 0x32620180, 0xaf4500a8,
-0xaf5000a0, 0x10400002, 0xaf4600a4, 0xa34b05c3,
-0x10a00085, 0x2054021, 0x91020000, 0x3821,
-0x3042000f, 0x25080, 0x32c20002, 0x10400012,
-0x10a1821, 0x32620002, 0x10400010, 0x32c20001,
-0x1002021, 0x94820000, 0x24840002, 0xe23821,
-0x83102b, 0x1440fffb, 0x30e2ffff, 0x71c02,
-0x623821, 0x71c02, 0x30e2ffff, 0x623821,
-0x71027, 0xa502000a, 0x32c20001, 0x1040006a,
-0x32620001, 0x10400068, 0x0, 0x8f4200a8,
-0x10400065, 0x0, 0x8f4200a0, 0x8f4300a8,
-0x431021, 0x904c0009, 0x318900ff, 0x39230006,
-0x3182b, 0x39220011, 0x2102b, 0x621824,
-0x1060000c, 0x3c050006, 0x8f4200a4, 0x3c040001,
-0x248467d4, 0xafa20010, 0x8f4200a0, 0x34a54600,
-0x1203821, 0xc002b3b, 0xafa20014, 0x1000004e,
-0x0, 0x32c20004, 0x14400013, 0x2821,
-0x316200ff, 0x14400004, 0x0, 0x95020002,
-0x1000000d, 0x4a2823, 0x9505000c, 0x9502000e,
-0x95030010, 0xa22821, 0xa32821, 0x95030012,
-0x91040009, 0x95020002, 0xa32821, 0xa42821,
-0x4a1023, 0xa22821, 0x2002021, 0x94820000,
-0x24840002, 0xe23821, 0x88102b, 0x1440fffb,
-0x71c02, 0x30e2ffff, 0x623821, 0x71c02,
-0x30e2ffff, 0x623821, 0x1a52821, 0x51c02,
-0x30a2ffff, 0x622821, 0x51c02, 0x30a2ffff,
-0x622821, 0xa72823, 0x51402, 0xa22821,
-0x30a5ffff, 0x50a00001, 0x3405ffff, 0x316200ff,
-0x14400008, 0x318300ff, 0x8f4300a0, 0x8f4200a8,
-0x624021, 0x91020000, 0x3042000f, 0x25080,
-0x318300ff, 0x24020006, 0x14620003, 0x10a1021,
-0x10000002, 0x24440010, 0x24440006, 0x316200ff,
-0x14400006, 0x0, 0x94820000, 0xa22821,
-0x51c02, 0x30a2ffff, 0x622821, 0x934205c3,
-0x10400003, 0x32620100, 0x50400003, 0xa4850000,
-0x52827, 0xa4850000, 0x9622000e, 0x8f43009c,
-0x621821, 0x32a200ff, 0x10400007, 0xaf43009c,
-0x3c024000, 0x2021025, 0xafa20020, 0x8f42009c,
-0x10000003, 0x5e1025, 0xafb00020, 0x8f42009c,
-0xafa20024, 0x32620080, 0x10400010, 0x32620100,
-0x8f4200b4, 0x24430001, 0x210c0, 0x571021,
-0xaf4300b4, 0x8fa30020, 0x8fa40024, 0x3c010001,
-0x220821, 0xac2338e8, 0x3c010001, 0x220821,
-0xac2438ec, 0x100000a5, 0x32c20020, 0x10400064,
-0x0, 0x8f4200b4, 0x24430001, 0x210c0,
-0x571021, 0xaf4300b4, 0x8fa30020, 0x8fa40024,
-0x3c010001, 0x220821, 0xac2338e8, 0x3c010001,
-0x220821, 0xac2438ec, 0x8f4200b4, 0x10400051,
-0x3821, 0x3c090001, 0x352938e8, 0x3c08001f,
-0x3508ffff, 0x240bffff, 0x340affff, 0x710c0,
-0x571021, 0x491021, 0x8c430000, 0x8c440004,
-0xafa30028, 0xafa4002c, 0x8f8200fc, 0x8fa30028,
-0x8fa4002c, 0xac430000, 0xac440004, 0x24420008,
-0xaf8200f0, 0x8f42008c, 0x2442ffff, 0xaf42008c,
-0x97a2002e, 0x8f440270, 0x8f450274, 0x401821,
-0x1021, 0xa32821, 0xa3302b, 0x822021,
-0x862021, 0xaf440270, 0xaf450274, 0x8fa20028,
-0x481024, 0x90430000, 0x30630001, 0x1460000b,
-0x402021, 0x8f420278, 0x8f43027c, 0x24630001,
-0x2c640001, 0x441021, 0xaf420278, 0xaf43027c,
-0x8f420278, 0x1000001a, 0x8f43027c, 0x8c820000,
-0x144b000e, 0x0, 0x94820004, 0x144a000b,
-0x0, 0x8f420288, 0x8f43028c, 0x24630001,
-0x2c640001, 0x441021, 0xaf420288, 0xaf43028c,
-0x8f420288, 0x1000000a, 0x8f43028c, 0x8f420280,
-0x8f430284, 0x24630001, 0x2c640001, 0x441021,
-0xaf420280, 0xaf430284, 0x8f420280, 0x8f430284,
-0x8f4200b4, 0x24e70001, 0xe2102b, 0x1440ffb8,
-0x710c0, 0xa34005c3, 0x1000003f, 0xaf4000b4,
-0x8f8200fc, 0x8fa30020, 0x8fa40024, 0xac430000,
-0xac440004, 0x24420008, 0xaf8200f0, 0x8f42009c,
-0x8f46008c, 0x8f440270, 0x8f450274, 0x401821,
-0x1021, 0x24c6ffff, 0xaf46008c, 0xa32821,
-0xa3302b, 0x822021, 0x862021, 0xaf440270,
-0xaf450274, 0x92020000, 0x30420001, 0x1440000c,
-0x2402ffff, 0x8f420278, 0x8f43027c, 0x24630001,
-0x2c640001, 0x441021, 0xaf420278, 0xaf43027c,
-0x8f420278, 0x8f43027c, 0x1000001c, 0x32c20020,
-0x8e030000, 0x1462000f, 0x3402ffff, 0x96030004,
-0x1462000c, 0x0, 0x8f420288, 0x8f43028c,
-0x24630001, 0x2c640001, 0x441021, 0xaf420288,
-0xaf43028c, 0x8f420288, 0x8f43028c, 0x1000000b,
-0x32c20020, 0x8f420280, 0x8f430284, 0x24630001,
-0x2c640001, 0x441021, 0xaf420280, 0xaf430284,
-0x8f420280, 0x8f430284, 0x32c20020, 0x10400005,
-0xaf40009c, 0x8f420358, 0x2442ffff, 0xaf420358,
-0x8f420358, 0x8e22001c, 0x8f430040, 0x24420001,
-0x2463ffff, 0x431024, 0xaf42002c, 0x32420060,
-0x14400008, 0x32c20010, 0x8f420034, 0x24420001,
-0xaf420034, 0x8c03023c, 0x43102b, 0x14400102,
-0x32c20010, 0x10400018, 0x24070008, 0x8f440170,
-0x8f450174, 0x8f43002c, 0x8f48000c, 0x8f860120,
-0x24020080, 0xafa20010, 0xafa30014, 0xafa80018,
-0x8f42010c, 0x40f809, 0x24c6001c, 0x10400047,
-0x24020001, 0x8f420300, 0x8f43002c, 0x24420001,
-0xaf420300, 0x8f420300, 0x24020001, 0xa34205c1,
-0x1000007c, 0xaf430038, 0x8f440170, 0x8f450174,
-0x8f43002c, 0x8f48000c, 0x8f860120, 0x24020020,
-0xafa20010, 0xafa30014, 0xafa80018, 0x8f42010c,
-0x40f809, 0x24c6001c, 0x10400057, 0x24020001,
-0x10000065, 0x0, 0x32420012, 0x10400075,
-0x32420001, 0x9622000e, 0x8f43009c, 0x621821,
-0x32c20020, 0x10400005, 0xaf43009c, 0x8f420358,
-0x2442ffff, 0xaf420358, 0x8f420358, 0x8e22001c,
-0x8f430040, 0x24420001, 0x2463ffff, 0x431024,
-0xaf42002c, 0x32420010, 0x14400008, 0x32c20010,
-0x8f420034, 0x24420001, 0xaf420034, 0x8c03023c,
-0x43102b, 0x144000bc, 0x32c20010, 0x10400028,
-0x24070008, 0x8f440170, 0x8f450174, 0x8f43002c,
-0x8f48000c, 0x8f860120, 0x24020080, 0xafa20010,
-0xafa30014, 0xafa80018, 0x8f42010c, 0x40f809,
-0x24c6001c, 0x14400011, 0x24020001, 0x3c010001,
-0x370821, 0xa02240f1, 0x8f820124, 0xafa20010,
-0x8f820128, 0x3c040001, 0x248467c4, 0xafa20014,
-0x8f46002c, 0x8f870120, 0x3c050009, 0xc002b3b,
-0x34a51100, 0x10000036, 0x0, 0x8f420300,
-0x8f43002c, 0x24420001, 0xaf420300, 0x8f420300,
-0x24020001, 0xa34205c1, 0x10000026, 0xaf430038,
-0x8f440170, 0x8f450174, 0x8f43002c, 0x8f48000c,
-0x8f860120, 0x24020020, 0xafa20010, 0xafa30014,
-0xafa80018, 0x8f42010c, 0x40f809, 0x24c6001c,
-0x14400011, 0x24020001, 0x3c010001, 0x370821,
-0xa02240f0, 0x8f820124, 0xafa20010, 0x8f820128,
-0x3c040001, 0x248467b8, 0xafa20014, 0x8f46002c,
-0x8f870120, 0x3c050009, 0xc002b3b, 0x34a50900,
-0x1000000f, 0x0, 0x8f420300, 0x24420001,
-0xaf420300, 0x8f420300, 0x8f42002c, 0xa34005c1,
-0xaf420038, 0x3c010001, 0x370821, 0xa02040f1,
-0x3c010001, 0x370821, 0xa02040f0, 0xaf400034,
-0x8f420314, 0x24420001, 0xaf420314, 0x10000062,
-0x8f420314, 0x10400022, 0x32427000, 0x8e25001c,
-0x8f420028, 0xa22023, 0x4810003, 0x0,
-0x8f420040, 0x822021, 0x8f420358, 0x8f430000,
-0xaf450028, 0x441021, 0x10600007, 0xaf420358,
-0xaf80004c, 0x8f82004c, 0x1040fffd, 0x0,
-0x10000005, 0x0, 0xaf800048, 0x8f820048,
-0x1040fffd, 0x0, 0x8f820060, 0x34420008,
-0xaf820060, 0x8f420000, 0x10400003, 0x0,
-0x10000041, 0xaf80004c, 0x1000003f, 0xaf800048,
-0x1040002f, 0x32421000, 0x1040000c, 0x32424000,
-0x8e23001c, 0x8f420050, 0x622023, 0x4820001,
-0x24840200, 0x8f42035c, 0x441021, 0xaf42035c,
-0x8f420368, 0x1000001a, 0xaf430050, 0x1040000c,
-0x32c28000, 0x8e23001c, 0x8f420070, 0x622023,
-0x4820001, 0x24840400, 0x8f420364, 0x441021,
-0xaf420364, 0x8f420368, 0x1000000d, 0xaf430070,
-0x1040000e, 0x3c020800, 0x8e23001c, 0x8f420060,
-0x622023, 0x4820001, 0x24840100, 0x8f420360,
-0x441021, 0xaf420360, 0x8f420368, 0xaf430060,
-0x441021, 0xaf420368, 0x3c020800, 0x2c21024,
-0x50400011, 0x36940040, 0x1000000f, 0x0,
-0x32420048, 0x10400007, 0x24150001, 0x8e22001c,
-0x3c03ffff, 0x43f024, 0x3042ffff, 0x1000fd75,
-0xae22001c, 0x32420100, 0x10400003, 0x0,
-0xc002bd8, 0x0, 0x8fbf0050, 0x8fbe004c,
-0x8fb50048, 0x8fb30044, 0x8fb20040, 0x8fb1003c,
-0x8fb00038, 0x3e00008, 0x27bd0058, 0x3e00008,
-0x0, 0x0, 0x0, 0x8f8300e4,
-0x8f8200e0, 0x2404fff8, 0x441024, 0x621026,
-0x2102b, 0x21023, 0x3e00008, 0x621024,
-0x3e00008, 0x0, 0x27bdffe0, 0xafbf001c,
-0xafb00018, 0x8f8600c4, 0x8f8400e0, 0x8f8500e4,
-0x2402fff8, 0x821824, 0x10a30009, 0x27623ff8,
-0x14a20002, 0x24a20008, 0x27623000, 0x408021,
-0x16030005, 0x30820004, 0x10400004, 0xc02021,
-0x10000022, 0x1021, 0x8e040000, 0x8f42011c,
-0x14a20003, 0x0, 0x8f420120, 0xaf420114,
-0x8ca30000, 0x8f420148, 0x831823, 0x43102b,
-0x10400003, 0x0, 0x8f420148, 0x621821,
-0x94a20006, 0x24420050, 0x62102b, 0x1440000f,
-0xa01021, 0xafa40010, 0xafa30014, 0x8ca60000,
-0x8ca70004, 0x3c040001, 0xc002b3b, 0x24846894,
-0x8f42020c, 0x24420001, 0xaf42020c, 0x8f42020c,
-0x1021, 0xaf9000e8, 0xaf9000e4, 0x8fbf001c,
-0x8fb00018, 0x3e00008, 0x27bd0020, 0x3e00008,
-0x0, 0x8f8400e0, 0x8f8800c4, 0x8f8300e8,
-0x2402fff8, 0x823824, 0xe32023, 0x2c821000,
-0x50400001, 0x24841000, 0x420c2, 0x801821,
-0x8f440258, 0x8f45025c, 0x1021, 0xa32821,
-0xa3302b, 0x822021, 0x862021, 0xaf440258,
-0xaf45025c, 0x8f8300c8, 0x8f420148, 0x1032023,
-0x82102b, 0x14400004, 0x801821, 0x8f420148,
-0x822021, 0x801821, 0x8f440250, 0x8f450254,
-0x1021, 0xa32821, 0xa3302b, 0x822021,
-0x862021, 0xaf440250, 0xaf450254, 0xaf8800c8,
-0xaf8700e4, 0xaf8700e8, 0x3e00008, 0x0,
-0x27bdff30, 0x240a0001, 0xafbf00c8, 0xafbe00c4,
-0xafb500c0, 0xafb300bc, 0xafb200b8, 0xafb100b4,
-0xafb000b0, 0xa3a00097, 0xafa00044, 0xafaa005c,
-0x934205c4, 0xa7a0008e, 0x1040000a, 0xa7a00086,
-0x8f4b00c4, 0xafab0064, 0x8f4a00c0, 0xafaa006c,
-0x8f4b00cc, 0xafab0074, 0x8f4a00c8, 0x10000129,
-0xafaa007c, 0x8f420114, 0x40f809, 0x0,
-0x403021, 0x10c0034f, 0x0, 0x8cc20000,
-0x8cc30004, 0xafa20020, 0xafa30024, 0x8fab0024,
-0x8faa0020, 0x3162ffff, 0x2442fffc, 0xafa2006c,
-0x3c020006, 0x2c21024, 0xafab007c, 0x14400015,
-0xafaa0064, 0x91420000, 0x30420001, 0x10400011,
-0x2402ffff, 0x8d430000, 0x14620004, 0x3402ffff,
-0x95430004, 0x1062000b, 0x0, 0xc0024bb,
-0x8fa40064, 0x304200ff, 0x14400006, 0x0,
-0x8f420118, 0x40f809, 0x0, 0x1000032d,
-0x0, 0x8fa20024, 0x3c03ffbf, 0x3463ffff,
-0x431024, 0x3c03ffff, 0x431824, 0x14600003,
-0xafa20024, 0x10000040, 0x1821, 0x3c020080,
-0x621024, 0x10400007, 0x0, 0x8f42038c,
-0x24420001, 0xaf42038c, 0x8f42038c, 0x10000036,
-0x24030001, 0x8f420210, 0x24420001, 0xaf420210,
-0x8f420210, 0x3c020001, 0x621024, 0x10400006,
-0x3c020002, 0x8f4201c4, 0x24420001, 0xaf4201c4,
-0x8f4201c4, 0x3c020002, 0x621024, 0x10400006,
-0x3c020004, 0x8f42037c, 0x24420001, 0xaf42037c,
-0x8f42037c, 0x3c020004, 0x621024, 0x10400006,
-0x3c020008, 0x8f420380, 0x24420001, 0xaf420380,
-0x8f420380, 0x3c020008, 0x621024, 0x10400006,
-0x3c020010, 0x8f420384, 0x24420001, 0xaf420384,
-0x8f420384, 0x3c020010, 0x621024, 0x10400006,
-0x3c020020, 0x8f4201c0, 0x24420001, 0xaf4201c0,
-0x8f4201c0, 0x3c020020, 0x621024, 0x10400006,
-0x24030001, 0x8f420388, 0x24420001, 0xaf420388,
-0x8f420388, 0x24030001, 0x8c020260, 0x8fab006c,
-0x4b102b, 0x10400014, 0x307000ff, 0x8f4201e8,
-0x24420001, 0xaf4201e8, 0x8f4201e8, 0x8faa007c,
-0x8f8200e0, 0x354a0100, 0xafaa007c, 0xafa20010,
-0x8f8200e4, 0x24100001, 0x3c040001, 0x248468a0,
-0xafa20014, 0x8fa60020, 0x8fa70024, 0x3c050007,
-0xc002b3b, 0x34a50800, 0x12000010, 0x3c020080,
-0x2c21024, 0x1440000e, 0x32c20400, 0x8fab007c,
-0x3c020080, 0x34420100, 0x1621024, 0x10400005,
-0x0, 0x8f42020c, 0x24420001, 0xaf42020c,
-0x8f42020c, 0x100002b0, 0x8fa3006c, 0x32c20400,
-0x10400015, 0x34028100, 0x8faa0064, 0x9543000c,
-0x14620012, 0x3c020100, 0x240b0200, 0xa7ab008e,
-0x9542000e, 0x8d430008, 0x8d440004, 0x8d450000,
-0x8faa006c, 0x8fab0064, 0x254afffc, 0xafaa006c,
-0xa7a20086, 0xad63000c, 0xad640008, 0xad650004,
-0x256b0004, 0xafab0064, 0x3c020100, 0x2c21024,
-0x10400004, 0x0, 0x8faa006c, 0x254a0004,
-0xafaa006c, 0x8f4200bc, 0x5040000a, 0xafa00074,
-0x8fab006c, 0x4b102b, 0x50400006, 0xafa00074,
-0x8f4200bc, 0x1621023, 0xafa20074, 0x8f4a00bc,
-0xafaa006c, 0x8f420080, 0x8fab006c, 0x4b102b,
-0x10400056, 0x32c28000, 0x1040005e, 0x240a0003,
-0x32c21000, 0x1040005b, 0xafaa005c, 0x10000058,
-0x240b0004, 0x8f420350, 0x2403ffbf, 0x283a024,
-0x24420001, 0xaf420350, 0x1000024f, 0x8f420350,
-0x2c2b025, 0x2402ffbf, 0x282a024, 0x8f830128,
-0x3c040001, 0x248468d0, 0x26620001, 0xafa20014,
-0xafa30010, 0x8f860120, 0x8f870124, 0x3c050007,
-0xc002b3b, 0x34a52250, 0x1000023f, 0x0,
-0x2c2b025, 0x2402ffbf, 0x282a024, 0x8f830128,
-0x3c040001, 0x248468d0, 0x24020002, 0xafa20014,
-0xafa30010, 0x8f860120, 0x8f870124, 0x3c050007,
-0xc002b3b, 0x34a52450, 0x1000022f, 0x0,
-0x8ea20000, 0x8ea30004, 0x3c040001, 0x248468e8,
-0xafb00010, 0xafbe0014, 0x8ea70018, 0x34a52800,
-0xc002b3b, 0x603021, 0x10000223, 0x0,
-0xa6b1000a, 0x8f820124, 0x3c040001, 0x248468f0,
-0xafbe0014, 0xafa20010, 0x8f460044, 0x8f870120,
-0x3c050007, 0xc002b3b, 0x34a53000, 0x10000216,
-0x0, 0xa6b1000a, 0xa6b2000e, 0x8f820124,
-0x3c040001, 0x248468fc, 0xafbe0014, 0xafa20010,
-0x8f460044, 0x8f870120, 0x3c050007, 0xc002b3b,
-0x34a53200, 0x10000208, 0x0, 0x8f420084,
-0x8faa006c, 0x4a102b, 0x14400007, 0x3c020001,
-0x2c21024, 0x10400004, 0x0, 0x240b0002,
-0xafab005c, 0x8faa006c, 0x1140021b, 0x27ab0020,
-0xafab00a4, 0x3c0a001f, 0x354affff, 0xafaa009c,
-0x8fab005c, 0x240a0001, 0x556a0021, 0x240a0002,
-0x8f430054, 0x8f420050, 0x1062000b, 0x274b0054,
-0x8f5e0054, 0x3403ecc0, 0xafab004c, 0x27c20001,
-0x304201ff, 0xafa20054, 0x1e1140, 0x431021,
-0x1000006b, 0x2e2a821, 0x8f420044, 0x8faa006c,
-0x3c040001, 0x248468ac, 0xafaa0014, 0xafa20010,
-0x8f460054, 0x8f470050, 0x3c050007, 0xc002b3b,
-0x34a51300, 0x8f430350, 0x2402ffbf, 0x282a024,
-0x24630001, 0xaf430350, 0x100001d3, 0x8f420350,
-0x156a001d, 0x0, 0x8f430074, 0x8f420070,
-0x1062000a, 0x274b0074, 0x8f5e0074, 0xafab004c,
-0x27c20001, 0x304203ff, 0xafa20054, 0x1e1140,
-0x24426cc0, 0x1000004a, 0x2e2a821, 0x8f420044,
-0x8faa006c, 0x3c040001, 0x248468b8, 0x3c050007,
-0xafaa0014, 0xafa20010, 0x8f460074, 0x8f470070,
-0x34a51500, 0x240b0001, 0xc002b3b, 0xafab005c,
-0x1000ffc3, 0x0, 0x8f430064, 0x8f420060,
-0x1062001a, 0x274a0064, 0x8f5e0064, 0x8fab005c,
-0xafaa004c, 0x27c20001, 0x304200ff, 0xafa20054,
-0x24020004, 0x1562000e, 0x1e1140, 0x1e1180,
-0x24420cc0, 0x2e21021, 0xafa20044, 0x9442002a,
-0x8faa0044, 0x8fab006c, 0x4b102b, 0x10400024,
-0x25550020, 0x240a0001, 0x10000021, 0xa3aa0097,
-0x24424cc0, 0x1000001e, 0x2e2a821, 0x8f420044,
-0x8fab006c, 0x3c040001, 0x248468c4, 0xafab0014,
-0xafa20010, 0x8f460064, 0x8f470060, 0x3c050007,
-0xc002b3b, 0x34a51800, 0x3c020008, 0x2c21024,
-0x1440ff34, 0x0, 0x8f420370, 0x240a0001,
-0xafaa005c, 0x24420001, 0xaf420370, 0x1000ff90,
-0x8f420370, 0x27a30036, 0x131040, 0x621821,
-0x94620000, 0x441021, 0x10000020, 0xa4620000,
-0x8fab0064, 0xaeab0018, 0x93a20097, 0x10400072,
-0x9821, 0x8faa0044, 0x8fa4006c, 0x8fa300a4,
-0x25420020, 0xafa20028, 0x25420008, 0xafa20030,
-0x25420010, 0xafaa002c, 0xafa20034, 0x9542002a,
-0xa7a20038, 0x95420018, 0xa7a2003a, 0x9542001a,
-0xa7a2003c, 0x9542001c, 0xa7a2003e, 0x94620018,
-0x24630002, 0x822023, 0x1880ffde, 0x26730001,
-0x2e620004, 0x1440fff9, 0x0, 0x8f4200fc,
-0x26650001, 0xa2102a, 0x1440002b, 0x24030001,
-0x8f83012c, 0x10600023, 0x0, 0x8f820124,
-0x431023, 0x22143, 0x58800001, 0x24840040,
-0x8f820128, 0x431023, 0x21943, 0x58600001,
-0x24630040, 0x64102a, 0x54400001, 0x602021,
-0xaf4400fc, 0x8f4200fc, 0xa2102a, 0x10400011,
-0x24030001, 0x10000015, 0x306200ff, 0x8fab0064,
-0x96070018, 0xafab0010, 0x8e220008, 0x3c040001,
-0x248468dc, 0x8c430004, 0x8c420000, 0x34a52400,
-0x2403021, 0xc002b3b, 0xafa30014, 0x1000002b,
-0x0, 0x8f420334, 0x1821, 0x24420001,
-0xaf420334, 0x8f420334, 0x306200ff, 0x5040fedc,
-0x3c020800, 0x12600021, 0x9021, 0x8fb100a4,
-0x2208021, 0x8e220008, 0x96070018, 0x8fa60064,
-0x8c440000, 0x8c450004, 0x240a0001, 0xafaa0010,
-0xafbe0014, 0x8f420008, 0xafa20018, 0x8f42010c,
-0x40f809, 0x0, 0x1040ffd8, 0x3c050007,
-0x96020018, 0x8fab0064, 0x8faa009c, 0x1625821,
-0x14b102b, 0x10400004, 0xafab0064, 0x8f420148,
-0x1625823, 0xafab0064, 0x26100002, 0x26520001,
-0x253102b, 0x1440ffe3, 0x26310004, 0x8fb0006c,
-0x10000036, 0x97b10038, 0x8f4200fc, 0x24050002,
-0xa2102a, 0x1440001b, 0x24030001, 0x8f83012c,
-0x10600013, 0x0, 0x8f820124, 0x431023,
-0x22143, 0x58800001, 0x24840040, 0x8f820128,
-0x431023, 0x21943, 0x58600001, 0x24630040,
-0x64102a, 0x54400001, 0x602021, 0xaf4400fc,
-0x8f4200fc, 0xa2102a, 0x14400006, 0x24030001,
-0x8f420334, 0x1821, 0x24420001, 0xaf420334,
-0x8f420334, 0x306200ff, 0x1040fea5, 0x3c020800,
-0x96b1000a, 0x8fb0006c, 0x3223ffff, 0x70102b,
-0x54400001, 0x608021, 0x8ea40000, 0x8ea50004,
-0x240b0001, 0xafab0010, 0xafbe0014, 0x8f420008,
-0x8fa60064, 0xafa20018, 0x8f42010c, 0x40f809,
-0x2003821, 0x1040fea2, 0x3c050007, 0x96a3000e,
-0x97aa008e, 0x11400007, 0x609021, 0x934205c4,
-0x14400004, 0x0, 0x97ab0086, 0x6a1825,
-0xa6ab0016, 0x8faa007c, 0x3c02ffff, 0x1421024,
-0x10400003, 0xa1402, 0x34630400, 0xa6a20014,
-0x8fab006c, 0x560b0072, 0xa6a3000e, 0x34620004,
-0xa6a2000e, 0x8faa0074, 0x16a1021, 0xa6a2000a,
-0x8f430044, 0x8f4401a0, 0x8f4501a4, 0x34028000,
-0xafa20010, 0x8f420044, 0x2a03021, 0x24070020,
-0xafa20014, 0x8f42000c, 0x31940, 0x604821,
-0xafa20018, 0x8f42010c, 0x4021, 0xa92821,
-0xa9182b, 0x882021, 0x40f809, 0x832021,
-0x5040fe7f, 0xa6b2000e, 0x8f420368, 0xafa0006c,
-0xa34005c4, 0x2442ffff, 0xaf420368, 0x8fab005c,
-0x240a0001, 0x8f420368, 0x156a0006, 0x240a0002,
-0x8f42035c, 0x2442ffff, 0xaf42035c, 0x1000000c,
-0x8f42035c, 0x156a0006, 0x0, 0x8f420364,
-0x2442ffff, 0xaf420364, 0x10000005, 0x8f420364,
-0x8f420360, 0x2442ffff, 0xaf420360, 0x8f420360,
-0x8faa0054, 0x8fab004c, 0xad6a0000, 0x8f420044,
-0x8f440088, 0x8f430078, 0x24420001, 0x441024,
-0x24630001, 0xaf420044, 0xaf430078, 0x8c020240,
-0x62182b, 0x14600075, 0x24070008, 0x8f440168,
-0x8f45016c, 0x8f430044, 0x8f48000c, 0x8f860120,
-0x24020040, 0xafa20010, 0xafa30014, 0xafa80018,
-0x8f42010c, 0x40f809, 0x24c6001c, 0x14400011,
-0x240b0001, 0x3c010001, 0x370821, 0xa02b40f2,
-0x8f820124, 0xafa20010, 0x8f820128, 0x3c040001,
-0x2484688c, 0xafa20014, 0x8f460044, 0x8f870120,
-0x3c050009, 0xc002b3b, 0x34a51300, 0x1000000b,
-0x0, 0x8f420304, 0x24420001, 0xaf420304,
-0x8f420304, 0x8f420044, 0xaf42007c, 0x3c010001,
-0x370821, 0xa02040f2, 0xaf400078, 0x8f420318,
-0x24420001, 0xaf420318, 0x10000048, 0x8f420318,
-0xa6b0000a, 0x8f430044, 0x8f4401a0, 0x8f4501a4,
-0x34028000, 0xafa20010, 0x8f420044, 0x2a03021,
-0x24070020, 0xafa20014, 0x8f42000c, 0x31940,
-0x604821, 0xafa20018, 0x8f42010c, 0x4021,
-0xa92821, 0xa9182b, 0x882021, 0x40f809,
-0x832021, 0x1040fe1f, 0x240a0001, 0xa34a05c4,
-0x8fab006c, 0x8faa0064, 0x1705823, 0xafab006c,
-0x8fab009c, 0x1505021, 0x16a102b, 0x10400004,
-0xafaa0064, 0x8f420148, 0x1425023, 0xafaa0064,
-0x8f420368, 0x2442ffff, 0xaf420368, 0x8faa005c,
-0x240b0001, 0x8f420368, 0x154b0006, 0x240b0002,
-0x8f42035c, 0x2442ffff, 0xaf42035c, 0x1000000c,
-0x8f42035c, 0x114b0006, 0x0, 0x8f420360,
-0x2442ffff, 0xaf420360, 0x10000005, 0x8f420360,
-0x8f420364, 0x2442ffff, 0xaf420364, 0x8f420364,
-0x8fab0054, 0x8faa004c, 0xad4b0000, 0x8f420044,
-0x8f440088, 0x8f430078, 0x24420001, 0x441024,
-0x24630001, 0xaf420044, 0xaf430078, 0x8faa006c,
-0x1540fe0b, 0x0, 0x8fab006c, 0x1160001e,
-0x0, 0x934205c4, 0x10400009, 0x0,
-0x8faa0064, 0xaf4a00c4, 0xaf4b00c0, 0x8fab007c,
-0xaf4b00c8, 0x8faa0074, 0x1000000e, 0xaf4a00cc,
-0x97ab008e, 0x1160000b, 0x34038100, 0x8fa20020,
-0x8c46000c, 0xa443000c, 0x97aa0086, 0x8c440004,
-0x8c450008, 0xa44a000e, 0xac440000, 0xac450004,
-0xac460008, 0x8f42034c, 0x24420001, 0xaf42034c,
-0x10000010, 0x8f42034c, 0x8fab007c, 0x3164ffff,
-0x2484fffc, 0x801821, 0x8f440250, 0x8f450254,
-0x8f460118, 0x1021, 0xa32821, 0xa3382b,
-0x822021, 0x872021, 0xaf440250, 0xc0f809,
-0xaf450254, 0x8fbf00c8, 0x8fbe00c4, 0x8fb500c0,
-0x8fb300bc, 0x8fb200b8, 0x8fb100b4, 0x8fb000b0,
-0x3e00008, 0x27bd00d0, 0x3e00008, 0x0,
-0x27bdff38, 0x240b0001, 0xafbf00c0, 0xafbe00bc,
-0xafb500b8, 0xafb300b4, 0xafb200b0, 0xafb100ac,
-0xafb000a8, 0xa3a00087, 0xafa00044, 0xafab005c,
-0x934205c4, 0xa7a00076, 0x10400007, 0xa7a0007e,
-0x8f4c00c0, 0xafac0064, 0x8f4b00c8, 0x8f5e00c4,
-0x10000130, 0xafab006c, 0x8f420114, 0x40f809,
-0x0, 0x403021, 0x10c002a1, 0x0,
-0x8cc20000, 0x8cc30004, 0xafa20020, 0xafa30024,
-0x8fac0024, 0x8fbe0020, 0x3182ffff, 0x2442fffc,
-0xafa20064, 0x3c020006, 0x2c21024, 0x14400015,
-0xafac006c, 0x93c20000, 0x30420001, 0x10400011,
-0x2402ffff, 0x8fc30000, 0x14620004, 0x3402ffff,
-0x97c30004, 0x1062000b, 0x0, 0xc0024bb,
-0x3c02021, 0x304200ff, 0x14400006, 0x0,
-0x8f420118, 0x40f809, 0x0, 0x10000280,
-0x0, 0x8fa20024, 0x3c03ffbf, 0x3463ffff,
-0x431024, 0x3c03ffff, 0x431824, 0x14600003,
-0xafa20024, 0x10000040, 0x8021, 0x3c020080,
-0x621024, 0x10400007, 0x0, 0x8f42038c,
-0x24420001, 0xaf42038c, 0x8f42038c, 0x10000036,
-0x24100001, 0x8f420210, 0x24420001, 0xaf420210,
-0x8f420210, 0x3c020001, 0x621024, 0x10400006,
-0x3c020002, 0x8f4201c4, 0x24420001, 0xaf4201c4,
-0x8f4201c4, 0x3c020002, 0x621024, 0x10400006,
-0x3c020004, 0x8f42037c, 0x24420001, 0xaf42037c,
-0x8f42037c, 0x3c020004, 0x621024, 0x10400006,
-0x3c020008, 0x8f420380, 0x24420001, 0xaf420380,
-0x8f420380, 0x3c020008, 0x621024, 0x10400006,
-0x3c020010, 0x8f420384, 0x24420001, 0xaf420384,
-0x8f420384, 0x3c020010, 0x621024, 0x10400006,
-0x3c020020, 0x8f4201c0, 0x24420001, 0xaf4201c0,
-0x8f4201c0, 0x3c020020, 0x621024, 0x10400006,
-0x24100001, 0x8f420388, 0x24420001, 0xaf420388,
-0x8f420388, 0x24100001, 0x8c020260, 0x8fab0064,
-0x4b102b, 0x10400015, 0x320200ff, 0x8f4201e8,
-0x24420001, 0xaf4201e8, 0x8f4201e8, 0x8fac006c,
-0x8f8200e0, 0x358c0100, 0xafac006c, 0xafa20010,
-0x8f8200e4, 0x24100001, 0x3c040001, 0x248468a0,
-0xafa20014, 0x8fa60020, 0x8fa70024, 0x3c050007,
-0xc002b3b, 0x34a53600, 0x320200ff, 0x10400010,
-0x3c020080, 0x2c21024, 0x1440000e, 0x32c20400,
-0x8fab006c, 0x3c020080, 0x34420100, 0x1621024,
-0x10400005, 0x0, 0x8f42020c, 0x24420001,
-0xaf42020c, 0x8f42020c, 0x10000202, 0x8fa30064,
-0x32c20400, 0x10400012, 0x34028100, 0x97c3000c,
-0x1462000f, 0x0, 0x240c0200, 0xa7ac0076,
-0x97c2000e, 0x8fc30008, 0x8fc40004, 0x8fab0064,
-0x8fc50000, 0x256bfffc, 0xafab0064, 0xa7a2007e,
-0xafc3000c, 0xafc40008, 0xafc50004, 0x27de0004,
-0x8fa70064, 0x320200ff, 0x14400034, 0x3c020100,
-0x97c4000c, 0x2c8305dd, 0x38828870, 0x2c420001,
-0x621825, 0x10600015, 0x2821, 0x32c20800,
-0x10400015, 0x24020800, 0x97c30014, 0x14620012,
-0x3402aaaa, 0x97c3000e, 0x14620007, 0x2021,
-0x97c30010, 0x24020300, 0x14620004, 0x801021,
-0x97c20012, 0x2c440001, 0x801021, 0x54400006,
-0x24050016, 0x10000004, 0x0, 0x24020800,
-0x50820001, 0x2405000e, 0x10a00013, 0x3c52021,
-0x24830009, 0x3c02001f, 0x3442ffff, 0x43102b,
-0x10400003, 0x0, 0x8f420148, 0x621823,
-0x90620000, 0x38430006, 0x2c630001, 0x38420011,
-0x2c420001, 0x621825, 0x10600004, 0x3c020100,
-0x94820002, 0x453821, 0x3c020100, 0x2c21024,
-0x5040000e, 0xafa70064, 0x8fac0064, 0x10ec0008,
-0x3c050007, 0x3c040001, 0x24846908, 0x8fa60064,
-0x34a54000, 0xafa00010, 0xc002b3b, 0xafa00014,
-0x8fab0064, 0x256b0004, 0xafab0064, 0x8f420080,
-0x8fac0064, 0x4c102b, 0x1040002c, 0x32c28000,
-0x10400034, 0x240b0003, 0x32c21000, 0x10400031,
-0xafab005c, 0x1000002e, 0x240c0004, 0x8f420350,
-0x2403ffbf, 0x283a024, 0x24420001, 0xaf420350,
-0x10000173, 0x8f420350, 0x3c020800, 0x2c2b025,
-0x2402ffbf, 0x282a024, 0x8f830128, 0x3c040001,
-0x248468d0, 0x26620001, 0xafa20014, 0xafa30010,
-0x8f860120, 0x8f870124, 0x3c050007, 0xc002b3b,
-0x34a55300, 0x10000162, 0x0, 0x8ea20000,
-0x8ea30004, 0x3c040001, 0x248468e8, 0xafb00010,
-0xafb10014, 0x8ea70018, 0x34a55900, 0xc002b3b,
-0x603021, 0x10000156, 0x0, 0x8f420084,
-0x8fab0064, 0x4b102b, 0x14400007, 0x3c020001,
-0x2c21024, 0x10400004, 0x0, 0x240c0002,
-0xafac005c, 0x8fab0064, 0x11600166, 0x27ac0020,
-0xafac008c, 0x8fab005c, 0x240c0001, 0x556c0021,
-0x240c0002, 0x8f430054, 0x8f420050, 0x1062000b,
-0x274b0054, 0x8f510054, 0x3403ecc0, 0xafab004c,
-0x26220001, 0x304201ff, 0xafa20054, 0x111140,
-0x431021, 0x1000006b, 0x2e2a821, 0x8f420044,
-0x8fac0064, 0x3c040001, 0x248468ac, 0xafac0014,
-0xafa20010, 0x8f460054, 0x8f470050, 0x3c050007,
-0xc002b3b, 0x34a54300, 0x8f430350, 0x2402ffbf,
-0x282a024, 0x24630001, 0xaf430350, 0x10000124,
-0x8f420350, 0x156c001d, 0x0, 0x8f430074,
-0x8f420070, 0x1062000a, 0x274b0074, 0x8f510074,
-0xafab004c, 0x26220001, 0x304203ff, 0xafa20054,
-0x111140, 0x24426cc0, 0x1000004a, 0x2e2a821,
-0x8f420044, 0x8fac0064, 0x3c040001, 0x248468b8,
-0x3c050007, 0xafac0014, 0xafa20010, 0x8f460074,
-0x8f470070, 0x34a54500, 0x240b0001, 0xc002b3b,
-0xafab005c, 0x1000ffc3, 0x0, 0x8f430064,
-0x8f420060, 0x1062001a, 0x274c0064, 0x8f510064,
-0x8fab005c, 0xafac004c, 0x26220001, 0x304200ff,
-0xafa20054, 0x24020004, 0x1562000e, 0x111140,
-0x111180, 0x24420cc0, 0x2e21021, 0xafa20044,
-0x9442002a, 0x8fac0044, 0x8fab0064, 0x4b102b,
-0x10400024, 0x25950020, 0x240c0001, 0x10000021,
-0xa3ac0087, 0x24424cc0, 0x1000001e, 0x2e2a821,
-0x8f420044, 0x8fab0064, 0x3c040001, 0x248468c4,
-0xafab0014, 0xafa20010, 0x8f460064, 0x8f470060,
-0x3c050007, 0xc002b3b, 0x34a54800, 0x3c020008,
-0x2c21024, 0x1440ff61, 0x0, 0x8f420370,
-0x240c0001, 0xafac005c, 0x24420001, 0xaf420370,
-0x1000ff90, 0x8f420370, 0x27a30036, 0x131040,
-0x621821, 0x94620000, 0x441021, 0x1000001f,
-0xa4620000, 0xaebe0018, 0x93a20087, 0x10400084,
-0x9821, 0x8fab0044, 0x8fa40064, 0x8fa3008c,
-0x25620020, 0xafa20028, 0x25620008, 0xafa20030,
-0x25620010, 0xafab002c, 0xafa20034, 0x9562002a,
-0xa7a20038, 0x95620018, 0xa7a2003a, 0x9562001a,
-0xa7a2003c, 0x9562001c, 0xa7a2003e, 0x94620018,
-0x24630002, 0x822023, 0x1880ffdf, 0x26730001,
-0x2e620004, 0x1440fff9, 0x0, 0x8f4200fc,
-0x262102a, 0x14400030, 0x24030001, 0x8f83012c,
-0x10600028, 0x0, 0x8f820124, 0x431023,
-0x22143, 0x58800001, 0x24840040, 0x8f820128,
-0x431023, 0x21943, 0x58600001, 0x24630040,
-0x64102a, 0x54400001, 0x602021, 0xaf4400fc,
-0x8f4200fc, 0x262102a, 0x10400016, 0x24030001,
-0x1000001a, 0x306200ff, 0x8fac008c, 0x101040,
-0x4c1021, 0x94470018, 0x101080, 0x4c1021,
-0xafbe0010, 0x8c420008, 0x3c040001, 0x248468dc,
-0x3c050007, 0x8c430004, 0x8c420000, 0x34a55500,
-0x2003021, 0xc002b3b, 0xafa30014, 0x10000039,
-0x0, 0x8f420334, 0x1821, 0x24420001,
-0xaf420334, 0x8f420334, 0x306200ff, 0x1040ff06,
-0x8021, 0x8f430008, 0x2402fbff, 0x1260002d,
-0x625024, 0x3c0b4000, 0x22b4025, 0x8fb1008c,
-0x2669ffff, 0x2209021, 0x8e420008, 0x96270018,
-0x8c440000, 0x8c450004, 0x56090004, 0x240b0001,
-0x240c0002, 0x10000002, 0xafac0010, 0xafab0010,
-0x16000004, 0xafa80014, 0x8f420008, 0x10000002,
-0xafa20018, 0xafaa0018, 0x8f42010c, 0x3c03021,
-0xafa80098, 0xafa9009c, 0x40f809, 0xafaa00a0,
-0x8fa80098, 0x8fa9009c, 0x8faa00a0, 0x1040ffc2,
-0x3c02001f, 0x96230018, 0x3442ffff, 0x3c3f021,
-0x5e102b, 0x10400003, 0x26310002, 0x8f420148,
-0x3c2f023, 0x26100001, 0x213102b, 0x1440ffda,
-0x26520004, 0x8fb00064, 0x1000001a, 0x0,
-0x96a3000a, 0x8fb00064, 0x70102b, 0x54400001,
-0x608021, 0x8ea40000, 0x8ea50004, 0x8fab005c,
-0x240c0002, 0xafac0010, 0x934305c4, 0xb1700,
-0x10600003, 0x2223025, 0x3c020800, 0xc23025,
-0xafa60014, 0x8f420008, 0xafa20018, 0x8f42010c,
-0x3c03021, 0x40f809, 0x2003821, 0x1040fecb,
-0x3c050007, 0x97ac0076, 0x11800007, 0x96a3000e,
-0x934205c4, 0x14400004, 0x0, 0x97ab007e,
-0x6c1825, 0xa6ab0016, 0x8fac006c, 0x3c02ffff,
-0x1821024, 0x10400003, 0xc1402, 0x34630400,
-0xa6a20014, 0xa6b0000a, 0x8fab0064, 0x560b0006,
-0x3d0f021, 0x34620004, 0xafa00064, 0xa6a2000e,
-0x1000000d, 0xa34005c4, 0x8fac0064, 0x3c02001f,
-0x3442ffff, 0x5e102b, 0x1906023, 0xafac0064,
-0xa6a3000e, 0x240b0001, 0x10400003, 0xa34b05c4,
-0x8f420148, 0x3c2f023, 0x8fab0054, 0x8fac004c,
-0xad8b0000, 0x8fac0064, 0x1580feba, 0x0,
-0x8fab0064, 0x1160001b, 0x0, 0x934205c4,
-0x10400006, 0x0, 0xaf5e00c4, 0xaf4b00c0,
-0x8fac006c, 0x1000000e, 0xaf4c00c8, 0x97ab0076,
-0x1160000b, 0x34038100, 0x8fa20020, 0x8c46000c,
-0xa443000c, 0x97ac007e, 0x8c440004, 0x8c450008,
-0xa44c000e, 0xac440000, 0xac450004, 0xac460008,
-0x8f42034c, 0x24420001, 0xaf42034c, 0x10000010,
-0x8f42034c, 0x8fab006c, 0x3164ffff, 0x2484fffc,
-0x801821, 0x8f440250, 0x8f450254, 0x8f460118,
-0x1021, 0xa32821, 0xa3382b, 0x822021,
-0x872021, 0xaf440250, 0xc0f809, 0xaf450254,
-0x8fbf00c0, 0x8fbe00bc, 0x8fb500b8, 0x8fb300b4,
-0x8fb200b0, 0x8fb100ac, 0x8fb000a8, 0x3e00008,
-0x27bd00c8, 0x3e00008, 0x0, 0x27bdffd8,
-0xafbf0024, 0xafb00020, 0x8f43004c, 0x8f420048,
-0x10620034, 0x0, 0x8f430048, 0x8f42004c,
-0x622023, 0x4820001, 0x24840200, 0x8f430054,
-0x8f42004c, 0x43102b, 0x14400004, 0x24020200,
-0x8f43004c, 0x10000005, 0x431023, 0x8f420054,
-0x8f43004c, 0x431023, 0x2442ffff, 0x405021,
-0x8a102a, 0x54400001, 0x805021, 0x8f49004c,
-0x8f48004c, 0x8f440188, 0x8f45018c, 0x8f46004c,
-0x24071000, 0xafa70010, 0x84140, 0x1001821,
-0x12a4821, 0x313001ff, 0xafb00014, 0x8f470014,
-0x1021, 0x63140, 0xafa70018, 0xa32821,
-0xa3382b, 0x822021, 0x872021, 0x3402ecc0,
-0xc23021, 0x8f420108, 0x2e63021, 0x40f809,
-0xa3940, 0x54400001, 0xaf50004c, 0x8f43004c,
-0x8f420048, 0x14620018, 0x0, 0x8f420000,
-0x10400007, 0x0, 0xaf80004c, 0x8f82004c,
-0x1040fffd, 0x0, 0x10000005, 0x0,
-0xaf800048, 0x8f820048, 0x1040fffd, 0x0,
-0x8f820060, 0x2403fdff, 0x431024, 0xaf820060,
-0x8f420000, 0x10400003, 0x0, 0x10000002,
-0xaf80004c, 0xaf800048, 0x8fbf0024, 0x8fb00020,
-0x3e00008, 0x27bd0028, 0x3e00008, 0x0,
-0x27bdffd8, 0xafbf0024, 0xafb00020, 0x8f43005c,
-0x8f420058, 0x10620049, 0x0, 0x8f430058,
-0x8f42005c, 0x622023, 0x4820001, 0x24840100,
-0x8f430064, 0x8f42005c, 0x43102b, 0x14400004,
-0x24020100, 0x8f43005c, 0x10000005, 0x431023,
-0x8f420064, 0x8f43005c, 0x431023, 0x2442ffff,
-0x403821, 0x87102a, 0x54400001, 0x803821,
-0x8f42005c, 0x471021, 0x305000ff, 0x32c21000,
-0x10400015, 0x24082000, 0x8f49005c, 0x8f440190,
-0x8f450194, 0x8f46005c, 0x73980, 0xafa80010,
-0xafb00014, 0x8f480014, 0x94980, 0x1201821,
-0x1021, 0xa32821, 0xa3482b, 0x822021,
-0x892021, 0x63180, 0xafa80018, 0x8f420108,
-0x10000014, 0x24c60cc0, 0x8f49005c, 0x8f440190,
-0x8f450194, 0x8f46005c, 0x73940, 0xafa80010,
-0xafb00014, 0x8f480014, 0x94940, 0x1201821,
-0x1021, 0xa32821, 0xa3482b, 0x822021,
-0x892021, 0x63140, 0xafa80018, 0x8f420108,
-0x24c64cc0, 0x40f809, 0x2e63021, 0x54400001,
-0xaf50005c, 0x8f43005c, 0x8f420058, 0x14620018,
-0x0, 0x8f420000, 0x10400007, 0x0,
-0xaf80004c, 0x8f82004c, 0x1040fffd, 0x0,
-0x10000005, 0x0, 0xaf800048, 0x8f820048,
-0x1040fffd, 0x0, 0x8f820060, 0x2403feff,
-0x431024, 0xaf820060, 0x8f420000, 0x10400003,
-0x0, 0x10000002, 0xaf80004c, 0xaf800048,
-0x8fbf0024, 0x8fb00020, 0x3e00008, 0x27bd0028,
-0x3e00008, 0x0, 0x27bdffd8, 0xafbf0024,
-0xafb00020, 0x8f43006c, 0x8f420068, 0x10620033,
-0x0, 0x8f430068, 0x8f42006c, 0x622023,
-0x4820001, 0x24840400, 0x8f430074, 0x8f42006c,
-0x43102b, 0x14400004, 0x24020400, 0x8f43006c,
-0x10000005, 0x431023, 0x8f420074, 0x8f43006c,
-0x431023, 0x2442ffff, 0x405021, 0x8a102a,
-0x54400001, 0x805021, 0x8f49006c, 0x8f48006c,
-0x8f440198, 0x8f45019c, 0x8f46006c, 0x24074000,
-0xafa70010, 0x84140, 0x1001821, 0x12a4821,
-0x313003ff, 0xafb00014, 0x8f470014, 0x1021,
-0x63140, 0x24c66cc0, 0xafa70018, 0xa32821,
-0xa3382b, 0x822021, 0x872021, 0x8f420108,
-0x2e63021, 0x40f809, 0xa3940, 0x54400001,
-0xaf50006c, 0x8f43006c, 0x8f420068, 0x14620018,
-0x0, 0x8f420000, 0x10400007, 0x0,
-0xaf80004c, 0x8f82004c, 0x1040fffd, 0x0,
-0x10000005, 0x0, 0xaf800048, 0x8f820048,
-0x1040fffd, 0x0, 0x8f820060, 0x2403f7ff,
-0x431024, 0xaf820060, 0x8f420000, 0x10400003,
-0x0, 0x10000002, 0xaf80004c, 0xaf800048,
-0x8fbf0024, 0x8fb00020, 0x3e00008, 0x27bd0028,
-0x3e00008, 0x0, 0x8f4200fc, 0x3c030001,
-0x8f4400f8, 0x346330c8, 0x24420001, 0xaf4200fc,
-0x8f850128, 0x2e31021, 0x54820004, 0x24820008,
-0x3c020001, 0x34422ec8, 0x2e21021, 0x401821,
-0xaf4300f8, 0xac600000, 0x8f4200f4, 0x14620004,
-0x3c020001, 0x24a20020, 0x1000000f, 0xaf820128,
-0x8f4300f8, 0x344230c8, 0x2e21021, 0x54620004,
-0x24620008, 0x3c020001, 0x34422ec8, 0x2e21021,
-0x401821, 0x8c620004, 0x21140, 0xa21021,
-0xaf820128, 0xac600000, 0x8ca30018, 0x30620070,
-0x1040002d, 0x30620020, 0x10400004, 0x3c020010,
-0x2c21024, 0x1040000d, 0x0, 0x30620040,
-0x10400004, 0x3c020020, 0x2c21024, 0x10400007,
-0x0, 0x30620010, 0x1040001f, 0x3c020040,
-0x2c21024, 0x1440001c, 0x0, 0x8f820040,
-0x30420001, 0x14400008, 0x2021, 0x8c030104,
-0x24020001, 0x50620005, 0x24040001, 0x8c020264,
-0x10400003, 0x801021, 0x24040001, 0x801021,
-0x10400006, 0x0, 0x8f42030c, 0x24420001,
-0xaf42030c, 0x10000008, 0x8f42030c, 0x8f820044,
-0x34420004, 0xaf820044, 0x8f420308, 0x24420001,
-0xaf420308, 0x8f420308, 0x3e00008, 0x0,
-0x3e00008, 0x0, 0x27bdff98, 0xafbf0060,
-0xafbe005c, 0xafb50058, 0xafb30054, 0xafb20050,
-0xafb1004c, 0xafb00048, 0x8f4200fc, 0x24420001,
-0xaf4200fc, 0x8f880128, 0x25020020, 0xaf820128,
-0x8d030018, 0x30620070, 0x1040002e, 0x30620020,
-0x10400004, 0x3c020010, 0x2c21024, 0x1040000d,
-0x0, 0x30620040, 0x10400004, 0x3c020020,
-0x2c21024, 0x10400007, 0x0, 0x30620010,
-0x104001a9, 0x3c020040, 0x2c21024, 0x144001a6,
-0x0, 0x8f820040, 0x30420001, 0x14400008,
-0x2021, 0x8c030104, 0x24020001, 0x50620005,
-0x24040001, 0x8c020264, 0x10400003, 0x801021,
-0x24040001, 0x801021, 0x10400006, 0x0,
-0x8f42030c, 0x24420001, 0xaf42030c, 0x10000192,
-0x8f42030c, 0x8f820044, 0x34420004, 0xaf820044,
-0x8f420308, 0x24420001, 0xaf420308, 0x1000018a,
-0x8f420308, 0x30620002, 0x1040014b, 0x3c020800,
-0x8d1e001c, 0x1e5702, 0xafaa0034, 0x950a0016,
-0x3c22024, 0xafaa0024, 0x8faa0034, 0x24020001,
-0x15420006, 0x33deffff, 0x1e1140, 0x3403ecc0,
-0x431021, 0x10000010, 0x2e2a821, 0x24020002,
-0x15420005, 0x24020003, 0x1e1140, 0x24426cc0,
-0x10000009, 0x2e2a821, 0x15420005, 0x1e1180,
-0x1e1140, 0x24424cc0, 0x10000003, 0x2e2a821,
-0x571021, 0x24550ce0, 0x96a2000e, 0x304afffc,
-0x30420400, 0x10400003, 0xafaa002c, 0x100000e1,
-0x8821, 0x10800004, 0x8821, 0x97b10026,
-0x100000dd, 0xa6b10012, 0x8eb30018, 0x966a000c,
-0xa7aa003e, 0x97a5003e, 0x2ca305dd, 0x38a28870,
-0x2c420001, 0x621825, 0x10600015, 0x2021,
-0x32c20800, 0x10400015, 0x24020800, 0x96630014,
-0x14620012, 0x3402aaaa, 0x9663000e, 0x14620007,
-0x2821, 0x96630010, 0x24020300, 0x14620004,
-0xa01021, 0x96620012, 0x2c450001, 0xa01021,
-0x54400006, 0x24040016, 0x10000004, 0x0,
-0x24020800, 0x50a20001, 0x2404000e, 0x108000b9,
-0x2649021, 0x92420000, 0x3042000f, 0x28080,
-0x32c20100, 0x10400020, 0x2501821, 0x3c020020,
-0x43102b, 0x1440000e, 0x2402021, 0x2821,
-0x94820000, 0x24840002, 0xa22821, 0x83102b,
-0x1440fffb, 0x30a2ffff, 0x51c02, 0x622821,
-0x51c02, 0x30a2ffff, 0x10000009, 0x622821,
-0x8f470148, 0x8f420110, 0x102842, 0x3c060020,
-0x40f809, 0xafa80040, 0x3045ffff, 0x8fa80040,
-0x50a00001, 0x3405ffff, 0x8faa002c, 0x354a0002,
-0x10000002, 0xafaa002c, 0x2821, 0x32c20080,
-0x10400090, 0xa6a50010, 0x26430009, 0x3c02001f,
-0x3442ffff, 0x43102b, 0x10400003, 0x0,
-0x8f420148, 0x621823, 0x90660000, 0x30c200ff,
-0x38430006, 0x2c630001, 0x38420011, 0x2c420001,
-0x621825, 0x1060007f, 0x24020800, 0x8821,
-0x97a3003e, 0x1462000f, 0x2602021, 0x96710000,
-0x96620002, 0x96630004, 0x96640006, 0x2228821,
-0x2238821, 0x2248821, 0x96620008, 0x9663000a,
-0x9664000c, 0x2228821, 0x2238821, 0x10000007,
-0x2248821, 0x94820000, 0x24840002, 0x2228821,
-0x92102b, 0x1440fffb, 0x0, 0x111c02,
-0x3222ffff, 0x628821, 0x111c02, 0x3222ffff,
-0x628821, 0x32c20200, 0x10400003, 0x26440006,
-0x1000003e, 0x8021, 0x3c05001f, 0x34a5ffff,
-0xa4102b, 0x10400003, 0x0, 0x8f420148,
-0x822023, 0x94820000, 0x30421fff, 0x10400004,
-0x2644000c, 0x96420002, 0x10000030, 0x508023,
-0x96420002, 0x26430014, 0x508023, 0x3c020020,
-0x43102b, 0x1440000a, 0xd08021, 0x9642000c,
-0x2028021, 0x9642000e, 0x96430010, 0x96440012,
-0x2028021, 0x2038021, 0x10000020, 0x2048021,
-0xa4102b, 0x10400003, 0x0, 0x8f420148,
-0x822023, 0x94820000, 0x24840002, 0x2028021,
-0xa4102b, 0x10400003, 0x0, 0x8f420148,
-0x822023, 0x94820000, 0x24840002, 0x2028021,
-0xa4102b, 0x10400003, 0x0, 0x8f420148,
-0x822023, 0x94820000, 0x24840002, 0x2028021,
-0xa4102b, 0x10400003, 0x0, 0x8f420148,
-0x822023, 0x94820000, 0x2028021, 0x3c020100,
-0x2c21024, 0x1040000e, 0x0, 0x8faa002c,
-0x31420004, 0x1040000a, 0x0, 0x9504000e,
-0x2642021, 0xc003eec, 0x2484fffc, 0x3042ffff,
-0x2228821, 0x111c02, 0x3222ffff, 0x628821,
-0x8faa0024, 0x1518823, 0x111402, 0x2228821,
-0x2308821, 0x111402, 0x2228821, 0x3231ffff,
-0x52200001, 0x3411ffff, 0x8faa002c, 0x354a0001,
-0xafaa002c, 0xa6b10012, 0x97aa002e, 0xa6aa000e,
-0x8faa002c, 0x31420004, 0x10400002, 0x24091000,
-0x34098000, 0x8f480044, 0x8f4401a0, 0x8f4501a4,
-0xafa90010, 0x8f490044, 0x84140, 0x1001821,
-0xafa90014, 0x8f48000c, 0x2a03021, 0x24070020,
-0xafa80018, 0x8f48010c, 0x1021, 0xa32821,
-0xa3482b, 0x822021, 0x100f809, 0x892021,
-0x1440000b, 0x0, 0x8f820128, 0x3c040001,
-0x24846914, 0xafbe0014, 0xafa20010, 0x8f860124,
-0x8f870120, 0x3c050007, 0xc002b3b, 0x34a59920,
-0x8f420368, 0x2442ffff, 0xaf420368, 0x8f420044,
-0x8f430088, 0x24420001, 0x431024, 0xaf420044,
-0x8faa0034, 0x8f440368, 0x24020001, 0x15420006,
-0x24020002, 0x8f42035c, 0x2442ffff, 0xaf42035c,
-0x10000049, 0x8f42035c, 0x15420006, 0x0,
-0x8f420364, 0x2442ffff, 0xaf420364, 0x10000042,
-0x8f420364, 0x8f420360, 0x2442ffff, 0xaf420360,
-0x1000003d, 0x8f420360, 0x30621000, 0x10400005,
-0x30628000, 0x8f420078, 0x24420001, 0x10000036,
-0xaf420078, 0x10400034, 0x0, 0x8f420078,
-0x24420001, 0xaf420078, 0x8c030240, 0x43102b,
-0x1440002d, 0x24070008, 0x8f440168, 0x8f45016c,
-0x8f430044, 0x8f48000c, 0x8f860120, 0x24020040,
-0xafa20010, 0xafa30014, 0xafa80018, 0x8f42010c,
-0x40f809, 0x24c6001c, 0x14400011, 0x24020001,
-0x3c010001, 0x370821, 0xa02240f2, 0x8f820124,
-0xafa20010, 0x8f820128, 0x3c040001, 0x2484688c,
-0xafa20014, 0x8f460044, 0x8f870120, 0x3c050009,
-0xc002b3b, 0x34a51300, 0x1000000b, 0x0,
-0x8f420304, 0x24420001, 0xaf420304, 0x8f420304,
-0x8f420044, 0xaf42007c, 0x3c010001, 0x370821,
-0xa02040f2, 0xaf400078, 0x8f420318, 0x24420001,
-0xaf420318, 0x8f420318, 0x8fbf0060, 0x8fbe005c,
-0x8fb50058, 0x8fb30054, 0x8fb20050, 0x8fb1004c,
-0x8fb00048, 0x3e00008, 0x27bd0068, 0x3e00008,
-0x0, 0x0, 0x0, 0x8f42013c,
-0xaf8200c0, 0x8f42013c, 0xaf8200c4, 0x8f42013c,
-0xaf8200c8, 0x8f420138, 0xaf8200d0, 0x8f420138,
-0xaf8200d4, 0x8f420138, 0x3e00008, 0xaf8200d8,
-0x27bdffe0, 0x27840208, 0x24050200, 0xafbf0018,
-0xc002bbf, 0x24060008, 0x8c020204, 0xc004012,
-0xaf820210, 0x3c020001, 0x8c426d94, 0x30420002,
-0x1040000e, 0x2021, 0x8c060248, 0x24020002,
-0x3c010001, 0xac226d98, 0xc005104, 0x24050002,
-0x2021, 0x8c060248, 0x24020001, 0x3c010001,
-0xac226d98, 0x10000011, 0x24050001, 0x8c060248,
-0x24020004, 0x3c010001, 0xac226d98, 0xc005104,
-0x24050004, 0x3c020001, 0x8c426d94, 0x30420001,
-0x10400008, 0x24020001, 0x3c010001, 0xac226d98,
-0x2021, 0x24050001, 0x3c06601b, 0xc005104,
-0x0, 0x3c040001, 0x248469d0, 0x8f420150,
-0x8f430154, 0x3c050008, 0x8f460158, 0x21640,
-0x31940, 0x34630403, 0x431025, 0x633c0,
-0x461025, 0xaf82021c, 0xafa00010, 0xafa00014,
-0x8f86021c, 0x34a50200, 0xc002b3b, 0x3821,
-0x3c010001, 0xac206d90, 0x3c010001, 0xac206da8,
-0x8fbf0018, 0x3e00008, 0x27bd0020, 0x27bdffe0,
-0x3c050008, 0x34a50300, 0xafbf0018, 0xafa00010,
-0xafa00014, 0x8f860200, 0x3c040001, 0x248469dc,
-0xc002b3b, 0x3821, 0x8f420410, 0x24420001,
-0xaf420410, 0x8f420410, 0x8fbf0018, 0x3e00008,
-0x27bd0020, 0x27bdffd8, 0xafbf0020, 0xafb1001c,
-0xafb00018, 0x8f4203a4, 0x24420001, 0xaf4203a4,
-0x8f4203a4, 0x8f900220, 0x8f8200e0, 0xafa20010,
-0x8f8200e4, 0xafa20014, 0x8f8600c4, 0x8f8700c8,
-0x3c040001, 0x248469e8, 0xc002b3b, 0x2002821,
-0x3c044000, 0x2041024, 0x504000b4, 0x3c040100,
-0x8f4203bc, 0x24420001, 0xaf4203bc, 0x8f4203bc,
-0x8f8700c4, 0x8f8300c8, 0x8f420148, 0x671823,
-0x43102b, 0x10400003, 0x0, 0x8f420148,
-0x621821, 0x10600005, 0x0, 0x8f42014c,
-0x43102b, 0x1040000b, 0x0, 0x8f8200e0,
-0x8f430124, 0xaf42011c, 0xaf430114, 0x8f820220,
-0x3c0308ff, 0x3463fffb, 0x431024, 0x100000ce,
-0x441025, 0x8f820220, 0x3c0308ff, 0x3463ffff,
-0x431024, 0x34420004, 0xaf820220, 0x8f8200e0,
-0x8f430124, 0xaf42011c, 0xaf430114, 0x8f8600c8,
-0x8f840120, 0x8f830124, 0x10000005, 0x2821,
-0x14620002, 0x24620020, 0x27624800, 0x401821,
-0x1064000c, 0x30a200ff, 0x8c620018, 0x30420003,
-0x1040fff7, 0x27624fe0, 0x8f4203d0, 0x24050001,
-0x24420001, 0xaf4203d0, 0x8f4203d0, 0x8c660008,
-0x30a200ff, 0x14400058, 0x0, 0x934205c4,
-0x14400055, 0x0, 0x8f8700c4, 0x8f8800e0,
-0x8f8400e4, 0x2402fff8, 0x1024024, 0x1041023,
-0x218c3, 0x4620001, 0x24630200, 0x10600005,
-0x24020001, 0x10620009, 0x0, 0x1000001f,
-0x0, 0x8f4203c0, 0xe03021, 0x24420001,
-0xaf4203c0, 0x10000040, 0x8f4203c0, 0x8f4203c4,
-0x24420001, 0xaf4203c4, 0x8c860000, 0x8f420148,
-0x8f4303c4, 0xe61823, 0x43102b, 0x10400004,
-0x2c62233f, 0x8f420148, 0x621821, 0x2c62233f,
-0x14400031, 0x0, 0x8f42020c, 0x24420001,
-0xaf42020c, 0x8f42020c, 0xe03021, 0x24820008,
-0xaf8200e4, 0x10000028, 0xaf8200e8, 0x8f4203c8,
-0x24420001, 0xaf4203c8, 0x8f4203c8, 0x8c850000,
-0x8f420148, 0xa71823, 0x43102b, 0x10400003,
-0x0, 0x8f420148, 0x621821, 0x8f42014c,
-0x43102b, 0x5440000a, 0xa03021, 0x8f42020c,
-0x24420001, 0xaf42020c, 0x8f42020c, 0x24820008,
-0xaf8200e4, 0x8f8400e4, 0x1488ffec, 0xaf8400e8,
-0x1488000d, 0x27623000, 0x14820002, 0x2482fff8,
-0x27623ff8, 0x94430006, 0x3c02001f, 0x3442ffff,
-0xc33021, 0x46102b, 0x10400003, 0x0,
-0x8f420148, 0xc23023, 0xaf8600c8, 0x8f8300c4,
-0x8f420148, 0xc31823, 0x43102b, 0x10400003,
-0x0, 0x8f420148, 0x621821, 0x10600005,
-0x0, 0x8f42014c, 0x43102b, 0x50400008,
-0x3c02fdff, 0x8f820220, 0x3c0308ff, 0x3463fffb,
-0x431024, 0x3c034000, 0x1000003f, 0x431025,
-0x8f4303cc, 0x3442ffff, 0x282a024, 0x24630001,
-0xaf4303cc, 0x10000039, 0x8f4203cc, 0x2041024,
-0x1040000e, 0x3c110200, 0x8f4203a8, 0x24420001,
-0xaf4203a8, 0x8f4203a8, 0x8f820220, 0x3c0308ff,
-0x3463ffff, 0x431024, 0x441025, 0xc003daf,
-0xaf820220, 0x10000029, 0x0, 0x2111024,
-0x50400008, 0x3c110400, 0x8f4203ac, 0x24420001,
-0xaf4203ac, 0xc003daf, 0x8f4203ac, 0x10000019,
-0x0, 0x2111024, 0x1040001c, 0x0,
-0x8f830224, 0x24021402, 0x14620009, 0x3c050008,
-0x3c040001, 0x248469f4, 0xafa00010, 0xafa00014,
-0x8f860224, 0x34a50500, 0xc002b3b, 0x3821,
-0x8f4203b0, 0x24420001, 0xaf4203b0, 0x8f4203b0,
-0x8f820220, 0x2002021, 0x34420002, 0xc004e9c,
-0xaf820220, 0x8f820220, 0x3c0308ff, 0x3463ffff,
-0x431024, 0x511025, 0xaf820220, 0x8fbf0020,
-0x8fb1001c, 0x8fb00018, 0x3e00008, 0x27bd0028,
-0x3e00008, 0x0, 0x3c020001, 0x8c426da8,
-0x27bdffb0, 0xafbf0048, 0xafbe0044, 0xafb50040,
-0xafb3003c, 0xafb20038, 0xafb10034, 0x1040000f,
-0xafb00030, 0x3c040001, 0x24846a00, 0x3c050008,
-0xafa00010, 0xafa00014, 0x8f860220, 0x34a50600,
-0x24020001, 0x3c010001, 0xac206da8, 0x3c010001,
-0xac226d9c, 0xc002b3b, 0x3821, 0x3c037fff,
-0x8c020268, 0x3463ffff, 0x3c04fdff, 0x431024,
-0xac020268, 0x8f420004, 0x3484ffff, 0x30420002,
-0x10400092, 0x284a024, 0x3c040600, 0x34842000,
-0x8f420004, 0x2821, 0x2403fffd, 0x431024,
-0xaf420004, 0xafa40020, 0x8f5e0018, 0x27aa0020,
-0x240200ff, 0x13c20002, 0xafaa002c, 0x27c50001,
-0x8c020228, 0xa09021, 0x1642000e, 0x1e38c0,
-0x8f42033c, 0x24420001, 0xaf42033c, 0x8f42033c,
-0x8c020228, 0x3c040001, 0x24846998, 0x3c050009,
-0xafa00014, 0xafa20010, 0x8fa60020, 0x1000006d,
-0x34a50500, 0xf71021, 0x8fa30020, 0x8fa40024,
-0xac4304c0, 0xac4404c4, 0x8f830054, 0x8f820054,
-0x247003e8, 0x2021023, 0x2c4203e9, 0x1040001b,
-0x9821, 0xe08821, 0x263504c0, 0x8f440178,
-0x8f45017c, 0x2201821, 0x240a0004, 0xafaa0010,
-0xafb20014, 0x8f48000c, 0x1021, 0x2f53021,
-0xafa80018, 0x8f48010c, 0x24070008, 0xa32821,
-0xa3482b, 0x822021, 0x100f809, 0x892021,
-0x54400006, 0x24130001, 0x8f820054, 0x2021023,
-0x2c4203e9, 0x1440ffe9, 0x0, 0x326200ff,
-0x54400017, 0xaf520018, 0x8f420378, 0x24420001,
-0xaf420378, 0x8f420378, 0x8f820120, 0x8faa002c,
-0xafa20010, 0x8f820124, 0x3c040001, 0x248469a4,
-0x3c050009, 0xafa20014, 0x8d460000, 0x10000035,
-0x34a50600, 0x8f420308, 0x24130001, 0x24420001,
-0xaf420308, 0x8f420308, 0x1000001e, 0x326200ff,
-0x8f830054, 0x8f820054, 0x247003e8, 0x2021023,
-0x2c4203e9, 0x10400016, 0x9821, 0x3c150020,
-0x24110010, 0x8f42000c, 0x8f440160, 0x8f450164,
-0x8f860120, 0xafb10010, 0xafb20014, 0x551025,
-0xafa20018, 0x8f42010c, 0x24070008, 0x40f809,
-0x24c6001c, 0x1440ffe3, 0x0, 0x8f820054,
-0x2021023, 0x2c4203e9, 0x1440ffee, 0x0,
-0x326200ff, 0x14400011, 0x0, 0x8f420378,
-0x24420001, 0xaf420378, 0x8f420378, 0x8f820120,
-0x8faa002c, 0xafa20010, 0x8f820124, 0x3c040001,
-0x248469ac, 0x3c050009, 0xafa20014, 0x8d460000,
-0x34a50700, 0xc002b3b, 0x3c03821, 0x8f4202ec,
-0x24420001, 0xaf4202ec, 0x8f4202ec, 0x8fbf0048,
-0x8fbe0044, 0x8fb50040, 0x8fb3003c, 0x8fb20038,
-0x8fb10034, 0x8fb00030, 0x3e00008, 0x27bd0050,
-0x3c020001, 0x8c426da8, 0x27bdffe0, 0x1440000d,
-0xafbf0018, 0x3c040001, 0x24846a0c, 0x3c050008,
-0xafa00010, 0xafa00014, 0x8f860220, 0x34a50700,
-0x24020001, 0x3c010001, 0xac226da8, 0xc002b3b,
-0x3821, 0x3c020004, 0x2c21024, 0x10400007,
-0x0, 0x8f820220, 0x3c0308ff, 0x3463ffff,
-0x431024, 0x34420008, 0xaf820220, 0x3c050001,
-0x8ca56d98, 0x24020001, 0x14a20007, 0x2021,
-0xc00529b, 0x24050001, 0xac02026c, 0x8c03026c,
-0x10000006, 0x3c020007, 0xc00529b, 0x2021,
-0xac020268, 0x8c030268, 0x3c020007, 0x621824,
-0x3c020002, 0x5062000d, 0x3c0205f5, 0x43102b,
-0x14400006, 0x3c020004, 0x3c020001, 0x10620009,
-0x3c020098, 0x1000000b, 0x0, 0x14620009,
-0x3c023b9a, 0x10000004, 0x3442ca00, 0x10000002,
-0x3442e100, 0x34429680, 0xaf4201fc, 0x8f4201fc,
-0xaee20064, 0x8fbf0018, 0x3e00008, 0x27bd0020,
-0x0, 0x0, 0x0, 0x86102b,
-0x50400001, 0x872023, 0xc41023, 0x24843,
-0x125102b, 0x1040001b, 0x91040, 0x824021,
-0x88102b, 0x10400007, 0x1821, 0x94820000,
-0x24840002, 0x621821, 0x88102b, 0x1440fffb,
-0x0, 0x602021, 0xc73023, 0xa91023,
-0x21040, 0xc22821, 0xc5102b, 0x10400007,
-0x1821, 0x94c20000, 0x24c60002, 0x621821,
-0xc5102b, 0x1440fffb, 0x0, 0x1000000d,
-0x832021, 0x51040, 0x822821, 0x85102b,
-0x10400007, 0x1821, 0x94820000, 0x24840002,
-0x621821, 0x85102b, 0x1440fffb, 0x0,
-0x602021, 0x41c02, 0x3082ffff, 0x622021,
-0x41c02, 0x3082ffff, 0x622021, 0x3e00008,
-0x3082ffff, 0x3e00008, 0x0, 0x802821,
-0x30a20001, 0x1040002b, 0x3c03001f, 0x3463ffff,
-0x24a20004, 0x62102b, 0x54400007, 0x65102b,
-0x90a20001, 0x90a40003, 0x90a30000, 0x90a50002,
-0x1000002a, 0x441021, 0x10400003, 0x0,
-0x8f420148, 0xa22823, 0x90a40000, 0x24a50001,
-0x65102b, 0x10400003, 0x0, 0x8f420148,
-0xa22823, 0x90a20000, 0x24a50001, 0x21200,
-0x822021, 0x65102b, 0x10400003, 0x0,
-0x8f420148, 0xa22823, 0x90a20000, 0x24a50001,
-0x822021, 0x65102b, 0x10400003, 0x0,
-0x8f420148, 0xa22823, 0x90a20000, 0x1000002d,
-0x21200, 0x3463ffff, 0x24a20004, 0x62102b,
-0x5440000a, 0x65102b, 0x90a20000, 0x90a40002,
-0x90a30001, 0x90a50003, 0x441021, 0x21200,
-0x651821, 0x10000020, 0x432021, 0x10400003,
-0x0, 0x8f420148, 0xa22823, 0x90a20000,
-0x24a50001, 0x22200, 0x65102b, 0x10400003,
-0x0, 0x8f420148, 0xa22823, 0x90a20000,
-0x24a50001, 0x822021, 0x65102b, 0x10400003,
-0x0, 0x8f420148, 0xa22823, 0x90a20000,
-0x24a50001, 0x21200, 0x822021, 0x65102b,
-0x10400003, 0x0, 0x8f420148, 0xa22823,
-0x90a20000, 0x822021, 0x41c02, 0x3082ffff,
-0x622021, 0x41c02, 0x3082ffff, 0x622021,
-0x3e00008, 0x3082ffff, 0x0, 0x8f820220,
-0x34420002, 0xaf820220, 0x3c020002, 0x8c428ff8,
-0x30424000, 0x10400054, 0x24040001, 0x8f820200,
-0x24067fff, 0x8f830200, 0x30450002, 0x2402fffd,
-0x621824, 0xaf830200, 0xaf840204, 0x8f830054,
-0x8f820054, 0x10000002, 0x24630001, 0x8f820054,
-0x621023, 0x2c420002, 0x1440fffc, 0x0,
-0x8f820224, 0x1444004d, 0x42040, 0xc4102b,
-0x1040fff1, 0x0, 0x8f820200, 0x451025,
-0xaf820200, 0x8f820220, 0x34428000, 0xaf820220,
-0x8f830054, 0x8f820054, 0x10000002, 0x24630001,
-0x8f820054, 0x621023, 0x2c420002, 0x1440fffc,
-0x0, 0x8f820220, 0x3c030004, 0x431024,
-0x1440000f, 0x0, 0x8f820220, 0x3c03ffff,
-0x34637fff, 0x431024, 0xaf820220, 0x8f830054,
-0x8f820054, 0x10000002, 0x24630001, 0x8f820054,
-0x621023, 0x2c420002, 0x1440fffc, 0x0,
-0x8f820220, 0x3c030004, 0x431024, 0x1440000d,
-0x0, 0x8f820220, 0x34428000, 0xaf820220,
-0x8f830054, 0x8f820054, 0x10000002, 0x24630001,
-0x8f820054, 0x621023, 0x2c420002, 0x1440fffc,
-0x0, 0x8f820220, 0x3c030004, 0x431024,
-0x1040001b, 0x1021, 0x8f830220, 0x24020001,
-0x10000015, 0x3c04f700, 0x8f820220, 0x3c04f700,
-0x441025, 0xaf820220, 0x8f820220, 0x2403fffd,
-0x431024, 0xaf820220, 0x8f820220, 0x3c030300,
-0x431024, 0x14400003, 0x0, 0x10000008,
-0x1021, 0x8f820220, 0x34420002, 0xaf820220,
-0x8f830220, 0x24020001, 0x641825, 0xaf830220,
-0x3e00008, 0x0, 0x2021, 0x3c050100,
-0x24020001, 0xaf80021c, 0xaf820200, 0xaf820220,
-0x27625000, 0xaf8200c0, 0x27625000, 0xaf8200c4,
-0x27625000, 0xaf8200c8, 0x27625000, 0xaf8200d0,
-0x27625000, 0xaf8200d4, 0x27625000, 0xaf8200d8,
-0x27623000, 0xaf8200e0, 0x27623000, 0xaf8200e4,
-0x27623000, 0xaf8200e8, 0x27622800, 0xaf8200f0,
-0x27622800, 0xaf8200f4, 0x27622800, 0xaf8200f8,
-0x418c0, 0x24840001, 0x3631021, 0xac453004,
-0x3631021, 0xac403000, 0x28820200, 0x1440fff9,
-0x418c0, 0x2021, 0x418c0, 0x24840001,
-0x3631021, 0xac402804, 0x3631021, 0xac402800,
-0x28820100, 0x1440fff9, 0x418c0, 0xaf80023c,
-0x24030080, 0x24040100, 0xac600000, 0x24630004,
-0x64102b, 0x5440fffd, 0xac600000, 0x8f830040,
-0x3c02f000, 0x621824, 0x3c025000, 0x1062000c,
-0x43102b, 0x14400006, 0x3c026000, 0x3c024000,
-0x10620008, 0x24020800, 0x10000008, 0x0,
-0x10620004, 0x24020800, 0x10000004, 0x0,
-0x24020700, 0x3c010001, 0xac226dac, 0x3e00008,
-0x0, 0x3c020001, 0x8c426dbc, 0x27bdffd0,
-0xafbf002c, 0xafb20028, 0xafb10024, 0xafb00020,
-0x3c010001, 0x10400005, 0xac206d94, 0xc004d9e,
-0x0, 0x3c010001, 0xac206dbc, 0x8f830054,
-0x8f820054, 0x10000002, 0x24630064, 0x8f820054,
-0x621023, 0x2c420065, 0x1440fffc, 0x0,
-0xc004db9, 0x0, 0x24040001, 0x2821,
-0x27a60018, 0x34028000, 0xc0045be, 0xa7a20018,
-0x8f830054, 0x8f820054, 0x10000002, 0x24630064,
-0x8f820054, 0x621023, 0x2c420065, 0x1440fffc,
-0x24040001, 0x24050001, 0xc00457c, 0x27a60018,
-0x8f830054, 0x8f820054, 0x10000002, 0x24630064,
-0x8f820054, 0x621023, 0x2c420065, 0x1440fffc,
-0x24040001, 0x24050001, 0xc00457c, 0x27a60018,
-0x8f830054, 0x8f820054, 0x10000002, 0x24630064,
-0x8f820054, 0x621023, 0x2c420065, 0x1440fffc,
-0x24040001, 0x3c060001, 0x24c66f24, 0xc00457c,
-0x24050002, 0x8f830054, 0x8f820054, 0x10000002,
-0x24630064, 0x8f820054, 0x621023, 0x2c420065,
-0x1440fffc, 0x24040001, 0x24050003, 0x3c100001,
-0x26106f26, 0xc00457c, 0x2003021, 0x97a60018,
-0x3c070001, 0x94e76f24, 0x3c040001, 0x24846ae0,
-0xafa00014, 0x96020000, 0x3c05000d, 0x34a50100,
-0xc002b3b, 0xafa20010, 0x97a20018, 0x1040004d,
-0x24036040, 0x96020000, 0x3042fff0, 0x1443000c,
-0x24020020, 0x3c030001, 0x94636f24, 0x1462000b,
-0x24027830, 0x24020003, 0x3c010001, 0xac226d94,
-0x24020005, 0x3c010001, 0x1000003f, 0xac226f34,
-0x3c030001, 0x94636f24, 0x24027830, 0x1462000c,
-0x24030010, 0x3c020001, 0x94426f26, 0x3042fff0,
-0x14430007, 0x24020003, 0x3c010001, 0xac226d94,
-0x24020006, 0x3c010001, 0x1000002f, 0xac226f34,
-0x3c020001, 0x8c426d94, 0x3c030001, 0x94636f24,
-0x34420001, 0x3c010001, 0xac226d94, 0x24020015,
-0x1462000b, 0x0, 0x3c020001, 0x94426f26,
-0x3042fff0, 0x3843f420, 0x2c630001, 0x3842f430,
-0x2c420001, 0x621825, 0x1460001b, 0x24020003,
-0x3c030001, 0x94636f24, 0x24027810, 0x14620016,
-0x24020002, 0x3c020001, 0x94426f26, 0x3042fff0,
-0x14400011, 0x24020002, 0x1000000f, 0x24020004,
-0x3c020001, 0x8c426d94, 0x34420008, 0x3c010001,
-0xac226d94, 0x1000005e, 0x24020004, 0x3c020001,
-0x8c426d94, 0x34420004, 0x3c010001, 0x100000af,
-0xac226d94, 0x24020001, 0x3c010001, 0xac226f40,
-0x3c020001, 0x8c426d94, 0x30420002, 0x144000b2,
-0x3c09fff0, 0x24020e00, 0xaf820238, 0x8f840054,
-0x8f820054, 0x24030008, 0x3c010001, 0xac236d98,
-0x10000002, 0x248401f4, 0x8f820054, 0x821023,
-0x2c4201f5, 0x1440fffc, 0x3c0200c8, 0x344201fb,
-0xaf820238, 0x8f830054, 0x8f820054, 0x10000002,
-0x246301f4, 0x8f820054, 0x621023, 0x2c4201f5,
-0x1440fffc, 0x8021, 0x24120001, 0x24110009,
-0xc004482, 0x0, 0x3c010001, 0xac326db4,
-0xc004547, 0x0, 0x3c020001, 0x8c426db4,
-0x1451fffb, 0x3c0200c8, 0x344201f6, 0xaf820238,
-0x8f830054, 0x8f820054, 0x10000002, 0x2463000a,
-0x8f820054, 0x621023, 0x2c42000b, 0x1440fffc,
-0x0, 0x8f820220, 0x24040001, 0x34420002,
-0xaf820220, 0x8f830200, 0x24057fff, 0x2402fffd,
-0x621824, 0xaf830200, 0xaf840204, 0x8f830054,
-0x8f820054, 0x10000002, 0x24630001, 0x8f820054,
-0x621023, 0x2c420002, 0x1440fffc, 0x0,
-0x8f820224, 0x14440005, 0x34028000, 0x42040,
-0xa4102b, 0x1040fff0, 0x34028000, 0x1082ffa0,
-0x26100001, 0x2e020014, 0x1440ffcd, 0x24020004,
-0x3c010001, 0xac226d98, 0x8021, 0x24120009,
-0x3c11ffff, 0x36313f7f, 0xc004482, 0x0,
-0x24020001, 0x3c010001, 0xac226db4, 0xc004547,
-0x0, 0x3c020001, 0x8c426db4, 0x1452fffb,
-0x0, 0x8f820044, 0x511024, 0x34425080,
-0xaf820044, 0x8f830054, 0x8f820054, 0x10000002,
-0x2463000a, 0x8f820054, 0x621023, 0x2c42000b,
-0x1440fffc, 0x0, 0x8f820044, 0x511024,
-0x3442f080, 0xaf820044, 0x8f830054, 0x8f820054,
-0x10000002, 0x2463000a, 0x8f820054, 0x621023,
-0x2c42000b, 0x1440fffc, 0x0, 0x8f820220,
-0x3c03f700, 0x431025, 0xaf820220, 0x8f830054,
-0x8f820054, 0x10000002, 0x24630064, 0x8f820054,
-0x621023, 0x2c420065, 0x1440fffc, 0x0,
-0x8f820220, 0x24040001, 0x34420002, 0xaf820220,
-0x8f830200, 0x24057fff, 0x2402fffd, 0x621824,
-0xaf830200, 0xaf840204, 0x8f830054, 0x8f820054,
-0x10000002, 0x24630001, 0x8f820054, 0x621023,
-0x2c420002, 0x1440fffc, 0x0, 0x8f820224,
-0x14440005, 0x34028000, 0x42040, 0xa4102b,
-0x1040fff0, 0x34028000, 0x1082ff50, 0x26100001,
-0x2e020064, 0x1440ffb0, 0x0, 0x3c020001,
-0x8c426d94, 0x30420004, 0x14400007, 0x3c09fff0,
-0x8f820044, 0x3c03ffff, 0x34633f7f, 0x431024,
-0xaf820044, 0x3c09fff0, 0x3529bdc0, 0x3c060001,
-0x8cc66d94, 0x3c040001, 0x24846ae0, 0x24020001,
-0x3c010001, 0xac226d9c, 0x8f820054, 0x3c070001,
-0x8ce76f40, 0x3c030001, 0x94636f24, 0x3c080001,
-0x95086f26, 0x3c05000d, 0x34a50100, 0x3c010001,
-0xac206d98, 0x491021, 0x3c010001, 0xac226f30,
-0xafa30010, 0xc002b3b, 0xafa80014, 0x8fbf002c,
-0x8fb20028, 0x8fb10024, 0x8fb00020, 0x3e00008,
-0x27bd0030, 0x27bdffe8, 0x3c050001, 0x8ca56d98,
-0x24060004, 0x24020001, 0x14a20014, 0xafbf0010,
-0x3c020002, 0x8c428ffc, 0x30428000, 0x10400005,
-0x3c04000f, 0x3c030001, 0x8c636f40, 0x10000005,
-0x34844240, 0x3c040004, 0x3c030001, 0x8c636f40,
-0x348493e0, 0x24020005, 0x14620016, 0x0,
-0x3c04003d, 0x10000013, 0x34840900, 0x3c020002,
-0x8c428ff8, 0x30428000, 0x10400005, 0x3c04001e,
-0x3c030001, 0x8c636f40, 0x10000005, 0x34848480,
-0x3c04000f, 0x3c030001, 0x8c636f40, 0x34844240,
-0x24020005, 0x14620003, 0x0, 0x3c04007a,
-0x34841200, 0x3c020001, 0x8c426f30, 0x8f830054,
-0x441021, 0x431023, 0x44102b, 0x1440004c,
-0x0, 0x3c020001, 0x8c426da0, 0x14400048,
-0x0, 0x3c010001, 0x10c00025, 0xac206db0,
-0x3c090001, 0x8d296d94, 0x24070001, 0x3c044000,
-0x3c080002, 0x25088ffc, 0x250afffc, 0x52842,
-0x14a00002, 0x24c6ffff, 0x24050008, 0xa91024,
-0x10400010, 0x0, 0x14a70008, 0x0,
-0x8d020000, 0x441024, 0x1040000a, 0x0,
-0x3c010001, 0x10000007, 0xac256db0, 0x8d420000,
-0x441024, 0x10400003, 0x0, 0x3c010001,
-0xac276db0, 0x3c020001, 0x8c426db0, 0x6182b,
-0x2c420001, 0x431024, 0x5440ffe5, 0x52842,
-0x8f820054, 0x3c030001, 0x8c636db0, 0x3c010001,
-0xac226f30, 0x1060003b, 0x24020005, 0x3c030001,
-0x8c636f40, 0x3c010001, 0xac256d98, 0x14620012,
-0x24020001, 0x3c020002, 0x8c428ff8, 0x3c032000,
-0x34635000, 0x431024, 0x14400006, 0x24020001,
-0x3c010001, 0xac206f1c, 0x3c010001, 0xac226d98,
-0x24020001, 0x3c010001, 0xac226e24, 0x3c010001,
-0xac226da4, 0x24020001, 0x3c010001, 0xac226d9c,
-0x3c020001, 0x8c426db0, 0x1040001e, 0x0,
-0x3c020001, 0x8c426d9c, 0x10400008, 0x24020001,
-0x3c010001, 0xac206d9c, 0xaee204b8, 0x3c010001,
-0xac206e1c, 0x3c010001, 0xac226dd4, 0x8ee304b8,
-0x24020008, 0x10620005, 0x24020001, 0xc004239,
-0x0, 0x1000000b, 0x0, 0x3c030001,
-0x8c636d98, 0x10620007, 0x2402000e, 0x3c030002,
-0x8c638f90, 0x10620003, 0x0, 0xc004e9c,
-0x8f840220, 0x8fbf0010, 0x3e00008, 0x27bd0018,
-0x27bdffe0, 0x3c03fdff, 0x3c040001, 0x8c846d98,
-0x3c020001, 0x8c426dc0, 0x3463ffff, 0x283a024,
-0x14820006, 0xafbf0018, 0x8ee304b8, 0x3c020001,
-0x8c426dc4, 0x10620006, 0x0, 0x8ee204b8,
-0x3c010001, 0xac246dc0, 0x3c010001, 0xac226dc4,
-0x3c030001, 0x8c636d98, 0x24020002, 0x1062019c,
-0x2c620003, 0x10400005, 0x24020001, 0x1062000a,
-0x0, 0x10000226, 0x0, 0x24020004,
-0x106200b6, 0x24020008, 0x1062010a, 0x24020001,
-0x1000021f, 0x0, 0x8ee204b8, 0x2443ffff,
-0x2c620008, 0x1040021c, 0x31080, 0x3c010001,
-0x220821, 0x8c226af8, 0x400008, 0x0,
-0x3c030001, 0x8c636f40, 0x24020005, 0x14620010,
-0x0, 0x3c020001, 0x8c426da4, 0x10400008,
-0x24020003, 0xc004482, 0x0, 0x24020002,
-0xaee204b8, 0x3c010001, 0x10000002, 0xac206da4,
-0xaee204b8, 0x3c010001, 0x10000203, 0xac206d30,
-0xc004482, 0x0, 0x3c020001, 0x8c426da4,
-0x3c010001, 0xac206d30, 0x1440017a, 0x24020002,
-0x1000019d, 0x24020007, 0x3c030001, 0x8c636f40,
-0x24020005, 0x14620003, 0x24020001, 0x3c010001,
-0xac226dd0, 0xc0045ff, 0x0, 0x3c030001,
-0x8c636dd0, 0x10000174, 0x24020011, 0x3c050001,
-0x8ca56d98, 0x3c060002, 0x8cc68ffc, 0xc005104,
-0x2021, 0x24020005, 0x3c010001, 0xac206da4,
-0x100001e1, 0xaee204b8, 0x3c040001, 0x24846aec,
-0x3c05000f, 0x34a50100, 0x3021, 0x3821,
-0xafa00010, 0xc002b3b, 0xafa00014, 0x100001d6,
-0x0, 0x8f820220, 0x3c030004, 0x431024,
-0x14400175, 0x24020007, 0x8f830054, 0x3c020001,
-0x8c426f28, 0x2463d8f0, 0x431023, 0x2c422710,
-0x14400003, 0x24020001, 0x3c010001, 0xac226d9c,
-0x3c020002, 0x8c428ffc, 0x30425000, 0x104001c2,
-0x0, 0x8f820220, 0x30428000, 0x1040017d,
-0x0, 0x10000175, 0x0, 0x3c050001,
-0x8ca56d98, 0xc00529b, 0x2021, 0xc00551b,
-0x2021, 0x3c030002, 0x8c638ff4, 0x46101b0,
-0x24020001, 0x3c020008, 0x621024, 0x10400006,
-0x0, 0x8f820214, 0x3c03ffff, 0x431024,
-0x10000005, 0x3442251f, 0x8f820214, 0x3c03ffff,
-0x431024, 0x3442241f, 0xaf820214, 0x8f820220,
-0x3c030200, 0x34420002, 0xaf820220, 0x24020008,
-0xaee204b8, 0x8f820220, 0x283a025, 0x3c030004,
-0x431024, 0x14400016, 0x0, 0x3c020002,
-0x8c428ffc, 0x30425000, 0x1040000d, 0x0,
-0x8f820220, 0x30428000, 0x10400006, 0x0,
-0x8f820220, 0x3c03ffff, 0x34637fff, 0x10000003,
-0x431024, 0x8f820220, 0x34428000, 0xaf820220,
-0x8f820220, 0x3c03f700, 0x431025, 0xaf820220,
-0x3c030001, 0x8c636f40, 0x24020005, 0x1462000a,
-0x0, 0x3c020001, 0x94426f26, 0x24429fbc,
-0x2c420004, 0x10400004, 0x24040018, 0x24050002,
-0xc004ddb, 0x24060020, 0xc003e6d, 0x0,
-0x3c010001, 0x10000170, 0xac206e20, 0x8ee204b8,
-0x2443ffff, 0x2c620008, 0x1040016b, 0x31080,
-0x3c010001, 0x220821, 0x8c226b18, 0x400008,
-0x0, 0xc004547, 0x0, 0x3c030001,
-0x8c636db4, 0x100000e8, 0x24020009, 0x3c020002,
-0x8c428ff8, 0x30424000, 0x10400004, 0x0,
-0x8f820044, 0x10000006, 0x3442f080, 0x8f820044,
-0x3c03ffff, 0x34633f7f, 0x431024, 0x3442a080,
-0xaf820044, 0x8f830054, 0x100000ea, 0x24020004,
-0x8f830054, 0x3c020001, 0x8c426f28, 0x2463d8f0,
-0x431023, 0x2c422710, 0x14400147, 0x24020005,
-0x100000d8, 0x0, 0x8f820220, 0x3c03f700,
-0x431025, 0xaf820220, 0xaf800204, 0x3c010002,
-0x100000d6, 0xac208fe0, 0x8f830054, 0x3c020001,
-0x8c426f28, 0x2463fff6, 0x431023, 0x2c42000a,
-0x14400135, 0x24020007, 0x100000d7, 0x0,
-0xc003f50, 0x0, 0x1040012d, 0x24020001,
-0x8f820214, 0x3c03ffff, 0x3c040001, 0x8c846f1c,
-0x431024, 0x3442251f, 0xaf820214, 0x24020008,
-0x10800005, 0xaee204b8, 0x3c020001, 0x8c426e44,
-0x10400064, 0x24020001, 0x8f820220, 0x3c030008,
-0x431024, 0x1040006a, 0x3c020200, 0x10000078,
-0x0, 0x8ee204b8, 0x2443ffff, 0x2c620007,
-0x10400115, 0x31080, 0x3c010001, 0x220821,
-0x8c226b38, 0x400008, 0x0, 0xc003daf,
-0x0, 0x3c010001, 0xac206d9c, 0xaf800204,
-0x3c010002, 0xc004482, 0xac208fe0, 0x24020001,
-0x3c010001, 0xac226db4, 0x24020002, 0x10000102,
-0xaee204b8, 0xc004547, 0x0, 0x3c030001,
-0x8c636db4, 0x10000084, 0x24020009, 0x3c020002,
-0x8c428ff8, 0x30424000, 0x10400003, 0x3c0200c8,
-0x10000002, 0x344201f6, 0x344201fe, 0xaf820238,
-0x8f830054, 0x1000008b, 0x24020004, 0x8f830054,
-0x3c020001, 0x8c426f28, 0x2463d8f0, 0x431023,
-0x2c422710, 0x144000e8, 0x24020005, 0x10000079,
-0x0, 0x8f820220, 0x3c03f700, 0x431025,
-0xaf820220, 0xaf800204, 0x3c010002, 0x10000077,
-0xac208fe0, 0x8f830054, 0x3c020001, 0x8c426f28,
-0x2463fff6, 0x431023, 0x2c42000a, 0x144000d6,
-0x24020007, 0x10000078, 0x0, 0xc003f50,
-0x0, 0x104000ce, 0x24020001, 0x8f820214,
-0x3c03ffff, 0x3c040001, 0x8c846f1c, 0x431024,
-0x3442251f, 0xaf820214, 0x24020008, 0x1080000f,
-0xaee204b8, 0x3c020001, 0x8c426e44, 0x1440000b,
-0x0, 0x8f820220, 0x34420002, 0xaf820220,
-0x24020001, 0x3c010002, 0xac228f90, 0xc004e9c,
-0x8f840220, 0x10000016, 0x0, 0x8f820220,
-0x3c030008, 0x431024, 0x14400011, 0x3c020200,
-0x282a025, 0x2402000e, 0x3c010002, 0xac228f90,
-0xc00551b, 0x2021, 0x8f820220, 0x34420002,
-0xc003e6d, 0xaf820220, 0x3c050001, 0x8ca56d98,
-0xc00529b, 0x2021, 0x100000a3, 0x0,
-0x3c020001, 0x8c426e44, 0x1040009f, 0x0,
-0x3c020001, 0x8c426e40, 0x2442ffff, 0x3c010001,
-0xac226e40, 0x14400098, 0x24020002, 0x3c010001,
-0xac206e44, 0x3c010001, 0x10000093, 0xac226e40,
-0x8ee204b8, 0x2443ffff, 0x2c620007, 0x1040008e,
-0x31080, 0x3c010001, 0x220821, 0x8c226b58,
-0x400008, 0x0, 0x3c020001, 0x8c426da4,
-0x10400018, 0x24020005, 0xc004482, 0x0,
-0x24020002, 0xaee204b8, 0x3c010001, 0x1000007e,
-0xac206da4, 0xc004963, 0x0, 0x3c030001,
-0x8c636dd4, 0x24020006, 0x14620077, 0x24020003,
-0x10000075, 0xaee204b8, 0x3c050001, 0x8ca56d98,
-0x3c060002, 0x8cc68ff8, 0xc005104, 0x2021,
-0x24020005, 0x1000006c, 0xaee204b8, 0x8f820220,
-0x3c03f700, 0x431025, 0xaf820220, 0x8f830054,
-0x24020006, 0xaee204b8, 0x3c010001, 0x10000062,
-0xac236f28, 0x8f820220, 0x3c030004, 0x431024,
-0x10400003, 0x24020007, 0x1000005b, 0xaee204b8,
-0x8f830054, 0x3c020001, 0x8c426f28, 0x2463d8f0,
-0x431023, 0x2c422710, 0x14400003, 0x24020001,
-0x3c010001, 0xac226d9c, 0x3c020002, 0x8c428ff8,
-0x30425000, 0x1040004c, 0x0, 0x8f820220,
-0x30428000, 0x10400007, 0x0, 0x8f820220,
-0x3c03ffff, 0x34637fff, 0x431024, 0x10000042,
-0xaf820220, 0x8f820220, 0x34428000, 0x1000003e,
-0xaf820220, 0x3c050001, 0x8ca56d98, 0xc00529b,
-0x2021, 0xc00551b, 0x2021, 0x3c020002,
-0x8c428ff0, 0x4410032, 0x24020001, 0x8f820214,
-0x3c03ffff, 0x431024, 0x3442251f, 0xaf820214,
-0x24020008, 0xaee204b8, 0x8f820220, 0x34420002,
-0xaf820220, 0x8f820220, 0x3c030004, 0x431024,
-0x14400016, 0x0, 0x3c020002, 0x8c428ff8,
-0x30425000, 0x1040000d, 0x0, 0x8f820220,
-0x30428000, 0x10400006, 0x0, 0x8f820220,
-0x3c03ffff, 0x34637fff, 0x10000003, 0x431024,
-0x8f820220, 0x34428000, 0xaf820220, 0x8f820220,
-0x3c03f700, 0x431025, 0xaf820220, 0x3c020001,
-0x94426f26, 0x24429fbc, 0x2c420004, 0x10400004,
-0x24040018, 0x24050002, 0xc004ddb, 0x24060020,
-0xc003e6d, 0x0, 0x10000003, 0x0,
-0x3c010001, 0xac226d9c, 0x8fbf0018, 0x3e00008,
-0x27bd0020, 0x8f820200, 0x8f820220, 0x8f820220,
-0x34420004, 0xaf820220, 0x8f820200, 0x3c050001,
-0x8ca56d98, 0x34420004, 0xaf820200, 0x24020002,
-0x10a2004b, 0x2ca20003, 0x10400005, 0x24020001,
-0x10a2000a, 0x0, 0x100000b1, 0x0,
-0x24020004, 0x10a20072, 0x24020008, 0x10a20085,
-0x3c02f0ff, 0x100000aa, 0x0, 0x8f830050,
-0x3c02f0ff, 0x3442ffff, 0x3c040001, 0x8c846f40,
-0x621824, 0x3c020700, 0x621825, 0x24020e00,
-0x2484fffb, 0x2c840002, 0xaf830050, 0xaf850200,
-0xaf850220, 0x14800006, 0xaf820238, 0x8f820044,
-0x3c03ffff, 0x34633f7f, 0x431024, 0xaf820044,
-0x3c030001, 0x8c636f40, 0x24020005, 0x14620004,
-0x0, 0x8f820044, 0x34425000, 0xaf820044,
-0x3c020001, 0x8c426d88, 0x3c030001, 0x8c636f40,
-0x34420022, 0x2463fffc, 0x2c630002, 0x1460000c,
-0xaf820200, 0x3c020001, 0x8c426dac, 0x3c030001,
-0x8c636d90, 0x3c040001, 0x8c846d8c, 0x34428000,
-0x621825, 0x641825, 0x1000000a, 0x34620002,
-0x3c020001, 0x8c426d90, 0x3c030001, 0x8c636dac,
-0x3c040001, 0x8c846d8c, 0x431025, 0x441025,
-0x34420002, 0xaf820220, 0x1000002f, 0x24020001,
-0x24020e01, 0xaf820238, 0x8f830050, 0x3c02f0ff,
-0x3442ffff, 0x3c040001, 0x8c846f1c, 0x621824,
-0x3c020d00, 0x621825, 0x24020001, 0xaf830050,
-0xaf820200, 0xaf820220, 0x10800005, 0x3c033f00,
-0x3c020001, 0x8c426d80, 0x10000004, 0x34630070,
-0x3c020001, 0x8c426d80, 0x34630072, 0x431025,
-0xaf820200, 0x3c030001, 0x8c636d84, 0x3c02f700,
-0x621825, 0x3c020001, 0x8c426d90, 0x3c040001,
-0x8c846dac, 0x3c050001, 0x8ca56f40, 0x431025,
-0x441025, 0xaf820220, 0x24020005, 0x14a20006,
-0x24020001, 0x8f820044, 0x2403afff, 0x431024,
-0xaf820044, 0x24020001, 0x1000003d, 0xaf820238,
-0x8f830050, 0x3c02f0ff, 0x3442ffff, 0x3c040001,
-0x8c846f1c, 0x621824, 0x3c020a00, 0x621825,
-0x24020001, 0xaf830050, 0xaf820200, 0x1080001e,
-0xaf820220, 0x3c020001, 0x8c426e44, 0x1440001a,
-0x3c033f00, 0x3c020001, 0x8c426d80, 0x1000001a,
-0x346300e0, 0x8f830050, 0x3c040001, 0x8c846f1c,
-0x3442ffff, 0x621824, 0x1080000f, 0xaf830050,
-0x3c020001, 0x8c426e44, 0x1440000b, 0x3c043f00,
-0x3c030001, 0x8c636d80, 0x348400e0, 0x24020001,
-0xaf820200, 0xaf820220, 0x641825, 0xaf830200,
-0x10000008, 0x3c05f700, 0x3c020001, 0x8c426d80,
-0x3c033f00, 0x346300e2, 0x431025, 0xaf820200,
-0x3c05f700, 0x34a58000, 0x3c030001, 0x8c636d84,
-0x3c020001, 0x8c426d90, 0x3c040001, 0x8c846dac,
-0x651825, 0x431025, 0x441025, 0xaf820220,
-0x3e00008, 0x0, 0x3c030001, 0x8c636db4,
-0x3c020001, 0x8c426db8, 0x10620003, 0x24020002,
-0x3c010001, 0xac236db8, 0x1062001d, 0x2c620003,
-0x10400025, 0x24020001, 0x14620023, 0x24020004,
-0x3c030001, 0x8c636d98, 0x10620006, 0x24020008,
-0x1462000c, 0x3c0200c8, 0x344201fb, 0x10000009,
-0xaf820238, 0x24020e01, 0xaf820238, 0x8f820044,
-0x3c03ffff, 0x34633f7f, 0x431024, 0x34420080,
-0xaf820044, 0x8f830054, 0x24020002, 0x3c010001,
-0xac226db4, 0x3c010001, 0x1000000b, 0xac236f2c,
-0x8f830054, 0x3c020001, 0x8c426f2c, 0x2463d8f0,
-0x431023, 0x2c422710, 0x14400003, 0x24020009,
-0x3c010001, 0xac226db4, 0x3e00008, 0x0,
-0x0, 0x0, 0x0, 0x27bdffd8,
-0xafb20018, 0x809021, 0xafb3001c, 0xa09821,
-0xafb10014, 0xc08821, 0xafb00010, 0x8021,
-0xafbf0020, 0xa6200000, 0xc004d78, 0x24040001,
-0x26100001, 0x2e020020, 0x1440fffb, 0x0,
-0xc004d78, 0x2021, 0xc004d78, 0x24040001,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0x24100010, 0x2501024, 0x10400002, 0x2021,
-0x24040001, 0xc004d78, 0x108042, 0x1600fffa,
-0x2501024, 0x24100010, 0x2701024, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fffa, 0x2701024, 0xc004db9, 0x34108000,
-0xc004db9, 0x0, 0xc004d58, 0x0,
-0x50400005, 0x108042, 0x96220000, 0x501025,
-0xa6220000, 0x108042, 0x1600fff7, 0x0,
-0xc004db9, 0x0, 0x8fbf0020, 0x8fb3001c,
-0x8fb20018, 0x8fb10014, 0x8fb00010, 0x3e00008,
-0x27bd0028, 0x27bdffd8, 0xafb10014, 0x808821,
-0xafb20018, 0xa09021, 0xafb3001c, 0xc09821,
-0xafb00010, 0x8021, 0xafbf0020, 0xc004d78,
-0x24040001, 0x26100001, 0x2e020020, 0x1440fffb,
-0x0, 0xc004d78, 0x2021, 0xc004d78,
-0x24040001, 0xc004d78, 0x2021, 0xc004d78,
-0x24040001, 0x24100010, 0x2301024, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fffa, 0x2301024, 0x24100010, 0x2501024,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x2501024, 0xc004d78,
-0x24040001, 0xc004d78, 0x2021, 0x34108000,
-0x96620000, 0x501024, 0x10400002, 0x2021,
-0x24040001, 0xc004d78, 0x108042, 0x1600fff8,
-0x0, 0xc004db9, 0x0, 0x8fbf0020,
-0x8fb3001c, 0x8fb20018, 0x8fb10014, 0x8fb00010,
-0x3e00008, 0x27bd0028, 0x3c040001, 0x8c846dd0,
-0x3c020001, 0x8c426e18, 0x27bdffd8, 0xafbf0020,
-0xafb1001c, 0x10820003, 0xafb00018, 0x3c010001,
-0xac246e18, 0x3c030001, 0x8c636f40, 0x24020005,
-0x14620005, 0x2483ffff, 0xc004963, 0x0,
-0x1000034c, 0x0, 0x2c620013, 0x10400349,
-0x31080, 0x3c010001, 0x220821, 0x8c226b80,
-0x400008, 0x0, 0xc004db9, 0x8021,
-0x34028000, 0xa7a20010, 0x27b10010, 0xc004d78,
-0x24040001, 0x26100001, 0x2e020020, 0x1440fffb,
-0x0, 0xc004d78, 0x2021, 0xc004d78,
-0x24040001, 0xc004d78, 0x2021, 0xc004d78,
-0x24040001, 0x24100010, 0x32020001, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fffa, 0x32020001, 0x24100010, 0xc004d78,
-0x2021, 0x108042, 0x1600fffc, 0x0,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0x34108000, 0x96220000, 0x501024, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fff8, 0x0, 0xc004db9, 0x0,
-0x1000030e, 0x24020002, 0x27b10010, 0xa7a00010,
-0x8021, 0xc004d78, 0x24040001, 0x26100001,
-0x2e020020, 0x1440fffb, 0x0, 0xc004d78,
-0x2021, 0xc004d78, 0x24040001, 0xc004d78,
-0x24040001, 0xc004d78, 0x2021, 0x24100010,
-0x32020001, 0x10400002, 0x2021, 0x24040001,
-0xc004d78, 0x108042, 0x1600fffa, 0x32020001,
-0x24100010, 0xc004d78, 0x2021, 0x108042,
-0x1600fffc, 0x0, 0xc004db9, 0x34108000,
-0xc004db9, 0x0, 0xc004d58, 0x0,
-0x50400005, 0x108042, 0x96220000, 0x501025,
-0xa6220000, 0x108042, 0x1600fff7, 0x0,
-0xc004db9, 0x0, 0x97a20010, 0x30428000,
-0x144002dc, 0x24020003, 0x100002d8, 0x0,
-0x24021200, 0xa7a20010, 0x27b10010, 0x8021,
-0xc004d78, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0xc004d78, 0x2021, 0x108042, 0x1600fffc,
-0x0, 0xc004d78, 0x24040001, 0xc004d78,
-0x2021, 0x34108000, 0x96220000, 0x501024,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fff8, 0x0, 0xc004db9,
-0x0, 0x8f830054, 0x10000296, 0x24020004,
-0x8f830054, 0x3c020001, 0x8c426f3c, 0x2463ff9c,
-0x431023, 0x2c420064, 0x1440029e, 0x24020002,
-0x3c030001, 0x8c636f40, 0x10620297, 0x2c620003,
-0x14400296, 0x24020011, 0x24020003, 0x10620005,
-0x24020004, 0x10620291, 0x2402000f, 0x1000028f,
-0x24020011, 0x1000028d, 0x24020005, 0x24020014,
-0xa7a20010, 0x27b10010, 0x8021, 0xc004d78,
-0x24040001, 0x26100001, 0x2e020020, 0x1440fffb,
-0x0, 0xc004d78, 0x2021, 0xc004d78,
-0x24040001, 0xc004d78, 0x2021, 0xc004d78,
-0x24040001, 0x24100010, 0x32020001, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fffa, 0x32020001, 0x24100010, 0x32020012,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x32020012, 0xc004d78,
-0x24040001, 0xc004d78, 0x2021, 0x34108000,
-0x96220000, 0x501024, 0x10400002, 0x2021,
-0x24040001, 0xc004d78, 0x108042, 0x1600fff8,
-0x0, 0xc004db9, 0x0, 0x8f830054,
-0x10000248, 0x24020006, 0x8f830054, 0x3c020001,
-0x8c426f3c, 0x2463ff9c, 0x431023, 0x2c420064,
-0x14400250, 0x24020007, 0x1000024c, 0x0,
-0x24020006, 0xa7a20010, 0x27b10010, 0x8021,
-0xc004d78, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020013, 0x10400002, 0x2021, 0x24040001,
-0xc004d78, 0x108042, 0x1600fffa, 0x32020013,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0x34108000, 0x96220000, 0x501024, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fff8, 0x0, 0xc004db9, 0x0,
-0x8f830054, 0x10000207, 0x24020008, 0x8f830054,
-0x3c020001, 0x8c426f3c, 0x2463ff9c, 0x431023,
-0x2c420064, 0x1440020f, 0x24020009, 0x1000020b,
-0x0, 0x27b10010, 0xa7a00010, 0x8021,
-0xc004d78, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0xc004d78, 0x24040001,
-0xc004d78, 0x2021, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020018, 0x10400002, 0x2021, 0x24040001,
-0xc004d78, 0x108042, 0x1600fffa, 0x32020018,
-0xc004db9, 0x34108000, 0xc004db9, 0x0,
-0xc004d58, 0x0, 0x50400005, 0x108042,
-0x96220000, 0x501025, 0xa6220000, 0x108042,
-0x1600fff7, 0x0, 0xc004db9, 0x8021,
-0x97a20010, 0x27b10010, 0x34420001, 0xa7a20010,
-0xc004d78, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020018, 0x10400002, 0x2021, 0x24040001,
-0xc004d78, 0x108042, 0x1600fffa, 0x32020018,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0x34108000, 0x96220000, 0x501024, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fff8, 0x0, 0xc004db9, 0x0,
-0x8f830054, 0x10000193, 0x2402000a, 0x8f830054,
-0x3c020001, 0x8c426f3c, 0x2463ff9c, 0x431023,
-0x2c420064, 0x1440019b, 0x2402000b, 0x10000197,
-0x0, 0x27b10010, 0xa7a00010, 0x8021,
-0xc004d78, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0xc004d78, 0x24040001,
-0xc004d78, 0x2021, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020017, 0x10400002, 0x2021, 0x24040001,
-0xc004d78, 0x108042, 0x1600fffa, 0x32020017,
-0xc004db9, 0x34108000, 0xc004db9, 0x0,
-0xc004d58, 0x0, 0x50400005, 0x108042,
-0x96220000, 0x501025, 0xa6220000, 0x108042,
-0x1600fff7, 0x0, 0xc004db9, 0x8021,
-0x97a20010, 0x27b10010, 0x34420700, 0xa7a20010,
-0xc004d78, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020017, 0x10400002, 0x2021, 0x24040001,
-0xc004d78, 0x108042, 0x1600fffa, 0x32020017,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0x34108000, 0x96220000, 0x501024, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fff8, 0x0, 0xc004db9, 0x0,
-0x8f830054, 0x1000011f, 0x2402000c, 0x8f830054,
-0x3c020001, 0x8c426f3c, 0x2463ff9c, 0x431023,
-0x2c420064, 0x14400127, 0x24020012, 0x10000123,
-0x0, 0x27b10010, 0xa7a00010, 0x8021,
-0xc004d78, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0xc004d78, 0x24040001,
-0xc004d78, 0x2021, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020014, 0x10400002, 0x2021, 0x24040001,
-0xc004d78, 0x108042, 0x1600fffa, 0x32020014,
-0xc004db9, 0x34108000, 0xc004db9, 0x0,
-0xc004d58, 0x0, 0x50400005, 0x108042,
-0x96220000, 0x501025, 0xa6220000, 0x108042,
-0x1600fff7, 0x0, 0xc004db9, 0x8021,
-0x97a20010, 0x27b10010, 0x34420010, 0xa7a20010,
-0xc004d78, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020014, 0x10400002, 0x2021, 0x24040001,
-0xc004d78, 0x108042, 0x1600fffa, 0x32020014,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0x34108000, 0x96220000, 0x501024, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fff8, 0x0, 0xc004db9, 0x0,
-0x8f830054, 0x100000ab, 0x24020013, 0x8f830054,
-0x3c020001, 0x8c426f3c, 0x2463ff9c, 0x431023,
-0x2c420064, 0x144000b3, 0x2402000d, 0x100000af,
-0x0, 0x27b10010, 0xa7a00010, 0x8021,
-0xc004d78, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0xc004d78, 0x24040001,
-0xc004d78, 0x2021, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020018, 0x10400002, 0x2021, 0x24040001,
-0xc004d78, 0x108042, 0x1600fffa, 0x32020018,
-0xc004db9, 0x34108000, 0xc004db9, 0x0,
-0xc004d58, 0x0, 0x50400005, 0x108042,
-0x96220000, 0x501025, 0xa6220000, 0x108042,
-0x1600fff7, 0x0, 0xc004db9, 0x8021,
-0x97a20010, 0x27b10010, 0x3042fffe, 0xa7a20010,
-0xc004d78, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020018, 0x10400002, 0x2021, 0x24040001,
-0xc004d78, 0x108042, 0x1600fffa, 0x32020018,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0x34108000, 0x96220000, 0x501024, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fff8, 0x0, 0xc004db9, 0x0,
-0x8f830054, 0x10000037, 0x2402000e, 0x24020840,
-0xa7a20010, 0x27b10010, 0x8021, 0xc004d78,
-0x24040001, 0x26100001, 0x2e020020, 0x1440fffb,
-0x0, 0xc004d78, 0x2021, 0xc004d78,
-0x24040001, 0xc004d78, 0x2021, 0xc004d78,
-0x24040001, 0x24100010, 0x32020001, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fffa, 0x32020001, 0x24100010, 0x32020013,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x32020013, 0xc004d78,
-0x24040001, 0xc004d78, 0x2021, 0x34108000,
-0x96220000, 0x501024, 0x10400002, 0x2021,
-0x24040001, 0xc004d78, 0x108042, 0x1600fff8,
-0x0, 0xc004db9, 0x0, 0x8f830054,
-0x24020010, 0x3c010001, 0xac226dd0, 0x3c010001,
-0x1000000c, 0xac236f3c, 0x8f830054, 0x3c020001,
-0x8c426f3c, 0x2463ff9c, 0x431023, 0x2c420064,
-0x14400004, 0x0, 0x24020011, 0x3c010001,
-0xac226dd0, 0x8fbf0020, 0x8fb1001c, 0x8fb00018,
-0x3e00008, 0x27bd0028, 0x3c030001, 0x8c636d98,
-0x27bdffc8, 0x24020002, 0xafbf0034, 0xafb20030,
-0xafb1002c, 0x14620004, 0xafb00028, 0x3c120002,
-0x10000003, 0x8e528ff8, 0x3c120002, 0x8e528ffc,
-0x3c030001, 0x8c636dd4, 0x3c020001, 0x8c426e1c,
-0x50620004, 0x2463ffff, 0x3c010001, 0xac236e1c,
-0x2463ffff, 0x2c620006, 0x10400377, 0x31080,
-0x3c010001, 0x220821, 0x8c226bd8, 0x400008,
-0x0, 0x2021, 0x2821, 0xc004ddb,
-0x34068000, 0x24040010, 0x24050002, 0x24060002,
-0x24020002, 0xc004ddb, 0xa7a20018, 0x24020002,
-0x3c010001, 0x10000364, 0xac226dd4, 0x27b10018,
-0xa7a00018, 0x8021, 0xc004d78, 0x24040001,
-0x26100001, 0x2e020020, 0x1440fffb, 0x0,
-0xc004d78, 0x2021, 0xc004d78, 0x24040001,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0x24100010, 0x32020001, 0x10400002, 0x2021,
-0x24040001, 0xc004d78, 0x108042, 0x1600fffa,
-0x32020001, 0x24100010, 0xc004d78, 0x2021,
-0x108042, 0x1600fffc, 0x0, 0xc004db9,
-0x34108000, 0xc004db9, 0x0, 0xc004d58,
-0x0, 0x50400005, 0x108042, 0x96220000,
-0x501025, 0xa6220000, 0x108042, 0x1600fff7,
-0x0, 0xc004db9, 0x0, 0x97a20018,
-0x30428000, 0x14400004, 0x24020003, 0x3c010001,
-0xac226dd4, 0x24020003, 0x3c010001, 0x1000032a,
-0xac226dd4, 0x24040010, 0x24050002, 0x24060002,
-0x24020002, 0xc004ddb, 0xa7a20018, 0x3c030001,
-0x8c636e20, 0x24020001, 0x146201e1, 0x8021,
-0x27b10018, 0xa7a00018, 0xc004d78, 0x24040001,
-0x26100001, 0x2e020020, 0x1440fffb, 0x0,
-0xc004d78, 0x2021, 0xc004d78, 0x24040001,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0x24100010, 0x32020001, 0x10400002, 0x2021,
-0x24040001, 0xc004d78, 0x108042, 0x1600fffa,
-0x32020001, 0x24100010, 0x32020018, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fffa, 0x32020018, 0xc004db9, 0x34108000,
-0xc004db9, 0x0, 0xc004d58, 0x0,
-0x50400005, 0x108042, 0x96220000, 0x501025,
-0xa6220000, 0x108042, 0x1600fff7, 0x0,
-0xc004db9, 0x8021, 0x27b10018, 0xa7a00018,
-0xc004d78, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0xc004d78, 0x24040001,
-0xc004d78, 0x2021, 0x24100010, 0x32020001,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x32020001, 0x24100010,
-0x32020018, 0x10400002, 0x2021, 0x24040001,
-0xc004d78, 0x108042, 0x1600fffa, 0x32020018,
-0xc004db9, 0x34108000, 0xc004db9, 0x0,
-0xc004d58, 0x0, 0x50400005, 0x108042,
-0x96220000, 0x501025, 0xa6220000, 0x108042,
-0x1600fff7, 0x0, 0xc004db9, 0x8021,
-0x24040018, 0x2821, 0xc004ddb, 0x24060404,
-0xa7a0001a, 0xc004d78, 0x24040001, 0x26100001,
-0x2e020020, 0x1440fffb, 0x0, 0xc004d78,
-0x2021, 0xc004d78, 0x24040001, 0xc004d78,
-0x24040001, 0xc004d78, 0x2021, 0x24100010,
-0x32020001, 0x10400002, 0x2021, 0x24040001,
-0xc004d78, 0x108042, 0x1600fffa, 0x32020001,
-0x24100010, 0x32020018, 0x10400002, 0x2021,
-0x24040001, 0xc004d78, 0x108042, 0x1600fffa,
-0x32020018, 0xc004db9, 0x34108000, 0xc004db9,
-0x0, 0xc004d58, 0x0, 0x50400005,
-0x108042, 0x97a2001a, 0x501025, 0xa7a2001a,
-0x108042, 0x1600fff7, 0x0, 0xc004db9,
-0x8021, 0xa7a0001a, 0xc004d78, 0x24040001,
-0x26100001, 0x2e020020, 0x1440fffb, 0x0,
-0xc004d78, 0x2021, 0xc004d78, 0x24040001,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0x24100010, 0x32020001, 0x10400002, 0x2021,
-0x24040001, 0xc004d78, 0x108042, 0x1600fffa,
-0x32020001, 0x24100010, 0x32020018, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fffa, 0x32020018, 0xc004db9, 0x34108000,
-0xc004db9, 0x0, 0xc004d58, 0x0,
-0x50400005, 0x108042, 0x97a2001a, 0x501025,
-0xa7a2001a, 0x108042, 0x1600fff7, 0x0,
-0xc004db9, 0x8021, 0xa7a0001c, 0xc004d78,
-0x24040001, 0x26100001, 0x2e020020, 0x1440fffb,
-0x0, 0xc004d78, 0x2021, 0xc004d78,
-0x24040001, 0xc004d78, 0x24040001, 0xc004d78,
-0x2021, 0x24100010, 0xc004d78, 0x2021,
-0x108042, 0x1600fffc, 0x0, 0x24100010,
-0x3202001e, 0x10400002, 0x2021, 0x24040001,
-0xc004d78, 0x108042, 0x1600fffa, 0x3202001e,
-0xc004db9, 0x34108000, 0xc004db9, 0x0,
-0xc004d58, 0x0, 0x50400005, 0x108042,
-0x97a2001c, 0x501025, 0xa7a2001c, 0x108042,
-0x1600fff7, 0x0, 0xc004db9, 0x8021,
-0xa7a0001c, 0xc004d78, 0x24040001, 0x26100001,
-0x2e020020, 0x1440fffb, 0x0, 0xc004d78,
-0x2021, 0xc004d78, 0x24040001, 0xc004d78,
-0x24040001, 0xc004d78, 0x2021, 0x24100010,
-0xc004d78, 0x2021, 0x108042, 0x1600fffc,
-0x0, 0x24100010, 0x3202001e, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fffa, 0x3202001e, 0xc004db9, 0x34108000,
-0xc004db9, 0x0, 0xc004d58, 0x0,
-0x50400005, 0x108042, 0x97a2001c, 0x501025,
-0xa7a2001c, 0x108042, 0x1600fff7, 0x0,
-0xc004db9, 0x8021, 0x24020002, 0xa7a2001e,
-0xc004d78, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0x24100010, 0xc004d78,
-0x2021, 0x108042, 0x1600fffc, 0x0,
-0x24100010, 0x3202001e, 0x10400002, 0x2021,
-0x24040001, 0xc004d78, 0x108042, 0x1600fffa,
-0x3202001e, 0xc004d78, 0x24040001, 0xc004d78,
-0x2021, 0x34108000, 0x97a2001e, 0x501024,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fff8, 0x0, 0xc004db9,
-0x8021, 0xa7a00020, 0xc004d78, 0x24040001,
-0x26100001, 0x2e020020, 0x1440fffb, 0x0,
-0xc004d78, 0x2021, 0xc004d78, 0x24040001,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0x24100010, 0xc004d78, 0x2021, 0x108042,
-0x1600fffc, 0x0, 0x24100010, 0x3202001e,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x3202001e, 0xc004db9,
-0x34108000, 0xc004db9, 0x0, 0xc004d58,
-0x0, 0x50400005, 0x108042, 0x97a20020,
-0x501025, 0xa7a20020, 0x108042, 0x1600fff7,
-0x0, 0xc004db9, 0x8021, 0xa7a00020,
-0xc004d78, 0x24040001, 0x26100001, 0x2e020020,
-0x1440fffb, 0x0, 0xc004d78, 0x2021,
-0xc004d78, 0x24040001, 0xc004d78, 0x24040001,
-0xc004d78, 0x2021, 0x24100010, 0xc004d78,
-0x2021, 0x108042, 0x1600fffc, 0x0,
-0x24100010, 0x3202001e, 0x10400002, 0x2021,
-0x24040001, 0xc004d78, 0x108042, 0x1600fffa,
-0x3202001e, 0xc004db9, 0x34108000, 0xc004db9,
-0x0, 0xc004d58, 0x0, 0x50400005,
-0x108042, 0x97a20020, 0x501025, 0xa7a20020,
-0x108042, 0x1600fff7, 0x0, 0xc004db9,
-0x8021, 0xa7a00022, 0xc004d78, 0x24040001,
-0x26100001, 0x2e020020, 0x1440fffb, 0x0,
-0xc004d78, 0x2021, 0xc004d78, 0x24040001,
-0xc004d78, 0x2021, 0xc004d78, 0x24040001,
-0x24100010, 0xc004d78, 0x2021, 0x108042,
-0x1600fffc, 0x0, 0x24100010, 0xc004d78,
-0x2021, 0x108042, 0x1600fffc, 0x0,
-0xc004d78, 0x24040001, 0xc004d78, 0x2021,
-0x34108000, 0x97a20022, 0x501024, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fff8, 0x0, 0xc004db9, 0x0,
-0x24040018, 0x24050002, 0xc004ddb, 0x24060004,
-0x3c100001, 0x8e106e24, 0x24020001, 0x1602011d,
-0x0, 0x3c020001, 0x94426f26, 0x3c010001,
-0xac206e24, 0x24429fbc, 0x2c420004, 0x1040000c,
-0x24040009, 0x24050001, 0xc004ddb, 0x24060400,
-0x24040018, 0x24050001, 0xc004ddb, 0x24060020,
-0x24040018, 0x24050001, 0xc004ddb, 0x24062000,
-0x3c024000, 0x2421024, 0x10400123, 0x3c022000,
-0x2421024, 0x10400004, 0x0, 0x3c010001,
-0x10000003, 0xac306f1c, 0x3c010001, 0xac206f1c,
-0x3c030001, 0x8c636f34, 0x24020005, 0x146200f9,
-0x0, 0x3c020001, 0x8c426f1c, 0x10400067,
-0x3c020004, 0x2421024, 0x10400011, 0xa7a00018,
-0x3c020008, 0x2421024, 0x10400002, 0x24020200,
-0xa7a20018, 0x3c020010, 0x2421024, 0x10400004,
-0x0, 0x97a20018, 0x34420100, 0xa7a20018,
-0x97a60018, 0x24040009, 0x10000004, 0x2821,
-0x24040009, 0x2821, 0x3021, 0xc004ddb,
-0x0, 0x24020001, 0xa7a2001a, 0x3c020008,
-0x2421024, 0x1040000c, 0x3c020002, 0x2421024,
-0x10400002, 0x24020101, 0xa7a2001a, 0x3c020001,
-0x2421024, 0x10400005, 0x3c020010, 0x97a2001a,
-0x34420040, 0xa7a2001a, 0x3c020010, 0x2421024,
-0x1040000e, 0x3c020002, 0x2421024, 0x10400005,
-0x3c020001, 0x97a2001a, 0x34420080, 0xa7a2001a,
-0x3c020001, 0x2421024, 0x10400005, 0x3c0300a0,
-0x97a2001a, 0x34420020, 0xa7a2001a, 0x3c0300a0,
-0x2431024, 0x54430004, 0x3c020020, 0x97a2001a,
-0x1000000c, 0x34420400, 0x2421024, 0x50400004,
-0x3c020080, 0x97a2001a, 0x10000006, 0x34420800,
-0x2421024, 0x10400004, 0x0, 0x97a2001a,
-0x34420c00, 0xa7a2001a, 0x97a6001a, 0x24040004,
-0xc004ddb, 0x2821, 0x3c020004, 0x2421024,
-0x10400004, 0xa7a0001c, 0x32425000, 0x14400004,
-0x0, 0x32424000, 0x10400005, 0x2021,
-0xc004cf9, 0x2402021, 0x10000096, 0x0,
-0x97a6001c, 0x2821, 0x34c61200, 0xc004ddb,
-0xa7a6001c, 0x1000008f, 0x0, 0x2421024,
-0x10400004, 0xa7a00018, 0x32425000, 0x14400004,
-0x0, 0x32424000, 0x10400005, 0x3c020010,
-0xc004cf9, 0x2402021, 0x10000019, 0xa7a0001a,
-0x2421024, 0x10400004, 0x0, 0x97a20018,
-0x10000004, 0xa7a20018, 0x97a20018, 0x34420100,
-0xa7a20018, 0x3c020001, 0x2421024, 0x10400004,
-0x0, 0x97a20018, 0x10000004, 0xa7a20018,
-0x97a20018, 0x34422000, 0xa7a20018, 0x97a60018,
-0x2021, 0xc004ddb, 0x2821, 0xa7a0001a,
-0x8021, 0xc004d78, 0x24040001, 0x26100001,
-0x2e020020, 0x1440fffb, 0x0, 0xc004d78,
-0x2021, 0xc004d78, 0x24040001, 0xc004d78,
-0x24040001, 0xc004d78, 0x2021, 0x24100010,
-0x32020001, 0x10400002, 0x2021, 0x24040001,
-0xc004d78, 0x108042, 0x1600fffa, 0x32020001,
-0x24100010, 0xc004d78, 0x2021, 0x108042,
-0x1600fffc, 0x0, 0xc004db9, 0x34108000,
-0xc004db9, 0x0, 0xc004d58, 0x0,
-0x50400005, 0x108042, 0x97a2001a, 0x501025,
-0xa7a2001a, 0x108042, 0x1600fff7, 0x0,
-0xc004db9, 0x8021, 0xa7a0001a, 0xc004d78,
-0x24040001, 0x26100001, 0x2e020020, 0x1440fffb,
-0x0, 0xc004d78, 0x2021, 0xc004d78,
-0x24040001, 0xc004d78, 0x24040001, 0xc004d78,
-0x2021, 0x24100010, 0x32020001, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fffa, 0x32020001, 0x24100010, 0xc004d78,
-0x2021, 0x108042, 0x1600fffc, 0x0,
-0xc004db9, 0x34108000, 0xc004db9, 0x0,
-0xc004d58, 0x0, 0x50400005, 0x108042,
-0x97a2001a, 0x501025, 0xa7a2001a, 0x108042,
-0x1600fff7, 0x0, 0xc004db9, 0x0,
-0x3c040001, 0x24846bcc, 0x97a60018, 0x97a7001a,
-0x3c020001, 0x8c426d98, 0x3c030001, 0x8c636f1c,
-0x3c05000d, 0x34a50205, 0xafa20010, 0xc002b3b,
-0xafa30014, 0x8f830054, 0x24020004, 0x3c010001,
-0xac226dd4, 0x3c010001, 0x10000017, 0xac236f38,
-0x8f830054, 0x3c020001, 0x8c426f38, 0x2463ff9c,
-0x431023, 0x2c420064, 0x1440000f, 0x0,
-0x8f820220, 0x24030005, 0x3c010001, 0xac236dd4,
-0x3c03f700, 0x431025, 0x10000007, 0xaf820220,
-0x24020006, 0x3c010001, 0xac226dd4, 0x24020011,
-0x3c010001, 0xac226dd0, 0x8fbf0034, 0x8fb20030,
-0x8fb1002c, 0x8fb00028, 0x3e00008, 0x27bd0038,
-0x27bdffd8, 0xafb00018, 0x808021, 0xafb1001c,
-0x8821, 0x32024000, 0x10400013, 0xafbf0020,
-0x3c020010, 0x2021024, 0x2c420001, 0x21023,
-0x30434100, 0x3c020001, 0x2021024, 0x14400006,
-0x34714000, 0x3c020002, 0x2021024, 0x14400002,
-0x34716000, 0x34714040, 0x2021, 0x2821,
-0x10000036, 0x2203021, 0x32021000, 0x10400035,
-0x2021, 0x2821, 0xc004ddb, 0x24060040,
-0x24040018, 0x2821, 0xc004ddb, 0x24060c00,
-0x24040017, 0x2821, 0xc004ddb, 0x24060400,
-0x24040016, 0x2821, 0xc004ddb, 0x24060006,
-0x24040017, 0x2821, 0xc004ddb, 0x24062500,
-0x24040016, 0x2821, 0xc004ddb, 0x24060006,
-0x24040017, 0x2821, 0xc004ddb, 0x24064600,
-0x24040016, 0x2821, 0xc004ddb, 0x24060006,
-0x24040017, 0x2821, 0xc004ddb, 0x24066700,
-0x24040016, 0x2821, 0xc004ddb, 0x24060006,
-0x2404001f, 0x2821, 0xc004ddb, 0x24060010,
-0x24040009, 0x2821, 0xc004ddb, 0x24061500,
-0x24040009, 0x2821, 0x24061d00, 0xc004ddb,
-0x0, 0x3c040001, 0x24846bf0, 0x3c05000e,
-0x34a50100, 0x2003021, 0x2203821, 0xafa00010,
-0xc002b3b, 0xafa00014, 0x8fbf0020, 0x8fb1001c,
-0x8fb00018, 0x3e00008, 0x27bd0028, 0x8f850044,
-0x8f820044, 0x3c030001, 0x431025, 0x3c030008,
-0xaf820044, 0x8f840054, 0x8f820054, 0xa32824,
-0x10000002, 0x24840001, 0x8f820054, 0x821023,
-0x2c420002, 0x1440fffc, 0x0, 0x8f820044,
-0x3c03fffe, 0x3463ffff, 0x431024, 0xaf820044,
-0x8f830054, 0x8f820054, 0x10000002, 0x24630001,
-0x8f820054, 0x621023, 0x2c420002, 0x1440fffc,
-0x0, 0x3e00008, 0xa01021, 0x8f830044,
-0x3c02fff0, 0x3442ffff, 0x42480, 0x621824,
-0x3c020002, 0x822025, 0x641825, 0xaf830044,
-0x8f820044, 0x3c03fffe, 0x3463ffff, 0x431024,
-0xaf820044, 0x8f830054, 0x8f820054, 0x10000002,
-0x24630001, 0x8f820054, 0x621023, 0x2c420002,
-0x1440fffc, 0x0, 0x8f820044, 0x3c030001,
-0x431025, 0xaf820044, 0x8f830054, 0x8f820054,
-0x10000002, 0x24630001, 0x8f820054, 0x621023,
-0x2c420002, 0x1440fffc, 0x0, 0x3e00008,
-0x0, 0x8f820044, 0x2403ff7f, 0x431024,
-0xaf820044, 0x8f830054, 0x8f820054, 0x10000002,
-0x24630001, 0x8f820054, 0x621023, 0x2c420002,
-0x1440fffc, 0x0, 0x8f820044, 0x34420080,
-0xaf820044, 0x8f830054, 0x8f820054, 0x10000002,
-0x24630001, 0x8f820054, 0x621023, 0x2c420002,
-0x1440fffc, 0x0, 0x3e00008, 0x0,
-0x8f820044, 0x3c03fff0, 0x3463ffff, 0x431024,
-0xaf820044, 0x8f820044, 0x3c030001, 0x431025,
-0xaf820044, 0x8f830054, 0x8f820054, 0x10000002,
-0x24630001, 0x8f820054, 0x621023, 0x2c420002,
-0x1440fffc, 0x0, 0x8f820044, 0x3c03fffe,
-0x3463ffff, 0x431024, 0xaf820044, 0x8f830054,
-0x8f820054, 0x10000002, 0x24630001, 0x8f820054,
-0x621023, 0x2c420002, 0x1440fffc, 0x0,
-0x3e00008, 0x0, 0x27bdffc8, 0xafb30024,
-0x809821, 0xafbe002c, 0xa0f021, 0xafb20020,
-0xc09021, 0x33c2ffff, 0xafbf0030, 0xafb50028,
-0xafb1001c, 0xafb00018, 0x14400034, 0xa7b20010,
-0x3271ffff, 0x27b20010, 0x8021, 0xc004d78,
-0x24040001, 0x26100001, 0x2e020020, 0x1440fffb,
-0x0, 0xc004d78, 0x2021, 0xc004d78,
-0x24040001, 0xc004d78, 0x2021, 0xc004d78,
-0x24040001, 0x24100010, 0x32020001, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fffa, 0x32020001, 0x24100010, 0x2301024,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x2301024, 0xc004d78,
-0x24040001, 0xc004d78, 0x2021, 0x34108000,
-0x96420000, 0x501024, 0x10400002, 0x2021,
-0x24040001, 0xc004d78, 0x108042, 0x12000075,
-0x0, 0x1000fff6, 0x0, 0x3275ffff,
-0x27b10010, 0xa7a00010, 0x8021, 0xc004d78,
-0x24040001, 0x26100001, 0x2e020020, 0x1440fffb,
-0x0, 0xc004d78, 0x2021, 0xc004d78,
-0x24040001, 0xc004d78, 0x24040001, 0xc004d78,
-0x2021, 0x24100010, 0x32020001, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fffa, 0x32020001, 0x24100010, 0x2b01024,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x2b01024, 0xc004db9,
-0x34108000, 0xc004db9, 0x0, 0xc004d58,
-0x0, 0x50400005, 0x108042, 0x96220000,
-0x501025, 0xa6220000, 0x108042, 0x1600fff7,
-0x0, 0xc004db9, 0x0, 0x33c5ffff,
-0x24020001, 0x54a20004, 0x24020002, 0x97a20010,
-0x10000006, 0x521025, 0x14a20006, 0x3271ffff,
-0x97a20010, 0x121827, 0x431024, 0xa7a20010,
-0x3271ffff, 0x27b20010, 0x8021, 0xc004d78,
-0x24040001, 0x26100001, 0x2e020020, 0x1440fffb,
-0x0, 0xc004d78, 0x2021, 0xc004d78,
-0x24040001, 0xc004d78, 0x2021, 0xc004d78,
-0x24040001, 0x24100010, 0x32020001, 0x10400002,
-0x2021, 0x24040001, 0xc004d78, 0x108042,
-0x1600fffa, 0x32020001, 0x24100010, 0x2301024,
-0x10400002, 0x2021, 0x24040001, 0xc004d78,
-0x108042, 0x1600fffa, 0x2301024, 0xc004d78,
-0x24040001, 0xc004d78, 0x2021, 0x34108000,
-0x96420000, 0x501024, 0x10400002, 0x2021,
-0x24040001, 0xc004d78, 0x108042, 0x1600fff8,
-0x0, 0xc004db9, 0x0, 0x8fbf0030,
-0x8fbe002c, 0x8fb50028, 0x8fb30024, 0x8fb20020,
-0x8fb1001c, 0x8fb00018, 0x3e00008, 0x27bd0038,
-0x0, 0x0, 0x0, 0x27bdffe8,
-0xafbf0010, 0x8ee304b8, 0x24020008, 0x146201e0,
-0x0, 0x3c020001, 0x8c426f1c, 0x14400005,
-0x0, 0xc003daf, 0x8f840224, 0x100001d8,
-0x0, 0x8f820220, 0x3c030008, 0x431024,
-0x10400026, 0x24020001, 0x8f840224, 0x8f820220,
-0x3c030400, 0x431024, 0x10400006, 0x0,
-0x3c010002, 0xac208fa0, 0x3c010002, 0x1000000b,
-0xac208fc0, 0x3c030002, 0x24638fa0, 0x8c620000,
-0x24420001, 0xac620000, 0x2c420002, 0x14400003,
-0x24020001, 0x3c010002, 0xac228fc0, 0x3c020002,
-0x8c428fc0, 0x10400006, 0x30820040, 0x10400004,
-0x24020001, 0x3c010002, 0x10000003, 0xac228fc4,
-0x3c010002, 0xac208fc4, 0x3c010002, 0xac248f9c,
-0x3c010002, 0x1000000b, 0xac208fd0, 0x3c010002,
-0xac228fd0, 0x3c010002, 0xac208fc0, 0x3c010002,
-0xac208fa0, 0x3c010002, 0xac208fc4, 0x3c010002,
-0xac208f9c, 0x3c030002, 0x8c638f90, 0x3c020002,
-0x8c428f94, 0x50620004, 0x2463ffff, 0x3c010002,
-0xac238f94, 0x2463ffff, 0x2c62000e, 0x10400194,
-0x31080, 0x3c010001, 0x220821, 0x8c226c00,
-0x400008, 0x0, 0x24020002, 0x3c010002,
-0xac208fc0, 0x3c010002, 0xac208fa0, 0x3c010002,
-0xac208f9c, 0x3c010002, 0xac208fc4, 0x3c010002,
-0xac208fb8, 0x3c010002, 0xac208fb0, 0xaf800224,
-0x3c010002, 0xac228f90, 0x3c020002, 0x8c428fd0,
-0x1440004f, 0x3c02fdff, 0x3442ffff, 0xc003daf,
-0x282a024, 0xaf800204, 0x8f820200, 0x2403fffd,
-0x431024, 0xaf820200, 0x3c010002, 0xac208fe0,
-0x8f830054, 0x3c020002, 0x8c428fb8, 0x24040001,
-0x3c010002, 0xac248fcc, 0x24420001, 0x3c010002,
-0xac228fb8, 0x2c420004, 0x3c010002, 0xac238fb4,
-0x14400006, 0x24020003, 0x3c010001, 0xac246d9c,
-0x3c010002, 0x1000015e, 0xac208fb8, 0x3c010002,
-0x1000015b, 0xac228f90, 0x8f830054, 0x3c020002,
-0x8c428fb4, 0x2463d8f0, 0x431023, 0x2c422710,
-0x14400003, 0x24020004, 0x3c010002, 0xac228f90,
-0x3c020002, 0x8c428fd0, 0x14400021, 0x3c02fdff,
-0x3442ffff, 0x1000014a, 0x282a024, 0x3c040001,
-0x8c846f20, 0x3c010002, 0xc005084, 0xac208fa8,
-0x3c020002, 0x8c428fdc, 0xaf820204, 0x3c020002,
-0x8c428fd0, 0x14400012, 0x3c03fdff, 0x8f820204,
-0x3463ffff, 0x30420030, 0x1440012f, 0x283a024,
-0x3c030002, 0x8c638fdc, 0x24020005, 0x3c010002,
-0xac228f90, 0x3c010002, 0x10000131, 0xac238fe0,
-0x3c020002, 0x8c428fd0, 0x10400010, 0x3c02fdff,
-0x3c020001, 0x8c426e3c, 0x24420001, 0x3c010001,
-0xac226e3c, 0x2c420002, 0x14400125, 0x24020001,
-0x3c010001, 0xac226e44, 0x3c010001, 0xac206e3c,
-0x3c010001, 0x1000011e, 0xac226d9c, 0x3c030002,
-0x8c638fc0, 0x3442ffff, 0x10600119, 0x282a024,
-0x3c020002, 0x8c428f9c, 0x10400115, 0x0,
-0x3c010002, 0xac228fc8, 0x24020003, 0x3c010002,
-0xac228fa0, 0x100000b8, 0x24020006, 0x3c010002,
-0xac208fa8, 0x8f820204, 0x34420040, 0xaf820204,
-0x3c020002, 0x8c428fe0, 0x24030007, 0x3c010002,
-0xac238f90, 0x34420040, 0x3c010002, 0xac228fe0,
-0x3c020002, 0x8c428fc0, 0x10400005, 0x0,
-0x3c020002, 0x8c428f9c, 0x104000f0, 0x24020002,
-0x3c050002, 0x24a58fa0, 0x8ca20000, 0x2c424e21,
-0x104000ea, 0x24020002, 0x3c020002, 0x8c428fc4,
-0x104000ef, 0x2404ffbf, 0x3c020002, 0x8c428f9c,
-0x3c030002, 0x8c638fc8, 0x441024, 0x641824,
-0x10430004, 0x24020001, 0x3c010002, 0x100000e4,
-0xac228f90, 0x24020003, 0xaca20000, 0x24020008,
-0x3c010002, 0xac228f90, 0x3c020002, 0x8c428fcc,
-0x1040000c, 0x24020001, 0x3c040002, 0xc005091,
-0x8c848f9c, 0x3c020002, 0x8c428fe8, 0x14400005,
-0x24020001, 0x3c020002, 0x8c428fe4, 0x10400006,
-0x24020001, 0x3c010001, 0xac226d9c, 0x3c010002,
-0x100000cb, 0xac208fb8, 0x3c020002, 0x8c428fb0,
-0x3c030002, 0x8c638f9c, 0x2c420001, 0x210c0,
-0x30630008, 0x3c010002, 0xac228fb0, 0x3c010002,
-0xac238fac, 0x8f830054, 0x24020009, 0x3c010002,
-0xac228f90, 0x3c010002, 0x100000b9, 0xac238fb4,
-0x8f830054, 0x3c020002, 0x8c428fb4, 0x2463d8f0,
-0x431023, 0x2c422710, 0x1440009f, 0x0,
-0x3c020002, 0x8c428fc0, 0x10400005, 0x0,
-0x3c020002, 0x8c428f9c, 0x104000a0, 0x24020002,
-0x3c030002, 0x24638fa0, 0x8c620000, 0x2c424e21,
-0x1040009a, 0x24020002, 0x3c020002, 0x8c428fcc,
-0x1040000e, 0x0, 0x3c020002, 0x8c428f9c,
-0x3c010002, 0xac208fcc, 0x30420080, 0x1040002f,
-0x2402000c, 0x8f820204, 0x30420080, 0x1440000c,
-0x24020003, 0x10000029, 0x2402000c, 0x3c020002,
-0x8c428f9c, 0x30420080, 0x14400005, 0x24020003,
-0x8f820204, 0x30420080, 0x1040001f, 0x24020003,
-0xac620000, 0x2402000a, 0x3c010002, 0xac228f90,
-0x3c040002, 0x24848fd8, 0x8c820000, 0x3c030002,
-0x8c638fb0, 0x431025, 0xaf820204, 0x8c830000,
-0x3c040002, 0x8c848fb0, 0x2402000b, 0x3c010002,
-0xac228f90, 0x641825, 0x3c010002, 0xac238fe0,
-0x3c050002, 0x24a58fa0, 0x8ca20000, 0x2c424e21,
-0x10400066, 0x24020002, 0x3c020002, 0x8c428fd0,
-0x10400005, 0x0, 0x2402000c, 0x3c010002,
-0x10000067, 0xac228f90, 0x3c020002, 0x8c428fc0,
-0x10400063, 0x0, 0x3c040002, 0x8c848f9c,
-0x10800055, 0x30820008, 0x3c030002, 0x8c638fac,
-0x1062005b, 0x24020003, 0x3c010002, 0xac248fc8,
-0xaca20000, 0x24020006, 0x3c010002, 0x10000054,
-0xac228f90, 0x8f820200, 0x34420002, 0xaf820200,
-0x8f830054, 0x2402000d, 0x3c010002, 0xac228f90,
-0x3c010002, 0xac238fb4, 0x8f830054, 0x3c020002,
-0x8c428fb4, 0x2463d8f0, 0x431023, 0x2c422710,
-0x14400031, 0x0, 0x3c020002, 0x8c428fd0,
-0x10400020, 0x2402000e, 0x3c030002, 0x8c638fe4,
-0x3c010002, 0x14600015, 0xac228f90, 0xc003e6d,
-0x0, 0x3c050001, 0x8ca56d98, 0xc00529b,
-0x2021, 0x3c030001, 0x8c636d98, 0x24020004,
-0x14620005, 0x2403fffb, 0x3c020001, 0x8c426d94,
-0x10000003, 0x2403fff7, 0x3c020001, 0x8c426d94,
-0x431024, 0x3c010001, 0xac226d94, 0x8f830224,
-0x3c020200, 0x3c010002, 0xac238fec, 0x10000020,
-0x282a025, 0x3c020002, 0x8c428fc0, 0x10400005,
-0x0, 0x3c020002, 0x8c428f9c, 0x1040000f,
-0x24020002, 0x3c020002, 0x8c428fa0, 0x2c424e21,
-0x1040000a, 0x24020002, 0x3c020002, 0x8c428fc0,
-0x1040000f, 0x0, 0x3c020002, 0x8c428f9c,
-0x1440000b, 0x0, 0x24020002, 0x3c010002,
-0x10000007, 0xac228f90, 0x3c020002, 0x8c428fc0,
-0x10400003, 0x0, 0xc003daf, 0x0,
-0x8f820220, 0x3c03f700, 0x431025, 0xaf820220,
-0x8fbf0010, 0x3e00008, 0x27bd0018, 0x3c030002,
-0x24638fe8, 0x8c620000, 0x10400005, 0x34422000,
-0x3c010002, 0xac228fdc, 0x10000003, 0xac600000,
-0x3c010002, 0xac248fdc, 0x3e00008, 0x0,
-0x27bdffe0, 0x30820030, 0xafbf0018, 0x3c010002,
-0xac228fe4, 0x14400067, 0x3c02ffff, 0x34421f0e,
-0x821024, 0x14400061, 0x24020030, 0x30822000,
-0x1040005d, 0x30838000, 0x31a02, 0x30820001,
-0x21200, 0x3c040001, 0x8c846f20, 0x621825,
-0x331c2, 0x3c030001, 0x24636e48, 0x30828000,
-0x21202, 0x30840001, 0x42200, 0x441025,
-0x239c2, 0x61080, 0x431021, 0x471021,
-0x90430000, 0x24020001, 0x10620025, 0x0,
-0x10600007, 0x24020002, 0x10620013, 0x24020003,
-0x1062002c, 0x3c05000f, 0x10000037, 0x0,
-0x8f820200, 0x2403feff, 0x431024, 0xaf820200,
-0x8f820220, 0x3c03fffe, 0x3463ffff, 0x431024,
-0xaf820220, 0x3c010002, 0xac209004, 0x3c010002,
-0x10000034, 0xac20900c, 0x8f820200, 0x34420100,
-0xaf820200, 0x8f820220, 0x3c03fffe, 0x3463ffff,
-0x431024, 0xaf820220, 0x24020100, 0x3c010002,
-0xac229004, 0x3c010002, 0x10000026, 0xac20900c,
-0x8f820200, 0x2403feff, 0x431024, 0xaf820200,
-0x8f820220, 0x3c030001, 0x431025, 0xaf820220,
-0x3c010002, 0xac209004, 0x3c010002, 0x10000019,
-0xac23900c, 0x8f820200, 0x34420100, 0xaf820200,
-0x8f820220, 0x3c030001, 0x431025, 0xaf820220,
-0x24020100, 0x3c010002, 0xac229004, 0x3c010002,
-0x1000000c, 0xac23900c, 0x34a5ffff, 0x3c040001,
-0x24846c38, 0xafa30010, 0xc002b3b, 0xafa00014,
-0x10000004, 0x0, 0x24020030, 0x3c010002,
-0xac228fe8, 0x8fbf0018, 0x3e00008, 0x27bd0020,
-0x0, 0x0, 0x0, 0x27bdffc8,
-0xafb20028, 0x809021, 0xafb3002c, 0xa09821,
-0xafb00020, 0xc08021, 0x3c040001, 0x24846c50,
-0x3c050009, 0x3c020001, 0x8c426d98, 0x34a59001,
-0x2403021, 0x2603821, 0xafbf0030, 0xafb10024,
-0xa7a0001a, 0xafb00014, 0xc002b3b, 0xafa20010,
-0x24020002, 0x12620083, 0x2e620003, 0x10400005,
-0x24020001, 0x1262000a, 0x0, 0x10000173,
-0x0, 0x24020004, 0x126200f8, 0x24020008,
-0x126200f7, 0x3c02ffec, 0x1000016c, 0x0,
-0x3c020001, 0x8c426d94, 0x30420002, 0x14400004,
-0x128940, 0x3c02fffb, 0x3442ffff, 0x2028024,
-0x3c010002, 0x310821, 0xac308ffc, 0x3c024000,
-0x2021024, 0x1040004e, 0x1023c2, 0x30840030,
-0x101382, 0x3042001c, 0x3c030001, 0x24636dd8,
-0x431021, 0x823821, 0x3c020020, 0x2021024,
-0x10400006, 0x24020100, 0x3c010002, 0x310821,
-0xac229000, 0x10000005, 0x3c020080, 0x3c010002,
-0x310821, 0xac209000, 0x3c020080, 0x2021024,
-0x10400006, 0x121940, 0x3c020001, 0x3c010002,
-0x230821, 0x10000005, 0xac229008, 0x121140,
-0x3c010002, 0x220821, 0xac209008, 0x94e40000,
-0x3c030001, 0x8c636f40, 0x24020005, 0x10620010,
-0xa7a40018, 0x32024000, 0x10400002, 0x34824000,
-0xa7a20018, 0x24040001, 0x94e20002, 0x24050004,
-0x24e60002, 0x34420001, 0xc0045be, 0xa4e20002,
-0x24040001, 0x2821, 0xc0045be, 0x27a60018,
-0x3c020001, 0x8c426d98, 0x24110001, 0x3c010001,
-0xac316da4, 0x14530004, 0x32028000, 0xc003daf,
-0x0, 0x32028000, 0x1040011c, 0x0,
-0xc003daf, 0x0, 0x3c030001, 0x8c636f40,
-0x24020005, 0x10620115, 0x24020002, 0x3c010001,
-0xac316d9c, 0x3c010001, 0x10000110, 0xac226d98,
-0x24040001, 0x24050004, 0x27b0001a, 0xc0045be,
-0x2003021, 0x24040001, 0x2821, 0xc0045be,
-0x2003021, 0x3c020002, 0x511021, 0x8c428ff4,
-0x3c040001, 0x8c846d98, 0x3c03bfff, 0x3463ffff,
-0x3c010001, 0xac336da4, 0x431024, 0x3c010002,
-0x310821, 0x109300f7, 0xac228ff4, 0x100000f7,
-0x0, 0x3c022000, 0x2021024, 0x10400005,
-0x24020001, 0x3c010001, 0xac226f1c, 0x10000004,
-0x128940, 0x3c010001, 0xac206f1c, 0x128940,
-0x3c010002, 0x310821, 0xac308ff8, 0x3c024000,
-0x2021024, 0x14400014, 0x0, 0x3c020001,
-0x8c426f1c, 0x10400006, 0x24040004, 0x24050001,
-0xc004ddb, 0x24062000, 0x24020001, 0xaee204b8,
-0x3c020002, 0x511021, 0x8c428ff0, 0x3c03bfff,
-0x3463ffff, 0x431024, 0x3c010002, 0x310821,
-0x100000d0, 0xac228ff0, 0x3c020001, 0x8c426f1c,
-0x10400028, 0x3c0300a0, 0x2031024, 0x5443000d,
-0x3c020020, 0x3c020001, 0x8c426f20, 0x24030100,
-0x3c010002, 0x310821, 0xac239004, 0x3c030001,
-0x3c010002, 0x310821, 0xac23900c, 0x10000015,
-0x34420400, 0x2021024, 0x10400008, 0x24030100,
-0x3c020001, 0x8c426f20, 0x3c010002, 0x310821,
-0xac239004, 0x1000000b, 0x34420800, 0x3c020080,
-0x2021024, 0x1040002e, 0x3c030001, 0x3c020001,
-0x8c426f20, 0x3c010002, 0x310821, 0xac23900c,
-0x34420c00, 0x3c010001, 0xac226f20, 0x10000025,
-0x24040001, 0x3c020020, 0x2021024, 0x10400006,
-0x24020100, 0x3c010002, 0x310821, 0xac229004,
-0x10000005, 0x3c020080, 0x3c010002, 0x310821,
-0xac209004, 0x3c020080, 0x2021024, 0x10400007,
-0x121940, 0x3c020001, 0x3c010002, 0x230821,
-0xac22900c, 0x10000006, 0x24040001, 0x121140,
-0x3c010002, 0x220821, 0xac20900c, 0x24040001,
-0x2821, 0x27b0001e, 0xc00457c, 0x2003021,
-0x24040001, 0x2821, 0xc00457c, 0x2003021,
-0x24040001, 0x24050001, 0x27b0001c, 0xc00457c,
-0x2003021, 0x24040001, 0x24050001, 0xc00457c,
-0x2003021, 0x10000077, 0x0, 0x3c02ffec,
-0x3442ffff, 0x2028024, 0x3c020008, 0x2028025,
-0x121140, 0x3c010002, 0x220821, 0xac308ff8,
-0x3c022000, 0x2021024, 0x10400009, 0x0,
-0x3c020001, 0x8c426e44, 0x14400005, 0x24020001,
-0x3c010001, 0xac226f1c, 0x10000004, 0x3c024000,
-0x3c010001, 0xac206f1c, 0x3c024000, 0x2021024,
-0x1440001d, 0x24020e01, 0x3c030001, 0x8c636f1c,
-0xaf820238, 0x3c010001, 0xac206db0, 0x10600005,
-0x24022020, 0x3c010001, 0xac226f20, 0x24020001,
-0xaee204b8, 0x3c04bfff, 0x121940, 0x3c020002,
-0x431021, 0x8c428ff0, 0x3c050001, 0x8ca56d98,
-0x3484ffff, 0x441024, 0x3c010002, 0x230821,
-0xac228ff0, 0x24020001, 0x10a20044, 0x0,
-0x10000040, 0x0, 0x3c020001, 0x8c426f1c,
-0x1040001c, 0x24022000, 0x3c010001, 0xac226f20,
-0x3c0300a0, 0x2031024, 0x14430005, 0x121140,
-0x3402a000, 0x3c010001, 0x1000002d, 0xac226f20,
-0x3c030002, 0x621821, 0x8c638ff8, 0x3c020020,
-0x621024, 0x10400004, 0x24022001, 0x3c010001,
-0x10000023, 0xac226f20, 0x3c020080, 0x621024,
-0x1040001f, 0x3402a001, 0x3c010001, 0x1000001c,
-0xac226f20, 0x3c020020, 0x2021024, 0x10400007,
-0x121940, 0x24020100, 0x3c010002, 0x230821,
-0xac229004, 0x10000006, 0x3c020080, 0x121140,
-0x3c010002, 0x220821, 0xac209004, 0x3c020080,
-0x2021024, 0x10400006, 0x121940, 0x3c020001,
-0x3c010002, 0x230821, 0x10000005, 0xac22900c,
-0x121140, 0x3c010002, 0x220821, 0xac20900c,
-0x3c030001, 0x8c636d98, 0x24020001, 0x10620003,
-0x0, 0xc003daf, 0x0, 0x8fbf0030,
-0x8fb3002c, 0x8fb20028, 0x8fb10024, 0x8fb00020,
-0x3e00008, 0x27bd0038, 0x27bdffb0, 0xafb3003c,
-0x9821, 0xafb50040, 0xa821, 0xafb10034,
-0x8821, 0x24020002, 0xafbf0048, 0xafbe0044,
-0xafb20038, 0xafb00030, 0xafa4002c, 0xa7a0001a,
-0xa7a00018, 0xa7a00020, 0xa7a0001e, 0xa7a00022,
-0x10a20130, 0xa7a0001c, 0x2ca20003, 0x10400005,
-0x24020001, 0x10a2000a, 0x3c024000, 0x1000025d,
-0x2201021, 0x24020004, 0x10a2020a, 0x24020008,
-0x10a20208, 0x2201021, 0x10000256, 0x0,
-0x8fa8002c, 0x88140, 0x3c030002, 0x701821,
-0x8c638ffc, 0x621024, 0x14400009, 0x24040001,
-0x3c027fff, 0x3442ffff, 0x628824, 0x3c010002,
-0x300821, 0xac318ff4, 0x10000246, 0x2201021,
-0x24050001, 0xc00457c, 0x27a60018, 0x24040001,
-0x24050001, 0xc00457c, 0x27a60018, 0x97a20018,
-0x30420004, 0x104000d9, 0x3c114000, 0x3c020001,
-0x8c426f40, 0x2443ffff, 0x2c620006, 0x104000d9,
-0x31080, 0x3c010001, 0x220821, 0x8c226c68,
-0x400008, 0x0, 0x24040001, 0x24050011,
-0x27b0001a, 0xc00457c, 0x2003021, 0x24040001,
-0x24050011, 0xc00457c, 0x2003021, 0x97a3001a,
-0x30624000, 0x10400002, 0x3c150010, 0x3c150008,
-0x30628000, 0x104000aa, 0x3c130001, 0x100000a8,
-0x3c130002, 0x24040001, 0x24050014, 0x27b0001a,
-0xc00457c, 0x2003021, 0x24040001, 0x24050014,
-0xc00457c, 0x2003021, 0x97a3001a, 0x30621000,
-0x10400002, 0x3c150010, 0x3c150008, 0x30620800,
-0x10400097, 0x3c130001, 0x10000095, 0x3c130002,
-0x24040001, 0x24050019, 0x27b0001c, 0xc00457c,
-0x2003021, 0x24040001, 0x24050019, 0xc00457c,
-0x2003021, 0x97a2001c, 0x30430700, 0x24020400,
-0x10620027, 0x28620401, 0x1040000e, 0x24020200,
-0x1062001f, 0x28620201, 0x10400005, 0x24020100,
-0x5062001e, 0x3c130001, 0x1000001e, 0x24040001,
-0x24020300, 0x50620019, 0x3c130002, 0x10000019,
-0x24040001, 0x24020600, 0x1062000d, 0x28620601,
-0x10400005, 0x24020500, 0x5062000b, 0x3c130002,
-0x10000010, 0x24040001, 0x24020700, 0x1462000d,
-0x24040001, 0x3c130004, 0x1000000a, 0x3c150008,
-0x10000006, 0x3c130004, 0x10000005, 0x3c150008,
-0x3c130001, 0x10000002, 0x3c150008, 0x3c150010,
-0x24040001, 0x24050018, 0x27b0001e, 0xc00457c,
-0x2003021, 0x24040001, 0x24050018, 0xc00457c,
-0x2003021, 0x8fa8002c, 0x97a7001e, 0x81140,
-0x3c060002, 0xc23021, 0x8cc68ff4, 0x97a20022,
-0x3c100001, 0x26106c5c, 0x2002021, 0xafa20010,
-0x97a2001c, 0x3c05000c, 0x34a50303, 0xc002b3b,
-0xafa20014, 0x3c020004, 0x16620010, 0x3c020001,
-0x8f840054, 0x24030001, 0x24020002, 0x3c010001,
-0xac236d9c, 0x3c010001, 0xac226d98, 0x3c010001,
-0xac236da4, 0x3c010001, 0xac236e24, 0x3c010001,
-0xac246f30, 0x1000004f, 0x2b38825, 0x16620039,
-0x3c028000, 0x3c020001, 0x8c426e20, 0x1440001e,
-0x24040018, 0x2021, 0x2821, 0xc004ddb,
-0x34068000, 0x8f830054, 0x8f820054, 0x2b38825,
-0x10000002, 0x24630032, 0x8f820054, 0x621023,
-0x2c420033, 0x1440fffc, 0x0, 0x8f830054,
-0x24020001, 0x3c010001, 0xac226e20, 0x3c010001,
-0xac226d9c, 0x3c010001, 0xac226d98, 0x3c010001,
-0xac226da4, 0x3c010001, 0xac226e24, 0x3c010001,
-0x1000002c, 0xac236f30, 0x2821, 0xc004ddb,
-0x24060404, 0x2021, 0x2405001e, 0x27a60018,
-0x24020002, 0xc0045be, 0xa7a20018, 0x2021,
-0x2821, 0x27a60018, 0xc0045be, 0xa7a00018,
-0x24040018, 0x24050002, 0xc004ddb, 0x24060004,
-0x3c028000, 0x2221025, 0x2b31825, 0x10000015,
-0x438825, 0x2221025, 0x2751825, 0x438825,
-0x2002021, 0x97a6001c, 0x3c070001, 0x8ce76d98,
-0x3c05000c, 0x34a50326, 0xafb30010, 0xc002b3b,
-0xafb10014, 0x10000007, 0x0, 0x3c110002,
-0x2308821, 0x8e318ffc, 0x3c027fff, 0x3442ffff,
-0x2228824, 0x3c020001, 0x8c426da8, 0x1040001e,
-0x0, 0x3c020001, 0x8c426f1c, 0x10400002,
-0x3c022000, 0x2228825, 0x8fa8002c, 0x81140,
-0x3c010002, 0x220821, 0x8c229000, 0x10400003,
-0x3c020020, 0x10000005, 0x2228825, 0x3c02ffdf,
-0x3442ffff, 0x2228824, 0x8fa8002c, 0x81140,
-0x3c010002, 0x220821, 0x8c229008, 0x10400003,
-0x3c020080, 0x10000004, 0x2228825, 0x3c02ff7f,
-0x3442ffff, 0x2228824, 0x8fa8002c, 0x81140,
-0x3c010002, 0x220821, 0xac318ff4, 0x10000135,
-0x2201021, 0x8fa8002c, 0x8f140, 0x3c030002,
-0x7e1821, 0x8c638ff8, 0x3c024000, 0x621024,
-0x14400009, 0x24040001, 0x3c027fff, 0x3442ffff,
-0x628824, 0x3c010002, 0x3e0821, 0xac318ff0,
-0x10000124, 0x2201021, 0x2821, 0xc00457c,
-0x27a60018, 0x24040001, 0x2821, 0xc00457c,
-0x27a60018, 0x24040001, 0x24050001, 0x27b20020,
-0xc00457c, 0x2403021, 0x24040001, 0x24050001,
-0xc00457c, 0x2403021, 0x24040001, 0x24050004,
-0x27b1001e, 0xc00457c, 0x2203021, 0x24040001,
-0x24050004, 0xc00457c, 0x2203021, 0x24040001,
-0x24050005, 0x27b00022, 0xc00457c, 0x2003021,
-0x24040001, 0x24050005, 0xc00457c, 0x2003021,
-0x24040001, 0x24050010, 0xc00457c, 0x27a60018,
-0x24040001, 0x24050010, 0xc00457c, 0x27a60018,
-0x24040001, 0x2405000a, 0xc00457c, 0x2403021,
-0x24040001, 0x2405000a, 0xc00457c, 0x2403021,
-0x24040001, 0x24050018, 0xc00457c, 0x2203021,
-0x24040001, 0x24050018, 0xc00457c, 0x2203021,
-0x24040001, 0x24050001, 0xc00457c, 0x27a60018,
-0x24040001, 0x24050001, 0xc00457c, 0x27a60018,
-0x97a20018, 0x30420004, 0x10400066, 0x3c114000,
-0x3c030001, 0x8c636f34, 0x24020005, 0x14620067,
-0x24040001, 0x24050019, 0x27b0001c, 0xc00457c,
-0x2003021, 0x24040001, 0x24050019, 0xc00457c,
-0x2003021, 0x97a2001c, 0x30430700, 0x24020400,
-0x10620027, 0x28620401, 0x1040000e, 0x24020200,
-0x1062001f, 0x28620201, 0x10400005, 0x24020100,
-0x5062001e, 0x3c130001, 0x1000001e, 0x3c020004,
-0x24020300, 0x50620019, 0x3c130002, 0x10000019,
-0x3c020004, 0x24020600, 0x1062000d, 0x28620601,
-0x10400005, 0x24020500, 0x5062000b, 0x3c130002,
-0x10000010, 0x3c020004, 0x24020700, 0x1462000d,
-0x3c020004, 0x3c130004, 0x1000000a, 0x3c150008,
-0x10000006, 0x3c130004, 0x10000005, 0x3c150008,
-0x3c130001, 0x10000002, 0x3c150008, 0x3c150010,
-0x3c020004, 0x12620017, 0x3c028000, 0x8f820054,
-0x24100001, 0x3c010001, 0xac306d9c, 0x3c010001,
-0xac306d98, 0x3c010001, 0xac306da4, 0x3c010001,
-0xac306e24, 0x3c010001, 0xac226f30, 0x3c020001,
-0x16620022, 0x2758825, 0x2021, 0x2821,
-0xc004ddb, 0x34068000, 0x3c010001, 0x1000001b,
-0xac306e20, 0x2221025, 0x2b31825, 0x438825,
-0x97a6001c, 0x3c020001, 0x8c426f1c, 0x3c070001,
-0x8ce76d98, 0x3c040001, 0x24846c5c, 0xafa20010,
-0x97a2001e, 0x3c05000c, 0x34a50323, 0x3c010001,
-0xac206e20, 0xc002b3b, 0xafa20014, 0x10000007,
-0x0, 0x3c110002, 0x23e8821, 0x8e318ff0,
-0x3c027fff, 0x3442ffff, 0x2228824, 0x3c020001,
-0x8c426da8, 0x10400069, 0x0, 0x3c020001,
-0x8c426f1c, 0x10400002, 0x3c022000, 0x2228825,
-0x8fa8002c, 0x81140, 0x3c010002, 0x220821,
-0x8c229004, 0x10400003, 0x3c020020, 0x10000005,
-0x2228825, 0x3c02ffdf, 0x3442ffff, 0x2228824,
-0x8fa8002c, 0x81140, 0x3c010002, 0x220821,
-0x8c22900c, 0x10400003, 0x3c020080, 0x1000004f,
-0x2228825, 0x3c02ff7f, 0x3442ffff, 0x1000004b,
-0x2228824, 0x8fa8002c, 0x82940, 0x3c030002,
-0x651821, 0x8c638ff8, 0x3c024000, 0x621024,
-0x14400008, 0x3c027fff, 0x3442ffff, 0x628824,
-0x3c010002, 0x250821, 0xac318ff0, 0x10000041,
-0x2201021, 0x3c020001, 0x8c426da8, 0x10400034,
-0x3c11c00c, 0x3c020001, 0x8c426e44, 0x3c04c00c,
-0x34842000, 0x3c030001, 0x8c636f1c, 0x2102b,
-0x21023, 0x441024, 0x10600003, 0x518825,
-0x3c022000, 0x2228825, 0x3c020002, 0x451021,
-0x8c429004, 0x10400003, 0x3c020020, 0x10000004,
-0x2228825, 0x3c02ffdf, 0x3442ffff, 0x2228824,
-0x8fa8002c, 0x81140, 0x3c010002, 0x220821,
-0x8c22900c, 0x10400003, 0x3c020080, 0x10000004,
-0x2228825, 0x3c02ff7f, 0x3442ffff, 0x2228824,
-0x3c020001, 0x8c426e30, 0x10400002, 0x3c020800,
-0x2228825, 0x3c020001, 0x8c426e34, 0x10400002,
-0x3c020400, 0x2228825, 0x3c020001, 0x8c426e38,
-0x10400006, 0x3c020100, 0x10000004, 0x2228825,
-0x3c027fff, 0x3442ffff, 0x628824, 0x8fa8002c,
-0x81140, 0x3c010002, 0x220821, 0xac318ff0,
-0x2201021, 0x8fbf0048, 0x8fbe0044, 0x8fb50040,
-0x8fb3003c, 0x8fb20038, 0x8fb10034, 0x8fb00030,
-0x3e00008, 0x27bd0050, 0x27bdffd0, 0xafb20028,
-0x809021, 0xafbf002c, 0xafb10024, 0xafb00020,
-0x8f840200, 0x3c100001, 0x8e106d98, 0x8f860220,
-0x24020002, 0x1202005c, 0x2e020003, 0x10400005,
-0x24020001, 0x1202000a, 0x121940, 0x1000010c,
-0x0, 0x24020004, 0x120200bf, 0x24020008,
-0x120200be, 0x128940, 0x10000105, 0x0,
-0x3c050002, 0xa32821, 0x8ca58ffc, 0x3c100002,
-0x2038021, 0x8e108ff4, 0x3c024000, 0xa21024,
-0x10400038, 0x3c020008, 0x2021024, 0x10400020,
-0x34840002, 0x3c020002, 0x431021, 0x8c429000,
-0x10400005, 0x34840020, 0x34840100, 0x3c020020,
-0x10000006, 0x2028025, 0x2402feff, 0x822024,
-0x3c02ffdf, 0x3442ffff, 0x2028024, 0x121140,
-0x3c010002, 0x220821, 0x8c229008, 0x10400005,
-0x3c020001, 0xc23025, 0x3c020080, 0x10000016,
-0x2028025, 0x3c02fffe, 0x3442ffff, 0xc23024,
-0x3c02ff7f, 0x3442ffff, 0x1000000f, 0x2028024,
-0x2402fedf, 0x822024, 0x3c02fffe, 0x3442ffff,
-0xc23024, 0x3c02ff5f, 0x3442ffff, 0x2028024,
-0x3c010002, 0x230821, 0xac209000, 0x3c010002,
-0x230821, 0xac209008, 0xaf840200, 0xaf860220,
-0x8f820220, 0x34420002, 0xaf820220, 0x1000000a,
-0x121140, 0x3c02bfff, 0x3442ffff, 0x8f830200,
-0x2028024, 0x2402fffd, 0x621824, 0xc003daf,
-0xaf830200, 0x121140, 0x3c010002, 0x220821,
-0x100000b7, 0xac308ff4, 0x3c020001, 0x8c426f1c,
-0x10400069, 0x24050004, 0x24040001, 0xc00457c,
-0x27a60018, 0x24040001, 0x24050005, 0xc00457c,
-0x27a6001a, 0x97a30018, 0x97a2001a, 0x3c040001,
-0x24846e48, 0x30630c00, 0x31a82, 0x30420c00,
-0x21282, 0xa7a2001a, 0x21080, 0x441021,
-0x431021, 0xa7a30018, 0x90480000, 0x24020001,
-0x3103ffff, 0x10620029, 0x28620002, 0x10400005,
-0x0, 0x10600009, 0x0, 0x1000003d,
-0x0, 0x10700013, 0x24020003, 0x1062002c,
-0x0, 0x10000037, 0x0, 0x8f820200,
-0x2403feff, 0x431024, 0xaf820200, 0x8f820220,
-0x3c03fffe, 0x3463ffff, 0x431024, 0xaf820220,
-0x3c010002, 0xac209004, 0x3c010002, 0x10000032,
-0xac20900c, 0x8f820200, 0x34420100, 0xaf820200,
-0x8f820220, 0x3c03fffe, 0x3463ffff, 0x431024,
-0xaf820220, 0x24020100, 0x3c010002, 0xac229004,
-0x3c010002, 0x10000024, 0xac20900c, 0x8f820200,
-0x2403feff, 0x431024, 0xaf820200, 0x8f820220,
-0x3c030001, 0x431025, 0xaf820220, 0x3c010002,
-0xac209004, 0x3c010002, 0x10000017, 0xac23900c,
-0x8f820200, 0x34420100, 0xaf820200, 0x8f820220,
-0x3c030001, 0x431025, 0xaf820220, 0x24020100,
-0x3c010002, 0xac229004, 0x3c010002, 0x1000000a,
-0xac23900c, 0x3c040001, 0x24846c80, 0x97a6001a,
-0x97a70018, 0x3c050001, 0x34a5ffff, 0xafa80010,
-0xc002b3b, 0xafa00014, 0x8f820200, 0x34420002,
-0x1000004b, 0xaf820200, 0x128940, 0x3c050002,
-0xb12821, 0x8ca58ff8, 0x3c100002, 0x2118021,
-0x8e108ff0, 0x3c024000, 0xa21024, 0x14400010,
-0x0, 0x3c020001, 0x8c426f1c, 0x14400005,
-0x3c02bfff, 0x8f820200, 0x34420002, 0xaf820200,
-0x3c02bfff, 0x3442ffff, 0xc003daf, 0x2028024,
-0x3c010002, 0x310821, 0x10000031, 0xac308ff0,
-0x3c020001, 0x8c426f1c, 0x10400005, 0x3c020020,
-0x3c020001, 0x8c426e44, 0x10400025, 0x3c020020,
-0xa21024, 0x10400007, 0x34840020, 0x24020100,
-0x3c010002, 0x310821, 0xac229004, 0x10000006,
-0x34840100, 0x3c010002, 0x310821, 0xac209004,
-0x2402feff, 0x822024, 0x3c020080, 0xa21024,
-0x10400007, 0x121940, 0x3c020001, 0x3c010002,
-0x230821, 0xac22900c, 0x10000008, 0xc23025,
-0x121140, 0x3c010002, 0x220821, 0xac20900c,
-0x3c02fffe, 0x3442ffff, 0xc23024, 0xaf840200,
-0xaf860220, 0x8f820220, 0x34420002, 0xaf820220,
-0x121140, 0x3c010002, 0x220821, 0xac308ff0,
-0x8fbf002c, 0x8fb20028, 0x8fb10024, 0x8fb00020,
-0x3e00008, 0x27bd0030, 0x0, 0x1821,
-0x308400ff, 0x2405ffdf, 0x2406ffbf, 0x641007,
-0x30420001, 0x10400004, 0x0, 0x8f820044,
-0x10000003, 0x34420040, 0x8f820044, 0x461024,
-0xaf820044, 0x8f820044, 0x34420020, 0xaf820044,
-0x8f820044, 0x451024, 0xaf820044, 0x24630001,
-0x28620008, 0x5440ffee, 0x641007, 0x3e00008,
-0x0, 0x2c820008, 0x1040001b, 0x0,
-0x2405ffdf, 0x2406ffbf, 0x41880, 0x3c020001,
-0x24426e60, 0x621821, 0x24640004, 0x90620000,
-0x10400004, 0x0, 0x8f820044, 0x10000003,
-0x34420040, 0x8f820044, 0x461024, 0xaf820044,
-0x8f820044, 0x34420020, 0xaf820044, 0x8f820044,
-0x451024, 0xaf820044, 0x24630001, 0x64102b,
-0x1440ffee, 0x0, 0x3e00008, 0x0,
-0x0, 0x0, 0x0, 0x8f8400c4,
-0x8f8600e0, 0x8f8700e4, 0x2402fff8, 0xc22824,
-0x10e5001a, 0x27623ff8, 0x14e20002, 0x24e80008,
-0x27683000, 0x55050004, 0x8d0a0000, 0x30c20004,
-0x14400012, 0x805021, 0x8ce90000, 0x8f42013c,
-0x1494823, 0x49182b, 0x94eb0006, 0x10600002,
-0x25630050, 0x494821, 0x123182b, 0x50400003,
-0x8f4201fc, 0x3e00008, 0xe01021, 0xaf8800e8,
-0x24420001, 0xaf4201fc, 0xaf8800e4, 0x3e00008,
-0x1021, 0x3e00008, 0x0, 0x8f8300e4,
-0x27623ff8, 0x10620004, 0x24620008, 0xaf8200e8,
-0x3e00008, 0xaf8200e4, 0x27623000, 0xaf8200e8,
-0x3e00008, 0xaf8200e4, 0x3e00008, 0x0,
-0x0, 0x0, 0x0, 0x8f880120,
-0x27624fe0, 0x8f830128, 0x15020002, 0x25090020,
-0x27694800, 0x11230012, 0x8fa20010, 0xad040000,
-0xad050004, 0xad060008, 0xa507000e, 0x8fa30014,
-0xad020018, 0x8fa20018, 0xad03001c, 0x25030016,
-0xad020010, 0xad030014, 0xaf890120, 0x8f4300fc,
-0x24020001, 0x2463ffff, 0x3e00008, 0xaf4300fc,
-0x8f430324, 0x1021, 0x24630001, 0x3e00008,
-0xaf430324, 0x3e00008, 0x0, 0x8f880100,
-0x276247e0, 0x8f830108, 0x15020002, 0x25090020,
-0x27694000, 0x1123000f, 0x8fa20010, 0xad040000,
-0xad050004, 0xad060008, 0xa507000e, 0x8fa30014,
-0xad020018, 0x8fa20018, 0xad03001c, 0x25030016,
-0xad020010, 0xad030014, 0xaf890100, 0x3e00008,
-0x24020001, 0x8f430328, 0x1021, 0x24630001,
-0x3e00008, 0xaf430328, 0x3e00008, 0x0,
-0x0, 0x0, 0x0, 0x0 };
-static u32 tigon2FwRodata[(MAX_RODATA_LEN/4) + 1] __devinitdata = {
-0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x77322f63, 0x6f6d6d6f,
-0x6e2f6677, 0x6d61696e, 0x2e632c76, 0x20312e31,
-0x2e322e34, 0x35203139, 0x39392f30, 0x312f3234,
-0x2030303a, 0x31303a35, 0x35207368, 0x75616e67,
-0x20457870, 0x20240000, 0x65767452, 0x6e674600,
-0x51657674, 0x46000000, 0x51657674, 0x505f4600,
-0x4d657674, 0x526e6746, 0x0, 0x4d516576,
-0x74460000, 0x4d516576, 0x505f4600, 0x5173436f,
-0x6e495f46, 0x0, 0x5173436f, 0x6e734600,
-0x51725072, 0x6f644600, 0x6261644d, 0x656d537a,
-0x0, 0x68775665, 0x72000000, 0x62616448,
-0x77566572, 0x0, 0x2a2a4441, 0x574e5f41,
-0x0, 0x74785278, 0x4266537a, 0x0,
-0x62664174, 0x6e4d726b, 0x0, 0x7265645a,
-0x6f6e6531, 0x0, 0x70636943, 0x6f6e6600,
-0x67656e43, 0x6f6e6600, 0x2a646d61, 0x5244666c,
-0x0, 0x2a50414e, 0x49432a00, 0x2e2e2f2e,
-0x2e2f2e2e, 0x2f2e2e2f, 0x2e2e2f73, 0x72632f6e,
-0x69632f66, 0x77322f63, 0x6f6d6d6f, 0x6e2f6677,
-0x6d61696e, 0x2e630000, 0x72636246, 0x6c616773,
-0x0, 0x62616452, 0x78526362, 0x0,
-0x676c6f62, 0x466c6773, 0x0, 0x2b5f6469,
-0x73705f6c, 0x6f6f7000, 0x2b65765f, 0x68616e64,
-0x6c657200, 0x63616e74, 0x31446d61, 0x0,
-0x2b715f64, 0x6d615f74, 0x6f5f6e69, 0x635f636b,
-0x73756d00, 0x2b685f73, 0x656e645f, 0x64617461,
-0x5f726561, 0x64795f63, 0x6b73756d, 0x0,
-0x2b685f64, 0x6d615f72, 0x645f6173, 0x73697374,
-0x5f636b73, 0x756d0000, 0x74436b73, 0x6d4f6e00,
-0x2b715f64, 0x6d615f74, 0x6f5f6e69, 0x63000000,
-0x2b685f73, 0x656e645f, 0x64617461, 0x5f726561,
-0x64790000, 0x2b685f64, 0x6d615f72, 0x645f6173,
-0x73697374, 0x0, 0x74436b73, 0x6d4f6666,
-0x0, 0x2b685f73, 0x656e645f, 0x62645f72,
-0x65616479, 0x0, 0x68737453, 0x52696e67,
-0x0, 0x62616453, 0x52696e67, 0x0,
-0x6e696353, 0x52696e67, 0x0, 0x77446d61,
-0x416c6c41, 0x0, 0x2b715f64, 0x6d615f74,
-0x6f5f686f, 0x73745f63, 0x6b73756d, 0x0,
-0x2b685f6d, 0x61635f72, 0x785f636f, 0x6d705f63,
-0x6b73756d, 0x0, 0x2b685f64, 0x6d615f77,
-0x725f6173, 0x73697374, 0x5f636b73, 0x756d0000,
-0x72436b73, 0x6d4f6e00, 0x2b715f64, 0x6d615f74,
-0x6f5f686f, 0x73740000, 0x2b685f6d, 0x61635f72,
-0x785f636f, 0x6d700000, 0x2b685f64, 0x6d615f77,
-0x725f6173, 0x73697374, 0x0, 0x72436b73,
-0x6d4f6666, 0x0, 0x2b685f72, 0x6563765f,
-0x62645f72, 0x65616479, 0x0, 0x2b685f72,
-0x6563765f, 0x6a756d62, 0x6f5f6264, 0x5f726561,
-0x64790000, 0x2b685f72, 0x6563765f, 0x6d696e69,
-0x5f62645f, 0x72656164, 0x79000000, 0x2b6d685f,
-0x636f6d6d, 0x616e6400, 0x2b685f74, 0x696d6572,
-0x0, 0x2b685f64, 0x6f5f7570, 0x64617465,
-0x5f74785f, 0x636f6e73, 0x0, 0x2b685f64,
-0x6f5f7570, 0x64617465, 0x5f72785f, 0x70726f64,
-0x0, 0x2b636b73, 0x756d3136, 0x0,
-0x2b706565, 0x6b5f6d61, 0x635f7278, 0x5f776100,
-0x2b706565, 0x6b5f6d61, 0x635f7278, 0x0,
-0x2b646571, 0x5f6d6163, 0x5f727800, 0x2b685f6d,
-0x61635f72, 0x785f6174, 0x746e0000, 0x62616452,
-0x6574537a, 0x0, 0x72784264, 0x4266537a,
-0x0, 0x2b6e756c, 0x6c5f6861, 0x6e646c65,
-0x72000000, 0x66774f70, 0x4661696c, 0x0,
-0x2b685f75, 0x70646174, 0x655f6c65, 0x64340000,
-0x2b685f75, 0x70646174, 0x655f6c65, 0x64360000,
-0x2b685f75, 0x70646174, 0x655f6c65, 0x64320000,
-0x696e7453, 0x74617465, 0x0, 0x2a2a696e,
-0x69744370, 0x0, 0x23736372, 0x65616d00,
-0x69537461, 0x636b4572, 0x0, 0x70726f62,
-0x654d656d, 0x0, 0x2a2a4441, 0x574e5f42,
-0x0, 0x2b73775f, 0x646d615f, 0x61737369,
-0x73745f70, 0x6c75735f, 0x74696d65, 0x72000000,
-0x2b267072, 0x656c6f61, 0x645f7772, 0x5f646573,
-0x63720000, 0x2b267072, 0x656c6f61, 0x645f7264,
-0x5f646573, 0x63720000, 0x2b685f68, 0x665f7469,
-0x6d657200, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x77322f63, 0x6f6d6d6f,
-0x6e2f7469, 0x6d65722e, 0x632c7620, 0x312e312e,
-0x322e3335, 0x20313939, 0x392f3031, 0x2f323720,
-0x31393a30, 0x393a3530, 0x20686179, 0x65732045,
-0x78702024, 0x0, 0x65767452, 0x6e674600,
-0x51657674, 0x46000000, 0x51657674, 0x505f4600,
-0x4d657674, 0x526e6746, 0x0, 0x4d516576,
-0x74460000, 0x4d516576, 0x505f4600, 0x5173436f,
-0x6e495f46, 0x0, 0x5173436f, 0x6e734600,
-0x51725072, 0x6f644600, 0x542d446d, 0x61526432,
-0x0, 0x542d446d, 0x61526431, 0x0,
-0x542d446d, 0x61526442, 0x0, 0x542d446d,
-0x61577232, 0x0, 0x542d446d, 0x61577231,
-0x0, 0x542d446d, 0x61577242, 0x0,
-0x0, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x77322f63, 0x6f6d6d6f,
-0x6e2f636f, 0x6d6d616e, 0x642e632c, 0x7620312e,
-0x312e322e, 0x32382031, 0x3939392f, 0x30312f32,
-0x30203139, 0x3a34393a, 0x34392073, 0x6875616e,
-0x67204578, 0x70202400, 0x65767452, 0x6e674600,
-0x51657674, 0x46000000, 0x51657674, 0x505f4600,
-0x4d657674, 0x526e6746, 0x0, 0x4d516576,
-0x74460000, 0x4d516576, 0x505f4600, 0x5173436f,
-0x6e495f46, 0x0, 0x5173436f, 0x6e734600,
-0x51725072, 0x6f644600, 0x3f48636d, 0x644d6278,
-0x0, 0x3f636d64, 0x48737453, 0x0,
-0x3f636d64, 0x4d634d64, 0x0, 0x3f636d64,
-0x50726f6d, 0x0, 0x3f636d64, 0x4c696e6b,
-0x0, 0x3f636d64, 0x45727200, 0x86ac,
-0x8e5c, 0x8e5c, 0x8de4, 0x8b78,
-0x8e30, 0x8e5c, 0x8790, 0x8800,
-0x8990, 0x8a68, 0x8a34, 0x8e5c,
-0x8870, 0x8b24, 0x8e5c, 0x8b34,
-0x87b4, 0x8824, 0x0, 0x0,
-0x0, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x77322f63, 0x6f6d6d6f,
-0x6e2f6d63, 0x6173742e, 0x632c7620, 0x312e312e,
-0x322e3820, 0x31393938, 0x2f31322f, 0x30382030,
-0x323a3336, 0x3a333620, 0x73687561, 0x6e672045,
-0x78702024, 0x0, 0x65767452, 0x6e674600,
-0x51657674, 0x46000000, 0x51657674, 0x505f4600,
-0x4d657674, 0x526e6746, 0x0, 0x4d516576,
-0x74460000, 0x4d516576, 0x505f4600, 0x5173436f,
-0x6e495f46, 0x0, 0x5173436f, 0x6e734600,
-0x51725072, 0x6f644600, 0x6164644d, 0x63447570,
-0x0, 0x6164644d, 0x6346756c, 0x0,
-0x64656c4d, 0x634e6f45, 0x0, 0x0,
-0x0, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x77322f63, 0x6f6d6d6f,
-0x6e2f646d, 0x612e632c, 0x7620312e, 0x312e322e,
-0x32342031, 0x3939382f, 0x31322f32, 0x31203030,
-0x3a33333a, 0x30392073, 0x6875616e, 0x67204578,
-0x70202400, 0x65767452, 0x6e674600, 0x51657674,
-0x46000000, 0x51657674, 0x505f4600, 0x4d657674,
-0x526e6746, 0x0, 0x4d516576, 0x74460000,
-0x4d516576, 0x505f4600, 0x5173436f, 0x6e495f46,
-0x0, 0x5173436f, 0x6e734600, 0x51725072,
-0x6f644600, 0x7377446d, 0x614f6666, 0x0,
-0x31446d61, 0x4f6e0000, 0x7377446d, 0x614f6e00,
-0x2372446d, 0x6141544e, 0x0, 0x72446d61,
-0x41544e30, 0x0, 0x72446d61, 0x41544e31,
-0x0, 0x72446d61, 0x34476200, 0x2a50414e,
-0x49432a00, 0x2e2e2f2e, 0x2e2f2e2e, 0x2f2e2e2f,
-0x2e2e2f73, 0x72632f6e, 0x69632f66, 0x77322f63,
-0x6f6d6d6f, 0x6e2f646d, 0x612e6300, 0x2377446d,
-0x6141544e, 0x0, 0x77446d61, 0x41544e30,
-0x0, 0x77446d61, 0x41544e31, 0x0,
-0x77446d61, 0x34476200, 0x0, 0x0,
-0x0, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x77322f63, 0x6f6d6d6f,
-0x6e2f7472, 0x6163652e, 0x632c7620, 0x312e312e,
-0x322e3520, 0x31393938, 0x2f30392f, 0x33302031,
-0x383a3530, 0x3a323820, 0x73687561, 0x6e672045,
-0x78702024, 0x0, 0x0, 0x0,
-0x0, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x77322f63, 0x6f6d6d6f,
-0x6e2f6461, 0x74612e63, 0x2c762031, 0x2e312e32,
-0x2e313220, 0x31393939, 0x2f30312f, 0x32302031,
-0x393a3439, 0x3a353120, 0x73687561, 0x6e672045,
-0x78702024, 0x0, 0x46575f56, 0x45525349,
-0x4f4e3a20, 0x23312046, 0x72692041, 0x70722037,
-0x2031373a, 0x35373a35, 0x32205044, 0x54203230,
-0x30300000, 0x46575f43, 0x4f4d5049, 0x4c455f54,
-0x494d453a, 0x2031373a, 0x35373a35, 0x32000000,
-0x46575f43, 0x4f4d5049, 0x4c455f42, 0x593a2064,
-0x65767263, 0x73000000, 0x46575f43, 0x4f4d5049,
-0x4c455f48, 0x4f53543a, 0x20636f6d, 0x70757465,
-0x0, 0x46575f43, 0x4f4d5049, 0x4c455f44,
-0x4f4d4149, 0x4e3a2065, 0x6e672e61, 0x6374656f,
-0x6e2e636f, 0x6d000000, 0x46575f43, 0x4f4d5049,
-0x4c45523a, 0x20676363, 0x20766572, 0x73696f6e,
-0x20322e37, 0x2e320000, 0x0, 0x12041100,
-0x0, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x77322f63, 0x6f6d6d6f,
-0x6e2f6d65, 0x6d2e632c, 0x7620312e, 0x312e322e,
-0x35203139, 0x39382f30, 0x392f3330, 0x2031383a,
-0x35303a30, 0x38207368, 0x75616e67, 0x20457870,
-0x20240000, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x77322f63, 0x6f6d6d6f,
-0x6e2f7365, 0x6e642e63, 0x2c762031, 0x2e312e32,
-0x2e343420, 0x31393938, 0x2f31322f, 0x32312030,
-0x303a3333, 0x3a313820, 0x73687561, 0x6e672045,
-0x78702024, 0x0, 0x65767452, 0x6e674600,
-0x51657674, 0x46000000, 0x51657674, 0x505f4600,
-0x4d657674, 0x526e6746, 0x0, 0x4d516576,
-0x74460000, 0x4d516576, 0x505f4600, 0x5173436f,
-0x6e495f46, 0x0, 0x5173436f, 0x6e734600,
-0x51725072, 0x6f644600, 0x69736e74, 0x54637055,
-0x0, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x77322f63, 0x6f6d6d6f,
-0x6e2f7265, 0x63762e63, 0x2c762031, 0x2e312e32,
-0x2e353320, 0x31393939, 0x2f30312f, 0x31362030,
-0x323a3535, 0x3a343320, 0x73687561, 0x6e672045,
-0x78702024, 0x0, 0x65767452, 0x6e674600,
-0x51657674, 0x46000000, 0x51657674, 0x505f4600,
-0x4d657674, 0x526e6746, 0x0, 0x4d516576,
-0x74460000, 0x4d516576, 0x505f4600, 0x5173436f,
-0x6e495f46, 0x0, 0x5173436f, 0x6e734600,
-0x51725072, 0x6f644600, 0x724d6163, 0x43686b30,
-0x0, 0x72784672, 0x6d324c67, 0x0,
-0x72784e6f, 0x53744264, 0x0, 0x72784e6f,
-0x4d694264, 0x0, 0x72784e6f, 0x4a6d4264,
-0x0, 0x7278436b, 0x446d6146, 0x0,
-0x72785144, 0x6d457846, 0x0, 0x72785144,
-0x6d614600, 0x72785144, 0x4c426446, 0x0,
-0x72785144, 0x6d426446, 0x0, 0x72784372,
-0x63506164, 0x0, 0x72536d51, 0x446d6146,
-0x0, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x77322f63, 0x6f6d6d6f,
-0x6e2f6d61, 0x632e632c, 0x7620312e, 0x312e322e,
-0x32322031, 0x3939382f, 0x31322f30, 0x38203032,
-0x3a33363a, 0x33302073, 0x6875616e, 0x67204578,
-0x70202400, 0x65767452, 0x6e674600, 0x51657674,
-0x46000000, 0x51657674, 0x505f4600, 0x4d657674,
-0x526e6746, 0x0, 0x4d516576, 0x74460000,
-0x4d516576, 0x505f4600, 0x5173436f, 0x6e495f46,
-0x0, 0x5173436f, 0x6e734600, 0x51725072,
-0x6f644600, 0x6d616354, 0x68726573, 0x0,
-0x23744d61, 0x6341544e, 0x0, 0x23724d61,
-0x6341544e, 0x0, 0x72656d41, 0x73737274,
-0x0, 0x6c696e6b, 0x444f574e, 0x0,
-0x6c696e6b, 0x55500000, 0x0, 0x0,
-0x0, 0x24486561, 0x6465723a, 0x202f7072,
-0x6f6a6563, 0x74732f72, 0x63732f73, 0x772f6765,
-0x2f2e2f6e, 0x69632f66, 0x77322f63, 0x6f6d6d6f,
-0x6e2f636b, 0x73756d2e, 0x632c7620, 0x312e312e,
-0x322e3920, 0x31393939, 0x2f30312f, 0x31342030,
-0x303a3033, 0x3a343820, 0x73687561, 0x6e672045,
-0x78702024, 0x0, 0x65767452, 0x6e674600,
-0x51657674, 0x46000000, 0x51657674, 0x505f4600,
-0x4d657674, 0x526e6746, 0x0, 0x4d516576,
-0x74460000, 0x4d516576, 0x505f4600, 0x5173436f,
-0x6e495f46, 0x0, 0x5173436f, 0x6e734600,
-0x51725072, 0x6f644600, 0x0, 0x0,
-0x0, 0x50726f62, 0x65506879, 0x0,
-0x6c6e6b41, 0x53535254, 0x0, 0x109a4,
-0x10a1c, 0x10a50, 0x10a7c, 0x11050,
-0x10aa8, 0x10b10, 0x111fc, 0x10dc0,
-0x10c68, 0x10c80, 0x10cc4, 0x10cec,
-0x10d0c, 0x10d34, 0x111fc, 0x10dc0,
-0x10df8, 0x10e10, 0x10e40, 0x10e68,
-0x10e88, 0x10eb0, 0x0, 0x10fdc,
-0x11008, 0x1102c, 0x111fc, 0x11050,
-0x11078, 0x11108, 0x0, 0x0,
-0x0, 0x1186c, 0x1193c, 0x11a14,
-0x11ae4, 0x11b40, 0x11c1c, 0x11c44,
-0x11d20, 0x11d48, 0x11ef0, 0x11f18,
-0x120c0, 0x122b8, 0x1254c, 0x12460,
-0x1254c, 0x12578, 0x120e8, 0x12290,
-0x7273745f, 0x676d6969, 0x0, 0x12608,
-0x12640, 0x12728, 0x13374, 0x133b4,
-0x133cc, 0x7365746c, 0x6f6f7000, 0x0,
-0x0, 0x13bbc, 0x13bfc, 0x13c8c,
-0x13cd0, 0x13d34, 0x13dc0, 0x13df4,
-0x13e7c, 0x13f14, 0x13fe4, 0x14024,
-0x140a8, 0x140cc, 0x141dc, 0x646f4261,
-0x73655067, 0x0, 0x0, 0x0,
-0x0, 0x73746d61, 0x634c4e4b, 0x0,
-0x6765746d, 0x636c6e6b, 0x0, 0x14ed8,
-0x14ed8, 0x14b8c, 0x14bd8, 0x14c24,
-0x14ed8, 0x7365746d, 0x61636163, 0x74000000,
-0x0, 0x0 };
-static u32 tigon2FwData[(MAX_DATA_LEN/4) + 1] __devinitdata = {
-0x1,
-0x1, 0x1, 0xc001fc, 0x3ffc,
-0xc00000, 0x416c7465, 0x6f6e2041, 0x63654e49,
-0x43205600, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x416c7465,
-0x6f6e2041, 0x63654e49, 0x43205600, 0x42424242,
-0x0, 0x0, 0x0, 0x1ffffc,
-0x1fff7c, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x60cf00,
-0x60, 0xcf000000, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x3, 0x0,
-0x1, 0x0, 0x0, 0x0,
-0x1, 0x0, 0x1, 0x0,
-0x0, 0x0, 0x0, 0x1,
-0x1, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x1000000, 0x21000000,
-0x12000140, 0x0, 0x0, 0x20000000,
-0x120000a0, 0x0, 0x12000060, 0x12000180,
-0x120001e0, 0x0, 0x0, 0x0,
-0x1, 0x0, 0x0, 0x0,
-0x0, 0x0, 0x0, 0x2,
-0x0, 0x0, 0x30001, 0x1,
-0x30201, 0x0, 0x0, 0x1010101,
-0x1010100, 0x10100, 0x1010001, 0x10001,
-0x1000101, 0x101, 0x0, 0x0 };
diff --git a/drivers/net/amd8111e.c b/drivers/net/amd8111e.c
index 187ac6e..7709992 100644
--- a/drivers/net/amd8111e.c
+++ b/drivers/net/amd8111e.c
@@ -1813,6 +1813,25 @@
lp->ext_phy_addr = 1;
}
+static const struct net_device_ops amd8111e_netdev_ops = {
+ .ndo_open = amd8111e_open,
+ .ndo_stop = amd8111e_close,
+ .ndo_start_xmit = amd8111e_start_xmit,
+ .ndo_tx_timeout = amd8111e_tx_timeout,
+ .ndo_get_stats = amd8111e_get_stats,
+ .ndo_set_multicast_list = amd8111e_set_multicast_list,
+ .ndo_validate_addr = eth_validate_addr,
+ .ndo_set_mac_address = amd8111e_set_mac_address,
+ .ndo_do_ioctl = amd8111e_ioctl,
+ .ndo_change_mtu = amd8111e_change_mtu,
+#if AMD8111E_VLAN_TAG_USED
+ .ndo_vlan_rx_register = amd8111e_vlan_rx_register,
+#endif
+#ifdef CONFIG_NET_POLL_CONTROLLER
+ .ndo_poll_controller = amd8111e_poll,
+#endif
+};
+
static int __devinit amd8111e_probe_one(struct pci_dev *pdev,
const struct pci_device_id *ent)
{
@@ -1872,7 +1891,6 @@
#if AMD8111E_VLAN_TAG_USED
dev->features |= NETIF_F_HW_VLAN_TX | NETIF_F_HW_VLAN_RX ;
- dev->vlan_rx_register =amd8111e_vlan_rx_register;
#endif
lp = netdev_priv(dev);
@@ -1901,27 +1919,16 @@
if(dynamic_ipg[card_idx++])
lp->options |= OPTION_DYN_IPG_ENABLE;
+
/* Initialize driver entry points */
- dev->open = amd8111e_open;
- dev->hard_start_xmit = amd8111e_start_xmit;
- dev->stop = amd8111e_close;
- dev->get_stats = amd8111e_get_stats;
- dev->set_multicast_list = amd8111e_set_multicast_list;
- dev->set_mac_address = amd8111e_set_mac_address;
- dev->do_ioctl = amd8111e_ioctl;
- dev->change_mtu = amd8111e_change_mtu;
+ dev->netdev_ops = &amd8111e_netdev_ops;
SET_ETHTOOL_OPS(dev, &ops);
dev->irq =pdev->irq;
- dev->tx_timeout = amd8111e_tx_timeout;
dev->watchdog_timeo = AMD8111E_TX_TIMEOUT;
netif_napi_add(dev, &lp->napi, amd8111e_rx_poll, 32);
-#ifdef CONFIG_NET_POLL_CONTROLLER
- dev->poll_controller = amd8111e_poll;
-#endif
#if AMD8111E_VLAN_TAG_USED
dev->features |= NETIF_F_HW_VLAN_TX | NETIF_F_HW_VLAN_RX;
- dev->vlan_rx_register =amd8111e_vlan_rx_register;
#endif
/* Probe the external PHY */
amd8111e_probe_ext_phy(dev);
diff --git a/drivers/net/appletalk/ipddp.c b/drivers/net/appletalk/ipddp.c
index 9a0be9b..da64ba8 100644
--- a/drivers/net/appletalk/ipddp.c
+++ b/drivers/net/appletalk/ipddp.c
@@ -48,12 +48,18 @@
/* Index to functions, as function prototypes. */
static int ipddp_xmit(struct sk_buff *skb, struct net_device *dev);
-static struct net_device_stats *ipddp_get_stats(struct net_device *dev);
static int ipddp_create(struct ipddp_route *new_rt);
static int ipddp_delete(struct ipddp_route *rt);
static struct ipddp_route* ipddp_find_route(struct ipddp_route *rt);
static int ipddp_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd);
+static const struct net_device_ops ipddp_netdev_ops = {
+ .ndo_start_xmit = ipddp_xmit,
+ .ndo_do_ioctl = ipddp_ioctl,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
static struct net_device * __init ipddp_init(void)
{
@@ -61,7 +67,7 @@
struct net_device *dev;
int err;
- dev = alloc_etherdev(sizeof(struct net_device_stats));
+ dev = alloc_etherdev(0);
if (!dev)
return ERR_PTR(-ENOMEM);
@@ -71,9 +77,7 @@
printk(version);
/* Initalize the device structure. */
- dev->hard_start_xmit = ipddp_xmit;
- dev->get_stats = ipddp_get_stats;
- dev->do_ioctl = ipddp_ioctl;
+ dev->netdev_ops = &ipddp_netdev_ops;
dev->type = ARPHRD_IPDDP; /* IP over DDP tunnel */
dev->mtu = 585;
@@ -103,13 +107,6 @@
return dev;
}
-/*
- * Get the current statistics. This may be called with the card open or closed.
- */
-static struct net_device_stats *ipddp_get_stats(struct net_device *dev)
-{
- return netdev_priv(dev);
-}
/*
* Transmit LLAP/ELAP frame using aarp_send_ddp.
@@ -170,8 +167,8 @@
skb->protocol = htons(ETH_P_ATALK); /* Protocol has changed */
- ((struct net_device_stats *) netdev_priv(dev))->tx_packets++;
- ((struct net_device_stats *) netdev_priv(dev))->tx_bytes += skb->len;
+ dev->stats.tx_packets++;
+ dev->stats.tx_bytes += skb->len;
if(aarp_send_ddp(rt->dev, skb, &rt->at, NULL) < 0)
dev_kfree_skb(skb);
diff --git a/drivers/net/atp.c b/drivers/net/atp.c
index ea493ce..4317b3e 100644
--- a/drivers/net/atp.c
+++ b/drivers/net/atp.c
@@ -204,8 +204,7 @@
static void net_rx(struct net_device *dev);
static void read_block(long ioaddr, int length, unsigned char *buffer, int data_mode);
static int net_close(struct net_device *dev);
-static void set_rx_mode_8002(struct net_device *dev);
-static void set_rx_mode_8012(struct net_device *dev);
+static void set_rx_mode(struct net_device *dev);
static void tx_timeout(struct net_device *dev);
@@ -242,6 +241,17 @@
return -ENODEV;
}
+static const struct net_device_ops atp_netdev_ops = {
+ .ndo_open = net_open,
+ .ndo_stop = net_close,
+ .ndo_start_xmit = atp_send_packet,
+ .ndo_set_multicast_list = set_rx_mode,
+ .ndo_tx_timeout = tx_timeout,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
static int __init atp_probe1(long ioaddr)
{
struct net_device *dev = NULL;
@@ -342,12 +352,7 @@
if (dev->mem_end & 0xf)
net_debug = dev->mem_end & 7;
- dev->open = net_open;
- dev->stop = net_close;
- dev->hard_start_xmit = atp_send_packet;
- dev->set_multicast_list =
- lp->chip_type == RTL8002 ? &set_rx_mode_8002 : &set_rx_mode_8012;
- dev->tx_timeout = tx_timeout;
+ dev->netdev_ops = &atp_netdev_ops;
dev->watchdog_timeo = TX_TIMEOUT;
res = register_netdev(dev);
@@ -903,6 +908,17 @@
write_reg(ioaddr, CMR2, CMR2_IRQOUT); /* Switch back to page 0 */
}
+static void set_rx_mode(struct net_device *dev)
+{
+ struct net_local *lp = netdev_priv(dev);
+
+ if (lp->chip_type == RTL8002)
+ return set_rx_mode_8002(dev);
+ else
+ return set_rx_mode_8012(dev);
+}
+
+
static int __init atp_init_module(void) {
if (debug) /* Emit version even if no cards detected. */
printk(KERN_INFO "%s", version);
diff --git a/drivers/net/b44.c b/drivers/net/b44.c
index 0e7470a..6926ebe 100644
--- a/drivers/net/b44.c
+++ b/drivers/net/b44.c
@@ -2108,6 +2108,22 @@
return err;
}
+static const struct net_device_ops b44_netdev_ops = {
+ .ndo_open = b44_open,
+ .ndo_stop = b44_close,
+ .ndo_start_xmit = b44_start_xmit,
+ .ndo_get_stats = b44_get_stats,
+ .ndo_set_multicast_list = b44_set_rx_mode,
+ .ndo_set_mac_address = b44_set_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+ .ndo_do_ioctl = b44_ioctl,
+ .ndo_tx_timeout = b44_tx_timeout,
+ .ndo_change_mtu = b44_change_mtu,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+ .ndo_poll_controller = b44_poll_controller,
+#endif
+};
+
static int __devinit b44_init_one(struct ssb_device *sdev,
const struct ssb_device_id *ent)
{
@@ -2145,20 +2161,9 @@
bp->rx_pending = B44_DEF_RX_RING_PENDING;
bp->tx_pending = B44_DEF_TX_RING_PENDING;
- dev->open = b44_open;
- dev->stop = b44_close;
- dev->hard_start_xmit = b44_start_xmit;
- dev->get_stats = b44_get_stats;
- dev->set_multicast_list = b44_set_rx_mode;
- dev->set_mac_address = b44_set_mac_addr;
- dev->do_ioctl = b44_ioctl;
- dev->tx_timeout = b44_tx_timeout;
+ dev->netdev_ops = &b44_netdev_ops;
netif_napi_add(dev, &bp->napi, b44_poll, 64);
dev->watchdog_timeo = B44_TX_TIMEOUT;
-#ifdef CONFIG_NET_POLL_CONTROLLER
- dev->poll_controller = b44_poll_controller;
-#endif
- dev->change_mtu = b44_change_mtu;
dev->irq = sdev->irq;
SET_ETHTOOL_OPS(dev, &b44_ethtool_ops);
diff --git a/drivers/net/bnx2x_main.c b/drivers/net/bnx2x_main.c
index ef8103b..4be0584 100644
--- a/drivers/net/bnx2x_main.c
+++ b/drivers/net/bnx2x_main.c
@@ -8243,6 +8243,9 @@
struct bnx2x *bp = netdev_priv(dev);
int rc;
+ if (!netif_running(dev))
+ return -EAGAIN;
+
DP(BNX2X_MSG_NVM, "ethtool_eeprom: cmd %d\n"
DP_LEVEL " magic 0x%x offset 0x%x (%d) len 0x%x (%d)\n",
eeprom->cmd, eeprom->magic, eeprom->offset, eeprom->offset,
diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
index 460c2ca..9fb3883 100644
--- a/drivers/net/bonding/bond_main.c
+++ b/drivers/net/bonding/bond_main.c
@@ -4148,7 +4148,7 @@
bond_for_each_slave(bond, slave, i) {
pr_debug("s %p s->p %p c_m %p\n", slave,
- slave->prev, slave->dev->change_mtu);
+ slave->prev, slave->dev->netdev_ops->ndo_change_mtu);
res = dev_set_mtu(slave->dev, new_mtu);
diff --git a/drivers/net/cassini.c b/drivers/net/cassini.c
index 321f43d..840b3d1 100644
--- a/drivers/net/cassini.c
+++ b/drivers/net/cassini.c
@@ -4977,6 +4977,22 @@
pci_write_config_byte(pdev, PCI_LATENCY_TIMER, 0xff);
}
+static const struct net_device_ops cas_netdev_ops = {
+ .ndo_open = cas_open,
+ .ndo_stop = cas_close,
+ .ndo_start_xmit = cas_start_xmit,
+ .ndo_get_stats = cas_get_stats,
+ .ndo_set_multicast_list = cas_set_multicast,
+ .ndo_do_ioctl = cas_ioctl,
+ .ndo_tx_timeout = cas_tx_timeout,
+ .ndo_change_mtu = cas_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+ .ndo_poll_controller = cas_netpoll,
+#endif
+};
+
static int __devinit cas_init_one(struct pci_dev *pdev,
const struct pci_device_id *ent)
{
@@ -5166,22 +5182,13 @@
for (i = 0; i < N_RX_FLOWS; i++)
skb_queue_head_init(&cp->rx_flows[i]);
- dev->open = cas_open;
- dev->stop = cas_close;
- dev->hard_start_xmit = cas_start_xmit;
- dev->get_stats = cas_get_stats;
- dev->set_multicast_list = cas_set_multicast;
- dev->do_ioctl = cas_ioctl;
+ dev->netdev_ops = &cas_netdev_ops;
dev->ethtool_ops = &cas_ethtool_ops;
- dev->tx_timeout = cas_tx_timeout;
dev->watchdog_timeo = CAS_TX_TIMEOUT;
- dev->change_mtu = cas_change_mtu;
+
#ifdef USE_NAPI
netif_napi_add(dev, &cp->napi, cas_poll, 64);
#endif
-#ifdef CONFIG_NET_POLL_CONTROLLER
- dev->poll_controller = cas_netpoll;
-#endif
dev->irq = pdev->irq;
dev->dma = 0;
diff --git a/drivers/net/de600.c b/drivers/net/de600.c
index 970f820..de63f1d 100644
--- a/drivers/net/de600.c
+++ b/drivers/net/de600.c
@@ -378,6 +378,16 @@
*/
}
+static const struct net_device_ops de600_netdev_ops = {
+ .ndo_open = de600_open,
+ .ndo_stop = de600_close,
+ .ndo_start_xmit = de600_start_xmit,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
+
static struct net_device * __init de600_probe(void)
{
int i;
@@ -439,9 +449,7 @@
printk(", Ethernet Address: %pM\n", dev->dev_addr);
- dev->open = de600_open;
- dev->stop = de600_close;
- dev->hard_start_xmit = &de600_start_xmit;
+ dev->netdev_ops = &de600_netdev_ops;
dev->flags&=~IFF_MULTICAST;
diff --git a/drivers/net/de620.c b/drivers/net/de620.c
index bdfa894..d52f34c 100644
--- a/drivers/net/de620.c
+++ b/drivers/net/de620.c
@@ -784,6 +784,17 @@
return 0; /* all ok */
}
+static const struct net_device_ops de620_netdev_ops = {
+ .ndo_open = de620_open,
+ .ndo_stop = de620_close,
+ .ndo_start_xmit = de620_start_xmit,
+ .ndo_tx_timeout = de620_timeout,
+ .ndo_set_multicast_list = de620_set_multicast_list,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
/******************************************************************************
*
* Only start-up code below
@@ -861,12 +872,8 @@
else
printk(" UTP)\n");
- dev->open = de620_open;
- dev->stop = de620_close;
- dev->hard_start_xmit = de620_start_xmit;
- dev->tx_timeout = de620_timeout;
+ dev->netdev_ops = &de620_netdev_ops;
dev->watchdog_timeo = HZ*2;
- dev->set_multicast_list = de620_set_multicast_list;
/* base_addr and irq are already set, see above! */
diff --git a/drivers/net/e100.c b/drivers/net/e100.c
index 134b2d6..86bb876 100644
--- a/drivers/net/e100.c
+++ b/drivers/net/e100.c
@@ -161,6 +161,7 @@
#include <linux/skbuff.h>
#include <linux/ethtool.h>
#include <linux/string.h>
+#include <linux/firmware.h>
#include <asm/unaligned.h>
@@ -174,10 +175,17 @@
#define E100_WATCHDOG_PERIOD (2 * HZ)
#define E100_NAPI_WEIGHT 16
+#define FIRMWARE_D101M "e100/d101m_ucode.bin"
+#define FIRMWARE_D101S "e100/d101s_ucode.bin"
+#define FIRMWARE_D102E "e100/d102e_ucode.bin"
+
MODULE_DESCRIPTION(DRV_DESCRIPTION);
MODULE_AUTHOR(DRV_COPYRIGHT);
MODULE_LICENSE("GPL");
MODULE_VERSION(DRV_VERSION);
+MODULE_FIRMWARE(FIRMWARE_D101M);
+MODULE_FIRMWARE(FIRMWARE_D101S);
+MODULE_FIRMWARE(FIRMWARE_D102E);
static int debug = 3;
static int eeprom_bad_csum_allow = 0;
@@ -1049,178 +1057,6 @@
c[16], c[17], c[18], c[19], c[20], c[21], c[22], c[23]);
}
-/********************************************************/
-/* Micro code for 8086:1229 Rev 8 */
-/********************************************************/
-
-/* Parameter values for the D101M B-step */
-#define D101M_CPUSAVER_TIMER_DWORD 78
-#define D101M_CPUSAVER_BUNDLE_DWORD 65
-#define D101M_CPUSAVER_MIN_SIZE_DWORD 126
-
-#define D101M_B_RCVBUNDLE_UCODE \
-{\
-0x00550215, 0xFFFF0437, 0xFFFFFFFF, 0x06A70789, 0xFFFFFFFF, 0x0558FFFF, \
-0x000C0001, 0x00101312, 0x000C0008, 0x00380216, \
-0x0010009C, 0x00204056, 0x002380CC, 0x00380056, \
-0x0010009C, 0x00244C0B, 0x00000800, 0x00124818, \
-0x00380438, 0x00000000, 0x00140000, 0x00380555, \
-0x00308000, 0x00100662, 0x00100561, 0x000E0408, \
-0x00134861, 0x000C0002, 0x00103093, 0x00308000, \
-0x00100624, 0x00100561, 0x000E0408, 0x00100861, \
-0x000C007E, 0x00222C21, 0x000C0002, 0x00103093, \
-0x00380C7A, 0x00080000, 0x00103090, 0x00380C7A, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x0010009C, 0x00244C2D, 0x00010004, 0x00041000, \
-0x003A0437, 0x00044010, 0x0038078A, 0x00000000, \
-0x00100099, 0x00206C7A, 0x0010009C, 0x00244C48, \
-0x00130824, 0x000C0001, 0x00101213, 0x00260C75, \
-0x00041000, 0x00010004, 0x00130826, 0x000C0006, \
-0x002206A8, 0x0013C926, 0x00101313, 0x003806A8, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00080600, 0x00101B10, 0x00050004, 0x00100826, \
-0x00101210, 0x00380C34, 0x00000000, 0x00000000, \
-0x0021155B, 0x00100099, 0x00206559, 0x0010009C, \
-0x00244559, 0x00130836, 0x000C0000, 0x00220C62, \
-0x000C0001, 0x00101B13, 0x00229C0E, 0x00210C0E, \
-0x00226C0E, 0x00216C0E, 0x0022FC0E, 0x00215C0E, \
-0x00214C0E, 0x00380555, 0x00010004, 0x00041000, \
-0x00278C67, 0x00040800, 0x00018100, 0x003A0437, \
-0x00130826, 0x000C0001, 0x00220559, 0x00101313, \
-0x00380559, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00130831, 0x0010090B, 0x00124813, \
-0x000CFF80, 0x002606AB, 0x00041000, 0x00010004, \
-0x003806A8, 0x00000000, 0x00000000, 0x00000000, \
-}
-
-/********************************************************/
-/* Micro code for 8086:1229 Rev 9 */
-/********************************************************/
-
-/* Parameter values for the D101S */
-#define D101S_CPUSAVER_TIMER_DWORD 78
-#define D101S_CPUSAVER_BUNDLE_DWORD 67
-#define D101S_CPUSAVER_MIN_SIZE_DWORD 128
-
-#define D101S_RCVBUNDLE_UCODE \
-{\
-0x00550242, 0xFFFF047E, 0xFFFFFFFF, 0x06FF0818, 0xFFFFFFFF, 0x05A6FFFF, \
-0x000C0001, 0x00101312, 0x000C0008, 0x00380243, \
-0x0010009C, 0x00204056, 0x002380D0, 0x00380056, \
-0x0010009C, 0x00244F8B, 0x00000800, 0x00124818, \
-0x0038047F, 0x00000000, 0x00140000, 0x003805A3, \
-0x00308000, 0x00100610, 0x00100561, 0x000E0408, \
-0x00134861, 0x000C0002, 0x00103093, 0x00308000, \
-0x00100624, 0x00100561, 0x000E0408, 0x00100861, \
-0x000C007E, 0x00222FA1, 0x000C0002, 0x00103093, \
-0x00380F90, 0x00080000, 0x00103090, 0x00380F90, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x0010009C, 0x00244FAD, 0x00010004, 0x00041000, \
-0x003A047E, 0x00044010, 0x00380819, 0x00000000, \
-0x00100099, 0x00206FFD, 0x0010009A, 0x0020AFFD, \
-0x0010009C, 0x00244FC8, 0x00130824, 0x000C0001, \
-0x00101213, 0x00260FF7, 0x00041000, 0x00010004, \
-0x00130826, 0x000C0006, 0x00220700, 0x0013C926, \
-0x00101313, 0x00380700, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00080600, 0x00101B10, 0x00050004, 0x00100826, \
-0x00101210, 0x00380FB6, 0x00000000, 0x00000000, \
-0x002115A9, 0x00100099, 0x002065A7, 0x0010009A, \
-0x0020A5A7, 0x0010009C, 0x002445A7, 0x00130836, \
-0x000C0000, 0x00220FE4, 0x000C0001, 0x00101B13, \
-0x00229F8E, 0x00210F8E, 0x00226F8E, 0x00216F8E, \
-0x0022FF8E, 0x00215F8E, 0x00214F8E, 0x003805A3, \
-0x00010004, 0x00041000, 0x00278FE9, 0x00040800, \
-0x00018100, 0x003A047E, 0x00130826, 0x000C0001, \
-0x002205A7, 0x00101313, 0x003805A7, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00130831, \
-0x0010090B, 0x00124813, 0x000CFF80, 0x00260703, \
-0x00041000, 0x00010004, 0x00380700 \
-}
-
-/********************************************************/
-/* Micro code for the 8086:1229 Rev F/10 */
-/********************************************************/
-
-/* Parameter values for the D102 E-step */
-#define D102_E_CPUSAVER_TIMER_DWORD 42
-#define D102_E_CPUSAVER_BUNDLE_DWORD 54
-#define D102_E_CPUSAVER_MIN_SIZE_DWORD 46
-
-#define D102_E_RCVBUNDLE_UCODE \
-{\
-0x007D028F, 0x0E4204F9, 0x14ED0C85, 0x14FA14E9, 0x0EF70E36, 0x1FFF1FFF, \
-0x00E014B9, 0x00000000, 0x00000000, 0x00000000, \
-0x00E014BD, 0x00000000, 0x00000000, 0x00000000, \
-0x00E014D5, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00E014C1, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00E014C8, 0x00000000, 0x00000000, 0x00000000, \
-0x00200600, 0x00E014EE, 0x00000000, 0x00000000, \
-0x0030FF80, 0x00940E46, 0x00038200, 0x00102000, \
-0x00E00E43, 0x00000000, 0x00000000, 0x00000000, \
-0x00300006, 0x00E014FB, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00906E41, 0x00800E3C, 0x00E00E39, 0x00000000, \
-0x00906EFD, 0x00900EFD, 0x00E00EF8, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-0x00000000, 0x00000000, 0x00000000, 0x00000000, \
-}
-
-static void e100_setup_ucode(struct nic *nic, struct cb *cb, struct sk_buff *skb)
-{
-/* *INDENT-OFF* */
- static struct {
- u32 ucode[UCODE_SIZE + 1];
- u8 mac;
- u8 timer_dword;
- u8 bundle_dword;
- u8 min_size_dword;
- } ucode_opts[] = {
- { D101M_B_RCVBUNDLE_UCODE,
- mac_82559_D101M,
- D101M_CPUSAVER_TIMER_DWORD,
- D101M_CPUSAVER_BUNDLE_DWORD,
- D101M_CPUSAVER_MIN_SIZE_DWORD },
- { D101S_RCVBUNDLE_UCODE,
- mac_82559_D101S,
- D101S_CPUSAVER_TIMER_DWORD,
- D101S_CPUSAVER_BUNDLE_DWORD,
- D101S_CPUSAVER_MIN_SIZE_DWORD },
- { D102_E_RCVBUNDLE_UCODE,
- mac_82551_F,
- D102_E_CPUSAVER_TIMER_DWORD,
- D102_E_CPUSAVER_BUNDLE_DWORD,
- D102_E_CPUSAVER_MIN_SIZE_DWORD },
- { D102_E_RCVBUNDLE_UCODE,
- mac_82551_10,
- D102_E_CPUSAVER_TIMER_DWORD,
- D102_E_CPUSAVER_BUNDLE_DWORD,
- D102_E_CPUSAVER_MIN_SIZE_DWORD },
- { {0}, 0, 0, 0, 0}
- }, *opts;
-/* *INDENT-ON* */
-
/*************************************************************************
* CPUSaver parameters
*
@@ -1280,42 +1116,101 @@
#define BUNDLEMAX (u16)6
#define INTDELAY (u16)1536 /* 0x600 */
+/* Initialize firmware */
+static const struct firmware *e100_request_firmware(struct nic *nic)
+{
+ const char *fw_name;
+ const struct firmware *fw;
+ u8 timer, bundle, min_size;
+ int err;
+
/* do not load u-code for ICH devices */
if (nic->flags & ich)
- goto noloaducode;
+ return NULL;
/* Search for ucode match against h/w revision */
- for (opts = ucode_opts; opts->mac; opts++) {
- int i;
- u32 *ucode = opts->ucode;
- if (nic->mac != opts->mac)
- continue;
+ if (nic->mac == mac_82559_D101M)
+ fw_name = FIRMWARE_D101M;
+ else if (nic->mac == mac_82559_D101S)
+ fw_name = FIRMWARE_D101S;
+ else if (nic->mac == mac_82551_F || nic->mac == mac_82551_10)
+ fw_name = FIRMWARE_D102E;
+ else /* No ucode on other devices */
+ return NULL;
- /* Insert user-tunable settings */
- ucode[opts->timer_dword] &= 0xFFFF0000;
- ucode[opts->timer_dword] |= INTDELAY;
- ucode[opts->bundle_dword] &= 0xFFFF0000;
- ucode[opts->bundle_dword] |= BUNDLEMAX;
- ucode[opts->min_size_dword] &= 0xFFFF0000;
- ucode[opts->min_size_dword] |= (BUNDLESMALL) ? 0xFFFF : 0xFF80;
-
- for (i = 0; i < UCODE_SIZE; i++)
- cb->u.ucode[i] = cpu_to_le32(ucode[i]);
- cb->command = cpu_to_le16(cb_ucode | cb_el);
- return;
+ err = request_firmware(&fw, fw_name, &nic->pdev->dev);
+ if (err) {
+ DPRINTK(PROBE, ERR, "Failed to load firmware \"%s\": %d\n",
+ fw_name, err);
+ return ERR_PTR(err);
+ }
+ /* Firmware should be precisely UCODE_SIZE (words) plus three bytes
+ indicating the offsets for BUNDLESMALL, BUNDLEMAX, INTDELAY */
+ if (fw->size != UCODE_SIZE * 4 + 3) {
+ DPRINTK(PROBE, ERR, "Firmware \"%s\" has wrong size %zu\n",
+ fw_name, fw->size);
+ release_firmware(fw);
+ return ERR_PTR(-EINVAL);
}
-noloaducode:
- cb->command = cpu_to_le16(cb_nop | cb_el);
+ /* Read timer, bundle and min_size from end of firmware blob */
+ timer = fw->data[UCODE_SIZE * 4];
+ bundle = fw->data[UCODE_SIZE * 4 + 1];
+ min_size = fw->data[UCODE_SIZE * 4 + 2];
+
+ if (timer >= UCODE_SIZE || bundle >= UCODE_SIZE ||
+ min_size >= UCODE_SIZE) {
+ DPRINTK(PROBE, ERR,
+ "\"%s\" has bogus offset values (0x%x,0x%x,0x%x)\n",
+ fw_name, timer, bundle, min_size);
+ release_firmware(fw);
+ return ERR_PTR(-EINVAL);
+ }
+ /* OK, firmware is validated and ready to use... */
+ return fw;
}
-static inline int e100_exec_cb_wait(struct nic *nic, struct sk_buff *skb,
- void (*cb_prepare)(struct nic *, struct cb *, struct sk_buff *))
+static void e100_setup_ucode(struct nic *nic, struct cb *cb,
+ struct sk_buff *skb)
{
+ const struct firmware *fw = (void *)skb;
+ u8 timer, bundle, min_size;
+
+ /* It's not a real skb; we just abused the fact that e100_exec_cb
+ will pass it through to here... */
+ cb->skb = NULL;
+
+ /* firmware is stored as little endian already */
+ memcpy(cb->u.ucode, fw->data, UCODE_SIZE * 4);
+
+ /* Read timer, bundle and min_size from end of firmware blob */
+ timer = fw->data[UCODE_SIZE * 4];
+ bundle = fw->data[UCODE_SIZE * 4 + 1];
+ min_size = fw->data[UCODE_SIZE * 4 + 2];
+
+ /* Insert user-tunable settings in cb->u.ucode */
+ cb->u.ucode[timer] &= cpu_to_le32(0xFFFF0000);
+ cb->u.ucode[timer] |= cpu_to_le32(INTDELAY);
+ cb->u.ucode[bundle] &= cpu_to_le32(0xFFFF0000);
+ cb->u.ucode[bundle] |= cpu_to_le32(BUNDLEMAX);
+ cb->u.ucode[min_size] &= cpu_to_le32(0xFFFF0000);
+ cb->u.ucode[min_size] |= cpu_to_le32((BUNDLESMALL) ? 0xFFFF : 0xFF80);
+
+ cb->command = cpu_to_le16(cb_ucode | cb_el);
+}
+
+static inline int e100_load_ucode_wait(struct nic *nic)
+{
+ const struct firmware *fw;
int err = 0, counter = 50;
struct cb *cb = nic->cb_to_clean;
- if ((err = e100_exec_cb(nic, NULL, e100_setup_ucode)))
+ fw = e100_request_firmware(nic);
+ /* If it's NULL, then no ucode is required */
+ if (!fw || IS_ERR(fw))
+ return PTR_ERR(fw);
+
+ if ((err = e100_exec_cb(nic, (void *)fw, e100_setup_ucode)))
DPRINTK(PROBE,ERR, "ucode cmd failed with error %d\n", err);
/* must restart cuc */
@@ -1435,7 +1330,7 @@
return err;
if ((err = e100_exec_cmd(nic, ruc_load_base, 0)))
return err;
- if ((err = e100_exec_cb_wait(nic, NULL, e100_setup_ucode)))
+ if ((err = e100_load_ucode_wait(nic)))
return err;
if ((err = e100_exec_cb(nic, NULL, e100_configure)))
return err;
diff --git a/drivers/net/ehea/ehea_phyp.c b/drivers/net/ehea/ehea_phyp.c
index 2a33a61..8fe9dca 100644
--- a/drivers/net/ehea/ehea_phyp.c
+++ b/drivers/net/ehea/ehea_phyp.c
@@ -214,7 +214,7 @@
u64 *qp_handle, struct h_epas *h_epas)
{
u64 hret;
- u64 outs[PLPAR_HCALL9_BUFSIZE];
+ unsigned long outs[PLPAR_HCALL9_BUFSIZE];
u64 allocate_controls =
EHEA_BMASK_SET(H_ALL_RES_QP_EQPO, init_attr->low_lat_rq1 ? 1 : 0)
@@ -312,7 +312,7 @@
u64 *cq_handle, struct h_epas *epas)
{
u64 hret;
- u64 outs[PLPAR_HCALL9_BUFSIZE];
+ unsigned long outs[PLPAR_HCALL9_BUFSIZE];
hret = ehea_plpar_hcall9(H_ALLOC_HEA_RESOURCE,
outs,
@@ -374,7 +374,7 @@
struct ehea_eq_attr *eq_attr, u64 *eq_handle)
{
u64 hret, allocate_controls;
- u64 outs[PLPAR_HCALL9_BUFSIZE];
+ unsigned long outs[PLPAR_HCALL9_BUFSIZE];
/* resource type */
allocate_controls =
@@ -407,7 +407,7 @@
u16 *out_swr, u16 *out_rwr)
{
u64 hret;
- u64 outs[PLPAR_HCALL9_BUFSIZE];
+ unsigned long outs[PLPAR_HCALL9_BUFSIZE];
hret = ehea_plpar_hcall9(H_MODIFY_HEA_QP,
outs,
@@ -449,7 +449,7 @@
struct ehea_mr *mr)
{
u64 hret;
- u64 outs[PLPAR_HCALL9_BUFSIZE];
+ unsigned long outs[PLPAR_HCALL9_BUFSIZE];
hret = ehea_plpar_hcall9(H_REGISTER_SMR,
outs,
@@ -468,7 +468,7 @@
u64 ehea_h_disable_and_get_hea(const u64 adapter_handle, const u64 qp_handle)
{
- u64 outs[PLPAR_HCALL9_BUFSIZE];
+ unsigned long outs[PLPAR_HCALL9_BUFSIZE];
return ehea_plpar_hcall9(H_DISABLE_AND_GET_HEA,
outs,
@@ -493,7 +493,7 @@
const u32 pd, u64 *mr_handle, u32 *lkey)
{
u64 hret;
- u64 outs[PLPAR_HCALL9_BUFSIZE];
+ unsigned long outs[PLPAR_HCALL9_BUFSIZE];
hret = ehea_plpar_hcall9(H_ALLOC_HEA_RESOURCE,
outs,
@@ -564,7 +564,7 @@
const u8 cb_cat, const u64 select_mask,
void *cb_addr)
{
- u64 outs[PLPAR_HCALL9_BUFSIZE];
+ unsigned long outs[PLPAR_HCALL9_BUFSIZE];
u64 port_info;
u64 arr_index = 0;
u64 cb_logaddr = virt_to_abs(cb_addr);
diff --git a/drivers/net/enc28j60.c b/drivers/net/enc28j60.c
index cefe1d9..fc6cc03 100644
--- a/drivers/net/enc28j60.c
+++ b/drivers/net/enc28j60.c
@@ -1531,6 +1531,17 @@
return enc28j60_hw_init(priv);
}
+static const struct net_device_ops enc28j60_netdev_ops = {
+ .ndo_open = enc28j60_net_open,
+ .ndo_stop = enc28j60_net_close,
+ .ndo_start_xmit = enc28j60_send_packet,
+ .ndo_set_multicast_list = enc28j60_set_multicast_list,
+ .ndo_set_mac_address = enc28j60_set_mac_address,
+ .ndo_tx_timeout = enc28j60_tx_timeout,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
static int __devinit enc28j60_probe(struct spi_device *spi)
{
struct net_device *dev;
@@ -1585,12 +1596,7 @@
dev->if_port = IF_PORT_10BASET;
dev->irq = spi->irq;
- dev->open = enc28j60_net_open;
- dev->stop = enc28j60_net_close;
- dev->hard_start_xmit = enc28j60_send_packet;
- dev->set_multicast_list = &enc28j60_set_multicast_list;
- dev->set_mac_address = enc28j60_set_mac_address;
- dev->tx_timeout = &enc28j60_tx_timeout;
+ dev->netdev_ops = &enc28j60_netdev_ops;
dev->watchdog_timeo = TX_TIMEOUT;
SET_ETHTOOL_OPS(dev, &enc28j60_ethtool_ops);
diff --git a/drivers/net/epic100.c b/drivers/net/epic100.c
index f9b37c8..a539bc3 100644
--- a/drivers/net/epic100.c
+++ b/drivers/net/epic100.c
@@ -308,7 +308,18 @@
static struct net_device_stats *epic_get_stats(struct net_device *dev);
static void set_rx_mode(struct net_device *dev);
-
+static const struct net_device_ops epic_netdev_ops = {
+ .ndo_open = epic_open,
+ .ndo_stop = epic_close,
+ .ndo_start_xmit = epic_start_xmit,
+ .ndo_tx_timeout = epic_tx_timeout,
+ .ndo_get_stats = epic_get_stats,
+ .ndo_set_multicast_list = set_rx_mode,
+ .ndo_do_ioctl = netdev_ioctl,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
static int __devinit epic_init_one (struct pci_dev *pdev,
const struct pci_device_id *ent)
@@ -483,15 +494,9 @@
dev->if_port = ep->default_port = option;
/* The Epic-specific entries in the device structure. */
- dev->open = &epic_open;
- dev->hard_start_xmit = &epic_start_xmit;
- dev->stop = &epic_close;
- dev->get_stats = &epic_get_stats;
- dev->set_multicast_list = &set_rx_mode;
- dev->do_ioctl = &netdev_ioctl;
+ dev->netdev_ops = &epic_netdev_ops;
dev->ethtool_ops = &netdev_ethtool_ops;
dev->watchdog_timeo = TX_TIMEOUT;
- dev->tx_timeout = &epic_tx_timeout;
netif_napi_add(dev, &ep->napi, epic_poll, 64);
ret = register_netdev(dev);
diff --git a/drivers/net/fealnx.c b/drivers/net/fealnx.c
index 31ab1ff..daf7272 100644
--- a/drivers/net/fealnx.c
+++ b/drivers/net/fealnx.c
@@ -467,6 +467,18 @@
}
}
+static const struct net_device_ops netdev_ops = {
+ .ndo_open = netdev_open,
+ .ndo_stop = netdev_close,
+ .ndo_start_xmit = start_tx,
+ .ndo_get_stats = get_stats,
+ .ndo_set_multicast_list = set_rx_mode,
+ .ndo_do_ioctl = mii_ioctl,
+ .ndo_tx_timeout = fealnx_tx_timeout,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
static int __devinit fealnx_init_one(struct pci_dev *pdev,
const struct pci_device_id *ent)
@@ -649,15 +661,8 @@
np->mii.force_media = 1;
}
- /* The chip-specific entries in the device structure. */
- dev->open = &netdev_open;
- dev->hard_start_xmit = &start_tx;
- dev->stop = &netdev_close;
- dev->get_stats = &get_stats;
- dev->set_multicast_list = &set_rx_mode;
- dev->do_ioctl = &mii_ioctl;
+ dev->netdev_ops = &netdev_ops;
dev->ethtool_ops = &netdev_ethtool_ops;
- dev->tx_timeout = &fealnx_tx_timeout;
dev->watchdog_timeo = TX_TIMEOUT;
err = register_netdev(dev);
diff --git a/drivers/net/gianfar.c b/drivers/net/gianfar.c
index c672ecf..1b8deca 100644
--- a/drivers/net/gianfar.c
+++ b/drivers/net/gianfar.c
@@ -238,8 +238,8 @@
goto err_out;
}
- snprintf(priv->phy_bus_id, BUS_ID_SIZE, PHY_ID_FMT, "0",
- fixed_link[0]);
+ snprintf(priv->phy_bus_id, sizeof(priv->phy_bus_id),
+ PHY_ID_FMT, "0", fixed_link[0]);
} else {
phy = of_find_node_by_phandle(*ph);
@@ -256,7 +256,7 @@
of_node_put(mdio);
gfar_mdio_bus_name(bus_name, mdio);
- snprintf(priv->phy_bus_id, BUS_ID_SIZE, "%s:%02x",
+ snprintf(priv->phy_bus_id, sizeof(priv->phy_bus_id), "%s:%02x",
bus_name, *id);
}
@@ -1973,6 +1973,8 @@
case 1000:
tempval =
((tempval & ~(MACCFG2_IF)) | MACCFG2_GMII);
+
+ ecntrl &= ~(ECNTRL_R100);
break;
case 100:
case 10:
diff --git a/drivers/net/hp100.c b/drivers/net/hp100.c
index ebe7651..ad8be7e 100644
--- a/drivers/net/hp100.c
+++ b/drivers/net/hp100.c
@@ -425,6 +425,28 @@
}
#endif /* !MODULE && CONFIG_ISA */
+static const struct net_device_ops hp100_bm_netdev_ops = {
+ .ndo_open = hp100_open,
+ .ndo_stop = hp100_close,
+ .ndo_start_xmit = hp100_start_xmit_bm,
+ .ndo_get_stats = hp100_get_stats,
+ .ndo_set_multicast_list = hp100_set_multicast_list,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
+static const struct net_device_ops hp100_netdev_ops = {
+ .ndo_open = hp100_open,
+ .ndo_stop = hp100_close,
+ .ndo_start_xmit = hp100_start_xmit,
+ .ndo_get_stats = hp100_get_stats,
+ .ndo_set_multicast_list = hp100_set_multicast_list,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
static int __devinit hp100_probe1(struct net_device *dev, int ioaddr,
u_char bus, struct pci_dev *pci_dev)
{
@@ -657,16 +679,10 @@
lp->virt_memory_size = virt_memory_size;
lp->rx_ratio = hp100_rx_ratio; /* can be conf'd with insmod */
- dev->open = hp100_open;
- dev->stop = hp100_close;
-
if (lp->mode == 1) /* busmaster */
- dev->hard_start_xmit = hp100_start_xmit_bm;
+ dev->netdev_ops = &hp100_bm_netdev_ops;
else
- dev->hard_start_xmit = hp100_start_xmit;
-
- dev->get_stats = hp100_get_stats;
- dev->set_multicast_list = &hp100_set_multicast_list;
+ dev->netdev_ops = &hp100_netdev_ops;
/* Ask the card for which IRQ line it is configured */
if (bus == HP100_BUS_PCI) {
diff --git a/drivers/net/ibmveth.c b/drivers/net/ibmveth.c
index 9bc0f17..ca3bb9f 100644
--- a/drivers/net/ibmveth.c
+++ b/drivers/net/ibmveth.c
@@ -754,7 +754,7 @@
void (*done) (struct net_device *, u32))
{
struct ibmveth_adapter *adapter = netdev_priv(dev);
- u64 set_attr, clr_attr, ret_attr;
+ unsigned long set_attr, clr_attr, ret_attr;
long ret;
int rc1 = 0, rc2 = 0;
int restart = 0;
@@ -1209,7 +1209,7 @@
long ret;
struct net_device *netdev;
struct ibmveth_adapter *adapter;
- u64 set_attr, ret_attr;
+ unsigned long set_attr, ret_attr;
unsigned char *mac_addr_p;
unsigned int *mcastFilterSize_p;
diff --git a/drivers/net/ibmveth.h b/drivers/net/ibmveth.h
index d281869..ec76ace 100644
--- a/drivers/net/ibmveth.h
+++ b/drivers/net/ibmveth.h
@@ -39,11 +39,11 @@
#define IbmVethMcastRemoveFilter 0x2UL
#define IbmVethMcastClearFilterTable 0x3UL
-#define IBMVETH_ILLAN_PADDED_PKT_CSUM 0x0000000000002000ULL
-#define IBMVETH_ILLAN_TRUNK_PRI_MASK 0x0000000000000F00ULL
-#define IBMVETH_ILLAN_IPV6_TCP_CSUM 0x0000000000000004ULL
-#define IBMVETH_ILLAN_IPV4_TCP_CSUM 0x0000000000000002ULL
-#define IBMVETH_ILLAN_ACTIVE_TRUNK 0x0000000000000001ULL
+#define IBMVETH_ILLAN_PADDED_PKT_CSUM 0x0000000000002000UL
+#define IBMVETH_ILLAN_TRUNK_PRI_MASK 0x0000000000000F00UL
+#define IBMVETH_ILLAN_IPV6_TCP_CSUM 0x0000000000000004UL
+#define IBMVETH_ILLAN_IPV4_TCP_CSUM 0x0000000000000002UL
+#define IBMVETH_ILLAN_ACTIVE_TRUNK 0x0000000000000001UL
/* hcall macros */
#define h_register_logical_lan(ua, buflst, rxq, fltlst, mac) \
diff --git a/drivers/net/ipg.c b/drivers/net/ipg.c
index 7b6d435..360aa5e 100644
--- a/drivers/net/ipg.c
+++ b/drivers/net/ipg.c
@@ -2210,6 +2210,19 @@
pci_set_drvdata(pdev, NULL);
}
+static const struct net_device_ops ipg_netdev_ops = {
+ .ndo_open = ipg_nic_open,
+ .ndo_stop = ipg_nic_stop,
+ .ndo_start_xmit = ipg_nic_hard_start_xmit,
+ .ndo_get_stats = ipg_nic_get_stats,
+ .ndo_set_multicast_list = ipg_nic_set_multicast_list,
+ .ndo_do_ioctl = ipg_ioctl,
+ .ndo_tx_timeout = ipg_tx_timeout,
+ .ndo_change_mtu = ipg_nic_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
static int __devinit ipg_probe(struct pci_dev *pdev,
const struct pci_device_id *id)
{
@@ -2258,15 +2271,7 @@
/* Declare IPG NIC functions for Ethernet device methods.
*/
- dev->open = &ipg_nic_open;
- dev->stop = &ipg_nic_stop;
- dev->hard_start_xmit = &ipg_nic_hard_start_xmit;
- dev->get_stats = &ipg_nic_get_stats;
- dev->set_multicast_list = &ipg_nic_set_multicast_list;
- dev->do_ioctl = ipg_ioctl;
- dev->tx_timeout = ipg_tx_timeout;
- dev->change_mtu = &ipg_nic_change_mtu;
-
+ dev->netdev_ops = &ipg_netdev_ops;
SET_NETDEV_DEV(dev, &pdev->dev);
SET_ETHTOOL_OPS(dev, &ipg_ethtool_ops);
diff --git a/drivers/net/irda/ali-ircc.c b/drivers/net/irda/ali-ircc.c
index 3c58e67..17779f9 100644
--- a/drivers/net/irda/ali-ircc.c
+++ b/drivers/net/irda/ali-ircc.c
@@ -109,7 +109,6 @@
static int ali_ircc_net_close(struct net_device *dev);
static int ali_ircc_net_ioctl(struct net_device *dev, struct ifreq *rq, int cmd);
static void ali_ircc_change_speed(struct ali_ircc_cb *self, __u32 baud);
-static struct net_device_stats *ali_ircc_net_get_stats(struct net_device *dev);
/* SIR function */
static int ali_ircc_sir_hard_xmit(struct sk_buff *skb, struct net_device *dev);
@@ -366,7 +365,6 @@
dev->open = ali_ircc_net_open;
dev->stop = ali_ircc_net_close;
dev->do_ioctl = ali_ircc_net_ioctl;
- dev->get_stats = ali_ircc_net_get_stats;
err = register_netdev(dev);
if (err) {
@@ -876,7 +874,7 @@
* async_unwrap_char will deliver all found frames
*/
do {
- async_unwrap_char(self->netdev, &self->stats, &self->rx_buff,
+ async_unwrap_char(self->netdev, &self->netdev->stats, &self->rx_buff,
inb(iobase+UART_RX));
/* Make sure we don't stay here too long */
@@ -943,7 +941,7 @@
netif_wake_queue(self->netdev);
}
- self->stats.tx_packets++;
+ self->netdev->stats.tx_packets++;
/* Turn on receive interrupts */
outb(UART_IER_RDI, iobase+UART_IER);
@@ -1467,7 +1465,7 @@
self->tx_fifo.queue[self->tx_fifo.free].len = skb->len;
self->tx_fifo.tail += skb->len;
- self->stats.tx_bytes += skb->len;
+ dev->stats.tx_bytes += skb->len;
skb_copy_from_linear_data(skb, self->tx_fifo.queue[self->tx_fifo.free].start,
skb->len);
@@ -1661,12 +1659,12 @@
{
IRDA_ERROR("%s(), ********* LSR_FRAME_ABORT *********\n", __func__);
- self->stats.tx_errors++;
- self->stats.tx_fifo_errors++;
+ self->netdev->stats.tx_errors++;
+ self->netdev->stats.tx_fifo_errors++;
}
else
{
- self->stats.tx_packets++;
+ self->netdev->stats.tx_packets++;
}
/* Check if we need to change the speed */
@@ -1831,35 +1829,35 @@
IRDA_DEBUG(0,"%s(), ************* RX Errors ************ \n", __func__ );
/* Skip frame */
- self->stats.rx_errors++;
+ self->netdev->stats.rx_errors++;
self->rx_buff.data += len;
if (status & LSR_FIFO_UR)
{
- self->stats.rx_frame_errors++;
+ self->netdev->stats.rx_frame_errors++;
IRDA_DEBUG(0,"%s(), ************* FIFO Errors ************ \n", __func__ );
}
if (status & LSR_FRAME_ERROR)
{
- self->stats.rx_frame_errors++;
+ self->netdev->stats.rx_frame_errors++;
IRDA_DEBUG(0,"%s(), ************* FRAME Errors ************ \n", __func__ );
}
if (status & LSR_CRC_ERROR)
{
- self->stats.rx_crc_errors++;
+ self->netdev->stats.rx_crc_errors++;
IRDA_DEBUG(0,"%s(), ************* CRC Errors ************ \n", __func__ );
}
if(self->rcvFramesOverflow)
{
- self->stats.rx_frame_errors++;
+ self->netdev->stats.rx_frame_errors++;
IRDA_DEBUG(0,"%s(), ************* Overran DMA buffer ************ \n", __func__ );
}
if(len == 0)
{
- self->stats.rx_frame_errors++;
+ self->netdev->stats.rx_frame_errors++;
IRDA_DEBUG(0,"%s(), ********** Receive Frame Size = 0 ********* \n", __func__ );
}
}
@@ -1910,7 +1908,7 @@
IRDA_WARNING("%s(), memory squeeze, "
"dropping frame.\n",
__func__);
- self->stats.rx_dropped++;
+ self->netdev->stats.rx_dropped++;
return FALSE;
}
@@ -1924,8 +1922,8 @@
/* Move to next frame */
self->rx_buff.data += len;
- self->stats.rx_bytes += len;
- self->stats.rx_packets++;
+ self->netdev->stats.rx_bytes += len;
+ self->netdev->stats.rx_packets++;
skb->dev = self->netdev;
skb_reset_mac_header(skb);
@@ -1994,7 +1992,7 @@
self->tx_buff.len = async_wrap_skb(skb, self->tx_buff.data,
self->tx_buff.truesize);
- self->stats.tx_bytes += self->tx_buff.len;
+ self->netdev->stats.tx_bytes += self->tx_buff.len;
/* Turn on transmit finished interrupt. Will fire immediately! */
outb(UART_IER_THRI, iobase+UART_IER);
@@ -2111,17 +2109,6 @@
return status;
}
-static struct net_device_stats *ali_ircc_net_get_stats(struct net_device *dev)
-{
- struct ali_ircc_cb *self = netdev_priv(dev);
-
- IRDA_DEBUG(2, "%s(), ---------------- Start ----------------\n", __func__ );
-
- IRDA_DEBUG(2, "%s(), ----------------- End ------------------\n", __func__ );
-
- return &self->stats;
-}
-
static int ali_ircc_suspend(struct platform_device *dev, pm_message_t state)
{
struct ali_ircc_cb *self = platform_get_drvdata(dev);
diff --git a/drivers/net/irda/ali-ircc.h b/drivers/net/irda/ali-ircc.h
index ed35d99..0c8edb4 100644
--- a/drivers/net/irda/ali-ircc.h
+++ b/drivers/net/irda/ali-ircc.h
@@ -191,7 +191,6 @@
struct tx_fifo tx_fifo; /* Info about frames to be transmitted */
struct net_device *netdev; /* Yes! we are some kind of netdevice */
- struct net_device_stats stats;
struct irlap_cb *irlap; /* The link layer we are binded to */
struct qos_info qos; /* QoS capabilities for this device */
diff --git a/drivers/net/irda/au1000_ircc.h b/drivers/net/irda/au1000_ircc.h
index b4763f2..c072c09 100644
--- a/drivers/net/irda/au1000_ircc.h
+++ b/drivers/net/irda/au1000_ircc.h
@@ -107,7 +107,6 @@
iobuff_t rx_buff;
struct net_device *netdev;
- struct net_device_stats stats;
struct timeval stamp;
struct timeval now;
diff --git a/drivers/net/irda/au1k_ir.c b/drivers/net/irda/au1k_ir.c
index 6c4b53f..75a1d0a 100644
--- a/drivers/net/irda/au1k_ir.c
+++ b/drivers/net/irda/au1k_ir.c
@@ -53,7 +53,6 @@
static int au1k_irda_rx(struct net_device *);
static void au1k_irda_interrupt(int, void *);
static void au1k_tx_timeout(struct net_device *);
-static struct net_device_stats *au1k_irda_stats(struct net_device *);
static int au1k_irda_ioctl(struct net_device *, struct ifreq *, int);
static int au1k_irda_set_speed(struct net_device *dev, int speed);
@@ -213,7 +212,6 @@
dev->open = au1k_irda_start;
dev->hard_start_xmit = au1k_irda_hard_xmit;
dev->stop = au1k_irda_stop;
- dev->get_stats = au1k_irda_stats;
dev->do_ioctl = au1k_irda_ioctl;
dev->tx_timeout = au1k_tx_timeout;
@@ -832,13 +830,6 @@
return ret;
}
-
-static struct net_device_stats *au1k_irda_stats(struct net_device *dev)
-{
- struct au1k_private *aup = netdev_priv(dev);
- return &aup->stats;
-}
-
MODULE_AUTHOR("Pete Popov <ppopov@mvista.com>");
MODULE_DESCRIPTION("Au1000 IrDA Device Driver");
diff --git a/drivers/net/irda/donauboe.h b/drivers/net/irda/donauboe.h
index 1e67720..0dbd193 100644
--- a/drivers/net/irda/donauboe.h
+++ b/drivers/net/irda/donauboe.h
@@ -308,7 +308,6 @@
struct toshoboe_cb
{
struct net_device *netdev; /* Yes! we are some kind of netdevice */
- struct net_device_stats stats;
struct tty_driver ttydev;
struct irlap_cb *irlap; /* The link layer we are binded to */
diff --git a/drivers/net/irda/irda-usb.c b/drivers/net/irda/irda-usb.c
index 205e4e8..29118f5 100644
--- a/drivers/net/irda/irda-usb.c
+++ b/drivers/net/irda/irda-usb.c
@@ -122,7 +122,6 @@
static int irda_usb_net_close(struct net_device *dev);
static int irda_usb_net_ioctl(struct net_device *dev, struct ifreq *rq, int cmd);
static void irda_usb_net_timeout(struct net_device *dev);
-static struct net_device_stats *irda_usb_net_get_stats(struct net_device *dev);
/************************ TRANSMIT ROUTINES ************************/
/*
@@ -525,13 +524,13 @@
/* Ask USB to send the packet - Irq disabled -> GFP_ATOMIC */
if ((res = usb_submit_urb(urb, GFP_ATOMIC))) {
IRDA_WARNING("%s(), failed Tx URB\n", __func__);
- self->stats.tx_errors++;
+ netdev->stats.tx_errors++;
/* Let USB recover : We will catch that in the watchdog */
/*netif_start_queue(netdev);*/
} else {
/* Increment packet stats */
- self->stats.tx_packets++;
- self->stats.tx_bytes += skb->len;
+ netdev->stats.tx_packets++;
+ netdev->stats.tx_bytes += skb->len;
netdev->trans_start = jiffies;
}
@@ -677,7 +676,7 @@
IRDA_DEBUG(0, "%s: Tx timed out, urb->status=%d, urb->transfer_flags=0x%04X\n", netdev->name, urb->status, urb->transfer_flags);
/* Increase error count */
- self->stats.tx_errors++;
+ netdev->stats.tx_errors++;
#ifdef IU_BUG_KICK_TIMEOUT
/* Can't be a bad idea to reset the speed ;-) - Jean II */
@@ -826,7 +825,7 @@
if (urb->status != 0) {
switch (urb->status) {
case -EILSEQ:
- self->stats.rx_crc_errors++;
+ self->netdev->stats.rx_crc_errors++;
/* Also precursor to a hot-unplug on UHCI. */
/* Fallthrough... */
case -ECONNRESET:
@@ -839,7 +838,7 @@
case -ETIME:
/* Usually precursor to a hot-unplug on OHCI. */
default:
- self->stats.rx_errors++;
+ self->netdev->stats.rx_errors++;
IRDA_DEBUG(0, "%s(), RX status %d, transfer_flags 0x%04X \n", __func__, urb->status, urb->transfer_flags);
break;
}
@@ -890,7 +889,7 @@
IRDA_SKB_MAX_MTU);
if (!newskb) {
- self->stats.rx_dropped++;
+ self->netdev->stats.rx_dropped++;
/* We could deliver the current skb, but this would stall
* the Rx path. Better drop the packet... Jean II */
goto done;
@@ -927,8 +926,8 @@
netif_rx(dataskb);
/* Keep stats up to date */
- self->stats.rx_bytes += len;
- self->stats.rx_packets++;
+ self->netdev->stats.rx_bytes += len;
+ self->netdev->stats.rx_packets++;
done:
/* Note : at this point, the URB we've just received (urb)
@@ -1342,14 +1341,6 @@
}
/*------------------------------------------------------------------*/
-/*
- * Get device stats (for /proc/net/dev and ifconfig)
- */
-static struct net_device_stats *irda_usb_net_get_stats(struct net_device *dev)
-{
- struct irda_usb_cb *self = netdev_priv(dev);
- return &self->stats;
-}
/********************* IRDA CONFIG SUBROUTINES *********************/
/*
@@ -1428,7 +1419,6 @@
netdev->watchdog_timeo = 250*HZ/1000; /* 250 ms > USB timeout */
netdev->open = irda_usb_net_open;
netdev->stop = irda_usb_net_close;
- netdev->get_stats = irda_usb_net_get_stats;
netdev->do_ioctl = irda_usb_net_ioctl;
return register_netdev(netdev);
diff --git a/drivers/net/irda/irda-usb.h b/drivers/net/irda/irda-usb.h
index a0ca9c1..ac0443d 100644
--- a/drivers/net/irda/irda-usb.h
+++ b/drivers/net/irda/irda-usb.h
@@ -152,7 +152,6 @@
struct urb *speed_urb; /* URB used to send speed commands */
struct net_device *netdev; /* Yes! we are some kind of netdev. */
- struct net_device_stats stats;
struct irlap_cb *irlap; /* The link layer we are binded to */
struct qos_info qos;
char *speed_buff; /* Buffer for speed changes */
diff --git a/drivers/net/irda/kingsun-sir.c b/drivers/net/irda/kingsun-sir.c
index c747c87..b4a6171 100644
--- a/drivers/net/irda/kingsun-sir.c
+++ b/drivers/net/irda/kingsun-sir.c
@@ -105,7 +105,7 @@
struct usb_device *usbdev; /* init: probe_irda */
struct net_device *netdev; /* network layer */
struct irlap_cb *irlap; /* The link layer we are binded to */
- struct net_device_stats stats; /* network statistics */
+
struct qos_info qos;
__u8 *in_buf; /* receive buffer */
@@ -186,12 +186,12 @@
case -EPIPE:
break;
default:
- kingsun->stats.tx_errors++;
+ netdev->stats.tx_errors++;
netif_start_queue(netdev);
}
} else {
- kingsun->stats.tx_packets++;
- kingsun->stats.tx_bytes += skb->len;
+ netdev->stats.tx_packets++;
+ netdev->stats.tx_bytes += skb->len;
}
dev_kfree_skb(skb);
@@ -232,7 +232,7 @@
if (bytes[0] >= 1 && bytes[0] < kingsun->max_rx) {
for (i = 1; i <= bytes[0]; i++) {
async_unwrap_char(kingsun->netdev,
- &kingsun->stats,
+ &kingsun->netdev->stats,
&kingsun->rx_buff, bytes[i]);
}
do_gettimeofday(&kingsun->rx_time);
@@ -418,15 +418,6 @@
return ret;
}
-/*
- * Get device stats (for /proc/net/dev and ifconfig)
- */
-static struct net_device_stats *
-kingsun_net_get_stats(struct net_device *netdev)
-{
- struct kingsun_cb *kingsun = netdev_priv(netdev);
- return &kingsun->stats;
-}
/*
* This routine is called by the USB subsystem for each new device
@@ -532,7 +523,6 @@
net->hard_start_xmit = kingsun_hard_xmit;
net->open = kingsun_net_open;
net->stop = kingsun_net_close;
- net->get_stats = kingsun_net_get_stats;
net->do_ioctl = kingsun_net_ioctl;
ret = register_netdev(net);
diff --git a/drivers/net/irda/ks959-sir.c b/drivers/net/irda/ks959-sir.c
index 600d96f..55322fb 100644
--- a/drivers/net/irda/ks959-sir.c
+++ b/drivers/net/irda/ks959-sir.c
@@ -174,7 +174,7 @@
struct usb_device *usbdev; /* init: probe_irda */
struct net_device *netdev; /* network layer */
struct irlap_cb *irlap; /* The link layer we are binded to */
- struct net_device_stats stats; /* network statistics */
+
struct qos_info qos;
struct usb_ctrlrequest *tx_setuprequest;
@@ -366,7 +366,7 @@
case -EPIPE:
break;
default:
- kingsun->stats.tx_errors++;
+ netdev->stats.tx_errors++;
netif_start_queue(netdev);
}
}
@@ -416,12 +416,12 @@
case -EPIPE:
break;
default:
- kingsun->stats.tx_errors++;
+ netdev->stats.tx_errors++;
netif_start_queue(netdev);
}
} else {
- kingsun->stats.tx_packets++;
- kingsun->stats.tx_bytes += skb->len;
+ netdev->stats.tx_packets++;
+ netdev->stats.tx_bytes += skb->len;
}
@@ -469,7 +469,7 @@
*/
if (kingsun->rx_variable_xormask != 0) {
async_unwrap_char(kingsun->netdev,
- &kingsun->stats,
+ &kingsun->netdev->stats,
&kingsun->rx_unwrap_buff,
bytes[i]);
}
@@ -669,15 +669,6 @@
}
/*
- * Get device stats (for /proc/net/dev and ifconfig)
- */
-static struct net_device_stats *ks959_net_get_stats(struct net_device *netdev)
-{
- struct ks959_cb *kingsun = netdev_priv(netdev);
- return &kingsun->stats;
-}
-
-/*
* This routine is called by the USB subsystem for each new device
* in the system. We need to check if the device is ours, and in
* this case start handling it.
@@ -792,7 +783,6 @@
net->hard_start_xmit = ks959_hard_xmit;
net->open = ks959_net_open;
net->stop = ks959_net_close;
- net->get_stats = ks959_net_get_stats;
net->do_ioctl = ks959_net_ioctl;
ret = register_netdev(net);
diff --git a/drivers/net/irda/ksdazzle-sir.c b/drivers/net/irda/ksdazzle-sir.c
index 0e7f893..5b327b0 100644
--- a/drivers/net/irda/ksdazzle-sir.c
+++ b/drivers/net/irda/ksdazzle-sir.c
@@ -140,7 +140,7 @@
struct usb_device *usbdev; /* init: probe_irda */
struct net_device *netdev; /* network layer */
struct irlap_cb *irlap; /* The link layer we are binded to */
- struct net_device_stats stats; /* network statistics */
+
struct qos_info qos;
struct urb *tx_urb;
@@ -278,7 +278,7 @@
case -EPIPE:
break;
default:
- kingsun->stats.tx_errors++;
+ netdev->stats.tx_errors++;
netif_start_queue(netdev);
}
}
@@ -329,12 +329,12 @@
case -EPIPE:
break;
default:
- kingsun->stats.tx_errors++;
+ netdev->stats.tx_errors++;
netif_start_queue(netdev);
}
} else {
- kingsun->stats.tx_packets++;
- kingsun->stats.tx_bytes += skb->len;
+ netdev->stats.tx_packets++;
+ netdev->stats.tx_bytes += skb->len;
}
@@ -348,9 +348,10 @@
static void ksdazzle_rcv_irq(struct urb *urb)
{
struct ksdazzle_cb *kingsun = urb->context;
+ struct net_device *netdev = kingsun->netdev;
/* in process of stopping, just drop data */
- if (!netif_running(kingsun->netdev)) {
+ if (!netif_running(netdev)) {
kingsun->receiving = 0;
return;
}
@@ -368,7 +369,7 @@
unsigned int i;
for (i = 0; i < urb->actual_length; i++) {
- async_unwrap_char(kingsun->netdev, &kingsun->stats,
+ async_unwrap_char(netdev, &netdev->stats,
&kingsun->rx_unwrap_buff, bytes[i]);
}
kingsun->receiving =
@@ -562,16 +563,6 @@
}
/*
- * Get device stats (for /proc/net/dev and ifconfig)
- */
-static struct net_device_stats *ksdazzle_net_get_stats(struct net_device
- *netdev)
-{
- struct ksdazzle_cb *kingsun = netdev_priv(netdev);
- return &kingsun->stats;
-}
-
-/*
* This routine is called by the USB subsystem for each new device
* in the system. We need to check if the device is ours, and in
* this case start handling it.
@@ -696,7 +687,6 @@
net->hard_start_xmit = ksdazzle_hard_xmit;
net->open = ksdazzle_net_open;
net->stop = ksdazzle_net_close;
- net->get_stats = ksdazzle_net_get_stats;
net->do_ioctl = ksdazzle_net_ioctl;
ret = register_netdev(net);
diff --git a/drivers/net/irda/mcs7780.c b/drivers/net/irda/mcs7780.c
index 904c961..7eafdca 100644
--- a/drivers/net/irda/mcs7780.c
+++ b/drivers/net/irda/mcs7780.c
@@ -403,8 +403,8 @@
if(unlikely(new_len <= 0)) {
IRDA_ERROR("%s short frame length %d\n",
mcs->netdev->name, new_len);
- ++mcs->stats.rx_errors;
- ++mcs->stats.rx_length_errors;
+ ++mcs->netdev->stats.rx_errors;
+ ++mcs->netdev->stats.rx_length_errors;
return;
}
fcs = 0;
@@ -413,14 +413,14 @@
if(fcs != GOOD_FCS) {
IRDA_ERROR("crc error calc 0x%x len %d\n",
fcs, new_len);
- mcs->stats.rx_errors++;
- mcs->stats.rx_crc_errors++;
+ mcs->netdev->stats.rx_errors++;
+ mcs->netdev->stats.rx_crc_errors++;
return;
}
skb = dev_alloc_skb(new_len + 1);
if(unlikely(!skb)) {
- ++mcs->stats.rx_dropped;
+ ++mcs->netdev->stats.rx_dropped;
return;
}
@@ -433,8 +433,8 @@
netif_rx(skb);
- mcs->stats.rx_packets++;
- mcs->stats.rx_bytes += new_len;
+ mcs->netdev->stats.rx_packets++;
+ mcs->netdev->stats.rx_bytes += new_len;
return;
}
@@ -458,22 +458,22 @@
if(unlikely(new_len <= 0)) {
IRDA_ERROR("%s short frame length %d\n",
mcs->netdev->name, new_len);
- ++mcs->stats.rx_errors;
- ++mcs->stats.rx_length_errors;
+ ++mcs->netdev->stats.rx_errors;
+ ++mcs->netdev->stats.rx_length_errors;
return;
}
fcs = ~(crc32_le(~0, buf, new_len));
if(fcs != get_unaligned_le32(buf + new_len)) {
IRDA_ERROR("crc error calc 0x%x len %d\n", fcs, new_len);
- mcs->stats.rx_errors++;
- mcs->stats.rx_crc_errors++;
+ mcs->netdev->stats.rx_errors++;
+ mcs->netdev->stats.rx_crc_errors++;
return;
}
skb = dev_alloc_skb(new_len + 1);
if(unlikely(!skb)) {
- ++mcs->stats.rx_dropped;
+ ++mcs->netdev->stats.rx_dropped;
return;
}
@@ -486,8 +486,8 @@
netif_rx(skb);
- mcs->stats.rx_packets++;
- mcs->stats.rx_bytes += new_len;
+ mcs->netdev->stats.rx_packets++;
+ mcs->netdev->stats.rx_bytes += new_len;
return;
}
@@ -756,14 +756,6 @@
return ret;
}
-
-/* Get device stats for /proc/net/dev and ifconfig */
-static struct net_device_stats *mcs_net_get_stats(struct net_device *netdev)
-{
- struct mcs_cb *mcs = netdev_priv(netdev);
- return &mcs->stats;
-}
-
/* Receive callback function. */
static void mcs_receive_irq(struct urb *urb)
{
@@ -786,14 +778,14 @@
*/
/* SIR speed */
if(mcs->speed < 576000) {
- async_unwrap_char(mcs->netdev, &mcs->stats,
+ async_unwrap_char(mcs->netdev, &mcs->netdev->stats,
&mcs->rx_buff, 0xc0);
for (i = 0; i < urb->actual_length; i++)
- async_unwrap_char(mcs->netdev, &mcs->stats,
+ async_unwrap_char(mcs->netdev, &mcs->netdev->stats,
&mcs->rx_buff, bytes[i]);
- async_unwrap_char(mcs->netdev, &mcs->stats,
+ async_unwrap_char(mcs->netdev, &mcs->netdev->stats,
&mcs->rx_buff, 0xc1);
}
/* MIR speed */
@@ -868,12 +860,12 @@
case -EPIPE:
break;
default:
- mcs->stats.tx_errors++;
+ mcs->netdev->stats.tx_errors++;
netif_start_queue(ndev);
}
} else {
- mcs->stats.tx_packets++;
- mcs->stats.tx_bytes += skb->len;
+ mcs->netdev->stats.tx_packets++;
+ mcs->netdev->stats.tx_bytes += skb->len;
}
dev_kfree_skb(skb);
@@ -931,7 +923,6 @@
ndev->hard_start_xmit = mcs_hard_xmit;
ndev->open = mcs_net_open;
ndev->stop = mcs_net_close;
- ndev->get_stats = mcs_net_get_stats;
ndev->do_ioctl = mcs_net_ioctl;
if (!intf->cur_altsetting)
diff --git a/drivers/net/irda/mcs7780.h b/drivers/net/irda/mcs7780.h
index b18148c..6bdc621 100644
--- a/drivers/net/irda/mcs7780.h
+++ b/drivers/net/irda/mcs7780.h
@@ -104,7 +104,6 @@
struct usb_device *usbdev; /* init: probe_irda */
struct net_device *netdev; /* network layer */
struct irlap_cb *irlap; /* The link layer we are binded to */
- struct net_device_stats stats; /* network statistics */
struct qos_info qos;
unsigned int speed; /* Current speed */
unsigned int new_speed; /* new speed */
@@ -154,7 +153,6 @@
static int mcs_net_ioctl(struct net_device *netdev, struct ifreq *rq, int cmd);
static int mcs_net_close(struct net_device *netdev);
static int mcs_net_open(struct net_device *netdev);
-static struct net_device_stats *mcs_net_get_stats(struct net_device *netdev);
static void mcs_receive_irq(struct urb *urb);
static void mcs_send_irq(struct urb *urb);
diff --git a/drivers/net/irda/nsc-ircc.c b/drivers/net/irda/nsc-ircc.c
index 2c6bf2d..61e509c 100644
--- a/drivers/net/irda/nsc-ircc.c
+++ b/drivers/net/irda/nsc-ircc.c
@@ -185,7 +185,6 @@
static int nsc_ircc_net_open(struct net_device *dev);
static int nsc_ircc_net_close(struct net_device *dev);
static int nsc_ircc_net_ioctl(struct net_device *dev, struct ifreq *rq, int cmd);
-static struct net_device_stats *nsc_ircc_net_get_stats(struct net_device *dev);
/* Globals */
static int pnp_registered;
@@ -446,7 +445,6 @@
dev->open = nsc_ircc_net_open;
dev->stop = nsc_ircc_net_close;
dev->do_ioctl = nsc_ircc_net_ioctl;
- dev->get_stats = nsc_ircc_net_get_stats;
err = register_netdev(dev);
if (err) {
@@ -1401,7 +1399,7 @@
self->tx_buff.len = async_wrap_skb(skb, self->tx_buff.data,
self->tx_buff.truesize);
- self->stats.tx_bytes += self->tx_buff.len;
+ dev->stats.tx_bytes += self->tx_buff.len;
/* Add interrupt on tx low level (will fire immediately) */
switch_bank(iobase, BANK0);
@@ -1473,7 +1471,7 @@
self->tx_fifo.queue[self->tx_fifo.free].len = skb->len;
self->tx_fifo.tail += skb->len;
- self->stats.tx_bytes += skb->len;
+ dev->stats.tx_bytes += skb->len;
skb_copy_from_linear_data(skb, self->tx_fifo.queue[self->tx_fifo.free].start,
skb->len);
@@ -1652,13 +1650,13 @@
/* Check for underrrun! */
if (inb(iobase+ASCR) & ASCR_TXUR) {
- self->stats.tx_errors++;
- self->stats.tx_fifo_errors++;
+ self->netdev->stats.tx_errors++;
+ self->netdev->stats.tx_fifo_errors++;
/* Clear bit, by writing 1 into it */
outb(ASCR_TXUR, iobase+ASCR);
} else {
- self->stats.tx_packets++;
+ self->netdev->stats.tx_packets++;
}
/* Finished with this frame, so prepare for next */
@@ -1793,28 +1791,28 @@
if (status & FRM_ST_ERR_MSK) {
if (status & FRM_ST_LOST_FR) {
/* Add number of lost frames to stats */
- self->stats.rx_errors += len;
+ self->netdev->stats.rx_errors += len;
} else {
/* Skip frame */
- self->stats.rx_errors++;
+ self->netdev->stats.rx_errors++;
self->rx_buff.data += len;
if (status & FRM_ST_MAX_LEN)
- self->stats.rx_length_errors++;
+ self->netdev->stats.rx_length_errors++;
if (status & FRM_ST_PHY_ERR)
- self->stats.rx_frame_errors++;
+ self->netdev->stats.rx_frame_errors++;
if (status & FRM_ST_BAD_CRC)
- self->stats.rx_crc_errors++;
+ self->netdev->stats.rx_crc_errors++;
}
/* The errors below can be reported in both cases */
if (status & FRM_ST_OVR1)
- self->stats.rx_fifo_errors++;
+ self->netdev->stats.rx_fifo_errors++;
if (status & FRM_ST_OVR2)
- self->stats.rx_fifo_errors++;
+ self->netdev->stats.rx_fifo_errors++;
} else {
/*
* First we must make sure that the frame we
@@ -1863,7 +1861,7 @@
IRDA_WARNING("%s(), memory squeeze, "
"dropping frame.\n",
__func__);
- self->stats.rx_dropped++;
+ self->netdev->stats.rx_dropped++;
/* Restore bank register */
outb(bank, iobase+BSR);
@@ -1889,8 +1887,8 @@
/* Move to next frame */
self->rx_buff.data += len;
- self->stats.rx_bytes += len;
- self->stats.rx_packets++;
+ self->netdev->stats.rx_bytes += len;
+ self->netdev->stats.rx_packets++;
skb->dev = self->netdev;
skb_reset_mac_header(skb);
@@ -1920,8 +1918,8 @@
/* Receive all characters in Rx FIFO */
do {
byte = inb(iobase+RXD);
- async_unwrap_char(self->netdev, &self->stats, &self->rx_buff,
- byte);
+ async_unwrap_char(self->netdev, &self->netdev->stats,
+ &self->rx_buff, byte);
} while (inb(iobase+LSR) & LSR_RXDA); /* Data available */
}
@@ -1952,7 +1950,7 @@
self->ier = IER_TXLDL_IE;
else {
- self->stats.tx_packets++;
+ self->netdev->stats.tx_packets++;
netif_wake_queue(self->netdev);
self->ier = IER_TXEMP_IE;
}
@@ -2307,13 +2305,6 @@
return ret;
}
-static struct net_device_stats *nsc_ircc_net_get_stats(struct net_device *dev)
-{
- struct nsc_ircc_cb *self = netdev_priv(dev);
-
- return &self->stats;
-}
-
static int nsc_ircc_suspend(struct platform_device *dev, pm_message_t state)
{
struct nsc_ircc_cb *self = platform_get_drvdata(dev);
diff --git a/drivers/net/irda/nsc-ircc.h b/drivers/net/irda/nsc-ircc.h
index 71cd3c5..7ba7738 100644
--- a/drivers/net/irda/nsc-ircc.h
+++ b/drivers/net/irda/nsc-ircc.h
@@ -251,7 +251,6 @@
struct tx_fifo tx_fifo; /* Info about frames to be transmitted */
struct net_device *netdev; /* Yes! we are some kind of netdevice */
- struct net_device_stats stats;
struct irlap_cb *irlap; /* The link layer we are binded to */
struct qos_info qos; /* QoS capabilities for this device */
diff --git a/drivers/net/irda/pxaficp_ir.c b/drivers/net/irda/pxaficp_ir.c
index 004a9aa..31794c2 100644
--- a/drivers/net/irda/pxaficp_ir.c
+++ b/drivers/net/irda/pxaficp_ir.c
@@ -108,7 +108,6 @@
int txdma;
int rxdma;
- struct net_device_stats stats;
struct irlap_cb *irlap;
struct qos_info qos;
@@ -258,14 +257,15 @@
data = STRBR;
if (lsr & (LSR_OE | LSR_PE | LSR_FE | LSR_BI)) {
printk(KERN_DEBUG "pxa_ir: sir receiving error\n");
- si->stats.rx_errors++;
+ dev->stats.rx_errors++;
if (lsr & LSR_FE)
- si->stats.rx_frame_errors++;
+ dev->stats.rx_frame_errors++;
if (lsr & LSR_OE)
- si->stats.rx_fifo_errors++;
+ dev->stats.rx_fifo_errors++;
} else {
- si->stats.rx_bytes++;
- async_unwrap_char(dev, &si->stats, &si->rx_buff, data);
+ dev->stats.rx_bytes++;
+ async_unwrap_char(dev, &dev->stats,
+ &si->rx_buff, data);
}
lsr = STLSR;
}
@@ -277,8 +277,8 @@
case 0x0C: /* Character Timeout Indication */
do {
- si->stats.rx_bytes++;
- async_unwrap_char(dev, &si->stats, &si->rx_buff, STRBR);
+ dev->stats.rx_bytes++;
+ async_unwrap_char(dev, &dev->stats, &si->rx_buff, STRBR);
} while (STLSR & LSR_DR);
si->last_oscr = OSCR;
break;
@@ -290,9 +290,8 @@
}
if (si->tx_buff.len == 0) {
- si->stats.tx_packets++;
- si->stats.tx_bytes += si->tx_buff.data -
- si->tx_buff.head;
+ dev->stats.tx_packets++;
+ dev->stats.tx_bytes += si->tx_buff.data - si->tx_buff.head;
/* We need to ensure that the transmitter has finished. */
while ((STLSR & LSR_TEMT) == 0)
@@ -343,10 +342,10 @@
DCSR(channel) = dcsr & ~DCSR_RUN;
if (dcsr & DCSR_ENDINTR) {
- si->stats.tx_packets++;
- si->stats.tx_bytes += si->dma_tx_buff_len;
+ dev->stats.tx_packets++;
+ dev->stats.tx_bytes += si->dma_tx_buff_len;
} else {
- si->stats.tx_errors++;
+ dev->stats.tx_errors++;
}
while (ICSR1 & ICSR1_TBY)
@@ -392,14 +391,14 @@
data = ICDR;
if (stat & (ICSR1_CRE | ICSR1_ROR)) {
- si->stats.rx_errors++;
+ dev->stats.rx_errors++;
if (stat & ICSR1_CRE) {
printk(KERN_DEBUG "pxa_ir: fir receive CRC error\n");
- si->stats.rx_crc_errors++;
+ dev->stats.rx_crc_errors++;
}
if (stat & ICSR1_ROR) {
printk(KERN_DEBUG "pxa_ir: fir receive overrun\n");
- si->stats.rx_over_errors++;
+ dev->stats.rx_over_errors++;
}
} else {
si->dma_rx_buff[len++] = data;
@@ -415,14 +414,14 @@
if (icsr0 & ICSR0_FRE) {
printk(KERN_ERR "pxa_ir: dropping erroneous frame\n");
- si->stats.rx_dropped++;
+ dev->stats.rx_dropped++;
return;
}
skb = alloc_skb(len+1,GFP_ATOMIC);
if (!skb) {
printk(KERN_ERR "pxa_ir: fir out of memory for receive skb\n");
- si->stats.rx_dropped++;
+ dev->stats.rx_dropped++;
return;
}
@@ -437,8 +436,8 @@
skb->protocol = htons(ETH_P_IRDA);
netif_rx(skb);
- si->stats.rx_packets++;
- si->stats.rx_bytes += len;
+ dev->stats.rx_packets++;
+ dev->stats.rx_bytes += len;
}
}
@@ -457,10 +456,10 @@
if (icsr0 & (ICSR0_FRE | ICSR0_RAB)) {
if (icsr0 & ICSR0_FRE) {
printk(KERN_DEBUG "pxa_ir: fir receive frame error\n");
- si->stats.rx_frame_errors++;
+ dev->stats.rx_frame_errors++;
} else {
printk(KERN_DEBUG "pxa_ir: fir receive abort\n");
- si->stats.rx_errors++;
+ dev->stats.rx_errors++;
}
ICSR0 = icsr0 & (ICSR0_FRE | ICSR0_RAB);
}
@@ -589,12 +588,6 @@
return ret;
}
-static struct net_device_stats *pxa_irda_stats(struct net_device *dev)
-{
- struct pxa_irda *si = netdev_priv(dev);
- return &si->stats;
-}
-
static void pxa_irda_startup(struct pxa_irda *si)
{
/* Disable STUART interrupts */
@@ -857,7 +850,6 @@
dev->open = pxa_irda_start;
dev->stop = pxa_irda_stop;
dev->do_ioctl = pxa_irda_ioctl;
- dev->get_stats = pxa_irda_stats;
irda_init_max_qos_capabilies(&si->qos);
diff --git a/drivers/net/irda/sa1100_ir.c b/drivers/net/irda/sa1100_ir.c
index d302bcf..7a2b003 100644
--- a/drivers/net/irda/sa1100_ir.c
+++ b/drivers/net/irda/sa1100_ir.c
@@ -60,7 +60,6 @@
dma_regs_t *txdma;
dma_regs_t *rxdma;
- struct net_device_stats stats;
struct device *dev;
struct irda_platform_data *pdata;
struct irlap_cb *irlap;
@@ -375,13 +374,13 @@
data = Ser2UTDR;
if (stat & (UTSR1_FRE | UTSR1_ROR)) {
- si->stats.rx_errors++;
+ dev->stats.rx_errors++;
if (stat & UTSR1_FRE)
- si->stats.rx_frame_errors++;
+ dev->stats.rx_frame_errors++;
if (stat & UTSR1_ROR)
- si->stats.rx_fifo_errors++;
+ dev->stats.rx_fifo_errors++;
} else
- async_unwrap_char(dev, &si->stats, &si->rx_buff, data);
+ async_unwrap_char(dev, &dev->stats, &si->rx_buff, data);
status = Ser2UTSR0;
}
@@ -396,9 +395,9 @@
* There are at least 4 bytes in the FIFO. Read 3 bytes
* and leave the rest to the block below.
*/
- async_unwrap_char(dev, &si->stats, &si->rx_buff, Ser2UTDR);
- async_unwrap_char(dev, &si->stats, &si->rx_buff, Ser2UTDR);
- async_unwrap_char(dev, &si->stats, &si->rx_buff, Ser2UTDR);
+ async_unwrap_char(dev, &dev->stats, &si->rx_buff, Ser2UTDR);
+ async_unwrap_char(dev, &dev->stats, &si->rx_buff, Ser2UTDR);
+ async_unwrap_char(dev, &dev->stats, &si->rx_buff, Ser2UTDR);
}
if (status & (UTSR0_RFS | UTSR0_RID)) {
@@ -406,7 +405,7 @@
* Fifo contains more than 1 character.
*/
do {
- async_unwrap_char(dev, &si->stats, &si->rx_buff,
+ async_unwrap_char(dev, &dev->stats, &si->rx_buff,
Ser2UTDR);
} while (Ser2UTSR1 & UTSR1_RNE);
@@ -422,8 +421,8 @@
} while (Ser2UTSR1 & UTSR1_TNF && si->tx_buff.len);
if (si->tx_buff.len == 0) {
- si->stats.tx_packets++;
- si->stats.tx_bytes += si->tx_buff.data -
+ dev->stats.tx_packets++;
+ dev->stats.tx_bytes += si->tx_buff.data -
si->tx_buff.head;
/*
@@ -482,11 +481,11 @@
data = Ser2HSDR;
if (stat & (HSSR1_CRE | HSSR1_ROR)) {
- si->stats.rx_errors++;
+ dev->stats.rx_errors++;
if (stat & HSSR1_CRE)
- si->stats.rx_crc_errors++;
+ dev->stats.rx_crc_errors++;
if (stat & HSSR1_ROR)
- si->stats.rx_frame_errors++;
+ dev->stats.rx_frame_errors++;
} else
skb->data[len++] = data;
@@ -505,8 +504,8 @@
skb->dev = dev;
skb_reset_mac_header(skb);
skb->protocol = htons(ETH_P_IRDA);
- si->stats.rx_packets++;
- si->stats.rx_bytes += len;
+ dev->stats.rx_packets++;
+ dev->stats.rx_bytes += len;
/*
* Before we pass the buffer up, allocate a new one.
@@ -545,10 +544,10 @@
* from the fifo.
*/
if (Ser2HSSR0 & (HSSR0_FRE | HSSR0_RAB)) {
- si->stats.rx_errors++;
+ dev->stats.rx_errors++;
if (Ser2HSSR0 & HSSR0_FRE)
- si->stats.rx_frame_errors++;
+ dev->stats.rx_frame_errors++;
/*
* Clear out the DMA...
@@ -633,8 +632,8 @@
*/
if (skb) {
dma_unmap_single(si->dev, si->txbuf_dma, skb->len, DMA_TO_DEVICE);
- si->stats.tx_packets ++;
- si->stats.tx_bytes += skb->len;
+ dev->stats.tx_packets ++;
+ dev->stats.tx_bytes += skb->len;
dev_kfree_skb_irq(skb);
}
@@ -762,12 +761,6 @@
return ret;
}
-static struct net_device_stats *sa1100_irda_stats(struct net_device *dev)
-{
- struct sa1100_irda *si = netdev_priv(dev);
- return &si->stats;
-}
-
static int sa1100_irda_start(struct net_device *dev)
{
struct sa1100_irda *si = netdev_priv(dev);
@@ -924,7 +917,6 @@
dev->open = sa1100_irda_start;
dev->stop = sa1100_irda_stop;
dev->do_ioctl = sa1100_irda_ioctl;
- dev->get_stats = sa1100_irda_stats;
dev->irq = IRQ_Ser2ICP;
irda_init_max_qos_capabilies(&si->qos);
diff --git a/drivers/net/irda/sir-dev.h b/drivers/net/irda/sir-dev.h
index 2a57bc6..6d5b1e2 100644
--- a/drivers/net/irda/sir-dev.h
+++ b/drivers/net/irda/sir-dev.h
@@ -160,7 +160,6 @@
struct sir_dev {
struct net_device *netdev;
- struct net_device_stats stats;
struct irlap_cb *irlap;
diff --git a/drivers/net/irda/sir_dev.c b/drivers/net/irda/sir_dev.c
index ceef040..5b58624 100644
--- a/drivers/net/irda/sir_dev.c
+++ b/drivers/net/irda/sir_dev.c
@@ -455,8 +455,8 @@
if ((skb=dev->tx_skb) != NULL) {
dev->tx_skb = NULL;
dev_kfree_skb_any(skb);
- dev->stats.tx_errors++;
- dev->stats.tx_dropped++;
+ dev->netdev->stats.tx_errors++;
+ dev->netdev->stats.tx_dropped++;
}
dev->tx_buff.len = 0;
}
@@ -493,8 +493,8 @@
if ((skb=dev->tx_skb) != NULL) {
dev->tx_skb = NULL;
- dev->stats.tx_packets++;
- dev->stats.tx_bytes += skb->len;
+ dev->netdev->stats.tx_packets++;
+ dev->netdev->stats.tx_bytes += skb->len;
dev_kfree_skb_any(skb);
}
@@ -548,7 +548,7 @@
* just update stats and set media busy
*/
irda_device_set_media_busy(dev->netdev, TRUE);
- dev->stats.rx_dropped++;
+ dev->netdev->stats.rx_dropped++;
IRDA_DEBUG(0, "%s; rx-drop: %zd\n", __func__, count);
return 0;
}
@@ -557,7 +557,7 @@
if (likely(atomic_read(&dev->enable_rx))) {
while (count--)
/* Unwrap and destuff one byte */
- async_unwrap_char(dev->netdev, &dev->stats,
+ async_unwrap_char(dev->netdev, &dev->netdev->stats,
&dev->rx_buff, *cp++);
} else {
while (count--) {
@@ -582,13 +582,6 @@
/* callbacks from network layer */
-static struct net_device_stats *sirdev_get_stats(struct net_device *ndev)
-{
- struct sir_dev *dev = netdev_priv(ndev);
-
- return (dev) ? &dev->stats : NULL;
-}
-
static int sirdev_hard_xmit(struct sk_buff *skb, struct net_device *ndev)
{
struct sir_dev *dev = netdev_priv(ndev);
@@ -654,7 +647,7 @@
*/
atomic_set(&dev->enable_rx, 0);
if (unlikely(sirdev_is_receiving(dev)))
- dev->stats.collisions++;
+ dev->netdev->stats.collisions++;
actual = dev->drv->do_write(dev, dev->tx_buff.data, dev->tx_buff.len);
@@ -669,8 +662,8 @@
IRDA_ERROR("%s: drv->do_write failed (%d)\n",
__func__, actual);
dev_kfree_skb_any(skb);
- dev->stats.tx_errors++;
- dev->stats.tx_dropped++;
+ dev->netdev->stats.tx_errors++;
+ dev->netdev->stats.tx_dropped++;
netif_wake_queue(ndev);
}
spin_unlock_irqrestore(&dev->tx_lock, flags);
@@ -918,7 +911,6 @@
ndev->hard_start_xmit = sirdev_hard_xmit;
ndev->open = sirdev_open;
ndev->stop = sirdev_close;
- ndev->get_stats = sirdev_get_stats;
ndev->do_ioctl = sirdev_ioctl;
if (register_netdev(ndev)) {
diff --git a/drivers/net/irda/smsc-ircc2.c b/drivers/net/irda/smsc-ircc2.c
index 5d09e15..dd73cce 100644
--- a/drivers/net/irda/smsc-ircc2.c
+++ b/drivers/net/irda/smsc-ircc2.c
@@ -150,7 +150,6 @@
/* Private data for each instance */
struct smsc_ircc_cb {
struct net_device *netdev; /* Yes! we are some kind of netdevice */
- struct net_device_stats stats;
struct irlap_cb *irlap; /* The link layer we are binded to */
chipio_t io; /* IrDA controller information */
@@ -215,7 +214,6 @@
#if SMSC_IRCC2_C_NET_TIMEOUT
static void smsc_ircc_timeout(struct net_device *dev);
#endif
-static struct net_device_stats *smsc_ircc_net_get_stats(struct net_device *dev);
static int smsc_ircc_is_receiving(struct smsc_ircc_cb *self);
static void smsc_ircc_probe_transceiver(struct smsc_ircc_cb *self);
static void smsc_ircc_set_transceiver_for_speed(struct smsc_ircc_cb *self, u32 speed);
@@ -529,7 +527,6 @@
dev->open = smsc_ircc_net_open;
dev->stop = smsc_ircc_net_close;
dev->do_ioctl = smsc_ircc_net_ioctl;
- dev->get_stats = smsc_ircc_net_get_stats;
self = netdev_priv(dev);
self->netdev = dev;
@@ -834,13 +831,6 @@
return ret;
}
-static struct net_device_stats *smsc_ircc_net_get_stats(struct net_device *dev)
-{
- struct smsc_ircc_cb *self = netdev_priv(dev);
-
- return &self->stats;
-}
-
#if SMSC_IRCC2_C_NET_TIMEOUT
/*
* Function smsc_ircc_timeout (struct net_device *dev)
@@ -920,7 +910,7 @@
self->tx_buff.len = async_wrap_skb(skb, self->tx_buff.data,
self->tx_buff.truesize);
- self->stats.tx_bytes += self->tx_buff.len;
+ dev->stats.tx_bytes += self->tx_buff.len;
/* Turn on transmit finished interrupt. Will fire immediately! */
outb(UART_IER_THRI, self->io.sir_base + UART_IER);
@@ -1320,16 +1310,16 @@
/* Check for underrun! */
register_bank(iobase, 0);
if (inb(iobase + IRCC_LSR) & IRCC_LSR_UNDERRUN) {
- self->stats.tx_errors++;
- self->stats.tx_fifo_errors++;
+ self->netdev->stats.tx_errors++;
+ self->netdev->stats.tx_fifo_errors++;
/* Reset error condition */
register_bank(iobase, 0);
outb(IRCC_MASTER_ERROR_RESET, iobase + IRCC_MASTER);
outb(0x00, iobase + IRCC_MASTER);
} else {
- self->stats.tx_packets++;
- self->stats.tx_bytes += self->tx_buff.len;
+ self->netdev->stats.tx_packets++;
+ self->netdev->stats.tx_bytes += self->tx_buff.len;
}
/* Check if it's time to change the speed */
@@ -1429,15 +1419,15 @@
/* Look for errors */
if (lsr & (IRCC_LSR_FRAME_ERROR | IRCC_LSR_CRC_ERROR | IRCC_LSR_SIZE_ERROR)) {
- self->stats.rx_errors++;
+ self->netdev->stats.rx_errors++;
if (lsr & IRCC_LSR_FRAME_ERROR)
- self->stats.rx_frame_errors++;
+ self->netdev->stats.rx_frame_errors++;
if (lsr & IRCC_LSR_CRC_ERROR)
- self->stats.rx_crc_errors++;
+ self->netdev->stats.rx_crc_errors++;
if (lsr & IRCC_LSR_SIZE_ERROR)
- self->stats.rx_length_errors++;
+ self->netdev->stats.rx_length_errors++;
if (lsr & (IRCC_LSR_UNDERRUN | IRCC_LSR_OVERRUN))
- self->stats.rx_length_errors++;
+ self->netdev->stats.rx_length_errors++;
return;
}
@@ -1460,8 +1450,8 @@
skb_reserve(skb, 1);
memcpy(skb_put(skb, len), self->rx_buff.data, len);
- self->stats.rx_packets++;
- self->stats.rx_bytes += len;
+ self->netdev->stats.rx_packets++;
+ self->netdev->stats.rx_bytes += len;
skb->dev = self->netdev;
skb_reset_mac_header(skb);
@@ -1489,7 +1479,7 @@
* async_unwrap_char will deliver all found frames
*/
do {
- async_unwrap_char(self->netdev, &self->stats, &self->rx_buff,
+ async_unwrap_char(self->netdev, &self->netdev->stats, &self->rx_buff,
inb(iobase + UART_RX));
/* Make sure we don't stay here to long */
@@ -1992,7 +1982,7 @@
/* Tell network layer that we want more frames */
netif_wake_queue(self->netdev);
}
- self->stats.tx_packets++;
+ self->netdev->stats.tx_packets++;
if (self->io.speed <= 115200) {
/*
diff --git a/drivers/net/irda/stir4200.c b/drivers/net/irda/stir4200.c
index ca4cd92..8b1658c 100644
--- a/drivers/net/irda/stir4200.c
+++ b/drivers/net/irda/stir4200.c
@@ -164,7 +164,7 @@
struct usb_device *usbdev; /* init: probe_irda */
struct net_device *netdev; /* network layer */
struct irlap_cb *irlap; /* The link layer we are binded to */
- struct net_device_stats stats; /* network statistics */
+
struct qos_info qos;
unsigned speed; /* Current speed */
@@ -323,16 +323,16 @@
pr_debug("%s: short frame len %d\n",
stir->netdev->name, len);
- ++stir->stats.rx_errors;
- ++stir->stats.rx_length_errors;
+ ++stir->netdev->stats.rx_errors;
+ ++stir->netdev->stats.rx_length_errors;
return;
}
fcs = ~(crc32_le(~0, rx_buff->data, len));
if (fcs != get_unaligned_le32(rx_buff->data + len)) {
pr_debug("crc error calc 0x%x len %d\n", fcs, len);
- stir->stats.rx_errors++;
- stir->stats.rx_crc_errors++;
+ stir->netdev->stats.rx_errors++;
+ stir->netdev->stats.rx_crc_errors++;
return;
}
@@ -340,7 +340,7 @@
if (len < IRDA_RX_COPY_THRESHOLD) {
nskb = dev_alloc_skb(len + 1);
if (unlikely(!nskb)) {
- ++stir->stats.rx_dropped;
+ ++stir->netdev->stats.rx_dropped;
return;
}
skb_reserve(nskb, 1);
@@ -349,7 +349,7 @@
} else {
nskb = dev_alloc_skb(rx_buff->truesize);
if (unlikely(!nskb)) {
- ++stir->stats.rx_dropped;
+ ++stir->netdev->stats.rx_dropped;
return;
}
skb_reserve(nskb, 1);
@@ -366,8 +366,8 @@
netif_rx(skb);
- stir->stats.rx_packets++;
- stir->stats.rx_bytes += len;
+ stir->netdev->stats.rx_packets++;
+ stir->netdev->stats.rx_bytes += len;
rx_buff->data = rx_buff->head;
rx_buff->len = 0;
@@ -437,7 +437,7 @@
if (unlikely(rx_buff->len >= rx_buff->truesize)) {
pr_debug("%s: fir frame exceeds %d\n",
stir->netdev->name, rx_buff->truesize);
- ++stir->stats.rx_over_errors;
+ ++stir->netdev->stats.rx_over_errors;
goto error_recovery;
}
@@ -445,10 +445,10 @@
continue;
frame_error:
- ++stir->stats.rx_frame_errors;
+ ++stir->netdev->stats.rx_frame_errors;
error_recovery:
- ++stir->stats.rx_errors;
+ ++stir->netdev->stats.rx_errors;
rx_buff->state = OUTSIDE_FRAME;
rx_buff->in_frame = FALSE;
}
@@ -461,7 +461,7 @@
int i;
for (i = 0; i < len; i++)
- async_unwrap_char(stir->netdev, &stir->stats,
+ async_unwrap_char(stir->netdev, &stir->netdev->stats,
&stir->rx_buff, bytes[i]);
}
@@ -692,7 +692,7 @@
usb_kill_urb(stir->rx_urb);
if (stir->rx_buff.in_frame)
- stir->stats.collisions++;
+ stir->netdev->stats.collisions++;
}
/*
* Wrap data in socket buffer and send it.
@@ -718,15 +718,15 @@
if (!first_frame)
fifo_txwait(stir, wraplen);
- stir->stats.tx_packets++;
- stir->stats.tx_bytes += skb->len;
+ stir->netdev->stats.tx_packets++;
+ stir->netdev->stats.tx_bytes += skb->len;
stir->netdev->trans_start = jiffies;
pr_debug("send %d (%d)\n", skb->len, wraplen);
if (usb_bulk_msg(stir->usbdev, usb_sndbulkpipe(stir->usbdev, 1),
stir->io_buf, wraplen,
NULL, TRANSMIT_TIMEOUT))
- stir->stats.tx_errors++;
+ stir->netdev->stats.tx_errors++;
}
/*
@@ -1008,15 +1008,6 @@
}
/*
- * Get device stats (for /proc/net/dev and ifconfig)
- */
-static struct net_device_stats *stir_net_get_stats(struct net_device *netdev)
-{
- struct stir_cb *stir = netdev_priv(netdev);
- return &stir->stats;
-}
-
-/*
* This routine is called by the USB subsystem for each new device
* in the system. We need to check if the device is ours, and in
* this case start handling it.
@@ -1066,7 +1057,6 @@
net->hard_start_xmit = stir_hard_xmit;
net->open = stir_net_open;
net->stop = stir_net_close;
- net->get_stats = stir_net_get_stats;
net->do_ioctl = stir_net_ioctl;
ret = register_netdev(net);
diff --git a/drivers/net/irda/via-ircc.c b/drivers/net/irda/via-ircc.c
index 74c78cf..8b3e545 100644
--- a/drivers/net/irda/via-ircc.c
+++ b/drivers/net/irda/via-ircc.c
@@ -101,8 +101,6 @@
static int via_ircc_net_close(struct net_device *dev);
static int via_ircc_net_ioctl(struct net_device *dev, struct ifreq *rq,
int cmd);
-static struct net_device_stats *via_ircc_net_get_stats(struct net_device
- *dev);
static void via_ircc_change_dongle_speed(int iobase, int speed,
int dongle_id);
static int RxTimerHandler(struct via_ircc_cb *self, int iobase);
@@ -434,7 +432,6 @@
dev->open = via_ircc_net_open;
dev->stop = via_ircc_net_close;
dev->do_ioctl = via_ircc_net_ioctl;
- dev->get_stats = via_ircc_net_get_stats;
err = register_netdev(dev);
if (err)
@@ -855,7 +852,7 @@
async_wrap_skb(skb, self->tx_buff.data,
self->tx_buff.truesize);
- self->stats.tx_bytes += self->tx_buff.len;
+ dev->stats.tx_bytes += self->tx_buff.len;
/* Send this frame with old speed */
SetBaudRate(iobase, self->io.speed);
SetPulseWidth(iobase, 12);
@@ -921,7 +918,7 @@
self->tx_fifo.queue[self->tx_fifo.free].len = skb->len;
self->tx_fifo.tail += skb->len;
- self->stats.tx_bytes += skb->len;
+ dev->stats.tx_bytes += skb->len;
skb_copy_from_linear_data(skb,
self->tx_fifo.queue[self->tx_fifo.free].start, skb->len);
self->tx_fifo.len++;
@@ -990,12 +987,12 @@
/* Clear bit, by writing 1 into it */
Tx_status = GetTXStatus(iobase);
if (Tx_status & 0x08) {
- self->stats.tx_errors++;
- self->stats.tx_fifo_errors++;
+ self->netdev->stats.tx_errors++;
+ self->netdev->stats.tx_fifo_errors++;
hwreset(self);
// how to clear underrrun ?
} else {
- self->stats.tx_packets++;
+ self->netdev->stats.tx_packets++;
ResetChip(iobase, 3);
ResetChip(iobase, 4);
}
@@ -1119,8 +1116,8 @@
}
// Move to next frame
self->rx_buff.data += len;
- self->stats.rx_bytes += len;
- self->stats.rx_packets++;
+ self->netdev->stats.rx_bytes += len;
+ self->netdev->stats.rx_packets++;
skb->dev = self->netdev;
skb_reset_mac_header(skb);
skb->protocol = htons(ETH_P_IRDA);
@@ -1180,7 +1177,7 @@
*/
if ((skb == NULL) || (skb->data == NULL)
|| (self->rx_buff.data == NULL) || (len < 6)) {
- self->stats.rx_dropped++;
+ self->netdev->stats.rx_dropped++;
return TRUE;
}
skb_reserve(skb, 1);
@@ -1192,8 +1189,8 @@
// Move to next frame
self->rx_buff.data += len;
- self->stats.rx_bytes += len;
- self->stats.rx_packets++;
+ self->netdev->stats.rx_bytes += len;
+ self->netdev->stats.rx_packets++;
skb->dev = self->netdev;
skb_reset_mac_header(skb);
skb->protocol = htons(ETH_P_IRDA);
@@ -1220,13 +1217,13 @@
IRDA_DEBUG(2, "%s(): len=%x\n", __func__, len);
if ((len - 4) < 2) {
- self->stats.rx_dropped++;
+ self->netdev->stats.rx_dropped++;
return FALSE;
}
skb = dev_alloc_skb(len + 1);
if (skb == NULL) {
- self->stats.rx_dropped++;
+ self->netdev->stats.rx_dropped++;
return FALSE;
}
skb_reserve(skb, 1);
@@ -1238,8 +1235,8 @@
st_fifo->tail = 0;
// Move to next frame
self->rx_buff.data += len;
- self->stats.rx_bytes += len;
- self->stats.rx_packets++;
+ self->netdev->stats.rx_bytes += len;
+ self->netdev->stats.rx_packets++;
skb->dev = self->netdev;
skb_reset_mac_header(skb);
skb->protocol = htons(ETH_P_IRDA);
@@ -1295,7 +1292,7 @@
*/
if ((skb == NULL) || (skb->data == NULL)
|| (self->rx_buff.data == NULL) || (len < 6)) {
- self->stats.rx_dropped++;
+ self->netdev->stats.rx_dropped++;
continue;
}
skb_reserve(skb, 1);
@@ -1307,8 +1304,8 @@
// Move to next frame
self->rx_buff.data += len;
- self->stats.rx_bytes += len;
- self->stats.rx_packets++;
+ self->netdev->stats.rx_bytes += len;
+ self->netdev->stats.rx_packets++;
skb->dev = self->netdev;
skb_reset_mac_header(skb);
skb->protocol = htons(ETH_P_IRDA);
@@ -1523,7 +1520,7 @@
IRDA_ASSERT(dev != NULL, return -1;);
self = netdev_priv(dev);
- self->stats.rx_packets = 0;
+ dev->stats.rx_packets = 0;
IRDA_ASSERT(self != NULL, return 0;);
iobase = self->io.fir_base;
if (request_irq(self->io.irq, via_ircc_interrupt, 0, dev->name, dev)) {
@@ -1660,14 +1657,6 @@
return ret;
}
-static struct net_device_stats *via_ircc_net_get_stats(struct net_device
- *dev)
-{
- struct via_ircc_cb *self = netdev_priv(dev);
-
- return &self->stats;
-}
-
MODULE_AUTHOR("VIA Technologies,inc");
MODULE_DESCRIPTION("VIA IrDA Device Driver");
MODULE_LICENSE("GPL");
diff --git a/drivers/net/irda/via-ircc.h b/drivers/net/irda/via-ircc.h
index 403c3f7..d9d1db0 100644
--- a/drivers/net/irda/via-ircc.h
+++ b/drivers/net/irda/via-ircc.h
@@ -95,7 +95,6 @@
struct tx_fifo tx_fifo; /* Info about frames to be transmitted */
struct net_device *netdev; /* Yes! we are some kind of netdevice */
- struct net_device_stats stats;
struct irlap_cb *irlap; /* The link layer we are binded to */
struct qos_info qos; /* QoS capabilities for this device */
diff --git a/drivers/net/irda/vlsi_ir.c b/drivers/net/irda/vlsi_ir.c
index 0d30f8d..723c458 100644
--- a/drivers/net/irda/vlsi_ir.c
+++ b/drivers/net/irda/vlsi_ir.c
@@ -291,14 +291,14 @@
now.tv_sec - idev->last_rx.tv_sec - delta1, delta2);
seq_printf(seq, "RX: packets=%lu / bytes=%lu / errors=%lu / dropped=%lu",
- idev->stats.rx_packets, idev->stats.rx_bytes, idev->stats.rx_errors,
- idev->stats.rx_dropped);
+ ndev->stats.rx_packets, ndev->stats.rx_bytes, ndev->stats.rx_errors,
+ ndev->stats.rx_dropped);
seq_printf(seq, " / overrun=%lu / length=%lu / frame=%lu / crc=%lu\n",
- idev->stats.rx_over_errors, idev->stats.rx_length_errors,
- idev->stats.rx_frame_errors, idev->stats.rx_crc_errors);
+ ndev->stats.rx_over_errors, ndev->stats.rx_length_errors,
+ ndev->stats.rx_frame_errors, ndev->stats.rx_crc_errors);
seq_printf(seq, "TX: packets=%lu / bytes=%lu / errors=%lu / dropped=%lu / fifo=%lu\n",
- idev->stats.tx_packets, idev->stats.tx_bytes, idev->stats.tx_errors,
- idev->stats.tx_dropped, idev->stats.tx_fifo_errors);
+ ndev->stats.tx_packets, ndev->stats.tx_bytes, ndev->stats.tx_errors,
+ ndev->stats.tx_dropped, ndev->stats.tx_fifo_errors);
}
@@ -651,21 +651,21 @@
if (ret < 0) {
ret = -ret;
- idev->stats.rx_errors++;
+ ndev->stats.rx_errors++;
if (ret & VLSI_RX_DROP)
- idev->stats.rx_dropped++;
+ ndev->stats.rx_dropped++;
if (ret & VLSI_RX_OVER)
- idev->stats.rx_over_errors++;
+ ndev->stats.rx_over_errors++;
if (ret & VLSI_RX_LENGTH)
- idev->stats.rx_length_errors++;
+ ndev->stats.rx_length_errors++;
if (ret & VLSI_RX_FRAME)
- idev->stats.rx_frame_errors++;
+ ndev->stats.rx_frame_errors++;
if (ret & VLSI_RX_CRC)
- idev->stats.rx_crc_errors++;
+ ndev->stats.rx_crc_errors++;
}
else if (ret > 0) {
- idev->stats.rx_packets++;
- idev->stats.rx_bytes += ret;
+ ndev->stats.rx_packets++;
+ ndev->stats.rx_bytes += ret;
}
}
@@ -686,6 +686,7 @@
static void vlsi_unarm_rx(vlsi_irda_dev_t *idev)
{
+ struct net_device *ndev = pci_get_drvdata(idev->pdev);
struct vlsi_ring *r = idev->rx_ring;
struct ring_descr *rd;
int ret;
@@ -711,21 +712,21 @@
if (ret < 0) {
ret = -ret;
- idev->stats.rx_errors++;
+ ndev->stats.rx_errors++;
if (ret & VLSI_RX_DROP)
- idev->stats.rx_dropped++;
+ ndev->stats.rx_dropped++;
if (ret & VLSI_RX_OVER)
- idev->stats.rx_over_errors++;
+ ndev->stats.rx_over_errors++;
if (ret & VLSI_RX_LENGTH)
- idev->stats.rx_length_errors++;
+ ndev->stats.rx_length_errors++;
if (ret & VLSI_RX_FRAME)
- idev->stats.rx_frame_errors++;
+ ndev->stats.rx_frame_errors++;
if (ret & VLSI_RX_CRC)
- idev->stats.rx_crc_errors++;
+ ndev->stats.rx_crc_errors++;
}
else if (ret > 0) {
- idev->stats.rx_packets++;
- idev->stats.rx_bytes += ret;
+ ndev->stats.rx_packets++;
+ ndev->stats.rx_bytes += ret;
}
}
}
@@ -1050,8 +1051,8 @@
drop:
IRDA_WARNING("%s: dropping packet - %s\n", __func__, msg);
dev_kfree_skb_any(skb);
- idev->stats.tx_errors++;
- idev->stats.tx_dropped++;
+ ndev->stats.tx_errors++;
+ ndev->stats.tx_dropped++;
/* Don't even think about returning NET_XMIT_DROP (=1) here!
* In fact any retval!=0 causes the packet scheduler to requeue the
* packet for later retry of transmission - which isn't exactly
@@ -1078,15 +1079,15 @@
if (ret < 0) {
ret = -ret;
- idev->stats.tx_errors++;
+ ndev->stats.tx_errors++;
if (ret & VLSI_TX_DROP)
- idev->stats.tx_dropped++;
+ ndev->stats.tx_dropped++;
if (ret & VLSI_TX_FIFO)
- idev->stats.tx_fifo_errors++;
+ ndev->stats.tx_fifo_errors++;
}
else if (ret > 0){
- idev->stats.tx_packets++;
- idev->stats.tx_bytes += ret;
+ ndev->stats.tx_packets++;
+ ndev->stats.tx_bytes += ret;
}
}
@@ -1122,6 +1123,7 @@
static void vlsi_unarm_tx(vlsi_irda_dev_t *idev)
{
+ struct net_device *ndev = pci_get_drvdata(idev->pdev);
struct vlsi_ring *r = idev->tx_ring;
struct ring_descr *rd;
int ret;
@@ -1145,15 +1147,15 @@
if (ret < 0) {
ret = -ret;
- idev->stats.tx_errors++;
+ ndev->stats.tx_errors++;
if (ret & VLSI_TX_DROP)
- idev->stats.tx_dropped++;
+ ndev->stats.tx_dropped++;
if (ret & VLSI_TX_FIFO)
- idev->stats.tx_fifo_errors++;
+ ndev->stats.tx_fifo_errors++;
}
else if (ret > 0){
- idev->stats.tx_packets++;
- idev->stats.tx_bytes += ret;
+ ndev->stats.tx_packets++;
+ ndev->stats.tx_bytes += ret;
}
}
@@ -1373,13 +1375,6 @@
/**************************************************************/
-static struct net_device_stats * vlsi_get_stats(struct net_device *ndev)
-{
- vlsi_irda_dev_t *idev = netdev_priv(ndev);
-
- return &idev->stats;
-}
-
static void vlsi_tx_timeout(struct net_device *ndev)
{
vlsi_irda_dev_t *idev = netdev_priv(ndev);
@@ -1615,7 +1610,6 @@
ndev->open = vlsi_open;
ndev->stop = vlsi_close;
- ndev->get_stats = vlsi_get_stats;
ndev->hard_start_xmit = vlsi_hard_start_xmit;
ndev->do_ioctl = vlsi_ioctl;
ndev->tx_timeout = vlsi_tx_timeout;
diff --git a/drivers/net/irda/vlsi_ir.h b/drivers/net/irda/vlsi_ir.h
index 9b18843..3050d1a 100644
--- a/drivers/net/irda/vlsi_ir.h
+++ b/drivers/net/irda/vlsi_ir.h
@@ -712,7 +712,6 @@
typedef struct vlsi_irda_dev {
struct pci_dev *pdev;
- struct net_device_stats stats;
struct irlap_cb *irlap;
diff --git a/drivers/net/irda/w83977af_ir.c b/drivers/net/irda/w83977af_ir.c
index 30ec913..dc0a2e4 100644
--- a/drivers/net/irda/w83977af_ir.c
+++ b/drivers/net/irda/w83977af_ir.c
@@ -102,7 +102,6 @@
static int w83977af_net_open(struct net_device *dev);
static int w83977af_net_close(struct net_device *dev);
static int w83977af_net_ioctl(struct net_device *dev, struct ifreq *rq, int cmd);
-static struct net_device_stats *w83977af_net_get_stats(struct net_device *dev);
/*
* Function w83977af_init ()
@@ -237,7 +236,6 @@
dev->open = w83977af_net_open;
dev->stop = w83977af_net_close;
dev->do_ioctl = w83977af_net_ioctl;
- dev->get_stats = w83977af_net_get_stats;
err = register_netdev(dev);
if (err) {
@@ -702,13 +700,13 @@
if (inb(iobase+AUDR) & AUDR_UNDR) {
IRDA_DEBUG(0, "%s(), Transmit underrun!\n", __func__ );
- self->stats.tx_errors++;
- self->stats.tx_fifo_errors++;
+ self->netdev->stats.tx_errors++;
+ self->netdev->stats.tx_fifo_errors++;
/* Clear bit, by writing 1 to it */
outb(AUDR_UNDR, iobase+AUDR);
} else
- self->stats.tx_packets++;
+ self->netdev->stats.tx_packets++;
if (self->new_speed) {
@@ -846,28 +844,28 @@
if (status & FS_FO_ERR_MSK) {
if (status & FS_FO_LST_FR) {
/* Add number of lost frames to stats */
- self->stats.rx_errors += len;
+ self->netdev->stats.rx_errors += len;
} else {
/* Skip frame */
- self->stats.rx_errors++;
+ self->netdev->stats.rx_errors++;
self->rx_buff.data += len;
if (status & FS_FO_MX_LEX)
- self->stats.rx_length_errors++;
+ self->netdev->stats.rx_length_errors++;
if (status & FS_FO_PHY_ERR)
- self->stats.rx_frame_errors++;
+ self->netdev->stats.rx_frame_errors++;
if (status & FS_FO_CRC_ERR)
- self->stats.rx_crc_errors++;
+ self->netdev->stats.rx_crc_errors++;
}
/* The errors below can be reported in both cases */
if (status & FS_FO_RX_OV)
- self->stats.rx_fifo_errors++;
+ self->netdev->stats.rx_fifo_errors++;
if (status & FS_FO_FSF_OV)
- self->stats.rx_fifo_errors++;
+ self->netdev->stats.rx_fifo_errors++;
} else {
/* Check if we have transferred all data to memory */
@@ -917,7 +915,7 @@
/* Move to next frame */
self->rx_buff.data += len;
- self->stats.rx_packets++;
+ self->netdev->stats.rx_packets++;
skb->dev = self->netdev;
skb_reset_mac_header(skb);
@@ -951,7 +949,7 @@
/* Receive all characters in Rx FIFO */
do {
byte = inb(iobase+RBR);
- async_unwrap_char(self->netdev, &self->stats, &self->rx_buff,
+ async_unwrap_char(self->netdev, &self->netdev->stats, &self->rx_buff,
byte);
} while (inb(iobase+USR) & USR_RDR); /* Data available */
}
@@ -994,7 +992,7 @@
outb(AUDR_SFEND, iobase+AUDR);
outb(set, iobase+SSR);
- self->stats.tx_packets++;
+ self->netdev->stats.tx_packets++;
/* Feed me more packets */
netif_wake_queue(self->netdev);
@@ -1336,13 +1334,6 @@
return ret;
}
-static struct net_device_stats *w83977af_net_get_stats(struct net_device *dev)
-{
- struct w83977af_ir *self = netdev_priv(dev);
-
- return &self->stats;
-}
-
MODULE_AUTHOR("Dag Brattli <dagb@cs.uit.no>");
MODULE_DESCRIPTION("Winbond W83977AF IrDA Device Driver");
MODULE_LICENSE("GPL");
diff --git a/drivers/net/irda/w83977af_ir.h b/drivers/net/irda/w83977af_ir.h
index 87c3975..fefe9b1 100644
--- a/drivers/net/irda/w83977af_ir.h
+++ b/drivers/net/irda/w83977af_ir.h
@@ -172,7 +172,6 @@
int tx_len; /* Number of frames in tx_buff */
struct net_device *netdev; /* Yes! we are some kind of netdevice */
- struct net_device_stats stats;
struct irlap_cb *irlap; /* The link layer we are binded to */
struct qos_info qos; /* QoS capabilities for this device */
diff --git a/drivers/net/mlx4/en_params.c b/drivers/net/mlx4/en_params.c
index cfeef0f..c1bd040 100644
--- a/drivers/net/mlx4/en_params.c
+++ b/drivers/net/mlx4/en_params.c
@@ -399,8 +399,10 @@
rx_size = roundup_pow_of_two(param->rx_pending);
rx_size = max_t(u32, rx_size, MLX4_EN_MIN_RX_SIZE);
+ rx_size = min_t(u32, rx_size, MLX4_EN_MAX_RX_SIZE);
tx_size = roundup_pow_of_two(param->tx_pending);
tx_size = max_t(u32, tx_size, MLX4_EN_MIN_TX_SIZE);
+ tx_size = min_t(u32, tx_size, MLX4_EN_MAX_TX_SIZE);
if (rx_size == priv->prof->rx_ring_size &&
tx_size == priv->prof->tx_ring_size)
@@ -440,8 +442,8 @@
struct mlx4_en_dev *mdev = priv->mdev;
memset(param, 0, sizeof(*param));
- param->rx_max_pending = mdev->dev->caps.max_rq_sg;
- param->tx_max_pending = mdev->dev->caps.max_sq_sg;
+ param->rx_max_pending = MLX4_EN_MAX_RX_SIZE;
+ param->tx_max_pending = MLX4_EN_MAX_TX_SIZE;
param->rx_pending = mdev->profile.prof[priv->port].rx_ring_size;
param->tx_pending = mdev->profile.prof[priv->port].tx_ring_size;
}
diff --git a/drivers/net/mlx4/en_tx.c b/drivers/net/mlx4/en_tx.c
index ff4d752..4afd599 100644
--- a/drivers/net/mlx4/en_tx.c
+++ b/drivers/net/mlx4/en_tx.c
@@ -203,19 +203,21 @@
/* Optimize the common case when there are no wraparounds */
if (likely((void *) tx_desc + tx_info->nr_txbb * TXBB_SIZE <= end)) {
- if (tx_info->linear) {
- pci_unmap_single(mdev->pdev,
- (dma_addr_t) be64_to_cpu(data->addr),
+ if (!tx_info->inl) {
+ if (tx_info->linear) {
+ pci_unmap_single(mdev->pdev,
+ (dma_addr_t) be64_to_cpu(data->addr),
be32_to_cpu(data->byte_count),
PCI_DMA_TODEVICE);
- ++data;
- }
+ ++data;
+ }
- for (i = 0; i < frags; i++) {
- frag = &skb_shinfo(skb)->frags[i];
- pci_unmap_page(mdev->pdev,
- (dma_addr_t) be64_to_cpu(data[i].addr),
- frag->size, PCI_DMA_TODEVICE);
+ for (i = 0; i < frags; i++) {
+ frag = &skb_shinfo(skb)->frags[i];
+ pci_unmap_page(mdev->pdev,
+ (dma_addr_t) be64_to_cpu(data[i].addr),
+ frag->size, PCI_DMA_TODEVICE);
+ }
}
/* Stamp the freed descriptor */
for (i = 0; i < tx_info->nr_txbb * TXBB_SIZE; i += STAMP_STRIDE) {
@@ -224,27 +226,29 @@
}
} else {
- if ((void *) data >= end) {
- data = (struct mlx4_wqe_data_seg *)
- (ring->buf + ((void *) data - end));
- }
+ if (!tx_info->inl) {
+ if ((void *) data >= end) {
+ data = (struct mlx4_wqe_data_seg *)
+ (ring->buf + ((void *) data - end));
+ }
- if (tx_info->linear) {
- pci_unmap_single(mdev->pdev,
- (dma_addr_t) be64_to_cpu(data->addr),
+ if (tx_info->linear) {
+ pci_unmap_single(mdev->pdev,
+ (dma_addr_t) be64_to_cpu(data->addr),
be32_to_cpu(data->byte_count),
PCI_DMA_TODEVICE);
- ++data;
- }
+ ++data;
+ }
- for (i = 0; i < frags; i++) {
- /* Check for wraparound before unmapping */
- if ((void *) data >= end)
- data = (struct mlx4_wqe_data_seg *) ring->buf;
- frag = &skb_shinfo(skb)->frags[i];
- pci_unmap_page(mdev->pdev,
+ for (i = 0; i < frags; i++) {
+ /* Check for wraparound before unmapping */
+ if ((void *) data >= end)
+ data = (struct mlx4_wqe_data_seg *) ring->buf;
+ frag = &skb_shinfo(skb)->frags[i];
+ pci_unmap_page(mdev->pdev,
(dma_addr_t) be64_to_cpu(data->addr),
frag->size, PCI_DMA_TODEVICE);
+ }
}
/* Stamp the freed descriptor */
for (i = 0; i < tx_info->nr_txbb * TXBB_SIZE; i += STAMP_STRIDE) {
@@ -790,8 +794,11 @@
wmb();
data->byte_count = cpu_to_be32(skb_headlen(skb) - lso_header_size);
}
- } else
+ tx_info->inl = 0;
+ } else {
build_inline_wqe(tx_desc, skb, real_size, &vlan_tag, tx_ind, fragptr);
+ tx_info->inl = 1;
+ }
ring->prod += nr_txbb;
diff --git a/drivers/net/mlx4/mlx4_en.h b/drivers/net/mlx4/mlx4_en.h
index 2e96c7b..e9af32d 100644
--- a/drivers/net/mlx4/mlx4_en.h
+++ b/drivers/net/mlx4/mlx4_en.h
@@ -115,6 +115,10 @@
};
#define MLX4_EN_MAX_RX_FRAGS 4
+/* Maximum ring sizes */
+#define MLX4_EN_MAX_TX_SIZE 8192
+#define MLX4_EN_MAX_RX_SIZE 8192
+
/* Minimum ring size for our page-allocation sceme to work */
#define MLX4_EN_MIN_RX_SIZE (MLX4_EN_ALLOC_SIZE / SMP_CACHE_BYTES)
#define MLX4_EN_MIN_TX_SIZE (4096 / TXBB_SIZE)
@@ -202,6 +206,7 @@
u32 nr_txbb;
u8 linear;
u8 data_offset;
+ u8 inl;
};
diff --git a/drivers/net/natsemi.c b/drivers/net/natsemi.c
index 478edb9..c5dec54 100644
--- a/drivers/net/natsemi.c
+++ b/drivers/net/natsemi.c
@@ -779,6 +779,22 @@
}
+static const struct net_device_ops natsemi_netdev_ops = {
+ .ndo_open = netdev_open,
+ .ndo_stop = netdev_close,
+ .ndo_start_xmit = start_tx,
+ .ndo_get_stats = get_stats,
+ .ndo_set_multicast_list = set_rx_mode,
+ .ndo_change_mtu = natsemi_change_mtu,
+ .ndo_do_ioctl = netdev_ioctl,
+ .ndo_tx_timeout = ns_tx_timeout,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+ .ndo_poll_controller = natsemi_poll_controller,
+#endif
+};
+
static int __devinit natsemi_probe1 (struct pci_dev *pdev,
const struct pci_device_id *ent)
{
@@ -911,20 +927,9 @@
if (find_cnt < MAX_UNITS && full_duplex[find_cnt])
np->full_duplex = 1;
- /* The chip-specific entries in the device structure. */
- dev->open = &netdev_open;
- dev->hard_start_xmit = &start_tx;
- dev->stop = &netdev_close;
- dev->get_stats = &get_stats;
- dev->set_multicast_list = &set_rx_mode;
- dev->change_mtu = &natsemi_change_mtu;
- dev->do_ioctl = &netdev_ioctl;
- dev->tx_timeout = &ns_tx_timeout;
+ dev->netdev_ops = &natsemi_netdev_ops;
dev->watchdog_timeo = TX_TIMEOUT;
-#ifdef CONFIG_NET_POLL_CONTROLLER
- dev->poll_controller = &natsemi_poll_controller;
-#endif
SET_ETHTOOL_OPS(dev, ðtool_ops);
if (mtu)
diff --git a/drivers/net/ns83820.c b/drivers/net/ns83820.c
index 46b0772..42021ac 100644
--- a/drivers/net/ns83820.c
+++ b/drivers/net/ns83820.c
@@ -1957,6 +1957,9 @@
.ndo_set_multicast_list = ns83820_set_multicast,
.ndo_validate_addr = eth_validate_addr,
.ndo_tx_timeout = ns83820_tx_timeout,
+#ifdef NS83820_VLAN_ACCEL_SUPPORT
+ .ndo_vlan_rx_register = ns83820_vlan_rx_register,
+#endif
};
static int __devinit ns83820_init_one(struct pci_dev *pci_dev,
@@ -2216,7 +2219,6 @@
#ifdef NS83820_VLAN_ACCEL_SUPPORT
/* We also support hardware vlan acceleration */
ndev->features |= NETIF_F_HW_VLAN_TX | NETIF_F_HW_VLAN_RX;
- ndev->vlan_rx_register = ns83820_vlan_rx_register;
#endif
if (using_dac) {
diff --git a/drivers/net/pcnet32.c b/drivers/net/pcnet32.c
index 044b7b0..665a428 100644
--- a/drivers/net/pcnet32.c
+++ b/drivers/net/pcnet32.c
@@ -1568,6 +1568,22 @@
return err;
}
+static const struct net_device_ops pcnet32_netdev_ops = {
+ .ndo_open = pcnet32_open,
+ .ndo_stop = pcnet32_close,
+ .ndo_start_xmit = pcnet32_start_xmit,
+ .ndo_tx_timeout = pcnet32_tx_timeout,
+ .ndo_get_stats = pcnet32_get_stats,
+ .ndo_set_multicast_list = pcnet32_set_multicast_list,
+ .ndo_do_ioctl = pcnet32_ioctl,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+ .ndo_poll_controller = pcnet32_poll_controller,
+#endif
+};
+
/* pcnet32_probe1
* Called from both pcnet32_probe_vlbus and pcnet_probe_pci.
* pdev will be NULL when called from pcnet32_probe_vlbus.
@@ -1934,20 +1950,10 @@
lp->watchdog_timer.function = (void *)&pcnet32_watchdog;
/* The PCNET32-specific entries in the device structure. */
- dev->open = &pcnet32_open;
- dev->hard_start_xmit = &pcnet32_start_xmit;
- dev->stop = &pcnet32_close;
- dev->get_stats = &pcnet32_get_stats;
- dev->set_multicast_list = &pcnet32_set_multicast_list;
- dev->do_ioctl = &pcnet32_ioctl;
+ dev->netdev_ops = &pcnet32_netdev_ops;
dev->ethtool_ops = &pcnet32_ethtool_ops;
- dev->tx_timeout = pcnet32_tx_timeout;
dev->watchdog_timeo = (5 * HZ);
-#ifdef CONFIG_NET_POLL_CONTROLLER
- dev->poll_controller = pcnet32_poll_controller;
-#endif
-
/* Fill in the generic fields of the device structure. */
if (register_netdev(dev))
goto err_free_ring;
@@ -2276,7 +2282,7 @@
if (lp->chip_version >= PCNET32_79C970A) {
/* Print the link status and start the watchdog */
pcnet32_check_media(dev, 1);
- mod_timer(&(lp->watchdog_timer), PCNET32_WATCHDOG_TIMEOUT);
+ mod_timer(&lp->watchdog_timer, PCNET32_WATCHDOG_TIMEOUT);
}
i = 0;
@@ -2911,7 +2917,7 @@
pcnet32_check_media(dev, 0);
spin_unlock_irqrestore(&lp->lock, flags);
- mod_timer(&(lp->watchdog_timer), PCNET32_WATCHDOG_TIMEOUT);
+ mod_timer(&lp->watchdog_timer, round_jiffies(PCNET32_WATCHDOG_TIMEOUT));
}
static int pcnet32_pm_suspend(struct pci_dev *pdev, pm_message_t state)
diff --git a/drivers/net/plip.c b/drivers/net/plip.c
index 0c46d60..0be0f0b 100644
--- a/drivers/net/plip.c
+++ b/drivers/net/plip.c
@@ -265,6 +265,13 @@
.cache = plip_hard_header_cache,
};
+static const struct net_device_ops plip_netdev_ops = {
+ .ndo_open = plip_open,
+ .ndo_stop = plip_close,
+ .ndo_start_xmit = plip_tx_packet,
+ .ndo_do_ioctl = plip_ioctl,
+};
+
/* Entry point of PLIP driver.
Probe the hardware, and register/initialize the driver.
@@ -280,15 +287,11 @@
struct net_local *nl = netdev_priv(dev);
/* Then, override parts of it */
- dev->hard_start_xmit = plip_tx_packet;
- dev->open = plip_open;
- dev->stop = plip_close;
- dev->do_ioctl = plip_ioctl;
-
dev->tx_queue_len = 10;
dev->flags = IFF_POINTOPOINT|IFF_NOARP;
memset(dev->dev_addr, 0xfc, ETH_ALEN);
+ dev->netdev_ops = &plip_netdev_ops;
dev->header_ops = &plip_header_ops;
diff --git a/drivers/net/r6040.c b/drivers/net/r6040.c
index 53bbddf..cf3a082 100644
--- a/drivers/net/r6040.c
+++ b/drivers/net/r6040.c
@@ -49,8 +49,8 @@
#include <asm/processor.h>
#define DRV_NAME "r6040"
-#define DRV_VERSION "0.19"
-#define DRV_RELDATE "18Dec2008"
+#define DRV_VERSION "0.20"
+#define DRV_RELDATE "07Jan2009"
/* PHY CHIP Address */
#define PHY1_ADDR 1 /* For MAC1 */
@@ -200,7 +200,7 @@
static char version[] __devinitdata = KERN_INFO DRV_NAME
": RDC R6040 NAPI net driver,"
- "version "DRV_VERSION " (" DRV_RELDATE ")\n";
+ "version "DRV_VERSION " (" DRV_RELDATE ")";
static int phy_table[] = { PHY1_ADDR, PHY2_ADDR };
@@ -330,7 +330,7 @@
do {
skb = netdev_alloc_skb(dev, MAX_BUF_SIZE);
if (!skb) {
- printk(KERN_ERR "%s: failed to alloc skb for rx\n", dev->name);
+ printk(KERN_ERR DRV_NAME "%s: failed to alloc skb for rx\n", dev->name);
rc = -ENOMEM;
goto err_exit;
}
@@ -1077,20 +1077,20 @@
/* this should always be supported */
err = pci_set_dma_mask(pdev, DMA_32BIT_MASK);
if (err) {
- printk(KERN_ERR DRV_NAME "32-bit PCI DMA addresses"
+ printk(KERN_ERR DRV_NAME ": 32-bit PCI DMA addresses"
"not supported by the card\n");
goto err_out;
}
err = pci_set_consistent_dma_mask(pdev, DMA_32BIT_MASK);
if (err) {
- printk(KERN_ERR DRV_NAME "32-bit PCI DMA addresses"
+ printk(KERN_ERR DRV_NAME ": 32-bit PCI DMA addresses"
"not supported by the card\n");
goto err_out;
}
/* IO Size check */
if (pci_resource_len(pdev, 0) < io_size) {
- printk(KERN_ERR DRV_NAME "Insufficient PCI resources, aborting\n");
+ printk(KERN_ERR DRV_NAME ": Insufficient PCI resources, aborting\n");
err = -EIO;
goto err_out;
}
@@ -1100,7 +1100,7 @@
dev = alloc_etherdev(sizeof(struct r6040_private));
if (!dev) {
- printk(KERN_ERR DRV_NAME "Failed to allocate etherdev\n");
+ printk(KERN_ERR DRV_NAME ": Failed to allocate etherdev\n");
err = -ENOMEM;
goto err_out;
}
@@ -1116,11 +1116,15 @@
ioaddr = pci_iomap(pdev, bar, io_size);
if (!ioaddr) {
- printk(KERN_ERR "ioremap failed for device %s\n",
+ printk(KERN_ERR DRV_NAME ": ioremap failed for device %s\n",
pci_name(pdev));
err = -EIO;
goto err_out_free_res;
}
+ /* If PHY status change register is still set to zero it means the
+ * bootloader didn't initialize it */
+ if (ioread16(ioaddr + PHY_CC) == 0)
+ iowrite16(0x9f07, ioaddr + PHY_CC);
/* Init system & device */
lp->base = ioaddr;
@@ -1137,6 +1141,11 @@
adrp[1] = ioread16(ioaddr + MID_0M);
adrp[2] = ioread16(ioaddr + MID_0H);
+ /* Some bootloader/BIOSes do not initialize
+ * MAC address, warn about that */
+ if (!(adrp[0] || adrp[1] || adrp[2]))
+ printk(KERN_WARNING DRV_NAME ": MAC address not initialized\n");
+
/* Link new device into r6040_root_dev */
lp->pdev = pdev;
lp->dev = dev;
diff --git a/drivers/net/sb1000.c b/drivers/net/sb1000.c
index be30253..fc0e38b 100644
--- a/drivers/net/sb1000.c
+++ b/drivers/net/sb1000.c
@@ -134,6 +134,16 @@
};
MODULE_DEVICE_TABLE(pnp, sb1000_pnp_ids);
+static const struct net_device_ops sb1000_netdev_ops = {
+ .ndo_open = sb1000_open,
+ .ndo_start_xmit = sb1000_start_xmit,
+ .ndo_do_ioctl = sb1000_dev_ioctl,
+ .ndo_stop = sb1000_close,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
static int
sb1000_probe_one(struct pnp_dev *pdev, const struct pnp_device_id *id)
{
@@ -192,11 +202,7 @@
if (sb1000_debug > 0)
printk(KERN_NOTICE "%s", version);
- /* The SB1000-specific entries in the device structure. */
- dev->open = sb1000_open;
- dev->do_ioctl = sb1000_dev_ioctl;
- dev->hard_start_xmit = sb1000_start_xmit;
- dev->stop = sb1000_close;
+ dev->netdev_ops = &sb1000_netdev_ops;
/* hardware address is 0:0:serial_number */
dev->dev_addr[2] = serial_number >> 24 & 0xff;
diff --git a/drivers/net/sis190.c b/drivers/net/sis190.c
index 83cc3c5..a973268 100644
--- a/drivers/net/sis190.c
+++ b/drivers/net/sis190.c
@@ -1782,6 +1782,21 @@
generic_mii_ioctl(&tp->mii_if, if_mii(ifr), cmd, NULL);
}
+static const struct net_device_ops sis190_netdev_ops = {
+ .ndo_open = sis190_open,
+ .ndo_stop = sis190_close,
+ .ndo_do_ioctl = sis190_ioctl,
+ .ndo_start_xmit = sis190_start_xmit,
+ .ndo_tx_timeout = sis190_tx_timeout,
+ .ndo_set_multicast_list = sis190_set_rx_mode,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+ .ndo_poll_controller = sis190_netpoll,
+#endif
+};
+
static int __devinit sis190_init_one(struct pci_dev *pdev,
const struct pci_device_id *ent)
{
@@ -1815,19 +1830,12 @@
INIT_WORK(&tp->phy_task, sis190_phy_task);
- dev->open = sis190_open;
- dev->stop = sis190_close;
- dev->do_ioctl = sis190_ioctl;
- dev->tx_timeout = sis190_tx_timeout;
- dev->watchdog_timeo = SIS190_TX_TIMEOUT;
- dev->hard_start_xmit = sis190_start_xmit;
-#ifdef CONFIG_NET_POLL_CONTROLLER
- dev->poll_controller = sis190_netpoll;
-#endif
- dev->set_multicast_list = sis190_set_rx_mode;
+ dev->netdev_ops = &sis190_netdev_ops;
+
SET_ETHTOOL_OPS(dev, &sis190_ethtool_ops);
dev->irq = pdev->irq;
dev->base_addr = (unsigned long) 0xdead;
+ dev->watchdog_timeo = SIS190_TX_TIMEOUT;
spin_lock_init(&tp->lock);
diff --git a/drivers/net/slip.c b/drivers/net/slip.c
index 8e1c0ba..5c61d5f 100644
--- a/drivers/net/slip.c
+++ b/drivers/net/slip.c
@@ -603,7 +603,6 @@
dev->mtu = sl->mtu;
dev->type = ARPHRD_SLIP + sl->mode;
#ifdef SL_CHECK_TRANSMIT
- dev->tx_timeout = sl_tx_timeout;
dev->watchdog_timeo = 20*HZ;
#endif
return 0;
@@ -617,19 +616,26 @@
sl_free_bufs(sl);
}
+static const struct net_device_ops sl_netdev_ops = {
+ .ndo_init = sl_init,
+ .ndo_uninit = sl_uninit,
+ .ndo_open = sl_open,
+ .ndo_stop = sl_close,
+ .ndo_start_xmit = sl_xmit,
+ .ndo_get_stats = sl_get_stats,
+ .ndo_change_mtu = sl_change_mtu,
+ .ndo_tx_timeout = sl_tx_timeout,
+#ifdef CONFIG_SLIP_SMART
+ .ndo_do_ioctl = sl_ioctl,
+#endif
+};
+
+
static void sl_setup(struct net_device *dev)
{
- dev->init = sl_init;
- dev->uninit = sl_uninit;
- dev->open = sl_open;
+ dev->netdev_ops = &sl_netdev_ops;
dev->destructor = free_netdev;
- dev->stop = sl_close;
- dev->get_stats = sl_get_stats;
- dev->change_mtu = sl_change_mtu;
- dev->hard_start_xmit = sl_xmit;
-#ifdef CONFIG_SLIP_SMART
- dev->do_ioctl = sl_ioctl;
-#endif
+
dev->hard_header_len = 0;
dev->addr_len = 0;
dev->tx_queue_len = 10;
diff --git a/drivers/net/starfire.c b/drivers/net/starfire.c
index 57fb1f7..da3a76b 100644
--- a/drivers/net/starfire.c
+++ b/drivers/net/starfire.c
@@ -648,6 +648,24 @@
#endif /* VLAN_SUPPORT */
+static const struct net_device_ops netdev_ops = {
+ .ndo_open = netdev_open,
+ .ndo_stop = netdev_close,
+ .ndo_start_xmit = start_tx,
+ .ndo_tx_timeout = tx_timeout,
+ .ndo_get_stats = get_stats,
+ .ndo_set_multicast_list = &set_rx_mode,
+ .ndo_do_ioctl = netdev_ioctl,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+#ifdef VLAN_SUPPORT
+ .ndo_vlan_rx_register = netdev_vlan_rx_register,
+ .ndo_vlan_rx_add_vid = netdev_vlan_rx_add_vid,
+ .ndo_vlan_rx_kill_vid = netdev_vlan_rx_kill_vid,
+#endif
+};
+
static int __devinit starfire_init_one(struct pci_dev *pdev,
const struct pci_device_id *ent)
{
@@ -710,11 +728,9 @@
if (enable_hw_cksum)
dev->features |= NETIF_F_IP_CSUM | NETIF_F_SG;
#endif /* ZEROCOPY */
+
#ifdef VLAN_SUPPORT
dev->features |= NETIF_F_HW_VLAN_RX | NETIF_F_HW_VLAN_FILTER;
- dev->vlan_rx_register = netdev_vlan_rx_register;
- dev->vlan_rx_add_vid = netdev_vlan_rx_add_vid;
- dev->vlan_rx_kill_vid = netdev_vlan_rx_kill_vid;
#endif /* VLAN_RX_KILL_VID */
#ifdef ADDR_64BITS
dev->features |= NETIF_F_HIGHDMA;
@@ -810,18 +826,12 @@
}
}
- /* The chip-specific entries in the device structure. */
- dev->open = &netdev_open;
- dev->hard_start_xmit = &start_tx;
- dev->tx_timeout = tx_timeout;
+ dev->netdev_ops = &netdev_ops;
dev->watchdog_timeo = TX_TIMEOUT;
- netif_napi_add(dev, &np->napi, netdev_poll, max_interrupt_work);
- dev->stop = &netdev_close;
- dev->get_stats = &get_stats;
- dev->set_multicast_list = &set_rx_mode;
- dev->do_ioctl = &netdev_ioctl;
SET_ETHTOOL_OPS(dev, ðtool_ops);
+ netif_napi_add(dev, &np->napi, netdev_poll, max_interrupt_work);
+
if (mtu)
dev->mtu = mtu;
diff --git a/drivers/net/sundance.c b/drivers/net/sundance.c
index 698893b..feaf0e0 100644
--- a/drivers/net/sundance.c
+++ b/drivers/net/sundance.c
@@ -449,6 +449,19 @@
}
}
+static const struct net_device_ops netdev_ops = {
+ .ndo_open = netdev_open,
+ .ndo_stop = netdev_close,
+ .ndo_start_xmit = start_tx,
+ .ndo_get_stats = get_stats,
+ .ndo_set_multicast_list = set_rx_mode,
+ .ndo_do_ioctl = netdev_ioctl,
+ .ndo_tx_timeout = tx_timeout,
+ .ndo_change_mtu = change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
static int __devinit sundance_probe1 (struct pci_dev *pdev,
const struct pci_device_id *ent)
{
@@ -530,16 +543,10 @@
np->mii_if.reg_num_mask = 0x1f;
/* The chip-specific entries in the device structure. */
- dev->open = &netdev_open;
- dev->hard_start_xmit = &start_tx;
- dev->stop = &netdev_close;
- dev->get_stats = &get_stats;
- dev->set_multicast_list = &set_rx_mode;
- dev->do_ioctl = &netdev_ioctl;
+ dev->netdev_ops = &netdev_ops;
SET_ETHTOOL_OPS(dev, ðtool_ops);
- dev->tx_timeout = &tx_timeout;
dev->watchdog_timeo = TX_TIMEOUT;
- dev->change_mtu = &change_mtu;
+
pci_set_drvdata(pdev, dev);
i = register_netdev(dev);
diff --git a/drivers/net/sungem.c b/drivers/net/sungem.c
index 8a74604..86c765d 100644
--- a/drivers/net/sungem.c
+++ b/drivers/net/sungem.c
@@ -2989,6 +2989,19 @@
}
}
+static const struct net_device_ops gem_netdev_ops = {
+ .ndo_open = gem_open,
+ .ndo_stop = gem_close,
+ .ndo_start_xmit = gem_start_xmit,
+ .ndo_get_stats = gem_get_stats,
+ .ndo_set_multicast_list = gem_set_multicast,
+ .ndo_do_ioctl = gem_ioctl,
+ .ndo_tx_timeout = gem_tx_timeout,
+ .ndo_change_mtu = gem_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
static int __devinit gem_init_one(struct pci_dev *pdev,
const struct pci_device_id *ent)
{
@@ -3142,17 +3155,10 @@
if (gem_get_device_address(gp))
goto err_out_free_consistent;
- dev->open = gem_open;
- dev->stop = gem_close;
- dev->hard_start_xmit = gem_start_xmit;
- dev->get_stats = gem_get_stats;
- dev->set_multicast_list = gem_set_multicast;
- dev->do_ioctl = gem_ioctl;
+ dev->netdev_ops = &gem_netdev_ops;
netif_napi_add(dev, &gp->napi, gem_poll, 64);
dev->ethtool_ops = &gem_ethtool_ops;
- dev->tx_timeout = gem_tx_timeout;
dev->watchdog_timeo = 5 * HZ;
- dev->change_mtu = gem_change_mtu;
dev->irq = pdev->irq;
dev->dma = 0;
dev->set_mac_address = gem_set_mac_address;
diff --git a/drivers/net/sunhme.c b/drivers/net/sunhme.c
index b22d335..7a72a31 100644
--- a/drivers/net/sunhme.c
+++ b/drivers/net/sunhme.c
@@ -2607,6 +2607,18 @@
}
#endif /* CONFIG_PCI */
+static const struct net_device_ops hme_netdev_ops = {
+ .ndo_open = happy_meal_open,
+ .ndo_stop = happy_meal_close,
+ .ndo_start_xmit = happy_meal_start_xmit,
+ .ndo_tx_timeout = happy_meal_tx_timeout,
+ .ndo_get_stats = happy_meal_get_stats,
+ .ndo_set_multicast_list = happy_meal_set_multicast,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
#ifdef CONFIG_SBUS
static int __devinit happy_meal_sbus_probe_one(struct of_device *op, int is_qfe)
{
@@ -2750,12 +2762,7 @@
init_timer(&hp->happy_timer);
hp->dev = dev;
- dev->open = &happy_meal_open;
- dev->stop = &happy_meal_close;
- dev->hard_start_xmit = &happy_meal_start_xmit;
- dev->get_stats = &happy_meal_get_stats;
- dev->set_multicast_list = &happy_meal_set_multicast;
- dev->tx_timeout = &happy_meal_tx_timeout;
+ dev->netdev_ops = &hme_netdev_ops;
dev->watchdog_timeo = 5*HZ;
dev->ethtool_ops = &hme_ethtool_ops;
@@ -3076,12 +3083,7 @@
init_timer(&hp->happy_timer);
hp->dev = dev;
- dev->open = &happy_meal_open;
- dev->stop = &happy_meal_close;
- dev->hard_start_xmit = &happy_meal_start_xmit;
- dev->get_stats = &happy_meal_get_stats;
- dev->set_multicast_list = &happy_meal_set_multicast;
- dev->tx_timeout = &happy_meal_tx_timeout;
+ dev->netdev_ops = &hme_netdev_ops;
dev->watchdog_timeo = 5*HZ;
dev->ethtool_ops = &hme_ethtool_ops;
dev->irq = pdev->irq;
diff --git a/drivers/net/tlan.c b/drivers/net/tlan.c
index 85ef8b7..68b967b 100644
--- a/drivers/net/tlan.c
+++ b/drivers/net/tlan.c
@@ -831,6 +831,21 @@
}
#endif
+static const struct net_device_ops TLan_netdev_ops = {
+ .ndo_open = TLan_Open,
+ .ndo_stop = TLan_Close,
+ .ndo_start_xmit = TLan_StartTx,
+ .ndo_tx_timeout = TLan_tx_timeout,
+ .ndo_get_stats = TLan_GetStats,
+ .ndo_set_multicast_list = TLan_SetMulticastList,
+ .ndo_do_ioctl = TLan_ioctl,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+ .ndo_poll_controller = TLan_Poll,
+#endif
+};
@@ -892,16 +907,7 @@
netif_carrier_off(dev);
/* Device methods */
- dev->open = &TLan_Open;
- dev->hard_start_xmit = &TLan_StartTx;
- dev->stop = &TLan_Close;
- dev->get_stats = &TLan_GetStats;
- dev->set_multicast_list = &TLan_SetMulticastList;
- dev->do_ioctl = &TLan_ioctl;
-#ifdef CONFIG_NET_POLL_CONTROLLER
- dev->poll_controller = &TLan_Poll;
-#endif
- dev->tx_timeout = &TLan_tx_timeout;
+ dev->netdev_ops = &TLan_netdev_ops;
dev->watchdog_timeo = TX_TIMEOUT;
return 0;
diff --git a/drivers/net/tulip/de2104x.c b/drivers/net/tulip/de2104x.c
index 5166be9..d5d53b6 100644
--- a/drivers/net/tulip/de2104x.c
+++ b/drivers/net/tulip/de2104x.c
@@ -1922,6 +1922,18 @@
goto fill_defaults;
}
+static const struct net_device_ops de_netdev_ops = {
+ .ndo_open = de_open,
+ .ndo_stop = de_close,
+ .ndo_set_multicast_list = de_set_rx_mode,
+ .ndo_start_xmit = de_start_xmit,
+ .ndo_get_stats = de_get_stats,
+ .ndo_tx_timeout = de_tx_timeout,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
static int __devinit de_init_one (struct pci_dev *pdev,
const struct pci_device_id *ent)
{
@@ -1944,14 +1956,9 @@
if (!dev)
return -ENOMEM;
+ dev->netdev_ops = &de_netdev_ops;
SET_NETDEV_DEV(dev, &pdev->dev);
- dev->open = de_open;
- dev->stop = de_close;
- dev->set_multicast_list = de_set_rx_mode;
- dev->hard_start_xmit = de_start_xmit;
- dev->get_stats = de_get_stats;
dev->ethtool_ops = &de_ethtool_ops;
- dev->tx_timeout = de_tx_timeout;
dev->watchdog_timeo = TX_TIMEOUT;
de = netdev_priv(dev);
diff --git a/drivers/net/tulip/de4x5.c b/drivers/net/tulip/de4x5.c
index 67bfd6f..6418f74 100644
--- a/drivers/net/tulip/de4x5.c
+++ b/drivers/net/tulip/de4x5.c
@@ -1077,6 +1077,18 @@
mdelay(2); /* Wait for 2ms */\
}
+static const struct net_device_ops de4x5_netdev_ops = {
+ .ndo_open = de4x5_open,
+ .ndo_stop = de4x5_close,
+ .ndo_start_xmit = de4x5_queue_pkt,
+ .ndo_get_stats = de4x5_get_stats,
+ .ndo_set_multicast_list = set_multicast_list,
+ .ndo_do_ioctl = de4x5_ioctl,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address= eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
static int __devinit
de4x5_hw_init(struct net_device *dev, u_long iobase, struct device *gendev)
@@ -1258,13 +1270,7 @@
/* The DE4X5-specific entries in the device structure. */
SET_NETDEV_DEV(dev, gendev);
- dev->open = &de4x5_open;
- dev->hard_start_xmit = &de4x5_queue_pkt;
- dev->stop = &de4x5_close;
- dev->get_stats = &de4x5_get_stats;
- dev->set_multicast_list = &set_multicast_list;
- dev->do_ioctl = &de4x5_ioctl;
-
+ dev->netdev_ops = &de4x5_netdev_ops;
dev->mem_start = 0;
/* Fill in the generic fields of the device structure. */
diff --git a/drivers/net/tulip/dmfe.c b/drivers/net/tulip/dmfe.c
index 28a5c51..2e5c999 100644
--- a/drivers/net/tulip/dmfe.c
+++ b/drivers/net/tulip/dmfe.c
@@ -257,9 +257,6 @@
u8 wol_mode; /* user WOL settings */
struct timer_list timer;
- /* System defined statistic counter */
- struct net_device_stats stats;
-
/* Driver defined statistic counter */
unsigned long tx_fifo_underrun;
unsigned long tx_loss_carrier;
@@ -316,7 +313,6 @@
static int dmfe_open(struct DEVICE *);
static int dmfe_start_xmit(struct sk_buff *, struct DEVICE *);
static int dmfe_stop(struct DEVICE *);
-static struct net_device_stats * dmfe_get_stats(struct DEVICE *);
static void dmfe_set_filter_mode(struct DEVICE *);
static const struct ethtool_ops netdev_ethtool_ops;
static u16 read_srom_word(long ,int);
@@ -351,6 +347,19 @@
/* DM910X network board routine ---------------------------- */
+static const struct net_device_ops netdev_ops = {
+ .ndo_open = dmfe_open,
+ .ndo_stop = dmfe_stop,
+ .ndo_start_xmit = dmfe_start_xmit,
+ .ndo_set_multicast_list = dmfe_set_filter_mode,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+ .ndo_poll_controller = poll_dmfe,
+#endif
+};
+
/*
* Search DM910X board ,allocate space and register it
*/
@@ -442,14 +451,7 @@
dev->base_addr = db->ioaddr;
dev->irq = pdev->irq;
pci_set_drvdata(pdev, dev);
- dev->open = &dmfe_open;
- dev->hard_start_xmit = &dmfe_start_xmit;
- dev->stop = &dmfe_stop;
- dev->get_stats = &dmfe_get_stats;
- dev->set_multicast_list = &dmfe_set_filter_mode;
-#ifdef CONFIG_NET_POLL_CONTROLLER
- dev->poll_controller = &poll_dmfe;
-#endif
+ dev->netdev_ops = &netdev_ops;
dev->ethtool_ops = &netdev_ethtool_ops;
netif_carrier_off(dev);
spin_lock_init(&db->lock);
@@ -867,15 +869,15 @@
/* A packet sent completed */
db->tx_packet_cnt--;
- db->stats.tx_packets++;
+ dev->stats.tx_packets++;
/* Transmit statistic counter */
if ( tdes0 != 0x7fffffff ) {
/* printk(DRV_NAME ": tdes0=%x\n", tdes0); */
- db->stats.collisions += (tdes0 >> 3) & 0xf;
- db->stats.tx_bytes += le32_to_cpu(txptr->tdes1) & 0x7ff;
+ dev->stats.collisions += (tdes0 >> 3) & 0xf;
+ dev->stats.tx_bytes += le32_to_cpu(txptr->tdes1) & 0x7ff;
if (tdes0 & TDES0_ERR_MASK) {
- db->stats.tx_errors++;
+ dev->stats.tx_errors++;
if (tdes0 & 0x0002) { /* UnderRun */
db->tx_fifo_underrun++;
@@ -969,13 +971,13 @@
if (rdes0 & 0x8000) {
/* This is a error packet */
//printk(DRV_NAME ": rdes0: %lx\n", rdes0);
- db->stats.rx_errors++;
+ dev->stats.rx_errors++;
if (rdes0 & 1)
- db->stats.rx_fifo_errors++;
+ dev->stats.rx_fifo_errors++;
if (rdes0 & 2)
- db->stats.rx_crc_errors++;
+ dev->stats.rx_crc_errors++;
if (rdes0 & 0x80)
- db->stats.rx_length_errors++;
+ dev->stats.rx_length_errors++;
}
if ( !(rdes0 & 0x8000) ||
@@ -1008,8 +1010,8 @@
skb->protocol = eth_type_trans(skb, dev);
netif_rx(skb);
- db->stats.rx_packets++;
- db->stats.rx_bytes += rxlen;
+ dev->stats.rx_packets++;
+ dev->stats.rx_bytes += rxlen;
}
} else {
/* Reuse SKB buffer when the packet is error */
@@ -1024,20 +1026,6 @@
db->rx_ready_ptr = rxptr;
}
-
-/*
- * Get statistics from driver.
- */
-
-static struct net_device_stats * dmfe_get_stats(struct DEVICE *dev)
-{
- struct dmfe_board_info *db = netdev_priv(dev);
-
- DMFE_DBUG(0, "dmfe_get_stats", 0);
- return &db->stats;
-}
-
-
/*
* Set DM910X multicast address
*/
@@ -1161,7 +1149,7 @@
/* Operating Mode Check */
if ( (db->dm910x_chk_mode & 0x1) &&
- (db->stats.rx_packets > MAX_CHECK_PACKET) )
+ (dev->stats.rx_packets > MAX_CHECK_PACKET) )
db->dm910x_chk_mode = 0x4;
/* Dynamic reset DM910X : system error or transmit time-out */
diff --git a/drivers/net/tulip/tulip_core.c b/drivers/net/tulip/tulip_core.c
index ff84bab..bee75fa 100644
--- a/drivers/net/tulip/tulip_core.c
+++ b/drivers/net/tulip/tulip_core.c
@@ -1225,6 +1225,22 @@
return 0;
}
+static const struct net_device_ops tulip_netdev_ops = {
+ .ndo_open = tulip_open,
+ .ndo_start_xmit = tulip_start_xmit,
+ .ndo_tx_timeout = tulip_tx_timeout,
+ .ndo_stop = tulip_close,
+ .ndo_get_stats = tulip_get_stats,
+ .ndo_do_ioctl = private_ioctl,
+ .ndo_set_multicast_list = set_rx_mode,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+ .ndo_poll_controller = poll_tulip,
+#endif
+};
+
static int __devinit tulip_init_one (struct pci_dev *pdev,
const struct pci_device_id *ent)
{
@@ -1601,20 +1617,11 @@
}
/* The Tulip-specific entries in the device structure. */
- dev->open = tulip_open;
- dev->hard_start_xmit = tulip_start_xmit;
- dev->tx_timeout = tulip_tx_timeout;
+ dev->netdev_ops = &tulip_netdev_ops;
dev->watchdog_timeo = TX_TIMEOUT;
#ifdef CONFIG_TULIP_NAPI
netif_napi_add(dev, &tp->napi, tulip_poll, 16);
#endif
- dev->stop = tulip_close;
- dev->get_stats = tulip_get_stats;
- dev->do_ioctl = private_ioctl;
- dev->set_multicast_list = set_rx_mode;
-#ifdef CONFIG_NET_POLL_CONTROLLER
- dev->poll_controller = &poll_tulip;
-#endif
SET_ETHTOOL_OPS(dev, &ops);
if (register_netdev(dev))
diff --git a/drivers/net/tulip/uli526x.c b/drivers/net/tulip/uli526x.c
index 00cbc52..030e02e 100644
--- a/drivers/net/tulip/uli526x.c
+++ b/drivers/net/tulip/uli526x.c
@@ -168,9 +168,6 @@
u8 wait_reset; /* Hardware failed, need to reset */
struct timer_list timer;
- /* System defined statistic counter */
- struct net_device_stats stats;
-
/* Driver defined statistic counter */
unsigned long tx_fifo_underrun;
unsigned long tx_loss_carrier;
@@ -220,7 +217,6 @@
static int uli526x_open(struct net_device *);
static int uli526x_start_xmit(struct sk_buff *, struct net_device *);
static int uli526x_stop(struct net_device *);
-static struct net_device_stats * uli526x_get_stats(struct net_device *);
static void uli526x_set_filter_mode(struct net_device *);
static const struct ethtool_ops netdev_ethtool_ops;
static u16 read_srom_word(long, int);
@@ -251,6 +247,19 @@
/* ULI526X network board routine ---------------------------- */
+static const struct net_device_ops netdev_ops = {
+ .ndo_open = uli526x_open,
+ .ndo_stop = uli526x_stop,
+ .ndo_start_xmit = uli526x_start_xmit,
+ .ndo_set_multicast_list = uli526x_set_filter_mode,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+ .ndo_poll_controller = uli526x_poll,
+#endif
+};
+
/*
* Search ULI526X board, allocate space and register it
*/
@@ -335,15 +344,9 @@
pci_set_drvdata(pdev, dev);
/* Register some necessary functions */
- dev->open = &uli526x_open;
- dev->hard_start_xmit = &uli526x_start_xmit;
- dev->stop = &uli526x_stop;
- dev->get_stats = &uli526x_get_stats;
- dev->set_multicast_list = &uli526x_set_filter_mode;
+ dev->netdev_ops = &netdev_ops;
dev->ethtool_ops = &netdev_ethtool_ops;
-#ifdef CONFIG_NET_POLL_CONTROLLER
- dev->poll_controller = &uli526x_poll;
-#endif
+
spin_lock_init(&db->lock);
@@ -733,7 +736,8 @@
* Free TX resource after TX complete
*/
-static void uli526x_free_tx_pkt(struct net_device *dev, struct uli526x_board_info * db)
+static void uli526x_free_tx_pkt(struct net_device *dev,
+ struct uli526x_board_info * db)
{
struct tx_desc *txptr;
u32 tdes0;
@@ -747,15 +751,15 @@
/* A packet sent completed */
db->tx_packet_cnt--;
- db->stats.tx_packets++;
+ dev->stats.tx_packets++;
/* Transmit statistic counter */
if ( tdes0 != 0x7fffffff ) {
/* printk(DRV_NAME ": tdes0=%x\n", tdes0); */
- db->stats.collisions += (tdes0 >> 3) & 0xf;
- db->stats.tx_bytes += le32_to_cpu(txptr->tdes1) & 0x7ff;
+ dev->stats.collisions += (tdes0 >> 3) & 0xf;
+ dev->stats.tx_bytes += le32_to_cpu(txptr->tdes1) & 0x7ff;
if (tdes0 & TDES0_ERR_MASK) {
- db->stats.tx_errors++;
+ dev->stats.tx_errors++;
if (tdes0 & 0x0002) { /* UnderRun */
db->tx_fifo_underrun++;
if ( !(db->cr6_data & CR6_SFT) ) {
@@ -825,13 +829,13 @@
if (rdes0 & 0x8000) {
/* This is a error packet */
//printk(DRV_NAME ": rdes0: %lx\n", rdes0);
- db->stats.rx_errors++;
+ dev->stats.rx_errors++;
if (rdes0 & 1)
- db->stats.rx_fifo_errors++;
+ dev->stats.rx_fifo_errors++;
if (rdes0 & 2)
- db->stats.rx_crc_errors++;
+ dev->stats.rx_crc_errors++;
if (rdes0 & 0x80)
- db->stats.rx_length_errors++;
+ dev->stats.rx_length_errors++;
}
if ( !(rdes0 & 0x8000) ||
@@ -854,8 +858,8 @@
skb->protocol = eth_type_trans(skb, dev);
netif_rx(skb);
- db->stats.rx_packets++;
- db->stats.rx_bytes += rxlen;
+ dev->stats.rx_packets++;
+ dev->stats.rx_bytes += rxlen;
} else {
/* Reuse SKB buffer when the packet is error */
@@ -872,19 +876,6 @@
/*
- * Get statistics from driver.
- */
-
-static struct net_device_stats * uli526x_get_stats(struct net_device *dev)
-{
- struct uli526x_board_info *db = netdev_priv(dev);
-
- ULI526X_DBUG(0, "uli526x_get_stats", 0);
- return &db->stats;
-}
-
-
-/*
* Set ULI526X multicast address
*/
diff --git a/drivers/net/tulip/winbond-840.c b/drivers/net/tulip/winbond-840.c
index 022d99a..f467bf8 100644
--- a/drivers/net/tulip/winbond-840.c
+++ b/drivers/net/tulip/winbond-840.c
@@ -343,7 +343,18 @@
static const struct ethtool_ops netdev_ethtool_ops;
static int netdev_close(struct net_device *dev);
-
+static const struct net_device_ops netdev_ops = {
+ .ndo_open = netdev_open,
+ .ndo_stop = netdev_close,
+ .ndo_start_xmit = start_tx,
+ .ndo_get_stats = get_stats,
+ .ndo_set_multicast_list = set_rx_mode,
+ .ndo_do_ioctl = netdev_ioctl,
+ .ndo_tx_timeout = tx_timeout,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
static int __devinit w840_probe1 (struct pci_dev *pdev,
const struct pci_device_id *ent)
@@ -420,14 +431,8 @@
np->mii_if.force_media = 1;
/* The chip-specific entries in the device structure. */
- dev->open = &netdev_open;
- dev->hard_start_xmit = &start_tx;
- dev->stop = &netdev_close;
- dev->get_stats = &get_stats;
- dev->set_multicast_list = &set_rx_mode;
- dev->do_ioctl = &netdev_ioctl;
+ dev->netdev_ops = &netdev_ops;
dev->ethtool_ops = &netdev_ethtool_ops;
- dev->tx_timeout = &tx_timeout;
dev->watchdog_timeo = TX_TIMEOUT;
i = register_netdev(dev);
@@ -1555,7 +1560,7 @@
* rtnl_lock, & netif_device_detach after the rtnl_unlock.
* - get_stats:
* spin_lock_irq(np->lock), doesn't touch hw if not present
- * - hard_start_xmit:
+ * - start_xmit:
* synchronize_irq + netif_tx_disable;
* - tx_timeout:
* netif_device_detach + netif_tx_disable;
diff --git a/drivers/net/tulip/xircom_cb.c b/drivers/net/tulip/xircom_cb.c
index 13c8703..c2ca9f4 100644
--- a/drivers/net/tulip/xircom_cb.c
+++ b/drivers/net/tulip/xircom_cb.c
@@ -104,10 +104,8 @@
*/
spinlock_t lock;
-
struct pci_dev *pdev;
struct net_device *dev;
- struct net_device_stats stats;
};
@@ -119,7 +117,6 @@
static int xircom_open(struct net_device *dev);
static int xircom_close(struct net_device *dev);
static void xircom_up(struct xircom_private *card);
-static struct net_device_stats *xircom_get_stats(struct net_device *dev);
#ifdef CONFIG_NET_POLL_CONTROLLER
static void xircom_poll_controller(struct net_device *dev);
#endif
@@ -194,6 +191,18 @@
.get_drvinfo = netdev_get_drvinfo,
};
+static const struct net_device_ops netdev_ops = {
+ .ndo_open = xircom_open,
+ .ndo_stop = xircom_close,
+ .ndo_start_xmit = xircom_start_xmit,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+ .ndo_poll_controller = xircom_poll_controller,
+#endif
+};
+
/* xircom_probe is the code that gets called on device insertion.
it sets up the hardware and registers the device to the networklayer.
@@ -266,13 +275,7 @@
read_mac_address(private);
setup_descriptors(private);
- dev->open = &xircom_open;
- dev->hard_start_xmit = &xircom_start_xmit;
- dev->stop = &xircom_close;
- dev->get_stats = &xircom_get_stats;
-#ifdef CONFIG_NET_POLL_CONTROLLER
- dev->poll_controller = &xircom_poll_controller;
-#endif
+ dev->netdev_ops = &netdev_ops;
SET_ETHTOOL_OPS(dev, &netdev_ethtool_ops);
pci_set_drvdata(pdev, dev);
@@ -497,14 +500,6 @@
}
-
-static struct net_device_stats *xircom_get_stats(struct net_device *dev)
-{
- struct xircom_private *card = netdev_priv(dev);
- return &card->stats;
-}
-
-
#ifdef CONFIG_NET_POLL_CONTROLLER
static void xircom_poll_controller(struct net_device *dev)
{
@@ -1193,7 +1188,7 @@
skb = dev_alloc_skb(pkt_len + 2);
if (skb == NULL) {
- card->stats.rx_dropped++;
+ dev->stats.rx_dropped++;
goto out;
}
skb_reserve(skb, 2);
@@ -1201,8 +1196,8 @@
skb_put(skb, pkt_len);
skb->protocol = eth_type_trans(skb, dev);
netif_rx(skb);
- card->stats.rx_packets++;
- card->stats.rx_bytes += pkt_len;
+ dev->stats.rx_packets++;
+ dev->stats.rx_bytes += pkt_len;
out:
/* give the buffer back to the card */
@@ -1232,16 +1227,16 @@
#endif
if (status > 0) { /* bit 31 is 0 when done */
if (card->tx_skb[descnr]!=NULL) {
- card->stats.tx_bytes += card->tx_skb[descnr]->len;
+ dev->stats.tx_bytes += card->tx_skb[descnr]->len;
dev_kfree_skb_irq(card->tx_skb[descnr]);
}
card->tx_skb[descnr] = NULL;
/* Bit 8 in the status field is 1 if there was a collision */
if (status&(1<<8))
- card->stats.collisions++;
+ dev->stats.collisions++;
card->tx_buffer[4*descnr] = 0; /* descriptor is free again */
netif_wake_queue (dev);
- card->stats.tx_packets++;
+ dev->stats.tx_packets++;
}
leave("investigate_write_descriptor");
diff --git a/drivers/net/typhoon.c b/drivers/net/typhoon.c
index 0009f4e..3af9a95 100644
--- a/drivers/net/typhoon.c
+++ b/drivers/net/typhoon.c
@@ -2296,6 +2296,19 @@
return mode;
}
+static const struct net_device_ops typhoon_netdev_ops = {
+ .ndo_open = typhoon_open,
+ .ndo_stop = typhoon_close,
+ .ndo_start_xmit = typhoon_start_tx,
+ .ndo_set_multicast_list = typhoon_set_rx_mode,
+ .ndo_tx_timeout = typhoon_tx_timeout,
+ .ndo_get_stats = typhoon_get_stats,
+ .ndo_validate_addr = eth_validate_addr,
+ .ndo_set_mac_address = typhoon_set_mac_address,
+ .ndo_change_mtu = eth_change_mtu,
+ .ndo_vlan_rx_register = typhoon_vlan_rx_register,
+};
+
static int __devinit
typhoon_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
{
@@ -2495,16 +2508,9 @@
}
/* The chip-specific entries in the device structure. */
- dev->open = typhoon_open;
- dev->hard_start_xmit = typhoon_start_tx;
- dev->stop = typhoon_close;
- dev->set_multicast_list = typhoon_set_rx_mode;
- dev->tx_timeout = typhoon_tx_timeout;
+ dev->netdev_ops = &typhoon_netdev_ops;
netif_napi_add(dev, &tp->napi, typhoon_poll, 16);
dev->watchdog_timeo = TX_TIMEOUT;
- dev->get_stats = typhoon_get_stats;
- dev->set_mac_address = typhoon_set_mac_address;
- dev->vlan_rx_register = typhoon_vlan_rx_register;
SET_ETHTOOL_OPS(dev, &typhoon_ethtool_ops);
diff --git a/drivers/net/usb/dm9601.c b/drivers/net/usb/dm9601.c
index edd244f..5b67bbf 100644
--- a/drivers/net/usb/dm9601.c
+++ b/drivers/net/usb/dm9601.c
@@ -23,7 +23,7 @@
#include <linux/usb/usbnet.h>
/* datasheet:
- http://www.davicom.com.tw/big5/download/Data%20Sheet/DM9601-DS-P01-930914.pdf
+ http://ptm2.cc.utu.fi/ftp/network/cards/DM9601/From_NET/DM9601-DS-P01-930914.pdf
*/
/* control requests */
@@ -397,16 +397,24 @@
dm_write_reg_async(dev, DM_RX_CTRL, rx_ctl);
}
+static void __dm9601_set_mac_address(struct usbnet *dev)
+{
+ dm_write_async(dev, DM_PHY_ADDR, ETH_ALEN, dev->net->dev_addr);
+}
+
static int dm9601_set_mac_address(struct net_device *net, void *p)
{
struct sockaddr *addr = p;
struct usbnet *dev = netdev_priv(net);
- if (!is_valid_ether_addr(addr->sa_data))
+ if (!is_valid_ether_addr(addr->sa_data)) {
+ dev_err(&net->dev, "not setting invalid mac address %pM\n",
+ addr->sa_data);
return -EINVAL;
+ }
memcpy(net->dev_addr, addr->sa_data, net->addr_len);
- dm_write_async(dev, DM_PHY_ADDR, net->addr_len, net->dev_addr);
+ __dm9601_set_mac_address(dev);
return 0;
}
@@ -414,6 +422,7 @@
static int dm9601_bind(struct usbnet *dev, struct usb_interface *intf)
{
int ret;
+ u8 mac[ETH_ALEN];
ret = usbnet_get_endpoints(dev, intf);
if (ret)
@@ -438,12 +447,24 @@
udelay(20);
/* read MAC */
- if (dm_read(dev, DM_PHY_ADDR, ETH_ALEN, dev->net->dev_addr) < 0) {
+ if (dm_read(dev, DM_PHY_ADDR, ETH_ALEN, mac) < 0) {
printk(KERN_ERR "Error reading MAC address\n");
ret = -ENODEV;
goto out;
}
+ /*
+ * Overwrite the auto-generated address only with good ones.
+ */
+ if (is_valid_ether_addr(mac))
+ memcpy(dev->net->dev_addr, mac, ETH_ALEN);
+ else {
+ printk(KERN_WARNING
+ "dm9601: No valid MAC address in EEPROM, using %pM\n",
+ dev->net->dev_addr);
+ __dm9601_set_mac_address(dev);
+ }
+
/* power up phy */
dm_write_reg(dev, DM_GPR_CTRL, 1);
dm_write_reg(dev, DM_GPR_DATA, 0);
diff --git a/drivers/net/usb/kaweth.c b/drivers/net/usb/kaweth.c
index 3073ca2..7cb10a0 100644
--- a/drivers/net/usb/kaweth.c
+++ b/drivers/net/usb/kaweth.c
@@ -251,7 +251,6 @@
struct net_device_stats stats;
};
-
/****************************************************************
* kaweth_control
****************************************************************/
@@ -975,6 +974,17 @@
/****************************************************************
* kaweth_probe
****************************************************************/
+
+
+static const struct net_device_ops kaweth_netdev_ops = {
+ .ndo_open = kaweth_open,
+ .ndo_stop = kaweth_close,
+ .ndo_start_xmit = kaweth_start_xmit,
+ .ndo_tx_timeout = kaweth_tx_timeout,
+ .ndo_set_multicast_list = kaweth_set_rx_mode,
+ .ndo_get_stats = kaweth_netdev_stats,
+};
+
static int kaweth_probe(
struct usb_interface *intf,
const struct usb_device_id *id /* from id_table */
@@ -1147,22 +1157,13 @@
memcpy(netdev->dev_addr, &kaweth->configuration.hw_addr,
sizeof(kaweth->configuration.hw_addr));
- netdev->open = kaweth_open;
- netdev->stop = kaweth_close;
-
+ netdev->netdev_ops = &kaweth_netdev_ops;
netdev->watchdog_timeo = KAWETH_TX_TIMEOUT;
- netdev->tx_timeout = kaweth_tx_timeout;
-
- netdev->hard_start_xmit = kaweth_start_xmit;
- netdev->set_multicast_list = kaweth_set_rx_mode;
- netdev->get_stats = kaweth_netdev_stats;
netdev->mtu = le16_to_cpu(kaweth->configuration.segment_size);
SET_ETHTOOL_OPS(netdev, &ops);
/* kaweth is zeroed as part of alloc_netdev */
-
INIT_DELAYED_WORK(&kaweth->lowmem_work, kaweth_resubmit_tl);
-
usb_set_intfdata(intf, kaweth);
#if 0
diff --git a/drivers/net/usb/pegasus.c b/drivers/net/usb/pegasus.c
index d9241f1..a8228d8 100644
--- a/drivers/net/usb/pegasus.c
+++ b/drivers/net/usb/pegasus.c
@@ -93,6 +93,7 @@
MODULE_PARM_DESC (msg_level, "Override default message level");
MODULE_DEVICE_TABLE(usb, pegasus_ids);
+static const struct net_device_ops pegasus_netdev_ops;
static int update_eth_regs_async(pegasus_t *);
/* Aargh!!! I _really_ hate such tweaks */
@@ -1360,14 +1361,10 @@
pegasus->intf = intf;
pegasus->usb = dev;
pegasus->net = net;
- net->open = pegasus_open;
- net->stop = pegasus_close;
+
+
net->watchdog_timeo = PEGASUS_TX_TIMEOUT;
- net->tx_timeout = pegasus_tx_timeout;
- net->do_ioctl = pegasus_ioctl;
- net->hard_start_xmit = pegasus_start_xmit;
- net->set_multicast_list = pegasus_set_multicast;
- net->get_stats = pegasus_netdev_stats;
+ net->netdev_ops = &pegasus_netdev_ops;
SET_ETHTOOL_OPS(net, &ops);
pegasus->mii.dev = net;
pegasus->mii.mdio_read = mdio_read;
@@ -1482,6 +1479,16 @@
return 0;
}
+static const struct net_device_ops pegasus_netdev_ops = {
+ .ndo_open = pegasus_open,
+ .ndo_stop = pegasus_close,
+ .ndo_do_ioctl = pegasus_ioctl,
+ .ndo_start_xmit = pegasus_start_xmit,
+ .ndo_set_multicast_list = pegasus_set_multicast,
+ .ndo_get_stats = pegasus_netdev_stats,
+ .ndo_tx_timeout = pegasus_tx_timeout,
+};
+
static struct usb_driver pegasus_driver = {
.name = driver_name,
.probe = pegasus_probe,
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index b7004ff..43f6523 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -624,6 +624,18 @@
return 0;
}
+static const struct net_device_ops virtnet_netdev = {
+ .ndo_open = virtnet_open,
+ .ndo_stop = virtnet_close,
+ .ndo_start_xmit = start_xmit,
+ .ndo_validate_addr = eth_validate_addr,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_change_mtu = virtnet_change_mtu,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+ .ndo_poll_controller = virtnet_netpoll,
+#endif
+};
+
static int virtnet_probe(struct virtio_device *vdev)
{
int err;
@@ -636,14 +648,8 @@
return -ENOMEM;
/* Set up network device as normal. */
- dev->open = virtnet_open;
- dev->stop = virtnet_close;
- dev->hard_start_xmit = start_xmit;
- dev->change_mtu = virtnet_change_mtu;
+ dev->netdev_ops = &virtnet_netdev;
dev->features = NETIF_F_HIGHDMA;
-#ifdef CONFIG_NET_POLL_CONTROLLER
- dev->poll_controller = virtnet_netpoll;
-#endif
SET_ETHTOOL_OPS(dev, &virtnet_ethtool_ops);
SET_NETDEV_DEV(dev, &vdev->dev);
diff --git a/drivers/net/wimax/i2400m/usb.c b/drivers/net/wimax/i2400m/usb.c
index 6d4b65f..c6d9346 100644
--- a/drivers/net/wimax/i2400m/usb.c
+++ b/drivers/net/wimax/i2400m/usb.c
@@ -397,11 +397,13 @@
i2400m->bus_fw_name = I2400MU_FW_FILE_NAME;
i2400m->bus_bm_mac_addr_impaired = 0;
+#ifdef CONFIG_PM
iface->needs_remote_wakeup = 1; /* autosuspend (15s delay) */
device_init_wakeup(dev, 1);
usb_autopm_enable(i2400mu->usb_iface);
usb_dev->autosuspend_delay = 15 * HZ;
usb_dev->autosuspend_disabled = 0;
+#endif
result = i2400m_setup(i2400m, I2400M_BRI_MAC_REINIT);
if (result < 0) {
@@ -493,7 +495,9 @@
int result = 0;
struct device *dev = &iface->dev;
struct i2400mu *i2400mu = usb_get_intfdata(iface);
+#ifdef CONFIG_PM
struct usb_device *usb_dev = i2400mu->usb_dev;
+#endif
struct i2400m *i2400m = &i2400mu->i2400m;
d_fnstart(3, dev, "(iface %p pm_msg %u)\n", iface, pm_msg.event);
@@ -503,11 +507,13 @@
atomic_dec(&i2400mu->do_autopm);
result = i2400m_cmd_enter_powersave(i2400m);
atomic_inc(&i2400mu->do_autopm);
+#ifdef CONFIG_PM
if (result < 0 && usb_dev->auto_pm == 0) {
/* System suspend, can't fail */
dev_err(dev, "failed to suspend, will reset on resume\n");
result = 0;
}
+#endif
if (result < 0)
goto error_enter_powersave;
i2400mu_notification_release(i2400mu);
diff --git a/drivers/net/wireless/ath5k/dma.c b/drivers/net/wireless/ath5k/dma.c
index 7e2b1a6..b65b4fe 100644
--- a/drivers/net/wireless/ath5k/dma.c
+++ b/drivers/net/wireless/ath5k/dma.c
@@ -594,7 +594,7 @@
* XXX: BMISS interrupts may occur after association.
* I found this on 5210 code but it needs testing. If this is
* true we should disable them before assoc and re-enable them
- * after a successfull assoc + some jiffies.
+ * after a successful assoc + some jiffies.
interrupt_mask &= ~AR5K_INT_BMISS;
*/
}
diff --git a/drivers/net/wireless/zd1211rw/zd_mac.c b/drivers/net/wireless/zd1211rw/zd_mac.c
index 9caa96a..a611ad8 100644
--- a/drivers/net/wireless/zd1211rw/zd_mac.c
+++ b/drivers/net/wireless/zd1211rw/zd_mac.c
@@ -287,7 +287,7 @@
* @skb - a sk-buffer
* @flags: extra flags to set in the TX status info
* @ackssi: ACK signal strength
- * @success - True for successfull transmission of the frame
+ * @success - True for successful transmission of the frame
*
* This information calls ieee80211_tx_status_irqsafe() if required by the
* control information. It copies the control information into the status
diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
index 761635b..cd6184e 100644
--- a/drivers/net/xen-netfront.c
+++ b/drivers/net/xen-netfront.c
@@ -1105,6 +1105,16 @@
gnttab_free_grant_references(np->gref_rx_head);
}
+static const struct net_device_ops xennet_netdev_ops = {
+ .ndo_open = xennet_open,
+ .ndo_uninit = xennet_uninit,
+ .ndo_stop = xennet_close,
+ .ndo_start_xmit = xennet_start_xmit,
+ .ndo_change_mtu = xennet_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
static struct net_device * __devinit xennet_create_dev(struct xenbus_device *dev)
{
int i, err;
@@ -1161,12 +1171,9 @@
goto exit_free_tx;
}
- netdev->open = xennet_open;
- netdev->hard_start_xmit = xennet_start_xmit;
- netdev->stop = xennet_close;
+ netdev->netdev_ops = &xennet_netdev_ops;
+
netif_napi_add(netdev, &np->napi, xennet_poll, 64);
- netdev->uninit = xennet_uninit;
- netdev->change_mtu = xennet_change_mtu;
netdev->features = NETIF_F_IP_CSUM;
SET_ETHTOOL_OPS(netdev, &xennet_ethtool_ops);
diff --git a/drivers/oprofile/buffer_sync.c b/drivers/oprofile/buffer_sync.c
index 65e8294..9da5a4b 100644
--- a/drivers/oprofile/buffer_sync.c
+++ b/drivers/oprofile/buffer_sync.c
@@ -1,11 +1,12 @@
/**
* @file buffer_sync.c
*
- * @remark Copyright 2002 OProfile authors
+ * @remark Copyright 2002-2009 OProfile authors
* @remark Read the file COPYING
*
* @author John Levon <levon@movementarian.org>
* @author Barry Kasindorf
+ * @author Robert Richter <robert.richter@amd.com>
*
* This is the core of the buffer management. Each
* CPU buffer is processed and entered into the
@@ -315,88 +316,73 @@
add_event_entry(TRACE_BEGIN_CODE);
}
-#ifdef CONFIG_OPROFILE_IBS
-
-#define IBS_FETCH_CODE_SIZE 2
-#define IBS_OP_CODE_SIZE 5
-
-/*
- * Add IBS fetch and op entries to event buffer
- */
-static void add_ibs_begin(int cpu, int code, struct mm_struct *mm)
+static void add_data(struct op_entry *entry, struct mm_struct *mm)
{
- unsigned long rip;
- int i, count;
- unsigned long ibs_cookie = 0;
+ unsigned long code, pc, val;
+ unsigned long cookie;
off_t offset;
- struct op_sample *sample;
- sample = cpu_buffer_read_entry(cpu);
- if (!sample)
- goto Error;
- rip = sample->eip;
-
-#ifdef __LP64__
- rip += sample->event << 32;
-#endif
+ if (!op_cpu_buffer_get_data(entry, &code))
+ return;
+ if (!op_cpu_buffer_get_data(entry, &pc))
+ return;
+ if (!op_cpu_buffer_get_size(entry))
+ return;
if (mm) {
- ibs_cookie = lookup_dcookie(mm, rip, &offset);
+ cookie = lookup_dcookie(mm, pc, &offset);
- if (ibs_cookie == NO_COOKIE)
- offset = rip;
- if (ibs_cookie == INVALID_COOKIE) {
+ if (cookie == NO_COOKIE)
+ offset = pc;
+ if (cookie == INVALID_COOKIE) {
atomic_inc(&oprofile_stats.sample_lost_no_mapping);
- offset = rip;
+ offset = pc;
}
- if (ibs_cookie != last_cookie) {
- add_cookie_switch(ibs_cookie);
- last_cookie = ibs_cookie;
+ if (cookie != last_cookie) {
+ add_cookie_switch(cookie);
+ last_cookie = cookie;
}
} else
- offset = rip;
+ offset = pc;
add_event_entry(ESCAPE_CODE);
add_event_entry(code);
add_event_entry(offset); /* Offset from Dcookie */
- /* we send the Dcookie offset, but send the raw Linear Add also*/
- add_event_entry(sample->eip);
- add_event_entry(sample->event);
-
- if (code == IBS_FETCH_CODE)
- count = IBS_FETCH_CODE_SIZE; /*IBS FETCH is 2 int64s*/
- else
- count = IBS_OP_CODE_SIZE; /*IBS OP is 5 int64s*/
-
- for (i = 0; i < count; i++) {
- sample = cpu_buffer_read_entry(cpu);
- if (!sample)
- goto Error;
- add_event_entry(sample->eip);
- add_event_entry(sample->event);
- }
-
- return;
-
-Error:
- return;
+ while (op_cpu_buffer_get_data(entry, &val))
+ add_event_entry(val);
}
-#endif
-
-static void add_sample_entry(unsigned long offset, unsigned long event)
+static inline void add_sample_entry(unsigned long offset, unsigned long event)
{
add_event_entry(offset);
add_event_entry(event);
}
-static int add_us_sample(struct mm_struct *mm, struct op_sample *s)
+/*
+ * Add a sample to the global event buffer. If possible the
+ * sample is converted into a persistent dentry/offset pair
+ * for later lookup from userspace. Return 0 on failure.
+ */
+static int
+add_sample(struct mm_struct *mm, struct op_sample *s, int in_kernel)
{
unsigned long cookie;
off_t offset;
+ if (in_kernel) {
+ add_sample_entry(s->eip, s->event);
+ return 1;
+ }
+
+ /* add userspace sample */
+
+ if (!mm) {
+ atomic_inc(&oprofile_stats.sample_lost_no_mm);
+ return 0;
+ }
+
cookie = lookup_dcookie(mm, s->eip, &offset);
if (cookie == INVALID_COOKIE) {
@@ -415,25 +401,6 @@
}
-/* Add a sample to the global event buffer. If possible the
- * sample is converted into a persistent dentry/offset pair
- * for later lookup from userspace.
- */
-static int
-add_sample(struct mm_struct *mm, struct op_sample *s, int in_kernel)
-{
- if (in_kernel) {
- add_sample_entry(s->eip, s->event);
- return 1;
- } else if (mm) {
- return add_us_sample(mm, s);
- } else {
- atomic_inc(&oprofile_stats.sample_lost_no_mm);
- }
- return 0;
-}
-
-
static void release_mm(struct mm_struct *mm)
{
if (!mm)
@@ -526,66 +493,69 @@
{
struct mm_struct *mm = NULL;
struct mm_struct *oldmm;
+ unsigned long val;
struct task_struct *new;
unsigned long cookie = 0;
int in_kernel = 1;
sync_buffer_state state = sb_buffer_start;
unsigned int i;
unsigned long available;
+ unsigned long flags;
+ struct op_entry entry;
+ struct op_sample *sample;
mutex_lock(&buffer_mutex);
add_cpu_switch(cpu);
- cpu_buffer_reset(cpu);
- available = cpu_buffer_entries(cpu);
+ op_cpu_buffer_reset(cpu);
+ available = op_cpu_buffer_entries(cpu);
for (i = 0; i < available; ++i) {
- struct op_sample *s = cpu_buffer_read_entry(cpu);
- if (!s)
+ sample = op_cpu_buffer_read_entry(&entry, cpu);
+ if (!sample)
break;
- if (is_code(s->eip)) {
- switch (s->event) {
- case 0:
- case CPU_IS_KERNEL:
- /* kernel/userspace switch */
- in_kernel = s->event;
- if (state == sb_buffer_start)
- state = sb_sample_start;
- add_kernel_ctx_switch(s->event);
- break;
- case CPU_TRACE_BEGIN:
+ if (is_code(sample->eip)) {
+ flags = sample->event;
+ if (flags & TRACE_BEGIN) {
state = sb_bt_start;
add_trace_begin();
- break;
-#ifdef CONFIG_OPROFILE_IBS
- case IBS_FETCH_BEGIN:
- state = sb_bt_start;
- add_ibs_begin(cpu, IBS_FETCH_CODE, mm);
- break;
- case IBS_OP_BEGIN:
- state = sb_bt_start;
- add_ibs_begin(cpu, IBS_OP_CODE, mm);
- break;
-#endif
- default:
+ }
+ if (flags & KERNEL_CTX_SWITCH) {
+ /* kernel/userspace switch */
+ in_kernel = flags & IS_KERNEL;
+ if (state == sb_buffer_start)
+ state = sb_sample_start;
+ add_kernel_ctx_switch(flags & IS_KERNEL);
+ }
+ if (flags & USER_CTX_SWITCH
+ && op_cpu_buffer_get_data(&entry, &val)) {
/* userspace context switch */
+ new = (struct task_struct *)val;
oldmm = mm;
- new = (struct task_struct *)s->event;
release_mm(oldmm);
mm = take_tasks_mm(new);
if (mm != oldmm)
cookie = get_exec_dcookie(mm);
add_user_ctx_switch(new, cookie);
- break;
}
- } else if (state >= sb_bt_start &&
- !add_sample(mm, s, in_kernel)) {
- if (state == sb_bt_start) {
- state = sb_bt_ignore;
- atomic_inc(&oprofile_stats.bt_lost_no_mapping);
- }
+ if (op_cpu_buffer_get_size(&entry))
+ add_data(&entry, mm);
+ continue;
+ }
+
+ if (state < sb_bt_start)
+ /* ignore sample */
+ continue;
+
+ if (add_sample(mm, sample, in_kernel))
+ continue;
+
+ /* ignore backtraces if failed to add a sample */
+ if (state == sb_bt_start) {
+ state = sb_bt_ignore;
+ atomic_inc(&oprofile_stats.bt_lost_no_mapping);
}
}
release_mm(mm);
diff --git a/drivers/oprofile/cpu_buffer.c b/drivers/oprofile/cpu_buffer.c
index 6109096..2e03b6d 100644
--- a/drivers/oprofile/cpu_buffer.c
+++ b/drivers/oprofile/cpu_buffer.c
@@ -1,11 +1,12 @@
/**
* @file cpu_buffer.c
*
- * @remark Copyright 2002 OProfile authors
+ * @remark Copyright 2002-2009 OProfile authors
* @remark Read the file COPYING
*
* @author John Levon <levon@movementarian.org>
* @author Barry Kasindorf <barry.kasindorf@amd.com>
+ * @author Robert Richter <robert.richter@amd.com>
*
* Each CPU has a local buffer that stores PC value/event
* pairs. We also log context switches when we notice them.
@@ -45,8 +46,8 @@
* can be changed to a single buffer solution when the ring buffer
* access is implemented as non-locking atomic code.
*/
-struct ring_buffer *op_ring_buffer_read;
-struct ring_buffer *op_ring_buffer_write;
+static struct ring_buffer *op_ring_buffer_read;
+static struct ring_buffer *op_ring_buffer_write;
DEFINE_PER_CPU(struct oprofile_cpu_buffer, cpu_buffer);
static void wq_sync_buffer(struct work_struct *work);
@@ -54,6 +55,19 @@
#define DEFAULT_TIMER_EXPIRE (HZ / 10)
static int work_enabled;
+unsigned long oprofile_get_cpu_buffer_size(void)
+{
+ return oprofile_cpu_buffer_size;
+}
+
+void oprofile_cpu_buffer_inc_smpl_lost(void)
+{
+ struct oprofile_cpu_buffer *cpu_buf
+ = &__get_cpu_var(cpu_buffer);
+
+ cpu_buf->sample_lost_overflow++;
+}
+
void free_cpu_buffers(void)
{
if (op_ring_buffer_read)
@@ -64,24 +78,11 @@
op_ring_buffer_write = NULL;
}
-unsigned long oprofile_get_cpu_buffer_size(void)
-{
- return fs_cpu_buffer_size;
-}
-
-void oprofile_cpu_buffer_inc_smpl_lost(void)
-{
- struct oprofile_cpu_buffer *cpu_buf
- = &__get_cpu_var(cpu_buffer);
-
- cpu_buf->sample_lost_overflow++;
-}
-
int alloc_cpu_buffers(void)
{
int i;
- unsigned long buffer_size = fs_cpu_buffer_size;
+ unsigned long buffer_size = oprofile_cpu_buffer_size;
op_ring_buffer_read = ring_buffer_alloc(buffer_size, OP_BUFFER_FLAGS);
if (!op_ring_buffer_read)
@@ -97,8 +98,6 @@
b->last_is_kernel = -1;
b->tracing = 0;
b->buffer_size = buffer_size;
- b->tail_pos = 0;
- b->head_pos = 0;
b->sample_received = 0;
b->sample_lost_overflow = 0;
b->backtrace_aborted = 0;
@@ -145,47 +144,156 @@
flush_scheduled_work();
}
-static inline int
-add_sample(struct oprofile_cpu_buffer *cpu_buf,
- unsigned long pc, unsigned long event)
+/*
+ * This function prepares the cpu buffer to write a sample.
+ *
+ * Struct op_entry is used during operations on the ring buffer while
+ * struct op_sample contains the data that is stored in the ring
+ * buffer. Struct entry can be uninitialized. The function reserves a
+ * data array that is specified by size. Use
+ * op_cpu_buffer_write_commit() after preparing the sample. In case of
+ * errors a null pointer is returned, otherwise the pointer to the
+ * sample.
+ *
+ */
+struct op_sample
+*op_cpu_buffer_write_reserve(struct op_entry *entry, unsigned long size)
+{
+ entry->event = ring_buffer_lock_reserve
+ (op_ring_buffer_write, sizeof(struct op_sample) +
+ size * sizeof(entry->sample->data[0]), &entry->irq_flags);
+ if (entry->event)
+ entry->sample = ring_buffer_event_data(entry->event);
+ else
+ entry->sample = NULL;
+
+ if (!entry->sample)
+ return NULL;
+
+ entry->size = size;
+ entry->data = entry->sample->data;
+
+ return entry->sample;
+}
+
+int op_cpu_buffer_write_commit(struct op_entry *entry)
+{
+ return ring_buffer_unlock_commit(op_ring_buffer_write, entry->event,
+ entry->irq_flags);
+}
+
+struct op_sample *op_cpu_buffer_read_entry(struct op_entry *entry, int cpu)
+{
+ struct ring_buffer_event *e;
+ e = ring_buffer_consume(op_ring_buffer_read, cpu, NULL);
+ if (e)
+ goto event;
+ if (ring_buffer_swap_cpu(op_ring_buffer_read,
+ op_ring_buffer_write,
+ cpu))
+ return NULL;
+ e = ring_buffer_consume(op_ring_buffer_read, cpu, NULL);
+ if (e)
+ goto event;
+ return NULL;
+
+event:
+ entry->event = e;
+ entry->sample = ring_buffer_event_data(e);
+ entry->size = (ring_buffer_event_length(e) - sizeof(struct op_sample))
+ / sizeof(entry->sample->data[0]);
+ entry->data = entry->sample->data;
+ return entry->sample;
+}
+
+unsigned long op_cpu_buffer_entries(int cpu)
+{
+ return ring_buffer_entries_cpu(op_ring_buffer_read, cpu)
+ + ring_buffer_entries_cpu(op_ring_buffer_write, cpu);
+}
+
+static int
+op_add_code(struct oprofile_cpu_buffer *cpu_buf, unsigned long backtrace,
+ int is_kernel, struct task_struct *task)
{
struct op_entry entry;
- int ret;
+ struct op_sample *sample;
+ unsigned long flags;
+ int size;
- ret = cpu_buffer_write_entry(&entry);
- if (ret)
- return ret;
+ flags = 0;
- entry.sample->eip = pc;
- entry.sample->event = event;
+ if (backtrace)
+ flags |= TRACE_BEGIN;
- ret = cpu_buffer_write_commit(&entry);
- if (ret)
- return ret;
+ /* notice a switch from user->kernel or vice versa */
+ is_kernel = !!is_kernel;
+ if (cpu_buf->last_is_kernel != is_kernel) {
+ cpu_buf->last_is_kernel = is_kernel;
+ flags |= KERNEL_CTX_SWITCH;
+ if (is_kernel)
+ flags |= IS_KERNEL;
+ }
+
+ /* notice a task switch */
+ if (cpu_buf->last_task != task) {
+ cpu_buf->last_task = task;
+ flags |= USER_CTX_SWITCH;
+ }
+
+ if (!flags)
+ /* nothing to do */
+ return 0;
+
+ if (flags & USER_CTX_SWITCH)
+ size = 1;
+ else
+ size = 0;
+
+ sample = op_cpu_buffer_write_reserve(&entry, size);
+ if (!sample)
+ return -ENOMEM;
+
+ sample->eip = ESCAPE_CODE;
+ sample->event = flags;
+
+ if (size)
+ op_cpu_buffer_add_data(&entry, (unsigned long)task);
+
+ op_cpu_buffer_write_commit(&entry);
return 0;
}
static inline int
-add_code(struct oprofile_cpu_buffer *buffer, unsigned long value)
+op_add_sample(struct oprofile_cpu_buffer *cpu_buf,
+ unsigned long pc, unsigned long event)
{
- return add_sample(buffer, ESCAPE_CODE, value);
+ struct op_entry entry;
+ struct op_sample *sample;
+
+ sample = op_cpu_buffer_write_reserve(&entry, 0);
+ if (!sample)
+ return -ENOMEM;
+
+ sample->eip = pc;
+ sample->event = event;
+
+ return op_cpu_buffer_write_commit(&entry);
}
-/* This must be safe from any context. It's safe writing here
- * because of the head/tail separation of the writer and reader
- * of the CPU buffer.
+/*
+ * This must be safe from any context.
*
* is_kernel is needed because on some architectures you cannot
* tell if you are in kernel or user space simply by looking at
* pc. We tag this in the buffer by generating kernel enter/exit
* events whenever is_kernel changes
*/
-static int log_sample(struct oprofile_cpu_buffer *cpu_buf, unsigned long pc,
- int is_kernel, unsigned long event)
+static int
+log_sample(struct oprofile_cpu_buffer *cpu_buf, unsigned long pc,
+ unsigned long backtrace, int is_kernel, unsigned long event)
{
- struct task_struct *task;
-
cpu_buf->sample_received++;
if (pc == ESCAPE_CODE) {
@@ -193,25 +301,10 @@
return 0;
}
- is_kernel = !!is_kernel;
+ if (op_add_code(cpu_buf, backtrace, is_kernel, current))
+ goto fail;
- task = current;
-
- /* notice a switch from user->kernel or vice versa */
- if (cpu_buf->last_is_kernel != is_kernel) {
- cpu_buf->last_is_kernel = is_kernel;
- if (add_code(cpu_buf, is_kernel))
- goto fail;
- }
-
- /* notice a task switch */
- if (cpu_buf->last_task != task) {
- cpu_buf->last_task = task;
- if (add_code(cpu_buf, (unsigned long)task))
- goto fail;
- }
-
- if (add_sample(cpu_buf, pc, event))
+ if (op_add_sample(cpu_buf, pc, event))
goto fail;
return 1;
@@ -221,109 +314,102 @@
return 0;
}
-static int oprofile_begin_trace(struct oprofile_cpu_buffer *cpu_buf)
+static inline void oprofile_begin_trace(struct oprofile_cpu_buffer *cpu_buf)
{
- add_code(cpu_buf, CPU_TRACE_BEGIN);
cpu_buf->tracing = 1;
- return 1;
}
-static void oprofile_end_trace(struct oprofile_cpu_buffer *cpu_buf)
+static inline void oprofile_end_trace(struct oprofile_cpu_buffer *cpu_buf)
{
cpu_buf->tracing = 0;
}
-void oprofile_add_ext_sample(unsigned long pc, struct pt_regs * const regs,
- unsigned long event, int is_kernel)
+static inline void
+__oprofile_add_ext_sample(unsigned long pc, struct pt_regs * const regs,
+ unsigned long event, int is_kernel)
{
struct oprofile_cpu_buffer *cpu_buf = &__get_cpu_var(cpu_buffer);
-
- if (!backtrace_depth) {
- log_sample(cpu_buf, pc, is_kernel, event);
- return;
- }
-
- if (!oprofile_begin_trace(cpu_buf))
- return;
+ unsigned long backtrace = oprofile_backtrace_depth;
/*
* if log_sample() fail we can't backtrace since we lost the
* source of this event
*/
- if (log_sample(cpu_buf, pc, is_kernel, event))
- oprofile_ops.backtrace(regs, backtrace_depth);
+ if (!log_sample(cpu_buf, pc, backtrace, is_kernel, event))
+ /* failed */
+ return;
+
+ if (!backtrace)
+ return;
+
+ oprofile_begin_trace(cpu_buf);
+ oprofile_ops.backtrace(regs, backtrace);
oprofile_end_trace(cpu_buf);
}
+void oprofile_add_ext_sample(unsigned long pc, struct pt_regs * const regs,
+ unsigned long event, int is_kernel)
+{
+ __oprofile_add_ext_sample(pc, regs, event, is_kernel);
+}
+
void oprofile_add_sample(struct pt_regs * const regs, unsigned long event)
{
int is_kernel = !user_mode(regs);
unsigned long pc = profile_pc(regs);
- oprofile_add_ext_sample(pc, regs, event, is_kernel);
+ __oprofile_add_ext_sample(pc, regs, event, is_kernel);
}
-#ifdef CONFIG_OPROFILE_IBS
-
-#define MAX_IBS_SAMPLE_SIZE 14
-
-void oprofile_add_ibs_sample(struct pt_regs * const regs,
- unsigned int * const ibs_sample, int ibs_code)
+/*
+ * Add samples with data to the ring buffer.
+ *
+ * Use oprofile_add_data(&entry, val) to add data and
+ * oprofile_write_commit(&entry) to commit the sample.
+ */
+void
+oprofile_write_reserve(struct op_entry *entry, struct pt_regs * const regs,
+ unsigned long pc, int code, int size)
{
+ struct op_sample *sample;
int is_kernel = !user_mode(regs);
struct oprofile_cpu_buffer *cpu_buf = &__get_cpu_var(cpu_buffer);
- struct task_struct *task;
- int fail = 0;
cpu_buf->sample_received++;
- /* notice a switch from user->kernel or vice versa */
- if (cpu_buf->last_is_kernel != is_kernel) {
- if (add_code(cpu_buf, is_kernel))
- goto fail;
- cpu_buf->last_is_kernel = is_kernel;
- }
-
- /* notice a task switch */
- if (!is_kernel) {
- task = current;
- if (cpu_buf->last_task != task) {
- if (add_code(cpu_buf, (unsigned long)task))
- goto fail;
- cpu_buf->last_task = task;
- }
- }
-
- fail = fail || add_code(cpu_buf, ibs_code);
- fail = fail || add_sample(cpu_buf, ibs_sample[0], ibs_sample[1]);
- fail = fail || add_sample(cpu_buf, ibs_sample[2], ibs_sample[3]);
- fail = fail || add_sample(cpu_buf, ibs_sample[4], ibs_sample[5]);
-
- if (ibs_code == IBS_OP_BEGIN) {
- fail = fail || add_sample(cpu_buf, ibs_sample[6], ibs_sample[7]);
- fail = fail || add_sample(cpu_buf, ibs_sample[8], ibs_sample[9]);
- fail = fail || add_sample(cpu_buf, ibs_sample[10], ibs_sample[11]);
- }
-
- if (fail)
+ /* no backtraces for samples with data */
+ if (op_add_code(cpu_buf, 0, is_kernel, current))
goto fail;
- if (backtrace_depth)
- oprofile_ops.backtrace(regs, backtrace_depth);
+ sample = op_cpu_buffer_write_reserve(entry, size + 2);
+ if (!sample)
+ goto fail;
+ sample->eip = ESCAPE_CODE;
+ sample->event = 0; /* no flags */
+
+ op_cpu_buffer_add_data(entry, code);
+ op_cpu_buffer_add_data(entry, pc);
return;
fail:
cpu_buf->sample_lost_overflow++;
- return;
}
-#endif
+int oprofile_add_data(struct op_entry *entry, unsigned long val)
+{
+ return op_cpu_buffer_add_data(entry, val);
+}
+
+int oprofile_write_commit(struct op_entry *entry)
+{
+ return op_cpu_buffer_write_commit(entry);
+}
void oprofile_add_pc(unsigned long pc, int is_kernel, unsigned long event)
{
struct oprofile_cpu_buffer *cpu_buf = &__get_cpu_var(cpu_buffer);
- log_sample(cpu_buf, pc, is_kernel, event);
+ log_sample(cpu_buf, pc, 0, is_kernel, event);
}
void oprofile_add_trace(unsigned long pc)
@@ -340,7 +426,7 @@
if (pc == ESCAPE_CODE)
goto fail;
- if (add_sample(cpu_buf, pc, 0))
+ if (op_add_sample(cpu_buf, pc, 0))
goto fail;
return;
diff --git a/drivers/oprofile/cpu_buffer.h b/drivers/oprofile/cpu_buffer.h
index aacb0f0..63f81c4 100644
--- a/drivers/oprofile/cpu_buffer.h
+++ b/drivers/oprofile/cpu_buffer.h
@@ -1,10 +1,11 @@
/**
* @file cpu_buffer.h
*
- * @remark Copyright 2002 OProfile authors
+ * @remark Copyright 2002-2009 OProfile authors
* @remark Read the file COPYING
*
* @author John Levon <levon@movementarian.org>
+ * @author Robert Richter <robert.richter@amd.com>
*/
#ifndef OPROFILE_CPU_BUFFER_H
@@ -31,17 +32,12 @@
struct op_sample {
unsigned long eip;
unsigned long event;
+ unsigned long data[0];
};
-struct op_entry {
- struct ring_buffer_event *event;
- struct op_sample *sample;
- unsigned long irq_flags;
-};
+struct op_entry;
struct oprofile_cpu_buffer {
- volatile unsigned long head_pos;
- volatile unsigned long tail_pos;
unsigned long buffer_size;
struct task_struct *last_task;
int last_is_kernel;
@@ -54,8 +50,6 @@
struct delayed_work work;
};
-extern struct ring_buffer *op_ring_buffer_read;
-extern struct ring_buffer *op_ring_buffer_write;
DECLARE_PER_CPU(struct oprofile_cpu_buffer, cpu_buffer);
/*
@@ -64,7 +58,7 @@
* reset these to invalid values; the next sample collected will
* populate the buffer with proper values to initialize the buffer
*/
-static inline void cpu_buffer_reset(int cpu)
+static inline void op_cpu_buffer_reset(int cpu)
{
struct oprofile_cpu_buffer *cpu_buf = &per_cpu(cpu_buffer, cpu);
@@ -72,55 +66,48 @@
cpu_buf->last_task = NULL;
}
-static inline int cpu_buffer_write_entry(struct op_entry *entry)
+struct op_sample
+*op_cpu_buffer_write_reserve(struct op_entry *entry, unsigned long size);
+int op_cpu_buffer_write_commit(struct op_entry *entry);
+struct op_sample *op_cpu_buffer_read_entry(struct op_entry *entry, int cpu);
+unsigned long op_cpu_buffer_entries(int cpu);
+
+/* returns the remaining free size of data in the entry */
+static inline
+int op_cpu_buffer_add_data(struct op_entry *entry, unsigned long val)
{
- entry->event = ring_buffer_lock_reserve(op_ring_buffer_write,
- sizeof(struct op_sample),
- &entry->irq_flags);
- if (entry->event)
- entry->sample = ring_buffer_event_data(entry->event);
- else
- entry->sample = NULL;
-
- if (!entry->sample)
- return -ENOMEM;
-
- return 0;
+ if (!entry->size)
+ return 0;
+ *entry->data = val;
+ entry->size--;
+ entry->data++;
+ return entry->size;
}
-static inline int cpu_buffer_write_commit(struct op_entry *entry)
+/* returns the size of data in the entry */
+static inline
+int op_cpu_buffer_get_size(struct op_entry *entry)
{
- return ring_buffer_unlock_commit(op_ring_buffer_write, entry->event,
- entry->irq_flags);
+ return entry->size;
}
-static inline struct op_sample *cpu_buffer_read_entry(int cpu)
+/* returns 0 if empty or the size of data including the current value */
+static inline
+int op_cpu_buffer_get_data(struct op_entry *entry, unsigned long *val)
{
- struct ring_buffer_event *e;
- e = ring_buffer_consume(op_ring_buffer_read, cpu, NULL);
- if (e)
- return ring_buffer_event_data(e);
- if (ring_buffer_swap_cpu(op_ring_buffer_read,
- op_ring_buffer_write,
- cpu))
- return NULL;
- e = ring_buffer_consume(op_ring_buffer_read, cpu, NULL);
- if (e)
- return ring_buffer_event_data(e);
- return NULL;
+ int size = entry->size;
+ if (!size)
+ return 0;
+ *val = *entry->data;
+ entry->size--;
+ entry->data++;
+ return size;
}
-/* "acquire" as many cpu buffer slots as we can */
-static inline unsigned long cpu_buffer_entries(int cpu)
-{
- return ring_buffer_entries_cpu(op_ring_buffer_read, cpu)
- + ring_buffer_entries_cpu(op_ring_buffer_write, cpu);
-}
-
-/* transient events for the CPU buffer -> event buffer */
-#define CPU_IS_KERNEL 1
-#define CPU_TRACE_BEGIN 2
-#define IBS_FETCH_BEGIN 3
-#define IBS_OP_BEGIN 4
+/* extra data flags */
+#define KERNEL_CTX_SWITCH (1UL << 0)
+#define IS_KERNEL (1UL << 1)
+#define TRACE_BEGIN (1UL << 2)
+#define USER_CTX_SWITCH (1UL << 3)
#endif /* OPROFILE_CPU_BUFFER_H */
diff --git a/drivers/oprofile/event_buffer.c b/drivers/oprofile/event_buffer.c
index 191a320..2b7ae36 100644
--- a/drivers/oprofile/event_buffer.c
+++ b/drivers/oprofile/event_buffer.c
@@ -73,8 +73,8 @@
unsigned long flags;
spin_lock_irqsave(&oprofilefs_lock, flags);
- buffer_size = fs_buffer_size;
- buffer_watershed = fs_buffer_watershed;
+ buffer_size = oprofile_buffer_size;
+ buffer_watershed = oprofile_buffer_watershed;
spin_unlock_irqrestore(&oprofilefs_lock, flags);
if (buffer_watershed >= buffer_size)
diff --git a/drivers/oprofile/oprof.c b/drivers/oprofile/oprof.c
index cd37590..3cffce9 100644
--- a/drivers/oprofile/oprof.c
+++ b/drivers/oprofile/oprof.c
@@ -23,7 +23,7 @@
struct oprofile_operations oprofile_ops;
unsigned long oprofile_started;
-unsigned long backtrace_depth;
+unsigned long oprofile_backtrace_depth;
static unsigned long is_setup;
static DEFINE_MUTEX(start_mutex);
@@ -172,7 +172,7 @@
goto out;
}
- backtrace_depth = val;
+ oprofile_backtrace_depth = val;
out:
mutex_unlock(&start_mutex);
diff --git a/drivers/oprofile/oprof.h b/drivers/oprofile/oprof.h
index 5df0c21..c288d3c 100644
--- a/drivers/oprofile/oprof.h
+++ b/drivers/oprofile/oprof.h
@@ -21,12 +21,12 @@
struct oprofile_operations;
-extern unsigned long fs_buffer_size;
-extern unsigned long fs_cpu_buffer_size;
-extern unsigned long fs_buffer_watershed;
+extern unsigned long oprofile_buffer_size;
+extern unsigned long oprofile_cpu_buffer_size;
+extern unsigned long oprofile_buffer_watershed;
extern struct oprofile_operations oprofile_ops;
extern unsigned long oprofile_started;
-extern unsigned long backtrace_depth;
+extern unsigned long oprofile_backtrace_depth;
struct super_block;
struct dentry;
diff --git a/drivers/oprofile/oprofile_files.c b/drivers/oprofile/oprofile_files.c
index d820199..5d36ffc 100644
--- a/drivers/oprofile/oprofile_files.c
+++ b/drivers/oprofile/oprofile_files.c
@@ -14,17 +14,18 @@
#include "oprofile_stats.h"
#include "oprof.h"
-#define FS_BUFFER_SIZE_DEFAULT 131072
-#define FS_CPU_BUFFER_SIZE_DEFAULT 8192
-#define FS_BUFFER_WATERSHED_DEFAULT 32768 /* FIXME: tune */
+#define BUFFER_SIZE_DEFAULT 131072
+#define CPU_BUFFER_SIZE_DEFAULT 8192
+#define BUFFER_WATERSHED_DEFAULT 32768 /* FIXME: tune */
-unsigned long fs_buffer_size;
-unsigned long fs_cpu_buffer_size;
-unsigned long fs_buffer_watershed;
+unsigned long oprofile_buffer_size;
+unsigned long oprofile_cpu_buffer_size;
+unsigned long oprofile_buffer_watershed;
static ssize_t depth_read(struct file *file, char __user *buf, size_t count, loff_t *offset)
{
- return oprofilefs_ulong_to_user(backtrace_depth, buf, count, offset);
+ return oprofilefs_ulong_to_user(oprofile_backtrace_depth, buf, count,
+ offset);
}
@@ -125,16 +126,16 @@
void oprofile_create_files(struct super_block *sb, struct dentry *root)
{
/* reinitialize default values */
- fs_buffer_size = FS_BUFFER_SIZE_DEFAULT;
- fs_cpu_buffer_size = FS_CPU_BUFFER_SIZE_DEFAULT;
- fs_buffer_watershed = FS_BUFFER_WATERSHED_DEFAULT;
+ oprofile_buffer_size = BUFFER_SIZE_DEFAULT;
+ oprofile_cpu_buffer_size = CPU_BUFFER_SIZE_DEFAULT;
+ oprofile_buffer_watershed = BUFFER_WATERSHED_DEFAULT;
oprofilefs_create_file(sb, root, "enable", &enable_fops);
oprofilefs_create_file_perm(sb, root, "dump", &dump_fops, 0666);
oprofilefs_create_file(sb, root, "buffer", &event_buffer_fops);
- oprofilefs_create_ulong(sb, root, "buffer_size", &fs_buffer_size);
- oprofilefs_create_ulong(sb, root, "buffer_watershed", &fs_buffer_watershed);
- oprofilefs_create_ulong(sb, root, "cpu_buffer_size", &fs_cpu_buffer_size);
+ oprofilefs_create_ulong(sb, root, "buffer_size", &oprofile_buffer_size);
+ oprofilefs_create_ulong(sb, root, "buffer_watershed", &oprofile_buffer_watershed);
+ oprofilefs_create_ulong(sb, root, "cpu_buffer_size", &oprofile_cpu_buffer_size);
oprofilefs_create_file(sb, root, "cpu_type", &cpu_type_fops);
oprofilefs_create_file(sb, root, "backtrace_depth", &depth_fops);
oprofilefs_create_file(sb, root, "pointer_size", &pointer_size_fops);
diff --git a/drivers/parisc/asp.c b/drivers/parisc/asp.c
index 8213691..79311335 100644
--- a/drivers/parisc/asp.c
+++ b/drivers/parisc/asp.c
@@ -71,8 +71,7 @@
*/
#define ASP_INTERRUPT_ADDR 0xf0800000
-int __init
-asp_init_chip(struct parisc_device *dev)
+static int __init asp_init_chip(struct parisc_device *dev)
{
struct gsc_irq gsc_irq;
int ret;
diff --git a/drivers/parisc/ccio-dma.c b/drivers/parisc/ccio-dma.c
index dcc1e99..cd4dd7e 100644
--- a/drivers/parisc/ccio-dma.c
+++ b/drivers/parisc/ccio-dma.c
@@ -555,7 +555,7 @@
* (Load Coherence Index) instruction. The 8 bits used for the virtual
* index are bits 12:19 of the value returned by LCI.
*/
-void CCIO_INLINE
+static void CCIO_INLINE
ccio_io_pdir_entry(u64 *pdir_ptr, space_t sid, unsigned long vba,
unsigned long hints)
{
@@ -1578,8 +1578,6 @@
ioc_count++;
- parisc_vmerge_boundary = IOVP_SIZE;
- parisc_vmerge_max_size = BITS_PER_LONG * IOVP_SIZE;
parisc_has_iommu();
return 0;
}
diff --git a/drivers/parisc/dino.c b/drivers/parisc/dino.c
index 77cc8bf..d539d9d 100644
--- a/drivers/parisc/dino.c
+++ b/drivers/parisc/dino.c
@@ -287,7 +287,7 @@
DINO_PORT_OUT(w, 16, 2)
DINO_PORT_OUT(l, 32, 0)
-struct pci_port_ops dino_port_ops = {
+static struct pci_port_ops dino_port_ops = {
.inb = dino_in8,
.inw = dino_in16,
.inl = dino_in32,
@@ -690,7 +690,7 @@
}
-struct pci_bios_ops dino_bios_ops = {
+static struct pci_bios_ops dino_bios_ops = {
.init = dino_bios_init,
.fixup_bus = dino_fixup_bus
};
diff --git a/drivers/parisc/hppb.c b/drivers/parisc/hppb.c
index 65eee67..1385641 100644
--- a/drivers/parisc/hppb.c
+++ b/drivers/parisc/hppb.c
@@ -29,7 +29,7 @@
struct hppb_card *next;
};
-struct hppb_card hppb_card_head = {
+static struct hppb_card hppb_card_head = {
.hpa = 0,
.next = NULL,
};
diff --git a/drivers/parisc/lasi.c b/drivers/parisc/lasi.c
index bee5100..e65727c 100644
--- a/drivers/parisc/lasi.c
+++ b/drivers/parisc/lasi.c
@@ -107,7 +107,7 @@
#else
-void __init lasi_led_init(unsigned long lasi_hpa)
+static void __init lasi_led_init(unsigned long lasi_hpa)
{
unsigned long datareg;
@@ -163,8 +163,7 @@
gsc_writel(0x02, datareg);
}
-int __init
-lasi_init_chip(struct parisc_device *dev)
+static int __init lasi_init_chip(struct parisc_device *dev)
{
extern void (*chassis_power_off)(void);
struct gsc_asic *lasi;
diff --git a/drivers/parisc/lba_pci.c b/drivers/parisc/lba_pci.c
index a28c894..d8233de 100644
--- a/drivers/parisc/lba_pci.c
+++ b/drivers/parisc/lba_pci.c
@@ -824,7 +824,7 @@
}
-struct pci_bios_ops lba_bios_ops = {
+static struct pci_bios_ops lba_bios_ops = {
.init = lba_bios_init,
.fixup_bus = lba_fixup_bus,
};
diff --git a/drivers/parisc/sba_iommu.c b/drivers/parisc/sba_iommu.c
index bc73b96..3fac8f8 100644
--- a/drivers/parisc/sba_iommu.c
+++ b/drivers/parisc/sba_iommu.c
@@ -561,7 +561,7 @@
* IOMMU uses little endian for the pdir.
*/
-void SBA_INLINE
+static void SBA_INLINE
sba_io_pdir_entry(u64 *pdir_ptr, space_t sid, unsigned long vba,
unsigned long hint)
{
@@ -1874,7 +1874,7 @@
{ 0, }
};
-int sba_driver_callback(struct parisc_device *);
+static int sba_driver_callback(struct parisc_device *);
static struct parisc_driver sba_driver = {
.name = MODULE_NAME,
@@ -1887,8 +1887,7 @@
** If so, initialize the chip and tell other partners in crime they
** have work to do.
*/
-int
-sba_driver_callback(struct parisc_device *dev)
+static int sba_driver_callback(struct parisc_device *dev)
{
struct sba_device *sba_dev;
u32 func_class;
@@ -1979,8 +1978,6 @@
proc_create("sba_iommu-bitmap", 0, root, &sba_proc_bitmap_fops);
#endif
- parisc_vmerge_boundary = IOVP_SIZE;
- parisc_vmerge_max_size = IOVP_SIZE * BITS_PER_LONG;
parisc_has_iommu();
return 0;
}
diff --git a/drivers/parisc/wax.c b/drivers/parisc/wax.c
index 892a83b..da9d5ad 100644
--- a/drivers/parisc/wax.c
+++ b/drivers/parisc/wax.c
@@ -68,8 +68,7 @@
// gsc_writel(0xFFFFFFFF, base+0x2000); /* RS232-B on Wax */
}
-int __init
-wax_init_chip(struct parisc_device *dev)
+static int __init wax_init_chip(struct parisc_device *dev)
{
struct gsc_asic *wax;
struct parisc_device *parent;
diff --git a/drivers/pci/hotplug/acpi_pcihp.c b/drivers/pci/hotplug/acpi_pcihp.c
index c62ab8d..1c11418 100644
--- a/drivers/pci/hotplug/acpi_pcihp.c
+++ b/drivers/pci/hotplug/acpi_pcihp.c
@@ -33,7 +33,6 @@
#include <linux/pci-acpi.h>
#include <acpi/acpi.h>
#include <acpi/acpi_bus.h>
-#include <acpi/actypes.h>
#define MY_NAME "acpi_pcihp"
diff --git a/drivers/pci/hotplug/pciehp.h b/drivers/pci/hotplug/pciehp.h
index 27fd18f..db85284 100644
--- a/drivers/pci/hotplug/pciehp.h
+++ b/drivers/pci/hotplug/pciehp.h
@@ -217,7 +217,6 @@
#ifdef CONFIG_ACPI
#include <acpi/acpi.h>
#include <acpi/acpi_bus.h>
-#include <acpi/actypes.h>
#include <linux/pci-acpi.h>
extern void __init pciehp_acpi_slot_detection_init(void);
diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c
index 3582512..deea8a1 100644
--- a/drivers/pci/pci-acpi.c
+++ b/drivers/pci/pci-acpi.c
@@ -13,8 +13,6 @@
#include <linux/module.h>
#include <linux/pci-aspm.h>
#include <acpi/acpi.h>
-#include <acpi/acnamesp.h>
-#include <acpi/acresrc.h>
#include <acpi/acpi_bus.h>
#include <linux/pci-acpi.h>
diff --git a/drivers/platform/Kconfig b/drivers/platform/Kconfig
new file mode 100644
index 0000000..9652c3f
--- /dev/null
+++ b/drivers/platform/Kconfig
@@ -0,0 +1,5 @@
+# drivers/platform/Kconfig
+
+if X86
+source "drivers/platform/x86/Kconfig"
+endif
diff --git a/drivers/platform/Makefile b/drivers/platform/Makefile
new file mode 100644
index 0000000..782953a
--- /dev/null
+++ b/drivers/platform/Makefile
@@ -0,0 +1,5 @@
+#
+# Makefile for linux/drivers/platform
+#
+
+obj-$(CONFIG_X86) += x86/
diff --git a/drivers/platform/x86/Kconfig b/drivers/platform/x86/Kconfig
new file mode 100644
index 0000000..e65448e
--- /dev/null
+++ b/drivers/platform/x86/Kconfig
@@ -0,0 +1,375 @@
+#
+# X86 Platform Specific Drivers
+#
+
+menuconfig X86_PLATFORM_DEVICES
+ bool "X86 Platform Specific Device Drivers"
+ default y
+ ---help---
+ Say Y here to get to see options for device drivers for various
+ x86 platforms, including vendor-specific laptop extension drivers.
+ This option alone does not add any kernel code.
+
+ If you say N, all options in this submenu will be skipped and disabled.
+
+if X86_PLATFORM_DEVICES
+
+config ACER_WMI
+ tristate "Acer WMI Laptop Extras (EXPERIMENTAL)"
+ depends on EXPERIMENTAL
+ depends on ACPI
+ depends on LEDS_CLASS
+ depends on NEW_LEDS
+ depends on BACKLIGHT_CLASS_DEVICE
+ depends on SERIO_I8042
+ depends on RFKILL
+ select ACPI_WMI
+ ---help---
+ This is a driver for newer Acer (and Wistron) laptops. It adds
+ wireless radio and bluetooth control, and on some laptops,
+ exposes the mail LED and LCD backlight.
+
+ For more information about this driver see
+ <file:Documentation/laptops/acer-wmi.txt>
+
+ If you have an ACPI-WMI compatible Acer/ Wistron laptop, say Y or M
+ here.
+
+config ASUS_LAPTOP
+ tristate "Asus Laptop Extras (EXPERIMENTAL)"
+ depends on ACPI
+ depends on EXPERIMENTAL && !ACPI_ASUS
+ depends on LEDS_CLASS
+ depends on NEW_LEDS
+ depends on BACKLIGHT_CLASS_DEVICE
+ ---help---
+ This is the new Linux driver for Asus laptops. It may also support some
+ MEDION, JVC or VICTOR laptops. It makes all the extra buttons generate
+ standard ACPI events that go through /proc/acpi/events. It also adds
+ support for video output switching, LCD backlight control, Bluetooth and
+ Wlan control, and most importantly, allows you to blink those fancy LEDs.
+
+ For more information and a userspace daemon for handling the extra
+ buttons see <http://acpi4asus.sf.net/>.
+
+ If you have an ACPI-compatible ASUS laptop, say Y or M here.
+
+config FUJITSU_LAPTOP
+ tristate "Fujitsu Laptop Extras"
+ depends on ACPI
+ depends on INPUT
+ depends on BACKLIGHT_CLASS_DEVICE
+ ---help---
+ This is a driver for laptops built by Fujitsu:
+
+ * P2xxx/P5xxx/S6xxx/S7xxx series Lifebooks
+ * Possibly other Fujitsu laptop models
+ * Tested with S6410 and S7020
+
+ It adds support for LCD brightness control and some hotkeys.
+
+ If you have a Fujitsu laptop, say Y or M here.
+
+config FUJITSU_LAPTOP_DEBUG
+ bool "Verbose debug mode for Fujitsu Laptop Extras"
+ depends on FUJITSU_LAPTOP
+ default n
+ ---help---
+ Enables extra debug output from the fujitsu extras driver, at the
+ expense of a slight increase in driver size.
+
+ If you are not sure, say N here.
+
+config TC1100_WMI
+ tristate "HP Compaq TC1100 Tablet WMI Extras (EXPERIMENTAL)"
+ depends on !X86_64
+ depends on EXPERIMENTAL
+ depends on ACPI
+ select ACPI_WMI
+ ---help---
+ This is a driver for the WMI extensions (wireless and bluetooth power
+ control) of the HP Compaq TC1100 tablet.
+
+config HP_WMI
+ tristate "HP WMI extras"
+ depends on ACPI_WMI
+ depends on INPUT
+ depends on RFKILL
+ help
+ Say Y here if you want to support WMI-based hotkeys on HP laptops and
+ to read data from WMI such as docking or ambient light sensor state.
+
+ To compile this driver as a module, choose M here: the module will
+ be called hp-wmi.
+
+config MSI_LAPTOP
+ tristate "MSI Laptop Extras"
+ depends on ACPI
+ depends on BACKLIGHT_CLASS_DEVICE
+ ---help---
+ This is a driver for laptops built by MSI (MICRO-STAR
+ INTERNATIONAL):
+
+ MSI MegaBook S270 (MS-1013)
+ Cytron/TCM/Medion/Tchibo MD96100/SAM2000
+
+ It adds support for Bluetooth, WLAN and LCD brightness control.
+
+ More information about this driver is available at
+ <http://0pointer.de/lennart/tchibo.html>.
+
+ If you have an MSI S270 laptop, say Y or M here.
+
+config PANASONIC_LAPTOP
+ tristate "Panasonic Laptop Extras"
+ depends on INPUT && ACPI
+ depends on BACKLIGHT_CLASS_DEVICE
+ ---help---
+ This driver adds support for access to backlight control and hotkeys
+ on Panasonic Let's Note laptops.
+
+ If you have a Panasonic Let's note laptop (such as the R1(N variant),
+ R2, R3, R5, T2, W2 and Y2 series), say Y.
+
+config COMPAL_LAPTOP
+ tristate "Compal Laptop Extras"
+ depends on ACPI
+ depends on BACKLIGHT_CLASS_DEVICE
+ ---help---
+ This is a driver for laptops built by Compal:
+
+ Compal FL90/IFL90
+ Compal FL91/IFL91
+ Compal FL92/JFL92
+ Compal FT00/IFT00
+
+ It adds support for Bluetooth, WLAN and LCD brightness control.
+
+ If you have an Compal FL9x/IFL9x/FT00 laptop, say Y or M here.
+
+config SONY_LAPTOP
+ tristate "Sony Laptop Extras"
+ depends on ACPI
+ select BACKLIGHT_CLASS_DEVICE
+ depends on INPUT
+ ---help---
+ This mini-driver drives the SNC and SPIC devices present in the ACPI
+ BIOS of the Sony Vaio laptops.
+
+ It gives access to some extra laptop functionalities like Bluetooth,
+ screen brightness control, Fn keys and allows powering on/off some
+ devices.
+
+ Read <file:Documentation/laptops/sony-laptop.txt> for more information.
+
+config SONYPI_COMPAT
+ bool "Sonypi compatibility"
+ depends on SONY_LAPTOP
+ ---help---
+ Build the sonypi driver compatibility code into the sony-laptop driver.
+
+config THINKPAD_ACPI
+ tristate "ThinkPad ACPI Laptop Extras"
+ depends on ACPI
+ select BACKLIGHT_LCD_SUPPORT
+ select BACKLIGHT_CLASS_DEVICE
+ select HWMON
+ select NVRAM
+ select INPUT
+ select NEW_LEDS
+ select LEDS_CLASS
+ select NET
+ select RFKILL
+ ---help---
+ This is a driver for the IBM and Lenovo ThinkPad laptops. It adds
+ support for Fn-Fx key combinations, Bluetooth control, video
+ output switching, ThinkLight control, UltraBay eject and more.
+ For more information about this driver see
+ <file:Documentation/laptops/thinkpad-acpi.txt> and
+ <http://ibm-acpi.sf.net/> .
+
+ This driver was formerly known as ibm-acpi.
+
+ If you have an IBM or Lenovo ThinkPad laptop, say Y or M here.
+
+config THINKPAD_ACPI_DEBUG
+ bool "Verbose debug mode"
+ depends on THINKPAD_ACPI
+ default n
+ ---help---
+ Enables extra debugging information, at the expense of a slightly
+ increase in driver size.
+
+ If you are not sure, say N here.
+
+config THINKPAD_ACPI_DOCK
+ bool "Legacy Docking Station Support"
+ depends on THINKPAD_ACPI
+ depends on ACPI_DOCK=n
+ default n
+ ---help---
+ Allows the thinkpad_acpi driver to handle docking station events.
+ This support was made obsolete by the generic ACPI docking station
+ support (CONFIG_ACPI_DOCK). It will allow locking and removing the
+ laptop from the docking station, but will not properly connect PCI
+ devices.
+
+ If you are not sure, say N here.
+
+config THINKPAD_ACPI_BAY
+ bool "Legacy Removable Bay Support"
+ depends on THINKPAD_ACPI
+ default y
+ ---help---
+ Allows the thinkpad_acpi driver to handle removable bays. It will
+ electrically disable the device in the bay, and also generate
+ notifications when the bay lever is ejected or inserted.
+
+ If you are not sure, say Y here.
+
+config THINKPAD_ACPI_VIDEO
+ bool "Video output control support"
+ depends on THINKPAD_ACPI
+ default y
+ ---help---
+ Allows the thinkpad_acpi driver to provide an interface to control
+ the various video output ports.
+
+ This feature often won't work well, depending on ThinkPad model,
+ display state, video output devices in use, whether there is a X
+ server running, phase of the moon, and the current mood of
+ Schroedinger's cat. If you can use X.org's RandR to control
+ your ThinkPad's video output ports instead of this feature,
+ don't think twice: do it and say N here to save some memory.
+
+ If you are not sure, say Y here.
+
+config THINKPAD_ACPI_HOTKEY_POLL
+ bool "Support NVRAM polling for hot keys"
+ depends on THINKPAD_ACPI
+ default y
+ ---help---
+ Some thinkpad models benefit from NVRAM polling to detect a few of
+ the hot key press events. If you know your ThinkPad model does not
+ need to do NVRAM polling to support any of the hot keys you use,
+ unselecting this option will save about 1kB of memory.
+
+ ThinkPads T40 and newer, R52 and newer, and X31 and newer are
+ unlikely to need NVRAM polling in their latest BIOS versions.
+
+ NVRAM polling can detect at most the following keys: ThinkPad/Access
+ IBM, Zoom, Switch Display (fn+F7), ThinkLight, Volume up/down/mute,
+ Brightness up/down, Display Expand (fn+F8), Hibernate (fn+F12).
+
+ If you are not sure, say Y here. The driver enables polling only if
+ it is strictly necessary to do so.
+
+config INTEL_MENLOW
+ tristate "Thermal Management driver for Intel menlow platform"
+ depends on ACPI_THERMAL
+ select THERMAL
+ ---help---
+ ACPI thermal management enhancement driver on
+ Intel Menlow platform.
+
+ If unsure, say N.
+
+config EEEPC_LAPTOP
+ tristate "Eee PC Hotkey Driver (EXPERIMENTAL)"
+ depends on ACPI
+ depends on EXPERIMENTAL
+ select BACKLIGHT_CLASS_DEVICE
+ select HWMON
+ select RFKILL
+ ---help---
+ This driver supports the Fn-Fx keys on Eee PC laptops.
+ It also adds the ability to switch camera/wlan on/off.
+
+ If you have an Eee PC laptop, say Y or M here.
+
+
+config ACPI_WMI
+ tristate "WMI (EXPERIMENTAL)"
+ depends on ACPI
+ depends on EXPERIMENTAL
+ help
+ This driver adds support for the ACPI-WMI (Windows Management
+ Instrumentation) mapper device (PNP0C14) found on some systems.
+
+ ACPI-WMI is a proprietary extension to ACPI to expose parts of the
+ ACPI firmware to userspace - this is done through various vendor
+ defined methods and data blocks in a PNP0C14 device, which are then
+ made available for userspace to call.
+
+ The implementation of this in Linux currently only exposes this to
+ other kernel space drivers.
+
+ This driver is a required dependency to build the firmware specific
+ drivers needed on many machines, including Acer and HP laptops.
+
+ It is safe to enable this driver even if your DSDT doesn't define
+ any ACPI-WMI devices.
+
+config ACPI_ASUS
+ tristate "ASUS/Medion Laptop Extras"
+ depends on ACPI
+ select BACKLIGHT_CLASS_DEVICE
+ ---help---
+ This driver provides support for extra features of ACPI-compatible
+ ASUS laptops. As some of Medion laptops are made by ASUS, it may also
+ support some Medion laptops (such as 9675 for example). It makes all
+ the extra buttons generate standard ACPI events that go through
+ /proc/acpi/events, and (on some models) adds support for changing the
+ display brightness and output, switching the LCD backlight on and off,
+ and most importantly, allows you to blink those fancy LEDs intended
+ for reporting mail and wireless status.
+
+ Note: display switching code is currently considered EXPERIMENTAL,
+ toying with these values may even lock your machine.
+
+ All settings are changed via /proc/acpi/asus directory entries. Owner
+ and group for these entries can be set with asus_uid and asus_gid
+ parameters.
+
+ More information and a userspace daemon for handling the extra buttons
+ at <http://sourceforge.net/projects/acpi4asus/>.
+
+ If you have an ACPI-compatible ASUS laptop, say Y or M here. This
+ driver is still under development, so if your laptop is unsupported or
+ something works not quite as expected, please use the mailing list
+ available on the above page (acpi4asus-user@lists.sourceforge.net).
+
+ NOTE: This driver is deprecated and will probably be removed soon,
+ use asus-laptop instead.
+
+config ACPI_TOSHIBA
+ tristate "Toshiba Laptop Extras"
+ depends on ACPI
+ depends on INPUT
+ select INPUT_POLLDEV
+ select NET
+ select RFKILL
+ select BACKLIGHT_CLASS_DEVICE
+ ---help---
+ This driver adds support for access to certain system settings
+ on "legacy free" Toshiba laptops. These laptops can be recognized by
+ their lack of a BIOS setup menu and APM support.
+
+ On these machines, all system configuration is handled through the
+ ACPI. This driver is required for access to controls not covered
+ by the general ACPI drivers, such as LCD brightness, video output,
+ etc.
+
+ This driver differs from the non-ACPI Toshiba laptop driver (located
+ under "Processor type and features") in several aspects.
+ Configuration is accessed by reading and writing text files in the
+ /proc tree instead of by program interface to /dev. Furthermore, no
+ power management functions are exposed, as those are handled by the
+ general ACPI drivers.
+
+ More information about this driver is available at
+ <http://memebeam.org/toys/ToshibaAcpiDriver>.
+
+ If you have a legacy free Toshiba laptop (such as the Libretto L1
+ series), say Y.
+endif # X86_PLATFORM_DEVICES
diff --git a/drivers/platform/x86/Makefile b/drivers/platform/x86/Makefile
new file mode 100644
index 0000000..1e9de2a
--- /dev/null
+++ b/drivers/platform/x86/Makefile
@@ -0,0 +1,19 @@
+#
+# Makefile for linux/drivers/platform/x86
+# x86 Platform-Specific Drivers
+#
+obj-$(CONFIG_ASUS_LAPTOP) += asus-laptop.o
+obj-$(CONFIG_EEEPC_LAPTOP) += eeepc-laptop.o
+obj-$(CONFIG_MSI_LAPTOP) += msi-laptop.o
+obj-$(CONFIG_COMPAL_LAPTOP) += compal-laptop.o
+obj-$(CONFIG_ACER_WMI) += acer-wmi.o
+obj-$(CONFIG_HP_WMI) += hp-wmi.o
+obj-$(CONFIG_TC1100_WMI) += tc1100-wmi.o
+obj-$(CONFIG_SONY_LAPTOP) += sony-laptop.o
+obj-$(CONFIG_THINKPAD_ACPI) += thinkpad_acpi.o
+obj-$(CONFIG_FUJITSU_LAPTOP) += fujitsu-laptop.o
+obj-$(CONFIG_PANASONIC_LAPTOP) += panasonic-laptop.o
+obj-$(CONFIG_INTEL_MENLOW) += intel_menlow.o
+obj-$(CONFIG_ACPI_WMI) += wmi.o
+obj-$(CONFIG_ACPI_ASUS) += asus_acpi.o
+obj-$(CONFIG_ACPI_TOSHIBA) += toshiba_acpi.o
diff --git a/drivers/misc/acer-wmi.c b/drivers/platform/x86/acer-wmi.c
similarity index 100%
rename from drivers/misc/acer-wmi.c
rename to drivers/platform/x86/acer-wmi.c
diff --git a/drivers/misc/asus-laptop.c b/drivers/platform/x86/asus-laptop.c
similarity index 100%
rename from drivers/misc/asus-laptop.c
rename to drivers/platform/x86/asus-laptop.c
diff --git a/drivers/acpi/asus_acpi.c b/drivers/platform/x86/asus_acpi.c
similarity index 100%
rename from drivers/acpi/asus_acpi.c
rename to drivers/platform/x86/asus_acpi.c
diff --git a/drivers/misc/compal-laptop.c b/drivers/platform/x86/compal-laptop.c
similarity index 100%
rename from drivers/misc/compal-laptop.c
rename to drivers/platform/x86/compal-laptop.c
diff --git a/drivers/misc/eeepc-laptop.c b/drivers/platform/x86/eeepc-laptop.c
similarity index 100%
rename from drivers/misc/eeepc-laptop.c
rename to drivers/platform/x86/eeepc-laptop.c
diff --git a/drivers/misc/fujitsu-laptop.c b/drivers/platform/x86/fujitsu-laptop.c
similarity index 74%
rename from drivers/misc/fujitsu-laptop.c
rename to drivers/platform/x86/fujitsu-laptop.c
index a7dd3e9..65dc415 100644
--- a/drivers/misc/fujitsu-laptop.c
+++ b/drivers/platform/x86/fujitsu-laptop.c
@@ -3,6 +3,7 @@
/*
Copyright (C) 2007,2008 Jonathan Woithe <jwoithe@physics.adelaide.edu.au>
Copyright (C) 2008 Peter Gruber <nokos@gmx.net>
+ Copyright (C) 2008 Tony Vroon <tony@linx.net>
Based on earlier work:
Copyright (C) 2003 Shane Spencer <shane@bogomip.com>
Adrian Yee <brewt-fujitsu@brewt.org>
@@ -65,8 +66,11 @@
#include <linux/kfifo.h>
#include <linux/video_output.h>
#include <linux/platform_device.h>
+#ifdef CONFIG_LEDS_CLASS
+#include <linux/leds.h>
+#endif
-#define FUJITSU_DRIVER_VERSION "0.4.3"
+#define FUJITSU_DRIVER_VERSION "0.5.0"
#define FUJITSU_LCD_N_LEVELS 8
@@ -83,6 +87,24 @@
#define ACPI_VIDEO_NOTIFY_INC_BRIGHTNESS 0x86
#define ACPI_VIDEO_NOTIFY_DEC_BRIGHTNESS 0x87
+/* FUNC interface - command values */
+#define FUNC_RFKILL 0x1000
+#define FUNC_LEDS 0x1001
+#define FUNC_BUTTONS 0x1002
+#define FUNC_BACKLIGHT 0x1004
+
+/* FUNC interface - responses */
+#define UNSUPPORTED_CMD 0x80000000
+
+#ifdef CONFIG_LEDS_CLASS
+/* FUNC interface - LED control */
+#define FUNC_LED_OFF 0x1
+#define FUNC_LED_ON 0x30001
+#define KEYBOARD_LAMPS 0x100
+#define LOGOLAMP_POWERON 0x2000
+#define LOGOLAMP_ALWAYS 0x4000
+#endif
+
/* Hotkey details */
#define KEY1_CODE 0x410 /* codes for the keys in the GIRB register */
#define KEY2_CODE 0x411
@@ -133,7 +155,6 @@
static struct fujitsu_t *fujitsu;
static int use_alt_lcd_levels = -1;
-static int disable_brightness_keys = -1;
static int disable_brightness_adjust = -1;
/* Device used to access other hotkeys on the laptop */
@@ -145,8 +166,9 @@
struct platform_device *pf_device;
struct kfifo *fifo;
spinlock_t fifo_lock;
-
- unsigned int irb; /* info about the pressed buttons */
+ int rfkill_state;
+ int logolamp_registered;
+ int kblamps_registered;
};
static struct fujitsu_hotkey_t *fujitsu_hotkey;
@@ -154,12 +176,139 @@
static void acpi_fujitsu_hotkey_notify(acpi_handle handle, u32 event,
void *data);
+#ifdef CONFIG_LEDS_CLASS
+static enum led_brightness logolamp_get(struct led_classdev *cdev);
+static void logolamp_set(struct led_classdev *cdev,
+ enum led_brightness brightness);
+
+struct led_classdev logolamp_led = {
+ .name = "fujitsu::logolamp",
+ .brightness_get = logolamp_get,
+ .brightness_set = logolamp_set
+};
+
+static enum led_brightness kblamps_get(struct led_classdev *cdev);
+static void kblamps_set(struct led_classdev *cdev,
+ enum led_brightness brightness);
+
+struct led_classdev kblamps_led = {
+ .name = "fujitsu::kblamps",
+ .brightness_get = kblamps_get,
+ .brightness_set = kblamps_set
+};
+#endif
+
#ifdef CONFIG_FUJITSU_LAPTOP_DEBUG
static u32 dbg_level = 0x03;
#endif
static void acpi_fujitsu_notify(acpi_handle handle, u32 event, void *data);
+/* Fujitsu ACPI interface function */
+
+static int call_fext_func(int cmd, int arg0, int arg1, int arg2)
+{
+ acpi_status status = AE_OK;
+ union acpi_object params[4] = {
+ { .type = ACPI_TYPE_INTEGER },
+ { .type = ACPI_TYPE_INTEGER },
+ { .type = ACPI_TYPE_INTEGER },
+ { .type = ACPI_TYPE_INTEGER }
+ };
+ struct acpi_object_list arg_list = { 4, ¶ms[0] };
+ struct acpi_buffer output;
+ union acpi_object out_obj;
+ acpi_handle handle = NULL;
+
+ status = acpi_get_handle(fujitsu_hotkey->acpi_handle, "FUNC", &handle);
+ if (ACPI_FAILURE(status)) {
+ vdbg_printk(FUJLAPTOP_DBG_ERROR,
+ "FUNC interface is not present\n");
+ return -ENODEV;
+ }
+
+ params[0].integer.value = cmd;
+ params[1].integer.value = arg0;
+ params[2].integer.value = arg1;
+ params[3].integer.value = arg2;
+
+ output.length = sizeof(out_obj);
+ output.pointer = &out_obj;
+
+ status = acpi_evaluate_object(handle, NULL, &arg_list, &output);
+ if (ACPI_FAILURE(status)) {
+ vdbg_printk(FUJLAPTOP_DBG_WARN,
+ "FUNC 0x%x (args 0x%x, 0x%x, 0x%x) call failed\n",
+ cmd, arg0, arg1, arg2);
+ return -ENODEV;
+ }
+
+ if (out_obj.type != ACPI_TYPE_INTEGER) {
+ vdbg_printk(FUJLAPTOP_DBG_WARN,
+ "FUNC 0x%x (args 0x%x, 0x%x, 0x%x) did not "
+ "return an integer\n",
+ cmd, arg0, arg1, arg2);
+ return -ENODEV;
+ }
+
+ vdbg_printk(FUJLAPTOP_DBG_TRACE,
+ "FUNC 0x%x (args 0x%x, 0x%x, 0x%x) returned 0x%x\n",
+ cmd, arg0, arg1, arg2, (int)out_obj.integer.value);
+ return out_obj.integer.value;
+}
+
+#ifdef CONFIG_LEDS_CLASS
+/* LED class callbacks */
+
+static void logolamp_set(struct led_classdev *cdev,
+ enum led_brightness brightness)
+{
+ if (brightness >= LED_FULL) {
+ call_fext_func(FUNC_LEDS, 0x1, LOGOLAMP_POWERON, FUNC_LED_ON);
+ call_fext_func(FUNC_LEDS, 0x1, LOGOLAMP_ALWAYS, FUNC_LED_ON);
+ } else if (brightness >= LED_HALF) {
+ call_fext_func(FUNC_LEDS, 0x1, LOGOLAMP_POWERON, FUNC_LED_ON);
+ call_fext_func(FUNC_LEDS, 0x1, LOGOLAMP_ALWAYS, FUNC_LED_OFF);
+ } else {
+ call_fext_func(FUNC_LEDS, 0x1, LOGOLAMP_POWERON, FUNC_LED_OFF);
+ }
+}
+
+static void kblamps_set(struct led_classdev *cdev,
+ enum led_brightness brightness)
+{
+ if (brightness >= LED_FULL)
+ call_fext_func(FUNC_LEDS, 0x1, KEYBOARD_LAMPS, FUNC_LED_ON);
+ else
+ call_fext_func(FUNC_LEDS, 0x1, KEYBOARD_LAMPS, FUNC_LED_OFF);
+}
+
+static enum led_brightness logolamp_get(struct led_classdev *cdev)
+{
+ enum led_brightness brightness = LED_OFF;
+ int poweron, always;
+
+ poweron = call_fext_func(FUNC_LEDS, 0x2, LOGOLAMP_POWERON, 0x0);
+ if (poweron == FUNC_LED_ON) {
+ brightness = LED_HALF;
+ always = call_fext_func(FUNC_LEDS, 0x2, LOGOLAMP_ALWAYS, 0x0);
+ if (always == FUNC_LED_ON)
+ brightness = LED_FULL;
+ }
+ return brightness;
+}
+
+static enum led_brightness kblamps_get(struct led_classdev *cdev)
+{
+ enum led_brightness brightness = LED_OFF;
+
+ if (call_fext_func(FUNC_LEDS, 0x2, KEYBOARD_LAMPS, 0x0) == FUNC_LED_ON)
+ brightness = LED_FULL;
+
+ return brightness;
+}
+#endif
+
/* Hardware access for LCD brightness control */
static int set_lcd_level(int level)
@@ -263,44 +412,34 @@
return fujitsu->max_brightness;
}
-static int get_lcd_level_alt(void)
-{
- unsigned long long state = 0;
- acpi_status status = AE_OK;
-
- vdbg_printk(FUJLAPTOP_DBG_TRACE, "get lcd level via GBLS\n");
-
- status =
- acpi_evaluate_integer(fujitsu->acpi_handle, "GBLS", NULL, &state);
- if (status < 0)
- return status;
-
- fujitsu->brightness_level = state & 0x0fffffff;
-
- if (state & 0x80000000)
- fujitsu->brightness_changed = 1;
- else
- fujitsu->brightness_changed = 0;
-
- return fujitsu->brightness_level;
-}
-
/* Backlight device stuff */
static int bl_get_brightness(struct backlight_device *b)
{
- if (use_alt_lcd_levels)
- return get_lcd_level_alt();
- else
- return get_lcd_level();
+ return get_lcd_level();
}
static int bl_update_status(struct backlight_device *b)
{
- if (use_alt_lcd_levels)
- return set_lcd_level_alt(b->props.brightness);
+ int ret;
+ if (b->props.power == 4)
+ ret = call_fext_func(FUNC_BACKLIGHT, 0x1, 0x4, 0x3);
else
- return set_lcd_level(b->props.brightness);
+ ret = call_fext_func(FUNC_BACKLIGHT, 0x1, 0x4, 0x0);
+ if (ret != 0)
+ vdbg_printk(FUJLAPTOP_DBG_ERROR,
+ "Unable to adjust backlight power, error code %i\n",
+ ret);
+
+ if (use_alt_lcd_levels)
+ ret = set_lcd_level_alt(b->props.brightness);
+ else
+ ret = set_lcd_level(b->props.brightness);
+ if (ret != 0)
+ vdbg_printk(FUJLAPTOP_DBG_ERROR,
+ "Unable to adjust LCD brightness, error code %i\n",
+ ret);
+ return ret;
}
static struct backlight_ops fujitsubl_ops = {
@@ -344,10 +483,7 @@
int ret;
- if (use_alt_lcd_levels)
- ret = get_lcd_level_alt();
- else
- ret = get_lcd_level();
+ ret = get_lcd_level();
if (ret < 0)
return ret;
@@ -372,36 +508,13 @@
if (ret < 0)
return ret;
- if (use_alt_lcd_levels)
- ret = get_lcd_level_alt();
- else
- ret = get_lcd_level();
+ ret = get_lcd_level();
if (ret < 0)
return ret;
return count;
}
-/* Hardware access for hotkey device */
-
-static int get_irb(void)
-{
- unsigned long long state = 0;
- acpi_status status = AE_OK;
-
- vdbg_printk(FUJLAPTOP_DBG_TRACE, "Get irb\n");
-
- status =
- acpi_evaluate_integer(fujitsu_hotkey->acpi_handle, "GIRB", NULL,
- &state);
- if (status < 0)
- return status;
-
- fujitsu_hotkey->irb = state;
-
- return fujitsu_hotkey->irb;
-}
-
static ssize_t
ignore_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t count)
@@ -409,15 +522,57 @@
return count;
}
+static ssize_t
+show_lid_state(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ if (fujitsu_hotkey->rfkill_state == UNSUPPORTED_CMD)
+ return sprintf(buf, "unknown\n");
+ if (fujitsu_hotkey->rfkill_state & 0x100)
+ return sprintf(buf, "open\n");
+ else
+ return sprintf(buf, "closed\n");
+}
+
+static ssize_t
+show_dock_state(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ if (fujitsu_hotkey->rfkill_state == UNSUPPORTED_CMD)
+ return sprintf(buf, "unknown\n");
+ if (fujitsu_hotkey->rfkill_state & 0x200)
+ return sprintf(buf, "docked\n");
+ else
+ return sprintf(buf, "undocked\n");
+}
+
+static ssize_t
+show_radios_state(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ if (fujitsu_hotkey->rfkill_state == UNSUPPORTED_CMD)
+ return sprintf(buf, "unknown\n");
+ if (fujitsu_hotkey->rfkill_state & 0x20)
+ return sprintf(buf, "on\n");
+ else
+ return sprintf(buf, "killed\n");
+}
+
static DEVICE_ATTR(max_brightness, 0444, show_max_brightness, ignore_store);
static DEVICE_ATTR(brightness_changed, 0444, show_brightness_changed,
ignore_store);
static DEVICE_ATTR(lcd_level, 0644, show_lcd_level, store_lcd_level);
+static DEVICE_ATTR(lid, 0444, show_lid_state, ignore_store);
+static DEVICE_ATTR(dock, 0444, show_dock_state, ignore_store);
+static DEVICE_ATTR(radios, 0444, show_radios_state, ignore_store);
static struct attribute *fujitsupf_attributes[] = {
&dev_attr_brightness_changed.attr,
&dev_attr_max_brightness.attr,
&dev_attr_lcd_level.attr,
+ &dev_attr_lid.attr,
+ &dev_attr_dock.attr,
+ &dev_attr_radios.attr,
NULL
};
@@ -435,24 +590,16 @@
static void dmi_check_cb_common(const struct dmi_system_id *id)
{
acpi_handle handle;
- int have_blnf;
printk(KERN_INFO "fujitsu-laptop: Identified laptop model '%s'.\n",
id->ident);
- have_blnf = ACPI_SUCCESS
- (acpi_get_handle(NULL, "\\_SB.PCI0.GFX0.LCD.BLNF", &handle));
if (use_alt_lcd_levels == -1) {
- vdbg_printk(FUJLAPTOP_DBG_TRACE, "auto-detecting usealt\n");
- use_alt_lcd_levels = 1;
- }
- if (disable_brightness_keys == -1) {
- vdbg_printk(FUJLAPTOP_DBG_TRACE,
- "auto-detecting disable_keys\n");
- disable_brightness_keys = have_blnf ? 1 : 0;
- }
- if (disable_brightness_adjust == -1) {
- vdbg_printk(FUJLAPTOP_DBG_TRACE,
- "auto-detecting disable_adjust\n");
- disable_brightness_adjust = have_blnf ? 0 : 1;
+ if (ACPI_SUCCESS(acpi_get_handle(NULL,
+ "\\_SB.PCI0.LPCB.FJEX.SBL2", &handle)))
+ use_alt_lcd_levels = 1;
+ else
+ use_alt_lcd_levels = 0;
+ vdbg_printk(FUJLAPTOP_DBG_TRACE, "auto-detected usealt as "
+ "%i\n", use_alt_lcd_levels);
}
}
@@ -581,19 +728,14 @@
/* do config (detect defaults) */
use_alt_lcd_levels = use_alt_lcd_levels == 1 ? 1 : 0;
- disable_brightness_keys = disable_brightness_keys == 1 ? 1 : 0;
disable_brightness_adjust = disable_brightness_adjust == 1 ? 1 : 0;
vdbg_printk(FUJLAPTOP_DBG_INFO,
- "config: [alt interface: %d], [key disable: %d], [adjust disable: %d]\n",
- use_alt_lcd_levels, disable_brightness_keys,
- disable_brightness_adjust);
+ "config: [alt interface: %d], [adjust disable: %d]\n",
+ use_alt_lcd_levels, disable_brightness_adjust);
if (get_max_brightness() <= 0)
fujitsu->max_brightness = FUJITSU_LCD_N_LEVELS;
- if (use_alt_lcd_levels)
- get_lcd_level_alt();
- else
- get_lcd_level();
+ get_lcd_level();
return result;
@@ -644,43 +786,23 @@
case ACPI_FUJITSU_NOTIFY_CODE1:
keycode = 0;
oldb = fujitsu->brightness_level;
- get_lcd_level(); /* the alt version always yields changed */
+ get_lcd_level();
newb = fujitsu->brightness_level;
vdbg_printk(FUJLAPTOP_DBG_TRACE,
"brightness button event [%i -> %i (%i)]\n",
oldb, newb, fujitsu->brightness_changed);
- if (oldb == newb && fujitsu->brightness_changed) {
- keycode = 0;
- if (disable_brightness_keys != 1) {
- if (oldb == 0) {
- acpi_bus_generate_proc_event
- (fujitsu->dev,
- ACPI_VIDEO_NOTIFY_DEC_BRIGHTNESS,
- 0);
- keycode = KEY_BRIGHTNESSDOWN;
- } else if (oldb ==
- (fujitsu->max_brightness) - 1) {
- acpi_bus_generate_proc_event
- (fujitsu->dev,
- ACPI_VIDEO_NOTIFY_INC_BRIGHTNESS,
- 0);
- keycode = KEY_BRIGHTNESSUP;
- }
- }
- } else if (oldb < newb) {
+ if (oldb < newb) {
if (disable_brightness_adjust != 1) {
if (use_alt_lcd_levels)
set_lcd_level_alt(newb);
else
set_lcd_level(newb);
}
- if (disable_brightness_keys != 1) {
- acpi_bus_generate_proc_event(fujitsu->dev,
- ACPI_VIDEO_NOTIFY_INC_BRIGHTNESS, 0);
- keycode = KEY_BRIGHTNESSUP;
- }
+ acpi_bus_generate_proc_event(fujitsu->dev,
+ ACPI_VIDEO_NOTIFY_INC_BRIGHTNESS, 0);
+ keycode = KEY_BRIGHTNESSUP;
} else if (oldb > newb) {
if (disable_brightness_adjust != 1) {
if (use_alt_lcd_levels)
@@ -688,13 +810,9 @@
else
set_lcd_level(newb);
}
- if (disable_brightness_keys != 1) {
- acpi_bus_generate_proc_event(fujitsu->dev,
- ACPI_VIDEO_NOTIFY_DEC_BRIGHTNESS, 0);
- keycode = KEY_BRIGHTNESSDOWN;
- }
- } else {
- keycode = KEY_UNKNOWN;
+ acpi_bus_generate_proc_event(fujitsu->dev,
+ ACPI_VIDEO_NOTIFY_DEC_BRIGHTNESS, 0);
+ keycode = KEY_BRIGHTNESSDOWN;
}
break;
default:
@@ -771,7 +889,8 @@
input->id.bustype = BUS_HOST;
input->id.product = 0x06;
input->dev.parent = &device->dev;
- input->evbit[0] = BIT(EV_KEY);
+
+ set_bit(EV_KEY, input->evbit);
set_bit(fujitsu->keycode1, input->keybit);
set_bit(fujitsu->keycode2, input->keybit);
set_bit(fujitsu->keycode3, input->keybit);
@@ -803,10 +922,44 @@
printk(KERN_ERR "_INI Method failed\n");
}
- i = 0; /* Discard hotkey ringbuffer */
- while (get_irb() != 0 && (i++) < MAX_HOTKEY_RINGBUFFER_SIZE) ;
+ i = 0;
+ while (call_fext_func(FUNC_BUTTONS, 0x1, 0x0, 0x0) != 0
+ && (i++) < MAX_HOTKEY_RINGBUFFER_SIZE)
+ ; /* No action, result is discarded */
vdbg_printk(FUJLAPTOP_DBG_INFO, "Discarded %i ringbuffer entries\n", i);
+ fujitsu_hotkey->rfkill_state =
+ call_fext_func(FUNC_RFKILL, 0x4, 0x0, 0x0);
+
+ /* Suspect this is a keymap of the application panel, print it */
+ printk(KERN_INFO "fujitsu-laptop: BTNI: [0x%x]\n",
+ call_fext_func(FUNC_BUTTONS, 0x0, 0x0, 0x0));
+
+ #ifdef CONFIG_LEDS_CLASS
+ if (call_fext_func(FUNC_LEDS, 0x0, 0x0, 0x0) & LOGOLAMP_POWERON) {
+ result = led_classdev_register(&fujitsu->pf_device->dev,
+ &logolamp_led);
+ if (result == 0) {
+ fujitsu_hotkey->logolamp_registered = 1;
+ } else {
+ printk(KERN_ERR "fujitsu-laptop: Could not register "
+ "LED handler for logo lamp, error %i\n", result);
+ }
+ }
+
+ if ((call_fext_func(FUNC_LEDS, 0x0, 0x0, 0x0) & KEYBOARD_LAMPS) &&
+ (call_fext_func(FUNC_BUTTONS, 0x0, 0x0, 0x0) == 0x0)) {
+ result = led_classdev_register(&fujitsu->pf_device->dev,
+ &kblamps_led);
+ if (result == 0) {
+ fujitsu_hotkey->kblamps_registered = 1;
+ } else {
+ printk(KERN_ERR "fujitsu-laptop: Could not register "
+ "LED handler for keyboard lamps, error %i\n", result);
+ }
+ }
+ #endif
+
return result;
end:
@@ -852,16 +1005,15 @@
input = fujitsu_hotkey->input;
- vdbg_printk(FUJLAPTOP_DBG_TRACE, "Hotkey event\n");
+ fujitsu_hotkey->rfkill_state =
+ call_fext_func(FUNC_RFKILL, 0x4, 0x0, 0x0);
switch (event) {
case ACPI_FUJITSU_NOTIFY_CODE1:
i = 0;
- while ((irb = get_irb()) != 0
- && (i++) < MAX_HOTKEY_RINGBUFFER_SIZE) {
- vdbg_printk(FUJLAPTOP_DBG_TRACE, "GIRB result [%x]\n",
- irb);
-
+ while ((irb =
+ call_fext_func(FUNC_BUTTONS, 0x1, 0x0, 0x0)) != 0
+ && (i++) < MAX_HOTKEY_RINGBUFFER_SIZE) {
switch (irb & 0x4ff) {
case KEY1_CODE:
keycode = fujitsu->keycode1;
@@ -1035,6 +1187,15 @@
goto fail_hotkey1;
}
+ /* Sync backlight power status (needs FUJ02E3 device, hence deferred) */
+
+ if (!acpi_video_backlight_support()) {
+ if (call_fext_func(FUNC_BACKLIGHT, 0x2, 0x4, 0x0) == 3)
+ fujitsu->bl_device->props.power = 4;
+ else
+ fujitsu->bl_device->props.power = 0;
+ }
+
printk(KERN_INFO "fujitsu-laptop: driver " FUJITSU_DRIVER_VERSION
" successfully loaded.\n");
@@ -1074,6 +1235,14 @@
static void __exit fujitsu_cleanup(void)
{
+ #ifdef CONFIG_LEDS_CLASS
+ if (fujitsu_hotkey->logolamp_registered != 0)
+ led_classdev_unregister(&logolamp_led);
+
+ if (fujitsu_hotkey->kblamps_registered != 0)
+ led_classdev_unregister(&kblamps_led);
+ #endif
+
sysfs_remove_group(&fujitsu->pf_device->dev.kobj,
&fujitsupf_attribute_group);
platform_device_unregister(fujitsu->pf_device);
@@ -1098,9 +1267,6 @@
module_param(use_alt_lcd_levels, uint, 0644);
MODULE_PARM_DESC(use_alt_lcd_levels,
"Use alternative interface for lcd_levels (needed for Lifebook s6410).");
-module_param(disable_brightness_keys, uint, 0644);
-MODULE_PARM_DESC(disable_brightness_keys,
- "Disable brightness keys (eg. if they are already handled by the generic ACPI_VIDEO device).");
module_param(disable_brightness_adjust, uint, 0644);
MODULE_PARM_DESC(disable_brightness_adjust, "Disable brightness adjustment .");
#ifdef CONFIG_FUJITSU_LAPTOP_DEBUG
@@ -1108,12 +1274,13 @@
MODULE_PARM_DESC(debug, "Sets debug level bit-mask");
#endif
-MODULE_AUTHOR("Jonathan Woithe, Peter Gruber");
+MODULE_AUTHOR("Jonathan Woithe, Peter Gruber, Tony Vroon");
MODULE_DESCRIPTION("Fujitsu laptop extras support");
MODULE_VERSION(FUJITSU_DRIVER_VERSION);
MODULE_LICENSE("GPL");
MODULE_ALIAS("dmi:*:svnFUJITSUSIEMENS:*:pvr:rvnFUJITSU:rnFJNB1D3:*:cvrS6410:*");
+MODULE_ALIAS("dmi:*:svnFUJITSUSIEMENS:*:pvr:rvnFUJITSU:rnFJNB1E6:*:cvrS6420:*");
MODULE_ALIAS("dmi:*:svnFUJITSU:*:pvr:rvnFUJITSU:rnFJNB19C:*:cvrS7020:*");
static struct pnp_device_id pnp_ids[] = {
diff --git a/drivers/misc/hp-wmi.c b/drivers/platform/x86/hp-wmi.c
similarity index 100%
rename from drivers/misc/hp-wmi.c
rename to drivers/platform/x86/hp-wmi.c
diff --git a/drivers/misc/intel_menlow.c b/drivers/platform/x86/intel_menlow.c
similarity index 100%
rename from drivers/misc/intel_menlow.c
rename to drivers/platform/x86/intel_menlow.c
diff --git a/drivers/misc/msi-laptop.c b/drivers/platform/x86/msi-laptop.c
similarity index 100%
rename from drivers/misc/msi-laptop.c
rename to drivers/platform/x86/msi-laptop.c
diff --git a/drivers/misc/panasonic-laptop.c b/drivers/platform/x86/panasonic-laptop.c
similarity index 97%
rename from drivers/misc/panasonic-laptop.c
rename to drivers/platform/x86/panasonic-laptop.c
index 4a1bc64..f30db36 100644
--- a/drivers/misc/panasonic-laptop.c
+++ b/drivers/platform/x86/panasonic-laptop.c
@@ -241,8 +241,6 @@
};
acpi_status status = AE_OK;
- ACPI_FUNCTION_TRACE("acpi_pcc_write_sset");
-
status = acpi_evaluate_object(pcc->handle, METHOD_HKEY_SSET,
¶ms, NULL);
@@ -254,8 +252,6 @@
unsigned long long s;
acpi_status status;
- ACPI_FUNCTION_TRACE("acpi_pcc_get_sqty");
-
status = acpi_evaluate_integer(device->handle, METHOD_HKEY_SQTY,
NULL, &s);
if (ACPI_SUCCESS(status))
@@ -274,8 +270,6 @@
union acpi_object *hkey = NULL;
int i;
- ACPI_FUNCTION_TRACE("acpi_pcc_retrieve_biosdata");
-
status = acpi_evaluate_object(pcc->handle, METHOD_HKEY_SINF, 0,
&buffer);
if (ACPI_FAILURE(status)) {
@@ -501,8 +495,6 @@
int key_code, hkey_num;
unsigned long long result;
- ACPI_FUNCTION_TRACE("acpi_pcc_generate_keyinput");
-
rc = acpi_evaluate_integer(pcc->handle, METHOD_HKEY_QUERY,
NULL, &result);
if (!ACPI_SUCCESS(rc)) {
@@ -538,8 +530,6 @@
{
struct pcc_acpi *pcc = (struct pcc_acpi *) data;
- ACPI_FUNCTION_TRACE("acpi_pcc_hotkey_notify");
-
switch (event) {
case HKEY_NOTIFY:
acpi_pcc_generate_keyinput(pcc);
@@ -554,8 +544,6 @@
{
int i, rc;
- ACPI_FUNCTION_TRACE("acpi_pcc_init_input");
-
pcc->input_dev = input_allocate_device();
if (!pcc->input_dev) {
ACPI_DEBUG_PRINT((ACPI_DB_ERROR,
@@ -597,8 +585,6 @@
struct pcc_acpi *pcc = acpi_driver_data(device);
acpi_status status = AE_OK;
- ACPI_FUNCTION_TRACE("acpi_pcc_hotkey_resume");
-
if (device == NULL || pcc == NULL)
return -EINVAL;
@@ -616,8 +602,6 @@
struct pcc_acpi *pcc;
int num_sifr, result;
- ACPI_FUNCTION_TRACE("acpi_pcc_hotkey_add");
-
if (!device)
return -EINVAL;
@@ -714,8 +698,6 @@
{
int result = 0;
- ACPI_FUNCTION_TRACE("acpi_pcc_init");
-
if (acpi_disabled)
return -ENODEV;
@@ -733,8 +715,6 @@
{
struct pcc_acpi *pcc = acpi_driver_data(device);
- ACPI_FUNCTION_TRACE("acpi_pcc_hotkey_remove");
-
if (!device || !pcc)
return -EINVAL;
@@ -757,8 +737,6 @@
static void __exit acpi_pcc_exit(void)
{
- ACPI_FUNCTION_TRACE("acpi_pcc_exit");
-
acpi_bus_unregister_driver(&acpi_pcc_driver);
}
diff --git a/drivers/misc/sony-laptop.c b/drivers/platform/x86/sony-laptop.c
similarity index 99%
rename from drivers/misc/sony-laptop.c
rename to drivers/platform/x86/sony-laptop.c
index 571b211..537959d 100644
--- a/drivers/misc/sony-laptop.c
+++ b/drivers/platform/x86/sony-laptop.c
@@ -935,14 +935,17 @@
static acpi_status sony_walk_callback(acpi_handle handle, u32 level,
void *context, void **return_value)
{
- struct acpi_namespace_node *node;
- union acpi_operand_object *operand;
+ struct acpi_device_info *info;
+ struct acpi_buffer buffer = {ACPI_ALLOCATE_BUFFER, NULL};
- node = (struct acpi_namespace_node *)handle;
- operand = (union acpi_operand_object *)node->object;
+ if (ACPI_SUCCESS(acpi_get_object_info(handle, &buffer))) {
+ info = buffer.pointer;
- printk(KERN_WARNING DRV_PFX "method: name: %4.4s, args %X\n", node->name.ascii,
- (u32) operand->method.param_count);
+ printk(KERN_WARNING DRV_PFX "method: name: %4.4s, args %X\n",
+ (char *)&info->name, info->param_count);
+
+ kfree(buffer.pointer);
+ }
return AE_OK;
}
diff --git a/drivers/misc/tc1100-wmi.c b/drivers/platform/x86/tc1100-wmi.c
similarity index 99%
rename from drivers/misc/tc1100-wmi.c
rename to drivers/platform/x86/tc1100-wmi.c
index f25e4c9..b4a4aa9 100644
--- a/drivers/misc/tc1100-wmi.c
+++ b/drivers/platform/x86/tc1100-wmi.c
@@ -30,7 +30,6 @@
#include <linux/init.h>
#include <linux/types.h>
#include <acpi/acpi.h>
-#include <acpi/actypes.h>
#include <acpi/acpi_bus.h>
#include <acpi/acpi_drivers.h>
#include <linux/platform_device.h>
diff --git a/drivers/misc/thinkpad_acpi.c b/drivers/platform/x86/thinkpad_acpi.c
similarity index 99%
rename from drivers/misc/thinkpad_acpi.c
rename to drivers/platform/x86/thinkpad_acpi.c
index 899766e..3478453 100644
--- a/drivers/misc/thinkpad_acpi.c
+++ b/drivers/platform/x86/thinkpad_acpi.c
@@ -76,7 +76,6 @@
#include <linux/workqueue.h>
#include <acpi/acpi_drivers.h>
-#include <acpi/acnamesp.h>
#include <linux/pci_ids.h>
diff --git a/drivers/acpi/toshiba_acpi.c b/drivers/platform/x86/toshiba_acpi.c
similarity index 100%
rename from drivers/acpi/toshiba_acpi.c
rename to drivers/platform/x86/toshiba_acpi.c
diff --git a/drivers/acpi/wmi.c b/drivers/platform/x86/wmi.c
similarity index 100%
rename from drivers/acpi/wmi.c
rename to drivers/platform/x86/wmi.c
diff --git a/drivers/pnp/pnpacpi/core.c b/drivers/pnp/pnpacpi/core.c
index 383e47c..2834846 100644
--- a/drivers/pnp/pnpacpi/core.c
+++ b/drivers/pnp/pnpacpi/core.c
@@ -23,7 +23,6 @@
#include <linux/pnp.h>
#include <linux/mod_devicetable.h>
#include <acpi/acpi_bus.h>
-#include <acpi/actypes.h>
#include "../base.h"
#include "pnpacpi.h"
diff --git a/drivers/regulator/core.c b/drivers/regulator/core.c
index 02a7744..f511a40 100644
--- a/drivers/regulator/core.c
+++ b/drivers/regulator/core.c
@@ -29,7 +29,7 @@
static LIST_HEAD(regulator_list);
static LIST_HEAD(regulator_map_list);
-/**
+/*
* struct regulator_dev
*
* Voltage / Current regulator class device. One for each regulator.
@@ -56,7 +56,7 @@
void *reg_data; /* regulator_dev data */
};
-/**
+/*
* struct regulator_map
*
* Used to provide symbolic supply names to devices.
@@ -79,7 +79,7 @@
int uA_load;
int min_uV;
int max_uV;
- int enabled; /* client has called enabled */
+ int enabled; /* count of client enables */
char *supply_name;
struct device_attribute dev_attr;
struct regulator_dev *rdev;
@@ -174,6 +174,16 @@
/* operating mode constraint check */
static int regulator_check_mode(struct regulator_dev *rdev, int mode)
{
+ switch (mode) {
+ case REGULATOR_MODE_FAST:
+ case REGULATOR_MODE_NORMAL:
+ case REGULATOR_MODE_IDLE:
+ case REGULATOR_MODE_STANDBY:
+ break;
+ default:
+ return -EINVAL;
+ }
+
if (!rdev->constraints) {
printk(KERN_ERR "%s: no constraints for %s\n", __func__,
rdev->desc->name);
@@ -232,6 +242,7 @@
return ret;
}
+static DEVICE_ATTR(microvolts, 0444, regulator_uV_show, NULL);
static ssize_t regulator_uA_show(struct device *dev,
struct device_attribute *attr, char *buf)
@@ -240,6 +251,7 @@
return sprintf(buf, "%d\n", _regulator_get_current_limit(rdev));
}
+static DEVICE_ATTR(microamps, 0444, regulator_uA_show, NULL);
static ssize_t regulator_name_show(struct device *dev,
struct device_attribute *attr, char *buf)
@@ -257,12 +269,8 @@
return sprintf(buf, "%s\n", name);
}
-static ssize_t regulator_opmode_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+static ssize_t regulator_print_opmode(char *buf, int mode)
{
- struct regulator_dev *rdev = dev_get_drvdata(dev);
- int mode = _regulator_get_mode(rdev);
-
switch (mode) {
case REGULATOR_MODE_FAST:
return sprintf(buf, "fast\n");
@@ -276,12 +284,17 @@
return sprintf(buf, "unknown\n");
}
-static ssize_t regulator_state_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+static ssize_t regulator_opmode_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
{
struct regulator_dev *rdev = dev_get_drvdata(dev);
- int state = _regulator_is_enabled(rdev);
+ return regulator_print_opmode(buf, _regulator_get_mode(rdev));
+}
+static DEVICE_ATTR(opmode, 0444, regulator_opmode_show, NULL);
+
+static ssize_t regulator_print_state(char *buf, int state)
+{
if (state > 0)
return sprintf(buf, "enabled\n");
else if (state == 0)
@@ -290,6 +303,15 @@
return sprintf(buf, "unknown\n");
}
+static ssize_t regulator_state_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct regulator_dev *rdev = dev_get_drvdata(dev);
+
+ return regulator_print_state(buf, _regulator_is_enabled(rdev));
+}
+static DEVICE_ATTR(state, 0444, regulator_state_show, NULL);
+
static ssize_t regulator_min_uA_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
@@ -300,6 +322,7 @@
return sprintf(buf, "%d\n", rdev->constraints->min_uA);
}
+static DEVICE_ATTR(min_microamps, 0444, regulator_min_uA_show, NULL);
static ssize_t regulator_max_uA_show(struct device *dev,
struct device_attribute *attr, char *buf)
@@ -311,6 +334,7 @@
return sprintf(buf, "%d\n", rdev->constraints->max_uA);
}
+static DEVICE_ATTR(max_microamps, 0444, regulator_max_uA_show, NULL);
static ssize_t regulator_min_uV_show(struct device *dev,
struct device_attribute *attr, char *buf)
@@ -322,6 +346,7 @@
return sprintf(buf, "%d\n", rdev->constraints->min_uV);
}
+static DEVICE_ATTR(min_microvolts, 0444, regulator_min_uV_show, NULL);
static ssize_t regulator_max_uV_show(struct device *dev,
struct device_attribute *attr, char *buf)
@@ -333,6 +358,7 @@
return sprintf(buf, "%d\n", rdev->constraints->max_uV);
}
+static DEVICE_ATTR(max_microvolts, 0444, regulator_max_uV_show, NULL);
static ssize_t regulator_total_uA_show(struct device *dev,
struct device_attribute *attr, char *buf)
@@ -347,6 +373,7 @@
mutex_unlock(&rdev->mutex);
return sprintf(buf, "%d\n", uA);
}
+static DEVICE_ATTR(requested_microamps, 0444, regulator_total_uA_show, NULL);
static ssize_t regulator_num_users_show(struct device *dev,
struct device_attribute *attr, char *buf)
@@ -374,153 +401,106 @@
{
struct regulator_dev *rdev = dev_get_drvdata(dev);
- if (!rdev->constraints)
- return sprintf(buf, "not defined\n");
return sprintf(buf, "%d\n", rdev->constraints->state_mem.uV);
}
+static DEVICE_ATTR(suspend_mem_microvolts, 0444,
+ regulator_suspend_mem_uV_show, NULL);
static ssize_t regulator_suspend_disk_uV_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct regulator_dev *rdev = dev_get_drvdata(dev);
- if (!rdev->constraints)
- return sprintf(buf, "not defined\n");
return sprintf(buf, "%d\n", rdev->constraints->state_disk.uV);
}
+static DEVICE_ATTR(suspend_disk_microvolts, 0444,
+ regulator_suspend_disk_uV_show, NULL);
static ssize_t regulator_suspend_standby_uV_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct regulator_dev *rdev = dev_get_drvdata(dev);
- if (!rdev->constraints)
- return sprintf(buf, "not defined\n");
return sprintf(buf, "%d\n", rdev->constraints->state_standby.uV);
}
-
-static ssize_t suspend_opmode_show(struct regulator_dev *rdev,
- unsigned int mode, char *buf)
-{
- switch (mode) {
- case REGULATOR_MODE_FAST:
- return sprintf(buf, "fast\n");
- case REGULATOR_MODE_NORMAL:
- return sprintf(buf, "normal\n");
- case REGULATOR_MODE_IDLE:
- return sprintf(buf, "idle\n");
- case REGULATOR_MODE_STANDBY:
- return sprintf(buf, "standby\n");
- }
- return sprintf(buf, "unknown\n");
-}
+static DEVICE_ATTR(suspend_standby_microvolts, 0444,
+ regulator_suspend_standby_uV_show, NULL);
static ssize_t regulator_suspend_mem_mode_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct regulator_dev *rdev = dev_get_drvdata(dev);
- if (!rdev->constraints)
- return sprintf(buf, "not defined\n");
- return suspend_opmode_show(rdev,
- rdev->constraints->state_mem.mode, buf);
+ return regulator_print_opmode(buf,
+ rdev->constraints->state_mem.mode);
}
+static DEVICE_ATTR(suspend_mem_mode, 0444,
+ regulator_suspend_mem_mode_show, NULL);
static ssize_t regulator_suspend_disk_mode_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct regulator_dev *rdev = dev_get_drvdata(dev);
- if (!rdev->constraints)
- return sprintf(buf, "not defined\n");
- return suspend_opmode_show(rdev,
- rdev->constraints->state_disk.mode, buf);
+ return regulator_print_opmode(buf,
+ rdev->constraints->state_disk.mode);
}
+static DEVICE_ATTR(suspend_disk_mode, 0444,
+ regulator_suspend_disk_mode_show, NULL);
static ssize_t regulator_suspend_standby_mode_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct regulator_dev *rdev = dev_get_drvdata(dev);
- if (!rdev->constraints)
- return sprintf(buf, "not defined\n");
- return suspend_opmode_show(rdev,
- rdev->constraints->state_standby.mode, buf);
+ return regulator_print_opmode(buf,
+ rdev->constraints->state_standby.mode);
}
+static DEVICE_ATTR(suspend_standby_mode, 0444,
+ regulator_suspend_standby_mode_show, NULL);
static ssize_t regulator_suspend_mem_state_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct regulator_dev *rdev = dev_get_drvdata(dev);
- if (!rdev->constraints)
- return sprintf(buf, "not defined\n");
-
- if (rdev->constraints->state_mem.enabled)
- return sprintf(buf, "enabled\n");
- else
- return sprintf(buf, "disabled\n");
+ return regulator_print_state(buf,
+ rdev->constraints->state_mem.enabled);
}
+static DEVICE_ATTR(suspend_mem_state, 0444,
+ regulator_suspend_mem_state_show, NULL);
static ssize_t regulator_suspend_disk_state_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct regulator_dev *rdev = dev_get_drvdata(dev);
- if (!rdev->constraints)
- return sprintf(buf, "not defined\n");
-
- if (rdev->constraints->state_disk.enabled)
- return sprintf(buf, "enabled\n");
- else
- return sprintf(buf, "disabled\n");
+ return regulator_print_state(buf,
+ rdev->constraints->state_disk.enabled);
}
+static DEVICE_ATTR(suspend_disk_state, 0444,
+ regulator_suspend_disk_state_show, NULL);
static ssize_t regulator_suspend_standby_state_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct regulator_dev *rdev = dev_get_drvdata(dev);
- if (!rdev->constraints)
- return sprintf(buf, "not defined\n");
-
- if (rdev->constraints->state_standby.enabled)
- return sprintf(buf, "enabled\n");
- else
- return sprintf(buf, "disabled\n");
+ return regulator_print_state(buf,
+ rdev->constraints->state_standby.enabled);
}
+static DEVICE_ATTR(suspend_standby_state, 0444,
+ regulator_suspend_standby_state_show, NULL);
+
+/*
+ * These are the only attributes are present for all regulators.
+ * Other attributes are a function of regulator functionality.
+ */
static struct device_attribute regulator_dev_attrs[] = {
__ATTR(name, 0444, regulator_name_show, NULL),
- __ATTR(microvolts, 0444, regulator_uV_show, NULL),
- __ATTR(microamps, 0444, regulator_uA_show, NULL),
- __ATTR(opmode, 0444, regulator_opmode_show, NULL),
- __ATTR(state, 0444, regulator_state_show, NULL),
- __ATTR(min_microvolts, 0444, regulator_min_uV_show, NULL),
- __ATTR(min_microamps, 0444, regulator_min_uA_show, NULL),
- __ATTR(max_microvolts, 0444, regulator_max_uV_show, NULL),
- __ATTR(max_microamps, 0444, regulator_max_uA_show, NULL),
- __ATTR(requested_microamps, 0444, regulator_total_uA_show, NULL),
__ATTR(num_users, 0444, regulator_num_users_show, NULL),
__ATTR(type, 0444, regulator_type_show, NULL),
- __ATTR(suspend_mem_microvolts, 0444,
- regulator_suspend_mem_uV_show, NULL),
- __ATTR(suspend_disk_microvolts, 0444,
- regulator_suspend_disk_uV_show, NULL),
- __ATTR(suspend_standby_microvolts, 0444,
- regulator_suspend_standby_uV_show, NULL),
- __ATTR(suspend_mem_mode, 0444,
- regulator_suspend_mem_mode_show, NULL),
- __ATTR(suspend_disk_mode, 0444,
- regulator_suspend_disk_mode_show, NULL),
- __ATTR(suspend_standby_mode, 0444,
- regulator_suspend_standby_mode_show, NULL),
- __ATTR(suspend_mem_state, 0444,
- regulator_suspend_mem_state_show, NULL),
- __ATTR(suspend_disk_state, 0444,
- regulator_suspend_disk_state_show, NULL),
- __ATTR(suspend_standby_state, 0444,
- regulator_suspend_standby_state_show, NULL),
__ATTR_NULL,
};
@@ -675,7 +655,8 @@
/**
* set_machine_constraints - sets regulator constraints
- * @regulator: regulator source
+ * @rdev: regulator source
+ * @constraints: constraints to apply
*
* Allows platform initialisation code to define and constrain
* regulator circuits e.g. valid voltage/current ranges, etc. NOTE:
@@ -750,8 +731,8 @@
/**
* set_supply - set regulator supply regulator
- * @regulator: regulator name
- * @supply: supply regulator name
+ * @rdev: regulator name
+ * @supply_rdev: supply regulator name
*
* Called by platform initialisation code to set the supply regulator for this
* regulator. This ensures that a regulators supply will also be enabled by the
@@ -778,9 +759,9 @@
/**
* set_consumer_device_supply: Bind a regulator to a symbolic supply
- * @regulator: regulator source
- * @dev: device the supply applies to
- * @supply: symbolic name for supply
+ * @rdev: regulator source
+ * @consumer_dev: device the supply applies to
+ * @supply: symbolic name for supply
*
* Allows platform initialisation code to map physical regulator
* sources to symbolic names for supplies for use by devices. Devices
@@ -795,6 +776,20 @@
if (supply == NULL)
return -EINVAL;
+ list_for_each_entry(node, ®ulator_map_list, list) {
+ if (consumer_dev != node->dev)
+ continue;
+ if (strcmp(node->supply, supply) != 0)
+ continue;
+
+ dev_dbg(consumer_dev, "%s/%s is '%s' supply; fail %s/%s\n",
+ dev_name(&node->regulator->dev),
+ node->regulator->desc->name,
+ supply,
+ dev_name(&rdev->dev), rdev->desc->name);
+ return -EBUSY;
+ }
+
node = kmalloc(sizeof(struct regulator_map), GFP_KERNEL);
if (node == NULL)
return -ENOMEM;
@@ -963,16 +958,13 @@
if (regulator == NULL || IS_ERR(regulator))
return;
- if (regulator->enabled) {
- printk(KERN_WARNING "Releasing supply %s while enabled\n",
- regulator->supply_name);
- WARN_ON(regulator->enabled);
- regulator_disable(regulator);
- }
-
mutex_lock(®ulator_list_mutex);
rdev = regulator->rdev;
+ if (WARN(regulator->enabled, "Releasing supply %s while enabled\n",
+ regulator->supply_name))
+ _regulator_disable(rdev);
+
/* remove any sysfs entries */
if (regulator->dev) {
sysfs_remove_link(&rdev->dev.kobj, regulator->supply_name);
@@ -1034,29 +1026,26 @@
* regulator_enable - enable regulator output
* @regulator: regulator source
*
- * Enable the regulator output at the predefined voltage or current value.
+ * Request that the regulator be enabled with the regulator output at
+ * the predefined voltage or current value. Calls to regulator_enable()
+ * must be balanced with calls to regulator_disable().
+ *
* NOTE: the output value can be set by other drivers, boot loader or may be
* hardwired in the regulator.
- * NOTE: calls to regulator_enable() must be balanced with calls to
- * regulator_disable().
*/
int regulator_enable(struct regulator *regulator)
{
- int ret;
+ struct regulator_dev *rdev = regulator->rdev;
+ int ret = 0;
- if (regulator->enabled) {
- printk(KERN_CRIT "Regulator %s already enabled\n",
- regulator->supply_name);
- WARN_ON(regulator->enabled);
- return 0;
- }
-
- mutex_lock(®ulator->rdev->mutex);
- regulator->enabled = 1;
- ret = _regulator_enable(regulator->rdev);
- if (ret != 0)
- regulator->enabled = 0;
- mutex_unlock(®ulator->rdev->mutex);
+ mutex_lock(&rdev->mutex);
+ if (regulator->enabled == 0)
+ ret = _regulator_enable(rdev);
+ else if (regulator->enabled < 0)
+ ret = -EIO;
+ if (ret == 0)
+ regulator->enabled++;
+ mutex_unlock(&rdev->mutex);
return ret;
}
EXPORT_SYMBOL_GPL(regulator_enable);
@@ -1100,27 +1089,31 @@
* regulator_disable - disable regulator output
* @regulator: regulator source
*
- * Disable the regulator output voltage or current.
- * NOTE: this will only disable the regulator output if no other consumer
- * devices have it enabled.
- * NOTE: calls to regulator_enable() must be balanced with calls to
+ * Disable the regulator output voltage or current. Calls to
+ * regulator_enable() must be balanced with calls to
* regulator_disable().
+ *
+ * NOTE: this will only disable the regulator output if no other consumer
+ * devices have it enabled, the regulator device supports disabling and
+ * machine constraints permit this operation.
*/
int regulator_disable(struct regulator *regulator)
{
- int ret;
+ struct regulator_dev *rdev = regulator->rdev;
+ int ret = 0;
- if (!regulator->enabled) {
- printk(KERN_ERR "%s: not in use by this consumer\n",
- __func__);
- return 0;
- }
-
- mutex_lock(®ulator->rdev->mutex);
- regulator->enabled = 0;
- regulator->uA_load = 0;
- ret = _regulator_disable(regulator->rdev);
- mutex_unlock(®ulator->rdev->mutex);
+ mutex_lock(&rdev->mutex);
+ if (regulator->enabled == 1) {
+ ret = _regulator_disable(rdev);
+ if (ret == 0)
+ regulator->uA_load = 0;
+ } else if (WARN(regulator->enabled <= 0,
+ "unbalanced disables for supply %s\n",
+ regulator->supply_name))
+ ret = -EIO;
+ if (ret == 0)
+ regulator->enabled--;
+ mutex_unlock(&rdev->mutex);
return ret;
}
EXPORT_SYMBOL_GPL(regulator_disable);
@@ -1196,7 +1189,13 @@
* regulator_is_enabled - is the regulator output enabled
* @regulator: regulator source
*
- * Returns zero for disabled otherwise return number of enable requests.
+ * Returns positive if the regulator driver backing the source/client
+ * has requested that the device be enabled, zero if it hasn't, else a
+ * negative errno code.
+ *
+ * Note that the device backing this regulator handle can have multiple
+ * users, so it might be enabled even if regulator_enable() was never
+ * called for this particular source.
*/
int regulator_is_enabled(struct regulator *regulator)
{
@@ -1219,7 +1218,7 @@
*
* NOTE: If the regulator is shared between several devices then the lowest
* request voltage that meets the system constraints will be used.
- * NOTE: Regulator system constraints must be set for this regulator before
+ * Regulator system constraints must be set for this regulator before
* calling this function otherwise this call will fail.
*/
int regulator_set_voltage(struct regulator *regulator, int min_uV, int max_uV)
@@ -1493,7 +1492,8 @@
mode = rdev->desc->ops->get_optimum_mode(rdev,
input_uV, output_uV,
total_uA_load);
- if (ret <= 0) {
+ ret = regulator_check_mode(rdev, mode);
+ if (ret < 0) {
printk(KERN_ERR "%s: failed to get optimum mode for %s @"
" %d uA %d -> %d uV\n", __func__, rdev->desc->name,
total_uA_load, input_uV, output_uV);
@@ -1501,7 +1501,7 @@
}
ret = rdev->desc->ops->set_mode(rdev, mode);
- if (ret <= 0) {
+ if (ret < 0) {
printk(KERN_ERR "%s: failed to set optimum mode %x for %s\n",
__func__, mode, rdev->desc->name);
goto out;
@@ -1516,7 +1516,7 @@
/**
* regulator_register_notifier - register regulator event notifier
* @regulator: regulator source
- * @notifier_block: notifier block
+ * @nb: notifier block
*
* Register notifier block to receive regulator events.
*/
@@ -1531,7 +1531,7 @@
/**
* regulator_unregister_notifier - unregister regulator event notifier
* @regulator: regulator source
- * @notifier_block: notifier block
+ * @nb: notifier block
*
* Unregister regulator event notifier block.
*/
@@ -1697,9 +1697,9 @@
/**
* regulator_notifier_call_chain - call regulator event notifier
- * @regulator: regulator source
+ * @rdev: regulator source
* @event: notifier block
- * @data:
+ * @data: callback-specific data.
*
* Called by regulator drivers to notify clients a regulator event has
* occurred. We also notify regulator clients downstream.
@@ -1713,10 +1713,122 @@
}
EXPORT_SYMBOL_GPL(regulator_notifier_call_chain);
+/*
+ * To avoid cluttering sysfs (and memory) with useless state, only
+ * create attributes that can be meaningfully displayed.
+ */
+static int add_regulator_attributes(struct regulator_dev *rdev)
+{
+ struct device *dev = &rdev->dev;
+ struct regulator_ops *ops = rdev->desc->ops;
+ int status = 0;
+
+ /* some attributes need specific methods to be displayed */
+ if (ops->get_voltage) {
+ status = device_create_file(dev, &dev_attr_microvolts);
+ if (status < 0)
+ return status;
+ }
+ if (ops->get_current_limit) {
+ status = device_create_file(dev, &dev_attr_microamps);
+ if (status < 0)
+ return status;
+ }
+ if (ops->get_mode) {
+ status = device_create_file(dev, &dev_attr_opmode);
+ if (status < 0)
+ return status;
+ }
+ if (ops->is_enabled) {
+ status = device_create_file(dev, &dev_attr_state);
+ if (status < 0)
+ return status;
+ }
+
+ /* some attributes are type-specific */
+ if (rdev->desc->type == REGULATOR_CURRENT) {
+ status = device_create_file(dev, &dev_attr_requested_microamps);
+ if (status < 0)
+ return status;
+ }
+
+ /* all the other attributes exist to support constraints;
+ * don't show them if there are no constraints, or if the
+ * relevant supporting methods are missing.
+ */
+ if (!rdev->constraints)
+ return status;
+
+ /* constraints need specific supporting methods */
+ if (ops->set_voltage) {
+ status = device_create_file(dev, &dev_attr_min_microvolts);
+ if (status < 0)
+ return status;
+ status = device_create_file(dev, &dev_attr_max_microvolts);
+ if (status < 0)
+ return status;
+ }
+ if (ops->set_current_limit) {
+ status = device_create_file(dev, &dev_attr_min_microamps);
+ if (status < 0)
+ return status;
+ status = device_create_file(dev, &dev_attr_max_microamps);
+ if (status < 0)
+ return status;
+ }
+
+ /* suspend mode constraints need multiple supporting methods */
+ if (!(ops->set_suspend_enable && ops->set_suspend_disable))
+ return status;
+
+ status = device_create_file(dev, &dev_attr_suspend_standby_state);
+ if (status < 0)
+ return status;
+ status = device_create_file(dev, &dev_attr_suspend_mem_state);
+ if (status < 0)
+ return status;
+ status = device_create_file(dev, &dev_attr_suspend_disk_state);
+ if (status < 0)
+ return status;
+
+ if (ops->set_suspend_voltage) {
+ status = device_create_file(dev,
+ &dev_attr_suspend_standby_microvolts);
+ if (status < 0)
+ return status;
+ status = device_create_file(dev,
+ &dev_attr_suspend_mem_microvolts);
+ if (status < 0)
+ return status;
+ status = device_create_file(dev,
+ &dev_attr_suspend_disk_microvolts);
+ if (status < 0)
+ return status;
+ }
+
+ if (ops->set_suspend_mode) {
+ status = device_create_file(dev,
+ &dev_attr_suspend_standby_mode);
+ if (status < 0)
+ return status;
+ status = device_create_file(dev,
+ &dev_attr_suspend_mem_mode);
+ if (status < 0)
+ return status;
+ status = device_create_file(dev,
+ &dev_attr_suspend_disk_mode);
+ if (status < 0)
+ return status;
+ }
+
+ return status;
+}
+
/**
* regulator_register - register regulator
- * @regulator: regulator source
- * @reg_data: private regulator data
+ * @regulator_desc: regulator to register
+ * @dev: struct device for the regulator
+ * @driver_data: private regulator data
*
* Called by regulator drivers to register a regulator.
* Returns 0 on success.
@@ -1761,45 +1873,37 @@
/* preform any regulator specific init */
if (init_data->regulator_init) {
ret = init_data->regulator_init(rdev->reg_data);
- if (ret < 0) {
- kfree(rdev);
- rdev = ERR_PTR(ret);
- goto out;
- }
- }
-
- /* set regulator constraints */
- ret = set_machine_constraints(rdev, &init_data->constraints);
- if (ret < 0) {
- kfree(rdev);
- rdev = ERR_PTR(ret);
- goto out;
+ if (ret < 0)
+ goto clean;
}
/* register with sysfs */
rdev->dev.class = ®ulator_class;
rdev->dev.parent = dev;
- snprintf(rdev->dev.bus_id, sizeof(rdev->dev.bus_id),
- "regulator.%d", atomic_inc_return(®ulator_no) - 1);
+ dev_set_name(&rdev->dev, "regulator.%d",
+ atomic_inc_return(®ulator_no) - 1);
ret = device_register(&rdev->dev);
- if (ret != 0) {
- kfree(rdev);
- rdev = ERR_PTR(ret);
- goto out;
- }
+ if (ret != 0)
+ goto clean;
dev_set_drvdata(&rdev->dev, rdev);
+ /* set regulator constraints */
+ ret = set_machine_constraints(rdev, &init_data->constraints);
+ if (ret < 0)
+ goto scrub;
+
+ /* add attributes supported by this regulator */
+ ret = add_regulator_attributes(rdev);
+ if (ret < 0)
+ goto scrub;
+
/* set supply regulator if it exists */
if (init_data->supply_regulator_dev) {
ret = set_supply(rdev,
dev_get_drvdata(init_data->supply_regulator_dev));
- if (ret < 0) {
- device_unregister(&rdev->dev);
- kfree(rdev);
- rdev = ERR_PTR(ret);
- goto out;
- }
+ if (ret < 0)
+ goto scrub;
}
/* add consumers devices */
@@ -1811,10 +1915,7 @@
for (--i; i >= 0; i--)
unset_consumer_device_supply(rdev,
init_data->consumer_supplies[i].dev);
- device_unregister(&rdev->dev);
- kfree(rdev);
- rdev = ERR_PTR(ret);
- goto out;
+ goto scrub;
}
}
@@ -1822,12 +1923,19 @@
out:
mutex_unlock(®ulator_list_mutex);
return rdev;
+
+scrub:
+ device_unregister(&rdev->dev);
+clean:
+ kfree(rdev);
+ rdev = ERR_PTR(ret);
+ goto out;
}
EXPORT_SYMBOL_GPL(regulator_register);
/**
* regulator_unregister - unregister regulator
- * @regulator: regulator source
+ * @rdev: regulator to unregister
*
* Called by regulator drivers to unregister a regulator.
*/
@@ -1846,7 +1954,7 @@
EXPORT_SYMBOL_GPL(regulator_unregister);
/**
- * regulator_suspend_prepare: prepare regulators for system wide suspend
+ * regulator_suspend_prepare - prepare regulators for system wide suspend
* @state: system suspend state
*
* Configure each regulator with it's suspend operating parameters for state.
@@ -1882,7 +1990,7 @@
/**
* rdev_get_drvdata - get rdev regulator driver data
- * @regulator: regulator
+ * @rdev: regulator
*
* Get rdev regulator driver private data. This call can be used in the
* regulator driver context.
@@ -1919,7 +2027,7 @@
/**
* regulator_get_id - get regulator ID
- * @regulator: regulator
+ * @rdev: regulator
*/
int rdev_get_id(struct regulator_dev *rdev)
{
diff --git a/drivers/regulator/da903x.c b/drivers/regulator/da903x.c
index 773b29c..fe77730 100644
--- a/drivers/regulator/da903x.c
+++ b/drivers/regulator/da903x.c
@@ -102,7 +102,7 @@
uint8_t val, mask;
if (check_range(info, min_uV, max_uV)) {
- pr_err("invalid voltage range (%d, %d) uV", min_uV, max_uV);
+ pr_err("invalid voltage range (%d, %d) uV\n", min_uV, max_uV);
return -EINVAL;
}
@@ -159,7 +159,7 @@
if (ret)
return ret;
- return reg_val & (1 << info->enable_bit);
+ return !!(reg_val & (1 << info->enable_bit));
}
/* DA9030 specific operations */
@@ -172,7 +172,7 @@
int ret;
if (check_range(info, min_uV, max_uV)) {
- pr_err("invalid voltage range (%d, %d) uV", min_uV, max_uV);
+ pr_err("invalid voltage range (%d, %d) uV\n", min_uV, max_uV);
return -EINVAL;
}
@@ -199,7 +199,7 @@
int thresh;
if (check_range(info, min_uV, max_uV)) {
- pr_err("invalid voltage range (%d, %d) uV", min_uV, max_uV);
+ pr_err("invalid voltage range (%d, %d) uV\n", min_uV, max_uV);
return -EINVAL;
}
@@ -248,7 +248,7 @@
int ret;
if (check_range(info, min_uV, max_uV)) {
- pr_err("invalid voltage range (%d, %d) uV", min_uV, max_uV);
+ pr_err("invalid voltage range (%d, %d) uV\n", min_uV, max_uV);
return -EINVAL;
}
@@ -273,7 +273,7 @@
uint8_t val, mask;
if (check_range(info, min_uV, max_uV)) {
- pr_err("invalid voltage range (%d, %d) uV", min_uV, max_uV);
+ pr_err("invalid voltage range (%d, %d) uV\n", min_uV, max_uV);
return -EINVAL;
}
diff --git a/drivers/rtc/rtc-ds1307.c b/drivers/rtc/rtc-ds1307.c
index 162330b..7e5155e 100644
--- a/drivers/rtc/rtc-ds1307.c
+++ b/drivers/rtc/rtc-ds1307.c
@@ -86,13 +86,11 @@
struct ds1307 {
- u8 reg_addr;
u8 regs[11];
enum ds_type type;
unsigned long flags;
#define HAS_NVRAM 0 /* bit 0 == sysfs file active */
#define HAS_ALARM 1 /* bit 1 == irq claimed */
- struct i2c_msg msg[2];
struct i2c_client *client;
struct rtc_device *rtc;
struct work_struct work;
@@ -204,13 +202,9 @@
int tmp;
/* read the RTC date and time registers all at once */
- ds1307->reg_addr = 0;
- ds1307->msg[1].flags = I2C_M_RD;
- ds1307->msg[1].len = 7;
-
- tmp = i2c_transfer(to_i2c_adapter(ds1307->client->dev.parent),
- ds1307->msg, 2);
- if (tmp != 2) {
+ tmp = i2c_smbus_read_i2c_block_data(ds1307->client,
+ DS1307_REG_SECS, 7, ds1307->regs);
+ if (tmp != 7) {
dev_err(dev, "%s error %d\n", "read", tmp);
return -EIO;
}
@@ -257,7 +251,6 @@
t->tm_hour, t->tm_mday,
t->tm_mon, t->tm_year, t->tm_wday);
- *buf++ = 0; /* first register addr */
buf[DS1307_REG_SECS] = bin2bcd(t->tm_sec);
buf[DS1307_REG_MIN] = bin2bcd(t->tm_min);
buf[DS1307_REG_HOUR] = bin2bcd(t->tm_hour);
@@ -282,23 +275,19 @@
break;
}
- ds1307->msg[1].flags = 0;
- ds1307->msg[1].len = 8;
-
dev_dbg(dev, "%s: %02x %02x %02x %02x %02x %02x %02x\n",
"write", buf[0], buf[1], buf[2], buf[3],
buf[4], buf[5], buf[6]);
- result = i2c_transfer(to_i2c_adapter(ds1307->client->dev.parent),
- &ds1307->msg[1], 1);
- if (result != 1) {
- dev_err(dev, "%s error %d\n", "write", tmp);
- return -EIO;
+ result = i2c_smbus_write_i2c_block_data(ds1307->client, 0, 7, buf);
+ if (result < 0) {
+ dev_err(dev, "%s error %d\n", "write", result);
+ return result;
}
return 0;
}
-static int ds1307_read_alarm(struct device *dev, struct rtc_wkalrm *t)
+static int ds1337_read_alarm(struct device *dev, struct rtc_wkalrm *t)
{
struct i2c_client *client = to_i2c_client(dev);
struct ds1307 *ds1307 = i2c_get_clientdata(client);
@@ -308,13 +297,9 @@
return -EINVAL;
/* read all ALARM1, ALARM2, and status registers at once */
- ds1307->reg_addr = DS1339_REG_ALARM1_SECS;
- ds1307->msg[1].flags = I2C_M_RD;
- ds1307->msg[1].len = 9;
-
- ret = i2c_transfer(to_i2c_adapter(client->dev.parent),
- ds1307->msg, 2);
- if (ret != 2) {
+ ret = i2c_smbus_read_i2c_block_data(client,
+ DS1339_REG_ALARM1_SECS, 9, ds1307->regs);
+ if (ret != 9) {
dev_err(dev, "%s error %d\n", "alarm read", ret);
return -EIO;
}
@@ -353,7 +338,7 @@
return 0;
}
-static int ds1307_set_alarm(struct device *dev, struct rtc_wkalrm *t)
+static int ds1337_set_alarm(struct device *dev, struct rtc_wkalrm *t)
{
struct i2c_client *client = to_i2c_client(dev);
struct ds1307 *ds1307 = i2c_get_clientdata(client);
@@ -371,13 +356,9 @@
t->enabled, t->pending);
/* read current status of both alarms and the chip */
- ds1307->reg_addr = DS1339_REG_ALARM1_SECS;
- ds1307->msg[1].flags = I2C_M_RD;
- ds1307->msg[1].len = 9;
-
- ret = i2c_transfer(to_i2c_adapter(client->dev.parent),
- ds1307->msg, 2);
- if (ret != 2) {
+ ret = i2c_smbus_read_i2c_block_data(client,
+ DS1339_REG_ALARM1_SECS, 9, buf);
+ if (ret != 9) {
dev_err(dev, "%s error %d\n", "alarm write", ret);
return -EIO;
}
@@ -392,7 +373,6 @@
ds1307->regs[6], control, status);
/* set ALARM1, using 24 hour and day-of-month modes */
- *buf++ = DS1339_REG_ALARM1_SECS; /* first register addr */
buf[0] = bin2bcd(t->time.tm_sec);
buf[1] = bin2bcd(t->time.tm_min);
buf[2] = bin2bcd(t->time.tm_hour);
@@ -411,14 +391,11 @@
}
buf[8] = status & ~(DS1337_BIT_A1I | DS1337_BIT_A2I);
- ds1307->msg[1].flags = 0;
- ds1307->msg[1].len = 10;
-
- ret = i2c_transfer(to_i2c_adapter(client->dev.parent),
- &ds1307->msg[1], 1);
- if (ret != 1) {
+ ret = i2c_smbus_write_i2c_block_data(client,
+ DS1339_REG_ALARM1_SECS, 9, buf);
+ if (ret < 0) {
dev_err(dev, "can't set alarm time\n");
- return -EIO;
+ return ret;
}
return 0;
@@ -475,8 +452,8 @@
static const struct rtc_class_ops ds13xx_rtc_ops = {
.read_time = ds1307_get_time,
.set_time = ds1307_set_time,
- .read_alarm = ds1307_read_alarm,
- .set_alarm = ds1307_set_alarm,
+ .read_alarm = ds1337_read_alarm,
+ .set_alarm = ds1337_set_alarm,
.ioctl = ds1307_ioctl,
};
@@ -490,7 +467,6 @@
{
struct i2c_client *client;
struct ds1307 *ds1307;
- struct i2c_msg msg[2];
int result;
client = kobj_to_i2c_client(kobj);
@@ -503,24 +479,10 @@
if (unlikely(!count))
return count;
- msg[0].addr = client->addr;
- msg[0].flags = 0;
- msg[0].len = 1;
- msg[0].buf = buf;
-
- buf[0] = 8 + off;
-
- msg[1].addr = client->addr;
- msg[1].flags = I2C_M_RD;
- msg[1].len = count;
- msg[1].buf = buf;
-
- result = i2c_transfer(to_i2c_adapter(client->dev.parent), msg, 2);
- if (result != 2) {
+ result = i2c_smbus_read_i2c_block_data(client, 8 + off, count, buf);
+ if (result < 0)
dev_err(&client->dev, "%s error %d\n", "nvram read", result);
- return -EIO;
- }
- return count;
+ return result;
}
static ssize_t
@@ -528,8 +490,7 @@
char *buf, loff_t off, size_t count)
{
struct i2c_client *client;
- u8 buffer[NVRAM_SIZE + 1];
- int ret;
+ int result;
client = kobj_to_i2c_client(kobj);
@@ -540,11 +501,12 @@
if (unlikely(!count))
return count;
- buffer[0] = 8 + off;
- memcpy(buffer + 1, buf, count);
-
- ret = i2c_master_send(client, buffer, count + 1);
- return (ret < 0) ? ret : (ret - 1);
+ result = i2c_smbus_write_i2c_block_data(client, 8 + off, count, buf);
+ if (result < 0) {
+ dev_err(&client->dev, "%s error %d\n", "nvram write", result);
+ return result;
+ }
+ return count;
}
static struct bin_attribute nvram = {
@@ -571,9 +533,11 @@
const struct chip_desc *chip = &chips[id->driver_data];
struct i2c_adapter *adapter = to_i2c_adapter(client->dev.parent);
int want_irq = false;
+ unsigned char *buf;
if (!i2c_check_functionality(adapter,
- I2C_FUNC_I2C | I2C_FUNC_SMBUS_WRITE_BYTE_DATA))
+ I2C_FUNC_SMBUS_WRITE_BYTE_DATA |
+ I2C_FUNC_SMBUS_I2C_BLOCK))
return -EIO;
if (!(ds1307 = kzalloc(sizeof(struct ds1307), GFP_KERNEL)))
@@ -581,18 +545,8 @@
ds1307->client = client;
i2c_set_clientdata(client, ds1307);
-
- ds1307->msg[0].addr = client->addr;
- ds1307->msg[0].flags = 0;
- ds1307->msg[0].len = 1;
- ds1307->msg[0].buf = &ds1307->reg_addr;
-
- ds1307->msg[1].addr = client->addr;
- ds1307->msg[1].flags = I2C_M_RD;
- ds1307->msg[1].len = sizeof(ds1307->regs);
- ds1307->msg[1].buf = ds1307->regs;
-
ds1307->type = id->driver_data;
+ buf = ds1307->regs;
switch (ds1307->type) {
case ds_1337:
@@ -602,21 +556,15 @@
INIT_WORK(&ds1307->work, ds1307_work);
want_irq = true;
}
-
- ds1307->reg_addr = DS1337_REG_CONTROL;
- ds1307->msg[1].len = 2;
-
/* get registers that the "rtc" read below won't read... */
- tmp = i2c_transfer(adapter, ds1307->msg, 2);
+ tmp = i2c_smbus_read_i2c_block_data(ds1307->client,
+ DS1337_REG_CONTROL, 2, buf);
if (tmp != 2) {
pr_debug("read error %d\n", tmp);
err = -EIO;
goto exit_free;
}
- ds1307->reg_addr = 0;
- ds1307->msg[1].len = sizeof(ds1307->regs);
-
/* oscillator off? turn it on, so clock can tick. */
if (ds1307->regs[0] & DS1337_BIT_nEOSC)
ds1307->regs[0] &= ~DS1337_BIT_nEOSC;
@@ -647,9 +595,8 @@
read_rtc:
/* read RTC registers */
-
- tmp = i2c_transfer(adapter, ds1307->msg, 2);
- if (tmp != 2) {
+ tmp = i2c_smbus_read_i2c_block_data(ds1307->client, 0, 8, buf);
+ if (tmp != 8) {
pr_debug("read error %d\n", tmp);
err = -EIO;
goto exit_free;
@@ -707,22 +654,6 @@
break;
}
- tmp = ds1307->regs[DS1307_REG_SECS];
- tmp = bcd2bin(tmp & 0x7f);
- if (tmp > 60)
- goto exit_bad;
- tmp = bcd2bin(ds1307->regs[DS1307_REG_MIN] & 0x7f);
- if (tmp > 60)
- goto exit_bad;
-
- tmp = bcd2bin(ds1307->regs[DS1307_REG_MDAY] & 0x3f);
- if (tmp == 0 || tmp > 31)
- goto exit_bad;
-
- tmp = bcd2bin(ds1307->regs[DS1307_REG_MONTH] & 0x1f);
- if (tmp == 0 || tmp > 12)
- goto exit_bad;
-
tmp = ds1307->regs[DS1307_REG_HOUR];
switch (ds1307->type) {
case ds_1340:
@@ -779,13 +710,6 @@
return 0;
-exit_bad:
- dev_dbg(&client->dev, "%s: %02x %02x %02x %02x %02x %02x %02x\n",
- "bogus register",
- ds1307->regs[0], ds1307->regs[1],
- ds1307->regs[2], ds1307->regs[3],
- ds1307->regs[4], ds1307->regs[5],
- ds1307->regs[6]);
exit_irq:
if (ds1307->rtc)
rtc_device_unregister(ds1307->rtc);
diff --git a/drivers/rtc/rtc-parisc.c b/drivers/rtc/rtc-parisc.c
index 346d633..c6bfa6f 100644
--- a/drivers/rtc/rtc-parisc.c
+++ b/drivers/rtc/rtc-parisc.c
@@ -34,7 +34,8 @@
static int parisc_set_time(struct device *dev, struct rtc_time *tm)
{
struct parisc_rtc *p = dev_get_drvdata(dev);
- unsigned long flags, ret;
+ unsigned long flags;
+ int ret;
spin_lock_irqsave(&p->lock, flags);
ret = set_rtc_time(tm);
diff --git a/drivers/s390/block/dasd_3990_erp.c b/drivers/s390/block/dasd_3990_erp.c
index b8f9c00..d82aad5 100644
--- a/drivers/s390/block/dasd_3990_erp.c
+++ b/drivers/s390/block/dasd_3990_erp.c
@@ -2621,7 +2621,7 @@
}
}
- /* double-check if current erp/cqr was successfull */
+ /* double-check if current erp/cqr was successful */
if ((cqr->irb.scsw.cmd.cstat == 0x00) &&
(cqr->irb.scsw.cmd.dstat ==
(DEV_STAT_CHN_END | DEV_STAT_DEV_END))) {
diff --git a/drivers/s390/block/dasd_int.h b/drivers/s390/block/dasd_int.h
index 05a1453..4a39084 100644
--- a/drivers/s390/block/dasd_int.h
+++ b/drivers/s390/block/dasd_int.h
@@ -199,7 +199,7 @@
#define DASD_CQR_ERROR 0x82 /* request is completed with error */
#define DASD_CQR_CLEAR_PENDING 0x83 /* request is clear pending */
#define DASD_CQR_CLEARED 0x84 /* request was cleared */
-#define DASD_CQR_SUCCESS 0x85 /* request was successfull */
+#define DASD_CQR_SUCCESS 0x85 /* request was successful */
/* per dasd_ccw_req flags */
diff --git a/drivers/s390/char/tape_3590.c b/drivers/s390/char/tape_3590.c
index 4005c44..71605a1 100644
--- a/drivers/s390/char/tape_3590.c
+++ b/drivers/s390/char/tape_3590.c
@@ -801,7 +801,7 @@
static inline int
tape_3590_erp_succeded(struct tape_device *device, struct tape_request *request)
{
- DBF_EVENT(3, "Error Recovery successfull for %s\n",
+ DBF_EVENT(3, "Error Recovery successful for %s\n",
tape_op_verbose[request->op]);
return tape_3590_done(device, request);
}
diff --git a/drivers/s390/cio/cio.c b/drivers/s390/cio/cio.c
index 06b7182..659f8a7 100644
--- a/drivers/s390/cio/cio.c
+++ b/drivers/s390/cio/cio.c
@@ -379,7 +379,7 @@
if (ccode < 0) /* -EIO if msch gets a program check. */
return ccode;
switch (ccode) {
- case 0: /* successfull */
+ case 0: /* successful */
if (stsch(sch->schid, &schib) ||
!css_sch_is_valid(&schib))
return -ENODEV;
diff --git a/drivers/s390/cio/qdio_main.c b/drivers/s390/cio/qdio_main.c
index 744f928..10cb0f8 100644
--- a/drivers/s390/cio/qdio_main.c
+++ b/drivers/s390/cio/qdio_main.c
@@ -114,7 +114,7 @@
* @count: count of buffers to examine
* @auto_ack: automatically acknowledge buffers
*
- * Returns the number of successfull extracted equal buffer states.
+ * Returns the number of successfully extracted equal buffer states.
* Stops processing if a state is different from the last buffers state.
*/
static int qdio_do_eqbs(struct qdio_q *q, unsigned char *state,
diff --git a/drivers/s390/net/qeth_l2_main.c b/drivers/s390/net/qeth_l2_main.c
index 591a2b3..c4f1b04 100644
--- a/drivers/s390/net/qeth_l2_main.c
+++ b/drivers/s390/net/qeth_l2_main.c
@@ -916,6 +916,21 @@
.get_drvinfo = qeth_core_get_drvinfo,
};
+static struct net_device_ops qeth_l2_netdev_ops = {
+ .ndo_open = qeth_l2_open,
+ .ndo_stop = qeth_l2_stop,
+ .ndo_get_stats = qeth_get_stats,
+ .ndo_start_xmit = qeth_l2_hard_start_xmit,
+ .ndo_validate_addr = eth_validate_addr,
+ .ndo_set_multicast_list = qeth_l2_set_multicast_list,
+ .ndo_do_ioctl = qeth_l2_do_ioctl,
+ .ndo_set_mac_address = qeth_l2_set_mac_address,
+ .ndo_change_mtu = qeth_change_mtu,
+ .ndo_vlan_rx_add_vid = qeth_l2_vlan_rx_add_vid,
+ .ndo_vlan_rx_kill_vid = qeth_l2_vlan_rx_kill_vid,
+ .ndo_tx_timeout = qeth_tx_timeout,
+};
+
static int qeth_l2_setup_netdev(struct qeth_card *card)
{
switch (card->info.type) {
@@ -937,19 +952,9 @@
return -ENODEV;
card->dev->ml_priv = card;
- card->dev->tx_timeout = &qeth_tx_timeout;
card->dev->watchdog_timeo = QETH_TX_TIMEOUT;
- card->dev->open = qeth_l2_open;
- card->dev->stop = qeth_l2_stop;
- card->dev->hard_start_xmit = qeth_l2_hard_start_xmit;
- card->dev->do_ioctl = qeth_l2_do_ioctl;
- card->dev->get_stats = qeth_get_stats;
- card->dev->change_mtu = qeth_change_mtu;
- card->dev->set_multicast_list = qeth_l2_set_multicast_list;
- card->dev->vlan_rx_kill_vid = qeth_l2_vlan_rx_kill_vid;
- card->dev->vlan_rx_add_vid = qeth_l2_vlan_rx_add_vid;
- card->dev->set_mac_address = qeth_l2_set_mac_address;
card->dev->mtu = card->info.initial_mtu;
+ card->dev->netdev_ops = &qeth_l2_netdev_ops;
if (card->info.type != QETH_CARD_TYPE_OSN)
SET_ETHTOOL_OPS(card->dev, &qeth_l2_ethtool_ops);
else
diff --git a/drivers/s390/net/qeth_l3_main.c b/drivers/s390/net/qeth_l3_main.c
index 4693ee4..68d623a 100644
--- a/drivers/s390/net/qeth_l3_main.c
+++ b/drivers/s390/net/qeth_l3_main.c
@@ -1829,28 +1829,6 @@
static void qeth_l3_vlan_rx_add_vid(struct net_device *dev, unsigned short vid)
{
- struct net_device *vlandev;
- struct qeth_card *card = dev->ml_priv;
- struct in_device *in_dev;
-
- if (card->info.type == QETH_CARD_TYPE_IQD)
- return;
-
- vlandev = vlan_group_get_device(card->vlangrp, vid);
- vlandev->neigh_setup = qeth_l3_neigh_setup;
-
- in_dev = in_dev_get(vlandev);
-#ifdef CONFIG_SYSCTL
- neigh_sysctl_unregister(in_dev->arp_parms);
-#endif
- neigh_parms_release(&arp_tbl, in_dev->arp_parms);
-
- in_dev->arp_parms = neigh_parms_alloc(vlandev, &arp_tbl);
-#ifdef CONFIG_SYSCTL
- neigh_sysctl_register(vlandev, in_dev->arp_parms, NET_IPV4,
- NET_IPV4_NEIGH, "ipv4", NULL, NULL);
-#endif
- in_dev_put(in_dev);
return;
}
@@ -2916,6 +2894,21 @@
return 0;
}
+static struct net_device_ops qeth_l3_netdev_ops = {
+ .ndo_open = qeth_l3_open,
+ .ndo_stop = qeth_l3_stop,
+ .ndo_get_stats = qeth_get_stats,
+ .ndo_start_xmit = qeth_l3_hard_start_xmit,
+ .ndo_validate_addr = eth_validate_addr,
+ .ndo_set_multicast_list = qeth_l3_set_multicast_list,
+ .ndo_do_ioctl = qeth_l3_do_ioctl,
+ .ndo_change_mtu = qeth_change_mtu,
+ .ndo_vlan_rx_register = qeth_l3_vlan_rx_register,
+ .ndo_vlan_rx_add_vid = qeth_l3_vlan_rx_add_vid,
+ .ndo_vlan_rx_kill_vid = qeth_l3_vlan_rx_kill_vid,
+ .ndo_tx_timeout = qeth_tx_timeout,
+};
+
static int qeth_l3_setup_netdev(struct qeth_card *card)
{
if (card->info.type == QETH_CARD_TYPE_OSAE) {
@@ -2930,7 +2923,8 @@
card->dev = alloc_etherdev(0);
if (!card->dev)
return -ENODEV;
- card->dev->neigh_setup = qeth_l3_neigh_setup;
+ qeth_l3_netdev_ops.ndo_neigh_setup =
+ qeth_l3_neigh_setup;
/*IPv6 address autoconfiguration stuff*/
qeth_l3_get_unique_id(card);
@@ -2947,21 +2941,10 @@
} else
return -ENODEV;
- card->dev->hard_start_xmit = qeth_l3_hard_start_xmit;
card->dev->ml_priv = card;
- card->dev->tx_timeout = &qeth_tx_timeout;
card->dev->watchdog_timeo = QETH_TX_TIMEOUT;
- card->dev->open = qeth_l3_open;
- card->dev->stop = qeth_l3_stop;
- card->dev->do_ioctl = qeth_l3_do_ioctl;
- card->dev->get_stats = qeth_get_stats;
- card->dev->change_mtu = qeth_change_mtu;
- card->dev->set_multicast_list = qeth_l3_set_multicast_list;
- card->dev->vlan_rx_register = qeth_l3_vlan_rx_register;
- card->dev->vlan_rx_add_vid = qeth_l3_vlan_rx_add_vid;
- card->dev->vlan_rx_kill_vid = qeth_l3_vlan_rx_kill_vid;
card->dev->mtu = card->info.initial_mtu;
- card->dev->set_mac_address = NULL;
+ card->dev->netdev_ops = &qeth_l3_netdev_ops;
SET_ETHTOOL_OPS(card->dev, &qeth_l3_ethtool_ops);
card->dev->features |= NETIF_F_HW_VLAN_TX |
NETIF_F_HW_VLAN_RX |
diff --git a/drivers/scsi/Kconfig b/drivers/scsi/Kconfig
index b732297..256c7be 100644
--- a/drivers/scsi/Kconfig
+++ b/drivers/scsi/Kconfig
@@ -884,6 +884,7 @@
tristate "IBM Virtual SCSI support"
depends on PPC_PSERIES || PPC_ISERIES
select SCSI_SRP_ATTRS
+ select VIOPATH if PPC_ISERIES
help
This is the IBM POWER Virtual SCSI Client
diff --git a/drivers/scsi/NCR_D700.c b/drivers/scsi/NCR_D700.c
index 9e64b21..c889d84 100644
--- a/drivers/scsi/NCR_D700.c
+++ b/drivers/scsi/NCR_D700.c
@@ -318,7 +318,7 @@
return -ENOMEM;
p->dev = dev;
- snprintf(p->name, sizeof(p->name), "D700(%s)", dev->bus_id);
+ snprintf(p->name, sizeof(p->name), "D700(%s)", dev_name(dev));
if (request_irq(irq, NCR_D700_intr, IRQF_SHARED, p->name, p)) {
printk(KERN_ERR "D700: request_irq failed\n");
kfree(p);
diff --git a/drivers/scsi/a2091.c b/drivers/scsi/a2091.c
index 07d572f..37dd471 100644
--- a/drivers/scsi/a2091.c
+++ b/drivers/scsi/a2091.c
@@ -169,10 +169,8 @@
continue;
instance = scsi_register (tpnt, sizeof (struct WD33C93_hostdata));
- if (instance == NULL) {
- release_mem_region(address, 256);
- continue;
- }
+ if (instance == NULL)
+ goto release;
instance->base = ZTWO_VADDR(address);
instance->irq = IRQ_AMIGA_PORTS;
instance->unique_id = z->slotaddr;
@@ -183,10 +181,18 @@
HDATA(instance)->fast = 0;
HDATA(instance)->dma_mode = CTRL_DMA;
wd33c93_init(instance, regs, dma_setup, dma_stop, WD33C93_FS_8_10);
- request_irq(IRQ_AMIGA_PORTS, a2091_intr, IRQF_SHARED, "A2091 SCSI",
- instance);
+ if (request_irq(IRQ_AMIGA_PORTS, a2091_intr, IRQF_SHARED, "A2091 SCSI",
+ instance))
+ goto unregister;
DMA(instance)->CNTR = CNTR_PDMD | CNTR_INTEN;
num_a2091++;
+ continue;
+
+unregister:
+ scsi_unregister(instance);
+ wd33c93_release();
+release:
+ release_mem_region(address, 256);
}
return num_a2091;
diff --git a/drivers/scsi/advansys.c b/drivers/scsi/advansys.c
index 2f60272..7507d8b 100644
--- a/drivers/scsi/advansys.c
+++ b/drivers/scsi/advansys.c
@@ -2527,7 +2527,7 @@
{
struct asc_board *boardp = shost_priv(s);
- printk("Scsi_Host at addr 0x%p, device %s\n", s, boardp->dev->bus_id);
+ printk("Scsi_Host at addr 0x%p, device %s\n", s, dev_name(boardp->dev));
printk(" host_busy %u, host_no %d, last_reset %d,\n",
s->host_busy, s->host_no, (unsigned)s->last_reset);
diff --git a/drivers/scsi/aic94xx/aic94xx_tmf.c b/drivers/scsi/aic94xx/aic94xx_tmf.c
index d4640ef..78eb86f 100644
--- a/drivers/scsi/aic94xx/aic94xx_tmf.c
+++ b/drivers/scsi/aic94xx/aic94xx_tmf.c
@@ -189,7 +189,7 @@
asd_clear_nexus_I_T(dev, NEXUS_PHASE_PRE);
/* send a hard reset */
ASD_DPRINTK("sending %s reset to %s\n",
- reset_type ? "hard" : "soft", phy->dev.bus_id);
+ reset_type ? "hard" : "soft", dev_name(&phy->dev));
res = sas_phy_reset(phy, reset_type);
if (res == TMF_RESP_FUNC_COMPLETE) {
/* wait for the maximum settle time */
diff --git a/drivers/scsi/cxgb3i/cxgb3i_ddp.c b/drivers/scsi/cxgb3i/cxgb3i_ddp.c
index 1a41f04..08f3a09 100644
--- a/drivers/scsi/cxgb3i/cxgb3i_ddp.c
+++ b/drivers/scsi/cxgb3i/cxgb3i_ddp.c
@@ -11,6 +11,7 @@
*/
#include <linux/skbuff.h>
+#include <linux/scatterlist.h>
/* from cxgb3 LLD */
#include "common.h"
diff --git a/drivers/scsi/gvp11.c b/drivers/scsi/gvp11.c
index ca73637..5d1bf7e 100644
--- a/drivers/scsi/gvp11.c
+++ b/drivers/scsi/gvp11.c
@@ -329,12 +329,16 @@
(epc & GVP_SCSICLKMASK) ? WD33C93_FS_8_10
: WD33C93_FS_12_15);
- request_irq(IRQ_AMIGA_PORTS, gvp11_intr, IRQF_SHARED, "GVP11 SCSI",
- instance);
+ if (request_irq(IRQ_AMIGA_PORTS, gvp11_intr, IRQF_SHARED, "GVP11 SCSI",
+ instance))
+ goto unregister;
DMA(instance)->CNTR = GVP11_DMAC_INT_ENABLE;
num_gvp11++;
continue;
+unregister:
+ scsi_unregister(instance);
+ wd33c93_release();
release:
release_mem_region(address, 256);
}
diff --git a/drivers/scsi/hosts.c b/drivers/scsi/hosts.c
index 3fdbb13..aa670a1 100644
--- a/drivers/scsi/hosts.c
+++ b/drivers/scsi/hosts.c
@@ -388,8 +388,7 @@
shost->dma_boundary = 0xffffffff;
device_initialize(&shost->shost_gendev);
- snprintf(shost->shost_gendev.bus_id, BUS_ID_SIZE, "host%d",
- shost->host_no);
+ dev_set_name(&shost->shost_gendev, "host%d", shost->host_no);
#ifndef CONFIG_SYSFS_DEPRECATED
shost->shost_gendev.bus = &scsi_bus_type;
#endif
@@ -398,8 +397,7 @@
device_initialize(&shost->shost_dev);
shost->shost_dev.parent = &shost->shost_gendev;
shost->shost_dev.class = &shost_class;
- snprintf(shost->shost_dev.bus_id, BUS_ID_SIZE, "host%d",
- shost->host_no);
+ dev_set_name(&shost->shost_dev, "host%d", shost->host_no);
shost->shost_dev.groups = scsi_sysfs_shost_attr_groups;
shost->ehandler = kthread_run(scsi_error_handler, shost,
diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c
index 44f202f..ee0739b 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc.c
@@ -101,7 +101,7 @@
{ IBMVFC_FABRIC_MAPPED, IBMVFC_UNABLE_TO_ESTABLISH, DID_ERROR, 1, 1, "unable to establish" },
{ IBMVFC_FABRIC_MAPPED, IBMVFC_XPORT_FAULT, DID_OK, 1, 0, "transport fault" },
{ IBMVFC_FABRIC_MAPPED, IBMVFC_CMD_TIMEOUT, DID_TIME_OUT, 1, 1, "command timeout" },
- { IBMVFC_FABRIC_MAPPED, IBMVFC_ENETDOWN, DID_NO_CONNECT, 1, 1, "network down" },
+ { IBMVFC_FABRIC_MAPPED, IBMVFC_ENETDOWN, DID_TRANSPORT_DISRUPTED, 1, 1, "network down" },
{ IBMVFC_FABRIC_MAPPED, IBMVFC_HW_FAILURE, DID_ERROR, 1, 1, "hardware failure" },
{ IBMVFC_FABRIC_MAPPED, IBMVFC_LINK_DOWN_ERR, DID_REQUEUE, 0, 0, "link down" },
{ IBMVFC_FABRIC_MAPPED, IBMVFC_LINK_DEAD_ERR, DID_ERROR, 0, 0, "link dead" },
@@ -115,11 +115,11 @@
{ IBMVFC_VIOS_FAILURE, IBMVFC_CRQ_FAILURE, DID_REQUEUE, 1, 1, "CRQ failure" },
{ IBMVFC_VIOS_FAILURE, IBMVFC_SW_FAILURE, DID_ERROR, 0, 1, "software failure" },
- { IBMVFC_VIOS_FAILURE, IBMVFC_INVALID_PARAMETER, DID_ABORT, 0, 1, "invalid parameter" },
- { IBMVFC_VIOS_FAILURE, IBMVFC_MISSING_PARAMETER, DID_ABORT, 0, 1, "missing parameter" },
+ { IBMVFC_VIOS_FAILURE, IBMVFC_INVALID_PARAMETER, DID_ERROR, 0, 1, "invalid parameter" },
+ { IBMVFC_VIOS_FAILURE, IBMVFC_MISSING_PARAMETER, DID_ERROR, 0, 1, "missing parameter" },
{ IBMVFC_VIOS_FAILURE, IBMVFC_HOST_IO_BUS, DID_ERROR, 1, 1, "host I/O bus failure" },
- { IBMVFC_VIOS_FAILURE, IBMVFC_TRANS_CANCELLED, DID_ABORT, 0, 1, "transaction cancelled" },
- { IBMVFC_VIOS_FAILURE, IBMVFC_TRANS_CANCELLED_IMPLICIT, DID_ABORT, 0, 1, "transaction cancelled implicit" },
+ { IBMVFC_VIOS_FAILURE, IBMVFC_TRANS_CANCELLED, DID_ERROR, 0, 1, "transaction cancelled" },
+ { IBMVFC_VIOS_FAILURE, IBMVFC_TRANS_CANCELLED_IMPLICIT, DID_ERROR, 0, 1, "transaction cancelled implicit" },
{ IBMVFC_VIOS_FAILURE, IBMVFC_INSUFFICIENT_RESOURCE, DID_REQUEUE, 1, 1, "insufficient resources" },
{ IBMVFC_VIOS_FAILURE, IBMVFC_PLOGI_REQUIRED, DID_ERROR, 0, 1, "port login required" },
{ IBMVFC_VIOS_FAILURE, IBMVFC_COMMAND_FAILED, DID_ERROR, 1, 1, "command failed" },
@@ -1145,10 +1145,10 @@
login_info->async.len = vhost->async_crq.size * sizeof(*vhost->async_crq.msgs);
strncpy(login_info->partition_name, vhost->partition_name, IBMVFC_MAX_NAME);
strncpy(login_info->device_name,
- vhost->host->shost_gendev.bus_id, IBMVFC_MAX_NAME);
+ dev_name(&vhost->host->shost_gendev), IBMVFC_MAX_NAME);
location = of_get_property(of_node, "ibm,loc-code", NULL);
- location = location ? location : vhost->dev->bus_id;
+ location = location ? location : dev_name(vhost->dev);
strncpy(login_info->drc_name, location, IBMVFC_MAX_NAME);
}
diff --git a/drivers/scsi/ibmvscsi/ibmvscsi.c b/drivers/scsi/ibmvscsi/ibmvscsi.c
index 868d35e..5c541f7 100644
--- a/drivers/scsi/ibmvscsi/ibmvscsi.c
+++ b/drivers/scsi/ibmvscsi/ibmvscsi.c
@@ -89,6 +89,7 @@
static int max_channel = 3;
static int init_timeout = 5;
static int max_requests = IBMVSCSI_MAX_REQUESTS_DEFAULT;
+static int max_events = IBMVSCSI_MAX_REQUESTS_DEFAULT + 2;
static struct scsi_transport_template *ibmvscsi_transport_template;
@@ -1633,7 +1634,7 @@
static unsigned long ibmvscsi_get_desired_dma(struct vio_dev *vdev)
{
/* iu_storage data allocated in initialize_event_pool */
- unsigned long desired_io = max_requests * sizeof(union viosrp_iu);
+ unsigned long desired_io = max_events * sizeof(union viosrp_iu);
/* add io space for sg data */
desired_io += (IBMVSCSI_MAX_SECTORS_DEFAULT * 512 *
@@ -1657,7 +1658,6 @@
vdev->dev.driver_data = NULL;
- driver_template.can_queue = max_requests - 2;
host = scsi_host_alloc(&driver_template, sizeof(*hostdata));
if (!host) {
dev_err(&vdev->dev, "couldn't allocate host data\n");
@@ -1673,12 +1673,12 @@
atomic_set(&hostdata->request_limit, -1);
hostdata->host->max_sectors = IBMVSCSI_MAX_SECTORS_DEFAULT;
- rc = ibmvscsi_ops->init_crq_queue(&hostdata->queue, hostdata, max_requests);
+ rc = ibmvscsi_ops->init_crq_queue(&hostdata->queue, hostdata, max_events);
if (rc != 0 && rc != H_RESOURCE) {
dev_err(&vdev->dev, "couldn't initialize crq. rc=%d\n", rc);
goto init_crq_failed;
}
- if (initialize_event_pool(&hostdata->pool, max_requests, hostdata) != 0) {
+ if (initialize_event_pool(&hostdata->pool, max_events, hostdata) != 0) {
dev_err(&vdev->dev, "couldn't initialize event pool\n");
goto init_pool_failed;
}
@@ -1730,7 +1730,7 @@
add_host_failed:
release_event_pool(&hostdata->pool, hostdata);
init_pool_failed:
- ibmvscsi_ops->release_crq_queue(&hostdata->queue, hostdata, max_requests);
+ ibmvscsi_ops->release_crq_queue(&hostdata->queue, hostdata, max_events);
init_crq_failed:
scsi_host_put(host);
scsi_host_alloc_failed:
@@ -1742,7 +1742,7 @@
struct ibmvscsi_host_data *hostdata = vdev->dev.driver_data;
release_event_pool(&hostdata->pool, hostdata);
ibmvscsi_ops->release_crq_queue(&hostdata->queue, hostdata,
- max_requests);
+ max_events);
srp_remove_host(hostdata->host);
scsi_remove_host(hostdata->host);
@@ -1779,6 +1779,10 @@
{
int ret;
+ /* Ensure we have two requests to do error recovery */
+ driver_template.can_queue = max_requests;
+ max_events = max_requests + 2;
+
if (firmware_has_feature(FW_FEATURE_ISERIES))
ibmvscsi_ops = &iseriesvscsi_ops;
else if (firmware_has_feature(FW_FEATURE_VIO))
diff --git a/drivers/scsi/ipr.c b/drivers/scsi/ipr.c
index 0edfb1f..841f460 100644
--- a/drivers/scsi/ipr.c
+++ b/drivers/scsi/ipr.c
@@ -2184,7 +2184,7 @@
sizeof(struct ipr_dump_entry_header);
driver_dump->location_entry.hdr.data_type = IPR_DUMP_DATA_TYPE_ASCII;
driver_dump->location_entry.hdr.id = IPR_DUMP_LOCATION_ID;
- strcpy(driver_dump->location_entry.location, ioa_cfg->pdev->dev.bus_id);
+ strcpy(driver_dump->location_entry.location, dev_name(&ioa_cfg->pdev->dev));
driver_dump->hdr.num_entries++;
}
diff --git a/drivers/scsi/ipr.h b/drivers/scsi/ipr.h
index 5945914..8f872f8 100644
--- a/drivers/scsi/ipr.h
+++ b/drivers/scsi/ipr.h
@@ -1272,7 +1272,7 @@
struct ipr_dump_location_entry {
struct ipr_dump_entry_header hdr;
- u8 location[BUS_ID_SIZE];
+ u8 location[20];
}__attribute__((packed));
struct ipr_dump_trace_entry {
diff --git a/drivers/scsi/lasi700.c b/drivers/scsi/lasi700.c
index 3126824..4a4e695 100644
--- a/drivers/scsi/lasi700.c
+++ b/drivers/scsi/lasi700.c
@@ -103,8 +103,7 @@
hostdata = kzalloc(sizeof(*hostdata), GFP_KERNEL);
if (!hostdata) {
- printk(KERN_ERR "%s: Failed to allocate host data\n",
- dev->dev.bus_id);
+ dev_printk(KERN_ERR, dev, "Failed to allocate host data\n");
return -ENOMEM;
}
diff --git a/drivers/scsi/libsas/sas_discover.c b/drivers/scsi/libsas/sas_discover.c
index 709a6f7..facc5bfc 100644
--- a/drivers/scsi/libsas/sas_discover.c
+++ b/drivers/scsi/libsas/sas_discover.c
@@ -169,7 +169,7 @@
if (res) {
printk("sas: driver on pcidev %s cannot handle "
"device %llx, error:%d\n",
- sas_ha->dev->bus_id,
+ dev_name(sas_ha->dev),
SAS_ADDR(dev->sas_addr), res);
}
}
diff --git a/drivers/scsi/libsas/sas_dump.c b/drivers/scsi/libsas/sas_dump.c
index bf34a23..c17c250 100644
--- a/drivers/scsi/libsas/sas_dump.c
+++ b/drivers/scsi/libsas/sas_dump.c
@@ -56,7 +56,7 @@
void sas_dprint_hae(struct sas_ha_struct *sas_ha, enum ha_event he)
{
- SAS_DPRINTK("ha %s: %s event\n", sas_ha->dev->bus_id,
+ SAS_DPRINTK("ha %s: %s event\n", dev_name(sas_ha->dev),
sas_hae_str[he]);
}
diff --git a/drivers/scsi/libsas/sas_host_smp.c b/drivers/scsi/libsas/sas_host_smp.c
index 16f9312..d110a36 100644
--- a/drivers/scsi/libsas/sas_host_smp.c
+++ b/drivers/scsi/libsas/sas_host_smp.c
@@ -199,8 +199,8 @@
break;
case SMP_DISCOVER:
- req->data_len =- 16;
- if (req->data_len < 0) {
+ req->data_len -= 16;
+ if ((int)req->data_len < 0) {
req->data_len = 0;
error = -EINVAL;
goto out;
@@ -215,8 +215,8 @@
break;
case SMP_REPORT_PHY_SATA:
- req->data_len =- 16;
- if (req->data_len < 0) {
+ req->data_len -= 16;
+ if ((int)req->data_len < 0) {
req->data_len = 0;
error = -EINVAL;
goto out;
@@ -238,8 +238,8 @@
break;
case SMP_PHY_CONTROL:
- req->data_len =- 44;
- if (req->data_len < 0) {
+ req->data_len -= 44;
+ if ((int)req->data_len < 0) {
req->data_len = 0;
error = -EINVAL;
goto out;
diff --git a/drivers/scsi/libsas/sas_port.c b/drivers/scsi/libsas/sas_port.c
index 139935a..e6ac59c 100644
--- a/drivers/scsi/libsas/sas_port.c
+++ b/drivers/scsi/libsas/sas_port.c
@@ -113,7 +113,7 @@
sas_port_add_phy(port->port, phy->phy);
SAS_DPRINTK("%s added to %s, phy_mask:0x%x (%16llx)\n",
- phy->phy->dev.bus_id,port->port->dev.bus_id,
+ dev_name(&phy->phy->dev), dev_name(&port->port->dev),
port->phy_mask,
SAS_ADDR(port->attached_sas_addr));
diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index 4c77038..6c86731 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -1795,12 +1795,13 @@
int
lpfc_online(struct lpfc_hba *phba)
{
- struct lpfc_vport *vport = phba->pport;
+ struct lpfc_vport *vport;
struct lpfc_vport **vports;
int i;
if (!phba)
return 0;
+ vport = phba->pport;
if (!(vport->fc_flag & FC_OFFLINE_MODE))
return 0;
diff --git a/drivers/scsi/mvsas.c b/drivers/scsi/mvsas.c
index 1dd70d7..23e5a87 100644
--- a/drivers/scsi/mvsas.c
+++ b/drivers/scsi/mvsas.c
@@ -2959,7 +2959,7 @@
/* enable auto port detection */
mw32(GBL_PORT_TYPE, MODE_AUTO_DET_EN);
- msleep(100);
+ msleep(1100);
/* init and reset phys */
for (i = 0; i < mvi->chip->n_phy; i++) {
u32 lo = be32_to_cpu(*(u32 *)&mvi->sas_addr[4]);
diff --git a/drivers/scsi/pcmcia/aha152x_stub.c b/drivers/scsi/pcmcia/aha152x_stub.c
index 165ff88..67cde01 100644
--- a/drivers/scsi/pcmcia/aha152x_stub.c
+++ b/drivers/scsi/pcmcia/aha152x_stub.c
@@ -114,7 +114,7 @@
link->io.NumPorts1 = 0x20;
link->io.Attributes1 = IO_DATA_PATH_WIDTH_AUTO;
link->io.IOAddrLines = 10;
- link->irq.Attributes = IRQ_TYPE_EXCLUSIVE;
+ link->irq.Attributes = IRQ_TYPE_DYNAMIC_SHARING;
link->irq.IRQInfo1 = IRQ_LEVEL_ID;
link->conf.Attributes = CONF_ENABLE_IRQ;
link->conf.IntType = INT_MEMORY_AND_IO;
diff --git a/drivers/scsi/qla2xxx/qla_attr.c b/drivers/scsi/qla2xxx/qla_attr.c
index cd53627..c7acef5 100644
--- a/drivers/scsi/qla2xxx/qla_attr.c
+++ b/drivers/scsi/qla2xxx/qla_attr.c
@@ -303,7 +303,7 @@
else if (start == (ha->flt_region_boot * 4) ||
start == (ha->flt_region_fw * 4))
valid = 1;
- else if (IS_QLA25XX(ha) &&
+ else if ((IS_QLA25XX(ha) || IS_QLA81XX(ha)) &&
start == (ha->flt_region_vpd_nvram * 4))
valid = 1;
if (!valid) {
@@ -815,6 +815,21 @@
ha->qla_stats.total_isp_aborts);
}
+static ssize_t
+qla2x00_mpi_version_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ scsi_qla_host_t *vha = shost_priv(class_to_shost(dev));
+ struct qla_hw_data *ha = vha->hw;
+
+ if (!IS_QLA81XX(ha))
+ return snprintf(buf, PAGE_SIZE, "\n");
+
+ return snprintf(buf, PAGE_SIZE, "%02x.%02x.%02x.%02x (%x)\n",
+ ha->mpi_version[0], ha->mpi_version[1], ha->mpi_version[2],
+ ha->mpi_version[3], ha->mpi_capabilities);
+}
+
static DEVICE_ATTR(driver_version, S_IRUGO, qla2x00_drvr_version_show, NULL);
static DEVICE_ATTR(fw_version, S_IRUGO, qla2x00_fw_version_show, NULL);
static DEVICE_ATTR(serial_num, S_IRUGO, qla2x00_serial_num_show, NULL);
@@ -839,6 +854,7 @@
NULL);
static DEVICE_ATTR(total_isp_aborts, S_IRUGO, qla2x00_total_isp_aborts_show,
NULL);
+static DEVICE_ATTR(mpi_version, S_IRUGO, qla2x00_mpi_version_show, NULL);
struct device_attribute *qla2x00_host_attrs[] = {
&dev_attr_driver_version,
@@ -858,6 +874,7 @@
&dev_attr_optrom_fcode_version,
&dev_attr_optrom_fw_version,
&dev_attr_total_isp_aborts,
+ &dev_attr_mpi_version,
NULL,
};
@@ -892,6 +909,9 @@
case PORT_SPEED_8GB:
speed = FC_PORTSPEED_8GBIT;
break;
+ case PORT_SPEED_10GB:
+ speed = FC_PORTSPEED_10GBIT;
+ break;
}
fc_host_speed(shost) = speed;
}
@@ -1382,7 +1402,9 @@
fc_host_max_npiv_vports(vha->host) = ha->max_npiv_vports;
fc_host_npiv_vports_inuse(vha->host) = ha->cur_vport_count;
- if (IS_QLA25XX(ha))
+ if (IS_QLA81XX(ha))
+ speed = FC_PORTSPEED_10GBIT;
+ else if (IS_QLA25XX(ha))
speed = FC_PORTSPEED_8GBIT | FC_PORTSPEED_4GBIT |
FC_PORTSPEED_2GBIT | FC_PORTSPEED_1GBIT;
else if (IS_QLA24XX_TYPE(ha))
diff --git a/drivers/scsi/qla2xxx/qla_dbg.c b/drivers/scsi/qla2xxx/qla_dbg.c
index 1cf7777..34760f8 100644
--- a/drivers/scsi/qla2xxx/qla_dbg.c
+++ b/drivers/scsi/qla2xxx/qla_dbg.c
@@ -310,6 +310,76 @@
*buf++ = htons(RD_REG_WORD(dmp_reg++));
}
+static inline void *
+qla24xx_copy_eft(struct qla_hw_data *ha, void *ptr)
+{
+ if (!ha->eft)
+ return ptr;
+
+ memcpy(ptr, ha->eft, ntohl(ha->fw_dump->eft_size));
+ return ptr + ntohl(ha->fw_dump->eft_size);
+}
+
+static inline void *
+qla25xx_copy_fce(struct qla_hw_data *ha, void *ptr, uint32_t **last_chain)
+{
+ uint32_t cnt;
+ uint32_t *iter_reg;
+ struct qla2xxx_fce_chain *fcec = ptr;
+
+ if (!ha->fce)
+ return ptr;
+
+ *last_chain = &fcec->type;
+ fcec->type = __constant_htonl(DUMP_CHAIN_FCE);
+ fcec->chain_size = htonl(sizeof(struct qla2xxx_fce_chain) +
+ fce_calc_size(ha->fce_bufs));
+ fcec->size = htonl(fce_calc_size(ha->fce_bufs));
+ fcec->addr_l = htonl(LSD(ha->fce_dma));
+ fcec->addr_h = htonl(MSD(ha->fce_dma));
+
+ iter_reg = fcec->eregs;
+ for (cnt = 0; cnt < 8; cnt++)
+ *iter_reg++ = htonl(ha->fce_mb[cnt]);
+
+ memcpy(iter_reg, ha->fce, ntohl(fcec->size));
+
+ return iter_reg;
+}
+
+static inline void *
+qla25xx_copy_mq(struct qla_hw_data *ha, void *ptr, uint32_t **last_chain)
+{
+ uint32_t cnt, que_idx;
+ uint8_t req_cnt, rsp_cnt, que_cnt;
+ struct qla2xxx_mq_chain *mq = ptr;
+ struct device_reg_25xxmq __iomem *reg;
+
+ if (!ha->mqenable)
+ return ptr;
+
+ mq = ptr;
+ *last_chain = &mq->type;
+ mq->type = __constant_htonl(DUMP_CHAIN_MQ);
+ mq->chain_size = __constant_htonl(sizeof(struct qla2xxx_mq_chain));
+
+ req_cnt = find_first_zero_bit(ha->req_qid_map, ha->max_queues);
+ rsp_cnt = find_first_zero_bit(ha->rsp_qid_map, ha->max_queues);
+ que_cnt = req_cnt > rsp_cnt ? req_cnt : rsp_cnt;
+ mq->count = htonl(que_cnt);
+ for (cnt = 0; cnt < que_cnt; cnt++) {
+ reg = (struct device_reg_25xxmq *) ((void *)
+ ha->mqiobase + cnt * QLA_QUE_PAGE);
+ que_idx = cnt * 4;
+ mq->qregs[que_idx] = htonl(RD_REG_DWORD(®->req_q_in));
+ mq->qregs[que_idx+1] = htonl(RD_REG_DWORD(®->req_q_out));
+ mq->qregs[que_idx+2] = htonl(RD_REG_DWORD(®->rsp_q_in));
+ mq->qregs[que_idx+3] = htonl(RD_REG_DWORD(®->rsp_q_out));
+ }
+
+ return ptr + sizeof(struct qla2xxx_mq_chain);
+}
+
/**
* qla2300_fw_dump() - Dumps binary data from the 2300 firmware.
* @ha: HA context
@@ -913,8 +983,8 @@
goto qla24xx_fw_dump_failed_0;
nxt = qla2xxx_copy_queues(ha, nxt);
- if (ha->eft)
- memcpy(nxt, ha->eft, ntohl(ha->fw_dump->eft_size));
+
+ qla24xx_copy_eft(ha, nxt);
qla24xx_fw_dump_failed_0:
if (rval != QLA_SUCCESS) {
@@ -942,19 +1012,14 @@
uint32_t risc_address;
struct qla_hw_data *ha = vha->hw;
struct device_reg_24xx __iomem *reg = &ha->iobase->isp24;
- struct device_reg_25xxmq __iomem *reg25;
uint32_t __iomem *dmp_reg;
uint32_t *iter_reg;
uint16_t __iomem *mbx_reg;
unsigned long flags;
struct qla25xx_fw_dump *fw;
uint32_t ext_mem_cnt;
- void *nxt;
- struct qla2xxx_fce_chain *fcec;
- struct qla2xxx_mq_chain *mq = NULL;
- uint32_t qreg_size;
- uint8_t req_cnt, rsp_cnt, que_cnt;
- uint32_t que_idx;
+ void *nxt, *nxt_chain;
+ uint32_t *last_chain = NULL;
struct scsi_qla_host *base_vha = pci_get_drvdata(ha->pdev);
risc_address = ext_mem_cnt = 0;
@@ -1001,28 +1066,6 @@
fw->pcie_regs[2] = htonl(RD_REG_DWORD(dmp_reg));
fw->pcie_regs[3] = htonl(RD_REG_DWORD(®->iobase_window));
- /* Multi queue registers */
- if (ha->mqenable) {
- qreg_size = sizeof(struct qla2xxx_mq_chain);
- mq = kzalloc(qreg_size, GFP_KERNEL);
- if (!mq)
- goto qla25xx_fw_dump_failed_0;
- req_cnt = find_first_zero_bit(ha->req_qid_map, ha->max_queues);
- rsp_cnt = find_first_zero_bit(ha->rsp_qid_map, ha->max_queues);
- que_cnt = req_cnt > rsp_cnt ? req_cnt : rsp_cnt;
- mq->count = htonl(que_cnt);
- mq->chain_size = htonl(qreg_size);
- mq->type = __constant_htonl(DUMP_CHAIN_MQ);
- for (cnt = 0; cnt < que_cnt; cnt++) {
- reg25 = (struct device_reg_25xxmq *) ((void *)
- ha->mqiobase + cnt * QLA_QUE_PAGE);
- que_idx = cnt * 4;
- mq->qregs[que_idx] = htonl(reg25->req_q_in);
- mq->qregs[que_idx+1] = htonl(reg25->req_q_out);
- mq->qregs[que_idx+2] = htonl(reg25->rsp_q_in);
- mq->qregs[que_idx+3] = htonl(reg25->rsp_q_out);
- }
- }
WRT_REG_DWORD(®->iobase_window, 0x00);
RD_REG_DWORD(®->iobase_window);
@@ -1240,6 +1283,10 @@
iter_reg = qla24xx_read_window(reg, 0x61B0, 16, iter_reg);
qla24xx_read_window(reg, 0x6F00, 16, iter_reg);
+ /* Multi queue registers */
+ nxt_chain = qla25xx_copy_mq(ha, (void *)ha->fw_dump + ha->chain_offset,
+ &last_chain);
+
rval = qla24xx_soft_reset(ha);
if (rval != QLA_SUCCESS)
goto qla25xx_fw_dump_failed_0;
@@ -1249,37 +1296,16 @@
if (rval != QLA_SUCCESS)
goto qla25xx_fw_dump_failed_0;
- /* Fibre Channel Trace Buffer. */
nxt = qla2xxx_copy_queues(ha, nxt);
- if (ha->eft)
- memcpy(nxt, ha->eft, ntohl(ha->fw_dump->eft_size));
- /* Fibre Channel Event Buffer. */
- if (!ha->fce)
- goto qla25xx_fw_dump_failed_0;
+ nxt = qla24xx_copy_eft(ha, nxt);
- ha->fw_dump->version |= __constant_htonl(DUMP_CHAIN_VARIANT);
-
- if (ha->mqenable) {
- nxt = nxt + ntohl(ha->fw_dump->eft_size);
- memcpy(nxt, mq, qreg_size);
- kfree(mq);
- fcec = nxt + qreg_size;
- } else {
- fcec = nxt + ntohl(ha->fw_dump->eft_size);
+ /* Chain entries -- started with MQ. */
+ qla25xx_copy_fce(ha, nxt_chain, &last_chain);
+ if (last_chain) {
+ ha->fw_dump->version |= __constant_htonl(DUMP_CHAIN_VARIANT);
+ *last_chain |= __constant_htonl(DUMP_CHAIN_LAST);
}
- fcec->type = __constant_htonl(DUMP_CHAIN_FCE | DUMP_CHAIN_LAST);
- fcec->chain_size = htonl(sizeof(struct qla2xxx_fce_chain) +
- fce_calc_size(ha->fce_bufs));
- fcec->size = htonl(fce_calc_size(ha->fce_bufs));
- fcec->addr_l = htonl(LSD(ha->fce_dma));
- fcec->addr_h = htonl(MSD(ha->fce_dma));
-
- iter_reg = fcec->eregs;
- for (cnt = 0; cnt < 8; cnt++)
- *iter_reg++ = htonl(ha->fce_mb[cnt]);
-
- memcpy(iter_reg, ha->fce, ntohl(fcec->size));
qla25xx_fw_dump_failed_0:
if (rval != QLA_SUCCESS) {
@@ -1298,6 +1324,330 @@
if (!hardware_locked)
spin_unlock_irqrestore(&ha->hardware_lock, flags);
}
+
+void
+qla81xx_fw_dump(scsi_qla_host_t *vha, int hardware_locked)
+{
+ int rval;
+ uint32_t cnt;
+ uint32_t risc_address;
+ struct qla_hw_data *ha = vha->hw;
+ struct device_reg_24xx __iomem *reg = &ha->iobase->isp24;
+ uint32_t __iomem *dmp_reg;
+ uint32_t *iter_reg;
+ uint16_t __iomem *mbx_reg;
+ unsigned long flags;
+ struct qla81xx_fw_dump *fw;
+ uint32_t ext_mem_cnt;
+ void *nxt, *nxt_chain;
+ uint32_t *last_chain = NULL;
+ struct scsi_qla_host *base_vha = pci_get_drvdata(ha->pdev);
+
+ risc_address = ext_mem_cnt = 0;
+ flags = 0;
+
+ if (!hardware_locked)
+ spin_lock_irqsave(&ha->hardware_lock, flags);
+
+ if (!ha->fw_dump) {
+ qla_printk(KERN_WARNING, ha,
+ "No buffer available for dump!!!\n");
+ goto qla81xx_fw_dump_failed;
+ }
+
+ if (ha->fw_dumped) {
+ qla_printk(KERN_WARNING, ha,
+ "Firmware has been previously dumped (%p) -- ignoring "
+ "request...\n", ha->fw_dump);
+ goto qla81xx_fw_dump_failed;
+ }
+ fw = &ha->fw_dump->isp.isp81;
+ qla2xxx_prep_dump(ha, ha->fw_dump);
+
+ fw->host_status = htonl(RD_REG_DWORD(®->host_status));
+
+ /* Pause RISC. */
+ rval = qla24xx_pause_risc(reg);
+ if (rval != QLA_SUCCESS)
+ goto qla81xx_fw_dump_failed_0;
+
+ /* Host/Risc registers. */
+ iter_reg = fw->host_risc_reg;
+ iter_reg = qla24xx_read_window(reg, 0x7000, 16, iter_reg);
+ qla24xx_read_window(reg, 0x7010, 16, iter_reg);
+
+ /* PCIe registers. */
+ WRT_REG_DWORD(®->iobase_addr, 0x7C00);
+ RD_REG_DWORD(®->iobase_addr);
+ WRT_REG_DWORD(®->iobase_window, 0x01);
+ dmp_reg = ®->iobase_c4;
+ fw->pcie_regs[0] = htonl(RD_REG_DWORD(dmp_reg++));
+ fw->pcie_regs[1] = htonl(RD_REG_DWORD(dmp_reg++));
+ fw->pcie_regs[2] = htonl(RD_REG_DWORD(dmp_reg));
+ fw->pcie_regs[3] = htonl(RD_REG_DWORD(®->iobase_window));
+
+ WRT_REG_DWORD(®->iobase_window, 0x00);
+ RD_REG_DWORD(®->iobase_window);
+
+ /* Host interface registers. */
+ dmp_reg = ®->flash_addr;
+ for (cnt = 0; cnt < sizeof(fw->host_reg) / 4; cnt++)
+ fw->host_reg[cnt] = htonl(RD_REG_DWORD(dmp_reg++));
+
+ /* Disable interrupts. */
+ WRT_REG_DWORD(®->ictrl, 0);
+ RD_REG_DWORD(®->ictrl);
+
+ /* Shadow registers. */
+ WRT_REG_DWORD(®->iobase_addr, 0x0F70);
+ RD_REG_DWORD(®->iobase_addr);
+ WRT_REG_DWORD(®->iobase_select, 0xB0000000);
+ fw->shadow_reg[0] = htonl(RD_REG_DWORD(®->iobase_sdata));
+
+ WRT_REG_DWORD(®->iobase_select, 0xB0100000);
+ fw->shadow_reg[1] = htonl(RD_REG_DWORD(®->iobase_sdata));
+
+ WRT_REG_DWORD(®->iobase_select, 0xB0200000);
+ fw->shadow_reg[2] = htonl(RD_REG_DWORD(®->iobase_sdata));
+
+ WRT_REG_DWORD(®->iobase_select, 0xB0300000);
+ fw->shadow_reg[3] = htonl(RD_REG_DWORD(®->iobase_sdata));
+
+ WRT_REG_DWORD(®->iobase_select, 0xB0400000);
+ fw->shadow_reg[4] = htonl(RD_REG_DWORD(®->iobase_sdata));
+
+ WRT_REG_DWORD(®->iobase_select, 0xB0500000);
+ fw->shadow_reg[5] = htonl(RD_REG_DWORD(®->iobase_sdata));
+
+ WRT_REG_DWORD(®->iobase_select, 0xB0600000);
+ fw->shadow_reg[6] = htonl(RD_REG_DWORD(®->iobase_sdata));
+
+ WRT_REG_DWORD(®->iobase_select, 0xB0700000);
+ fw->shadow_reg[7] = htonl(RD_REG_DWORD(®->iobase_sdata));
+
+ WRT_REG_DWORD(®->iobase_select, 0xB0800000);
+ fw->shadow_reg[8] = htonl(RD_REG_DWORD(®->iobase_sdata));
+
+ WRT_REG_DWORD(®->iobase_select, 0xB0900000);
+ fw->shadow_reg[9] = htonl(RD_REG_DWORD(®->iobase_sdata));
+
+ WRT_REG_DWORD(®->iobase_select, 0xB0A00000);
+ fw->shadow_reg[10] = htonl(RD_REG_DWORD(®->iobase_sdata));
+
+ /* RISC I/O register. */
+ WRT_REG_DWORD(®->iobase_addr, 0x0010);
+ fw->risc_io_reg = htonl(RD_REG_DWORD(®->iobase_window));
+
+ /* Mailbox registers. */
+ mbx_reg = ®->mailbox0;
+ for (cnt = 0; cnt < sizeof(fw->mailbox_reg) / 2; cnt++)
+ fw->mailbox_reg[cnt] = htons(RD_REG_WORD(mbx_reg++));
+
+ /* Transfer sequence registers. */
+ iter_reg = fw->xseq_gp_reg;
+ iter_reg = qla24xx_read_window(reg, 0xBF00, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xBF10, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xBF20, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xBF30, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xBF40, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xBF50, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xBF60, 16, iter_reg);
+ qla24xx_read_window(reg, 0xBF70, 16, iter_reg);
+
+ iter_reg = fw->xseq_0_reg;
+ iter_reg = qla24xx_read_window(reg, 0xBFC0, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xBFD0, 16, iter_reg);
+ qla24xx_read_window(reg, 0xBFE0, 16, iter_reg);
+
+ qla24xx_read_window(reg, 0xBFF0, 16, fw->xseq_1_reg);
+
+ /* Receive sequence registers. */
+ iter_reg = fw->rseq_gp_reg;
+ iter_reg = qla24xx_read_window(reg, 0xFF00, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xFF10, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xFF20, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xFF30, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xFF40, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xFF50, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xFF60, 16, iter_reg);
+ qla24xx_read_window(reg, 0xFF70, 16, iter_reg);
+
+ iter_reg = fw->rseq_0_reg;
+ iter_reg = qla24xx_read_window(reg, 0xFFC0, 16, iter_reg);
+ qla24xx_read_window(reg, 0xFFD0, 16, iter_reg);
+
+ qla24xx_read_window(reg, 0xFFE0, 16, fw->rseq_1_reg);
+ qla24xx_read_window(reg, 0xFFF0, 16, fw->rseq_2_reg);
+
+ /* Auxiliary sequence registers. */
+ iter_reg = fw->aseq_gp_reg;
+ iter_reg = qla24xx_read_window(reg, 0xB000, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xB010, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xB020, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xB030, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xB040, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xB050, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0xB060, 16, iter_reg);
+ qla24xx_read_window(reg, 0xB070, 16, iter_reg);
+
+ iter_reg = fw->aseq_0_reg;
+ iter_reg = qla24xx_read_window(reg, 0xB0C0, 16, iter_reg);
+ qla24xx_read_window(reg, 0xB0D0, 16, iter_reg);
+
+ qla24xx_read_window(reg, 0xB0E0, 16, fw->aseq_1_reg);
+ qla24xx_read_window(reg, 0xB0F0, 16, fw->aseq_2_reg);
+
+ /* Command DMA registers. */
+ qla24xx_read_window(reg, 0x7100, 16, fw->cmd_dma_reg);
+
+ /* Queues. */
+ iter_reg = fw->req0_dma_reg;
+ iter_reg = qla24xx_read_window(reg, 0x7200, 8, iter_reg);
+ dmp_reg = ®->iobase_q;
+ for (cnt = 0; cnt < 7; cnt++)
+ *iter_reg++ = htonl(RD_REG_DWORD(dmp_reg++));
+
+ iter_reg = fw->resp0_dma_reg;
+ iter_reg = qla24xx_read_window(reg, 0x7300, 8, iter_reg);
+ dmp_reg = ®->iobase_q;
+ for (cnt = 0; cnt < 7; cnt++)
+ *iter_reg++ = htonl(RD_REG_DWORD(dmp_reg++));
+
+ iter_reg = fw->req1_dma_reg;
+ iter_reg = qla24xx_read_window(reg, 0x7400, 8, iter_reg);
+ dmp_reg = ®->iobase_q;
+ for (cnt = 0; cnt < 7; cnt++)
+ *iter_reg++ = htonl(RD_REG_DWORD(dmp_reg++));
+
+ /* Transmit DMA registers. */
+ iter_reg = fw->xmt0_dma_reg;
+ iter_reg = qla24xx_read_window(reg, 0x7600, 16, iter_reg);
+ qla24xx_read_window(reg, 0x7610, 16, iter_reg);
+
+ iter_reg = fw->xmt1_dma_reg;
+ iter_reg = qla24xx_read_window(reg, 0x7620, 16, iter_reg);
+ qla24xx_read_window(reg, 0x7630, 16, iter_reg);
+
+ iter_reg = fw->xmt2_dma_reg;
+ iter_reg = qla24xx_read_window(reg, 0x7640, 16, iter_reg);
+ qla24xx_read_window(reg, 0x7650, 16, iter_reg);
+
+ iter_reg = fw->xmt3_dma_reg;
+ iter_reg = qla24xx_read_window(reg, 0x7660, 16, iter_reg);
+ qla24xx_read_window(reg, 0x7670, 16, iter_reg);
+
+ iter_reg = fw->xmt4_dma_reg;
+ iter_reg = qla24xx_read_window(reg, 0x7680, 16, iter_reg);
+ qla24xx_read_window(reg, 0x7690, 16, iter_reg);
+
+ qla24xx_read_window(reg, 0x76A0, 16, fw->xmt_data_dma_reg);
+
+ /* Receive DMA registers. */
+ iter_reg = fw->rcvt0_data_dma_reg;
+ iter_reg = qla24xx_read_window(reg, 0x7700, 16, iter_reg);
+ qla24xx_read_window(reg, 0x7710, 16, iter_reg);
+
+ iter_reg = fw->rcvt1_data_dma_reg;
+ iter_reg = qla24xx_read_window(reg, 0x7720, 16, iter_reg);
+ qla24xx_read_window(reg, 0x7730, 16, iter_reg);
+
+ /* RISC registers. */
+ iter_reg = fw->risc_gp_reg;
+ iter_reg = qla24xx_read_window(reg, 0x0F00, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x0F10, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x0F20, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x0F30, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x0F40, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x0F50, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x0F60, 16, iter_reg);
+ qla24xx_read_window(reg, 0x0F70, 16, iter_reg);
+
+ /* Local memory controller registers. */
+ iter_reg = fw->lmc_reg;
+ iter_reg = qla24xx_read_window(reg, 0x3000, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x3010, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x3020, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x3030, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x3040, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x3050, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x3060, 16, iter_reg);
+ qla24xx_read_window(reg, 0x3070, 16, iter_reg);
+
+ /* Fibre Protocol Module registers. */
+ iter_reg = fw->fpm_hdw_reg;
+ iter_reg = qla24xx_read_window(reg, 0x4000, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x4010, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x4020, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x4030, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x4040, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x4050, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x4060, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x4070, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x4080, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x4090, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x40A0, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x40B0, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x40C0, 16, iter_reg);
+ qla24xx_read_window(reg, 0x40D0, 16, iter_reg);
+
+ /* Frame Buffer registers. */
+ iter_reg = fw->fb_hdw_reg;
+ iter_reg = qla24xx_read_window(reg, 0x6000, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x6010, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x6020, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x6030, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x6040, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x6100, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x6130, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x6150, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x6170, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x6190, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x61B0, 16, iter_reg);
+ iter_reg = qla24xx_read_window(reg, 0x61C0, 16, iter_reg);
+ qla24xx_read_window(reg, 0x6F00, 16, iter_reg);
+
+ /* Multi queue registers */
+ nxt_chain = qla25xx_copy_mq(ha, (void *)ha->fw_dump + ha->chain_offset,
+ &last_chain);
+
+ rval = qla24xx_soft_reset(ha);
+ if (rval != QLA_SUCCESS)
+ goto qla81xx_fw_dump_failed_0;
+
+ rval = qla24xx_dump_memory(ha, fw->code_ram, sizeof(fw->code_ram),
+ &nxt);
+ if (rval != QLA_SUCCESS)
+ goto qla81xx_fw_dump_failed_0;
+
+ nxt = qla2xxx_copy_queues(ha, nxt);
+
+ nxt = qla24xx_copy_eft(ha, nxt);
+
+ /* Chain entries -- started with MQ. */
+ qla25xx_copy_fce(ha, nxt_chain, &last_chain);
+ if (last_chain) {
+ ha->fw_dump->version |= __constant_htonl(DUMP_CHAIN_VARIANT);
+ *last_chain |= __constant_htonl(DUMP_CHAIN_LAST);
+ }
+
+qla81xx_fw_dump_failed_0:
+ if (rval != QLA_SUCCESS) {
+ qla_printk(KERN_WARNING, ha,
+ "Failed to dump firmware (%x)!!!\n", rval);
+ ha->fw_dumped = 0;
+
+ } else {
+ qla_printk(KERN_INFO, ha,
+ "Firmware dump saved to temp buffer (%ld/%p).\n",
+ base_vha->host_no, ha->fw_dump);
+ ha->fw_dumped = 1;
+ }
+
+qla81xx_fw_dump_failed:
+ if (!hardware_locked)
+ spin_unlock_irqrestore(&ha->hardware_lock, flags);
+}
+
/****************************************************************************/
/* Driver Debug Functions. */
/****************************************************************************/
diff --git a/drivers/scsi/qla2xxx/qla_dbg.h b/drivers/scsi/qla2xxx/qla_dbg.h
index c1794a7..f660dd7 100644
--- a/drivers/scsi/qla2xxx/qla_dbg.h
+++ b/drivers/scsi/qla2xxx/qla_dbg.h
@@ -251,6 +251,45 @@
uint32_t ext_mem[1];
};
+struct qla81xx_fw_dump {
+ uint32_t host_status;
+ uint32_t host_risc_reg[32];
+ uint32_t pcie_regs[4];
+ uint32_t host_reg[32];
+ uint32_t shadow_reg[11];
+ uint32_t risc_io_reg;
+ uint16_t mailbox_reg[32];
+ uint32_t xseq_gp_reg[128];
+ uint32_t xseq_0_reg[48];
+ uint32_t xseq_1_reg[16];
+ uint32_t rseq_gp_reg[128];
+ uint32_t rseq_0_reg[32];
+ uint32_t rseq_1_reg[16];
+ uint32_t rseq_2_reg[16];
+ uint32_t aseq_gp_reg[128];
+ uint32_t aseq_0_reg[32];
+ uint32_t aseq_1_reg[16];
+ uint32_t aseq_2_reg[16];
+ uint32_t cmd_dma_reg[16];
+ uint32_t req0_dma_reg[15];
+ uint32_t resp0_dma_reg[15];
+ uint32_t req1_dma_reg[15];
+ uint32_t xmt0_dma_reg[32];
+ uint32_t xmt1_dma_reg[32];
+ uint32_t xmt2_dma_reg[32];
+ uint32_t xmt3_dma_reg[32];
+ uint32_t xmt4_dma_reg[32];
+ uint32_t xmt_data_dma_reg[16];
+ uint32_t rcvt0_data_dma_reg[32];
+ uint32_t rcvt1_data_dma_reg[32];
+ uint32_t risc_gp_reg[128];
+ uint32_t lmc_reg[128];
+ uint32_t fpm_hdw_reg[224];
+ uint32_t fb_hdw_reg[208];
+ uint32_t code_ram[0x2000];
+ uint32_t ext_mem[1];
+};
+
#define EFT_NUM_BUFFERS 4
#define EFT_BYTES_PER_BUFFER 0x4000
#define EFT_SIZE ((EFT_BYTES_PER_BUFFER) * (EFT_NUM_BUFFERS))
@@ -313,5 +352,6 @@
struct qla2300_fw_dump isp23;
struct qla24xx_fw_dump isp24;
struct qla25xx_fw_dump isp25;
+ struct qla81xx_fw_dump isp81;
} isp;
};
diff --git a/drivers/scsi/qla2xxx/qla_def.h b/drivers/scsi/qla2xxx/qla_def.h
index a29c952..023ee77 100644
--- a/drivers/scsi/qla2xxx/qla_def.h
+++ b/drivers/scsi/qla2xxx/qla_def.h
@@ -187,7 +187,6 @@
* SCSI Request Block
*/
typedef struct srb {
- struct scsi_qla_host *vha; /* HA the SP is queued on */
struct req_que *que;
struct fc_port *fcport;
@@ -2136,7 +2135,6 @@
/* Work events. */
enum qla_work_type {
QLA_EVT_AEN,
- QLA_EVT_HWE_LOG,
};
@@ -2151,10 +2149,6 @@
enum fc_host_event_code code;
u32 data;
} aen;
- struct {
- uint16_t code;
- uint16_t d1, d2, d3;
- } hwe;
} u;
};
@@ -2309,6 +2303,7 @@
#define PORT_SPEED_2GB 0x01
#define PORT_SPEED_4GB 0x03
#define PORT_SPEED_8GB 0x04
+#define PORT_SPEED_10GB 0x13
uint16_t link_data_rate; /* F/W operating speed */
uint8_t current_topology;
@@ -2328,6 +2323,7 @@
#define PCI_DEVICE_ID_QLOGIC_ISP2532 0x2532
#define PCI_DEVICE_ID_QLOGIC_ISP8432 0x8432
+#define PCI_DEVICE_ID_QLOGIC_ISP8001 0x8001
uint32_t device_type;
#define DT_ISP2100 BIT_0
#define DT_ISP2200 BIT_1
@@ -2342,7 +2338,8 @@
#define DT_ISP5432 BIT_10
#define DT_ISP2532 BIT_11
#define DT_ISP8432 BIT_12
-#define DT_ISP_LAST (DT_ISP8432 << 1)
+#define DT_ISP8001 BIT_13
+#define DT_ISP_LAST (DT_ISP8001 << 1)
#define DT_IIDMA BIT_26
#define DT_FWI2 BIT_27
@@ -2364,6 +2361,7 @@
#define IS_QLA5432(ha) (DT_MASK(ha) & DT_ISP5432)
#define IS_QLA2532(ha) (DT_MASK(ha) & DT_ISP2532)
#define IS_QLA8432(ha) (DT_MASK(ha) & DT_ISP8432)
+#define IS_QLA8001(ha) (DT_MASK(ha) & DT_ISP8001)
#define IS_QLA23XX(ha) (IS_QLA2300(ha) || IS_QLA2312(ha) || IS_QLA2322(ha) || \
IS_QLA6312(ha) || IS_QLA6322(ha))
@@ -2373,8 +2371,11 @@
#define IS_QLA84XX(ha) (IS_QLA8432(ha))
#define IS_QLA24XX_TYPE(ha) (IS_QLA24XX(ha) || IS_QLA54XX(ha) || \
IS_QLA84XX(ha))
+#define IS_QLA81XX(ha) (IS_QLA8001(ha))
#define IS_QLA2XXX_MIDTYPE(ha) (IS_QLA24XX(ha) || IS_QLA84XX(ha) || \
- IS_QLA25XX(ha))
+ IS_QLA25XX(ha) || IS_QLA81XX(ha))
+#define IS_NOPOLLING_TYPE(ha) ((IS_QLA25XX(ha) || IS_QLA81XX(ha)) && \
+ (ha)->flags.msix_enabled)
#define IS_IIDMA_CAPABLE(ha) ((ha)->device_type & DT_IIDMA)
#define IS_FWI2_CAPABLE(ha) ((ha)->device_type & DT_FWI2)
@@ -2472,6 +2473,9 @@
uint8_t fw_seriallink_options[4];
uint16_t fw_seriallink_options24[4];
+ uint8_t mpi_version[4];
+ uint32_t mpi_capabilities;
+
/* Firmware dump information. */
struct qla2xxx_fw_dump *fw_dump;
uint32_t fw_dump_len;
@@ -2480,6 +2484,7 @@
dma_addr_t eft_dma;
void *eft;
+ uint32_t chain_offset;
struct dentry *dfs_dir;
struct dentry *dfs_fce;
dma_addr_t fce_dma;
@@ -2489,10 +2494,6 @@
uint64_t fce_wr, fce_rd;
struct mutex fce_mutex;
- uint32_t hw_event_start;
- uint32_t hw_event_ptr;
- uint32_t hw_event_pause_errors;
-
uint32_t pci_attr;
uint16_t chip_revision;
@@ -2522,6 +2523,12 @@
uint8_t fcode_revision[16];
uint32_t fw_revision[4];
+ /* Offsets for flash/nvram access (set to ~0 if not used). */
+ uint32_t flash_conf_off;
+ uint32_t flash_data_off;
+ uint32_t nvram_conf_off;
+ uint32_t nvram_data_off;
+
uint32_t fdt_wrt_disable;
uint32_t fdt_erase_cmd;
uint32_t fdt_block_size;
@@ -2533,7 +2540,6 @@
uint32_t flt_region_boot;
uint32_t flt_region_fw;
uint32_t flt_region_vpd_nvram;
- uint32_t flt_region_hw_event;
uint32_t flt_region_npiv_conf;
/* Needed for BEACON */
@@ -2737,6 +2743,7 @@
#define OPTROM_SIZE_2322 0x100000
#define OPTROM_SIZE_24XX 0x100000
#define OPTROM_SIZE_25XX 0x200000
+#define OPTROM_SIZE_81XX 0x400000
#include "qla_gbl.h"
#include "qla_dbg.h"
diff --git a/drivers/scsi/qla2xxx/qla_dfs.c b/drivers/scsi/qla2xxx/qla_dfs.c
index 0e366a1..c66036d 100644
--- a/drivers/scsi/qla2xxx/qla_dfs.c
+++ b/drivers/scsi/qla2xxx/qla_dfs.c
@@ -113,7 +113,8 @@
qla2x00_dfs_setup(scsi_qla_host_t *vha)
{
struct qla_hw_data *ha = vha->hw;
- if (!IS_QLA25XX(ha))
+
+ if (!IS_QLA25XX(ha) && !IS_QLA81XX(ha))
goto out;
if (!ha->fce)
goto out;
diff --git a/drivers/scsi/qla2xxx/qla_fw.h b/drivers/scsi/qla2xxx/qla_fw.h
index ee1f1e7..7abb045 100644
--- a/drivers/scsi/qla2xxx/qla_fw.h
+++ b/drivers/scsi/qla2xxx/qla_fw.h
@@ -1215,9 +1215,10 @@
struct qla_flt_location {
uint8_t sig[4];
- uint32_t start_lo;
- uint32_t start_hi;
- uint16_t unused;
+ uint16_t start_lo;
+ uint16_t start_hi;
+ uint8_t version;
+ uint8_t unused[5];
uint16_t checksum;
};
@@ -1390,4 +1391,291 @@
uint32_t reserved[12];
};
+
+/* 81XX Support **************************************************************/
+
+#define MBA_DCBX_START 0x8016
+#define MBA_DCBX_COMPLETE 0x8030
+#define MBA_FCF_CONF_ERR 0x8031
+#define MBA_DCBX_PARAM_UPDATE 0x8032
+#define MBA_IDC_COMPLETE 0x8100
+#define MBA_IDC_NOTIFY 0x8101
+#define MBA_IDC_TIME_EXT 0x8102
+
+struct nvram_81xx {
+ /* NVRAM header. */
+ uint8_t id[4];
+ uint16_t nvram_version;
+ uint16_t reserved_0;
+
+ /* Firmware Initialization Control Block. */
+ uint16_t version;
+ uint16_t reserved_1;
+ uint16_t frame_payload_size;
+ uint16_t execution_throttle;
+ uint16_t exchange_count;
+ uint16_t reserved_2;
+
+ uint8_t port_name[WWN_SIZE];
+ uint8_t node_name[WWN_SIZE];
+
+ uint16_t login_retry_count;
+ uint16_t reserved_3;
+ uint16_t interrupt_delay_timer;
+ uint16_t login_timeout;
+
+ uint32_t firmware_options_1;
+ uint32_t firmware_options_2;
+ uint32_t firmware_options_3;
+
+ uint16_t reserved_4[4];
+
+ /* Offset 64. */
+ uint8_t enode_mac[6];
+ uint16_t reserved_5[5];
+
+ /* Offset 80. */
+ uint16_t reserved_6[24];
+
+ /* Offset 128. */
+ uint16_t reserved_7[64];
+
+ /*
+ * BIT 0 = Enable spinup delay
+ * BIT 1 = Disable BIOS
+ * BIT 2 = Enable Memory Map BIOS
+ * BIT 3 = Enable Selectable Boot
+ * BIT 4 = Disable RISC code load
+ * BIT 5 = Disable Serdes
+ * BIT 6 = Opt boot mode
+ * BIT 7 = Interrupt enable
+ *
+ * BIT 8 = EV Control enable
+ * BIT 9 = Enable lip reset
+ * BIT 10 = Enable lip full login
+ * BIT 11 = Enable target reset
+ * BIT 12 = Stop firmware
+ * BIT 13 = Enable nodename option
+ * BIT 14 = Default WWPN valid
+ * BIT 15 = Enable alternate WWN
+ *
+ * BIT 16 = CLP LUN string
+ * BIT 17 = CLP Target string
+ * BIT 18 = CLP BIOS enable string
+ * BIT 19 = CLP Serdes string
+ * BIT 20 = CLP WWPN string
+ * BIT 21 = CLP WWNN string
+ * BIT 22 =
+ * BIT 23 =
+ * BIT 24 = Keep WWPN
+ * BIT 25 = Temp WWPN
+ * BIT 26-31 =
+ */
+ uint32_t host_p;
+
+ uint8_t alternate_port_name[WWN_SIZE];
+ uint8_t alternate_node_name[WWN_SIZE];
+
+ uint8_t boot_port_name[WWN_SIZE];
+ uint16_t boot_lun_number;
+ uint16_t reserved_8;
+
+ uint8_t alt1_boot_port_name[WWN_SIZE];
+ uint16_t alt1_boot_lun_number;
+ uint16_t reserved_9;
+
+ uint8_t alt2_boot_port_name[WWN_SIZE];
+ uint16_t alt2_boot_lun_number;
+ uint16_t reserved_10;
+
+ uint8_t alt3_boot_port_name[WWN_SIZE];
+ uint16_t alt3_boot_lun_number;
+ uint16_t reserved_11;
+
+ /*
+ * BIT 0 = Selective Login
+ * BIT 1 = Alt-Boot Enable
+ * BIT 2 = Reserved
+ * BIT 3 = Boot Order List
+ * BIT 4 = Reserved
+ * BIT 5 = Selective LUN
+ * BIT 6 = Reserved
+ * BIT 7-31 =
+ */
+ uint32_t efi_parameters;
+
+ uint8_t reset_delay;
+ uint8_t reserved_12;
+ uint16_t reserved_13;
+
+ uint16_t boot_id_number;
+ uint16_t reserved_14;
+
+ uint16_t max_luns_per_target;
+ uint16_t reserved_15;
+
+ uint16_t port_down_retry_count;
+ uint16_t link_down_timeout;
+
+ /* FCode parameters. */
+ uint16_t fcode_parameter;
+
+ uint16_t reserved_16[3];
+
+ /* Offset 352. */
+ uint8_t reserved_17[4];
+ uint16_t reserved_18[5];
+ uint8_t reserved_19[2];
+ uint16_t reserved_20[8];
+
+ /* Offset 384. */
+ uint8_t reserved_21[16];
+ uint16_t reserved_22[8];
+
+ /* Offset 416. */
+ uint16_t reserved_23[32];
+
+ /* Offset 480. */
+ uint8_t model_name[16];
+
+ /* Offset 496. */
+ uint16_t feature_mask_l;
+ uint16_t feature_mask_h;
+ uint16_t reserved_24[2];
+
+ uint16_t subsystem_vendor_id;
+ uint16_t subsystem_device_id;
+
+ uint32_t checksum;
+};
+
+/*
+ * ISP Initialization Control Block.
+ * Little endian except where noted.
+ */
+#define ICB_VERSION 1
+struct init_cb_81xx {
+ uint16_t version;
+ uint16_t reserved_1;
+
+ uint16_t frame_payload_size;
+ uint16_t execution_throttle;
+ uint16_t exchange_count;
+
+ uint16_t reserved_2;
+
+ uint8_t port_name[WWN_SIZE]; /* Big endian. */
+ uint8_t node_name[WWN_SIZE]; /* Big endian. */
+
+ uint16_t response_q_inpointer;
+ uint16_t request_q_outpointer;
+
+ uint16_t login_retry_count;
+
+ uint16_t prio_request_q_outpointer;
+
+ uint16_t response_q_length;
+ uint16_t request_q_length;
+
+ uint16_t reserved_3;
+
+ uint16_t prio_request_q_length;
+
+ uint32_t request_q_address[2];
+ uint32_t response_q_address[2];
+ uint32_t prio_request_q_address[2];
+
+ uint8_t reserved_4[8];
+
+ uint16_t atio_q_inpointer;
+ uint16_t atio_q_length;
+ uint32_t atio_q_address[2];
+
+ uint16_t interrupt_delay_timer; /* 100us increments. */
+ uint16_t login_timeout;
+
+ /*
+ * BIT 0-3 = Reserved
+ * BIT 4 = Enable Target Mode
+ * BIT 5 = Disable Initiator Mode
+ * BIT 6 = Reserved
+ * BIT 7 = Reserved
+ *
+ * BIT 8-13 = Reserved
+ * BIT 14 = Node Name Option
+ * BIT 15-31 = Reserved
+ */
+ uint32_t firmware_options_1;
+
+ /*
+ * BIT 0 = Operation Mode bit 0
+ * BIT 1 = Operation Mode bit 1
+ * BIT 2 = Operation Mode bit 2
+ * BIT 3 = Operation Mode bit 3
+ * BIT 4-7 = Reserved
+ *
+ * BIT 8 = Enable Class 2
+ * BIT 9 = Enable ACK0
+ * BIT 10 = Reserved
+ * BIT 11 = Enable FC-SP Security
+ * BIT 12 = FC Tape Enable
+ * BIT 13 = Reserved
+ * BIT 14 = Enable Target PRLI Control
+ * BIT 15-31 = Reserved
+ */
+ uint32_t firmware_options_2;
+
+ /*
+ * BIT 0-3 = Reserved
+ * BIT 4 = FCP RSP Payload bit 0
+ * BIT 5 = FCP RSP Payload bit 1
+ * BIT 6 = Enable Receive Out-of-Order data frame handling
+ * BIT 7 = Reserved
+ *
+ * BIT 8 = Reserved
+ * BIT 9 = Enable Out-of-Order FCP_XFER_RDY relative offset handling
+ * BIT 10-16 = Reserved
+ * BIT 17 = Enable multiple FCFs
+ * BIT 18-20 = MAC addressing mode
+ * BIT 21-25 = Ethernet data rate
+ * BIT 26 = Enable ethernet header rx IOCB for ATIO q
+ * BIT 27 = Enable ethernet header rx IOCB for response q
+ * BIT 28 = SPMA selection bit 0
+ * BIT 28 = SPMA selection bit 1
+ * BIT 30-31 = Reserved
+ */
+ uint32_t firmware_options_3;
+
+ uint8_t reserved_5[8];
+
+ uint8_t enode_mac[6];
+
+ uint8_t reserved_6[10];
+};
+
+struct mid_init_cb_81xx {
+ struct init_cb_81xx init_cb;
+
+ uint16_t count;
+ uint16_t options;
+
+ struct mid_conf_entry_24xx entries[MAX_MULTI_ID_FABRIC];
+};
+
+#define FARX_ACCESS_FLASH_CONF_81XX 0x7FFD0000
+#define FARX_ACCESS_FLASH_DATA_81XX 0x7F800000
+
+/* 81XX Flash locations -- occupies second 2MB region. */
+#define FA_BOOT_CODE_ADDR_81 0x80000
+#define FA_RISC_CODE_ADDR_81 0xA0000
+#define FA_FW_AREA_ADDR_81 0xC0000
+#define FA_VPD_NVRAM_ADDR_81 0xD0000
+#define FA_FEATURE_ADDR_81 0xD4000
+#define FA_FLASH_DESCR_ADDR_81 0xD8000
+#define FA_FLASH_LAYOUT_ADDR_81 0xD8400
+#define FA_HW_EVENT0_ADDR_81 0xDC000
+#define FA_HW_EVENT1_ADDR_81 0xDC400
+#define FA_NPIV_CONF0_ADDR_81 0xD1000
+#define FA_NPIV_CONF1_ADDR_81 0xD2000
+
#endif
diff --git a/drivers/scsi/qla2xxx/qla_gbl.h b/drivers/scsi/qla2xxx/qla_gbl.h
index 0011e31..ba49133 100644
--- a/drivers/scsi/qla2xxx/qla_gbl.h
+++ b/drivers/scsi/qla2xxx/qla_gbl.h
@@ -28,8 +28,10 @@
extern void qla24xx_reset_adapter(struct scsi_qla_host *);
extern int qla2x00_nvram_config(struct scsi_qla_host *);
extern int qla24xx_nvram_config(struct scsi_qla_host *);
+extern int qla81xx_nvram_config(struct scsi_qla_host *);
extern void qla2x00_update_fw_options(struct scsi_qla_host *);
extern void qla24xx_update_fw_options(scsi_qla_host_t *);
+extern void qla81xx_update_fw_options(scsi_qla_host_t *);
extern int qla2x00_load_risc(struct scsi_qla_host *, uint32_t *);
extern int qla24xx_load_risc(scsi_qla_host_t *, uint32_t *);
@@ -69,8 +71,6 @@
extern void qla2x00_abort_all_cmds(scsi_qla_host_t *, int);
extern int qla2x00_post_aen_work(struct scsi_qla_host *, enum
fc_host_event_code, u32);
-extern int qla2x00_post_hwe_work(struct scsi_qla_host *, uint16_t , uint16_t,
- uint16_t, uint16_t);
extern void qla2x00_abort_fcport_cmds(fc_port_t *);
extern struct scsi_qla_host *qla2x00_create_host(struct scsi_host_template *,
@@ -143,7 +143,7 @@
extern void
qla2x00_get_fw_version(scsi_qla_host_t *, uint16_t *,
- uint16_t *, uint16_t *, uint16_t *, uint32_t *);
+ uint16_t *, uint16_t *, uint16_t *, uint32_t *, uint8_t *, uint32_t *);
extern int
qla2x00_get_fw_options(scsi_qla_host_t *, uint16_t *);
@@ -317,9 +317,6 @@
extern int qla2x00_get_flash_version(scsi_qla_host_t *, void *);
extern int qla24xx_get_flash_version(scsi_qla_host_t *, void *);
-extern int qla2xxx_hw_event_log(scsi_qla_host_t *, uint16_t , uint16_t,
- uint16_t, uint16_t);
-
extern int qla2xxx_get_flash_info(scsi_qla_host_t *);
extern int qla2xxx_get_vpd_field(scsi_qla_host_t *, char *, char *, size_t);
@@ -332,6 +329,7 @@
extern void qla2300_fw_dump(scsi_qla_host_t *, int);
extern void qla24xx_fw_dump(scsi_qla_host_t *, int);
extern void qla25xx_fw_dump(scsi_qla_host_t *, int);
+extern void qla81xx_fw_dump(scsi_qla_host_t *, int);
extern void qla2x00_dump_regs(scsi_qla_host_t *);
extern void qla2x00_dump_buffer(uint8_t *, uint32_t);
diff --git a/drivers/scsi/qla2xxx/qla_gs.c b/drivers/scsi/qla2xxx/qla_gs.c
index 0a6f7297..557f58d 100644
--- a/drivers/scsi/qla2xxx/qla_gs.c
+++ b/drivers/scsi/qla2xxx/qla_gs.c
@@ -1535,7 +1535,10 @@
eiter = (struct ct_fdmi_port_attr *) (entries + size);
eiter->type = __constant_cpu_to_be16(FDMI_PORT_SUPPORT_SPEED);
eiter->len = __constant_cpu_to_be16(4 + 4);
- if (IS_QLA25XX(ha))
+ if (IS_QLA81XX(ha))
+ eiter->a.sup_speed = __constant_cpu_to_be32(
+ FDMI_PORT_SPEED_10GB);
+ else if (IS_QLA25XX(ha))
eiter->a.sup_speed = __constant_cpu_to_be32(
FDMI_PORT_SPEED_1GB|FDMI_PORT_SPEED_2GB|
FDMI_PORT_SPEED_4GB|FDMI_PORT_SPEED_8GB);
@@ -1575,6 +1578,10 @@
eiter->a.cur_speed =
__constant_cpu_to_be32(FDMI_PORT_SPEED_8GB);
break;
+ case PORT_SPEED_10GB:
+ eiter->a.cur_speed =
+ __constant_cpu_to_be32(FDMI_PORT_SPEED_10GB);
+ break;
default:
eiter->a.cur_speed =
__constant_cpu_to_be32(FDMI_PORT_SPEED_UNKNOWN);
diff --git a/drivers/scsi/qla2xxx/qla_init.c b/drivers/scsi/qla2xxx/qla_init.c
index 52ed56e..2d4f32b 100644
--- a/drivers/scsi/qla2xxx/qla_init.c
+++ b/drivers/scsi/qla2xxx/qla_init.c
@@ -552,10 +552,6 @@
d2 = RD_REG_DWORD(®->ctrl_status);
barrier();
}
- if (cnt == 0 || hw_evt)
- qla2xxx_hw_event_log(vha, HW_EVENT_RESET_ERR,
- RD_REG_WORD(®->mailbox1), RD_REG_WORD(®->mailbox2),
- RD_REG_WORD(®->mailbox3));
WRT_REG_DWORD(®->hccr, HCCRX_SET_RISC_RESET);
RD_REG_DWORD(®->hccr);
@@ -574,6 +570,9 @@
}
spin_unlock_irqrestore(&ha->hardware_lock, flags);
+
+ if (IS_NOPOLLING_TYPE(ha))
+ ha->isp_ops->enable_intrs(ha);
}
/**
@@ -779,16 +778,19 @@
mem_size = (ha->fw_memory_size - 0x11000 + 1) *
sizeof(uint16_t);
} else if (IS_FWI2_CAPABLE(ha)) {
- fixed_size = IS_QLA25XX(ha) ?
- offsetof(struct qla25xx_fw_dump, ext_mem) :
- offsetof(struct qla24xx_fw_dump, ext_mem);
+ if (IS_QLA81XX(ha))
+ fixed_size = offsetof(struct qla81xx_fw_dump, ext_mem);
+ else if (IS_QLA25XX(ha))
+ fixed_size = offsetof(struct qla25xx_fw_dump, ext_mem);
+ else
+ fixed_size = offsetof(struct qla24xx_fw_dump, ext_mem);
mem_size = (ha->fw_memory_size - 0x100000 + 1) *
sizeof(uint32_t);
if (ha->mqenable)
mq_size = sizeof(struct qla2xxx_mq_chain);
/* Allocate memory for Fibre Channel Event Buffer. */
- if (!IS_QLA25XX(ha))
+ if (!IS_QLA25XX(ha) && !IS_QLA81XX(ha))
goto try_eft;
tc = dma_alloc_coherent(&ha->pdev->dev, FCE_SIZE, &tc_dma,
@@ -851,7 +853,9 @@
dump_size = offsetof(struct qla2xxx_fw_dump, isp);
dump_size += fixed_size + mem_size + req_q_size + rsp_q_size +
- mq_size + eft_size + fce_size;
+ eft_size;
+ ha->chain_offset = dump_size;
+ dump_size += mq_size + fce_size;
ha->fw_dump = vmalloc(dump_size);
if (!ha->fw_dump) {
@@ -987,7 +991,8 @@
&ha->fw_major_version,
&ha->fw_minor_version,
&ha->fw_subminor_version,
- &ha->fw_attributes, &ha->fw_memory_size);
+ &ha->fw_attributes, &ha->fw_memory_size,
+ ha->mpi_version, &ha->mpi_capabilities);
ha->flags.npiv_supported = 0;
if (IS_QLA2XXX_MIDTYPE(ha) &&
(ha->fw_attributes & BIT_2)) {
@@ -1665,10 +1670,6 @@
qla_printk(KERN_WARNING, ha, "Falling back to functioning (yet "
"invalid -- WWPN) defaults.\n");
- if (chksum)
- qla2xxx_hw_event_log(vha, HW_EVENT_NVRAM_CHKSUM_ERR, 0,
- MSW(chksum), LSW(chksum));
-
/*
* Set default initialization control block.
*/
@@ -4255,3 +4256,269 @@
return rval != QLA_SUCCESS || status[0] ? QLA_FUNCTION_FAILED:
QLA_SUCCESS;
}
+
+/* 81XX Support **************************************************************/
+
+int
+qla81xx_nvram_config(scsi_qla_host_t *vha)
+{
+ int rval;
+ struct init_cb_81xx *icb;
+ struct nvram_81xx *nv;
+ uint32_t *dptr;
+ uint8_t *dptr1, *dptr2;
+ uint32_t chksum;
+ uint16_t cnt;
+ struct qla_hw_data *ha = vha->hw;
+
+ rval = QLA_SUCCESS;
+ icb = (struct init_cb_81xx *)ha->init_cb;
+ nv = ha->nvram;
+
+ /* Determine NVRAM starting address. */
+ ha->nvram_size = sizeof(struct nvram_81xx);
+ ha->nvram_base = FA_NVRAM_FUNC0_ADDR;
+ ha->vpd_size = FA_NVRAM_VPD_SIZE;
+ ha->vpd_base = FA_NVRAM_VPD0_ADDR;
+ if (PCI_FUNC(ha->pdev->devfn) & 1) {
+ ha->nvram_base = FA_NVRAM_FUNC1_ADDR;
+ ha->vpd_base = FA_NVRAM_VPD1_ADDR;
+ }
+
+ /* Get VPD data into cache */
+ ha->vpd = ha->nvram + VPD_OFFSET;
+ ha->isp_ops->read_nvram(vha, (uint8_t *)ha->vpd,
+ ha->nvram_base - FA_NVRAM_FUNC0_ADDR, FA_NVRAM_VPD_SIZE * 4);
+
+ /* Get NVRAM data into cache and calculate checksum. */
+ dptr = (uint32_t *)nv;
+ ha->isp_ops->read_nvram(vha, (uint8_t *)dptr, ha->nvram_base,
+ ha->nvram_size);
+ for (cnt = 0, chksum = 0; cnt < ha->nvram_size >> 2; cnt++)
+ chksum += le32_to_cpu(*dptr++);
+
+ DEBUG5(printk("scsi(%ld): Contents of NVRAM\n", ha->host_no));
+ DEBUG5(qla2x00_dump_buffer((uint8_t *)nv, ha->nvram_size));
+
+ /* Bad NVRAM data, set defaults parameters. */
+ if (chksum || nv->id[0] != 'I' || nv->id[1] != 'S' || nv->id[2] != 'P'
+ || nv->id[3] != ' ' ||
+ nv->nvram_version < __constant_cpu_to_le16(ICB_VERSION)) {
+ /* Reset NVRAM data. */
+ qla_printk(KERN_WARNING, ha, "Inconsistent NVRAM detected: "
+ "checksum=0x%x id=%c version=0x%x.\n", chksum, nv->id[0],
+ le16_to_cpu(nv->nvram_version));
+ qla_printk(KERN_WARNING, ha, "Falling back to functioning (yet "
+ "invalid -- WWPN) defaults.\n");
+
+ /*
+ * Set default initialization control block.
+ */
+ memset(nv, 0, ha->nvram_size);
+ nv->nvram_version = __constant_cpu_to_le16(ICB_VERSION);
+ nv->version = __constant_cpu_to_le16(ICB_VERSION);
+ nv->frame_payload_size = __constant_cpu_to_le16(2048);
+ nv->execution_throttle = __constant_cpu_to_le16(0xFFFF);
+ nv->exchange_count = __constant_cpu_to_le16(0);
+ nv->port_name[0] = 0x21;
+ nv->port_name[1] = 0x00 + PCI_FUNC(ha->pdev->devfn);
+ nv->port_name[2] = 0x00;
+ nv->port_name[3] = 0xe0;
+ nv->port_name[4] = 0x8b;
+ nv->port_name[5] = 0x1c;
+ nv->port_name[6] = 0x55;
+ nv->port_name[7] = 0x86;
+ nv->node_name[0] = 0x20;
+ nv->node_name[1] = 0x00;
+ nv->node_name[2] = 0x00;
+ nv->node_name[3] = 0xe0;
+ nv->node_name[4] = 0x8b;
+ nv->node_name[5] = 0x1c;
+ nv->node_name[6] = 0x55;
+ nv->node_name[7] = 0x86;
+ nv->login_retry_count = __constant_cpu_to_le16(8);
+ nv->interrupt_delay_timer = __constant_cpu_to_le16(0);
+ nv->login_timeout = __constant_cpu_to_le16(0);
+ nv->firmware_options_1 =
+ __constant_cpu_to_le32(BIT_14|BIT_13|BIT_2|BIT_1);
+ nv->firmware_options_2 = __constant_cpu_to_le32(2 << 4);
+ nv->firmware_options_2 |= __constant_cpu_to_le32(BIT_12);
+ nv->firmware_options_3 = __constant_cpu_to_le32(2 << 13);
+ nv->host_p = __constant_cpu_to_le32(BIT_11|BIT_10);
+ nv->efi_parameters = __constant_cpu_to_le32(0);
+ nv->reset_delay = 5;
+ nv->max_luns_per_target = __constant_cpu_to_le16(128);
+ nv->port_down_retry_count = __constant_cpu_to_le16(30);
+ nv->link_down_timeout = __constant_cpu_to_le16(30);
+ nv->enode_mac[0] = 0x01;
+ nv->enode_mac[1] = 0x02;
+ nv->enode_mac[2] = 0x03;
+ nv->enode_mac[3] = 0x04;
+ nv->enode_mac[4] = 0x05;
+ nv->enode_mac[5] = 0x06 + PCI_FUNC(ha->pdev->devfn);
+
+ rval = 1;
+ }
+
+ /* Reset Initialization control block */
+ memset(icb, 0, sizeof(struct init_cb_81xx));
+
+ /* Copy 1st segment. */
+ dptr1 = (uint8_t *)icb;
+ dptr2 = (uint8_t *)&nv->version;
+ cnt = (uint8_t *)&icb->response_q_inpointer - (uint8_t *)&icb->version;
+ while (cnt--)
+ *dptr1++ = *dptr2++;
+
+ icb->login_retry_count = nv->login_retry_count;
+
+ /* Copy 2nd segment. */
+ dptr1 = (uint8_t *)&icb->interrupt_delay_timer;
+ dptr2 = (uint8_t *)&nv->interrupt_delay_timer;
+ cnt = (uint8_t *)&icb->reserved_5 -
+ (uint8_t *)&icb->interrupt_delay_timer;
+ while (cnt--)
+ *dptr1++ = *dptr2++;
+
+ memcpy(icb->enode_mac, nv->enode_mac, sizeof(icb->enode_mac));
+ /* Some boards (with valid NVRAMs) still have NULL enode_mac!! */
+ if (!memcmp(icb->enode_mac, "\0\0\0\0\0\0", sizeof(icb->enode_mac))) {
+ icb->enode_mac[0] = 0x01;
+ icb->enode_mac[1] = 0x02;
+ icb->enode_mac[2] = 0x03;
+ icb->enode_mac[3] = 0x04;
+ icb->enode_mac[4] = 0x05;
+ icb->enode_mac[5] = 0x06 + PCI_FUNC(ha->pdev->devfn);
+ }
+
+ /*
+ * Setup driver NVRAM options.
+ */
+ qla2x00_set_model_info(vha, nv->model_name, sizeof(nv->model_name),
+ "QLE81XX");
+
+ /* Use alternate WWN? */
+ if (nv->host_p & __constant_cpu_to_le32(BIT_15)) {
+ memcpy(icb->node_name, nv->alternate_node_name, WWN_SIZE);
+ memcpy(icb->port_name, nv->alternate_port_name, WWN_SIZE);
+ }
+
+ /* Prepare nodename */
+ if ((icb->firmware_options_1 & __constant_cpu_to_le32(BIT_14)) == 0) {
+ /*
+ * Firmware will apply the following mask if the nodename was
+ * not provided.
+ */
+ memcpy(icb->node_name, icb->port_name, WWN_SIZE);
+ icb->node_name[0] &= 0xF0;
+ }
+
+ /* Set host adapter parameters. */
+ ha->flags.disable_risc_code_load = 0;
+ ha->flags.enable_lip_reset = 0;
+ ha->flags.enable_lip_full_login =
+ le32_to_cpu(nv->host_p) & BIT_10 ? 1: 0;
+ ha->flags.enable_target_reset =
+ le32_to_cpu(nv->host_p) & BIT_11 ? 1: 0;
+ ha->flags.enable_led_scheme = 0;
+ ha->flags.disable_serdes = le32_to_cpu(nv->host_p) & BIT_5 ? 1: 0;
+
+ ha->operating_mode = (le32_to_cpu(icb->firmware_options_2) &
+ (BIT_6 | BIT_5 | BIT_4)) >> 4;
+
+ /* save HBA serial number */
+ ha->serial0 = icb->port_name[5];
+ ha->serial1 = icb->port_name[6];
+ ha->serial2 = icb->port_name[7];
+ memcpy(vha->node_name, icb->node_name, WWN_SIZE);
+ memcpy(vha->port_name, icb->port_name, WWN_SIZE);
+
+ icb->execution_throttle = __constant_cpu_to_le16(0xFFFF);
+
+ ha->retry_count = le16_to_cpu(nv->login_retry_count);
+
+ /* Set minimum login_timeout to 4 seconds. */
+ if (le16_to_cpu(nv->login_timeout) < ql2xlogintimeout)
+ nv->login_timeout = cpu_to_le16(ql2xlogintimeout);
+ if (le16_to_cpu(nv->login_timeout) < 4)
+ nv->login_timeout = __constant_cpu_to_le16(4);
+ ha->login_timeout = le16_to_cpu(nv->login_timeout);
+ icb->login_timeout = nv->login_timeout;
+
+ /* Set minimum RATOV to 100 tenths of a second. */
+ ha->r_a_tov = 100;
+
+ ha->loop_reset_delay = nv->reset_delay;
+
+ /* Link Down Timeout = 0:
+ *
+ * When Port Down timer expires we will start returning
+ * I/O's to OS with "DID_NO_CONNECT".
+ *
+ * Link Down Timeout != 0:
+ *
+ * The driver waits for the link to come up after link down
+ * before returning I/Os to OS with "DID_NO_CONNECT".
+ */
+ if (le16_to_cpu(nv->link_down_timeout) == 0) {
+ ha->loop_down_abort_time =
+ (LOOP_DOWN_TIME - LOOP_DOWN_TIMEOUT);
+ } else {
+ ha->link_down_timeout = le16_to_cpu(nv->link_down_timeout);
+ ha->loop_down_abort_time =
+ (LOOP_DOWN_TIME - ha->link_down_timeout);
+ }
+
+ /* Need enough time to try and get the port back. */
+ ha->port_down_retry_count = le16_to_cpu(nv->port_down_retry_count);
+ if (qlport_down_retry)
+ ha->port_down_retry_count = qlport_down_retry;
+
+ /* Set login_retry_count */
+ ha->login_retry_count = le16_to_cpu(nv->login_retry_count);
+ if (ha->port_down_retry_count ==
+ le16_to_cpu(nv->port_down_retry_count) &&
+ ha->port_down_retry_count > 3)
+ ha->login_retry_count = ha->port_down_retry_count;
+ else if (ha->port_down_retry_count > (int)ha->login_retry_count)
+ ha->login_retry_count = ha->port_down_retry_count;
+ if (ql2xloginretrycount)
+ ha->login_retry_count = ql2xloginretrycount;
+
+ /* Enable ZIO. */
+ if (!vha->flags.init_done) {
+ ha->zio_mode = le32_to_cpu(icb->firmware_options_2) &
+ (BIT_3 | BIT_2 | BIT_1 | BIT_0);
+ ha->zio_timer = le16_to_cpu(icb->interrupt_delay_timer) ?
+ le16_to_cpu(icb->interrupt_delay_timer): 2;
+ }
+ icb->firmware_options_2 &= __constant_cpu_to_le32(
+ ~(BIT_3 | BIT_2 | BIT_1 | BIT_0));
+ vha->flags.process_response_queue = 0;
+ if (ha->zio_mode != QLA_ZIO_DISABLED) {
+ ha->zio_mode = QLA_ZIO_MODE_6;
+
+ DEBUG2(printk("scsi(%ld): ZIO mode %d enabled; timer delay "
+ "(%d us).\n", vha->host_no, ha->zio_mode,
+ ha->zio_timer * 100));
+ qla_printk(KERN_INFO, ha,
+ "ZIO mode %d enabled; timer delay (%d us).\n",
+ ha->zio_mode, ha->zio_timer * 100);
+
+ icb->firmware_options_2 |= cpu_to_le32(
+ (uint32_t)ha->zio_mode);
+ icb->interrupt_delay_timer = cpu_to_le16(ha->zio_timer);
+ vha->flags.process_response_queue = 1;
+ }
+
+ if (rval) {
+ DEBUG2_3(printk(KERN_WARNING
+ "scsi(%ld): NVRAM configuration failed!\n", vha->host_no));
+ }
+ return (rval);
+}
+
+void
+qla81xx_update_fw_options(scsi_qla_host_t *ha)
+{
+}
diff --git a/drivers/scsi/qla2xxx/qla_iocb.c b/drivers/scsi/qla2xxx/qla_iocb.c
index 5bedc9d..2258152 100644
--- a/drivers/scsi/qla2xxx/qla_iocb.c
+++ b/drivers/scsi/qla2xxx/qla_iocb.c
@@ -173,7 +173,7 @@
return;
}
- vha = sp->vha;
+ vha = sp->fcport->vha;
req = sp->que;
cmd_pkt->control_flags |= cpu_to_le16(qla2x00_get_cmd_direction(sp));
@@ -234,7 +234,7 @@
return;
}
- vha = sp->vha;
+ vha = sp->fcport->vha;
req = sp->que;
cmd_pkt->control_flags |= cpu_to_le16(qla2x00_get_cmd_direction(sp));
@@ -294,7 +294,7 @@
/* Setup device pointers. */
ret = 0;
- vha = sp->vha;
+ vha = sp->fcport->vha;
ha = vha->hw;
reg = &ha->iobase->isp;
cmd = sp->cmd;
@@ -353,7 +353,6 @@
/* Build command packet */
req->current_outstanding_cmd = handle;
req->outstanding_cmds[handle] = sp;
- sp->vha = vha;
sp->que = req;
sp->cmd->host_scribble = (unsigned char *)(unsigned long)handle;
req->cnt -= req_cnt;
@@ -656,7 +655,7 @@
return;
}
- vha = sp->vha;
+ vha = sp->fcport->vha;
req = sp->que;
/* Set transfer direction */
@@ -723,7 +722,7 @@
struct req_que *req = NULL;
struct rsp_que *rsp = NULL;
struct scsi_cmnd *cmd = sp->cmd;
- struct scsi_qla_host *vha = sp->vha;
+ struct scsi_qla_host *vha = sp->fcport->vha;
struct qla_hw_data *ha = vha->hw;
uint16_t que_id;
@@ -791,7 +790,6 @@
/* Build command packet. */
req->current_outstanding_cmd = handle;
req->outstanding_cmds[handle] = sp;
- sp->vha = vha;
sp->cmd->host_scribble = (unsigned char *)(unsigned long)handle;
req->cnt -= req_cnt;
diff --git a/drivers/scsi/qla2xxx/qla_isr.c b/drivers/scsi/qla2xxx/qla_isr.c
index d5fb79a..789fc57 100644
--- a/drivers/scsi/qla2xxx/qla_isr.c
+++ b/drivers/scsi/qla2xxx/qla_isr.c
@@ -275,7 +275,7 @@
qla2x00_async_event(scsi_qla_host_t *vha, struct rsp_que *rsp, uint16_t *mb)
{
#define LS_UNKNOWN 2
- static char *link_speeds[5] = { "1", "2", "?", "4", "8" };
+ static char *link_speeds[] = { "1", "2", "?", "4", "8", "10" };
char *link_speed;
uint16_t handle_cnt;
uint16_t cnt;
@@ -288,6 +288,8 @@
/* Setup to process RIO completion. */
handle_cnt = 0;
+ if (IS_QLA81XX(ha))
+ goto skip_rio;
switch (mb[0]) {
case MBA_SCSI_COMPLETION:
handles[0] = le32_to_cpu((uint32_t)((mb[2] << 16) | mb[1]));
@@ -339,7 +341,7 @@
default:
break;
}
-
+skip_rio:
switch (mb[0]) {
case MBA_SCSI_COMPLETION: /* Fast Post */
if (!vha->flags.online)
@@ -362,7 +364,6 @@
"ISP System Error - mbx1=%xh mbx2=%xh mbx3=%xh.\n",
mb[1], mb[2], mb[3]);
- qla2x00_post_hwe_work(vha, mb[0], mb[1], mb[2], mb[3]);
ha->isp_ops->fw_dump(vha, 1);
if (IS_FWI2_CAPABLE(ha)) {
@@ -387,7 +388,6 @@
vha->host_no));
qla_printk(KERN_WARNING, ha, "ISP Request Transfer Error.\n");
- qla2x00_post_hwe_work(vha, mb[0], mb[1], mb[2], mb[3]);
set_bit(ISP_ABORT_NEEDED, &vha->dpc_flags);
break;
@@ -396,7 +396,6 @@
vha->host_no));
qla_printk(KERN_WARNING, ha, "ISP Response Transfer Error.\n");
- qla2x00_post_hwe_work(vha, mb[0], mb[1], mb[2], mb[3]);
set_bit(ISP_ABORT_NEEDED, &vha->dpc_flags);
break;
@@ -436,6 +435,8 @@
link_speed = link_speeds[LS_UNKNOWN];
if (mb[1] < 5)
link_speed = link_speeds[mb[1]];
+ else if (mb[1] == 0x13)
+ link_speed = link_speeds[5];
ha->link_data_rate = mb[1];
}
@@ -495,12 +496,17 @@
qla2x00_post_aen_work(vha, FCH_EVT_LIPRESET, mb[1]);
break;
+ /* case MBA_DCBX_COMPLETE: */
case MBA_POINT_TO_POINT: /* Point-to-Point */
if (IS_QLA2100(ha))
break;
- DEBUG2(printk("scsi(%ld): Asynchronous P2P MODE received.\n",
- vha->host_no));
+ if (IS_QLA81XX(ha))
+ DEBUG2(printk("scsi(%ld): DCBX Completed -- %04x %04x "
+ "%04x\n", vha->host_no, mb[1], mb[2], mb[3]));
+ else
+ DEBUG2(printk("scsi(%ld): Asynchronous P2P MODE "
+ "received.\n", vha->host_no));
/*
* Until there's a transition from loop down to loop up, treat
@@ -641,10 +647,7 @@
/* case MBA_RIO_RESPONSE: */
case MBA_ZIO_RESPONSE:
- DEBUG2(printk("scsi(%ld): [R|Z]IO update completion.\n",
- vha->host_no));
- DEBUG(printk(KERN_INFO
- "scsi(%ld): [R|Z]IO update completion.\n",
+ DEBUG3(printk("scsi(%ld): [R|Z]IO update completion.\n",
vha->host_no));
if (IS_FWI2_CAPABLE(ha))
@@ -698,6 +701,35 @@
}
spin_unlock_irqrestore(&ha->cs84xx->access_lock, flags);
break;
+ case MBA_DCBX_START:
+ DEBUG2(printk("scsi(%ld): DCBX Started -- %04x %04x %04x\n",
+ vha->host_no, mb[1], mb[2], mb[3]));
+ break;
+ case MBA_DCBX_PARAM_UPDATE:
+ DEBUG2(printk("scsi(%ld): DCBX Parameters Updated -- "
+ "%04x %04x %04x\n", vha->host_no, mb[1], mb[2], mb[3]));
+ break;
+ case MBA_FCF_CONF_ERR:
+ DEBUG2(printk("scsi(%ld): FCF Configuration Error -- "
+ "%04x %04x %04x\n", vha->host_no, mb[1], mb[2], mb[3]));
+ break;
+ case MBA_IDC_COMPLETE:
+ DEBUG2(printk("scsi(%ld): Inter-Driver Commucation "
+ "Complete -- %04x %04x %04x\n", vha->host_no, mb[1], mb[2],
+ mb[3]));
+ break;
+ case MBA_IDC_NOTIFY:
+ DEBUG2(printk("scsi(%ld): Inter-Driver Commucation "
+ "Request Notification -- %04x %04x %04x\n", vha->host_no,
+ mb[1], mb[2], mb[3]));
+ /**** Mailbox registers 4 - 7 valid!!! */
+ break;
+ case MBA_IDC_TIME_EXT:
+ DEBUG2(printk("scsi(%ld): Inter-Driver Commucation "
+ "Time Extension -- %04x %04x %04x\n", vha->host_no, mb[1],
+ mb[2], mb[3]));
+ /**** Mailbox registers 4 - 7 valid!!! */
+ break;
}
if (!vha->vp_idx && ha->num_vhosts)
@@ -1510,7 +1542,7 @@
struct qla_hw_data *ha = vha->hw;
struct device_reg_24xx __iomem *reg = &ha->iobase->isp24;
- if (!IS_QLA25XX(ha))
+ if (!IS_QLA25XX(ha) && !IS_QLA81XX(ha))
return;
rval = QLA_SUCCESS;
@@ -1590,12 +1622,6 @@
if (pci_channel_offline(ha->pdev))
break;
- if (ha->hw_event_pause_errors == 0)
- qla2x00_post_hwe_work(vha, HW_EVENT_PARITY_ERR,
- 0, MSW(stat), LSW(stat));
- else if (ha->hw_event_pause_errors < 0xffffffff)
- ha->hw_event_pause_errors++;
-
hccr = RD_REG_DWORD(®->hccr);
qla_printk(KERN_INFO, ha, "RISC paused -- HCCR=%x, "
@@ -1740,12 +1766,6 @@
if (pci_channel_offline(ha->pdev))
break;
- if (ha->hw_event_pause_errors == 0)
- qla2x00_post_hwe_work(vha, HW_EVENT_PARITY_ERR,
- 0, MSW(stat), LSW(stat));
- else if (ha->hw_event_pause_errors < 0xffffffff)
- ha->hw_event_pause_errors++;
-
hccr = RD_REG_DWORD(®->hccr);
qla_printk(KERN_INFO, ha, "RISC paused -- HCCR=%x, "
@@ -1944,7 +1964,8 @@
device_reg_t __iomem *reg = ha->iobase;
/* If possible, enable MSI-X. */
- if (!IS_QLA2432(ha) && !IS_QLA2532(ha) && !IS_QLA8432(ha))
+ if (!IS_QLA2432(ha) && !IS_QLA2532(ha) &&
+ !IS_QLA8432(ha) && !IS_QLA8001(ha))
goto skip_msix;
if (IS_QLA2432(ha) && (ha->pdev->revision < QLA_MSIX_CHIP_REV_24XX ||
@@ -1979,7 +2000,8 @@
"MSI-X: Falling back-to INTa mode -- %d.\n", ret);
skip_msix:
- if (!IS_QLA24XX(ha) && !IS_QLA2532(ha) && !IS_QLA8432(ha))
+ if (!IS_QLA24XX(ha) && !IS_QLA2532(ha) && !IS_QLA8432(ha) &&
+ !IS_QLA8001(ha))
goto skip_msi;
ret = pci_enable_msi(ha->pdev);
@@ -2000,6 +2022,12 @@
ha->flags.inta_enabled = 1;
clear_risc_ints:
+ /*
+ * FIXME: Noted that 8014s were being dropped during NK testing.
+ * Timing deltas during MSI-X/INTa transitions?
+ */
+ if (IS_QLA81XX(ha))
+ goto fail;
spin_lock_irq(&ha->hardware_lock);
if (IS_FWI2_CAPABLE(ha)) {
WRT_REG_DWORD(®->isp24.hccr, HCCRX_CLR_HOST_INT);
@@ -2044,7 +2072,7 @@
if (pkt && pkt->handle < MAX_OUTSTANDING_COMMANDS) {
sp = req->outstanding_cmds[pkt->handle];
if (sp)
- vha = sp->vha;
+ vha = sp->fcport->vha;
}
}
if (!vha)
diff --git a/drivers/scsi/qla2xxx/qla_mbx.c b/drivers/scsi/qla2xxx/qla_mbx.c
index a99976f..db4df45 100644
--- a/drivers/scsi/qla2xxx/qla_mbx.c
+++ b/drivers/scsi/qla2xxx/qla_mbx.c
@@ -123,8 +123,7 @@
/* Wait for mbx cmd completion until timeout */
- if (!abort_active && io_lock_on) {
-
+ if ((!abort_active && io_lock_on) || IS_NOPOLLING_TYPE(ha)) {
set_bit(MBX_INTR_WAIT, &ha->mbx_cmd_flags);
if (IS_FWI2_CAPABLE(ha))
@@ -218,7 +217,7 @@
/* Clean up */
ha->mcp = NULL;
- if (abort_active || !io_lock_on) {
+ if ((abort_active || !io_lock_on) && !IS_NOPOLLING_TYPE(ha)) {
DEBUG11(printk("%s(%ld): checking for additional resp "
"interrupt.\n", __func__, base_vha->host_no));
@@ -412,7 +411,8 @@
*/
void
qla2x00_get_fw_version(scsi_qla_host_t *vha, uint16_t *major, uint16_t *minor,
- uint16_t *subminor, uint16_t *attributes, uint32_t *memory)
+ uint16_t *subminor, uint16_t *attributes, uint32_t *memory, uint8_t *mpi,
+ uint32_t *mpi_caps)
{
int rval;
mbx_cmd_t mc;
@@ -423,6 +423,8 @@
mcp->mb[0] = MBC_GET_FIRMWARE_VERSION;
mcp->out_mb = MBX_0;
mcp->in_mb = MBX_6|MBX_5|MBX_4|MBX_3|MBX_2|MBX_1|MBX_0;
+ if (IS_QLA81XX(vha->hw))
+ mcp->in_mb |= MBX_13|MBX_12|MBX_11|MBX_10;
mcp->flags = 0;
mcp->tov = MBX_TOV_SECONDS;
rval = qla2x00_mailbox_command(vha, mcp);
@@ -436,6 +438,13 @@
*memory = 0x1FFFF; /* Defaults to 128KB. */
else
*memory = (mcp->mb[5] << 16) | mcp->mb[4];
+ if (IS_QLA81XX(vha->hw)) {
+ mpi[0] = mcp->mb[10] >> 8;
+ mpi[1] = mcp->mb[10] & 0xff;
+ mpi[2] = mcp->mb[11] >> 8;
+ mpi[3] = mcp->mb[11] & 0xff;
+ *mpi_caps = (mcp->mb[12] << 16) | mcp->mb[13];
+ }
if (rval != QLA_SUCCESS) {
/*EMPTY*/
@@ -568,7 +577,6 @@
qla2x00_mbx_reg_test(scsi_qla_host_t *vha)
{
int rval;
- struct qla_hw_data *ha = vha->hw;
mbx_cmd_t mc;
mbx_cmd_t *mcp = &mc;
@@ -595,14 +603,6 @@
if (mcp->mb[5] != 0xA5A5 || mcp->mb[6] != 0x5A5A ||
mcp->mb[7] != 0x2525)
rval = QLA_FUNCTION_FAILED;
- if (rval == QLA_FUNCTION_FAILED) {
- struct device_reg_24xx __iomem *reg =
- &ha->iobase->isp24;
-
- qla2xxx_hw_event_log(vha, HW_EVENT_ISP_ERR, 0,
- LSW(RD_REG_DWORD(®->hccr)),
- LSW(RD_REG_DWORD(®->istatus)));
- }
}
if (rval != QLA_SUCCESS) {
@@ -1363,7 +1363,13 @@
DEBUG11(printk("%s(%ld): entered.\n", __func__, vha->host_no));
- if (IS_FWI2_CAPABLE(vha->hw)) {
+ if (IS_QLA81XX(vha->hw)) {
+ /* Logout across all FCFs. */
+ mcp->mb[0] = MBC_LIP_FULL_LOGIN;
+ mcp->mb[1] = BIT_1;
+ mcp->mb[2] = 0;
+ mcp->out_mb = MBX_2|MBX_1|MBX_0;
+ } else if (IS_FWI2_CAPABLE(vha->hw)) {
mcp->mb[0] = MBC_LIP_FULL_LOGIN;
mcp->mb[1] = BIT_6;
mcp->mb[2] = 0;
@@ -1853,6 +1859,9 @@
mbx_cmd_t mc;
mbx_cmd_t *mcp = &mc;
+ if (IS_QLA81XX(vha->hw))
+ return QLA_SUCCESS;
+
DEBUG11(printk("qla2x00_full_login_lip(%ld): entered.\n",
vha->host_no));
@@ -2512,7 +2521,7 @@
mbx_cmd_t mc;
mbx_cmd_t *mcp = &mc;
- if (!IS_QLA25XX(vha->hw))
+ if (!IS_QLA25XX(vha->hw) && !IS_QLA81XX(vha->hw))
return QLA_FUNCTION_FAILED;
DEBUG11(printk("%s(%ld): entered.\n", __func__, vha->host_no));
@@ -3155,7 +3164,7 @@
mcp->mb[7] = LSW(MSD(rsp->dma));
mcp->mb[5] = rsp->length;
mcp->mb[11] = rsp->vp_idx;
- mcp->mb[14] = rsp->msix->vector;
+ mcp->mb[14] = rsp->msix->entry;
mcp->mb[13] = rsp->rid;
reg = (struct device_reg_25xxmq *)((void *)(ha->mqiobase) +
diff --git a/drivers/scsi/qla2xxx/qla_mid.c b/drivers/scsi/qla2xxx/qla_mid.c
index 386ffea..8863231 100644
--- a/drivers/scsi/qla2xxx/qla_mid.c
+++ b/drivers/scsi/qla2xxx/qla_mid.c
@@ -614,8 +614,10 @@
req->vp_idx = vp_idx;
req->qos = qos;
- if (ha->rsp_q_map[rsp_que])
+ if (ha->rsp_q_map[rsp_que]) {
req->rsp = ha->rsp_q_map[rsp_que];
+ req->rsp->req = req;
+ }
/* Use alternate PCI bus number */
if (MSB(req->rid))
options |= BIT_4;
diff --git a/drivers/scsi/qla2xxx/qla_os.c b/drivers/scsi/qla2xxx/qla_os.c
index 8ea9277..4a71f52 100644
--- a/drivers/scsi/qla2xxx/qla_os.c
+++ b/drivers/scsi/qla2xxx/qla_os.c
@@ -404,26 +404,9 @@
qla24xx_fw_version_str(struct scsi_qla_host *vha, char *str)
{
struct qla_hw_data *ha = vha->hw;
- sprintf(str, "%d.%02d.%02d ", ha->fw_major_version,
- ha->fw_minor_version,
- ha->fw_subminor_version);
- if (ha->fw_attributes & BIT_0)
- strcat(str, "[Class 2] ");
- if (ha->fw_attributes & BIT_1)
- strcat(str, "[IP] ");
- if (ha->fw_attributes & BIT_2)
- strcat(str, "[Multi-ID] ");
- if (ha->fw_attributes & BIT_3)
- strcat(str, "[SB-2] ");
- if (ha->fw_attributes & BIT_4)
- strcat(str, "[T10 CRC] ");
- if (ha->fw_attributes & BIT_5)
- strcat(str, "[VI] ");
- if (ha->fw_attributes & BIT_10)
- strcat(str, "[84XX] ");
- if (ha->fw_attributes & BIT_13)
- strcat(str, "[Experimental]");
+ sprintf(str, "%d.%02d.%02d (%x)", ha->fw_major_version,
+ ha->fw_minor_version, ha->fw_subminor_version, ha->fw_attributes);
return str;
}
@@ -438,7 +421,6 @@
if (!sp)
return sp;
- sp->vha = vha;
sp->fcport = fcport;
sp->cmd = cmd;
sp->que = ha->req_q_map[0];
@@ -1182,7 +1164,7 @@
continue;
for (cnt = 1; cnt < MAX_OUTSTANDING_COMMANDS; cnt++) {
sp = req->outstanding_cmds[cnt];
- if (sp && sp->vha == vha) {
+ if (sp && sp->fcport->vha == vha) {
req->outstanding_cmds[cnt] = NULL;
sp->cmd->result = res;
qla2x00_sp_compl(ha, sp);
@@ -1329,6 +1311,8 @@
unsigned long flags = 0;
struct device_reg_24xx __iomem *reg = &ha->iobase->isp24;
+ if (IS_NOPOLLING_TYPE(ha))
+ return;
spin_lock_irqsave(&ha->hardware_lock, flags);
ha->interrupts_on = 0;
WRT_REG_DWORD(®->ictrl, 0);
@@ -1488,6 +1472,44 @@
.rd_req_reg = qla24xx_rd_req_reg,
};
+static struct isp_operations qla81xx_isp_ops = {
+ .pci_config = qla25xx_pci_config,
+ .reset_chip = qla24xx_reset_chip,
+ .chip_diag = qla24xx_chip_diag,
+ .config_rings = qla24xx_config_rings,
+ .reset_adapter = qla24xx_reset_adapter,
+ .nvram_config = qla81xx_nvram_config,
+ .update_fw_options = qla81xx_update_fw_options,
+ .load_risc = qla24xx_load_risc,
+ .pci_info_str = qla24xx_pci_info_str,
+ .fw_version_str = qla24xx_fw_version_str,
+ .intr_handler = qla24xx_intr_handler,
+ .enable_intrs = qla24xx_enable_intrs,
+ .disable_intrs = qla24xx_disable_intrs,
+ .abort_command = qla24xx_abort_command,
+ .target_reset = qla24xx_abort_target,
+ .lun_reset = qla24xx_lun_reset,
+ .fabric_login = qla24xx_login_fabric,
+ .fabric_logout = qla24xx_fabric_logout,
+ .calc_req_entries = NULL,
+ .build_iocbs = NULL,
+ .prep_ms_iocb = qla24xx_prep_ms_iocb,
+ .prep_ms_fdmi_iocb = qla24xx_prep_ms_fdmi_iocb,
+ .read_nvram = qla25xx_read_nvram_data,
+ .write_nvram = qla25xx_write_nvram_data,
+ .fw_dump = qla81xx_fw_dump,
+ .beacon_on = qla24xx_beacon_on,
+ .beacon_off = qla24xx_beacon_off,
+ .beacon_blink = qla24xx_beacon_blink,
+ .read_optrom = qla25xx_read_optrom_data,
+ .write_optrom = qla24xx_write_optrom_data,
+ .get_flash_version = qla24xx_get_flash_version,
+ .start_scsi = qla24xx_start_scsi,
+ .wrt_req_reg = qla24xx_wrt_req_reg,
+ .wrt_rsp_reg = qla24xx_wrt_rsp_reg,
+ .rd_req_reg = qla24xx_rd_req_reg,
+};
+
static inline void
qla2x00_set_isp_flags(struct qla_hw_data *ha)
{
@@ -1567,6 +1589,13 @@
ha->device_type |= DT_IIDMA;
ha->fw_srisc_address = RISC_START_ADDRESS_2400;
break;
+ case PCI_DEVICE_ID_QLOGIC_ISP8001:
+ ha->device_type |= DT_ISP8001;
+ ha->device_type |= DT_ZIO_SUPPORTED;
+ ha->device_type |= DT_FWI2;
+ ha->device_type |= DT_IIDMA;
+ ha->fw_srisc_address = RISC_START_ADDRESS_2400;
+ break;
}
}
@@ -1629,7 +1658,7 @@
/* Determine queue resources */
ha->max_queues = 1;
- if (ql2xmaxqueues <= 1 || !IS_QLA25XX(ha))
+ if (ql2xmaxqueues <= 1 || (!IS_QLA25XX(ha) && !IS_QLA81XX(ha)))
goto mqiobase_exit;
ha->mqiobase = ioremap(pci_resource_start(ha->pdev, 3),
pci_resource_len(ha->pdev, 3));
@@ -1706,7 +1735,8 @@
pdev->device == PCI_DEVICE_ID_QLOGIC_ISP8432 ||
pdev->device == PCI_DEVICE_ID_QLOGIC_ISP5422 ||
pdev->device == PCI_DEVICE_ID_QLOGIC_ISP5432 ||
- pdev->device == PCI_DEVICE_ID_QLOGIC_ISP2532) {
+ pdev->device == PCI_DEVICE_ID_QLOGIC_ISP2532 ||
+ pdev->device == PCI_DEVICE_ID_QLOGIC_ISP8001) {
bars = pci_select_bars(pdev, IORESOURCE_MEM);
sht = &qla24xx_driver_template;
mem_only = 1;
@@ -1760,6 +1790,10 @@
rsp_length = RESPONSE_ENTRY_CNT_2100;
ha->max_loop_id = SNS_LAST_LOOP_ID_2100;
ha->gid_list_info_size = 4;
+ ha->flash_conf_off = ~0;
+ ha->flash_data_off = ~0;
+ ha->nvram_conf_off = ~0;
+ ha->nvram_data_off = ~0;
ha->isp_ops = &qla2100_isp_ops;
} else if (IS_QLA2200(ha)) {
ha->mbx_count = MAILBOX_REGISTER_COUNT;
@@ -1767,6 +1801,10 @@
rsp_length = RESPONSE_ENTRY_CNT_2100;
ha->max_loop_id = SNS_LAST_LOOP_ID_2100;
ha->gid_list_info_size = 4;
+ ha->flash_conf_off = ~0;
+ ha->flash_data_off = ~0;
+ ha->nvram_conf_off = ~0;
+ ha->nvram_data_off = ~0;
ha->isp_ops = &qla2100_isp_ops;
} else if (IS_QLA23XX(ha)) {
ha->mbx_count = MAILBOX_REGISTER_COUNT;
@@ -1776,6 +1814,10 @@
ha->gid_list_info_size = 6;
if (IS_QLA2322(ha) || IS_QLA6322(ha))
ha->optrom_size = OPTROM_SIZE_2322;
+ ha->flash_conf_off = ~0;
+ ha->flash_data_off = ~0;
+ ha->nvram_conf_off = ~0;
+ ha->nvram_data_off = ~0;
ha->isp_ops = &qla2300_isp_ops;
} else if (IS_QLA24XX_TYPE(ha)) {
ha->mbx_count = MAILBOX_REGISTER_COUNT;
@@ -1787,6 +1829,10 @@
ha->optrom_size = OPTROM_SIZE_24XX;
ha->nvram_npiv_size = QLA_MAX_VPORTS_QLA24XX;
ha->isp_ops = &qla24xx_isp_ops;
+ ha->flash_conf_off = FARX_ACCESS_FLASH_CONF;
+ ha->flash_data_off = FARX_ACCESS_FLASH_DATA;
+ ha->nvram_conf_off = FARX_ACCESS_NVRAM_CONF;
+ ha->nvram_data_off = FARX_ACCESS_NVRAM_DATA;
} else if (IS_QLA25XX(ha)) {
ha->mbx_count = MAILBOX_REGISTER_COUNT;
req_length = REQUEST_ENTRY_CNT_24XX;
@@ -1797,6 +1843,23 @@
ha->optrom_size = OPTROM_SIZE_25XX;
ha->nvram_npiv_size = QLA_MAX_VPORTS_QLA25XX;
ha->isp_ops = &qla25xx_isp_ops;
+ ha->flash_conf_off = FARX_ACCESS_FLASH_CONF;
+ ha->flash_data_off = FARX_ACCESS_FLASH_DATA;
+ ha->nvram_conf_off = FARX_ACCESS_NVRAM_CONF;
+ ha->nvram_data_off = FARX_ACCESS_NVRAM_DATA;
+ } else if (IS_QLA81XX(ha)) {
+ ha->mbx_count = MAILBOX_REGISTER_COUNT;
+ req_length = REQUEST_ENTRY_CNT_24XX;
+ rsp_length = RESPONSE_ENTRY_CNT_2300;
+ ha->max_loop_id = SNS_LAST_LOOP_ID_2300;
+ ha->init_cb_size = sizeof(struct mid_init_cb_81xx);
+ ha->gid_list_info_size = 8;
+ ha->optrom_size = OPTROM_SIZE_81XX;
+ ha->isp_ops = &qla81xx_isp_ops;
+ ha->flash_conf_off = FARX_ACCESS_FLASH_CONF_81XX;
+ ha->flash_data_off = FARX_ACCESS_FLASH_DATA_81XX;
+ ha->nvram_conf_off = ~0;
+ ha->nvram_data_off = ~0;
}
mutex_init(&ha->vport_lock);
@@ -2458,23 +2521,6 @@
return qla2x00_post_work(vha, e, 1);
}
-int
-qla2x00_post_hwe_work(struct scsi_qla_host *vha, uint16_t code, uint16_t d1,
- uint16_t d2, uint16_t d3)
-{
- struct qla_work_evt *e;
-
- e = qla2x00_alloc_work(vha, QLA_EVT_HWE_LOG, 1);
- if (!e)
- return QLA_FUNCTION_FAILED;
-
- e->u.hwe.code = code;
- e->u.hwe.d1 = d1;
- e->u.hwe.d2 = d2;
- e->u.hwe.d3 = d3;
- return qla2x00_post_work(vha, e, 1);
-}
-
static void
qla2x00_do_work(struct scsi_qla_host *vha)
{
@@ -2492,10 +2538,6 @@
fc_host_post_event(vha->host, fc_get_event_number(),
e->u.aen.code, e->u.aen.data);
break;
- case QLA_EVT_HWE_LOG:
- qla2xxx_hw_event_log(vha, e->u.hwe.code, e->u.hwe.d1,
- e->u.hwe.d2, e->u.hwe.d3);
- break;
}
if (e->flags & QLA_EVT_FLAG_FREE)
kfree(e);
@@ -2914,13 +2956,14 @@
/* Firmware interface routines. */
-#define FW_BLOBS 6
+#define FW_BLOBS 7
#define FW_ISP21XX 0
#define FW_ISP22XX 1
#define FW_ISP2300 2
#define FW_ISP2322 3
#define FW_ISP24XX 4
#define FW_ISP25XX 5
+#define FW_ISP81XX 6
#define FW_FILE_ISP21XX "ql2100_fw.bin"
#define FW_FILE_ISP22XX "ql2200_fw.bin"
@@ -2928,6 +2971,7 @@
#define FW_FILE_ISP2322 "ql2322_fw.bin"
#define FW_FILE_ISP24XX "ql2400_fw.bin"
#define FW_FILE_ISP25XX "ql2500_fw.bin"
+#define FW_FILE_ISP81XX "ql8100_fw.bin"
static DEFINE_MUTEX(qla_fw_lock);
@@ -2938,6 +2982,7 @@
{ .name = FW_FILE_ISP2322, .segs = { 0x800, 0x1c000, 0x1e000, 0 }, },
{ .name = FW_FILE_ISP24XX, },
{ .name = FW_FILE_ISP25XX, },
+ { .name = FW_FILE_ISP81XX, },
};
struct fw_blob *
@@ -2959,6 +3004,8 @@
blob = &qla_fw_blobs[FW_ISP24XX];
} else if (IS_QLA25XX(ha)) {
blob = &qla_fw_blobs[FW_ISP25XX];
+ } else if (IS_QLA81XX(ha)) {
+ blob = &qla_fw_blobs[FW_ISP81XX];
}
mutex_lock(&qla_fw_lock);
@@ -3112,6 +3159,7 @@
{ PCI_DEVICE(PCI_VENDOR_ID_QLOGIC, PCI_DEVICE_ID_QLOGIC_ISP5422) },
{ PCI_DEVICE(PCI_VENDOR_ID_QLOGIC, PCI_DEVICE_ID_QLOGIC_ISP5432) },
{ PCI_DEVICE(PCI_VENDOR_ID_QLOGIC, PCI_DEVICE_ID_QLOGIC_ISP2532) },
+ { PCI_DEVICE(PCI_VENDOR_ID_QLOGIC, PCI_DEVICE_ID_QLOGIC_ISP8001) },
{ 0 },
};
MODULE_DEVICE_TABLE(pci, qla2xxx_pci_tbl);
@@ -3200,3 +3248,4 @@
MODULE_FIRMWARE(FW_FILE_ISP2322);
MODULE_FIRMWARE(FW_FILE_ISP24XX);
MODULE_FIRMWARE(FW_FILE_ISP25XX);
+MODULE_FIRMWARE(FW_FILE_ISP81XX);
diff --git a/drivers/scsi/qla2xxx/qla_sup.c b/drivers/scsi/qla2xxx/qla_sup.c
index c538ee1..303f8ee 100644
--- a/drivers/scsi/qla2xxx/qla_sup.c
+++ b/drivers/scsi/qla2xxx/qla_sup.c
@@ -425,27 +425,27 @@
#define OPTROM_BURST_DWORDS (OPTROM_BURST_SIZE / 4)
static inline uint32_t
-flash_conf_to_access_addr(uint32_t faddr)
+flash_conf_addr(struct qla_hw_data *ha, uint32_t faddr)
{
- return FARX_ACCESS_FLASH_CONF | faddr;
+ return ha->flash_conf_off | faddr;
}
static inline uint32_t
-flash_data_to_access_addr(uint32_t faddr)
+flash_data_addr(struct qla_hw_data *ha, uint32_t faddr)
{
- return FARX_ACCESS_FLASH_DATA | faddr;
+ return ha->flash_data_off | faddr;
}
static inline uint32_t
-nvram_conf_to_access_addr(uint32_t naddr)
+nvram_conf_addr(struct qla_hw_data *ha, uint32_t naddr)
{
- return FARX_ACCESS_NVRAM_CONF | naddr;
+ return ha->nvram_conf_off | naddr;
}
static inline uint32_t
-nvram_data_to_access_addr(uint32_t naddr)
+nvram_data_addr(struct qla_hw_data *ha, uint32_t naddr)
{
- return FARX_ACCESS_NVRAM_DATA | naddr;
+ return ha->nvram_data_off | naddr;
}
static uint32_t
@@ -481,10 +481,12 @@
uint32_t dwords)
{
uint32_t i;
+ struct qla_hw_data *ha = vha->hw;
+
/* Dword reads to flash. */
for (i = 0; i < dwords; i++, faddr++)
- dwptr[i] = cpu_to_le32(qla24xx_read_flash_dword(vha->hw,
- flash_data_to_access_addr(faddr)));
+ dwptr[i] = cpu_to_le32(qla24xx_read_flash_dword(ha,
+ flash_data_addr(ha, faddr)));
return dwptr;
}
@@ -518,7 +520,7 @@
{
uint32_t ids;
- ids = qla24xx_read_flash_dword(ha, flash_data_to_access_addr(0xd03ab));
+ ids = qla24xx_read_flash_dword(ha, flash_conf_addr(ha, 0x03ab));
*man_id = LSB(ids);
*flash_id = MSB(ids);
@@ -530,8 +532,7 @@
* Example: ATMEL 0x00 01 45 1F
* Extract MFG and Dev ID from last two bytes.
*/
- ids = qla24xx_read_flash_dword(ha,
- flash_data_to_access_addr(0xd009f));
+ ids = qla24xx_read_flash_dword(ha, flash_conf_addr(ha, 0x009f));
*man_id = LSB(ids);
*flash_id = MSB(ids);
}
@@ -555,9 +556,13 @@
/* Begin with sane defaults. */
loc = locations[0];
- *start = IS_QLA24XX_TYPE(ha) ? FA_FLASH_LAYOUT_ADDR_24:
- FA_FLASH_LAYOUT_ADDR;
-
+ *start = 0;
+ if (IS_QLA24XX_TYPE(ha))
+ *start = FA_FLASH_LAYOUT_ADDR_24;
+ else if (IS_QLA25XX(ha))
+ *start = FA_FLASH_LAYOUT_ADDR;
+ else if (IS_QLA81XX(ha))
+ *start = FA_FLASH_LAYOUT_ADDR_81;
/* Begin with first PCI expansion ROM header. */
buf = (uint8_t *)req->ring;
dcode = (uint32_t *)req->ring;
@@ -618,6 +623,22 @@
qla2xxx_get_flt_info(scsi_qla_host_t *vha, uint32_t flt_addr)
{
const char *loc, *locations[] = { "DEF", "FLT" };
+ const uint32_t def_fw[] =
+ { FA_RISC_CODE_ADDR, FA_RISC_CODE_ADDR, FA_RISC_CODE_ADDR_81 };
+ const uint32_t def_boot[] =
+ { FA_BOOT_CODE_ADDR, FA_BOOT_CODE_ADDR, FA_BOOT_CODE_ADDR_81 };
+ const uint32_t def_vpd_nvram[] =
+ { FA_VPD_NVRAM_ADDR, FA_VPD_NVRAM_ADDR, FA_VPD_NVRAM_ADDR_81 };
+ const uint32_t def_fdt[] =
+ { FA_FLASH_DESCR_ADDR_24, FA_FLASH_DESCR_ADDR,
+ FA_FLASH_DESCR_ADDR_81 };
+ const uint32_t def_npiv_conf0[] =
+ { FA_NPIV_CONF0_ADDR_24, FA_NPIV_CONF0_ADDR,
+ FA_NPIV_CONF0_ADDR_81 };
+ const uint32_t def_npiv_conf1[] =
+ { FA_NPIV_CONF1_ADDR_24, FA_NPIV_CONF1_ADDR,
+ FA_NPIV_CONF1_ADDR_81 };
+ uint32_t def;
uint16_t *wptr;
uint16_t cnt, chksum;
uint32_t start;
@@ -676,20 +697,12 @@
case FLT_REG_FDT:
ha->flt_region_fdt = start;
break;
- case FLT_REG_HW_EVENT_0:
- if (!PCI_FUNC(ha->pdev->devfn))
- ha->flt_region_hw_event = start;
- break;
- case FLT_REG_HW_EVENT_1:
- if (PCI_FUNC(ha->pdev->devfn))
- ha->flt_region_hw_event = start;
- break;
case FLT_REG_NPIV_CONF_0:
- if (!PCI_FUNC(ha->pdev->devfn))
+ if (!(PCI_FUNC(ha->pdev->devfn) & 1))
ha->flt_region_npiv_conf = start;
break;
case FLT_REG_NPIV_CONF_1:
- if (PCI_FUNC(ha->pdev->devfn))
+ if (PCI_FUNC(ha->pdev->devfn) & 1)
ha->flt_region_npiv_conf = start;
break;
}
@@ -699,22 +712,24 @@
no_flash_data:
/* Use hardcoded defaults. */
loc = locations[0];
- ha->flt_region_fw = FA_RISC_CODE_ADDR;
- ha->flt_region_boot = FA_BOOT_CODE_ADDR;
- ha->flt_region_vpd_nvram = FA_VPD_NVRAM_ADDR;
- ha->flt_region_fdt = IS_QLA24XX_TYPE(ha) ? FA_FLASH_DESCR_ADDR_24:
- FA_FLASH_DESCR_ADDR;
- ha->flt_region_hw_event = !PCI_FUNC(ha->pdev->devfn) ?
- FA_HW_EVENT0_ADDR: FA_HW_EVENT1_ADDR;
- ha->flt_region_npiv_conf = !PCI_FUNC(ha->pdev->devfn) ?
- (IS_QLA24XX_TYPE(ha) ? FA_NPIV_CONF0_ADDR_24: FA_NPIV_CONF0_ADDR):
- (IS_QLA24XX_TYPE(ha) ? FA_NPIV_CONF1_ADDR_24: FA_NPIV_CONF1_ADDR);
+ def = 0;
+ if (IS_QLA24XX_TYPE(ha))
+ def = 0;
+ else if (IS_QLA25XX(ha))
+ def = 1;
+ else if (IS_QLA81XX(ha))
+ def = 2;
+ ha->flt_region_fw = def_fw[def];
+ ha->flt_region_boot = def_boot[def];
+ ha->flt_region_vpd_nvram = def_vpd_nvram[def];
+ ha->flt_region_fdt = def_fdt[def];
+ ha->flt_region_npiv_conf = !(PCI_FUNC(ha->pdev->devfn) & 1) ?
+ def_npiv_conf0[def]: def_npiv_conf1[def];
done:
DEBUG2(qla_printk(KERN_DEBUG, ha, "FLT[%s]: boot=0x%x fw=0x%x "
- "vpd_nvram=0x%x fdt=0x%x flt=0x%x hwe=0x%x npiv=0x%x.\n", loc,
+ "vpd_nvram=0x%x fdt=0x%x flt=0x%x npiv=0x%x.\n", loc,
ha->flt_region_boot, ha->flt_region_fw, ha->flt_region_vpd_nvram,
- ha->flt_region_fdt, ha->flt_region_flt, ha->flt_region_hw_event,
- ha->flt_region_npiv_conf));
+ ha->flt_region_fdt, ha->flt_region_flt, ha->flt_region_npiv_conf));
}
static void
@@ -757,14 +772,14 @@
mid = le16_to_cpu(fdt->man_id);
fid = le16_to_cpu(fdt->id);
ha->fdt_wrt_disable = fdt->wrt_disable_bits;
- ha->fdt_erase_cmd = flash_conf_to_access_addr(0x0300 | fdt->erase_cmd);
+ ha->fdt_erase_cmd = flash_conf_addr(ha, 0x0300 | fdt->erase_cmd);
ha->fdt_block_size = le32_to_cpu(fdt->block_size);
if (fdt->unprotect_sec_cmd) {
- ha->fdt_unprotect_sec_cmd = flash_conf_to_access_addr(0x0300 |
+ ha->fdt_unprotect_sec_cmd = flash_conf_addr(ha, 0x0300 |
fdt->unprotect_sec_cmd);
ha->fdt_protect_sec_cmd = fdt->protect_sec_cmd ?
- flash_conf_to_access_addr(0x0300 | fdt->protect_sec_cmd):
- flash_conf_to_access_addr(0x0336);
+ flash_conf_addr(ha, 0x0300 | fdt->protect_sec_cmd):
+ flash_conf_addr(ha, 0x0336);
}
goto done;
no_flash_data:
@@ -773,7 +788,7 @@
mid = man_id;
fid = flash_id;
ha->fdt_wrt_disable = 0x9c;
- ha->fdt_erase_cmd = flash_conf_to_access_addr(0x03d8);
+ ha->fdt_erase_cmd = flash_conf_addr(ha, 0x03d8);
switch (man_id) {
case 0xbf: /* STT flash. */
if (flash_id == 0x8e)
@@ -782,16 +797,16 @@
ha->fdt_block_size = FLASH_BLK_SIZE_32K;
if (flash_id == 0x80)
- ha->fdt_erase_cmd = flash_conf_to_access_addr(0x0352);
+ ha->fdt_erase_cmd = flash_conf_addr(ha, 0x0352);
break;
case 0x13: /* ST M25P80. */
ha->fdt_block_size = FLASH_BLK_SIZE_64K;
break;
case 0x1f: /* Atmel 26DF081A. */
ha->fdt_block_size = FLASH_BLK_SIZE_4K;
- ha->fdt_erase_cmd = flash_conf_to_access_addr(0x0320);
- ha->fdt_unprotect_sec_cmd = flash_conf_to_access_addr(0x0339);
- ha->fdt_protect_sec_cmd = flash_conf_to_access_addr(0x0336);
+ ha->fdt_erase_cmd = flash_conf_addr(ha, 0x0320);
+ ha->fdt_unprotect_sec_cmd = flash_conf_addr(ha, 0x0339);
+ ha->fdt_protect_sec_cmd = flash_conf_addr(ha, 0x0336);
break;
default:
/* Default to 64 kb sector size. */
@@ -813,7 +828,7 @@
uint32_t flt_addr;
struct qla_hw_data *ha = vha->hw;
- if (!IS_QLA24XX_TYPE(ha) && !IS_QLA25XX(ha))
+ if (!IS_QLA24XX_TYPE(ha) && !IS_QLA25XX(ha) && !IS_QLA81XX(ha))
return QLA_SUCCESS;
ret = qla2xxx_find_flt_start(vha, &flt_addr);
@@ -838,7 +853,7 @@
struct qla_npiv_entry *entry;
struct qla_hw_data *ha = vha->hw;
- if (!IS_QLA24XX_TYPE(ha) && !IS_QLA25XX(ha))
+ if (!IS_QLA24XX_TYPE(ha) && !IS_QLA25XX(ha) && !IS_QLA81XX(ha))
return;
ha->isp_ops->read_optrom(vha, (uint8_t *)&hdr,
@@ -930,9 +945,9 @@
return;
/* Disable flash write-protection. */
- qla24xx_write_flash_dword(ha, flash_conf_to_access_addr(0x101), 0);
+ qla24xx_write_flash_dword(ha, flash_conf_addr(ha, 0x101), 0);
/* Some flash parts need an additional zero-write to clear bits.*/
- qla24xx_write_flash_dword(ha, flash_conf_to_access_addr(0x101), 0);
+ qla24xx_write_flash_dword(ha, flash_conf_addr(ha, 0x101), 0);
}
static void
@@ -945,11 +960,10 @@
goto skip_wrt_protect;
/* Enable flash write-protection and wait for completion. */
- qla24xx_write_flash_dword(ha, flash_conf_to_access_addr(0x101),
+ qla24xx_write_flash_dword(ha, flash_conf_addr(ha, 0x101),
ha->fdt_wrt_disable);
for (cnt = 300; cnt &&
- qla24xx_read_flash_dword(ha,
- flash_conf_to_access_addr(0x005)) & BIT_0;
+ qla24xx_read_flash_dword(ha, flash_conf_addr(ha, 0x005)) & BIT_0;
cnt--) {
udelay(10);
}
@@ -977,7 +991,7 @@
ret = QLA_SUCCESS;
/* Prepare burst-capable write on supported ISPs. */
- if (IS_QLA25XX(ha) && !(faddr & 0xfff) &&
+ if ((IS_QLA25XX(ha) || IS_QLA81XX(ha)) && !(faddr & 0xfff) &&
dwords > OPTROM_BURST_DWORDS) {
optrom = dma_alloc_coherent(&ha->pdev->dev, OPTROM_BURST_SIZE,
&optrom_dma, GFP_KERNEL);
@@ -989,7 +1003,7 @@
}
rest_addr = (ha->fdt_block_size >> 2) - 1;
- sec_mask = 0x80000 - (ha->fdt_block_size >> 2);
+ sec_mask = (ha->optrom_size >> 2) - (ha->fdt_block_size >> 2);
qla24xx_unprotect_flash(ha);
@@ -1024,13 +1038,13 @@
*s = cpu_to_le32(*d);
ret = qla2x00_load_ram(vha, optrom_dma,
- flash_data_to_access_addr(faddr),
+ flash_data_addr(ha, faddr),
OPTROM_BURST_DWORDS);
if (ret != QLA_SUCCESS) {
qla_printk(KERN_WARNING, ha,
"Unable to burst-write optrom segment "
"(%x/%x/%llx).\n", ret,
- flash_data_to_access_addr(faddr),
+ flash_data_addr(ha, faddr),
(unsigned long long)optrom_dma);
qla_printk(KERN_WARNING, ha,
"Reverting to slow-write.\n");
@@ -1047,7 +1061,7 @@
}
ret = qla24xx_write_flash_dword(ha,
- flash_data_to_access_addr(faddr), cpu_to_le32(*dwptr));
+ flash_data_addr(ha, faddr), cpu_to_le32(*dwptr));
if (ret != QLA_SUCCESS) {
DEBUG9(printk("%s(%ld) Unable to program flash "
"address=%x data=%x.\n", __func__,
@@ -1098,12 +1112,13 @@
{
uint32_t i;
uint32_t *dwptr;
+ struct qla_hw_data *ha = vha->hw;
/* Dword reads to flash. */
dwptr = (uint32_t *)buf;
for (i = 0; i < bytes >> 2; i++, naddr++)
- dwptr[i] = cpu_to_le32(qla24xx_read_flash_dword(vha->hw,
- nvram_data_to_access_addr(naddr)));
+ dwptr[i] = cpu_to_le32(qla24xx_read_flash_dword(ha,
+ nvram_data_addr(ha, naddr)));
return buf;
}
@@ -1160,17 +1175,14 @@
RD_REG_DWORD(®->ctrl_status); /* PCI Posting. */
/* Disable NVRAM write-protection. */
- qla24xx_write_flash_dword(ha, nvram_conf_to_access_addr(0x101),
- 0);
- qla24xx_write_flash_dword(ha, nvram_conf_to_access_addr(0x101),
- 0);
+ qla24xx_write_flash_dword(ha, nvram_conf_addr(ha, 0x101), 0);
+ qla24xx_write_flash_dword(ha, nvram_conf_addr(ha, 0x101), 0);
/* Dword writes to flash. */
dwptr = (uint32_t *)buf;
for (i = 0; i < bytes >> 2; i++, naddr++, dwptr++) {
ret = qla24xx_write_flash_dword(ha,
- nvram_data_to_access_addr(naddr),
- cpu_to_le32(*dwptr));
+ nvram_data_addr(ha, naddr), cpu_to_le32(*dwptr));
if (ret != QLA_SUCCESS) {
DEBUG9(qla_printk("Unable to program nvram address=%x "
"data=%x.\n", naddr, *dwptr));
@@ -1179,8 +1191,7 @@
}
/* Enable NVRAM write-protection. */
- qla24xx_write_flash_dword(ha, nvram_conf_to_access_addr(0x101),
- 0x8c);
+ qla24xx_write_flash_dword(ha, nvram_conf_addr(ha, 0x101), 0x8c);
/* Disable flash write. */
WRT_REG_DWORD(®->ctrl_status,
@@ -1202,8 +1213,7 @@
dwptr = (uint32_t *)buf;
for (i = 0; i < bytes >> 2; i++, naddr++)
dwptr[i] = cpu_to_le32(qla24xx_read_flash_dword(ha,
- flash_data_to_access_addr(ha->flt_region_vpd_nvram |
- naddr)));
+ flash_data_addr(ha, ha->flt_region_vpd_nvram | naddr)));
return buf;
}
@@ -2246,12 +2256,12 @@
burst = left;
rval = qla2x00_dump_ram(vha, optrom_dma,
- flash_data_to_access_addr(faddr), burst);
+ flash_data_addr(ha, faddr), burst);
if (rval) {
qla_printk(KERN_WARNING, ha,
"Unable to burst-read optrom segment "
"(%x/%x/%llx).\n", rval,
- flash_data_to_access_addr(faddr),
+ flash_data_addr(ha, faddr),
(unsigned long long)optrom_dma);
qla_printk(KERN_WARNING, ha,
"Reverting to slow-read.\n");
@@ -2648,108 +2658,3 @@
return 0;
}
-
-static int
-qla2xxx_hw_event_store(scsi_qla_host_t *vha, uint32_t *fdata)
-{
- uint32_t d[2], faddr;
- struct qla_hw_data *ha = vha->hw;
-
- /* Locate first empty entry. */
- for (;;) {
- if (ha->hw_event_ptr >=
- ha->flt_region_hw_event + FA_HW_EVENT_SIZE) {
- DEBUG2(qla_printk(KERN_WARNING, ha,
- "HW event -- Log Full!\n"));
- return QLA_MEMORY_ALLOC_FAILED;
- }
-
- qla24xx_read_flash_data(vha, d, ha->hw_event_ptr, 2);
- faddr = flash_data_to_access_addr(ha->hw_event_ptr);
- ha->hw_event_ptr += FA_HW_EVENT_ENTRY_SIZE;
- if (d[0] == __constant_cpu_to_le32(0xffffffff) &&
- d[1] == __constant_cpu_to_le32(0xffffffff)) {
- qla24xx_unprotect_flash(ha);
-
- qla24xx_write_flash_dword(ha, faddr++,
- cpu_to_le32(jiffies));
- qla24xx_write_flash_dword(ha, faddr++, 0);
- qla24xx_write_flash_dword(ha, faddr++, *fdata++);
- qla24xx_write_flash_dword(ha, faddr++, *fdata);
-
- qla24xx_protect_flash(ha);
- break;
- }
- }
- return QLA_SUCCESS;
-}
-
-int
-qla2xxx_hw_event_log(scsi_qla_host_t *vha, uint16_t code, uint16_t d1,
- uint16_t d2, uint16_t d3)
-{
-#define QMARK(a, b, c, d) \
- cpu_to_le32(LSB(a) << 24 | LSB(b) << 16 | LSB(c) << 8 | LSB(d))
- struct qla_hw_data *ha = vha->hw;
- int rval;
- uint32_t marker[2], fdata[4];
-
- if (ha->flt_region_hw_event == 0)
- return QLA_FUNCTION_FAILED;
-
- DEBUG2(qla_printk(KERN_WARNING, ha,
- "HW event -- code=%x, d1=%x, d2=%x, d3=%x.\n", code, d1, d2, d3));
-
- /* If marker not already found, locate or write. */
- if (!ha->flags.hw_event_marker_found) {
- /* Create marker. */
- marker[0] = QMARK('L', ha->fw_major_version,
- ha->fw_minor_version, ha->fw_subminor_version);
- marker[1] = QMARK(QLA_DRIVER_MAJOR_VER, QLA_DRIVER_MINOR_VER,
- QLA_DRIVER_PATCH_VER, QLA_DRIVER_BETA_VER);
-
- /* Locate marker. */
- ha->hw_event_ptr = ha->flt_region_hw_event;
- for (;;) {
- qla24xx_read_flash_data(vha, fdata, ha->hw_event_ptr,
- 4);
- if (fdata[0] == __constant_cpu_to_le32(0xffffffff) &&
- fdata[1] == __constant_cpu_to_le32(0xffffffff))
- break;
- ha->hw_event_ptr += FA_HW_EVENT_ENTRY_SIZE;
- if (ha->hw_event_ptr >=
- ha->flt_region_hw_event + FA_HW_EVENT_SIZE) {
- DEBUG2(qla_printk(KERN_WARNING, ha,
- "HW event -- Log Full!\n"));
- return QLA_MEMORY_ALLOC_FAILED;
- }
- if (fdata[2] == marker[0] && fdata[3] == marker[1]) {
- ha->flags.hw_event_marker_found = 1;
- break;
- }
- }
- /* No marker, write it. */
- if (!ha->flags.hw_event_marker_found) {
- rval = qla2xxx_hw_event_store(vha, marker);
- if (rval != QLA_SUCCESS) {
- DEBUG2(qla_printk(KERN_WARNING, ha,
- "HW event -- Failed marker write=%x.!\n",
- rval));
- return rval;
- }
- ha->flags.hw_event_marker_found = 1;
- }
- }
-
- /* Store error. */
- fdata[0] = cpu_to_le32(code << 16 | d1);
- fdata[1] = cpu_to_le32(d2 << 16 | d3);
- rval = qla2xxx_hw_event_store(vha, fdata);
- if (rval != QLA_SUCCESS) {
- DEBUG2(qla_printk(KERN_WARNING, ha,
- "HW event -- Failed error write=%x.!\n",
- rval));
- }
-
- return rval;
-}
diff --git a/drivers/scsi/qla2xxx/qla_version.h b/drivers/scsi/qla2xxx/qla_version.h
index be22f3a..808bab6 100644
--- a/drivers/scsi/qla2xxx/qla_version.h
+++ b/drivers/scsi/qla2xxx/qla_version.h
@@ -7,9 +7,9 @@
/*
* Driver version
*/
-#define QLA2XXX_VERSION "8.02.03-k1"
+#define QLA2XXX_VERSION "8.03.00-k1"
#define QLA_DRIVER_MAJOR_VER 8
-#define QLA_DRIVER_MINOR_VER 2
-#define QLA_DRIVER_PATCH_VER 3
+#define QLA_DRIVER_MINOR_VER 3
+#define QLA_DRIVER_PATCH_VER 0
#define QLA_DRIVER_BETA_VER 0
diff --git a/drivers/scsi/raid_class.c b/drivers/scsi/raid_class.c
index 913a931..8e5c169 100644
--- a/drivers/scsi/raid_class.c
+++ b/drivers/scsi/raid_class.c
@@ -237,8 +237,7 @@
rc->dev.parent = get_device(component_dev);
rc->num = rd->component_count++;
- snprintf(rc->dev.bus_id, sizeof(rc->dev.bus_id),
- "component-%d", rc->num);
+ dev_set_name(&rc->dev, "component-%d", rc->num);
list_add_tail(&rc->node, &rd->component_list);
rc->dev.class = &raid_class.class;
err = device_add(&rc->dev);
diff --git a/drivers/scsi/scsi.c b/drivers/scsi/scsi.c
index f8b79d4..42e72a2 100644
--- a/drivers/scsi/scsi.c
+++ b/drivers/scsi/scsi.c
@@ -651,10 +651,6 @@
unsigned long timeout;
int rtn = 0;
- /*
- * We will use a queued command if possible, otherwise we will
- * emulate the queuing and calling of completion function ourselves.
- */
atomic_inc(&cmd->device->iorequest_cnt);
/* check if the device is still usable */
diff --git a/drivers/scsi/scsi_debug.c b/drivers/scsi/scsi_debug.c
index 27c633f..6eebd0b 100644
--- a/drivers/scsi/scsi_debug.c
+++ b/drivers/scsi/scsi_debug.c
@@ -2508,7 +2508,7 @@
}
static struct device pseudo_primary = {
- .bus_id = "pseudo_0",
+ .init_name = "pseudo_0",
.release = pseudo_0_release,
};
@@ -2680,7 +2680,7 @@
sdbg_host->dev.bus = &pseudo_lld_bus;
sdbg_host->dev.parent = &pseudo_primary;
sdbg_host->dev.release = &sdebug_release_adapter;
- sprintf(sdbg_host->dev.bus_id, "adapter%d", scsi_debug_add_host);
+ dev_set_name(&sdbg_host->dev, "adapter%d", scsi_debug_add_host);
error = device_register(&sdbg_host->dev);
diff --git a/drivers/scsi/scsi_error.c b/drivers/scsi/scsi_error.c
index d86ebea..ad6a137 100644
--- a/drivers/scsi/scsi_error.c
+++ b/drivers/scsi/scsi_error.c
@@ -124,34 +124,22 @@
enum blk_eh_timer_return scsi_times_out(struct request *req)
{
struct scsi_cmnd *scmd = req->special;
- enum blk_eh_timer_return (*eh_timed_out)(struct scsi_cmnd *);
enum blk_eh_timer_return rtn = BLK_EH_NOT_HANDLED;
scsi_log_completion(scmd, TIMEOUT_ERROR);
if (scmd->device->host->transportt->eh_timed_out)
- eh_timed_out = scmd->device->host->transportt->eh_timed_out;
+ rtn = scmd->device->host->transportt->eh_timed_out(scmd);
else if (scmd->device->host->hostt->eh_timed_out)
- eh_timed_out = scmd->device->host->hostt->eh_timed_out;
- else
- eh_timed_out = NULL;
+ rtn = scmd->device->host->hostt->eh_timed_out(scmd);
- if (eh_timed_out) {
- rtn = eh_timed_out(scmd);
- switch (rtn) {
- case BLK_EH_NOT_HANDLED:
- break;
- default:
- return rtn;
- }
- }
-
- if (unlikely(!scsi_eh_scmd_add(scmd, SCSI_EH_CANCEL_CMD))) {
+ if (unlikely(rtn == BLK_EH_NOT_HANDLED &&
+ !scsi_eh_scmd_add(scmd, SCSI_EH_CANCEL_CMD))) {
scmd->result |= DID_TIME_OUT << 16;
- return BLK_EH_HANDLED;
+ rtn = BLK_EH_HANDLED;
}
- return BLK_EH_NOT_HANDLED;
+ return rtn;
}
/**
diff --git a/drivers/scsi/scsi_ioctl.c b/drivers/scsi/scsi_ioctl.c
index 2ae4f8f..b98f763 100644
--- a/drivers/scsi/scsi_ioctl.c
+++ b/drivers/scsi/scsi_ioctl.c
@@ -167,10 +167,17 @@
static int scsi_ioctl_get_pci(struct scsi_device *sdev, void __user *arg)
{
struct device *dev = scsi_get_device(sdev->host);
+ const char *name;
if (!dev)
return -ENXIO;
- return copy_to_user(arg, dev->bus_id, sizeof(dev->bus_id))? -EFAULT: 0;
+
+ name = dev_name(dev);
+
+ /* compatibility with old ioctl which only returned
+ * 20 characters */
+ return copy_to_user(arg, name, min(strlen(name), (size_t)20))
+ ? -EFAULT: 0;
}
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index f2f51e0..940dc32 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -91,26 +91,19 @@
scsi_put_command(cmd);
}
-/*
- * Function: scsi_queue_insert()
+/**
+ * __scsi_queue_insert - private queue insertion
+ * @cmd: The SCSI command being requeued
+ * @reason: The reason for the requeue
+ * @unbusy: Whether the queue should be unbusied
*
- * Purpose: Insert a command in the midlevel queue.
- *
- * Arguments: cmd - command that we are adding to queue.
- * reason - why we are inserting command to queue.
- *
- * Lock status: Assumed that lock is not held upon entry.
- *
- * Returns: Nothing.
- *
- * Notes: We do this for one of two cases. Either the host is busy
- * and it cannot accept any more commands for the time being,
- * or the device returned QUEUE_FULL and can accept no more
- * commands.
- * Notes: This could be called either from an interrupt context or a
- * normal process context.
+ * This is a private queue insertion. The public interface
+ * scsi_queue_insert() always assumes the queue should be unbusied
+ * because it's always called before the completion. This function is
+ * for a requeue after completion, which should only occur in this
+ * file.
*/
-int scsi_queue_insert(struct scsi_cmnd *cmd, int reason)
+static int __scsi_queue_insert(struct scsi_cmnd *cmd, int reason, int unbusy)
{
struct Scsi_Host *host = cmd->device->host;
struct scsi_device *device = cmd->device;
@@ -150,7 +143,8 @@
* Decrement the counters, since these commands are no longer
* active on the host/device.
*/
- scsi_device_unbusy(device);
+ if (unbusy)
+ scsi_device_unbusy(device);
/*
* Requeue this command. It will go before all other commands
@@ -172,6 +166,29 @@
return 0;
}
+/*
+ * Function: scsi_queue_insert()
+ *
+ * Purpose: Insert a command in the midlevel queue.
+ *
+ * Arguments: cmd - command that we are adding to queue.
+ * reason - why we are inserting command to queue.
+ *
+ * Lock status: Assumed that lock is not held upon entry.
+ *
+ * Returns: Nothing.
+ *
+ * Notes: We do this for one of two cases. Either the host is busy
+ * and it cannot accept any more commands for the time being,
+ * or the device returned QUEUE_FULL and can accept no more
+ * commands.
+ * Notes: This could be called either from an interrupt context or a
+ * normal process context.
+ */
+int scsi_queue_insert(struct scsi_cmnd *cmd, int reason)
+{
+ return __scsi_queue_insert(cmd, reason, 1);
+}
/**
* scsi_execute - insert request and wait for the result
* @sdev: scsi device
@@ -684,6 +701,8 @@
scsi_run_queue(sdev->request_queue);
}
+static void __scsi_release_buffers(struct scsi_cmnd *, int);
+
/*
* Function: scsi_end_request()
*
@@ -732,6 +751,7 @@
* leftovers in the front of the
* queue, and goose the queue again.
*/
+ scsi_release_buffers(cmd);
scsi_requeue_command(q, cmd);
cmd = NULL;
}
@@ -743,6 +763,7 @@
* This will goose the queue request function at the end, so we don't
* need to worry about launching another command.
*/
+ __scsi_release_buffers(cmd, 0);
scsi_next_command(cmd);
return NULL;
}
@@ -798,6 +819,26 @@
__sg_free_table(&sdb->table, SCSI_MAX_SG_SEGMENTS, scsi_sg_free);
}
+static void __scsi_release_buffers(struct scsi_cmnd *cmd, int do_bidi_check)
+{
+
+ if (cmd->sdb.table.nents)
+ scsi_free_sgtable(&cmd->sdb);
+
+ memset(&cmd->sdb, 0, sizeof(cmd->sdb));
+
+ if (do_bidi_check && scsi_bidi_cmnd(cmd)) {
+ struct scsi_data_buffer *bidi_sdb =
+ cmd->request->next_rq->special;
+ scsi_free_sgtable(bidi_sdb);
+ kmem_cache_free(scsi_sdb_cache, bidi_sdb);
+ cmd->request->next_rq->special = NULL;
+ }
+
+ if (scsi_prot_sg_count(cmd))
+ scsi_free_sgtable(cmd->prot_sdb);
+}
+
/*
* Function: scsi_release_buffers()
*
@@ -817,21 +858,7 @@
*/
void scsi_release_buffers(struct scsi_cmnd *cmd)
{
- if (cmd->sdb.table.nents)
- scsi_free_sgtable(&cmd->sdb);
-
- memset(&cmd->sdb, 0, sizeof(cmd->sdb));
-
- if (scsi_bidi_cmnd(cmd)) {
- struct scsi_data_buffer *bidi_sdb =
- cmd->request->next_rq->special;
- scsi_free_sgtable(bidi_sdb);
- kmem_cache_free(scsi_sdb_cache, bidi_sdb);
- cmd->request->next_rq->special = NULL;
- }
-
- if (scsi_prot_sg_count(cmd))
- scsi_free_sgtable(cmd->prot_sdb);
+ __scsi_release_buffers(cmd, 1);
}
EXPORT_SYMBOL(scsi_release_buffers);
@@ -945,7 +972,6 @@
}
BUG_ON(blk_bidi_rq(req)); /* bidi not support for !blk_pc_request yet */
- scsi_release_buffers(cmd);
/*
* Next deal with any sectors which we were able to correctly
@@ -963,6 +989,8 @@
return;
this_count = blk_rq_bytes(req);
+ error = -EIO;
+
if (host_byte(result) == DID_RESET) {
/* Third party bus reset or reset for error recovery
* reasons. Just retry the command and see what
@@ -1004,13 +1032,18 @@
/* This will issue a new 6-byte command. */
cmd->device->use_10_for_rw = 0;
action = ACTION_REPREP;
+ } else if (sshdr.asc == 0x10) /* DIX */ {
+ description = "Host Data Integrity Failure";
+ action = ACTION_FAIL;
+ error = -EILSEQ;
} else
action = ACTION_FAIL;
break;
case ABORTED_COMMAND:
if (sshdr.asc == 0x10) { /* DIF */
+ description = "Target Data Integrity Failure";
action = ACTION_FAIL;
- description = "Data Integrity Failure";
+ error = -EILSEQ;
} else
action = ACTION_RETRY;
break;
@@ -1029,6 +1062,10 @@
case 0x09: /* self test in progress */
action = ACTION_DELAYED_RETRY;
break;
+ default:
+ description = "Device not ready";
+ action = ACTION_FAIL;
+ break;
}
} else {
description = "Device not ready";
@@ -1052,9 +1089,10 @@
switch (action) {
case ACTION_FAIL:
/* Give up and fail the remainder of the request */
+ scsi_release_buffers(cmd);
if (!(req->cmd_flags & REQ_QUIET)) {
if (description)
- scmd_printk(KERN_INFO, cmd, "%s",
+ scmd_printk(KERN_INFO, cmd, "%s\n",
description);
scsi_print_result(cmd);
if (driver_byte(result) & DRIVER_SENSE)
@@ -1067,15 +1105,16 @@
/* Unprep the request and put it back at the head of the queue.
* A new command will be prepared and issued.
*/
+ scsi_release_buffers(cmd);
scsi_requeue_command(q, cmd);
break;
case ACTION_RETRY:
/* Retry the same command immediately */
- scsi_queue_insert(cmd, SCSI_MLQUEUE_EH_RETRY);
+ __scsi_queue_insert(cmd, SCSI_MLQUEUE_EH_RETRY, 0);
break;
case ACTION_DELAYED_RETRY:
/* Retry the same command after a delay */
- scsi_queue_insert(cmd, SCSI_MLQUEUE_DEVICE_BUSY);
+ __scsi_queue_insert(cmd, SCSI_MLQUEUE_DEVICE_BUSY, 0);
break;
}
}
diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index 17914a3..66505bb 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -414,8 +414,7 @@
device_initialize(dev);
starget->reap_ref = 1;
dev->parent = get_device(parent);
- sprintf(dev->bus_id, "target%d:%d:%d",
- shost->host_no, channel, id);
+ dev_set_name(dev, "target%d:%d:%d", shost->host_no, channel, id);
#ifndef CONFIG_SYSFS_DEPRECATED
dev->bus = &scsi_bus_type;
#endif
@@ -1024,7 +1023,7 @@
if (rescan || !scsi_device_created(sdev)) {
SCSI_LOG_SCAN_BUS(3, printk(KERN_INFO
"scsi scan: device exists on %s\n",
- sdev->sdev_gendev.bus_id));
+ dev_name(&sdev->sdev_gendev)));
if (sdevp)
*sdevp = sdev;
else
@@ -1163,7 +1162,7 @@
struct Scsi_Host *shost = dev_to_shost(starget->dev.parent);
SCSI_LOG_SCAN_BUS(3, printk(KERN_INFO "scsi scan: Sequential scan of"
- "%s\n", starget->dev.bus_id));
+ "%s\n", dev_name(&starget->dev)));
max_dev_lun = min(max_scsi_luns, shost->max_lun);
/*
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index 93c28f3..da63802 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -1079,16 +1079,14 @@
device_initialize(&sdev->sdev_gendev);
sdev->sdev_gendev.bus = &scsi_bus_type;
sdev->sdev_gendev.type = &scsi_dev_type;
- sprintf(sdev->sdev_gendev.bus_id,"%d:%d:%d:%d",
- sdev->host->host_no, sdev->channel, sdev->id,
- sdev->lun);
-
+ dev_set_name(&sdev->sdev_gendev, "%d:%d:%d:%d",
+ sdev->host->host_no, sdev->channel, sdev->id, sdev->lun);
+
device_initialize(&sdev->sdev_dev);
sdev->sdev_dev.parent = &sdev->sdev_gendev;
sdev->sdev_dev.class = &sdev_class;
- snprintf(sdev->sdev_dev.bus_id, BUS_ID_SIZE,
- "%d:%d:%d:%d", sdev->host->host_no,
- sdev->channel, sdev->id, sdev->lun);
+ dev_set_name(&sdev->sdev_dev, "%d:%d:%d:%d",
+ sdev->host->host_no, sdev->channel, sdev->id, sdev->lun);
sdev->scsi_level = starget->scsi_level;
transport_setup_device(&sdev->sdev_gendev);
spin_lock_irqsave(shost->host_lock, flags);
diff --git a/drivers/scsi/scsi_transport_fc.c b/drivers/scsi/scsi_transport_fc.c
index 062304d..5f77417 100644
--- a/drivers/scsi/scsi_transport_fc.c
+++ b/drivers/scsi/scsi_transport_fc.c
@@ -2407,8 +2407,12 @@
/*
* Notify the driver that the rport is now dead. The LLDD will
* also guarantee that any communication to the rport is terminated
+ *
+ * Avoid this call if we already called it when we preserved the
+ * rport for the binding.
*/
- if (i->f->dev_loss_tmo_callbk)
+ if (!(rport->flags & FC_RPORT_DEVLOSS_CALLBK_DONE) &&
+ (i->f->dev_loss_tmo_callbk))
i->f->dev_loss_tmo_callbk(rport);
transport_remove_device(dev);
@@ -2486,8 +2490,8 @@
device_initialize(dev); /* takes self reference */
dev->parent = get_device(&shost->shost_gendev); /* parent reference */
dev->release = fc_rport_dev_release;
- sprintf(dev->bus_id, "rport-%d:%d-%d",
- shost->host_no, channel, rport->number);
+ dev_set_name(dev, "rport-%d:%d-%d",
+ shost->host_no, channel, rport->number);
transport_setup_device(dev);
error = device_add(dev);
@@ -2647,7 +2651,8 @@
spin_lock_irqsave(shost->host_lock, flags);
rport->flags &= ~(FC_RPORT_FAST_FAIL_TIMEDOUT |
- FC_RPORT_DEVLOSS_PENDING);
+ FC_RPORT_DEVLOSS_PENDING |
+ FC_RPORT_DEVLOSS_CALLBK_DONE);
/* if target, initiate a scan */
if (rport->scsi_target_id != -1) {
@@ -2944,6 +2949,7 @@
struct fc_rport *rport =
container_of(work, struct fc_rport, dev_loss_work.work);
struct Scsi_Host *shost = rport_to_shost(rport);
+ struct fc_internal *i = to_fc_internal(shost->transportt);
struct fc_host_attrs *fc_host = shost_to_fc_host(shost);
unsigned long flags;
@@ -3011,6 +3017,7 @@
rport->roles = FC_PORT_ROLE_UNKNOWN;
rport->port_state = FC_PORTSTATE_NOTPRESENT;
rport->flags &= ~FC_RPORT_FAST_FAIL_TIMEDOUT;
+ rport->flags |= FC_RPORT_DEVLOSS_CALLBK_DONE;
/*
* Pre-emptively kill I/O rather than waiting for the work queue
@@ -3046,8 +3053,18 @@
* all attached scsi devices.
*/
fc_queue_work(shost, &rport->stgt_delete_work);
+
+ /*
+ * Notify the driver that the rport is now dead. The LLDD will
+ * also guarantee that any communication to the rport is terminated
+ *
+ * Note: we set the CALLBK_DONE flag above to correspond
+ */
+ if (i->f->dev_loss_tmo_callbk)
+ i->f->dev_loss_tmo_callbk(rport);
}
+
/**
* fc_timeout_fail_rport_io - Timeout handler for a fast io failing on a disconnected SCSI target.
* @work: rport to terminate io on.
@@ -3164,8 +3181,8 @@
device_initialize(dev); /* takes self reference */
dev->parent = get_device(pdev); /* takes parent reference */
dev->release = fc_vport_dev_release;
- sprintf(dev->bus_id, "vport-%d:%d-%d",
- shost->host_no, channel, vport->number);
+ dev_set_name(dev, "vport-%d:%d-%d",
+ shost->host_no, channel, vport->number);
transport_setup_device(dev);
error = device_add(dev);
@@ -3188,19 +3205,19 @@
*/
if (pdev != &shost->shost_gendev) {
error = sysfs_create_link(&shost->shost_gendev.kobj,
- &dev->kobj, dev->bus_id);
+ &dev->kobj, dev_name(dev));
if (error)
printk(KERN_ERR
"%s: Cannot create vport symlinks for "
"%s, err=%d\n",
- __func__, dev->bus_id, error);
+ __func__, dev_name(dev), error);
}
spin_lock_irqsave(shost->host_lock, flags);
vport->flags &= ~FC_VPORT_CREATING;
spin_unlock_irqrestore(shost->host_lock, flags);
dev_printk(KERN_NOTICE, pdev,
- "%s created via shost%d channel %d\n", dev->bus_id,
+ "%s created via shost%d channel %d\n", dev_name(dev),
shost->host_no, channel);
*ret_vport = vport;
@@ -3297,7 +3314,7 @@
return stat;
if (dev->parent != &shost->shost_gendev)
- sysfs_remove_link(&shost->shost_gendev.kobj, dev->bus_id);
+ sysfs_remove_link(&shost->shost_gendev.kobj, dev_name(dev));
transport_remove_device(dev);
device_del(dev);
transport_destroy_device(dev);
@@ -3329,7 +3346,7 @@
dev_printk(KERN_ERR, vport->dev.parent,
"%s: %s could not be deleted created via "
"shost%d channel %d - error %d\n", __func__,
- vport->dev.bus_id, vport->shost->host_no,
+ dev_name(&vport->dev), vport->shost->host_no,
vport->channel, stat);
}
diff --git a/drivers/scsi/scsi_transport_iscsi.c b/drivers/scsi/scsi_transport_iscsi.c
index 4a803eb..75c9297 100644
--- a/drivers/scsi/scsi_transport_iscsi.c
+++ b/drivers/scsi/scsi_transport_iscsi.c
@@ -187,8 +187,7 @@
ep->id = id;
ep->dev.class = &iscsi_endpoint_class;
- snprintf(ep->dev.bus_id, BUS_ID_SIZE, "ep-%llu",
- (unsigned long long) id);
+ dev_set_name(&ep->dev, "ep-%llu", (unsigned long long) id);
err = device_register(&ep->dev);
if (err)
goto free_ep;
@@ -724,8 +723,7 @@
}
session->target_id = id;
- snprintf(session->dev.bus_id, BUS_ID_SIZE, "session%u",
- session->sid);
+ dev_set_name(&session->dev, "session%u", session->sid);
err = device_add(&session->dev);
if (err) {
iscsi_cls_session_printk(KERN_ERR, session,
@@ -898,8 +896,7 @@
if (!get_device(&session->dev))
goto free_conn;
- snprintf(conn->dev.bus_id, BUS_ID_SIZE, "connection%d:%u",
- session->sid, cid);
+ dev_set_name(&conn->dev, "connection%d:%u", session->sid, cid);
conn->dev.parent = &session->dev;
conn->dev.release = iscsi_conn_release;
err = device_register(&conn->dev);
@@ -1816,7 +1813,7 @@
priv->t.create_work_queue = 1;
priv->dev.class = &iscsi_transport_class;
- snprintf(priv->dev.bus_id, BUS_ID_SIZE, "%s", tt->name);
+ dev_set_name(&priv->dev, "%s", tt->name);
err = device_register(&priv->dev);
if (err)
goto free_priv;
diff --git a/drivers/scsi/scsi_transport_sas.c b/drivers/scsi/scsi_transport_sas.c
index 3666093..50988cb 100644
--- a/drivers/scsi/scsi_transport_sas.c
+++ b/drivers/scsi/scsi_transport_sas.c
@@ -207,7 +207,7 @@
struct request_queue *q;
int error;
struct device *dev;
- char namebuf[BUS_ID_SIZE];
+ char namebuf[20];
const char *name;
void (*release)(struct device *);
@@ -219,7 +219,7 @@
if (rphy) {
q = blk_init_queue(sas_non_host_smp_request, NULL);
dev = &rphy->dev;
- name = dev->bus_id;
+ name = dev_name(dev);
release = NULL;
} else {
q = blk_init_queue(sas_host_smp_request, NULL);
@@ -629,10 +629,10 @@
INIT_LIST_HEAD(&phy->port_siblings);
if (scsi_is_sas_expander_device(parent)) {
struct sas_rphy *rphy = dev_to_rphy(parent);
- sprintf(phy->dev.bus_id, "phy-%d:%d:%d", shost->host_no,
+ dev_set_name(&phy->dev, "phy-%d:%d:%d", shost->host_no,
rphy->scsi_target_id, number);
} else
- sprintf(phy->dev.bus_id, "phy-%d:%d", shost->host_no, number);
+ dev_set_name(&phy->dev, "phy-%d:%d", shost->host_no, number);
transport_setup_device(&phy->dev);
@@ -770,7 +770,7 @@
int res;
res = sysfs_create_link(&port->dev.kobj, &phy->dev.kobj,
- phy->dev.bus_id);
+ dev_name(&phy->dev));
if (res)
goto err;
res = sysfs_create_link(&phy->dev.kobj, &port->dev.kobj, "port");
@@ -785,7 +785,7 @@
static void sas_port_delete_link(struct sas_port *port,
struct sas_phy *phy)
{
- sysfs_remove_link(&port->dev.kobj, phy->dev.bus_id);
+ sysfs_remove_link(&port->dev.kobj, dev_name(&phy->dev));
sysfs_remove_link(&phy->dev.kobj, "port");
}
@@ -821,11 +821,11 @@
if (scsi_is_sas_expander_device(parent)) {
struct sas_rphy *rphy = dev_to_rphy(parent);
- sprintf(port->dev.bus_id, "port-%d:%d:%d", shost->host_no,
- rphy->scsi_target_id, port->port_identifier);
+ dev_set_name(&port->dev, "port-%d:%d:%d", shost->host_no,
+ rphy->scsi_target_id, port->port_identifier);
} else
- sprintf(port->dev.bus_id, "port-%d:%d", shost->host_no,
- port->port_identifier);
+ dev_set_name(&port->dev, "port-%d:%d", shost->host_no,
+ port->port_identifier);
transport_setup_device(&port->dev);
@@ -935,7 +935,7 @@
if (port->is_backlink) {
struct device *parent = port->dev.parent;
- sysfs_remove_link(&port->dev.kobj, parent->bus_id);
+ sysfs_remove_link(&port->dev.kobj, dev_name(parent));
port->is_backlink = 0;
}
@@ -984,7 +984,8 @@
/* If this trips, you added a phy that was already
* part of a different port */
if (unlikely(tmp != phy)) {
- dev_printk(KERN_ERR, &port->dev, "trying to add phy %s fails: it's already part of another port\n", phy->dev.bus_id);
+ dev_printk(KERN_ERR, &port->dev, "trying to add phy %s fails: it's already part of another port\n",
+ dev_name(&phy->dev));
BUG();
}
} else {
@@ -1023,7 +1024,7 @@
return;
port->is_backlink = 1;
res = sysfs_create_link(&port->dev.kobj, &parent->kobj,
- parent->bus_id);
+ dev_name(parent));
if (res)
goto err;
return;
@@ -1367,11 +1368,12 @@
rdev->rphy.dev.release = sas_end_device_release;
if (scsi_is_sas_expander_device(parent->dev.parent)) {
struct sas_rphy *rphy = dev_to_rphy(parent->dev.parent);
- sprintf(rdev->rphy.dev.bus_id, "end_device-%d:%d:%d",
- shost->host_no, rphy->scsi_target_id, parent->port_identifier);
+ dev_set_name(&rdev->rphy.dev, "end_device-%d:%d:%d",
+ shost->host_no, rphy->scsi_target_id,
+ parent->port_identifier);
} else
- sprintf(rdev->rphy.dev.bus_id, "end_device-%d:%d",
- shost->host_no, parent->port_identifier);
+ dev_set_name(&rdev->rphy.dev, "end_device-%d:%d",
+ shost->host_no, parent->port_identifier);
rdev->rphy.identify.device_type = SAS_END_DEVICE;
sas_rphy_initialize(&rdev->rphy);
transport_setup_device(&rdev->rphy.dev);
@@ -1411,8 +1413,8 @@
mutex_lock(&sas_host->lock);
rdev->rphy.scsi_target_id = sas_host->next_expander_id++;
mutex_unlock(&sas_host->lock);
- sprintf(rdev->rphy.dev.bus_id, "expander-%d:%d",
- shost->host_no, rdev->rphy.scsi_target_id);
+ dev_set_name(&rdev->rphy.dev, "expander-%d:%d",
+ shost->host_no, rdev->rphy.scsi_target_id);
rdev->rphy.identify.device_type = type;
sas_rphy_initialize(&rdev->rphy);
transport_setup_device(&rdev->rphy.dev);
@@ -1445,7 +1447,7 @@
transport_add_device(&rphy->dev);
transport_configure_device(&rphy->dev);
if (sas_bsg_initialize(shost, rphy))
- printk("fail to a bsg device %s\n", rphy->dev.bus_id);
+ printk("fail to a bsg device %s\n", dev_name(&rphy->dev));
mutex_lock(&sas_host->lock);
diff --git a/drivers/scsi/scsi_transport_srp.c b/drivers/scsi/scsi_transport_srp.c
index 8a7af95..21a045e 100644
--- a/drivers/scsi/scsi_transport_srp.c
+++ b/drivers/scsi/scsi_transport_srp.c
@@ -212,7 +212,7 @@
rport->roles = ids->roles;
id = atomic_inc_return(&to_srp_host_attrs(shost)->next_port_id);
- sprintf(rport->dev.bus_id, "port-%d:%d", shost->host_no, id);
+ dev_set_name(&rport->dev, "port-%d:%d", shost->host_no, id);
transport_setup_device(&rport->dev);
diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index e035c11..d57566b 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -1830,7 +1830,7 @@
device_initialize(&sdkp->dev);
sdkp->dev.parent = &sdp->sdev_gendev;
sdkp->dev.class = &sd_disk_class;
- strncpy(sdkp->dev.bus_id, sdp->sdev_gendev.bus_id, BUS_ID_SIZE);
+ dev_set_name(&sdkp->dev, dev_name(&sdp->sdev_gendev));
if (device_add(&sdkp->dev))
goto out_free_index;
diff --git a/drivers/scsi/sd_dif.c b/drivers/scsi/sd_dif.c
index 3ebb1f2..184dff4 100644
--- a/drivers/scsi/sd_dif.c
+++ b/drivers/scsi/sd_dif.c
@@ -142,7 +142,7 @@
static void sd_dif_type1_set_tag(void *prot, void *tag_buf, unsigned int sectors)
{
struct sd_dif_tuple *sdt = prot;
- char *tag = tag_buf;
+ u8 *tag = tag_buf;
unsigned int i, j;
for (i = 0, j = 0 ; i < sectors ; i++, j += 2, sdt++) {
@@ -154,7 +154,7 @@
static void sd_dif_type1_get_tag(void *prot, void *tag_buf, unsigned int sectors)
{
struct sd_dif_tuple *sdt = prot;
- char *tag = tag_buf;
+ u8 *tag = tag_buf;
unsigned int i, j;
for (i = 0, j = 0 ; i < sectors ; i++, j += 2, sdt++) {
@@ -256,7 +256,7 @@
static void sd_dif_type3_set_tag(void *prot, void *tag_buf, unsigned int sectors)
{
struct sd_dif_tuple *sdt = prot;
- char *tag = tag_buf;
+ u8 *tag = tag_buf;
unsigned int i, j;
for (i = 0, j = 0 ; i < sectors ; i++, j += 6, sdt++) {
@@ -269,7 +269,7 @@
static void sd_dif_type3_get_tag(void *prot, void *tag_buf, unsigned int sectors)
{
struct sd_dif_tuple *sdt = prot;
- char *tag = tag_buf;
+ u8 *tag = tag_buf;
unsigned int i, j;
for (i = 0, j = 0 ; i < sectors ; i++, j += 2, sdt++) {
@@ -374,7 +374,10 @@
else
csum_convert = 0;
+ BUG_ON(dif && (scmd->cmnd[0] == READ_6 || scmd->cmnd[0] == WRITE_6));
+
switch (scmd->cmnd[0]) {
+ case READ_6:
case READ_10:
case READ_12:
case READ_16:
@@ -390,6 +393,7 @@
break;
+ case WRITE_6:
case WRITE_10:
case WRITE_12:
case WRITE_16:
@@ -475,8 +479,9 @@
error:
kunmap_atomic(sdt, KM_USER0);
- sd_printk(KERN_ERR, sdkp, "%s: virt %u, phys %u, ref %u\n",
- __func__, virt, phys, be32_to_cpu(sdt->ref_tag));
+ sd_printk(KERN_ERR, sdkp, "%s: virt %u, phys %u, ref %u, app %4x\n",
+ __func__, virt, phys, be32_to_cpu(sdt->ref_tag),
+ be16_to_cpu(sdt->app_tag));
return -EIO;
}
diff --git a/drivers/scsi/ses.c b/drivers/scsi/ses.c
index 7f0df29..e946e05 100644
--- a/drivers/scsi/ses.c
+++ b/drivers/scsi/ses.c
@@ -526,7 +526,7 @@
if (!scomp)
goto err_free;
- edev = enclosure_register(cdev->parent, sdev->sdev_gendev.bus_id,
+ edev = enclosure_register(cdev->parent, dev_name(&sdev->sdev_gendev),
components, &ses_enclosure_callbacks);
if (IS_ERR(edev)) {
err = PTR_ERR(edev);
diff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c
index 5103855..8f0bd3f 100644
--- a/drivers/scsi/sg.c
+++ b/drivers/scsi/sg.c
@@ -1669,6 +1669,8 @@
md->pages = req_schp->pages;
md->page_order = req_schp->page_order;
md->nr_entries = req_schp->k_use_sg;
+ md->offset = 0;
+ md->null_mapped = hp->dxferp ? 0 : 1;
}
if (iov_count)
diff --git a/drivers/scsi/sgiwd93.c b/drivers/scsi/sgiwd93.c
index 31fe605..0807b26 100644
--- a/drivers/scsi/sgiwd93.c
+++ b/drivers/scsi/sgiwd93.c
@@ -297,7 +297,7 @@
return err;
}
-static void __exit sgiwd93_remove(struct platform_device *pdev)
+static int __exit sgiwd93_remove(struct platform_device *pdev)
{
struct Scsi_Host *host = platform_get_drvdata(pdev);
struct ip22_hostdata *hdata = (struct ip22_hostdata *) host->hostdata;
@@ -307,6 +307,7 @@
free_irq(pd->irq, host);
dma_free_noncoherent(&pdev->dev, HPC_DMA_SIZE, hdata->cpu, hdata->dma);
scsi_host_put(host);
+ return 0;
}
static struct platform_driver sgiwd93_driver = {
diff --git a/drivers/scsi/sim710.c b/drivers/scsi/sim710.c
index d63d229..6dc8b84 100644
--- a/drivers/scsi/sim710.c
+++ b/drivers/scsi/sim710.c
@@ -102,7 +102,7 @@
struct NCR_700_Host_Parameters *hostdata =
kzalloc(sizeof(struct NCR_700_Host_Parameters), GFP_KERNEL);
- printk(KERN_NOTICE "sim710: %s\n", dev->bus_id);
+ printk(KERN_NOTICE "sim710: %s\n", dev_name(dev));
printk(KERN_NOTICE "sim710: irq = %d, clock = %d, base = 0x%lx, scsi_id = %d\n",
irq, clock, base_addr, scsi_id);
@@ -305,7 +305,7 @@
scsi_id = ffs(val) - 1;
if(scsi_id > 7 || (val & ~(1<<scsi_id)) != 0) {
- printk(KERN_ERR "sim710.c, EISA card %s has incorrect scsi_id, setting to 7\n", dev->bus_id);
+ printk(KERN_ERR "sim710.c, EISA card %s has incorrect scsi_id, setting to 7\n", dev_name(dev));
scsi_id = 7;
}
} else {
diff --git a/drivers/scsi/sni_53c710.c b/drivers/scsi/sni_53c710.c
index 2bbef4c..77f0b2c 100644
--- a/drivers/scsi/sni_53c710.c
+++ b/drivers/scsi/sni_53c710.c
@@ -78,8 +78,7 @@
base = res->start;
hostdata = kzalloc(sizeof(*hostdata), GFP_KERNEL);
if (!hostdata) {
- printk(KERN_ERR "%s: Failed to allocate host data\n",
- dev->dev.bus_id);
+ dev_printk(KERN_ERR, dev, "Failed to allocate host data\n");
return -ENOMEM;
}
diff --git a/drivers/scsi/st.c b/drivers/scsi/st.c
index 7f3f317..c6f19ee 100644
--- a/drivers/scsi/st.c
+++ b/drivers/scsi/st.c
@@ -17,7 +17,7 @@
Last modified: 18-JAN-1998 Richard Gooch <rgooch@atnf.csiro.au> Devfs support
*/
-static const char *verstr = "20080504";
+static const char *verstr = "20081215";
#include <linux/module.h>
@@ -182,18 +182,16 @@
static int modes_defined;
-static struct st_buffer *new_tape_buffer(int, int, int);
static int enlarge_buffer(struct st_buffer *, int, int);
static void clear_buffer(struct st_buffer *);
static void normalize_buffer(struct st_buffer *);
static int append_to_buffer(const char __user *, struct st_buffer *, int);
static int from_buffer(struct st_buffer *, char __user *, int);
static void move_buffer_data(struct st_buffer *, int);
-static void buf_to_sg(struct st_buffer *, unsigned int);
-static int sgl_map_user_pages(struct scatterlist *, const unsigned int,
+static int sgl_map_user_pages(struct st_buffer *, const unsigned int,
unsigned long, size_t, int);
-static int sgl_unmap_user_pages(struct scatterlist *, const unsigned int, int);
+static int sgl_unmap_user_pages(struct st_buffer *, const unsigned int, int);
static int st_probe(struct device *);
static int st_remove(struct device *);
@@ -435,22 +433,6 @@
return (-EIO);
}
-
-/* Wakeup from interrupt */
-static void st_sleep_done(void *data, char *sense, int result, int resid)
-{
- struct st_request *SRpnt = data;
- struct scsi_tape *STp = SRpnt->stp;
-
- memcpy(SRpnt->sense, sense, SCSI_SENSE_BUFFERSIZE);
- (STp->buffer)->cmdstat.midlevel_result = SRpnt->result = result;
- (STp->buffer)->cmdstat.residual = resid;
- DEB( STp->write_pending = 0; )
-
- if (SRpnt->waiting)
- complete(SRpnt->waiting);
-}
-
static struct st_request *st_allocate_request(struct scsi_tape *stp)
{
struct st_request *streq;
@@ -475,6 +457,63 @@
kfree(streq);
}
+static void st_scsi_execute_end(struct request *req, int uptodate)
+{
+ struct st_request *SRpnt = req->end_io_data;
+ struct scsi_tape *STp = SRpnt->stp;
+
+ STp->buffer->cmdstat.midlevel_result = SRpnt->result = req->errors;
+ STp->buffer->cmdstat.residual = req->data_len;
+
+ if (SRpnt->waiting)
+ complete(SRpnt->waiting);
+
+ blk_rq_unmap_user(SRpnt->bio);
+ __blk_put_request(req->q, req);
+}
+
+static int st_scsi_execute(struct st_request *SRpnt, const unsigned char *cmd,
+ int data_direction, void *buffer, unsigned bufflen,
+ int timeout, int retries)
+{
+ struct request *req;
+ struct rq_map_data *mdata = &SRpnt->stp->buffer->map_data;
+ int err = 0;
+ int write = (data_direction == DMA_TO_DEVICE);
+
+ req = blk_get_request(SRpnt->stp->device->request_queue, write,
+ GFP_KERNEL);
+ if (!req)
+ return DRIVER_ERROR << 24;
+
+ req->cmd_type = REQ_TYPE_BLOCK_PC;
+ req->cmd_flags |= REQ_QUIET;
+
+ mdata->null_mapped = 1;
+
+ if (bufflen) {
+ err = blk_rq_map_user(req->q, req, mdata, NULL, bufflen,
+ GFP_KERNEL);
+ if (err) {
+ blk_put_request(req);
+ return DRIVER_ERROR << 24;
+ }
+ }
+
+ SRpnt->bio = req->bio;
+ req->cmd_len = COMMAND_SIZE(cmd[0]);
+ memset(req->cmd, 0, BLK_MAX_CDB);
+ memcpy(req->cmd, cmd, req->cmd_len);
+ req->sense = SRpnt->sense;
+ req->sense_len = 0;
+ req->timeout = timeout;
+ req->retries = retries;
+ req->end_io_data = SRpnt;
+
+ blk_execute_rq_nowait(req->q, NULL, req, 1, st_scsi_execute_end);
+ return 0;
+}
+
/* Do the scsi command. Waits until command performed if do_wait is true.
Otherwise write_behind_check() is used to check that the command
has finished. */
@@ -483,6 +522,8 @@
int bytes, int direction, int timeout, int retries, int do_wait)
{
struct completion *waiting;
+ struct rq_map_data *mdata = &STp->buffer->map_data;
+ int ret;
/* if async, make sure there's no command outstanding */
if (!do_wait && ((STp->buffer)->last_SRpnt)) {
@@ -510,21 +551,27 @@
init_completion(waiting);
SRpnt->waiting = waiting;
- if (!STp->buffer->do_dio)
- buf_to_sg(STp->buffer, bytes);
+ if (STp->buffer->do_dio) {
+ mdata->nr_entries = STp->buffer->sg_segs;
+ mdata->pages = STp->buffer->mapped_pages;
+ } else {
+ mdata->nr_entries =
+ DIV_ROUND_UP(bytes, PAGE_SIZE << mdata->page_order);
+ STp->buffer->map_data.pages = STp->buffer->reserved_pages;
+ STp->buffer->map_data.offset = 0;
+ }
memcpy(SRpnt->cmd, cmd, sizeof(SRpnt->cmd));
STp->buffer->cmdstat.have_sense = 0;
STp->buffer->syscall_result = 0;
- if (scsi_execute_async(STp->device, cmd, COMMAND_SIZE(cmd[0]), direction,
- &((STp->buffer)->sg[0]), bytes, (STp->buffer)->sg_segs,
- timeout, retries, SRpnt, st_sleep_done, GFP_KERNEL)) {
+ ret = st_scsi_execute(SRpnt, cmd, direction, NULL, bytes, timeout,
+ retries);
+ if (ret) {
/* could not allocate the buffer or request was too large */
(STp->buffer)->syscall_result = (-EBUSY);
(STp->buffer)->last_SRpnt = NULL;
- }
- else if (do_wait) {
+ } else if (do_wait) {
wait_for_completion(waiting);
SRpnt->waiting = NULL;
(STp->buffer)->syscall_result = st_chk_result(STp, SRpnt);
@@ -533,28 +580,6 @@
return SRpnt;
}
-static int st_scsi_kern_execute(struct st_request *streq,
- const unsigned char *cmd, int data_direction,
- void *buffer, unsigned bufflen, int timeout,
- int retries)
-{
- struct scsi_tape *stp = streq->stp;
- int ret, resid;
-
- stp->buffer->cmdstat.have_sense = 0;
- memcpy(streq->cmd, cmd, sizeof(streq->cmd));
-
- ret = scsi_execute(stp->device, cmd, data_direction, buffer, bufflen,
- streq->sense, timeout, retries, 0, &resid);
- if (driver_byte(ret) & DRIVER_ERROR)
- return -EBUSY;
-
- stp->buffer->cmdstat.midlevel_result = streq->result = ret;
- stp->buffer->cmdstat.residual = resid;
- stp->buffer->syscall_result = st_chk_result(stp, streq);
-
- return 0;
-}
/* Handle the write-behind checking (waits for completion). Returns -ENOSPC if
write has been correct but EOM early warning reached, -EIO if write ended in
@@ -627,7 +652,6 @@
{
struct st_request *SRpnt;
unsigned char cmd[MAX_COMMAND_SIZE];
- int ret;
cmd[0] = SPACE;
cmd[1] = 0x01; /* Space FileMarks */
@@ -641,26 +665,20 @@
DEBC(printk(ST_DEB_MSG "%s: Stepping over filemark %s.\n",
tape_name(STp), forward ? "forward" : "backward"));
- SRpnt = st_allocate_request(STp);
+ SRpnt = st_do_scsi(NULL, STp, cmd, 0, DMA_NONE,
+ STp->device->request_queue->rq_timeout,
+ MAX_RETRIES, 1);
if (!SRpnt)
- return STp->buffer->syscall_result;
+ return (STp->buffer)->syscall_result;
- ret = st_scsi_kern_execute(SRpnt, cmd, DMA_NONE, NULL, 0,
- STp->device->request_queue->rq_timeout,
- MAX_RETRIES);
- if (ret)
- goto out;
-
- ret = STp->buffer->syscall_result;
+ st_release_request(SRpnt);
+ SRpnt = NULL;
if ((STp->buffer)->cmdstat.midlevel_result != 0)
printk(KERN_ERR "%s: Stepping over filemark %s failed.\n",
tape_name(STp), forward ? "forward" : "backward");
-out:
- st_release_request(SRpnt);
-
- return ret;
+ return (STp->buffer)->syscall_result;
}
@@ -881,24 +899,21 @@
int attentions, waits, max_wait, scode;
int retval = CHKRES_READY, new_session = 0;
unsigned char cmd[MAX_COMMAND_SIZE];
- struct st_request *SRpnt;
+ struct st_request *SRpnt = NULL;
struct st_cmdstatus *cmdstatp = &STp->buffer->cmdstat;
- SRpnt = st_allocate_request(STp);
- if (!SRpnt)
- return STp->buffer->syscall_result;
-
max_wait = do_wait ? ST_BLOCK_SECONDS : 0;
for (attentions=waits=0; ; ) {
memset((void *) &cmd[0], 0, MAX_COMMAND_SIZE);
cmd[0] = TEST_UNIT_READY;
+ SRpnt = st_do_scsi(SRpnt, STp, cmd, 0, DMA_NONE,
+ STp->long_timeout, MAX_READY_RETRIES, 1);
- retval = st_scsi_kern_execute(SRpnt, cmd, DMA_NONE, NULL, 0,
- STp->long_timeout,
- MAX_READY_RETRIES);
- if (retval)
+ if (!SRpnt) {
+ retval = (STp->buffer)->syscall_result;
break;
+ }
if (cmdstatp->have_sense) {
@@ -942,8 +957,8 @@
break;
}
- st_release_request(SRpnt);
-
+ if (SRpnt != NULL)
+ st_release_request(SRpnt);
return retval;
}
@@ -1020,24 +1035,17 @@
}
}
- SRpnt = st_allocate_request(STp);
- if (!SRpnt) {
- retval = STp->buffer->syscall_result;
- goto err_out;
- }
-
if (STp->omit_blklims)
STp->min_block = STp->max_block = (-1);
else {
memset((void *) &cmd[0], 0, MAX_COMMAND_SIZE);
cmd[0] = READ_BLOCK_LIMITS;
- retval = st_scsi_kern_execute(SRpnt, cmd, DMA_FROM_DEVICE,
- STp->buffer->b_data, 6,
- STp->device->request_queue->rq_timeout,
- MAX_READY_RETRIES);
- if (retval) {
- st_release_request(SRpnt);
+ SRpnt = st_do_scsi(SRpnt, STp, cmd, 6, DMA_FROM_DEVICE,
+ STp->device->request_queue->rq_timeout,
+ MAX_READY_RETRIES, 1);
+ if (!SRpnt) {
+ retval = (STp->buffer)->syscall_result;
goto err_out;
}
@@ -1061,12 +1069,11 @@
cmd[0] = MODE_SENSE;
cmd[4] = 12;
- retval = st_scsi_kern_execute(SRpnt, cmd, DMA_FROM_DEVICE,
- STp->buffer->b_data, 12,
- STp->device->request_queue->rq_timeout,
- MAX_READY_RETRIES);
- if (retval) {
- st_release_request(SRpnt);
+ SRpnt = st_do_scsi(SRpnt, STp, cmd, 12, DMA_FROM_DEVICE,
+ STp->device->request_queue->rq_timeout,
+ MAX_READY_RETRIES, 1);
+ if (!SRpnt) {
+ retval = (STp->buffer)->syscall_result;
goto err_out;
}
@@ -1296,17 +1303,11 @@
cmd[0] = WRITE_FILEMARKS;
cmd[4] = 1 + STp->two_fm;
- SRpnt = st_allocate_request(STp);
+ SRpnt = st_do_scsi(NULL, STp, cmd, 0, DMA_NONE,
+ STp->device->request_queue->rq_timeout,
+ MAX_WRITE_RETRIES, 1);
if (!SRpnt) {
- result = STp->buffer->syscall_result;
- goto out;
- }
-
- result = st_scsi_kern_execute(SRpnt, cmd, DMA_NONE, NULL, 0,
- STp->device->request_queue->rq_timeout,
- MAX_WRITE_RETRIES);
- if (result) {
- st_release_request(SRpnt);
+ result = (STp->buffer)->syscall_result;
goto out;
}
@@ -1471,8 +1472,8 @@
if (i && ((unsigned long)buf & queue_dma_alignment(
STp->device->request_queue)) == 0) {
- i = sgl_map_user_pages(&(STbp->sg[0]), STbp->use_sg,
- (unsigned long)buf, count, (is_read ? READ : WRITE));
+ i = sgl_map_user_pages(STbp, STbp->use_sg, (unsigned long)buf,
+ count, (is_read ? READ : WRITE));
if (i > 0) {
STbp->do_dio = i;
STbp->buffer_bytes = 0; /* can be used as transfer counter */
@@ -1480,7 +1481,6 @@
else
STbp->do_dio = 0; /* fall back to buffering with any error */
STbp->sg_segs = STbp->do_dio;
- STbp->frp_sg_current = 0;
DEB(
if (STbp->do_dio) {
STp->nbr_dio++;
@@ -1526,7 +1526,7 @@
STbp = STp->buffer;
if (STbp->do_dio) {
- sgl_unmap_user_pages(&(STbp->sg[0]), STbp->do_dio, is_read);
+ sgl_unmap_user_pages(STbp, STbp->do_dio, is_read);
STbp->do_dio = 0;
STbp->sg_segs = 0;
}
@@ -2372,7 +2372,6 @@
{
unsigned char cmd[MAX_COMMAND_SIZE];
struct st_request *SRpnt;
- int ret;
memset(cmd, 0, MAX_COMMAND_SIZE);
cmd[0] = MODE_SENSE;
@@ -2381,17 +2380,14 @@
cmd[2] = page;
cmd[4] = 255;
- SRpnt = st_allocate_request(STp);
- if (!SRpnt)
- return STp->buffer->syscall_result;
+ SRpnt = st_do_scsi(NULL, STp, cmd, cmd[4], DMA_FROM_DEVICE,
+ STp->device->request_queue->rq_timeout, 0, 1);
+ if (SRpnt == NULL)
+ return (STp->buffer)->syscall_result;
- ret = st_scsi_kern_execute(SRpnt, cmd, DMA_FROM_DEVICE,
- STp->buffer->b_data, cmd[4],
- STp->device->request_queue->rq_timeout,
- MAX_RETRIES);
st_release_request(SRpnt);
- return ret ? : STp->buffer->syscall_result;
+ return STp->buffer->syscall_result;
}
@@ -2399,9 +2395,10 @@
in the buffer is correctly formatted. The long timeout is used if slow is non-zero. */
static int write_mode_page(struct scsi_tape *STp, int page, int slow)
{
- int pgo, timeout, ret = 0;
+ int pgo;
unsigned char cmd[MAX_COMMAND_SIZE];
struct st_request *SRpnt;
+ int timeout;
memset(cmd, 0, MAX_COMMAND_SIZE);
cmd[0] = MODE_SELECT;
@@ -2415,21 +2412,16 @@
(STp->buffer)->b_data[MH_OFF_DEV_SPECIFIC] &= ~MH_BIT_WP;
(STp->buffer)->b_data[pgo + MP_OFF_PAGE_NBR] &= MP_MSK_PAGE_NBR;
- SRpnt = st_allocate_request(STp);
- if (!SRpnt)
- return ret;
-
- timeout = slow ? STp->long_timeout :
- STp->device->request_queue->rq_timeout;
-
- ret = st_scsi_kern_execute(SRpnt, cmd, DMA_TO_DEVICE,
- STp->buffer->b_data, cmd[4], timeout, 0);
- if (!ret)
- ret = STp->buffer->syscall_result;
+ timeout = slow ?
+ STp->long_timeout : STp->device->request_queue->rq_timeout;
+ SRpnt = st_do_scsi(NULL, STp, cmd, cmd[4], DMA_TO_DEVICE,
+ timeout, 0, 1);
+ if (SRpnt == NULL)
+ return (STp->buffer)->syscall_result;
st_release_request(SRpnt);
- return ret;
+ return STp->buffer->syscall_result;
}
@@ -2547,16 +2539,13 @@
printk(ST_DEB_MSG "%s: Loading tape.\n", name);
);
- SRpnt = st_allocate_request(STp);
+ SRpnt = st_do_scsi(NULL, STp, cmd, 0, DMA_NONE,
+ timeout, MAX_RETRIES, 1);
if (!SRpnt)
- return STp->buffer->syscall_result;
-
- retval = st_scsi_kern_execute(SRpnt, cmd, DMA_NONE, NULL, 0, timeout,
- MAX_RETRIES);
- if (retval)
- goto out;
+ return (STp->buffer)->syscall_result;
retval = (STp->buffer)->syscall_result;
+ st_release_request(SRpnt);
if (!retval) { /* SCSI command successful */
@@ -2575,8 +2564,6 @@
STps = &(STp->ps[STp->partition]);
STps->drv_file = STps->drv_block = (-1);
}
-out:
- st_release_request(SRpnt);
return retval;
}
@@ -2852,15 +2839,12 @@
return (-ENOSYS);
}
- SRpnt = st_allocate_request(STp);
+ SRpnt = st_do_scsi(NULL, STp, cmd, datalen, direction,
+ timeout, MAX_RETRIES, 1);
if (!SRpnt)
return (STp->buffer)->syscall_result;
- ioctl_result = st_scsi_kern_execute(SRpnt, cmd, direction,
- STp->buffer->b_data, datalen,
- timeout, MAX_RETRIES);
- if (!ioctl_result)
- ioctl_result = (STp->buffer)->syscall_result;
+ ioctl_result = (STp->buffer)->syscall_result;
if (!ioctl_result) { /* SCSI command successful */
st_release_request(SRpnt);
@@ -3022,17 +3006,11 @@
if (!logical && !STp->scsi2_logical)
scmd[1] = 1;
}
-
- SRpnt = st_allocate_request(STp);
+ SRpnt = st_do_scsi(NULL, STp, scmd, 20, DMA_FROM_DEVICE,
+ STp->device->request_queue->rq_timeout,
+ MAX_READY_RETRIES, 1);
if (!SRpnt)
- return STp->buffer->syscall_result;
-
- result = st_scsi_kern_execute(SRpnt, scmd, DMA_FROM_DEVICE,
- STp->buffer->b_data, 20,
- STp->device->request_queue->rq_timeout,
- MAX_READY_RETRIES);
- if (result)
- goto out;
+ return (STp->buffer)->syscall_result;
if ((STp->buffer)->syscall_result != 0 ||
(STp->device->scsi_level >= SCSI_2 &&
@@ -3060,7 +3038,6 @@
DEBC(printk(ST_DEB_MSG "%s: Got tape pos. blk %d part %d.\n", name,
*block, *partition));
}
-out:
st_release_request(SRpnt);
SRpnt = NULL;
@@ -3135,14 +3112,10 @@
timeout = STp->device->request_queue->rq_timeout;
}
- SRpnt = st_allocate_request(STp);
+ SRpnt = st_do_scsi(NULL, STp, scmd, 0, DMA_NONE,
+ timeout, MAX_READY_RETRIES, 1);
if (!SRpnt)
- return STp->buffer->syscall_result;
-
- result = st_scsi_kern_execute(SRpnt, scmd, DMA_NONE, NULL, 0,
- timeout, MAX_READY_RETRIES);
- if (result)
- goto out;
+ return (STp->buffer)->syscall_result;
STps->drv_block = STps->drv_file = (-1);
STps->eof = ST_NOEOF;
@@ -3167,7 +3140,7 @@
STps->drv_block = STps->drv_file = 0;
result = 0;
}
-out:
+
st_release_request(SRpnt);
SRpnt = NULL;
@@ -3696,38 +3669,34 @@
/* Try to allocate a new tape buffer. Calling function must not hold
dev_arr_lock. */
-static struct st_buffer *
- new_tape_buffer(int from_initialization, int need_dma, int max_sg)
+static struct st_buffer *new_tape_buffer(int need_dma, int max_sg)
{
- int i, got = 0;
- gfp_t priority;
struct st_buffer *tb;
- if (from_initialization)
- priority = GFP_ATOMIC;
- else
- priority = GFP_KERNEL;
-
- i = sizeof(struct st_buffer) + (max_sg - 1) * sizeof(struct scatterlist) +
- max_sg * sizeof(struct st_buf_fragment);
- tb = kzalloc(i, priority);
+ tb = kzalloc(sizeof(struct st_buffer), GFP_ATOMIC);
if (!tb) {
printk(KERN_NOTICE "st: Can't allocate new tape buffer.\n");
return NULL;
}
- tb->frp_segs = tb->orig_frp_segs = 0;
+ tb->frp_segs = 0;
tb->use_sg = max_sg;
- tb->frp = (struct st_buf_fragment *)(&(tb->sg[0]) + max_sg);
-
tb->dma = need_dma;
- tb->buffer_size = got;
- sg_init_table(tb->sg, max_sg);
+ tb->buffer_size = 0;
+
+ tb->reserved_pages = kzalloc(max_sg * sizeof(struct page *),
+ GFP_ATOMIC);
+ if (!tb->reserved_pages) {
+ kfree(tb);
+ return NULL;
+ }
return tb;
}
/* Try to allocate enough space in the tape buffer */
+#define ST_MAX_ORDER 6
+
static int enlarge_buffer(struct st_buffer * STbuffer, int new_size, int need_dma)
{
int segs, nbr, max_segs, b_size, order, got;
@@ -3747,33 +3716,45 @@
priority = GFP_KERNEL | __GFP_NOWARN;
if (need_dma)
priority |= GFP_DMA;
- for (b_size = PAGE_SIZE, order=0; order <= 6 &&
- b_size < new_size - STbuffer->buffer_size;
- order++, b_size *= 2)
- ; /* empty */
+
+ if (STbuffer->cleared)
+ priority |= __GFP_ZERO;
+
+ if (STbuffer->frp_segs) {
+ order = STbuffer->map_data.page_order;
+ b_size = PAGE_SIZE << order;
+ } else {
+ for (b_size = PAGE_SIZE, order = 0;
+ order < ST_MAX_ORDER && b_size < new_size;
+ order++, b_size *= 2)
+ ; /* empty */
+ }
+ if (max_segs * (PAGE_SIZE << order) < new_size) {
+ if (order == ST_MAX_ORDER)
+ return 0;
+ normalize_buffer(STbuffer);
+ return enlarge_buffer(STbuffer, new_size, need_dma);
+ }
for (segs = STbuffer->frp_segs, got = STbuffer->buffer_size;
segs < max_segs && got < new_size;) {
- STbuffer->frp[segs].page = alloc_pages(priority, order);
- if (STbuffer->frp[segs].page == NULL) {
- if (new_size - got <= (max_segs - segs) * b_size / 2) {
- b_size /= 2; /* Large enough for the rest of the buffers */
- order--;
- continue;
- }
+ struct page *page;
+
+ page = alloc_pages(priority, order);
+ if (!page) {
DEB(STbuffer->buffer_size = got);
normalize_buffer(STbuffer);
return 0;
}
- STbuffer->frp[segs].length = b_size;
+
STbuffer->frp_segs += 1;
got += b_size;
STbuffer->buffer_size = got;
- if (STbuffer->cleared)
- memset(page_address(STbuffer->frp[segs].page), 0, b_size);
+ STbuffer->reserved_pages[segs] = page;
segs++;
}
- STbuffer->b_data = page_address(STbuffer->frp[0].page);
+ STbuffer->b_data = page_address(STbuffer->reserved_pages[0]);
+ STbuffer->map_data.page_order = order;
return 1;
}
@@ -3785,7 +3766,8 @@
int i;
for (i=0; i < st_bp->frp_segs; i++)
- memset(page_address(st_bp->frp[i].page), 0, st_bp->frp[i].length);
+ memset(page_address(st_bp->reserved_pages[i]), 0,
+ PAGE_SIZE << st_bp->map_data.page_order);
st_bp->cleared = 1;
}
@@ -3793,16 +3775,16 @@
/* Release the extra buffer */
static void normalize_buffer(struct st_buffer * STbuffer)
{
- int i, order;
+ int i, order = STbuffer->map_data.page_order;
- for (i = STbuffer->orig_frp_segs; i < STbuffer->frp_segs; i++) {
- order = get_order(STbuffer->frp[i].length);
- __free_pages(STbuffer->frp[i].page, order);
- STbuffer->buffer_size -= STbuffer->frp[i].length;
+ for (i = 0; i < STbuffer->frp_segs; i++) {
+ __free_pages(STbuffer->reserved_pages[i], order);
+ STbuffer->buffer_size -= (PAGE_SIZE << order);
}
- STbuffer->frp_segs = STbuffer->orig_frp_segs;
- STbuffer->frp_sg_current = 0;
+ STbuffer->frp_segs = 0;
STbuffer->sg_segs = 0;
+ STbuffer->map_data.page_order = 0;
+ STbuffer->map_data.offset = 0;
}
@@ -3811,18 +3793,19 @@
static int append_to_buffer(const char __user *ubp, struct st_buffer * st_bp, int do_count)
{
int i, cnt, res, offset;
+ int length = PAGE_SIZE << st_bp->map_data.page_order;
for (i = 0, offset = st_bp->buffer_bytes;
- i < st_bp->frp_segs && offset >= st_bp->frp[i].length; i++)
- offset -= st_bp->frp[i].length;
+ i < st_bp->frp_segs && offset >= length; i++)
+ offset -= length;
if (i == st_bp->frp_segs) { /* Should never happen */
printk(KERN_WARNING "st: append_to_buffer offset overflow.\n");
return (-EIO);
}
for (; i < st_bp->frp_segs && do_count > 0; i++) {
- cnt = st_bp->frp[i].length - offset < do_count ?
- st_bp->frp[i].length - offset : do_count;
- res = copy_from_user(page_address(st_bp->frp[i].page) + offset, ubp, cnt);
+ struct page *page = st_bp->reserved_pages[i];
+ cnt = length - offset < do_count ? length - offset : do_count;
+ res = copy_from_user(page_address(page) + offset, ubp, cnt);
if (res)
return (-EFAULT);
do_count -= cnt;
@@ -3842,18 +3825,19 @@
static int from_buffer(struct st_buffer * st_bp, char __user *ubp, int do_count)
{
int i, cnt, res, offset;
+ int length = PAGE_SIZE << st_bp->map_data.page_order;
for (i = 0, offset = st_bp->read_pointer;
- i < st_bp->frp_segs && offset >= st_bp->frp[i].length; i++)
- offset -= st_bp->frp[i].length;
+ i < st_bp->frp_segs && offset >= length; i++)
+ offset -= length;
if (i == st_bp->frp_segs) { /* Should never happen */
printk(KERN_WARNING "st: from_buffer offset overflow.\n");
return (-EIO);
}
for (; i < st_bp->frp_segs && do_count > 0; i++) {
- cnt = st_bp->frp[i].length - offset < do_count ?
- st_bp->frp[i].length - offset : do_count;
- res = copy_to_user(ubp, page_address(st_bp->frp[i].page) + offset, cnt);
+ struct page *page = st_bp->reserved_pages[i];
+ cnt = length - offset < do_count ? length - offset : do_count;
+ res = copy_to_user(ubp, page_address(page) + offset, cnt);
if (res)
return (-EFAULT);
do_count -= cnt;
@@ -3874,6 +3858,7 @@
{
int src_seg, dst_seg, src_offset = 0, dst_offset;
int count, total;
+ int length = PAGE_SIZE << st_bp->map_data.page_order;
if (offset == 0)
return;
@@ -3881,24 +3866,26 @@
total=st_bp->buffer_bytes - offset;
for (src_seg=0; src_seg < st_bp->frp_segs; src_seg++) {
src_offset = offset;
- if (src_offset < st_bp->frp[src_seg].length)
+ if (src_offset < length)
break;
- offset -= st_bp->frp[src_seg].length;
+ offset -= length;
}
st_bp->buffer_bytes = st_bp->read_pointer = total;
for (dst_seg=dst_offset=0; total > 0; ) {
- count = min(st_bp->frp[dst_seg].length - dst_offset,
- st_bp->frp[src_seg].length - src_offset);
- memmove(page_address(st_bp->frp[dst_seg].page) + dst_offset,
- page_address(st_bp->frp[src_seg].page) + src_offset, count);
+ struct page *dpage = st_bp->reserved_pages[dst_seg];
+ struct page *spage = st_bp->reserved_pages[src_seg];
+
+ count = min(length - dst_offset, length - src_offset);
+ memmove(page_address(dpage) + dst_offset,
+ page_address(spage) + src_offset, count);
src_offset += count;
- if (src_offset >= st_bp->frp[src_seg].length) {
+ if (src_offset >= length) {
src_seg++;
src_offset = 0;
}
dst_offset += count;
- if (dst_offset >= st_bp->frp[dst_seg].length) {
+ if (dst_offset >= length) {
dst_seg++;
dst_offset = 0;
}
@@ -3906,32 +3893,6 @@
}
}
-
-/* Fill the s/g list up to the length required for this transfer */
-static void buf_to_sg(struct st_buffer *STbp, unsigned int length)
-{
- int i;
- unsigned int count;
- struct scatterlist *sg;
- struct st_buf_fragment *frp;
-
- if (length == STbp->frp_sg_current)
- return; /* work already done */
-
- sg = &(STbp->sg[0]);
- frp = STbp->frp;
- for (i=count=0; count < length; i++) {
- if (length - count > frp[i].length)
- sg_set_page(&sg[i], frp[i].page, frp[i].length, 0);
- else
- sg_set_page(&sg[i], frp[i].page, length - count, 0);
- count += sg[i].length;
- }
- STbp->sg_segs = i;
- STbp->frp_sg_current = length;
-}
-
-
/* Validate the options from command line or module parameters */
static void validate_options(void)
{
@@ -4026,7 +3987,7 @@
SDp->request_queue->max_phys_segments);
if (st_max_sg_segs < i)
i = st_max_sg_segs;
- buffer = new_tape_buffer(1, (SDp->host)->unchecked_isa_dma, i);
+ buffer = new_tape_buffer((SDp->host)->unchecked_isa_dma, i);
if (buffer == NULL) {
printk(KERN_ERR
"st: Can't allocate new tape buffer. Device not attached.\n");
@@ -4280,8 +4241,8 @@
tpnt->device = NULL;
if (tpnt->buffer) {
- tpnt->buffer->orig_frp_segs = 0;
normalize_buffer(tpnt->buffer);
+ kfree(tpnt->buffer->reserved_pages);
kfree(tpnt->buffer);
}
@@ -4567,14 +4528,16 @@
}
/* The following functions may be useful for a larger audience. */
-static int sgl_map_user_pages(struct scatterlist *sgl, const unsigned int max_pages,
- unsigned long uaddr, size_t count, int rw)
+static int sgl_map_user_pages(struct st_buffer *STbp,
+ const unsigned int max_pages, unsigned long uaddr,
+ size_t count, int rw)
{
unsigned long end = (uaddr + count + PAGE_SIZE - 1) >> PAGE_SHIFT;
unsigned long start = uaddr >> PAGE_SHIFT;
const int nr_pages = end - start;
int res, i, j;
struct page **pages;
+ struct rq_map_data *mdata = &STbp->map_data;
/* User attempted Overflow! */
if ((uaddr + count) < uaddr)
@@ -4616,24 +4579,11 @@
flush_dcache_page(pages[i]);
}
- /* Populate the scatter/gather list */
- sg_set_page(&sgl[0], pages[0], 0, uaddr & ~PAGE_MASK);
- if (nr_pages > 1) {
- sgl[0].length = PAGE_SIZE - sgl[0].offset;
- count -= sgl[0].length;
- for (i=1; i < nr_pages ; i++) {
- sg_set_page(&sgl[i], pages[i],
- count < PAGE_SIZE ? count : PAGE_SIZE, 0);;
- count -= PAGE_SIZE;
- }
- }
- else {
- sgl[0].length = count;
- }
+ mdata->offset = uaddr & ~PAGE_MASK;
+ mdata->page_order = 0;
+ STbp->mapped_pages = pages;
- kfree(pages);
return nr_pages;
-
out_unmap:
if (res > 0) {
for (j=0; j < res; j++)
@@ -4646,13 +4596,13 @@
/* And unmap them... */
-static int sgl_unmap_user_pages(struct scatterlist *sgl, const unsigned int nr_pages,
- int dirtied)
+static int sgl_unmap_user_pages(struct st_buffer *STbp,
+ const unsigned int nr_pages, int dirtied)
{
int i;
for (i=0; i < nr_pages; i++) {
- struct page *page = sg_page(&sgl[i]);
+ struct page *page = STbp->mapped_pages[i];
if (dirtied)
SetPageDirty(page);
@@ -4661,6 +4611,8 @@
*/
page_cache_release(page);
}
+ kfree(STbp->mapped_pages);
+ STbp->mapped_pages = NULL;
return 0;
}
diff --git a/drivers/scsi/st.h b/drivers/scsi/st.h
index b92712f..544dc6b 100644
--- a/drivers/scsi/st.h
+++ b/drivers/scsi/st.h
@@ -29,6 +29,7 @@
int result;
struct scsi_tape *stp;
struct completion *waiting;
+ struct bio *bio;
};
/* The tape buffer descriptor. */
@@ -44,20 +45,13 @@
int syscall_result;
struct st_request *last_SRpnt;
struct st_cmdstatus cmdstat;
+ struct page **reserved_pages;
+ struct page **mapped_pages;
+ struct rq_map_data map_data;
unsigned char *b_data;
unsigned short use_sg; /* zero or max number of s/g segments for this adapter */
unsigned short sg_segs; /* number of segments in s/g list */
- unsigned short orig_frp_segs; /* number of segments allocated at first try */
unsigned short frp_segs; /* number of buffer segments */
- unsigned int frp_sg_current; /* driver buffer length currently in s/g list */
- struct st_buf_fragment *frp; /* the allocated buffer fragment list */
- struct scatterlist sg[1]; /* MUST BE last item */
-};
-
-/* The tape buffer fragment descriptor */
-struct st_buf_fragment {
- struct page *page;
- unsigned int length;
};
/* The tape mode definition */
diff --git a/drivers/scsi/zalon.c b/drivers/scsi/zalon.c
index 3c4a300..a8d61a6 100644
--- a/drivers/scsi/zalon.c
+++ b/drivers/scsi/zalon.c
@@ -137,8 +137,8 @@
goto fail;
if (request_irq(dev->irq, ncr53c8xx_intr, IRQF_SHARED, "zalon", host)) {
- printk(KERN_ERR "%s: irq problem with %d, detaching\n ",
- dev->dev.bus_id, dev->irq);
+ dev_printk(KERN_ERR, dev, "irq problem with %d, detaching\n ",
+ dev->irq);
goto fail;
}
diff --git a/drivers/serial/Kconfig b/drivers/serial/Kconfig
index b695ab3..3e525e3 100644
--- a/drivers/serial/Kconfig
+++ b/drivers/serial/Kconfig
@@ -457,7 +457,7 @@
config SERIAL_SAMSUNG_UARTS
int
- depends on SERIAL_SAMSUNG
+ depends on ARM && PLAT_S3C
default 2 if ARCH_S3C2400
default 4 if ARCH_S3C64XX || CPU_S3C2443
default 3
@@ -1320,13 +1320,30 @@
config SERIAL_OF_PLATFORM
tristate "Serial port on Open Firmware platform bus"
depends on PPC_OF
- depends on SERIAL_8250
+ depends on SERIAL_8250 || SERIAL_OF_PLATFORM_NWPSERIAL
help
If you have a PowerPC based system that has serial ports
on a platform specific bus, you should enable this option.
Currently, only 8250 compatible ports are supported, but
others can easily be added.
+config SERIAL_OF_PLATFORM_NWPSERIAL
+ tristate "NWP serial port driver"
+ depends on PPC_OF && PPC_DCR
+ select SERIAL_OF_PLATFORM
+ select SERIAL_CORE_CONSOLE
+ select SERIAL_CORE
+ help
+ This driver supports the cell network processor nwp serial
+ device.
+
+config SERIAL_OF_PLATFORM_NWPSERIAL_CONSOLE
+ bool "Console on NWP serial port"
+ depends on SERIAL_OF_PLATFORM_NWPSERIAL=y
+ select SERIAL_CORE_CONSOLE
+ help
+ Support for Console on the NWP serial ports.
+
config SERIAL_QE
tristate "Freescale QUICC Engine serial port support"
depends on QUICC_ENGINE
diff --git a/drivers/serial/Makefile b/drivers/serial/Makefile
index dfe775a..8844c0a 100644
--- a/drivers/serial/Makefile
+++ b/drivers/serial/Makefile
@@ -72,6 +72,7 @@
obj-$(CONFIG_SERIAL_UARTLITE) += uartlite.o
obj-$(CONFIG_SERIAL_NETX) += netx-serial.o
obj-$(CONFIG_SERIAL_OF_PLATFORM) += of_serial.o
+obj-$(CONFIG_SERIAL_OF_PLATFORM_NWPSERIAL) += nwpserial.o
obj-$(CONFIG_SERIAL_KS8695) += serial_ks8695.o
obj-$(CONFIG_KGDB_SERIAL_CONSOLE) += kgdboc.o
obj-$(CONFIG_SERIAL_QE) += ucc_uart.o
diff --git a/drivers/serial/nwpserial.c b/drivers/serial/nwpserial.c
new file mode 100644
index 0000000..32f3eaf
--- /dev/null
+++ b/drivers/serial/nwpserial.c
@@ -0,0 +1,475 @@
+/*
+ * Serial Port driver for a NWP uart device
+ *
+ * Copyright (C) 2008 IBM Corp., Benjamin Krill <ben@codiert.org>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ *
+ */
+#include <linux/init.h>
+#include <linux/console.h>
+#include <linux/serial.h>
+#include <linux/serial_reg.h>
+#include <linux/serial_core.h>
+#include <linux/tty.h>
+#include <linux/irqreturn.h>
+#include <linux/mutex.h>
+#include <linux/of_platform.h>
+#include <linux/of_device.h>
+#include <linux/nwpserial.h>
+#include <asm/prom.h>
+#include <asm/dcr.h>
+
+#define NWPSERIAL_NR 2
+
+#define NWPSERIAL_STATUS_RXVALID 0x1
+#define NWPSERIAL_STATUS_TXFULL 0x2
+
+struct nwpserial_port {
+ struct uart_port port;
+ dcr_host_t dcr_host;
+ unsigned int ier;
+ unsigned int mcr;
+};
+
+static DEFINE_MUTEX(nwpserial_mutex);
+static struct nwpserial_port nwpserial_ports[NWPSERIAL_NR];
+
+static void wait_for_bits(struct nwpserial_port *up, int bits)
+{
+ unsigned int status, tmout = 10000;
+
+ /* Wait up to 10ms for the character(s) to be sent. */
+ do {
+ status = dcr_read(up->dcr_host, UART_LSR);
+
+ if (--tmout == 0)
+ break;
+ udelay(1);
+ } while ((status & bits) != bits);
+}
+
+#ifdef CONFIG_SERIAL_OF_PLATFORM_NWPSERIAL_CONSOLE
+static void nwpserial_console_putchar(struct uart_port *port, int c)
+{
+ struct nwpserial_port *up;
+ up = container_of(port, struct nwpserial_port, port);
+ /* check if tx buffer is full */
+ wait_for_bits(up, UART_LSR_THRE);
+ dcr_write(up->dcr_host, UART_TX, c);
+ up->port.icount.tx++;
+}
+
+static void
+nwpserial_console_write(struct console *co, const char *s, unsigned int count)
+{
+ struct nwpserial_port *up = &nwpserial_ports[co->index];
+ unsigned long flags;
+ int locked = 1;
+
+ if (oops_in_progress)
+ locked = spin_trylock_irqsave(&up->port.lock, flags);
+ else
+ spin_lock_irqsave(&up->port.lock, flags);
+
+ /* save and disable interrupt */
+ up->ier = dcr_read(up->dcr_host, UART_IER);
+ dcr_write(up->dcr_host, UART_IER, up->ier & ~UART_IER_RDI);
+
+ uart_console_write(&up->port, s, count, nwpserial_console_putchar);
+
+ /* wait for transmitter to become emtpy */
+ while ((dcr_read(up->dcr_host, UART_LSR) & UART_LSR_THRE) == 0)
+ cpu_relax();
+
+ /* restore interrupt state */
+ dcr_write(up->dcr_host, UART_IER, up->ier);
+
+ if (locked)
+ spin_unlock_irqrestore(&up->port.lock, flags);
+}
+
+static struct uart_driver nwpserial_reg;
+static struct console nwpserial_console = {
+ .name = "ttySQ",
+ .write = nwpserial_console_write,
+ .device = uart_console_device,
+ .flags = CON_PRINTBUFFER,
+ .index = -1,
+ .data = &nwpserial_reg,
+};
+#define NWPSERIAL_CONSOLE (&nwpserial_console)
+#else
+#define NWPSERIAL_CONSOLE NULL
+#endif /* CONFIG_SERIAL_OF_PLATFORM_NWPSERIAL_CONSOLE */
+
+/**************************************************************************/
+
+static int nwpserial_request_port(struct uart_port *port)
+{
+ return 0;
+}
+
+static void nwpserial_release_port(struct uart_port *port)
+{
+ /* N/A */
+}
+
+static void nwpserial_config_port(struct uart_port *port, int flags)
+{
+ port->type = PORT_NWPSERIAL;
+}
+
+static irqreturn_t nwpserial_interrupt(int irq, void *dev_id)
+{
+ struct nwpserial_port *up = dev_id;
+ struct tty_struct *tty = up->port.info->port.tty;
+ irqreturn_t ret;
+ unsigned int iir;
+ unsigned char ch;
+
+ spin_lock(&up->port.lock);
+
+ /* check if the uart was the interrupt source. */
+ iir = dcr_read(up->dcr_host, UART_IIR);
+ if (!iir) {
+ ret = IRQ_NONE;
+ goto out;
+ }
+
+ do {
+ up->port.icount.rx++;
+ ch = dcr_read(up->dcr_host, UART_RX);
+ if (up->port.ignore_status_mask != NWPSERIAL_STATUS_RXVALID)
+ tty_insert_flip_char(tty, ch, TTY_NORMAL);
+ } while (dcr_read(up->dcr_host, UART_RX) & UART_LSR_DR);
+
+ tty_flip_buffer_push(tty);
+ ret = IRQ_HANDLED;
+
+out:
+ spin_unlock(&up->port.lock);
+ return ret;
+}
+
+static int nwpserial_startup(struct uart_port *port)
+{
+ struct nwpserial_port *up;
+ int err;
+
+ up = container_of(port, struct nwpserial_port, port);
+
+ /* disable flow control by default */
+ up->mcr = dcr_read(up->dcr_host, UART_MCR) & ~UART_MCR_AFE;
+ dcr_write(up->dcr_host, UART_MCR, up->mcr);
+
+ /* register interrupt handler */
+ err = request_irq(up->port.irq, nwpserial_interrupt,
+ IRQF_SHARED, "nwpserial", up);
+ if (err)
+ return err;
+
+ /* enable interrupts */
+ up->ier = UART_IER_RDI;
+ dcr_write(up->dcr_host, UART_IER, up->ier);
+
+ /* enable receiving */
+ up->port.ignore_status_mask &= ~NWPSERIAL_STATUS_RXVALID;
+
+ return 0;
+}
+
+static void nwpserial_shutdown(struct uart_port *port)
+{
+ struct nwpserial_port *up;
+ up = container_of(port, struct nwpserial_port, port);
+
+ /* disable receiving */
+ up->port.ignore_status_mask |= NWPSERIAL_STATUS_RXVALID;
+
+ /* disable interrupts from this port */
+ up->ier = 0;
+ dcr_write(up->dcr_host, UART_IER, up->ier);
+
+ /* free irq */
+ free_irq(up->port.irq, port);
+}
+
+static int nwpserial_verify_port(struct uart_port *port,
+ struct serial_struct *ser)
+{
+ return -EINVAL;
+}
+
+static const char *nwpserial_type(struct uart_port *port)
+{
+ return port->type == PORT_NWPSERIAL ? "nwpserial" : NULL;
+}
+
+static void nwpserial_set_termios(struct uart_port *port,
+ struct ktermios *termios, struct ktermios *old)
+{
+ struct nwpserial_port *up;
+ up = container_of(port, struct nwpserial_port, port);
+
+ up->port.read_status_mask = NWPSERIAL_STATUS_RXVALID
+ | NWPSERIAL_STATUS_TXFULL;
+
+ up->port.ignore_status_mask = 0;
+ /* ignore all characters if CREAD is not set */
+ if ((termios->c_cflag & CREAD) == 0)
+ up->port.ignore_status_mask |= NWPSERIAL_STATUS_RXVALID;
+
+ /* Copy back the old hardware settings */
+ if (old)
+ tty_termios_copy_hw(termios, old);
+}
+
+static void nwpserial_break_ctl(struct uart_port *port, int ctl)
+{
+ /* N/A */
+}
+
+static void nwpserial_enable_ms(struct uart_port *port)
+{
+ /* N/A */
+}
+
+static void nwpserial_stop_rx(struct uart_port *port)
+{
+ struct nwpserial_port *up;
+ up = container_of(port, struct nwpserial_port, port);
+ /* don't forward any more data (like !CREAD) */
+ up->port.ignore_status_mask = NWPSERIAL_STATUS_RXVALID;
+}
+
+static void nwpserial_putchar(struct nwpserial_port *up, unsigned char c)
+{
+ /* check if tx buffer is full */
+ wait_for_bits(up, UART_LSR_THRE);
+ dcr_write(up->dcr_host, UART_TX, c);
+ up->port.icount.tx++;
+}
+
+static void nwpserial_start_tx(struct uart_port *port)
+{
+ struct nwpserial_port *up;
+ struct circ_buf *xmit;
+ up = container_of(port, struct nwpserial_port, port);
+ xmit = &up->port.info->xmit;
+
+ if (port->x_char) {
+ nwpserial_putchar(up, up->port.x_char);
+ port->x_char = 0;
+ }
+
+ while (!(uart_circ_empty(xmit) || uart_tx_stopped(&up->port))) {
+ nwpserial_putchar(up, xmit->buf[xmit->tail]);
+ xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE-1);
+ }
+}
+
+static unsigned int nwpserial_get_mctrl(struct uart_port *port)
+{
+ return 0;
+}
+
+static void nwpserial_set_mctrl(struct uart_port *port, unsigned int mctrl)
+{
+ /* N/A */
+}
+
+static void nwpserial_stop_tx(struct uart_port *port)
+{
+ /* N/A */
+}
+
+static unsigned int nwpserial_tx_empty(struct uart_port *port)
+{
+ struct nwpserial_port *up;
+ unsigned long flags;
+ int ret;
+ up = container_of(port, struct nwpserial_port, port);
+
+ spin_lock_irqsave(&up->port.lock, flags);
+ ret = dcr_read(up->dcr_host, UART_LSR);
+ spin_unlock_irqrestore(&up->port.lock, flags);
+
+ return ret & UART_LSR_TEMT ? TIOCSER_TEMT : 0;
+}
+
+static struct uart_ops nwpserial_pops = {
+ .tx_empty = nwpserial_tx_empty,
+ .set_mctrl = nwpserial_set_mctrl,
+ .get_mctrl = nwpserial_get_mctrl,
+ .stop_tx = nwpserial_stop_tx,
+ .start_tx = nwpserial_start_tx,
+ .stop_rx = nwpserial_stop_rx,
+ .enable_ms = nwpserial_enable_ms,
+ .break_ctl = nwpserial_break_ctl,
+ .startup = nwpserial_startup,
+ .shutdown = nwpserial_shutdown,
+ .set_termios = nwpserial_set_termios,
+ .type = nwpserial_type,
+ .release_port = nwpserial_release_port,
+ .request_port = nwpserial_request_port,
+ .config_port = nwpserial_config_port,
+ .verify_port = nwpserial_verify_port,
+};
+
+static struct uart_driver nwpserial_reg = {
+ .owner = THIS_MODULE,
+ .driver_name = "nwpserial",
+ .dev_name = "ttySQ",
+ .major = TTY_MAJOR,
+ .minor = 68,
+ .nr = NWPSERIAL_NR,
+ .cons = NWPSERIAL_CONSOLE,
+};
+
+int nwpserial_register_port(struct uart_port *port)
+{
+ struct nwpserial_port *up = NULL;
+ int ret = -1;
+ int i;
+ static int first = 1;
+ int dcr_len;
+ int dcr_base;
+ struct device_node *dn;
+
+ mutex_lock(&nwpserial_mutex);
+
+ dn = to_of_device(port->dev)->node;
+ if (dn == NULL)
+ goto out;
+
+ /* get dcr base. */
+ dcr_base = dcr_resource_start(dn, 0);
+
+ /* find matching entry */
+ for (i = 0; i < NWPSERIAL_NR; i++)
+ if (nwpserial_ports[i].port.iobase == dcr_base) {
+ up = &nwpserial_ports[i];
+ break;
+ }
+
+ /* we didn't find a mtching entry, search for a free port */
+ if (up == NULL)
+ for (i = 0; i < NWPSERIAL_NR; i++)
+ if (nwpserial_ports[i].port.type == PORT_UNKNOWN &&
+ nwpserial_ports[i].port.iobase == 0) {
+ up = &nwpserial_ports[i];
+ break;
+ }
+
+ if (up == NULL) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ if (first)
+ uart_register_driver(&nwpserial_reg);
+ first = 0;
+
+ up->port.membase = port->membase;
+ up->port.irq = port->irq;
+ up->port.uartclk = port->uartclk;
+ up->port.fifosize = port->fifosize;
+ up->port.regshift = port->regshift;
+ up->port.iotype = port->iotype;
+ up->port.flags = port->flags;
+ up->port.mapbase = port->mapbase;
+ up->port.private_data = port->private_data;
+
+ if (port->dev)
+ up->port.dev = port->dev;
+
+ if (up->port.iobase != dcr_base) {
+ up->port.ops = &nwpserial_pops;
+ up->port.fifosize = 16;
+
+ spin_lock_init(&up->port.lock);
+
+ up->port.iobase = dcr_base;
+ dcr_len = dcr_resource_len(dn, 0);
+
+ up->dcr_host = dcr_map(dn, dcr_base, dcr_len);
+ if (!DCR_MAP_OK(up->dcr_host)) {
+ printk(KERN_ERR "Cannot map DCR resources for NWPSERIAL");
+ goto out;
+ }
+ }
+
+ ret = uart_add_one_port(&nwpserial_reg, &up->port);
+ if (ret == 0)
+ ret = up->port.line;
+
+out:
+ mutex_unlock(&nwpserial_mutex);
+
+ return ret;
+}
+EXPORT_SYMBOL(nwpserial_register_port);
+
+void nwpserial_unregister_port(int line)
+{
+ struct nwpserial_port *up = &nwpserial_ports[line];
+ mutex_lock(&nwpserial_mutex);
+ uart_remove_one_port(&nwpserial_reg, &up->port);
+
+ up->port.type = PORT_UNKNOWN;
+
+ mutex_unlock(&nwpserial_mutex);
+}
+EXPORT_SYMBOL(nwpserial_unregister_port);
+
+#ifdef CONFIG_SERIAL_OF_PLATFORM_NWPSERIAL_CONSOLE
+static int __init nwpserial_console_init(void)
+{
+ struct nwpserial_port *up = NULL;
+ struct device_node *dn;
+ const char *name;
+ int dcr_base;
+ int dcr_len;
+ int i;
+
+ /* search for a free port */
+ for (i = 0; i < NWPSERIAL_NR; i++)
+ if (nwpserial_ports[i].port.type == PORT_UNKNOWN) {
+ up = &nwpserial_ports[i];
+ break;
+ }
+
+ if (up == NULL)
+ return -1;
+
+ name = of_get_property(of_chosen, "linux,stdout-path", NULL);
+ if (name == NULL)
+ return -1;
+
+ dn = of_find_node_by_path(name);
+ if (!dn)
+ return -1;
+
+ spin_lock_init(&up->port.lock);
+ up->port.ops = &nwpserial_pops;
+ up->port.type = PORT_NWPSERIAL;
+ up->port.fifosize = 16;
+
+ dcr_base = dcr_resource_start(dn, 0);
+ dcr_len = dcr_resource_len(dn, 0);
+ up->port.iobase = dcr_base;
+
+ up->dcr_host = dcr_map(dn, dcr_base, dcr_len);
+ if (!DCR_MAP_OK(up->dcr_host)) {
+ printk("Cannot map DCR resources for SERIAL");
+ return -1;
+ }
+ register_console(&nwpserial_console);
+ return 0;
+}
+console_initcall(nwpserial_console_init);
+#endif /* CONFIG_SERIAL_OF_PLATFORM_NWPSERIAL_CONSOLE */
diff --git a/drivers/serial/of_serial.c b/drivers/serial/of_serial.c
index 8fa0ff5..a821e3a 100644
--- a/drivers/serial/of_serial.c
+++ b/drivers/serial/of_serial.c
@@ -14,6 +14,7 @@
#include <linux/serial_core.h>
#include <linux/serial_8250.h>
#include <linux/of_platform.h>
+#include <linux/nwpserial.h>
#include <asm/prom.h>
@@ -99,9 +100,16 @@
goto out;
switch (port_type) {
+#ifdef CONFIG_SERIAL_8250
case PORT_8250 ... PORT_MAX_8250:
ret = serial8250_register_port(&port);
break;
+#endif
+#ifdef CONFIG_SERIAL_OF_PLATFORM_NWPSERIAL
+ case PORT_NWPSERIAL:
+ ret = nwpserial_register_port(&port);
+ break;
+#endif
default:
/* need to add code for these */
case PORT_UNKNOWN:
@@ -129,9 +137,16 @@
{
struct of_serial_info *info = ofdev->dev.driver_data;
switch (info->type) {
+#ifdef CONFIG_SERIAL_8250
case PORT_8250 ... PORT_MAX_8250:
serial8250_unregister_port(info->line);
break;
+#endif
+#ifdef CONFIG_SERIAL_OF_PLATFORM_NWPSERIAL
+ case PORT_NWPSERIAL:
+ nwpserial_unregister_port(info->line);
+ break;
+#endif
default:
/* need to add code for these */
break;
@@ -148,6 +163,10 @@
{ .type = "serial", .compatible = "ns16450", .data = (void *)PORT_16450, },
{ .type = "serial", .compatible = "ns16550", .data = (void *)PORT_16550, },
{ .type = "serial", .compatible = "ns16750", .data = (void *)PORT_16750, },
+#ifdef CONFIG_SERIAL_OF_PLATFORM_NWPSERIAL
+ { .type = "serial", .compatible = "ibm,qpace-nwp-serial",
+ .data = (void *)PORT_NWPSERIAL, },
+#endif
{ .type = "serial", .data = (void *)PORT_UNKNOWN, },
{ /* end of list */ },
};
diff --git a/drivers/usb/gadget/f_phonet.c b/drivers/usb/gadget/f_phonet.c
index d8fc9b3..c0916c7 100644
--- a/drivers/usb/gadget/f_phonet.c
+++ b/drivers/usb/gadget/f_phonet.c
@@ -279,6 +279,13 @@
return err;
}
+static const struct net_device_ops pn_netdev_ops = {
+ .ndo_open = pn_net_open,
+ .ndo_stop = pn_net_close,
+ .ndo_start_xmit = pn_net_xmit,
+ .ndo_change_mtu = pn_net_mtu,
+};
+
static void pn_net_setup(struct net_device *dev)
{
dev->features = 0;
@@ -290,12 +297,9 @@
dev->addr_len = 1;
dev->tx_queue_len = 1;
+ dev->netdev_ops = &pn_netdev_ops;
dev->destructor = free_netdev;
dev->header_ops = &phonet_header_ops;
- dev->open = pn_net_open;
- dev->stop = pn_net_close;
- dev->hard_start_xmit = pn_net_xmit; /* mandatory */
- dev->change_mtu = pn_net_mtu;
}
/*-------------------------------------------------------------------------*/
diff --git a/drivers/usb/gadget/u_ether.c b/drivers/usb/gadget/u_ether.c
index d9739d5..96d65ca 100644
--- a/drivers/usb/gadget/u_ether.c
+++ b/drivers/usb/gadget/u_ether.c
@@ -716,6 +716,14 @@
static struct eth_dev *the_dev;
+static const struct net_device_ops eth_netdev_ops = {
+ .ndo_open = eth_open,
+ .ndo_stop = eth_stop,
+ .ndo_start_xmit = eth_start_xmit,
+ .ndo_change_mtu = ueth_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+};
/**
* gether_setup - initialize one ethernet-over-usb link
@@ -764,12 +772,8 @@
if (ethaddr)
memcpy(ethaddr, dev->host_mac, ETH_ALEN);
- net->change_mtu = ueth_change_mtu;
- net->hard_start_xmit = eth_start_xmit;
- net->open = eth_open;
- net->stop = eth_stop;
- /* watchdog_timeo, tx_timeout ... */
- /* set_multicast_list */
+ net->netdev_ops = ð_netdev_ops;
+
SET_ETHTOOL_OPS(net, &ops);
/* two kinds of host-initiated state changes:
diff --git a/drivers/uwb/i1480/i1480u-wlp/i1480u-wlp.h b/drivers/uwb/i1480/i1480u-wlp/i1480u-wlp.h
index 5f1b295..3421d33 100644
--- a/drivers/uwb/i1480/i1480u-wlp/i1480u-wlp.h
+++ b/drivers/uwb/i1480/i1480u-wlp/i1480u-wlp.h
@@ -221,7 +221,6 @@
struct net_device *net_dev;
spinlock_t lock;
- struct net_device_stats stats;
/* RX context handling */
struct sk_buff *rx_skb;
@@ -271,7 +270,6 @@
extern int i1480u_hard_start_xmit(struct sk_buff *, struct net_device *);
extern void i1480u_tx_timeout(struct net_device *);
extern int i1480u_set_config(struct net_device *, struct ifmap *);
-extern struct net_device_stats *i1480u_get_stats(struct net_device *);
extern int i1480u_change_mtu(struct net_device *, int);
extern void i1480u_uwb_notifs_cb(void *, struct uwb_dev *, enum uwb_notifs);
diff --git a/drivers/uwb/i1480/i1480u-wlp/lc.c b/drivers/uwb/i1480/i1480u-wlp/lc.c
index 049c05d..f272dfe 100644
--- a/drivers/uwb/i1480/i1480u-wlp/lc.c
+++ b/drivers/uwb/i1480/i1480u-wlp/lc.c
@@ -181,6 +181,15 @@
}
#endif
+static const struct net_device_ops i1480u_netdev_ops = {
+ .ndo_open = i1480u_open,
+ .ndo_stop = i1480u_stop,
+ .ndo_start_xmit = i1480u_hard_start_xmit,
+ .ndo_tx_timeout = i1480u_tx_timeout,
+ .ndo_set_config = i1480u_set_config,
+ .ndo_change_mtu = i1480u_change_mtu,
+};
+
static
int i1480u_add(struct i1480u *i1480u, struct usb_interface *iface)
{
@@ -235,13 +244,7 @@
net_dev->features |= NETIF_F_HIGHDMA;
net_dev->watchdog_timeo = 5*HZ; /* FIXME: a better default? */
- net_dev->open = i1480u_open;
- net_dev->stop = i1480u_stop;
- net_dev->hard_start_xmit = i1480u_hard_start_xmit;
- net_dev->tx_timeout = i1480u_tx_timeout;
- net_dev->get_stats = i1480u_get_stats;
- net_dev->set_config = i1480u_set_config;
- net_dev->change_mtu = i1480u_change_mtu;
+ net_dev->netdev_ops = &i1480u_netdev_ops;
#ifdef i1480u_FLOW_CONTROL
/* Notification endpoint setup (submitted when we open the device) */
diff --git a/drivers/uwb/i1480/i1480u-wlp/netdev.c b/drivers/uwb/i1480/i1480u-wlp/netdev.c
index e3873ff..7305553 100644
--- a/drivers/uwb/i1480/i1480u-wlp/netdev.c
+++ b/drivers/uwb/i1480/i1480u-wlp/netdev.c
@@ -262,15 +262,6 @@
return 0;
}
-
-/** Report statistics */
-struct net_device_stats *i1480u_get_stats(struct net_device *net_dev)
-{
- struct i1480u *i1480u = netdev_priv(net_dev);
- return &i1480u->stats;
-}
-
-
/**
*
* Change the interface config--we probably don't have to do anything.
diff --git a/drivers/uwb/i1480/i1480u-wlp/rx.c b/drivers/uwb/i1480/i1480u-wlp/rx.c
index 34f4cf9..25a2758 100644
--- a/drivers/uwb/i1480/i1480u-wlp/rx.c
+++ b/drivers/uwb/i1480/i1480u-wlp/rx.c
@@ -167,7 +167,7 @@
do { \
if (printk_ratelimit()) \
dev_err(&i1480u->usb_iface->dev, msg); \
- i1480u->stats.rx_dropped++; \
+ i1480u->net_dev->stats.rx_dropped++; \
} while (0)
@@ -193,10 +193,8 @@
if (!should_parse)
goto out;
i1480u->rx_skb->protocol = eth_type_trans(i1480u->rx_skb, net_dev);
- i1480u->stats.rx_packets++;
- i1480u->stats.rx_bytes += i1480u->rx_untd_pkt_size;
- net_dev->last_rx = jiffies;
- /* FIXME: flow control: check netif_rx() retval */
+ net_dev->stats.rx_packets++;
+ net_dev->stats.rx_bytes += i1480u->rx_untd_pkt_size;
netif_rx(i1480u->rx_skb); /* deliver */
out:
diff --git a/drivers/uwb/i1480/i1480u-wlp/tx.c b/drivers/uwb/i1480/i1480u-wlp/tx.c
index 39032cc..26bacc0 100644
--- a/drivers/uwb/i1480/i1480u-wlp/tx.c
+++ b/drivers/uwb/i1480/i1480u-wlp/tx.c
@@ -117,8 +117,8 @@
switch (urb->status) {
case 0:
spin_lock_irqsave(&i1480u->lock, flags);
- i1480u->stats.tx_packets++;
- i1480u->stats.tx_bytes += urb->actual_length;
+ net_dev->stats.tx_packets++;
+ net_dev->stats.tx_bytes += urb->actual_length;
spin_unlock_irqrestore(&i1480u->lock, flags);
break;
case -ECONNRESET: /* Not an error, but a controlled situation; */
@@ -530,7 +530,7 @@
return NETDEV_TX_OK;
error:
dev_kfree_skb_any(skb);
- i1480u->stats.tx_dropped++;
+ net_dev->stats.tx_dropped++;
out:
return NETDEV_TX_OK;
}
diff --git a/drivers/video/amba-clcd.c b/drivers/video/amba-clcd.c
index 2ac52fd..4e046fe 100644
--- a/drivers/video/amba-clcd.c
+++ b/drivers/video/amba-clcd.c
@@ -24,6 +24,7 @@
#include <linux/amba/bus.h>
#include <linux/amba/clcd.h>
#include <linux/clk.h>
+#include <linux/hardirq.h>
#include <asm/sizes.h>
diff --git a/drivers/w1/masters/Kconfig b/drivers/w1/masters/Kconfig
index 9061682..96d2f8e 100644
--- a/drivers/w1/masters/Kconfig
+++ b/drivers/w1/masters/Kconfig
@@ -34,6 +34,12 @@
This driver can also be built as a module. If so, the module
will be called ds2482.
+config W1_MASTER_MXC
+ tristate "Freescale MXC 1-wire busmaster"
+ depends on W1 && ARCH_MXC
+ help
+ Say Y here to enable MXC 1-wire host
+
config W1_MASTER_DS1WM
tristate "Maxim DS1WM 1-wire busmaster"
depends on W1 && ARM && HAVE_CLK
diff --git a/drivers/w1/masters/Makefile b/drivers/w1/masters/Makefile
index bc4714a..c5a3e96 100644
--- a/drivers/w1/masters/Makefile
+++ b/drivers/w1/masters/Makefile
@@ -5,6 +5,8 @@
obj-$(CONFIG_W1_MASTER_MATROX) += matrox_w1.o
obj-$(CONFIG_W1_MASTER_DS2490) += ds2490.o
obj-$(CONFIG_W1_MASTER_DS2482) += ds2482.o
+obj-$(CONFIG_W1_MASTER_MXC) += mxc_w1.o
+
obj-$(CONFIG_W1_MASTER_DS1WM) += ds1wm.o
obj-$(CONFIG_W1_MASTER_GPIO) += w1-gpio.o
obj-$(CONFIG_HDQ_MASTER_OMAP) += omap_hdq.o
diff --git a/drivers/w1/masters/mxc_w1.c b/drivers/w1/masters/mxc_w1.c
new file mode 100644
index 0000000..b9d74d0
--- /dev/null
+++ b/drivers/w1/masters/mxc_w1.c
@@ -0,0 +1,211 @@
+/*
+ * Copyright 2005-2008 Freescale Semiconductor, Inc. All Rights Reserved.
+ * Copyright 2008 Luotao Fu, kernel@pengutronix.de
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/interrupt.h>
+#include <linux/platform_device.h>
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/io.h>
+
+#include "../w1.h"
+#include "../w1_int.h"
+#include "../w1_log.h"
+
+/* According to the mx27 Datasheet the reset procedure should take up to about
+ * 1350us. We set the timeout to 500*100us = 50ms for sure */
+#define MXC_W1_RESET_TIMEOUT 500
+
+/*
+ * MXC W1 Register offsets
+ */
+#define MXC_W1_CONTROL 0x00
+#define MXC_W1_TIME_DIVIDER 0x02
+#define MXC_W1_RESET 0x04
+#define MXC_W1_COMMAND 0x06
+#define MXC_W1_TXRX 0x08
+#define MXC_W1_INTERRUPT 0x0A
+#define MXC_W1_INTERRUPT_EN 0x0C
+
+struct mxc_w1_device {
+ void __iomem *regs;
+ unsigned int clkdiv;
+ struct clk *clk;
+ struct w1_bus_master bus_master;
+};
+
+/*
+ * this is the low level routine to
+ * reset the device on the One Wire interface
+ * on the hardware
+ */
+static u8 mxc_w1_ds2_reset_bus(void *data)
+{
+ u8 reg_val;
+ unsigned int timeout_cnt = 0;
+ struct mxc_w1_device *dev = data;
+
+ __raw_writeb(0x80, (dev->regs + MXC_W1_CONTROL));
+
+ while (1) {
+ reg_val = __raw_readb(dev->regs + MXC_W1_CONTROL);
+
+ if (((reg_val >> 7) & 0x1) == 0 ||
+ timeout_cnt > MXC_W1_RESET_TIMEOUT)
+ break;
+ else
+ timeout_cnt++;
+
+ udelay(100);
+ }
+ return (reg_val >> 7) & 0x1;
+}
+
+/*
+ * this is the low level routine to read/write a bit on the One Wire
+ * interface on the hardware. It does write 0 if parameter bit is set
+ * to 0, otherwise a write 1/read.
+ */
+static u8 mxc_w1_ds2_touch_bit(void *data, u8 bit)
+{
+ struct mxc_w1_device *mdev = data;
+ void __iomem *ctrl_addr = mdev->regs + MXC_W1_CONTROL;
+ unsigned int timeout_cnt = 400; /* Takes max. 120us according to
+ * datasheet.
+ */
+
+ __raw_writeb((1 << (5 - bit)), ctrl_addr);
+
+ while (timeout_cnt--) {
+ if (!((__raw_readb(ctrl_addr) >> (5 - bit)) & 0x1))
+ break;
+
+ udelay(1);
+ }
+
+ return ((__raw_readb(ctrl_addr)) >> 3) & 0x1;
+}
+
+static int __init mxc_w1_probe(struct platform_device *pdev)
+{
+ struct mxc_w1_device *mdev;
+ struct resource *res;
+ int err = 0;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!res)
+ return -ENODEV;
+
+ mdev = kzalloc(sizeof(struct mxc_w1_device), GFP_KERNEL);
+ if (!mdev)
+ return -ENOMEM;
+
+ mdev->clk = clk_get(&pdev->dev, "owire_clk");
+ if (!mdev->clk) {
+ err = -ENODEV;
+ goto failed_clk;
+ }
+
+ mdev->clkdiv = (clk_get_rate(mdev->clk) / 1000000) - 1;
+
+ res = request_mem_region(res->start, resource_size(res),
+ "mxc_w1");
+ if (!res) {
+ err = -EBUSY;
+ goto failed_req;
+ }
+
+ mdev->regs = ioremap(res->start, resource_size(res));
+ if (!mdev->regs) {
+ printk(KERN_ERR "Cannot map frame buffer registers\n");
+ goto failed_ioremap;
+ }
+
+ clk_enable(mdev->clk);
+ __raw_writeb(mdev->clkdiv, mdev->regs + MXC_W1_TIME_DIVIDER);
+
+ mdev->bus_master.data = mdev;
+ mdev->bus_master.reset_bus = mxc_w1_ds2_reset_bus;
+ mdev->bus_master.touch_bit = mxc_w1_ds2_touch_bit;
+
+ err = w1_add_master_device(&mdev->bus_master);
+
+ if (err)
+ goto failed_add;
+
+ platform_set_drvdata(pdev, mdev);
+ return 0;
+
+failed_add:
+ iounmap(mdev->regs);
+failed_ioremap:
+ release_mem_region(res->start, resource_size(res));
+failed_req:
+ clk_put(mdev->clk);
+failed_clk:
+ kfree(mdev);
+ return err;
+}
+
+/*
+ * disassociate the w1 device from the driver
+ */
+static int mxc_w1_remove(struct platform_device *pdev)
+{
+ struct mxc_w1_device *mdev = platform_get_drvdata(pdev);
+ struct resource *res;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+
+ w1_remove_master_device(&mdev->bus_master);
+
+ iounmap(mdev->regs);
+ release_mem_region(res->start, resource_size(res));
+ clk_disable(mdev->clk);
+ clk_put(mdev->clk);
+
+ platform_set_drvdata(pdev, NULL);
+
+ return 0;
+}
+
+static struct platform_driver mxc_w1_driver = {
+ .driver = {
+ .name = "mxc_w1",
+ },
+ .probe = mxc_w1_probe,
+ .remove = mxc_w1_remove,
+};
+
+static int __init mxc_w1_init(void)
+{
+ return platform_driver_register(&mxc_w1_driver);
+}
+
+static void mxc_w1_exit(void)
+{
+ platform_driver_unregister(&mxc_w1_driver);
+}
+
+module_init(mxc_w1_init);
+module_exit(mxc_w1_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Freescale Semiconductors Inc");
+MODULE_DESCRIPTION("Driver for One-Wire on MXC");
diff --git a/drivers/w1/w1.h b/drivers/w1/w1.h
index 97304bd..d8a9709 100644
--- a/drivers/w1/w1.h
+++ b/drivers/w1/w1.h
@@ -210,6 +210,7 @@
int w1_reset_bus(struct w1_master *);
u8 w1_calc_crc8(u8 *, int);
void w1_write_block(struct w1_master *, const u8 *, int);
+void w1_touch_block(struct w1_master *, u8 *, int);
u8 w1_read_block(struct w1_master *, u8 *, int);
int w1_reset_select_slave(struct w1_slave *sl);
void w1_next_pullup(struct w1_master *, int);
diff --git a/drivers/w1/w1_io.c b/drivers/w1/w1_io.c
index 5139c25..442bd8b 100644
--- a/drivers/w1/w1_io.c
+++ b/drivers/w1/w1_io.c
@@ -238,7 +238,6 @@
* @param dev the master device
* @param buf pointer to the data to write
* @param len the number of bytes to write
- * @return the byte read
*/
void w1_write_block(struct w1_master *dev, const u8 *buf, int len)
{
@@ -256,6 +255,31 @@
EXPORT_SYMBOL_GPL(w1_write_block);
/**
+ * Touches a series of bytes.
+ *
+ * @param dev the master device
+ * @param buf pointer to the data to write
+ * @param len the number of bytes to write
+ */
+void w1_touch_block(struct w1_master *dev, u8 *buf, int len)
+{
+ int i, j;
+ u8 tmp;
+
+ for (i = 0; i < len; ++i) {
+ tmp = 0;
+ for (j = 0; j < 8; ++j) {
+ if (j == 7)
+ w1_pre_write(dev);
+ tmp |= w1_touch_bit(dev, (buf[i] >> j) & 0x1) << j;
+ }
+
+ buf[i] = tmp;
+ }
+}
+EXPORT_SYMBOL_GPL(w1_touch_block);
+
+/**
* Reads a series of bytes.
*
* @param dev the master device
diff --git a/drivers/w1/w1_netlink.c b/drivers/w1/w1_netlink.c
index 65c5ebd..fdf7285 100644
--- a/drivers/w1/w1_netlink.c
+++ b/drivers/w1/w1_netlink.c
@@ -47,21 +47,56 @@
cn_netlink_send(m, 0, GFP_KERNEL);
}
-static int w1_process_command_master(struct w1_master *dev, struct cn_msg *msg,
- struct w1_netlink_msg *hdr, struct w1_netlink_cmd *cmd)
+static void w1_send_slave(struct w1_master *dev, u64 rn)
{
- dev_dbg(&dev->dev, "%s: %s: cmd=%02x, len=%u.\n",
- __func__, dev->name, cmd->cmd, cmd->len);
+ struct cn_msg *msg = dev->priv;
+ struct w1_netlink_msg *hdr = (struct w1_netlink_msg *)(msg + 1);
+ struct w1_netlink_cmd *cmd = (struct w1_netlink_cmd *)(hdr + 1);
+ int avail;
- if (cmd->cmd != W1_CMD_SEARCH && cmd->cmd != W1_CMD_ALARM_SEARCH)
- return -EINVAL;
+ avail = dev->priv_size - cmd->len;
- w1_search_process(dev, (cmd->cmd == W1_CMD_ALARM_SEARCH)?W1_ALARM_SEARCH:W1_SEARCH);
+ if (avail > 8) {
+ u64 *data = (void *)(cmd + 1) + cmd->len;
+
+ *data = rn;
+ cmd->len += 8;
+ hdr->len += 8;
+ msg->len += 8;
+ return;
+ }
+
+ msg->ack++;
+ cn_netlink_send(msg, 0, GFP_KERNEL);
+
+ msg->len = sizeof(struct w1_netlink_msg) + sizeof(struct w1_netlink_cmd);
+ hdr->len = sizeof(struct w1_netlink_cmd);
+ cmd->len = 0;
+}
+
+static int w1_process_search_command(struct w1_master *dev, struct cn_msg *msg,
+ unsigned int avail)
+{
+ struct w1_netlink_msg *hdr = (struct w1_netlink_msg *)(msg + 1);
+ struct w1_netlink_cmd *cmd = (struct w1_netlink_cmd *)(hdr + 1);
+ int search_type = (cmd->cmd == W1_CMD_ALARM_SEARCH)?W1_ALARM_SEARCH:W1_SEARCH;
+
+ dev->priv = msg;
+ dev->priv_size = avail;
+
+ w1_search_devices(dev, search_type, w1_send_slave);
+
+ msg->ack = 0;
+ cn_netlink_send(msg, 0, GFP_KERNEL);
+
+ dev->priv = NULL;
+ dev->priv_size = 0;
+
return 0;
}
-static int w1_send_read_reply(struct w1_slave *sl, struct cn_msg *msg,
- struct w1_netlink_msg *hdr, struct w1_netlink_cmd *cmd)
+static int w1_send_read_reply(struct cn_msg *msg, struct w1_netlink_msg *hdr,
+ struct w1_netlink_cmd *cmd)
{
void *data;
struct w1_netlink_msg *h;
@@ -85,7 +120,8 @@
memcpy(c, cmd, sizeof(struct w1_netlink_cmd));
cm->ack = msg->seq+1;
- cm->len = sizeof(struct w1_netlink_msg) + sizeof(struct w1_netlink_cmd) + cmd->len;
+ cm->len = sizeof(struct w1_netlink_msg) +
+ sizeof(struct w1_netlink_cmd) + cmd->len;
h->len = sizeof(struct w1_netlink_cmd) + cmd->len;
@@ -98,36 +134,178 @@
return err;
}
-static int w1_process_command_slave(struct w1_slave *sl, struct cn_msg *msg,
+static int w1_process_command_io(struct w1_master *dev, struct cn_msg *msg,
struct w1_netlink_msg *hdr, struct w1_netlink_cmd *cmd)
{
int err = 0;
- dev_dbg(&sl->master->dev, "%s: %02x.%012llx.%02x: cmd=%02x, len=%u.\n",
- __func__, sl->reg_num.family, (unsigned long long)sl->reg_num.id, sl->reg_num.crc,
- cmd->cmd, cmd->len);
-
switch (cmd->cmd) {
- case W1_CMD_READ:
- w1_read_block(sl->master, cmd->data, cmd->len);
- w1_send_read_reply(sl, msg, hdr, cmd);
- break;
- case W1_CMD_WRITE:
- w1_write_block(sl->master, cmd->data, cmd->len);
- break;
- case W1_CMD_SEARCH:
- case W1_CMD_ALARM_SEARCH:
- w1_search_process(sl->master,
- (cmd->cmd == W1_CMD_ALARM_SEARCH)?W1_ALARM_SEARCH:W1_SEARCH);
- break;
- default:
- err = -1;
- break;
+ case W1_CMD_TOUCH:
+ w1_touch_block(dev, cmd->data, cmd->len);
+ w1_send_read_reply(msg, hdr, cmd);
+ break;
+ case W1_CMD_READ:
+ w1_read_block(dev, cmd->data, cmd->len);
+ w1_send_read_reply(msg, hdr, cmd);
+ break;
+ case W1_CMD_WRITE:
+ w1_write_block(dev, cmd->data, cmd->len);
+ break;
+ default:
+ err = -EINVAL;
+ break;
}
return err;
}
+static int w1_process_command_master(struct w1_master *dev, struct cn_msg *req_msg,
+ struct w1_netlink_msg *req_hdr, struct w1_netlink_cmd *req_cmd)
+{
+ int err = -EINVAL;
+ struct cn_msg *msg;
+ struct w1_netlink_msg *hdr;
+ struct w1_netlink_cmd *cmd;
+
+ msg = kzalloc(PAGE_SIZE, GFP_KERNEL);
+ if (!msg)
+ return -ENOMEM;
+
+ msg->id = req_msg->id;
+ msg->seq = req_msg->seq;
+ msg->ack = 0;
+ msg->len = sizeof(struct w1_netlink_msg) + sizeof(struct w1_netlink_cmd);
+
+ hdr = (struct w1_netlink_msg *)(msg + 1);
+ cmd = (struct w1_netlink_cmd *)(hdr + 1);
+
+ hdr->type = W1_MASTER_CMD;
+ hdr->id = req_hdr->id;
+ hdr->len = sizeof(struct w1_netlink_cmd);
+
+ cmd->cmd = req_cmd->cmd;
+ cmd->len = 0;
+
+ switch (cmd->cmd) {
+ case W1_CMD_SEARCH:
+ case W1_CMD_ALARM_SEARCH:
+ err = w1_process_search_command(dev, msg,
+ PAGE_SIZE - msg->len - sizeof(struct cn_msg));
+ break;
+ case W1_CMD_READ:
+ case W1_CMD_WRITE:
+ case W1_CMD_TOUCH:
+ err = w1_process_command_io(dev, req_msg, req_hdr, req_cmd);
+ break;
+ case W1_CMD_RESET:
+ err = w1_reset_bus(dev);
+ break;
+ default:
+ err = -EINVAL;
+ break;
+ }
+
+ kfree(msg);
+ return err;
+}
+
+static int w1_process_command_slave(struct w1_slave *sl, struct cn_msg *msg,
+ struct w1_netlink_msg *hdr, struct w1_netlink_cmd *cmd)
+{
+ dev_dbg(&sl->master->dev, "%s: %02x.%012llx.%02x: cmd=%02x, len=%u.\n",
+ __func__, sl->reg_num.family, (unsigned long long)sl->reg_num.id,
+ sl->reg_num.crc, cmd->cmd, cmd->len);
+
+ return w1_process_command_io(sl->master, msg, hdr, cmd);
+}
+
+static int w1_process_command_root(struct cn_msg *msg, struct w1_netlink_msg *mcmd)
+{
+ struct w1_master *m;
+ struct cn_msg *cn;
+ struct w1_netlink_msg *w;
+ u32 *id;
+
+ if (mcmd->type != W1_LIST_MASTERS) {
+ printk(KERN_NOTICE "%s: msg: %x.%x, wrong type: %u, len: %u.\n",
+ __func__, msg->id.idx, msg->id.val, mcmd->type, mcmd->len);
+ return -EPROTO;
+ }
+
+ cn = kmalloc(PAGE_SIZE, GFP_KERNEL);
+ if (!cn)
+ return -ENOMEM;
+
+ cn->id.idx = CN_W1_IDX;
+ cn->id.val = CN_W1_VAL;
+
+ cn->seq = msg->seq;
+ cn->ack = 1;
+ cn->len = sizeof(struct w1_netlink_msg);
+ w = (struct w1_netlink_msg *)(cn + 1);
+
+ w->type = W1_LIST_MASTERS;
+ w->status = 0;
+ w->len = 0;
+ id = (u32 *)(w + 1);
+
+ mutex_lock(&w1_mlock);
+ list_for_each_entry(m, &w1_masters, w1_master_entry) {
+ if (cn->len + sizeof(*id) > PAGE_SIZE - sizeof(struct cn_msg)) {
+ cn_netlink_send(cn, 0, GFP_KERNEL);
+ cn->ack++;
+ cn->len = sizeof(struct w1_netlink_msg);
+ w->len = 0;
+ id = (u32 *)(w + 1);
+ }
+
+ *id = m->id;
+ w->len += sizeof(*id);
+ cn->len += sizeof(*id);
+ id++;
+ }
+ cn->ack = 0;
+ cn_netlink_send(cn, 0, GFP_KERNEL);
+ mutex_unlock(&w1_mlock);
+
+ kfree(cn);
+ return 0;
+}
+
+static int w1_netlink_send_error(struct cn_msg *rcmsg, struct w1_netlink_msg *rmsg,
+ struct w1_netlink_cmd *rcmd, int error)
+{
+ struct cn_msg *cmsg;
+ struct w1_netlink_msg *msg;
+ struct w1_netlink_cmd *cmd;
+
+ cmsg = kzalloc(sizeof(*msg) + sizeof(*cmd) + sizeof(*cmsg), GFP_KERNEL);
+ if (!cmsg)
+ return -ENOMEM;
+
+ msg = (struct w1_netlink_msg *)(cmsg + 1);
+ cmd = (struct w1_netlink_cmd *)(msg + 1);
+
+ memcpy(cmsg, rcmsg, sizeof(*cmsg));
+ cmsg->len = sizeof(*msg);
+
+ memcpy(msg, rmsg, sizeof(*msg));
+ msg->len = 0;
+ msg->status = (short)-error;
+
+ if (rcmd) {
+ memcpy(cmd, rcmd, sizeof(*cmd));
+ cmd->len = 0;
+ msg->len += sizeof(*cmd);
+ cmsg->len += sizeof(*cmd);
+ }
+
+ error = cn_netlink_send(cmsg, 0, GFP_KERNEL);
+ kfree(cmsg);
+
+ return error;
+}
+
static void w1_cn_callback(void *data)
{
struct cn_msg *msg = data;
@@ -144,6 +322,7 @@
dev = NULL;
sl = NULL;
+ cmd = NULL;
memcpy(&id, m->id.id, sizeof(id));
#if 0
@@ -155,15 +334,15 @@
break;
}
- if (!mlen)
- goto out_cont;
-
if (m->type == W1_MASTER_CMD) {
dev = w1_search_master_id(m->id.mst.id);
} else if (m->type == W1_SLAVE_CMD) {
sl = w1_search_slave(&id);
if (sl)
dev = sl->master;
+ } else {
+ err = w1_process_command_root(msg, m);
+ goto out_cont;
}
if (!dev) {
@@ -171,6 +350,10 @@
goto out_cont;
}
+ err = 0;
+ if (!mlen)
+ goto out_cont;
+
mutex_lock(&dev->mutex);
if (sl && w1_reset_select_slave(sl)) {
@@ -187,9 +370,12 @@
}
if (sl)
- w1_process_command_slave(sl, msg, m, cmd);
+ err = w1_process_command_slave(sl, msg, m, cmd);
else
- w1_process_command_master(dev, msg, m, cmd);
+ err = w1_process_command_master(dev, msg, m, cmd);
+
+ w1_netlink_send_error(msg, m, cmd, err);
+ err = 0;
cmd_data += cmd->len + sizeof(struct w1_netlink_cmd);
mlen -= cmd->len + sizeof(struct w1_netlink_cmd);
@@ -200,6 +386,8 @@
atomic_dec(&sl->refcnt);
mutex_unlock(&dev->mutex);
out_cont:
+ if (!cmd || err)
+ w1_netlink_send_error(msg, m, cmd, err);
msg->len -= sizeof(struct w1_netlink_msg) + m->len;
m = (struct w1_netlink_msg *)(((u8 *)m) + sizeof(struct w1_netlink_msg) + m->len);
@@ -209,11 +397,6 @@
if (err == -ENODEV)
err = 0;
}
-#if 0
- if (err) {
- printk("%s: malformed message. Dropping.\n", __func__);
- }
-#endif
}
int w1_init_netlink(void)
diff --git a/drivers/w1/w1_netlink.h b/drivers/w1/w1_netlink.h
index 56122b9..27e950f 100644
--- a/drivers/w1/w1_netlink.h
+++ b/drivers/w1/w1_netlink.h
@@ -34,12 +34,13 @@
W1_MASTER_REMOVE,
W1_MASTER_CMD,
W1_SLAVE_CMD,
+ W1_LIST_MASTERS,
};
struct w1_netlink_msg
{
__u8 type;
- __u8 reserved;
+ __u8 status;
__u16 len;
union {
__u8 id[8];
@@ -51,10 +52,15 @@
__u8 data[0];
};
-#define W1_CMD_READ 0x0
-#define W1_CMD_WRITE 0x1
-#define W1_CMD_SEARCH 0x2
-#define W1_CMD_ALARM_SEARCH 0x3
+enum w1_commands {
+ W1_CMD_READ = 0,
+ W1_CMD_WRITE,
+ W1_CMD_SEARCH,
+ W1_CMD_ALARM_SEARCH,
+ W1_CMD_TOUCH,
+ W1_CMD_RESET,
+ W1_CMD_MAX,
+};
struct w1_netlink_cmd
{
diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
index 4b75a16..526187c 100644
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -17,3 +17,27 @@
is not accidentally visible to other domains. Is it more
secure, but slightly less efficient.
If in doubt, say yes.
+
+config XENFS
+ tristate "Xen filesystem"
+ depends on XEN
+ default y
+ help
+ The xen filesystem provides a way for domains to share
+ information with each other and with the hypervisor.
+ For example, by reading and writing the "xenbus" file, guests
+ may pass arbitrary information to the initial domain.
+ If in doubt, say yes.
+
+config XEN_COMPAT_XENFS
+ bool "Create compatibility mount point /proc/xen"
+ depends on XENFS
+ default y
+ help
+ The old xenstore userspace tools expect to find "xenbus"
+ under /proc/xen, but "xenbus" is now found at the root of the
+ xenfs filesystem. Selecting this causes the kernel to create
+ the compatibilty mount point /proc/xen if it is running on
+ a xen platform.
+ If in doubt, say yes.
+
diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile
index d2a8fdf..ff8accc 100644
--- a/drivers/xen/Makefile
+++ b/drivers/xen/Makefile
@@ -1,5 +1,7 @@
obj-y += grant-table.o features.o events.o manage.o
obj-y += xenbus/
+
obj-$(CONFIG_HOTPLUG_CPU) += cpu_hotplug.o
obj-$(CONFIG_XEN_XENCOMM) += xencomm.o
obj-$(CONFIG_XEN_BALLOON) += balloon.o
+obj-$(CONFIG_XENFS) += xenfs/
\ No newline at end of file
diff --git a/drivers/xen/xenbus/xenbus_client.c b/drivers/xen/xenbus/xenbus_client.c
index 9678b3e..92a1ef8 100644
--- a/drivers/xen/xenbus/xenbus_client.c
+++ b/drivers/xen/xenbus/xenbus_client.c
@@ -136,7 +136,6 @@
/**
* xenbus_switch_state
* @dev: xenbus device
- * @xbt: transaction handle
* @state: new state
*
* Advertise in the store a change of the given driver to the given new_state.
@@ -267,7 +266,7 @@
* @fmt: error message format
*
* Equivalent to xenbus_dev_error(dev, err, fmt, args), followed by
- * xenbus_switch_state(dev, NULL, XenbusStateClosing) to schedule an orderly
+ * xenbus_switch_state(dev, XenbusStateClosing) to schedule an orderly
* closedown of this driver and its peer.
*/
diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c
index b2a0318..773d1cf 100644
--- a/drivers/xen/xenbus/xenbus_probe.c
+++ b/drivers/xen/xenbus/xenbus_probe.c
@@ -40,6 +40,7 @@
#include <linux/ctype.h>
#include <linux/fcntl.h>
#include <linux/mm.h>
+#include <linux/proc_fs.h>
#include <linux/notifier.h>
#include <linux/kthread.h>
#include <linux/mutex.h>
@@ -55,7 +56,10 @@
#include "xenbus_comms.h"
#include "xenbus_probe.h"
+
int xen_store_evtchn;
+EXPORT_SYMBOL(xen_store_evtchn);
+
struct xenstore_domain_interface *xen_store_interface;
static unsigned long xen_store_mfn;
@@ -166,6 +170,9 @@
return read_otherend_details(xendev, "backend-id", "backend");
}
+static struct device_attribute xenbus_dev_attrs[] = {
+ __ATTR_NULL
+};
/* Bus type for frontend drivers. */
static struct xen_bus_type xenbus_frontend = {
@@ -174,12 +181,13 @@
.get_bus_id = frontend_bus_id,
.probe = xenbus_probe_frontend,
.bus = {
- .name = "xen",
- .match = xenbus_match,
- .uevent = xenbus_uevent,
- .probe = xenbus_dev_probe,
- .remove = xenbus_dev_remove,
- .shutdown = xenbus_dev_shutdown,
+ .name = "xen",
+ .match = xenbus_match,
+ .uevent = xenbus_uevent,
+ .probe = xenbus_dev_probe,
+ .remove = xenbus_dev_remove,
+ .shutdown = xenbus_dev_shutdown,
+ .dev_attrs = xenbus_dev_attrs,
},
};
@@ -852,6 +860,14 @@
if (!xen_initial_domain())
xenbus_probe(NULL);
+#ifdef CONFIG_XEN_COMPAT_XENFS
+ /*
+ * Create xenfs mountpoint in /proc for compatibility with
+ * utilities that expect to find "xenbus" under "/proc/xen".
+ */
+ proc_mkdir("xen", NULL);
+#endif
+
return 0;
out_unreg_back:
diff --git a/drivers/xen/xenbus/xenbus_xs.c b/drivers/xen/xenbus/xenbus_xs.c
index 7f2f91c..e325eab 100644
--- a/drivers/xen/xenbus/xenbus_xs.c
+++ b/drivers/xen/xenbus/xenbus_xs.c
@@ -184,6 +184,7 @@
return ret;
}
+EXPORT_SYMBOL(xenbus_dev_request_and_reply);
/* Send message to xs, get kmalloc'ed reply. ERR_PTR() on error. */
static void *xs_talkv(struct xenbus_transaction t,
diff --git a/drivers/xen/xenfs/Makefile b/drivers/xen/xenfs/Makefile
new file mode 100644
index 0000000..25275c3
--- /dev/null
+++ b/drivers/xen/xenfs/Makefile
@@ -0,0 +1,3 @@
+obj-$(CONFIG_XENFS) += xenfs.o
+
+xenfs-objs = super.o xenbus.o
\ No newline at end of file
diff --git a/drivers/xen/xenfs/super.c b/drivers/xen/xenfs/super.c
new file mode 100644
index 0000000..515741a
--- /dev/null
+++ b/drivers/xen/xenfs/super.c
@@ -0,0 +1,64 @@
+/*
+ * xenfs.c - a filesystem for passing info between the a domain and
+ * the hypervisor.
+ *
+ * 2008-10-07 Alex Zeffertt Replaced /proc/xen/xenbus with xenfs filesystem
+ * and /proc/xen compatibility mount point.
+ * Turned xenfs into a loadable module.
+ */
+
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/module.h>
+#include <linux/fs.h>
+#include <linux/magic.h>
+
+#include "xenfs.h"
+
+#include <asm/xen/hypervisor.h>
+
+MODULE_DESCRIPTION("Xen filesystem");
+MODULE_LICENSE("GPL");
+
+static int xenfs_fill_super(struct super_block *sb, void *data, int silent)
+{
+ static struct tree_descr xenfs_files[] = {
+ [2] = {"xenbus", &xenbus_file_ops, S_IRUSR|S_IWUSR},
+ {""},
+ };
+
+ return simple_fill_super(sb, XENFS_SUPER_MAGIC, xenfs_files);
+}
+
+static int xenfs_get_sb(struct file_system_type *fs_type,
+ int flags, const char *dev_name,
+ void *data, struct vfsmount *mnt)
+{
+ return get_sb_single(fs_type, flags, data, xenfs_fill_super, mnt);
+}
+
+static struct file_system_type xenfs_type = {
+ .owner = THIS_MODULE,
+ .name = "xenfs",
+ .get_sb = xenfs_get_sb,
+ .kill_sb = kill_litter_super,
+};
+
+static int __init xenfs_init(void)
+{
+ if (xen_pv_domain())
+ return register_filesystem(&xenfs_type);
+
+ printk(KERN_INFO "XENFS: not registering filesystem on non-xen platform\n");
+ return 0;
+}
+
+static void __exit xenfs_exit(void)
+{
+ if (xen_pv_domain())
+ unregister_filesystem(&xenfs_type);
+}
+
+module_init(xenfs_init);
+module_exit(xenfs_exit);
+
diff --git a/drivers/xen/xenfs/xenbus.c b/drivers/xen/xenfs/xenbus.c
new file mode 100644
index 0000000..875a4c5
--- /dev/null
+++ b/drivers/xen/xenfs/xenbus.c
@@ -0,0 +1,593 @@
+/*
+ * Driver giving user-space access to the kernel's xenbus connection
+ * to xenstore.
+ *
+ * Copyright (c) 2005, Christian Limpach
+ * Copyright (c) 2005, Rusty Russell, IBM Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version 2
+ * as published by the Free Software Foundation; or, when distributed
+ * separately from the Linux kernel or incorporated into other
+ * software packages, subject to the following license:
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this source file (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use, copy, modify,
+ * merge, publish, distribute, sublicense, and/or sell copies of the Software,
+ * and to permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ * Changes:
+ * 2008-10-07 Alex Zeffertt Replaced /proc/xen/xenbus with xenfs filesystem
+ * and /proc/xen compatibility mount point.
+ * Turned xenfs into a loadable module.
+ */
+
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/uio.h>
+#include <linux/notifier.h>
+#include <linux/wait.h>
+#include <linux/fs.h>
+#include <linux/poll.h>
+#include <linux/mutex.h>
+#include <linux/spinlock.h>
+#include <linux/mount.h>
+#include <linux/pagemap.h>
+#include <linux/uaccess.h>
+#include <linux/init.h>
+#include <linux/namei.h>
+#include <linux/string.h>
+
+#include "xenfs.h"
+#include "../xenbus/xenbus_comms.h"
+
+#include <xen/xenbus.h>
+#include <asm/xen/hypervisor.h>
+
+/*
+ * An element of a list of outstanding transactions, for which we're
+ * still waiting a reply.
+ */
+struct xenbus_transaction_holder {
+ struct list_head list;
+ struct xenbus_transaction handle;
+};
+
+/*
+ * A buffer of data on the queue.
+ */
+struct read_buffer {
+ struct list_head list;
+ unsigned int cons;
+ unsigned int len;
+ char msg[];
+};
+
+struct xenbus_file_priv {
+ /*
+ * msgbuffer_mutex is held while partial requests are built up
+ * and complete requests are acted on. It therefore protects
+ * the "transactions" and "watches" lists, and the partial
+ * request length and buffer.
+ *
+ * reply_mutex protects the reply being built up to return to
+ * usermode. It nests inside msgbuffer_mutex but may be held
+ * alone during a watch callback.
+ */
+ struct mutex msgbuffer_mutex;
+
+ /* In-progress transactions */
+ struct list_head transactions;
+
+ /* Active watches. */
+ struct list_head watches;
+
+ /* Partial request. */
+ unsigned int len;
+ union {
+ struct xsd_sockmsg msg;
+ char buffer[PAGE_SIZE];
+ } u;
+
+ /* Response queue. */
+ struct mutex reply_mutex;
+ struct list_head read_buffers;
+ wait_queue_head_t read_waitq;
+
+};
+
+/* Read out any raw xenbus messages queued up. */
+static ssize_t xenbus_file_read(struct file *filp,
+ char __user *ubuf,
+ size_t len, loff_t *ppos)
+{
+ struct xenbus_file_priv *u = filp->private_data;
+ struct read_buffer *rb;
+ unsigned i;
+ int ret;
+
+ mutex_lock(&u->reply_mutex);
+ while (list_empty(&u->read_buffers)) {
+ mutex_unlock(&u->reply_mutex);
+ ret = wait_event_interruptible(u->read_waitq,
+ !list_empty(&u->read_buffers));
+ if (ret)
+ return ret;
+ mutex_lock(&u->reply_mutex);
+ }
+
+ rb = list_entry(u->read_buffers.next, struct read_buffer, list);
+ i = 0;
+ while (i < len) {
+ unsigned sz = min((unsigned)len - i, rb->len - rb->cons);
+
+ ret = copy_to_user(ubuf + i, &rb->msg[rb->cons], sz);
+
+ i += sz - ret;
+ rb->cons += sz - ret;
+
+ if (ret != sz) {
+ if (i == 0)
+ i = -EFAULT;
+ goto out;
+ }
+
+ /* Clear out buffer if it has been consumed */
+ if (rb->cons == rb->len) {
+ list_del(&rb->list);
+ kfree(rb);
+ if (list_empty(&u->read_buffers))
+ break;
+ rb = list_entry(u->read_buffers.next,
+ struct read_buffer, list);
+ }
+ }
+
+out:
+ mutex_unlock(&u->reply_mutex);
+ return i;
+}
+
+/*
+ * Add a buffer to the queue. Caller must hold the appropriate lock
+ * if the queue is not local. (Commonly the caller will build up
+ * multiple queued buffers on a temporary local list, and then add it
+ * to the appropriate list under lock once all the buffers have een
+ * successfully allocated.)
+ */
+static int queue_reply(struct list_head *queue, const void *data, size_t len)
+{
+ struct read_buffer *rb;
+
+ if (len == 0)
+ return 0;
+
+ rb = kmalloc(sizeof(*rb) + len, GFP_KERNEL);
+ if (rb == NULL)
+ return -ENOMEM;
+
+ rb->cons = 0;
+ rb->len = len;
+
+ memcpy(rb->msg, data, len);
+
+ list_add_tail(&rb->list, queue);
+ return 0;
+}
+
+/*
+ * Free all the read_buffer s on a list.
+ * Caller must have sole reference to list.
+ */
+static void queue_cleanup(struct list_head *list)
+{
+ struct read_buffer *rb;
+
+ while (!list_empty(list)) {
+ rb = list_entry(list->next, struct read_buffer, list);
+ list_del(list->next);
+ kfree(rb);
+ }
+}
+
+struct watch_adapter {
+ struct list_head list;
+ struct xenbus_watch watch;
+ struct xenbus_file_priv *dev_data;
+ char *token;
+};
+
+static void free_watch_adapter(struct watch_adapter *watch)
+{
+ kfree(watch->watch.node);
+ kfree(watch->token);
+ kfree(watch);
+}
+
+static struct watch_adapter *alloc_watch_adapter(const char *path,
+ const char *token)
+{
+ struct watch_adapter *watch;
+
+ watch = kzalloc(sizeof(*watch), GFP_KERNEL);
+ if (watch == NULL)
+ goto out_fail;
+
+ watch->watch.node = kstrdup(path, GFP_KERNEL);
+ if (watch->watch.node == NULL)
+ goto out_free;
+
+ watch->token = kstrdup(token, GFP_KERNEL);
+ if (watch->token == NULL)
+ goto out_free;
+
+ return watch;
+
+out_free:
+ free_watch_adapter(watch);
+
+out_fail:
+ return NULL;
+}
+
+static void watch_fired(struct xenbus_watch *watch,
+ const char **vec,
+ unsigned int len)
+{
+ struct watch_adapter *adap;
+ struct xsd_sockmsg hdr;
+ const char *path, *token;
+ int path_len, tok_len, body_len, data_len = 0;
+ int ret;
+ LIST_HEAD(staging_q);
+
+ adap = container_of(watch, struct watch_adapter, watch);
+
+ path = vec[XS_WATCH_PATH];
+ token = adap->token;
+
+ path_len = strlen(path) + 1;
+ tok_len = strlen(token) + 1;
+ if (len > 2)
+ data_len = vec[len] - vec[2] + 1;
+ body_len = path_len + tok_len + data_len;
+
+ hdr.type = XS_WATCH_EVENT;
+ hdr.len = body_len;
+
+ mutex_lock(&adap->dev_data->reply_mutex);
+
+ ret = queue_reply(&staging_q, &hdr, sizeof(hdr));
+ if (!ret)
+ ret = queue_reply(&staging_q, path, path_len);
+ if (!ret)
+ ret = queue_reply(&staging_q, token, tok_len);
+ if (!ret && len > 2)
+ ret = queue_reply(&staging_q, vec[2], data_len);
+
+ if (!ret) {
+ /* success: pass reply list onto watcher */
+ list_splice_tail(&staging_q, &adap->dev_data->read_buffers);
+ wake_up(&adap->dev_data->read_waitq);
+ } else
+ queue_cleanup(&staging_q);
+
+ mutex_unlock(&adap->dev_data->reply_mutex);
+}
+
+static int xenbus_write_transaction(unsigned msg_type,
+ struct xenbus_file_priv *u)
+{
+ int rc, ret;
+ void *reply;
+ struct xenbus_transaction_holder *trans = NULL;
+ LIST_HEAD(staging_q);
+
+ if (msg_type == XS_TRANSACTION_START) {
+ trans = kmalloc(sizeof(*trans), GFP_KERNEL);
+ if (!trans) {
+ rc = -ENOMEM;
+ goto out;
+ }
+ }
+
+ reply = xenbus_dev_request_and_reply(&u->u.msg);
+ if (IS_ERR(reply)) {
+ kfree(trans);
+ rc = PTR_ERR(reply);
+ goto out;
+ }
+
+ if (msg_type == XS_TRANSACTION_START) {
+ trans->handle.id = simple_strtoul(reply, NULL, 0);
+
+ list_add(&trans->list, &u->transactions);
+ } else if (msg_type == XS_TRANSACTION_END) {
+ list_for_each_entry(trans, &u->transactions, list)
+ if (trans->handle.id == u->u.msg.tx_id)
+ break;
+ BUG_ON(&trans->list == &u->transactions);
+ list_del(&trans->list);
+
+ kfree(trans);
+ }
+
+ mutex_lock(&u->reply_mutex);
+ ret = queue_reply(&staging_q, &u->u.msg, sizeof(u->u.msg));
+ if (!ret)
+ ret = queue_reply(&staging_q, reply, u->u.msg.len);
+ if (!ret) {
+ list_splice_tail(&staging_q, &u->read_buffers);
+ wake_up(&u->read_waitq);
+ } else {
+ queue_cleanup(&staging_q);
+ rc = ret;
+ }
+ mutex_unlock(&u->reply_mutex);
+
+ kfree(reply);
+
+out:
+ return rc;
+}
+
+static int xenbus_write_watch(unsigned msg_type, struct xenbus_file_priv *u)
+{
+ struct watch_adapter *watch, *tmp_watch;
+ char *path, *token;
+ int err, rc;
+ LIST_HEAD(staging_q);
+
+ path = u->u.buffer + sizeof(u->u.msg);
+ token = memchr(path, 0, u->u.msg.len);
+ if (token == NULL) {
+ rc = -EILSEQ;
+ goto out;
+ }
+ token++;
+
+ if (msg_type == XS_WATCH) {
+ watch = alloc_watch_adapter(path, token);
+ if (watch == NULL) {
+ rc = -ENOMEM;
+ goto out;
+ }
+
+ watch->watch.callback = watch_fired;
+ watch->dev_data = u;
+
+ err = register_xenbus_watch(&watch->watch);
+ if (err) {
+ free_watch_adapter(watch);
+ rc = err;
+ goto out;
+ }
+ list_add(&watch->list, &u->watches);
+ } else {
+ list_for_each_entry_safe(watch, tmp_watch, &u->watches, list) {
+ if (!strcmp(watch->token, token) &&
+ !strcmp(watch->watch.node, path)) {
+ unregister_xenbus_watch(&watch->watch);
+ list_del(&watch->list);
+ free_watch_adapter(watch);
+ break;
+ }
+ }
+ }
+
+ /* Success. Synthesize a reply to say all is OK. */
+ {
+ struct {
+ struct xsd_sockmsg hdr;
+ char body[3];
+ } __packed reply = {
+ {
+ .type = msg_type,
+ .len = sizeof(reply.body)
+ },
+ "OK"
+ };
+
+ mutex_lock(&u->reply_mutex);
+ rc = queue_reply(&u->read_buffers, &reply, sizeof(reply));
+ mutex_unlock(&u->reply_mutex);
+ }
+
+out:
+ return rc;
+}
+
+static ssize_t xenbus_file_write(struct file *filp,
+ const char __user *ubuf,
+ size_t len, loff_t *ppos)
+{
+ struct xenbus_file_priv *u = filp->private_data;
+ uint32_t msg_type;
+ int rc = len;
+ int ret;
+ LIST_HEAD(staging_q);
+
+ /*
+ * We're expecting usermode to be writing properly formed
+ * xenbus messages. If they write an incomplete message we
+ * buffer it up. Once it is complete, we act on it.
+ */
+
+ /*
+ * Make sure concurrent writers can't stomp all over each
+ * other's messages and make a mess of our partial message
+ * buffer. We don't make any attemppt to stop multiple
+ * writers from making a mess of each other's incomplete
+ * messages; we're just trying to guarantee our own internal
+ * consistency and make sure that single writes are handled
+ * atomically.
+ */
+ mutex_lock(&u->msgbuffer_mutex);
+
+ /* Get this out of the way early to avoid confusion */
+ if (len == 0)
+ goto out;
+
+ /* Can't write a xenbus message larger we can buffer */
+ if ((len + u->len) > sizeof(u->u.buffer)) {
+ /* On error, dump existing buffer */
+ u->len = 0;
+ rc = -EINVAL;
+ goto out;
+ }
+
+ ret = copy_from_user(u->u.buffer + u->len, ubuf, len);
+
+ if (ret == len) {
+ rc = -EFAULT;
+ goto out;
+ }
+
+ /* Deal with a partial copy. */
+ len -= ret;
+ rc = len;
+
+ u->len += len;
+
+ /* Return if we haven't got a full message yet */
+ if (u->len < sizeof(u->u.msg))
+ goto out; /* not even the header yet */
+
+ /* If we're expecting a message that's larger than we can
+ possibly send, dump what we have and return an error. */
+ if ((sizeof(u->u.msg) + u->u.msg.len) > sizeof(u->u.buffer)) {
+ rc = -E2BIG;
+ u->len = 0;
+ goto out;
+ }
+
+ if (u->len < (sizeof(u->u.msg) + u->u.msg.len))
+ goto out; /* incomplete data portion */
+
+ /*
+ * OK, now we have a complete message. Do something with it.
+ */
+
+ msg_type = u->u.msg.type;
+
+ switch (msg_type) {
+ case XS_TRANSACTION_START:
+ case XS_TRANSACTION_END:
+ case XS_DIRECTORY:
+ case XS_READ:
+ case XS_GET_PERMS:
+ case XS_RELEASE:
+ case XS_GET_DOMAIN_PATH:
+ case XS_WRITE:
+ case XS_MKDIR:
+ case XS_RM:
+ case XS_SET_PERMS:
+ /* Send out a transaction */
+ ret = xenbus_write_transaction(msg_type, u);
+ break;
+
+ case XS_WATCH:
+ case XS_UNWATCH:
+ /* (Un)Ask for some path to be watched for changes */
+ ret = xenbus_write_watch(msg_type, u);
+ break;
+
+ default:
+ ret = -EINVAL;
+ break;
+ }
+ if (ret != 0)
+ rc = ret;
+
+ /* Buffered message consumed */
+ u->len = 0;
+
+ out:
+ mutex_unlock(&u->msgbuffer_mutex);
+ return rc;
+}
+
+static int xenbus_file_open(struct inode *inode, struct file *filp)
+{
+ struct xenbus_file_priv *u;
+
+ if (xen_store_evtchn == 0)
+ return -ENOENT;
+
+ nonseekable_open(inode, filp);
+
+ u = kzalloc(sizeof(*u), GFP_KERNEL);
+ if (u == NULL)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&u->transactions);
+ INIT_LIST_HEAD(&u->watches);
+ INIT_LIST_HEAD(&u->read_buffers);
+ init_waitqueue_head(&u->read_waitq);
+
+ mutex_init(&u->reply_mutex);
+ mutex_init(&u->msgbuffer_mutex);
+
+ filp->private_data = u;
+
+ return 0;
+}
+
+static int xenbus_file_release(struct inode *inode, struct file *filp)
+{
+ struct xenbus_file_priv *u = filp->private_data;
+ struct xenbus_transaction_holder *trans, *tmp;
+ struct watch_adapter *watch, *tmp_watch;
+
+ /*
+ * No need for locking here because there are no other users,
+ * by definition.
+ */
+
+ list_for_each_entry_safe(trans, tmp, &u->transactions, list) {
+ xenbus_transaction_end(trans->handle, 1);
+ list_del(&trans->list);
+ kfree(trans);
+ }
+
+ list_for_each_entry_safe(watch, tmp_watch, &u->watches, list) {
+ unregister_xenbus_watch(&watch->watch);
+ list_del(&watch->list);
+ free_watch_adapter(watch);
+ }
+
+ kfree(u);
+
+ return 0;
+}
+
+static unsigned int xenbus_file_poll(struct file *file, poll_table *wait)
+{
+ struct xenbus_file_priv *u = file->private_data;
+
+ poll_wait(file, &u->read_waitq, wait);
+ if (!list_empty(&u->read_buffers))
+ return POLLIN | POLLRDNORM;
+ return 0;
+}
+
+const struct file_operations xenbus_file_ops = {
+ .read = xenbus_file_read,
+ .write = xenbus_file_write,
+ .open = xenbus_file_open,
+ .release = xenbus_file_release,
+ .poll = xenbus_file_poll,
+};
diff --git a/drivers/xen/xenfs/xenfs.h b/drivers/xen/xenfs/xenfs.h
new file mode 100644
index 0000000..51f08b2
--- /dev/null
+++ b/drivers/xen/xenfs/xenfs.h
@@ -0,0 +1,6 @@
+#ifndef _XENFS_XENBUS_H
+#define _XENFS_XENBUS_H
+
+extern const struct file_operations xenbus_file_ops;
+
+#endif /* _XENFS_XENBUS_H */
diff --git a/firmware/Makefile b/firmware/Makefile
index 55d3082..ea1d28f 100644
--- a/firmware/Makefile
+++ b/firmware/Makefile
@@ -37,6 +37,8 @@
cxgb3/t3c_psram-1.1.0.bin \
cxgb3/t3fw-7.0.0.bin
fw-shipped-$(CONFIG_DVB_TTUSB_BUDGET) += ttusb-budget/dspbootcode.bin
+fw-shipped-$(CONFIG_E100) += e100/d101m_ucode.bin e100/d101s_ucode.bin \
+ e100/d102e_ucode.bin
fw-shipped-$(CONFIG_SMCTR) += tr_smctr.bin
fw-shipped-$(CONFIG_SND_KORG1212) += korg/k1212.dsp
fw-shipped-$(CONFIG_SND_MAESTRO3) += ess/maestro3_assp_kernel.fw \
diff --git a/firmware/WHENCE b/firmware/WHENCE
index 1bb2cf4..8b56513 100644
--- a/firmware/WHENCE
+++ b/firmware/WHENCE
@@ -360,6 +360,18 @@
--------------------------------------------------------------------------
+Driver: e100 -- Intel PRO/100 Ethernet NIC
+
+File: e100/d101m_ucode.bin
+File: e100/d101s_ucode.bin
+File: e100/d102e_ucode.bin
+
+Licence: Unknown
+
+Found in hex form in kernel source.
+
+--------------------------------------------------------------------------
+
Driver: acenic -- Alteon AceNIC Gigabit Ethernet card
File: acenic/tg1.bin
diff --git a/firmware/e100/d101m_ucode.bin.ihex b/firmware/e100/d101m_ucode.bin.ihex
new file mode 100644
index 0000000..12971ed
--- /dev/null
+++ b/firmware/e100/d101m_ucode.bin.ihex
@@ -0,0 +1,38 @@
+:10000000150255003704FFFFFFFFFFFF8907A70612
+:10001000FFFFFFFFFFFF580501000C001213100047
+:1000200008000C00160238009C001000564020000A
+:10003000CC802300560038009C0010000B4C24009C
+:1000400000080000184812003804380000000000C2
+:1000500000001400550538000080300062061000D2
+:100060006105100008040E006148130002000C0036
+:10007000933010000080300024061000610510004D
+:1000800008040E00610810007E000C00212C2200E4
+:1000900002000C00933010007A0C380000000800B9
+:1000A000903010007A0C38000000000000000000C2
+:1000B00000000000000000009C0010002D4C2400F7
+:1000C000040001000010040037043A00104004004E
+:1000D0008A07380000000000990010007A6C2000A8
+:1000E0009C001000484C24002408130001000C0060
+:1000F00013121000750C260000100400040001000B
+:100100002608130006000C00A806220026C91300CA
+:1001100013131000A80638000000000000000000C3
+:1001200000000000000000000000000000000000CF
+:10013000000000000000000000060800101B100076
+:10014000040005002608100010121000340C3800BE
+:1001500000000000000000005B1521009900100065
+:10016000596520009C0010005945240036081300F2
+:1001700000000C00620C220001000C00131B100098
+:100180000E9C22000E0C21000E6C22000E6C210031
+:100190000EFC22000E5C21000E4C2100550538009B
+:1001A0000400010000100400678C27000008040010
+:1001B0000081010037043A002608130001000C00FA
+:1001C00059052200131310005905380000000000E3
+:1001D000000000000000000000000000000000001F
+:1001E00000000000000000000000000031081300C3
+:1001F0000B0910001348120080FF0C00AB0626000C
+:100200000010040004000100A806380000000000EF
+:0B02100000000000000000004E417ED6
+:00000001FF
+/********************************************************/
+/* Micro code for 8086:1229 Rev 8 */
+/********************************************************/
diff --git a/firmware/e100/d101s_ucode.bin.ihex b/firmware/e100/d101s_ucode.bin.ihex
new file mode 100644
index 0000000..102c7fe
--- /dev/null
+++ b/firmware/e100/d101s_ucode.bin.ihex
@@ -0,0 +1,38 @@
+:10000000420255007E04FFFFFFFFFFFF1808FF06B6
+:10001000FFFFFFFFFFFFA60501000C0012131000F9
+:1000200008000C00430238009C00100056402000DD
+:10003000D0802300560038009C0010008B4F240015
+:1000400000080000184812007F043800000000007B
+:1000500000001400A30538000080300010061000D6
+:100060006105100008040E006148130002000C0036
+:10007000933010000080300024061000610510004D
+:1000800008040E00610810007E000C00A12F220061
+:1000900002000C0093301000900F380000000800A0
+:1000A00090301000900F38000000000000000000A9
+:1000B00000000000000000009C001000AD4F240074
+:1000C00004000100001004007E043A001040040007
+:1000D000190838000000000099001000FD6F200092
+:1000E0009A001000FDAF20009C001000C84F2400B3
+:1000F0002408130001000C0013121000F70F260053
+:1001000000100400040001002608130006000C0083
+:100110000007220026C9130013131000000738003F
+:1001200000000000000000000000000000000000CF
+:10013000000000000000000000060800101B100076
+:10014000040005002608100010121000B60F380039
+:100150000000000000000000A91521009900100017
+:10016000A76520009A001000A7A520009C001000A1
+:10017000A74524003608130000000C00E40F2200FD
+:1001800001000C00131B10008E9F22008E0F210017
+:100190008E6F22008E6F21008EFF22008E5F210065
+:1001A0008E4F2100A3053800040001000010040058
+:1001B000E98F270000080400008101007E043A0056
+:1001C0002608130001000C00A705220013131000DD
+:1001D000A70538000000000000000000000000003B
+:1001E000000000000000000000000000000000000F
+:1001F00000000000310813000B0910001348120022
+:1002000080FF0C000307260000100400040001001A
+:0B02100000073800000000004E438093
+:00000001FF
+/********************************************************/
+/* Micro code for 8086:1229 Rev 9 */
+/********************************************************/
diff --git a/firmware/e100/d102e_ucode.bin.ihex b/firmware/e100/d102e_ucode.bin.ihex
new file mode 100644
index 0000000..9e806da
--- /dev/null
+++ b/firmware/e100/d102e_ucode.bin.ihex
@@ -0,0 +1,38 @@
+:100000008F027D00F904420E850CED14E914FA14F8
+:10001000360EF70EFF1FFF1FB914E00000000000AE
+:100020000000000000000000BD14E000000000001F
+:100030000000000000000000D514E00000000000F7
+:1000400000000000000000000000000000000000B0
+:100050000000000000000000C114E00000000000EB
+:100060000000000000000000000000000000000090
+:100070000000000000000000000000000000000080
+:100080000000000000000000000000000000000070
+:100090000000000000000000C814E00000000000A4
+:1000A000000000000000000000062000EE14E00048
+:1000B000000000000000000080FF3000460E9400A9
+:1000C0000082030000201000430EE000000000004A
+:1000D000000000000000000006003000FB14E000FB
+:1000E0000000000000000000000000000000000010
+:1000F0000000000000000000000000000000000000
+:1001000000000000000000000000000000000000EF
+:100110000000000000000000416E90003C0E8000D6
+:10012000390EE00000000000FD6E9000FD0E900012
+:10013000F80EE000000000000000000000000000D9
+:1001400000000000000000000000000000000000AF
+:10015000000000000000000000000000000000009F
+:10016000000000000000000000000000000000008F
+:10017000000000000000000000000000000000007F
+:10018000000000000000000000000000000000006F
+:10019000000000000000000000000000000000005F
+:1001A000000000000000000000000000000000004F
+:1001B000000000000000000000000000000000003F
+:1001C000000000000000000000000000000000002F
+:1001D000000000000000000000000000000000001F
+:1001E000000000000000000000000000000000000F
+:1001F00000000000000000000000000000000000FF
+:1002000000000000000000000000000000000000EE
+:0B02100000000000000000002A362E55
+:00000001FF
+/********************************************************/
+/* Micro code for the 8086:1229 Rev F/10 */
+/********************************************************/
diff --git a/fs/Kconfig b/fs/Kconfig
index 3288358..02cff86 100644
--- a/fs/Kconfig
+++ b/fs/Kconfig
@@ -269,6 +269,25 @@
Posix Access Control Lists (ACLs) support permissions for users and
groups beyond the owner/group/world scheme.
+config BTRFS_FS
+ tristate "Btrfs filesystem (EXPERIMENTAL) Unstable disk format"
+ depends on EXPERIMENTAL
+ select LIBCRC32C
+ select ZLIB_INFLATE
+ select ZLIB_DEFLATE
+ help
+ Btrfs is a new filesystem with extents, writable snapshotting,
+ support for multiple devices and many more features.
+
+ Btrfs is highly experimental, and THE DISK FORMAT IS NOT YET
+ FINALIZED. You should say N here unless you are interested in
+ testing Btrfs with non-critical data.
+
+ To compile this file system support as a module, choose M here. The
+ module will be called btrfs.
+
+ If unsure, say N.
+
endif # BLOCK
source "fs/notify/Kconfig"
diff --git a/fs/Makefile b/fs/Makefile
index c830611..bc4e14d 100644
--- a/fs/Makefile
+++ b/fs/Makefile
@@ -119,4 +119,5 @@
obj-$(CONFIG_HPPFS) += hppfs/
obj-$(CONFIG_DEBUG_FS) += debugfs/
obj-$(CONFIG_OCFS2_FS) += ocfs2/
+obj-$(CONFIG_BTRFS_FS) += btrfs/
obj-$(CONFIG_GFS2_FS) += gfs2/
diff --git a/fs/binfmt_elf.c b/fs/binfmt_elf.c
index c41fa2a..e3ff2b9 100644
--- a/fs/binfmt_elf.c
+++ b/fs/binfmt_elf.c
@@ -152,8 +152,10 @@
elf_addr_t __user *sp;
elf_addr_t __user *u_platform;
elf_addr_t __user *u_base_platform;
+ elf_addr_t __user *u_rand_bytes;
const char *k_platform = ELF_PLATFORM;
const char *k_base_platform = ELF_BASE_PLATFORM;
+ unsigned char k_rand_bytes[16];
int items;
elf_addr_t *elf_info;
int ei_index = 0;
@@ -196,6 +198,15 @@
return -EFAULT;
}
+ /*
+ * Generate 16 random bytes for userspace PRNG seeding.
+ */
+ get_random_bytes(k_rand_bytes, sizeof(k_rand_bytes));
+ u_rand_bytes = (elf_addr_t __user *)
+ STACK_ALLOC(p, sizeof(k_rand_bytes));
+ if (__copy_to_user(u_rand_bytes, k_rand_bytes, sizeof(k_rand_bytes)))
+ return -EFAULT;
+
/* Create the ELF interpreter info */
elf_info = (elf_addr_t *)current->mm->saved_auxv;
/* update AT_VECTOR_SIZE_BASE if the number of NEW_AUX_ENT() changes */
@@ -228,6 +239,7 @@
NEW_AUX_ENT(AT_GID, cred->gid);
NEW_AUX_ENT(AT_EGID, cred->egid);
NEW_AUX_ENT(AT_SECURE, security_bprm_secureexec(bprm));
+ NEW_AUX_ENT(AT_RANDOM, (elf_addr_t)(unsigned long)u_rand_bytes);
NEW_AUX_ENT(AT_EXECFN, bprm->exec);
if (k_platform) {
NEW_AUX_ENT(AT_PLATFORM,
diff --git a/fs/bio.c b/fs/bio.c
index 711cee1..062299a 100644
--- a/fs/bio.c
+++ b/fs/bio.c
@@ -788,6 +788,7 @@
int i, ret;
int nr_pages = 0;
unsigned int len = 0;
+ unsigned int offset = map_data ? map_data->offset & ~PAGE_MASK : 0;
for (i = 0; i < iov_count; i++) {
unsigned long uaddr;
@@ -814,35 +815,42 @@
bio->bi_rw |= (!write_to_vm << BIO_RW);
ret = 0;
- i = 0;
- while (len) {
- unsigned int bytes;
- if (map_data)
- bytes = 1U << (PAGE_SHIFT + map_data->page_order);
- else
- bytes = PAGE_SIZE;
+ if (map_data) {
+ nr_pages = 1 << map_data->page_order;
+ i = map_data->offset / PAGE_SIZE;
+ }
+ while (len) {
+ unsigned int bytes = PAGE_SIZE;
+
+ bytes -= offset;
if (bytes > len)
bytes = len;
if (map_data) {
- if (i == map_data->nr_entries) {
+ if (i == map_data->nr_entries * nr_pages) {
ret = -ENOMEM;
break;
}
- page = map_data->pages[i++];
- } else
+
+ page = map_data->pages[i / nr_pages];
+ page += (i % nr_pages);
+
+ i++;
+ } else {
page = alloc_page(q->bounce_gfp | gfp_mask);
- if (!page) {
- ret = -ENOMEM;
- break;
+ if (!page) {
+ ret = -ENOMEM;
+ break;
+ }
}
- if (bio_add_pc_page(q, bio, page, bytes, 0) < bytes)
+ if (bio_add_pc_page(q, bio, page, bytes, offset) < bytes)
break;
len -= bytes;
+ offset = 0;
}
if (ret)
@@ -851,7 +859,7 @@
/*
* success
*/
- if (!write_to_vm) {
+ if (!write_to_vm && (!map_data || !map_data->null_mapped)) {
ret = __bio_copy_iov(bio, bio->bi_io_vec, iov, iov_count, 0, 0);
if (ret)
goto cleanup;
diff --git a/fs/block_dev.c b/fs/block_dev.c
index b957717..ac7031f 100644
--- a/fs/block_dev.c
+++ b/fs/block_dev.c
@@ -1005,6 +1005,7 @@
}
lock_kernel();
+ restart:
ret = -ENXIO;
disk = get_gendisk(bdev->bd_dev, &partno);
@@ -1025,6 +1026,19 @@
if (disk->fops->open) {
ret = disk->fops->open(bdev, mode);
+ if (ret == -ERESTARTSYS) {
+ /* Lost a race with 'disk' being
+ * deleted, try again.
+ * See md.c
+ */
+ disk_put_part(bdev->bd_part);
+ bdev->bd_part = NULL;
+ module_put(disk->fops->owner);
+ put_disk(disk);
+ bdev->bd_disk = NULL;
+ mutex_unlock(&bdev->bd_mutex);
+ goto restart;
+ }
if (ret)
goto out_clear;
}
@@ -1220,6 +1234,20 @@
return blkdev_ioctl(bdev, mode, cmd, arg);
}
+/*
+ * Try to release a page associated with block device when the system
+ * is under memory pressure.
+ */
+static int blkdev_releasepage(struct page *page, gfp_t wait)
+{
+ struct super_block *super = BDEV_I(page->mapping->host)->bdev.bd_super;
+
+ if (super && super->s_op->bdev_try_to_free_page)
+ return super->s_op->bdev_try_to_free_page(super, page, wait);
+
+ return try_to_free_buffers(page);
+}
+
static const struct address_space_operations def_blk_aops = {
.readpage = blkdev_readpage,
.writepage = blkdev_writepage,
@@ -1227,6 +1255,7 @@
.write_begin = blkdev_write_begin,
.write_end = blkdev_write_end,
.writepages = generic_writepages,
+ .releasepage = blkdev_releasepage,
.direct_IO = blkdev_direct_IO,
};
diff --git a/fs/btrfs/Makefile b/fs/btrfs/Makefile
new file mode 100644
index 0000000..d2cf5a5
--- /dev/null
+++ b/fs/btrfs/Makefile
@@ -0,0 +1,25 @@
+ifneq ($(KERNELRELEASE),)
+# kbuild part of makefile
+
+obj-$(CONFIG_BTRFS_FS) := btrfs.o
+btrfs-y := super.o ctree.o extent-tree.o print-tree.o root-tree.o dir-item.o \
+ file-item.o inode-item.o inode-map.o disk-io.o \
+ transaction.o inode.o file.o tree-defrag.o \
+ extent_map.o sysfs.o struct-funcs.o xattr.o ordered-data.o \
+ extent_io.o volumes.o async-thread.o ioctl.o locking.o orphan.o \
+ ref-cache.o export.o tree-log.o acl.o free-space-cache.o zlib.o \
+ compression.o
+else
+
+# Normal Makefile
+
+KERNELDIR := /lib/modules/`uname -r`/build
+all:
+ $(MAKE) -C $(KERNELDIR) M=`pwd` CONFIG_BTRFS_FS=m modules
+
+modules_install:
+ $(MAKE) -C $(KERNELDIR) M=`pwd` modules_install
+clean:
+ $(MAKE) -C $(KERNELDIR) M=`pwd` clean
+
+endif
diff --git a/fs/btrfs/acl.c b/fs/btrfs/acl.c
new file mode 100644
index 0000000..1d53b62
--- /dev/null
+++ b/fs/btrfs/acl.c
@@ -0,0 +1,351 @@
+/*
+ * Copyright (C) 2007 Red Hat. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/fs.h>
+#include <linux/string.h>
+#include <linux/xattr.h>
+#include <linux/posix_acl_xattr.h>
+#include <linux/posix_acl.h>
+#include <linux/sched.h>
+
+#include "ctree.h"
+#include "btrfs_inode.h"
+#include "xattr.h"
+
+#ifdef CONFIG_FS_POSIX_ACL
+
+static void btrfs_update_cached_acl(struct inode *inode,
+ struct posix_acl **p_acl,
+ struct posix_acl *acl)
+{
+ spin_lock(&inode->i_lock);
+ if (*p_acl && *p_acl != BTRFS_ACL_NOT_CACHED)
+ posix_acl_release(*p_acl);
+ *p_acl = posix_acl_dup(acl);
+ spin_unlock(&inode->i_lock);
+}
+
+static struct posix_acl *btrfs_get_acl(struct inode *inode, int type)
+{
+ int size;
+ const char *name;
+ char *value = NULL;
+ struct posix_acl *acl = NULL, **p_acl;
+
+ switch (type) {
+ case ACL_TYPE_ACCESS:
+ name = POSIX_ACL_XATTR_ACCESS;
+ p_acl = &BTRFS_I(inode)->i_acl;
+ break;
+ case ACL_TYPE_DEFAULT:
+ name = POSIX_ACL_XATTR_DEFAULT;
+ p_acl = &BTRFS_I(inode)->i_default_acl;
+ break;
+ default:
+ return ERR_PTR(-EINVAL);
+ }
+
+ spin_lock(&inode->i_lock);
+ if (*p_acl != BTRFS_ACL_NOT_CACHED)
+ acl = posix_acl_dup(*p_acl);
+ spin_unlock(&inode->i_lock);
+
+ if (acl)
+ return acl;
+
+
+ size = __btrfs_getxattr(inode, name, "", 0);
+ if (size > 0) {
+ value = kzalloc(size, GFP_NOFS);
+ if (!value)
+ return ERR_PTR(-ENOMEM);
+ size = __btrfs_getxattr(inode, name, value, size);
+ if (size > 0) {
+ acl = posix_acl_from_xattr(value, size);
+ btrfs_update_cached_acl(inode, p_acl, acl);
+ }
+ kfree(value);
+ } else if (size == -ENOENT) {
+ acl = NULL;
+ btrfs_update_cached_acl(inode, p_acl, acl);
+ }
+
+ return acl;
+}
+
+static int btrfs_xattr_get_acl(struct inode *inode, int type,
+ void *value, size_t size)
+{
+ struct posix_acl *acl;
+ int ret = 0;
+
+ acl = btrfs_get_acl(inode, type);
+
+ if (IS_ERR(acl))
+ return PTR_ERR(acl);
+ if (acl == NULL)
+ return -ENODATA;
+ ret = posix_acl_to_xattr(acl, value, size);
+ posix_acl_release(acl);
+
+ return ret;
+}
+
+/*
+ * Needs to be called with fs_mutex held
+ */
+static int btrfs_set_acl(struct inode *inode, struct posix_acl *acl, int type)
+{
+ int ret, size = 0;
+ const char *name;
+ struct posix_acl **p_acl;
+ char *value = NULL;
+ mode_t mode;
+
+ if (acl) {
+ ret = posix_acl_valid(acl);
+ if (ret < 0)
+ return ret;
+ ret = 0;
+ }
+
+ switch (type) {
+ case ACL_TYPE_ACCESS:
+ mode = inode->i_mode;
+ ret = posix_acl_equiv_mode(acl, &mode);
+ if (ret < 0)
+ return ret;
+ ret = 0;
+ inode->i_mode = mode;
+ name = POSIX_ACL_XATTR_ACCESS;
+ p_acl = &BTRFS_I(inode)->i_acl;
+ break;
+ case ACL_TYPE_DEFAULT:
+ if (!S_ISDIR(inode->i_mode))
+ return acl ? -EINVAL : 0;
+ name = POSIX_ACL_XATTR_DEFAULT;
+ p_acl = &BTRFS_I(inode)->i_default_acl;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (acl) {
+ size = posix_acl_xattr_size(acl->a_count);
+ value = kmalloc(size, GFP_NOFS);
+ if (!value) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ ret = posix_acl_to_xattr(acl, value, size);
+ if (ret < 0)
+ goto out;
+ }
+
+ ret = __btrfs_setxattr(inode, name, value, size, 0);
+
+out:
+ kfree(value);
+
+ if (!ret)
+ btrfs_update_cached_acl(inode, p_acl, acl);
+
+ return ret;
+}
+
+static int btrfs_xattr_set_acl(struct inode *inode, int type,
+ const void *value, size_t size)
+{
+ int ret = 0;
+ struct posix_acl *acl = NULL;
+
+ if (value) {
+ acl = posix_acl_from_xattr(value, size);
+ if (acl == NULL) {
+ value = NULL;
+ size = 0;
+ } else if (IS_ERR(acl)) {
+ return PTR_ERR(acl);
+ }
+ }
+
+ ret = btrfs_set_acl(inode, acl, type);
+
+ posix_acl_release(acl);
+
+ return ret;
+}
+
+
+static int btrfs_xattr_acl_access_get(struct inode *inode, const char *name,
+ void *value, size_t size)
+{
+ return btrfs_xattr_get_acl(inode, ACL_TYPE_ACCESS, value, size);
+}
+
+static int btrfs_xattr_acl_access_set(struct inode *inode, const char *name,
+ const void *value, size_t size, int flags)
+{
+ return btrfs_xattr_set_acl(inode, ACL_TYPE_ACCESS, value, size);
+}
+
+static int btrfs_xattr_acl_default_get(struct inode *inode, const char *name,
+ void *value, size_t size)
+{
+ return btrfs_xattr_get_acl(inode, ACL_TYPE_DEFAULT, value, size);
+}
+
+static int btrfs_xattr_acl_default_set(struct inode *inode, const char *name,
+ const void *value, size_t size, int flags)
+{
+ return btrfs_xattr_set_acl(inode, ACL_TYPE_DEFAULT, value, size);
+}
+
+int btrfs_check_acl(struct inode *inode, int mask)
+{
+ struct posix_acl *acl;
+ int error = -EAGAIN;
+
+ acl = btrfs_get_acl(inode, ACL_TYPE_ACCESS);
+
+ if (IS_ERR(acl))
+ return PTR_ERR(acl);
+ if (acl) {
+ error = posix_acl_permission(inode, acl, mask);
+ posix_acl_release(acl);
+ }
+
+ return error;
+}
+
+/*
+ * btrfs_init_acl is already generally called under fs_mutex, so the locking
+ * stuff has been fixed to work with that. If the locking stuff changes, we
+ * need to re-evaluate the acl locking stuff.
+ */
+int btrfs_init_acl(struct inode *inode, struct inode *dir)
+{
+ struct posix_acl *acl = NULL;
+ int ret = 0;
+
+ /* this happens with subvols */
+ if (!dir)
+ return 0;
+
+ if (!S_ISLNK(inode->i_mode)) {
+ if (IS_POSIXACL(dir)) {
+ acl = btrfs_get_acl(dir, ACL_TYPE_DEFAULT);
+ if (IS_ERR(acl))
+ return PTR_ERR(acl);
+ }
+
+ if (!acl)
+ inode->i_mode &= ~current->fs->umask;
+ }
+
+ if (IS_POSIXACL(dir) && acl) {
+ struct posix_acl *clone;
+ mode_t mode;
+
+ if (S_ISDIR(inode->i_mode)) {
+ ret = btrfs_set_acl(inode, acl, ACL_TYPE_DEFAULT);
+ if (ret)
+ goto failed;
+ }
+ clone = posix_acl_clone(acl, GFP_NOFS);
+ ret = -ENOMEM;
+ if (!clone)
+ goto failed;
+
+ mode = inode->i_mode;
+ ret = posix_acl_create_masq(clone, &mode);
+ if (ret >= 0) {
+ inode->i_mode = mode;
+ if (ret > 0) {
+ /* we need an acl */
+ ret = btrfs_set_acl(inode, clone,
+ ACL_TYPE_ACCESS);
+ }
+ }
+ }
+failed:
+ posix_acl_release(acl);
+
+ return ret;
+}
+
+int btrfs_acl_chmod(struct inode *inode)
+{
+ struct posix_acl *acl, *clone;
+ int ret = 0;
+
+ if (S_ISLNK(inode->i_mode))
+ return -EOPNOTSUPP;
+
+ if (!IS_POSIXACL(inode))
+ return 0;
+
+ acl = btrfs_get_acl(inode, ACL_TYPE_ACCESS);
+ if (IS_ERR(acl) || !acl)
+ return PTR_ERR(acl);
+
+ clone = posix_acl_clone(acl, GFP_KERNEL);
+ posix_acl_release(acl);
+ if (!clone)
+ return -ENOMEM;
+
+ ret = posix_acl_chmod_masq(clone, inode->i_mode);
+ if (!ret)
+ ret = btrfs_set_acl(inode, clone, ACL_TYPE_ACCESS);
+
+ posix_acl_release(clone);
+
+ return ret;
+}
+
+struct xattr_handler btrfs_xattr_acl_default_handler = {
+ .prefix = POSIX_ACL_XATTR_DEFAULT,
+ .get = btrfs_xattr_acl_default_get,
+ .set = btrfs_xattr_acl_default_set,
+};
+
+struct xattr_handler btrfs_xattr_acl_access_handler = {
+ .prefix = POSIX_ACL_XATTR_ACCESS,
+ .get = btrfs_xattr_acl_access_get,
+ .set = btrfs_xattr_acl_access_set,
+};
+
+#else /* CONFIG_FS_POSIX_ACL */
+
+int btrfs_acl_chmod(struct inode *inode)
+{
+ return 0;
+}
+
+int btrfs_init_acl(struct inode *inode, struct inode *dir)
+{
+ return 0;
+}
+
+int btrfs_check_acl(struct inode *inode, int mask)
+{
+ return 0;
+}
+
+#endif /* CONFIG_FS_POSIX_ACL */
diff --git a/fs/btrfs/async-thread.c b/fs/btrfs/async-thread.c
new file mode 100644
index 0000000..8e2fec0
--- /dev/null
+++ b/fs/btrfs/async-thread.c
@@ -0,0 +1,419 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/version.h>
+#include <linux/kthread.h>
+#include <linux/list.h>
+#include <linux/spinlock.h>
+# include <linux/freezer.h>
+#include "async-thread.h"
+
+#define WORK_QUEUED_BIT 0
+#define WORK_DONE_BIT 1
+#define WORK_ORDER_DONE_BIT 2
+
+/*
+ * container for the kthread task pointer and the list of pending work
+ * One of these is allocated per thread.
+ */
+struct btrfs_worker_thread {
+ /* pool we belong to */
+ struct btrfs_workers *workers;
+
+ /* list of struct btrfs_work that are waiting for service */
+ struct list_head pending;
+
+ /* list of worker threads from struct btrfs_workers */
+ struct list_head worker_list;
+
+ /* kthread */
+ struct task_struct *task;
+
+ /* number of things on the pending list */
+ atomic_t num_pending;
+
+ unsigned long sequence;
+
+ /* protects the pending list. */
+ spinlock_t lock;
+
+ /* set to non-zero when this thread is already awake and kicking */
+ int working;
+
+ /* are we currently idle */
+ int idle;
+};
+
+/*
+ * helper function to move a thread onto the idle list after it
+ * has finished some requests.
+ */
+static void check_idle_worker(struct btrfs_worker_thread *worker)
+{
+ if (!worker->idle && atomic_read(&worker->num_pending) <
+ worker->workers->idle_thresh / 2) {
+ unsigned long flags;
+ spin_lock_irqsave(&worker->workers->lock, flags);
+ worker->idle = 1;
+ list_move(&worker->worker_list, &worker->workers->idle_list);
+ spin_unlock_irqrestore(&worker->workers->lock, flags);
+ }
+}
+
+/*
+ * helper function to move a thread off the idle list after new
+ * pending work is added.
+ */
+static void check_busy_worker(struct btrfs_worker_thread *worker)
+{
+ if (worker->idle && atomic_read(&worker->num_pending) >=
+ worker->workers->idle_thresh) {
+ unsigned long flags;
+ spin_lock_irqsave(&worker->workers->lock, flags);
+ worker->idle = 0;
+ list_move_tail(&worker->worker_list,
+ &worker->workers->worker_list);
+ spin_unlock_irqrestore(&worker->workers->lock, flags);
+ }
+}
+
+static noinline int run_ordered_completions(struct btrfs_workers *workers,
+ struct btrfs_work *work)
+{
+ unsigned long flags;
+
+ if (!workers->ordered)
+ return 0;
+
+ set_bit(WORK_DONE_BIT, &work->flags);
+
+ spin_lock_irqsave(&workers->lock, flags);
+
+ while (!list_empty(&workers->order_list)) {
+ work = list_entry(workers->order_list.next,
+ struct btrfs_work, order_list);
+
+ if (!test_bit(WORK_DONE_BIT, &work->flags))
+ break;
+
+ /* we are going to call the ordered done function, but
+ * we leave the work item on the list as a barrier so
+ * that later work items that are done don't have their
+ * functions called before this one returns
+ */
+ if (test_and_set_bit(WORK_ORDER_DONE_BIT, &work->flags))
+ break;
+
+ spin_unlock_irqrestore(&workers->lock, flags);
+
+ work->ordered_func(work);
+
+ /* now take the lock again and call the freeing code */
+ spin_lock_irqsave(&workers->lock, flags);
+ list_del(&work->order_list);
+ work->ordered_free(work);
+ }
+
+ spin_unlock_irqrestore(&workers->lock, flags);
+ return 0;
+}
+
+/*
+ * main loop for servicing work items
+ */
+static int worker_loop(void *arg)
+{
+ struct btrfs_worker_thread *worker = arg;
+ struct list_head *cur;
+ struct btrfs_work *work;
+ do {
+ spin_lock_irq(&worker->lock);
+ while (!list_empty(&worker->pending)) {
+ cur = worker->pending.next;
+ work = list_entry(cur, struct btrfs_work, list);
+ list_del(&work->list);
+ clear_bit(WORK_QUEUED_BIT, &work->flags);
+
+ work->worker = worker;
+ spin_unlock_irq(&worker->lock);
+
+ work->func(work);
+
+ atomic_dec(&worker->num_pending);
+ /*
+ * unless this is an ordered work queue,
+ * 'work' was probably freed by func above.
+ */
+ run_ordered_completions(worker->workers, work);
+
+ spin_lock_irq(&worker->lock);
+ check_idle_worker(worker);
+
+ }
+ worker->working = 0;
+ if (freezing(current)) {
+ refrigerator();
+ } else {
+ set_current_state(TASK_INTERRUPTIBLE);
+ spin_unlock_irq(&worker->lock);
+ if (!kthread_should_stop())
+ schedule();
+ __set_current_state(TASK_RUNNING);
+ }
+ } while (!kthread_should_stop());
+ return 0;
+}
+
+/*
+ * this will wait for all the worker threads to shutdown
+ */
+int btrfs_stop_workers(struct btrfs_workers *workers)
+{
+ struct list_head *cur;
+ struct btrfs_worker_thread *worker;
+
+ list_splice_init(&workers->idle_list, &workers->worker_list);
+ while (!list_empty(&workers->worker_list)) {
+ cur = workers->worker_list.next;
+ worker = list_entry(cur, struct btrfs_worker_thread,
+ worker_list);
+ kthread_stop(worker->task);
+ list_del(&worker->worker_list);
+ kfree(worker);
+ }
+ return 0;
+}
+
+/*
+ * simple init on struct btrfs_workers
+ */
+void btrfs_init_workers(struct btrfs_workers *workers, char *name, int max)
+{
+ workers->num_workers = 0;
+ INIT_LIST_HEAD(&workers->worker_list);
+ INIT_LIST_HEAD(&workers->idle_list);
+ INIT_LIST_HEAD(&workers->order_list);
+ spin_lock_init(&workers->lock);
+ workers->max_workers = max;
+ workers->idle_thresh = 32;
+ workers->name = name;
+ workers->ordered = 0;
+}
+
+/*
+ * starts new worker threads. This does not enforce the max worker
+ * count in case you need to temporarily go past it.
+ */
+int btrfs_start_workers(struct btrfs_workers *workers, int num_workers)
+{
+ struct btrfs_worker_thread *worker;
+ int ret = 0;
+ int i;
+
+ for (i = 0; i < num_workers; i++) {
+ worker = kzalloc(sizeof(*worker), GFP_NOFS);
+ if (!worker) {
+ ret = -ENOMEM;
+ goto fail;
+ }
+
+ INIT_LIST_HEAD(&worker->pending);
+ INIT_LIST_HEAD(&worker->worker_list);
+ spin_lock_init(&worker->lock);
+ atomic_set(&worker->num_pending, 0);
+ worker->task = kthread_run(worker_loop, worker,
+ "btrfs-%s-%d", workers->name,
+ workers->num_workers + i);
+ worker->workers = workers;
+ if (IS_ERR(worker->task)) {
+ kfree(worker);
+ ret = PTR_ERR(worker->task);
+ goto fail;
+ }
+
+ spin_lock_irq(&workers->lock);
+ list_add_tail(&worker->worker_list, &workers->idle_list);
+ worker->idle = 1;
+ workers->num_workers++;
+ spin_unlock_irq(&workers->lock);
+ }
+ return 0;
+fail:
+ btrfs_stop_workers(workers);
+ return ret;
+}
+
+/*
+ * run through the list and find a worker thread that doesn't have a lot
+ * to do right now. This can return null if we aren't yet at the thread
+ * count limit and all of the threads are busy.
+ */
+static struct btrfs_worker_thread *next_worker(struct btrfs_workers *workers)
+{
+ struct btrfs_worker_thread *worker;
+ struct list_head *next;
+ int enforce_min = workers->num_workers < workers->max_workers;
+
+ /*
+ * if we find an idle thread, don't move it to the end of the
+ * idle list. This improves the chance that the next submission
+ * will reuse the same thread, and maybe catch it while it is still
+ * working
+ */
+ if (!list_empty(&workers->idle_list)) {
+ next = workers->idle_list.next;
+ worker = list_entry(next, struct btrfs_worker_thread,
+ worker_list);
+ return worker;
+ }
+ if (enforce_min || list_empty(&workers->worker_list))
+ return NULL;
+
+ /*
+ * if we pick a busy task, move the task to the end of the list.
+ * hopefully this will keep things somewhat evenly balanced.
+ * Do the move in batches based on the sequence number. This groups
+ * requests submitted at roughly the same time onto the same worker.
+ */
+ next = workers->worker_list.next;
+ worker = list_entry(next, struct btrfs_worker_thread, worker_list);
+ atomic_inc(&worker->num_pending);
+ worker->sequence++;
+
+ if (worker->sequence % workers->idle_thresh == 0)
+ list_move_tail(next, &workers->worker_list);
+ return worker;
+}
+
+/*
+ * selects a worker thread to take the next job. This will either find
+ * an idle worker, start a new worker up to the max count, or just return
+ * one of the existing busy workers.
+ */
+static struct btrfs_worker_thread *find_worker(struct btrfs_workers *workers)
+{
+ struct btrfs_worker_thread *worker;
+ unsigned long flags;
+
+again:
+ spin_lock_irqsave(&workers->lock, flags);
+ worker = next_worker(workers);
+ spin_unlock_irqrestore(&workers->lock, flags);
+
+ if (!worker) {
+ spin_lock_irqsave(&workers->lock, flags);
+ if (workers->num_workers >= workers->max_workers) {
+ struct list_head *fallback = NULL;
+ /*
+ * we have failed to find any workers, just
+ * return the force one
+ */
+ if (!list_empty(&workers->worker_list))
+ fallback = workers->worker_list.next;
+ if (!list_empty(&workers->idle_list))
+ fallback = workers->idle_list.next;
+ BUG_ON(!fallback);
+ worker = list_entry(fallback,
+ struct btrfs_worker_thread, worker_list);
+ spin_unlock_irqrestore(&workers->lock, flags);
+ } else {
+ spin_unlock_irqrestore(&workers->lock, flags);
+ /* we're below the limit, start another worker */
+ btrfs_start_workers(workers, 1);
+ goto again;
+ }
+ }
+ return worker;
+}
+
+/*
+ * btrfs_requeue_work just puts the work item back on the tail of the list
+ * it was taken from. It is intended for use with long running work functions
+ * that make some progress and want to give the cpu up for others.
+ */
+int btrfs_requeue_work(struct btrfs_work *work)
+{
+ struct btrfs_worker_thread *worker = work->worker;
+ unsigned long flags;
+
+ if (test_and_set_bit(WORK_QUEUED_BIT, &work->flags))
+ goto out;
+
+ spin_lock_irqsave(&worker->lock, flags);
+ atomic_inc(&worker->num_pending);
+ list_add_tail(&work->list, &worker->pending);
+
+ /* by definition we're busy, take ourselves off the idle
+ * list
+ */
+ if (worker->idle) {
+ spin_lock_irqsave(&worker->workers->lock, flags);
+ worker->idle = 0;
+ list_move_tail(&worker->worker_list,
+ &worker->workers->worker_list);
+ spin_unlock_irqrestore(&worker->workers->lock, flags);
+ }
+
+ spin_unlock_irqrestore(&worker->lock, flags);
+
+out:
+ return 0;
+}
+
+/*
+ * places a struct btrfs_work into the pending queue of one of the kthreads
+ */
+int btrfs_queue_worker(struct btrfs_workers *workers, struct btrfs_work *work)
+{
+ struct btrfs_worker_thread *worker;
+ unsigned long flags;
+ int wake = 0;
+
+ /* don't requeue something already on a list */
+ if (test_and_set_bit(WORK_QUEUED_BIT, &work->flags))
+ goto out;
+
+ worker = find_worker(workers);
+ if (workers->ordered) {
+ spin_lock_irqsave(&workers->lock, flags);
+ list_add_tail(&work->order_list, &workers->order_list);
+ spin_unlock_irqrestore(&workers->lock, flags);
+ } else {
+ INIT_LIST_HEAD(&work->order_list);
+ }
+
+ spin_lock_irqsave(&worker->lock, flags);
+ atomic_inc(&worker->num_pending);
+ check_busy_worker(worker);
+ list_add_tail(&work->list, &worker->pending);
+
+ /*
+ * avoid calling into wake_up_process if this thread has already
+ * been kicked
+ */
+ if (!worker->working)
+ wake = 1;
+ worker->working = 1;
+
+ spin_unlock_irqrestore(&worker->lock, flags);
+
+ if (wake)
+ wake_up_process(worker->task);
+out:
+ return 0;
+}
diff --git a/fs/btrfs/async-thread.h b/fs/btrfs/async-thread.h
new file mode 100644
index 0000000..31be4ed
--- /dev/null
+++ b/fs/btrfs/async-thread.h
@@ -0,0 +1,101 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __BTRFS_ASYNC_THREAD_
+#define __BTRFS_ASYNC_THREAD_
+
+struct btrfs_worker_thread;
+
+/*
+ * This is similar to a workqueue, but it is meant to spread the operations
+ * across all available cpus instead of just the CPU that was used to
+ * queue the work. There is also some batching introduced to try and
+ * cut down on context switches.
+ *
+ * By default threads are added on demand up to 2 * the number of cpus.
+ * Changing struct btrfs_workers->max_workers is one way to prevent
+ * demand creation of kthreads.
+ *
+ * the basic model of these worker threads is to embed a btrfs_work
+ * structure in your own data struct, and use container_of in a
+ * work function to get back to your data struct.
+ */
+struct btrfs_work {
+ /*
+ * func should be set to the function you want called
+ * your work struct is passed as the only arg
+ *
+ * ordered_func must be set for work sent to an ordered work queue,
+ * and it is called to complete a given work item in the same
+ * order they were sent to the queue.
+ */
+ void (*func)(struct btrfs_work *work);
+ void (*ordered_func)(struct btrfs_work *work);
+ void (*ordered_free)(struct btrfs_work *work);
+
+ /*
+ * flags should be set to zero. It is used to make sure the
+ * struct is only inserted once into the list.
+ */
+ unsigned long flags;
+
+ /* don't touch these */
+ struct btrfs_worker_thread *worker;
+ struct list_head list;
+ struct list_head order_list;
+};
+
+struct btrfs_workers {
+ /* current number of running workers */
+ int num_workers;
+
+ /* max number of workers allowed. changed by btrfs_start_workers */
+ int max_workers;
+
+ /* once a worker has this many requests or fewer, it is idle */
+ int idle_thresh;
+
+ /* force completions in the order they were queued */
+ int ordered;
+
+ /* list with all the work threads. The workers on the idle thread
+ * may be actively servicing jobs, but they haven't yet hit the
+ * idle thresh limit above.
+ */
+ struct list_head worker_list;
+ struct list_head idle_list;
+
+ /*
+ * when operating in ordered mode, this maintains the list
+ * of work items waiting for completion
+ */
+ struct list_head order_list;
+
+ /* lock for finding the next worker thread to queue on */
+ spinlock_t lock;
+
+ /* extra name for this worker, used for current->name */
+ char *name;
+};
+
+int btrfs_queue_worker(struct btrfs_workers *workers, struct btrfs_work *work);
+int btrfs_start_workers(struct btrfs_workers *workers, int num_workers);
+int btrfs_stop_workers(struct btrfs_workers *workers);
+void btrfs_init_workers(struct btrfs_workers *workers, char *name, int max);
+int btrfs_requeue_work(struct btrfs_work *work);
+#endif
diff --git a/fs/btrfs/btrfs_inode.h b/fs/btrfs/btrfs_inode.h
new file mode 100644
index 0000000..a8c9693
--- /dev/null
+++ b/fs/btrfs/btrfs_inode.h
@@ -0,0 +1,131 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __BTRFS_I__
+#define __BTRFS_I__
+
+#include "extent_map.h"
+#include "extent_io.h"
+#include "ordered-data.h"
+
+/* in memory btrfs inode */
+struct btrfs_inode {
+ /* which subvolume this inode belongs to */
+ struct btrfs_root *root;
+
+ /* key used to find this inode on disk. This is used by the code
+ * to read in roots of subvolumes
+ */
+ struct btrfs_key location;
+
+ /* the extent_tree has caches of all the extent mappings to disk */
+ struct extent_map_tree extent_tree;
+
+ /* the io_tree does range state (DIRTY, LOCKED etc) */
+ struct extent_io_tree io_tree;
+
+ /* special utility tree used to record which mirrors have already been
+ * tried when checksums fail for a given block
+ */
+ struct extent_io_tree io_failure_tree;
+
+ /* held while inesrting or deleting extents from files */
+ struct mutex extent_mutex;
+
+ /* held while logging the inode in tree-log.c */
+ struct mutex log_mutex;
+
+ /* used to order data wrt metadata */
+ struct btrfs_ordered_inode_tree ordered_tree;
+
+ /* standard acl pointers */
+ struct posix_acl *i_acl;
+ struct posix_acl *i_default_acl;
+
+ /* for keeping track of orphaned inodes */
+ struct list_head i_orphan;
+
+ /* list of all the delalloc inodes in the FS. There are times we need
+ * to write all the delalloc pages to disk, and this list is used
+ * to walk them all.
+ */
+ struct list_head delalloc_inodes;
+
+ /* full 64 bit generation number, struct vfs_inode doesn't have a big
+ * enough field for this.
+ */
+ u64 generation;
+
+ /* sequence number for NFS changes */
+ u64 sequence;
+
+ /*
+ * transid of the trans_handle that last modified this inode
+ */
+ u64 last_trans;
+ /*
+ * transid that last logged this inode
+ */
+ u64 logged_trans;
+
+ /*
+ * trans that last made a change that should be fully fsync'd. This
+ * gets reset to zero each time the inode is logged
+ */
+ u64 log_dirty_trans;
+
+ /* total number of bytes pending delalloc, used by stat to calc the
+ * real block usage of the file
+ */
+ u64 delalloc_bytes;
+
+ /*
+ * the size of the file stored in the metadata on disk. data=ordered
+ * means the in-memory i_size might be larger than the size on disk
+ * because not all the blocks are written yet.
+ */
+ u64 disk_i_size;
+
+ /* flags field from the on disk inode */
+ u32 flags;
+
+ /*
+ * if this is a directory then index_cnt is the counter for the index
+ * number for new files that are created
+ */
+ u64 index_cnt;
+
+ /* the start of block group preferred for allocations. */
+ u64 block_group;
+
+ struct inode vfs_inode;
+};
+
+static inline struct btrfs_inode *BTRFS_I(struct inode *inode)
+{
+ return container_of(inode, struct btrfs_inode, vfs_inode);
+}
+
+static inline void btrfs_i_size_write(struct inode *inode, u64 size)
+{
+ inode->i_size = size;
+ BTRFS_I(inode)->disk_i_size = size;
+}
+
+
+#endif
diff --git a/fs/btrfs/compat.h b/fs/btrfs/compat.h
new file mode 100644
index 0000000..7c4503e
--- /dev/null
+++ b/fs/btrfs/compat.h
@@ -0,0 +1,7 @@
+#ifndef _COMPAT_H_
+#define _COMPAT_H_
+
+#define btrfs_drop_nlink(inode) drop_nlink(inode)
+#define btrfs_inc_nlink(inode) inc_nlink(inode)
+
+#endif /* _COMPAT_H_ */
diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c
new file mode 100644
index 0000000..ee848d8
--- /dev/null
+++ b/fs/btrfs/compression.c
@@ -0,0 +1,709 @@
+/*
+ * Copyright (C) 2008 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/kernel.h>
+#include <linux/bio.h>
+#include <linux/buffer_head.h>
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <linux/pagemap.h>
+#include <linux/highmem.h>
+#include <linux/time.h>
+#include <linux/init.h>
+#include <linux/string.h>
+#include <linux/smp_lock.h>
+#include <linux/backing-dev.h>
+#include <linux/mpage.h>
+#include <linux/swap.h>
+#include <linux/writeback.h>
+#include <linux/bit_spinlock.h>
+#include <linux/version.h>
+#include <linux/pagevec.h>
+#include "compat.h"
+#include "ctree.h"
+#include "disk-io.h"
+#include "transaction.h"
+#include "btrfs_inode.h"
+#include "volumes.h"
+#include "ordered-data.h"
+#include "compression.h"
+#include "extent_io.h"
+#include "extent_map.h"
+
+struct compressed_bio {
+ /* number of bios pending for this compressed extent */
+ atomic_t pending_bios;
+
+ /* the pages with the compressed data on them */
+ struct page **compressed_pages;
+
+ /* inode that owns this data */
+ struct inode *inode;
+
+ /* starting offset in the inode for our pages */
+ u64 start;
+
+ /* number of bytes in the inode we're working on */
+ unsigned long len;
+
+ /* number of bytes on disk */
+ unsigned long compressed_len;
+
+ /* number of compressed pages in the array */
+ unsigned long nr_pages;
+
+ /* IO errors */
+ int errors;
+ int mirror_num;
+
+ /* for reads, this is the bio we are copying the data into */
+ struct bio *orig_bio;
+
+ /*
+ * the start of a variable length array of checksums only
+ * used by reads
+ */
+ u32 sums;
+};
+
+static inline int compressed_bio_size(struct btrfs_root *root,
+ unsigned long disk_size)
+{
+ u16 csum_size = btrfs_super_csum_size(&root->fs_info->super_copy);
+ return sizeof(struct compressed_bio) +
+ ((disk_size + root->sectorsize - 1) / root->sectorsize) *
+ csum_size;
+}
+
+static struct bio *compressed_bio_alloc(struct block_device *bdev,
+ u64 first_byte, gfp_t gfp_flags)
+{
+ struct bio *bio;
+ int nr_vecs;
+
+ nr_vecs = bio_get_nr_vecs(bdev);
+ bio = bio_alloc(gfp_flags, nr_vecs);
+
+ if (bio == NULL && (current->flags & PF_MEMALLOC)) {
+ while (!bio && (nr_vecs /= 2))
+ bio = bio_alloc(gfp_flags, nr_vecs);
+ }
+
+ if (bio) {
+ bio->bi_size = 0;
+ bio->bi_bdev = bdev;
+ bio->bi_sector = first_byte >> 9;
+ }
+ return bio;
+}
+
+static int check_compressed_csum(struct inode *inode,
+ struct compressed_bio *cb,
+ u64 disk_start)
+{
+ int ret;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct page *page;
+ unsigned long i;
+ char *kaddr;
+ u32 csum;
+ u32 *cb_sum = &cb->sums;
+
+ if (btrfs_test_flag(inode, NODATASUM))
+ return 0;
+
+ for (i = 0; i < cb->nr_pages; i++) {
+ page = cb->compressed_pages[i];
+ csum = ~(u32)0;
+
+ kaddr = kmap_atomic(page, KM_USER0);
+ csum = btrfs_csum_data(root, kaddr, csum, PAGE_CACHE_SIZE);
+ btrfs_csum_final(csum, (char *)&csum);
+ kunmap_atomic(kaddr, KM_USER0);
+
+ if (csum != *cb_sum) {
+ printk(KERN_INFO "btrfs csum failed ino %lu "
+ "extent %llu csum %u "
+ "wanted %u mirror %d\n", inode->i_ino,
+ (unsigned long long)disk_start,
+ csum, *cb_sum, cb->mirror_num);
+ ret = -EIO;
+ goto fail;
+ }
+ cb_sum++;
+
+ }
+ ret = 0;
+fail:
+ return ret;
+}
+
+/* when we finish reading compressed pages from the disk, we
+ * decompress them and then run the bio end_io routines on the
+ * decompressed pages (in the inode address space).
+ *
+ * This allows the checksumming and other IO error handling routines
+ * to work normally
+ *
+ * The compressed pages are freed here, and it must be run
+ * in process context
+ */
+static void end_compressed_bio_read(struct bio *bio, int err)
+{
+ struct extent_io_tree *tree;
+ struct compressed_bio *cb = bio->bi_private;
+ struct inode *inode;
+ struct page *page;
+ unsigned long index;
+ int ret;
+
+ if (err)
+ cb->errors = 1;
+
+ /* if there are more bios still pending for this compressed
+ * extent, just exit
+ */
+ if (!atomic_dec_and_test(&cb->pending_bios))
+ goto out;
+
+ inode = cb->inode;
+ ret = check_compressed_csum(inode, cb, (u64)bio->bi_sector << 9);
+ if (ret)
+ goto csum_failed;
+
+ /* ok, we're the last bio for this extent, lets start
+ * the decompression.
+ */
+ tree = &BTRFS_I(inode)->io_tree;
+ ret = btrfs_zlib_decompress_biovec(cb->compressed_pages,
+ cb->start,
+ cb->orig_bio->bi_io_vec,
+ cb->orig_bio->bi_vcnt,
+ cb->compressed_len);
+csum_failed:
+ if (ret)
+ cb->errors = 1;
+
+ /* release the compressed pages */
+ index = 0;
+ for (index = 0; index < cb->nr_pages; index++) {
+ page = cb->compressed_pages[index];
+ page->mapping = NULL;
+ page_cache_release(page);
+ }
+
+ /* do io completion on the original bio */
+ if (cb->errors) {
+ bio_io_error(cb->orig_bio);
+ } else {
+ int bio_index = 0;
+ struct bio_vec *bvec = cb->orig_bio->bi_io_vec;
+
+ /*
+ * we have verified the checksum already, set page
+ * checked so the end_io handlers know about it
+ */
+ while (bio_index < cb->orig_bio->bi_vcnt) {
+ SetPageChecked(bvec->bv_page);
+ bvec++;
+ bio_index++;
+ }
+ bio_endio(cb->orig_bio, 0);
+ }
+
+ /* finally free the cb struct */
+ kfree(cb->compressed_pages);
+ kfree(cb);
+out:
+ bio_put(bio);
+}
+
+/*
+ * Clear the writeback bits on all of the file
+ * pages for a compressed write
+ */
+static noinline int end_compressed_writeback(struct inode *inode, u64 start,
+ unsigned long ram_size)
+{
+ unsigned long index = start >> PAGE_CACHE_SHIFT;
+ unsigned long end_index = (start + ram_size - 1) >> PAGE_CACHE_SHIFT;
+ struct page *pages[16];
+ unsigned long nr_pages = end_index - index + 1;
+ int i;
+ int ret;
+
+ while (nr_pages > 0) {
+ ret = find_get_pages_contig(inode->i_mapping, index,
+ min_t(unsigned long,
+ nr_pages, ARRAY_SIZE(pages)), pages);
+ if (ret == 0) {
+ nr_pages -= 1;
+ index += 1;
+ continue;
+ }
+ for (i = 0; i < ret; i++) {
+ end_page_writeback(pages[i]);
+ page_cache_release(pages[i]);
+ }
+ nr_pages -= ret;
+ index += ret;
+ }
+ /* the inode may be gone now */
+ return 0;
+}
+
+/*
+ * do the cleanup once all the compressed pages hit the disk.
+ * This will clear writeback on the file pages and free the compressed
+ * pages.
+ *
+ * This also calls the writeback end hooks for the file pages so that
+ * metadata and checksums can be updated in the file.
+ */
+static void end_compressed_bio_write(struct bio *bio, int err)
+{
+ struct extent_io_tree *tree;
+ struct compressed_bio *cb = bio->bi_private;
+ struct inode *inode;
+ struct page *page;
+ unsigned long index;
+
+ if (err)
+ cb->errors = 1;
+
+ /* if there are more bios still pending for this compressed
+ * extent, just exit
+ */
+ if (!atomic_dec_and_test(&cb->pending_bios))
+ goto out;
+
+ /* ok, we're the last bio for this extent, step one is to
+ * call back into the FS and do all the end_io operations
+ */
+ inode = cb->inode;
+ tree = &BTRFS_I(inode)->io_tree;
+ cb->compressed_pages[0]->mapping = cb->inode->i_mapping;
+ tree->ops->writepage_end_io_hook(cb->compressed_pages[0],
+ cb->start,
+ cb->start + cb->len - 1,
+ NULL, 1);
+ cb->compressed_pages[0]->mapping = NULL;
+
+ end_compressed_writeback(inode, cb->start, cb->len);
+ /* note, our inode could be gone now */
+
+ /*
+ * release the compressed pages, these came from alloc_page and
+ * are not attached to the inode at all
+ */
+ index = 0;
+ for (index = 0; index < cb->nr_pages; index++) {
+ page = cb->compressed_pages[index];
+ page->mapping = NULL;
+ page_cache_release(page);
+ }
+
+ /* finally free the cb struct */
+ kfree(cb->compressed_pages);
+ kfree(cb);
+out:
+ bio_put(bio);
+}
+
+/*
+ * worker function to build and submit bios for previously compressed pages.
+ * The corresponding pages in the inode should be marked for writeback
+ * and the compressed pages should have a reference on them for dropping
+ * when the IO is complete.
+ *
+ * This also checksums the file bytes and gets things ready for
+ * the end io hooks.
+ */
+int btrfs_submit_compressed_write(struct inode *inode, u64 start,
+ unsigned long len, u64 disk_start,
+ unsigned long compressed_len,
+ struct page **compressed_pages,
+ unsigned long nr_pages)
+{
+ struct bio *bio = NULL;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct compressed_bio *cb;
+ unsigned long bytes_left;
+ struct extent_io_tree *io_tree = &BTRFS_I(inode)->io_tree;
+ int page_index = 0;
+ struct page *page;
+ u64 first_byte = disk_start;
+ struct block_device *bdev;
+ int ret;
+
+ WARN_ON(start & ((u64)PAGE_CACHE_SIZE - 1));
+ cb = kmalloc(compressed_bio_size(root, compressed_len), GFP_NOFS);
+ atomic_set(&cb->pending_bios, 0);
+ cb->errors = 0;
+ cb->inode = inode;
+ cb->start = start;
+ cb->len = len;
+ cb->mirror_num = 0;
+ cb->compressed_pages = compressed_pages;
+ cb->compressed_len = compressed_len;
+ cb->orig_bio = NULL;
+ cb->nr_pages = nr_pages;
+
+ bdev = BTRFS_I(inode)->root->fs_info->fs_devices->latest_bdev;
+
+ bio = compressed_bio_alloc(bdev, first_byte, GFP_NOFS);
+ bio->bi_private = cb;
+ bio->bi_end_io = end_compressed_bio_write;
+ atomic_inc(&cb->pending_bios);
+
+ /* create and submit bios for the compressed pages */
+ bytes_left = compressed_len;
+ for (page_index = 0; page_index < cb->nr_pages; page_index++) {
+ page = compressed_pages[page_index];
+ page->mapping = inode->i_mapping;
+ if (bio->bi_size)
+ ret = io_tree->ops->merge_bio_hook(page, 0,
+ PAGE_CACHE_SIZE,
+ bio, 0);
+ else
+ ret = 0;
+
+ page->mapping = NULL;
+ if (ret || bio_add_page(bio, page, PAGE_CACHE_SIZE, 0) <
+ PAGE_CACHE_SIZE) {
+ bio_get(bio);
+
+ /*
+ * inc the count before we submit the bio so
+ * we know the end IO handler won't happen before
+ * we inc the count. Otherwise, the cb might get
+ * freed before we're done setting it up
+ */
+ atomic_inc(&cb->pending_bios);
+ ret = btrfs_bio_wq_end_io(root->fs_info, bio, 0);
+ BUG_ON(ret);
+
+ ret = btrfs_csum_one_bio(root, inode, bio, start, 1);
+ BUG_ON(ret);
+
+ ret = btrfs_map_bio(root, WRITE, bio, 0, 1);
+ BUG_ON(ret);
+
+ bio_put(bio);
+
+ bio = compressed_bio_alloc(bdev, first_byte, GFP_NOFS);
+ bio->bi_private = cb;
+ bio->bi_end_io = end_compressed_bio_write;
+ bio_add_page(bio, page, PAGE_CACHE_SIZE, 0);
+ }
+ if (bytes_left < PAGE_CACHE_SIZE) {
+ printk("bytes left %lu compress len %lu nr %lu\n",
+ bytes_left, cb->compressed_len, cb->nr_pages);
+ }
+ bytes_left -= PAGE_CACHE_SIZE;
+ first_byte += PAGE_CACHE_SIZE;
+ cond_resched();
+ }
+ bio_get(bio);
+
+ ret = btrfs_bio_wq_end_io(root->fs_info, bio, 0);
+ BUG_ON(ret);
+
+ ret = btrfs_csum_one_bio(root, inode, bio, start, 1);
+ BUG_ON(ret);
+
+ ret = btrfs_map_bio(root, WRITE, bio, 0, 1);
+ BUG_ON(ret);
+
+ bio_put(bio);
+ return 0;
+}
+
+static noinline int add_ra_bio_pages(struct inode *inode,
+ u64 compressed_end,
+ struct compressed_bio *cb)
+{
+ unsigned long end_index;
+ unsigned long page_index;
+ u64 last_offset;
+ u64 isize = i_size_read(inode);
+ int ret;
+ struct page *page;
+ unsigned long nr_pages = 0;
+ struct extent_map *em;
+ struct address_space *mapping = inode->i_mapping;
+ struct pagevec pvec;
+ struct extent_map_tree *em_tree;
+ struct extent_io_tree *tree;
+ u64 end;
+ int misses = 0;
+
+ page = cb->orig_bio->bi_io_vec[cb->orig_bio->bi_vcnt - 1].bv_page;
+ last_offset = (page_offset(page) + PAGE_CACHE_SIZE);
+ em_tree = &BTRFS_I(inode)->extent_tree;
+ tree = &BTRFS_I(inode)->io_tree;
+
+ if (isize == 0)
+ return 0;
+
+ end_index = (i_size_read(inode) - 1) >> PAGE_CACHE_SHIFT;
+
+ pagevec_init(&pvec, 0);
+ while (last_offset < compressed_end) {
+ page_index = last_offset >> PAGE_CACHE_SHIFT;
+
+ if (page_index > end_index)
+ break;
+
+ rcu_read_lock();
+ page = radix_tree_lookup(&mapping->page_tree, page_index);
+ rcu_read_unlock();
+ if (page) {
+ misses++;
+ if (misses > 4)
+ break;
+ goto next;
+ }
+
+ page = alloc_page(mapping_gfp_mask(mapping) | GFP_NOFS);
+ if (!page)
+ break;
+
+ page->index = page_index;
+ /*
+ * what we want to do here is call add_to_page_cache_lru,
+ * but that isn't exported, so we reproduce it here
+ */
+ if (add_to_page_cache(page, mapping,
+ page->index, GFP_NOFS)) {
+ page_cache_release(page);
+ goto next;
+ }
+
+ /* open coding of lru_cache_add, also not exported */
+ page_cache_get(page);
+ if (!pagevec_add(&pvec, page))
+ __pagevec_lru_add_file(&pvec);
+
+ end = last_offset + PAGE_CACHE_SIZE - 1;
+ /*
+ * at this point, we have a locked page in the page cache
+ * for these bytes in the file. But, we have to make
+ * sure they map to this compressed extent on disk.
+ */
+ set_page_extent_mapped(page);
+ lock_extent(tree, last_offset, end, GFP_NOFS);
+ spin_lock(&em_tree->lock);
+ em = lookup_extent_mapping(em_tree, last_offset,
+ PAGE_CACHE_SIZE);
+ spin_unlock(&em_tree->lock);
+
+ if (!em || last_offset < em->start ||
+ (last_offset + PAGE_CACHE_SIZE > extent_map_end(em)) ||
+ (em->block_start >> 9) != cb->orig_bio->bi_sector) {
+ free_extent_map(em);
+ unlock_extent(tree, last_offset, end, GFP_NOFS);
+ unlock_page(page);
+ page_cache_release(page);
+ break;
+ }
+ free_extent_map(em);
+
+ if (page->index == end_index) {
+ char *userpage;
+ size_t zero_offset = isize & (PAGE_CACHE_SIZE - 1);
+
+ if (zero_offset) {
+ int zeros;
+ zeros = PAGE_CACHE_SIZE - zero_offset;
+ userpage = kmap_atomic(page, KM_USER0);
+ memset(userpage + zero_offset, 0, zeros);
+ flush_dcache_page(page);
+ kunmap_atomic(userpage, KM_USER0);
+ }
+ }
+
+ ret = bio_add_page(cb->orig_bio, page,
+ PAGE_CACHE_SIZE, 0);
+
+ if (ret == PAGE_CACHE_SIZE) {
+ nr_pages++;
+ page_cache_release(page);
+ } else {
+ unlock_extent(tree, last_offset, end, GFP_NOFS);
+ unlock_page(page);
+ page_cache_release(page);
+ break;
+ }
+next:
+ last_offset += PAGE_CACHE_SIZE;
+ }
+ if (pagevec_count(&pvec))
+ __pagevec_lru_add_file(&pvec);
+ return 0;
+}
+
+/*
+ * for a compressed read, the bio we get passed has all the inode pages
+ * in it. We don't actually do IO on those pages but allocate new ones
+ * to hold the compressed pages on disk.
+ *
+ * bio->bi_sector points to the compressed extent on disk
+ * bio->bi_io_vec points to all of the inode pages
+ * bio->bi_vcnt is a count of pages
+ *
+ * After the compressed pages are read, we copy the bytes into the
+ * bio we were passed and then call the bio end_io calls
+ */
+int btrfs_submit_compressed_read(struct inode *inode, struct bio *bio,
+ int mirror_num, unsigned long bio_flags)
+{
+ struct extent_io_tree *tree;
+ struct extent_map_tree *em_tree;
+ struct compressed_bio *cb;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ unsigned long uncompressed_len = bio->bi_vcnt * PAGE_CACHE_SIZE;
+ unsigned long compressed_len;
+ unsigned long nr_pages;
+ unsigned long page_index;
+ struct page *page;
+ struct block_device *bdev;
+ struct bio *comp_bio;
+ u64 cur_disk_byte = (u64)bio->bi_sector << 9;
+ u64 em_len;
+ u64 em_start;
+ struct extent_map *em;
+ int ret;
+ u32 *sums;
+
+ tree = &BTRFS_I(inode)->io_tree;
+ em_tree = &BTRFS_I(inode)->extent_tree;
+
+ /* we need the actual starting offset of this extent in the file */
+ spin_lock(&em_tree->lock);
+ em = lookup_extent_mapping(em_tree,
+ page_offset(bio->bi_io_vec->bv_page),
+ PAGE_CACHE_SIZE);
+ spin_unlock(&em_tree->lock);
+
+ compressed_len = em->block_len;
+ cb = kmalloc(compressed_bio_size(root, compressed_len), GFP_NOFS);
+ atomic_set(&cb->pending_bios, 0);
+ cb->errors = 0;
+ cb->inode = inode;
+ cb->mirror_num = mirror_num;
+ sums = &cb->sums;
+
+ cb->start = em->orig_start;
+ em_len = em->len;
+ em_start = em->start;
+
+ free_extent_map(em);
+ em = NULL;
+
+ cb->len = uncompressed_len;
+ cb->compressed_len = compressed_len;
+ cb->orig_bio = bio;
+
+ nr_pages = (compressed_len + PAGE_CACHE_SIZE - 1) /
+ PAGE_CACHE_SIZE;
+ cb->compressed_pages = kmalloc(sizeof(struct page *) * nr_pages,
+ GFP_NOFS);
+ bdev = BTRFS_I(inode)->root->fs_info->fs_devices->latest_bdev;
+
+ for (page_index = 0; page_index < nr_pages; page_index++) {
+ cb->compressed_pages[page_index] = alloc_page(GFP_NOFS |
+ __GFP_HIGHMEM);
+ }
+ cb->nr_pages = nr_pages;
+
+ add_ra_bio_pages(inode, em_start + em_len, cb);
+
+ /* include any pages we added in add_ra-bio_pages */
+ uncompressed_len = bio->bi_vcnt * PAGE_CACHE_SIZE;
+ cb->len = uncompressed_len;
+
+ comp_bio = compressed_bio_alloc(bdev, cur_disk_byte, GFP_NOFS);
+ comp_bio->bi_private = cb;
+ comp_bio->bi_end_io = end_compressed_bio_read;
+ atomic_inc(&cb->pending_bios);
+
+ for (page_index = 0; page_index < nr_pages; page_index++) {
+ page = cb->compressed_pages[page_index];
+ page->mapping = inode->i_mapping;
+ page->index = em_start >> PAGE_CACHE_SHIFT;
+
+ if (comp_bio->bi_size)
+ ret = tree->ops->merge_bio_hook(page, 0,
+ PAGE_CACHE_SIZE,
+ comp_bio, 0);
+ else
+ ret = 0;
+
+ page->mapping = NULL;
+ if (ret || bio_add_page(comp_bio, page, PAGE_CACHE_SIZE, 0) <
+ PAGE_CACHE_SIZE) {
+ bio_get(comp_bio);
+
+ ret = btrfs_bio_wq_end_io(root->fs_info, comp_bio, 0);
+ BUG_ON(ret);
+
+ /*
+ * inc the count before we submit the bio so
+ * we know the end IO handler won't happen before
+ * we inc the count. Otherwise, the cb might get
+ * freed before we're done setting it up
+ */
+ atomic_inc(&cb->pending_bios);
+
+ if (!btrfs_test_flag(inode, NODATASUM)) {
+ btrfs_lookup_bio_sums(root, inode, comp_bio,
+ sums);
+ }
+ sums += (comp_bio->bi_size + root->sectorsize - 1) /
+ root->sectorsize;
+
+ ret = btrfs_map_bio(root, READ, comp_bio,
+ mirror_num, 0);
+ BUG_ON(ret);
+
+ bio_put(comp_bio);
+
+ comp_bio = compressed_bio_alloc(bdev, cur_disk_byte,
+ GFP_NOFS);
+ comp_bio->bi_private = cb;
+ comp_bio->bi_end_io = end_compressed_bio_read;
+
+ bio_add_page(comp_bio, page, PAGE_CACHE_SIZE, 0);
+ }
+ cur_disk_byte += PAGE_CACHE_SIZE;
+ }
+ bio_get(comp_bio);
+
+ ret = btrfs_bio_wq_end_io(root->fs_info, comp_bio, 0);
+ BUG_ON(ret);
+
+ if (!btrfs_test_flag(inode, NODATASUM))
+ btrfs_lookup_bio_sums(root, inode, comp_bio, sums);
+
+ ret = btrfs_map_bio(root, READ, comp_bio, mirror_num, 0);
+ BUG_ON(ret);
+
+ bio_put(comp_bio);
+ return 0;
+}
diff --git a/fs/btrfs/compression.h b/fs/btrfs/compression.h
new file mode 100644
index 0000000..421f5b4
--- /dev/null
+++ b/fs/btrfs/compression.h
@@ -0,0 +1,47 @@
+/*
+ * Copyright (C) 2008 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __BTRFS_COMPRESSION_
+#define __BTRFS_COMPRESSION_
+
+int btrfs_zlib_decompress(unsigned char *data_in,
+ struct page *dest_page,
+ unsigned long start_byte,
+ size_t srclen, size_t destlen);
+int btrfs_zlib_compress_pages(struct address_space *mapping,
+ u64 start, unsigned long len,
+ struct page **pages,
+ unsigned long nr_dest_pages,
+ unsigned long *out_pages,
+ unsigned long *total_in,
+ unsigned long *total_out,
+ unsigned long max_out);
+int btrfs_zlib_decompress_biovec(struct page **pages_in,
+ u64 disk_start,
+ struct bio_vec *bvec,
+ int vcnt,
+ size_t srclen);
+void btrfs_zlib_exit(void);
+int btrfs_submit_compressed_write(struct inode *inode, u64 start,
+ unsigned long len, u64 disk_start,
+ unsigned long compressed_len,
+ struct page **compressed_pages,
+ unsigned long nr_pages);
+int btrfs_submit_compressed_read(struct inode *inode, struct bio *bio,
+ int mirror_num, unsigned long bio_flags);
+#endif
diff --git a/fs/btrfs/crc32c.h b/fs/btrfs/crc32c.h
new file mode 100644
index 0000000..6e1b3de
--- /dev/null
+++ b/fs/btrfs/crc32c.h
@@ -0,0 +1,29 @@
+/*
+ * Copyright (C) 2008 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __BTRFS_CRC32C__
+#define __BTRFS_CRC32C__
+#include <linux/crc32c.h>
+
+/*
+ * this file used to do more for selecting the HW version of crc32c,
+ * perhaps it will one day again soon.
+ */
+#define btrfs_crc32c(seed, data, length) crc32c(seed, data, length)
+#endif
+
diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
new file mode 100644
index 0000000..9e46c07
--- /dev/null
+++ b/fs/btrfs/ctree.c
@@ -0,0 +1,3953 @@
+/*
+ * Copyright (C) 2007,2008 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/sched.h>
+#include "ctree.h"
+#include "disk-io.h"
+#include "transaction.h"
+#include "print-tree.h"
+#include "locking.h"
+
+static int split_node(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root, struct btrfs_path *path, int level);
+static int split_leaf(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root, struct btrfs_key *ins_key,
+ struct btrfs_path *path, int data_size, int extend);
+static int push_node_left(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct extent_buffer *dst,
+ struct extent_buffer *src, int empty);
+static int balance_node_right(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct extent_buffer *dst_buf,
+ struct extent_buffer *src_buf);
+static int del_ptr(struct btrfs_trans_handle *trans, struct btrfs_root *root,
+ struct btrfs_path *path, int level, int slot);
+
+inline void btrfs_init_path(struct btrfs_path *p)
+{
+ memset(p, 0, sizeof(*p));
+}
+
+struct btrfs_path *btrfs_alloc_path(void)
+{
+ struct btrfs_path *path;
+ path = kmem_cache_alloc(btrfs_path_cachep, GFP_NOFS);
+ if (path) {
+ btrfs_init_path(path);
+ path->reada = 1;
+ }
+ return path;
+}
+
+/* this also releases the path */
+void btrfs_free_path(struct btrfs_path *p)
+{
+ btrfs_release_path(NULL, p);
+ kmem_cache_free(btrfs_path_cachep, p);
+}
+
+/*
+ * path release drops references on the extent buffers in the path
+ * and it drops any locks held by this path
+ *
+ * It is safe to call this on paths that no locks or extent buffers held.
+ */
+noinline void btrfs_release_path(struct btrfs_root *root, struct btrfs_path *p)
+{
+ int i;
+
+ for (i = 0; i < BTRFS_MAX_LEVEL; i++) {
+ p->slots[i] = 0;
+ if (!p->nodes[i])
+ continue;
+ if (p->locks[i]) {
+ btrfs_tree_unlock(p->nodes[i]);
+ p->locks[i] = 0;
+ }
+ free_extent_buffer(p->nodes[i]);
+ p->nodes[i] = NULL;
+ }
+}
+
+/*
+ * safely gets a reference on the root node of a tree. A lock
+ * is not taken, so a concurrent writer may put a different node
+ * at the root of the tree. See btrfs_lock_root_node for the
+ * looping required.
+ *
+ * The extent buffer returned by this has a reference taken, so
+ * it won't disappear. It may stop being the root of the tree
+ * at any time because there are no locks held.
+ */
+struct extent_buffer *btrfs_root_node(struct btrfs_root *root)
+{
+ struct extent_buffer *eb;
+ spin_lock(&root->node_lock);
+ eb = root->node;
+ extent_buffer_get(eb);
+ spin_unlock(&root->node_lock);
+ return eb;
+}
+
+/* loop around taking references on and locking the root node of the
+ * tree until you end up with a lock on the root. A locked buffer
+ * is returned, with a reference held.
+ */
+struct extent_buffer *btrfs_lock_root_node(struct btrfs_root *root)
+{
+ struct extent_buffer *eb;
+
+ while (1) {
+ eb = btrfs_root_node(root);
+ btrfs_tree_lock(eb);
+
+ spin_lock(&root->node_lock);
+ if (eb == root->node) {
+ spin_unlock(&root->node_lock);
+ break;
+ }
+ spin_unlock(&root->node_lock);
+
+ btrfs_tree_unlock(eb);
+ free_extent_buffer(eb);
+ }
+ return eb;
+}
+
+/* cowonly root (everything not a reference counted cow subvolume), just get
+ * put onto a simple dirty list. transaction.c walks this to make sure they
+ * get properly updated on disk.
+ */
+static void add_root_to_dirty_list(struct btrfs_root *root)
+{
+ if (root->track_dirty && list_empty(&root->dirty_list)) {
+ list_add(&root->dirty_list,
+ &root->fs_info->dirty_cowonly_roots);
+ }
+}
+
+/*
+ * used by snapshot creation to make a copy of a root for a tree with
+ * a given objectid. The buffer with the new root node is returned in
+ * cow_ret, and this func returns zero on success or a negative error code.
+ */
+int btrfs_copy_root(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct extent_buffer *buf,
+ struct extent_buffer **cow_ret, u64 new_root_objectid)
+{
+ struct extent_buffer *cow;
+ u32 nritems;
+ int ret = 0;
+ int level;
+ struct btrfs_root *new_root;
+
+ new_root = kmalloc(sizeof(*new_root), GFP_NOFS);
+ if (!new_root)
+ return -ENOMEM;
+
+ memcpy(new_root, root, sizeof(*new_root));
+ new_root->root_key.objectid = new_root_objectid;
+
+ WARN_ON(root->ref_cows && trans->transid !=
+ root->fs_info->running_transaction->transid);
+ WARN_ON(root->ref_cows && trans->transid != root->last_trans);
+
+ level = btrfs_header_level(buf);
+ nritems = btrfs_header_nritems(buf);
+
+ cow = btrfs_alloc_free_block(trans, new_root, buf->len, 0,
+ new_root_objectid, trans->transid,
+ level, buf->start, 0);
+ if (IS_ERR(cow)) {
+ kfree(new_root);
+ return PTR_ERR(cow);
+ }
+
+ copy_extent_buffer(cow, buf, 0, 0, cow->len);
+ btrfs_set_header_bytenr(cow, cow->start);
+ btrfs_set_header_generation(cow, trans->transid);
+ btrfs_set_header_owner(cow, new_root_objectid);
+ btrfs_clear_header_flag(cow, BTRFS_HEADER_FLAG_WRITTEN);
+
+ write_extent_buffer(cow, root->fs_info->fsid,
+ (unsigned long)btrfs_header_fsid(cow),
+ BTRFS_FSID_SIZE);
+
+ WARN_ON(btrfs_header_generation(buf) > trans->transid);
+ ret = btrfs_inc_ref(trans, new_root, buf, cow, NULL);
+ kfree(new_root);
+
+ if (ret)
+ return ret;
+
+ btrfs_mark_buffer_dirty(cow);
+ *cow_ret = cow;
+ return 0;
+}
+
+/*
+ * does the dirty work in cow of a single block. The parent block (if
+ * supplied) is updated to point to the new cow copy. The new buffer is marked
+ * dirty and returned locked. If you modify the block it needs to be marked
+ * dirty again.
+ *
+ * search_start -- an allocation hint for the new block
+ *
+ * empty_size -- a hint that you plan on doing more cow. This is the size in
+ * bytes the allocator should try to find free next to the block it returns.
+ * This is just a hint and may be ignored by the allocator.
+ *
+ * prealloc_dest -- if you have already reserved a destination for the cow,
+ * this uses that block instead of allocating a new one.
+ * btrfs_alloc_reserved_extent is used to finish the allocation.
+ */
+static noinline int __btrfs_cow_block(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct extent_buffer *buf,
+ struct extent_buffer *parent, int parent_slot,
+ struct extent_buffer **cow_ret,
+ u64 search_start, u64 empty_size,
+ u64 prealloc_dest)
+{
+ u64 parent_start;
+ struct extent_buffer *cow;
+ u32 nritems;
+ int ret = 0;
+ int level;
+ int unlock_orig = 0;
+
+ if (*cow_ret == buf)
+ unlock_orig = 1;
+
+ WARN_ON(!btrfs_tree_locked(buf));
+
+ if (parent)
+ parent_start = parent->start;
+ else
+ parent_start = 0;
+
+ WARN_ON(root->ref_cows && trans->transid !=
+ root->fs_info->running_transaction->transid);
+ WARN_ON(root->ref_cows && trans->transid != root->last_trans);
+
+ level = btrfs_header_level(buf);
+ nritems = btrfs_header_nritems(buf);
+
+ if (prealloc_dest) {
+ struct btrfs_key ins;
+
+ ins.objectid = prealloc_dest;
+ ins.offset = buf->len;
+ ins.type = BTRFS_EXTENT_ITEM_KEY;
+
+ ret = btrfs_alloc_reserved_extent(trans, root, parent_start,
+ root->root_key.objectid,
+ trans->transid, level, &ins);
+ BUG_ON(ret);
+ cow = btrfs_init_new_buffer(trans, root, prealloc_dest,
+ buf->len);
+ } else {
+ cow = btrfs_alloc_free_block(trans, root, buf->len,
+ parent_start,
+ root->root_key.objectid,
+ trans->transid, level,
+ search_start, empty_size);
+ }
+ if (IS_ERR(cow))
+ return PTR_ERR(cow);
+
+ copy_extent_buffer(cow, buf, 0, 0, cow->len);
+ btrfs_set_header_bytenr(cow, cow->start);
+ btrfs_set_header_generation(cow, trans->transid);
+ btrfs_set_header_owner(cow, root->root_key.objectid);
+ btrfs_clear_header_flag(cow, BTRFS_HEADER_FLAG_WRITTEN);
+
+ write_extent_buffer(cow, root->fs_info->fsid,
+ (unsigned long)btrfs_header_fsid(cow),
+ BTRFS_FSID_SIZE);
+
+ WARN_ON(btrfs_header_generation(buf) > trans->transid);
+ if (btrfs_header_generation(buf) != trans->transid) {
+ u32 nr_extents;
+ ret = btrfs_inc_ref(trans, root, buf, cow, &nr_extents);
+ if (ret)
+ return ret;
+
+ ret = btrfs_cache_ref(trans, root, buf, nr_extents);
+ WARN_ON(ret);
+ } else if (btrfs_header_owner(buf) == BTRFS_TREE_RELOC_OBJECTID) {
+ /*
+ * There are only two places that can drop reference to
+ * tree blocks owned by living reloc trees, one is here,
+ * the other place is btrfs_drop_subtree. In both places,
+ * we check reference count while tree block is locked.
+ * Furthermore, if reference count is one, it won't get
+ * increased by someone else.
+ */
+ u32 refs;
+ ret = btrfs_lookup_extent_ref(trans, root, buf->start,
+ buf->len, &refs);
+ BUG_ON(ret);
+ if (refs == 1) {
+ ret = btrfs_update_ref(trans, root, buf, cow,
+ 0, nritems);
+ clean_tree_block(trans, root, buf);
+ } else {
+ ret = btrfs_inc_ref(trans, root, buf, cow, NULL);
+ }
+ BUG_ON(ret);
+ } else {
+ ret = btrfs_update_ref(trans, root, buf, cow, 0, nritems);
+ if (ret)
+ return ret;
+ clean_tree_block(trans, root, buf);
+ }
+
+ if (root->root_key.objectid == BTRFS_TREE_RELOC_OBJECTID) {
+ ret = btrfs_reloc_tree_cache_ref(trans, root, cow, buf->start);
+ WARN_ON(ret);
+ }
+
+ if (buf == root->node) {
+ WARN_ON(parent && parent != buf);
+
+ spin_lock(&root->node_lock);
+ root->node = cow;
+ extent_buffer_get(cow);
+ spin_unlock(&root->node_lock);
+
+ if (buf != root->commit_root) {
+ btrfs_free_extent(trans, root, buf->start,
+ buf->len, buf->start,
+ root->root_key.objectid,
+ btrfs_header_generation(buf),
+ level, 1);
+ }
+ free_extent_buffer(buf);
+ add_root_to_dirty_list(root);
+ } else {
+ btrfs_set_node_blockptr(parent, parent_slot,
+ cow->start);
+ WARN_ON(trans->transid == 0);
+ btrfs_set_node_ptr_generation(parent, parent_slot,
+ trans->transid);
+ btrfs_mark_buffer_dirty(parent);
+ WARN_ON(btrfs_header_generation(parent) != trans->transid);
+ btrfs_free_extent(trans, root, buf->start, buf->len,
+ parent_start, btrfs_header_owner(parent),
+ btrfs_header_generation(parent), level, 1);
+ }
+ if (unlock_orig)
+ btrfs_tree_unlock(buf);
+ free_extent_buffer(buf);
+ btrfs_mark_buffer_dirty(cow);
+ *cow_ret = cow;
+ return 0;
+}
+
+/*
+ * cows a single block, see __btrfs_cow_block for the real work.
+ * This version of it has extra checks so that a block isn't cow'd more than
+ * once per transaction, as long as it hasn't been written yet
+ */
+noinline int btrfs_cow_block(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct extent_buffer *buf,
+ struct extent_buffer *parent, int parent_slot,
+ struct extent_buffer **cow_ret, u64 prealloc_dest)
+{
+ u64 search_start;
+ int ret;
+
+ if (trans->transaction != root->fs_info->running_transaction) {
+ printk(KERN_CRIT "trans %llu running %llu\n",
+ (unsigned long long)trans->transid,
+ (unsigned long long)
+ root->fs_info->running_transaction->transid);
+ WARN_ON(1);
+ }
+ if (trans->transid != root->fs_info->generation) {
+ printk(KERN_CRIT "trans %llu running %llu\n",
+ (unsigned long long)trans->transid,
+ (unsigned long long)root->fs_info->generation);
+ WARN_ON(1);
+ }
+
+ spin_lock(&root->fs_info->hash_lock);
+ if (btrfs_header_generation(buf) == trans->transid &&
+ btrfs_header_owner(buf) == root->root_key.objectid &&
+ !btrfs_header_flag(buf, BTRFS_HEADER_FLAG_WRITTEN)) {
+ *cow_ret = buf;
+ spin_unlock(&root->fs_info->hash_lock);
+ WARN_ON(prealloc_dest);
+ return 0;
+ }
+ spin_unlock(&root->fs_info->hash_lock);
+ search_start = buf->start & ~((u64)(1024 * 1024 * 1024) - 1);
+ ret = __btrfs_cow_block(trans, root, buf, parent,
+ parent_slot, cow_ret, search_start, 0,
+ prealloc_dest);
+ return ret;
+}
+
+/*
+ * helper function for defrag to decide if two blocks pointed to by a
+ * node are actually close by
+ */
+static int close_blocks(u64 blocknr, u64 other, u32 blocksize)
+{
+ if (blocknr < other && other - (blocknr + blocksize) < 32768)
+ return 1;
+ if (blocknr > other && blocknr - (other + blocksize) < 32768)
+ return 1;
+ return 0;
+}
+
+/*
+ * compare two keys in a memcmp fashion
+ */
+static int comp_keys(struct btrfs_disk_key *disk, struct btrfs_key *k2)
+{
+ struct btrfs_key k1;
+
+ btrfs_disk_key_to_cpu(&k1, disk);
+
+ if (k1.objectid > k2->objectid)
+ return 1;
+ if (k1.objectid < k2->objectid)
+ return -1;
+ if (k1.type > k2->type)
+ return 1;
+ if (k1.type < k2->type)
+ return -1;
+ if (k1.offset > k2->offset)
+ return 1;
+ if (k1.offset < k2->offset)
+ return -1;
+ return 0;
+}
+
+/*
+ * same as comp_keys only with two btrfs_key's
+ */
+static int comp_cpu_keys(struct btrfs_key *k1, struct btrfs_key *k2)
+{
+ if (k1->objectid > k2->objectid)
+ return 1;
+ if (k1->objectid < k2->objectid)
+ return -1;
+ if (k1->type > k2->type)
+ return 1;
+ if (k1->type < k2->type)
+ return -1;
+ if (k1->offset > k2->offset)
+ return 1;
+ if (k1->offset < k2->offset)
+ return -1;
+ return 0;
+}
+
+/*
+ * this is used by the defrag code to go through all the
+ * leaves pointed to by a node and reallocate them so that
+ * disk order is close to key order
+ */
+int btrfs_realloc_node(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct extent_buffer *parent,
+ int start_slot, int cache_only, u64 *last_ret,
+ struct btrfs_key *progress)
+{
+ struct extent_buffer *cur;
+ u64 blocknr;
+ u64 gen;
+ u64 search_start = *last_ret;
+ u64 last_block = 0;
+ u64 other;
+ u32 parent_nritems;
+ int end_slot;
+ int i;
+ int err = 0;
+ int parent_level;
+ int uptodate;
+ u32 blocksize;
+ int progress_passed = 0;
+ struct btrfs_disk_key disk_key;
+
+ parent_level = btrfs_header_level(parent);
+ if (cache_only && parent_level != 1)
+ return 0;
+
+ if (trans->transaction != root->fs_info->running_transaction)
+ WARN_ON(1);
+ if (trans->transid != root->fs_info->generation)
+ WARN_ON(1);
+
+ parent_nritems = btrfs_header_nritems(parent);
+ blocksize = btrfs_level_size(root, parent_level - 1);
+ end_slot = parent_nritems;
+
+ if (parent_nritems == 1)
+ return 0;
+
+ for (i = start_slot; i < end_slot; i++) {
+ int close = 1;
+
+ if (!parent->map_token) {
+ map_extent_buffer(parent,
+ btrfs_node_key_ptr_offset(i),
+ sizeof(struct btrfs_key_ptr),
+ &parent->map_token, &parent->kaddr,
+ &parent->map_start, &parent->map_len,
+ KM_USER1);
+ }
+ btrfs_node_key(parent, &disk_key, i);
+ if (!progress_passed && comp_keys(&disk_key, progress) < 0)
+ continue;
+
+ progress_passed = 1;
+ blocknr = btrfs_node_blockptr(parent, i);
+ gen = btrfs_node_ptr_generation(parent, i);
+ if (last_block == 0)
+ last_block = blocknr;
+
+ if (i > 0) {
+ other = btrfs_node_blockptr(parent, i - 1);
+ close = close_blocks(blocknr, other, blocksize);
+ }
+ if (!close && i < end_slot - 2) {
+ other = btrfs_node_blockptr(parent, i + 1);
+ close = close_blocks(blocknr, other, blocksize);
+ }
+ if (close) {
+ last_block = blocknr;
+ continue;
+ }
+ if (parent->map_token) {
+ unmap_extent_buffer(parent, parent->map_token,
+ KM_USER1);
+ parent->map_token = NULL;
+ }
+
+ cur = btrfs_find_tree_block(root, blocknr, blocksize);
+ if (cur)
+ uptodate = btrfs_buffer_uptodate(cur, gen);
+ else
+ uptodate = 0;
+ if (!cur || !uptodate) {
+ if (cache_only) {
+ free_extent_buffer(cur);
+ continue;
+ }
+ if (!cur) {
+ cur = read_tree_block(root, blocknr,
+ blocksize, gen);
+ } else if (!uptodate) {
+ btrfs_read_buffer(cur, gen);
+ }
+ }
+ if (search_start == 0)
+ search_start = last_block;
+
+ btrfs_tree_lock(cur);
+ err = __btrfs_cow_block(trans, root, cur, parent, i,
+ &cur, search_start,
+ min(16 * blocksize,
+ (end_slot - i) * blocksize), 0);
+ if (err) {
+ btrfs_tree_unlock(cur);
+ free_extent_buffer(cur);
+ break;
+ }
+ search_start = cur->start;
+ last_block = cur->start;
+ *last_ret = search_start;
+ btrfs_tree_unlock(cur);
+ free_extent_buffer(cur);
+ }
+ if (parent->map_token) {
+ unmap_extent_buffer(parent, parent->map_token,
+ KM_USER1);
+ parent->map_token = NULL;
+ }
+ return err;
+}
+
+/*
+ * The leaf data grows from end-to-front in the node.
+ * this returns the address of the start of the last item,
+ * which is the stop of the leaf data stack
+ */
+static inline unsigned int leaf_data_end(struct btrfs_root *root,
+ struct extent_buffer *leaf)
+{
+ u32 nr = btrfs_header_nritems(leaf);
+ if (nr == 0)
+ return BTRFS_LEAF_DATA_SIZE(root);
+ return btrfs_item_offset_nr(leaf, nr - 1);
+}
+
+/*
+ * extra debugging checks to make sure all the items in a key are
+ * well formed and in the proper order
+ */
+static int check_node(struct btrfs_root *root, struct btrfs_path *path,
+ int level)
+{
+ struct extent_buffer *parent = NULL;
+ struct extent_buffer *node = path->nodes[level];
+ struct btrfs_disk_key parent_key;
+ struct btrfs_disk_key node_key;
+ int parent_slot;
+ int slot;
+ struct btrfs_key cpukey;
+ u32 nritems = btrfs_header_nritems(node);
+
+ if (path->nodes[level + 1])
+ parent = path->nodes[level + 1];
+
+ slot = path->slots[level];
+ BUG_ON(nritems == 0);
+ if (parent) {
+ parent_slot = path->slots[level + 1];
+ btrfs_node_key(parent, &parent_key, parent_slot);
+ btrfs_node_key(node, &node_key, 0);
+ BUG_ON(memcmp(&parent_key, &node_key,
+ sizeof(struct btrfs_disk_key)));
+ BUG_ON(btrfs_node_blockptr(parent, parent_slot) !=
+ btrfs_header_bytenr(node));
+ }
+ BUG_ON(nritems > BTRFS_NODEPTRS_PER_BLOCK(root));
+ if (slot != 0) {
+ btrfs_node_key_to_cpu(node, &cpukey, slot - 1);
+ btrfs_node_key(node, &node_key, slot);
+ BUG_ON(comp_keys(&node_key, &cpukey) <= 0);
+ }
+ if (slot < nritems - 1) {
+ btrfs_node_key_to_cpu(node, &cpukey, slot + 1);
+ btrfs_node_key(node, &node_key, slot);
+ BUG_ON(comp_keys(&node_key, &cpukey) >= 0);
+ }
+ return 0;
+}
+
+/*
+ * extra checking to make sure all the items in a leaf are
+ * well formed and in the proper order
+ */
+static int check_leaf(struct btrfs_root *root, struct btrfs_path *path,
+ int level)
+{
+ struct extent_buffer *leaf = path->nodes[level];
+ struct extent_buffer *parent = NULL;
+ int parent_slot;
+ struct btrfs_key cpukey;
+ struct btrfs_disk_key parent_key;
+ struct btrfs_disk_key leaf_key;
+ int slot = path->slots[0];
+
+ u32 nritems = btrfs_header_nritems(leaf);
+
+ if (path->nodes[level + 1])
+ parent = path->nodes[level + 1];
+
+ if (nritems == 0)
+ return 0;
+
+ if (parent) {
+ parent_slot = path->slots[level + 1];
+ btrfs_node_key(parent, &parent_key, parent_slot);
+ btrfs_item_key(leaf, &leaf_key, 0);
+
+ BUG_ON(memcmp(&parent_key, &leaf_key,
+ sizeof(struct btrfs_disk_key)));
+ BUG_ON(btrfs_node_blockptr(parent, parent_slot) !=
+ btrfs_header_bytenr(leaf));
+ }
+ if (slot != 0 && slot < nritems - 1) {
+ btrfs_item_key(leaf, &leaf_key, slot);
+ btrfs_item_key_to_cpu(leaf, &cpukey, slot - 1);
+ if (comp_keys(&leaf_key, &cpukey) <= 0) {
+ btrfs_print_leaf(root, leaf);
+ printk(KERN_CRIT "slot %d offset bad key\n", slot);
+ BUG_ON(1);
+ }
+ if (btrfs_item_offset_nr(leaf, slot - 1) !=
+ btrfs_item_end_nr(leaf, slot)) {
+ btrfs_print_leaf(root, leaf);
+ printk(KERN_CRIT "slot %d offset bad\n", slot);
+ BUG_ON(1);
+ }
+ }
+ if (slot < nritems - 1) {
+ btrfs_item_key(leaf, &leaf_key, slot);
+ btrfs_item_key_to_cpu(leaf, &cpukey, slot + 1);
+ BUG_ON(comp_keys(&leaf_key, &cpukey) >= 0);
+ if (btrfs_item_offset_nr(leaf, slot) !=
+ btrfs_item_end_nr(leaf, slot + 1)) {
+ btrfs_print_leaf(root, leaf);
+ printk(KERN_CRIT "slot %d offset bad\n", slot);
+ BUG_ON(1);
+ }
+ }
+ BUG_ON(btrfs_item_offset_nr(leaf, 0) +
+ btrfs_item_size_nr(leaf, 0) != BTRFS_LEAF_DATA_SIZE(root));
+ return 0;
+}
+
+static noinline int check_block(struct btrfs_root *root,
+ struct btrfs_path *path, int level)
+{
+ return 0;
+ if (level == 0)
+ return check_leaf(root, path, level);
+ return check_node(root, path, level);
+}
+
+/*
+ * search for key in the extent_buffer. The items start at offset p,
+ * and they are item_size apart. There are 'max' items in p.
+ *
+ * the slot in the array is returned via slot, and it points to
+ * the place where you would insert key if it is not found in
+ * the array.
+ *
+ * slot may point to max if the key is bigger than all of the keys
+ */
+static noinline int generic_bin_search(struct extent_buffer *eb,
+ unsigned long p,
+ int item_size, struct btrfs_key *key,
+ int max, int *slot)
+{
+ int low = 0;
+ int high = max;
+ int mid;
+ int ret;
+ struct btrfs_disk_key *tmp = NULL;
+ struct btrfs_disk_key unaligned;
+ unsigned long offset;
+ char *map_token = NULL;
+ char *kaddr = NULL;
+ unsigned long map_start = 0;
+ unsigned long map_len = 0;
+ int err;
+
+ while (low < high) {
+ mid = (low + high) / 2;
+ offset = p + mid * item_size;
+
+ if (!map_token || offset < map_start ||
+ (offset + sizeof(struct btrfs_disk_key)) >
+ map_start + map_len) {
+ if (map_token) {
+ unmap_extent_buffer(eb, map_token, KM_USER0);
+ map_token = NULL;
+ }
+
+ err = map_private_extent_buffer(eb, offset,
+ sizeof(struct btrfs_disk_key),
+ &map_token, &kaddr,
+ &map_start, &map_len, KM_USER0);
+
+ if (!err) {
+ tmp = (struct btrfs_disk_key *)(kaddr + offset -
+ map_start);
+ } else {
+ read_extent_buffer(eb, &unaligned,
+ offset, sizeof(unaligned));
+ tmp = &unaligned;
+ }
+
+ } else {
+ tmp = (struct btrfs_disk_key *)(kaddr + offset -
+ map_start);
+ }
+ ret = comp_keys(tmp, key);
+
+ if (ret < 0)
+ low = mid + 1;
+ else if (ret > 0)
+ high = mid;
+ else {
+ *slot = mid;
+ if (map_token)
+ unmap_extent_buffer(eb, map_token, KM_USER0);
+ return 0;
+ }
+ }
+ *slot = low;
+ if (map_token)
+ unmap_extent_buffer(eb, map_token, KM_USER0);
+ return 1;
+}
+
+/*
+ * simple bin_search frontend that does the right thing for
+ * leaves vs nodes
+ */
+static int bin_search(struct extent_buffer *eb, struct btrfs_key *key,
+ int level, int *slot)
+{
+ if (level == 0) {
+ return generic_bin_search(eb,
+ offsetof(struct btrfs_leaf, items),
+ sizeof(struct btrfs_item),
+ key, btrfs_header_nritems(eb),
+ slot);
+ } else {
+ return generic_bin_search(eb,
+ offsetof(struct btrfs_node, ptrs),
+ sizeof(struct btrfs_key_ptr),
+ key, btrfs_header_nritems(eb),
+ slot);
+ }
+ return -1;
+}
+
+/* given a node and slot number, this reads the blocks it points to. The
+ * extent buffer is returned with a reference taken (but unlocked).
+ * NULL is returned on error.
+ */
+static noinline struct extent_buffer *read_node_slot(struct btrfs_root *root,
+ struct extent_buffer *parent, int slot)
+{
+ int level = btrfs_header_level(parent);
+ if (slot < 0)
+ return NULL;
+ if (slot >= btrfs_header_nritems(parent))
+ return NULL;
+
+ BUG_ON(level == 0);
+
+ return read_tree_block(root, btrfs_node_blockptr(parent, slot),
+ btrfs_level_size(root, level - 1),
+ btrfs_node_ptr_generation(parent, slot));
+}
+
+/*
+ * node level balancing, used to make sure nodes are in proper order for
+ * item deletion. We balance from the top down, so we have to make sure
+ * that a deletion won't leave an node completely empty later on.
+ */
+static noinline int balance_level(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, int level)
+{
+ struct extent_buffer *right = NULL;
+ struct extent_buffer *mid;
+ struct extent_buffer *left = NULL;
+ struct extent_buffer *parent = NULL;
+ int ret = 0;
+ int wret;
+ int pslot;
+ int orig_slot = path->slots[level];
+ int err_on_enospc = 0;
+ u64 orig_ptr;
+
+ if (level == 0)
+ return 0;
+
+ mid = path->nodes[level];
+ WARN_ON(!path->locks[level]);
+ WARN_ON(btrfs_header_generation(mid) != trans->transid);
+
+ orig_ptr = btrfs_node_blockptr(mid, orig_slot);
+
+ if (level < BTRFS_MAX_LEVEL - 1)
+ parent = path->nodes[level + 1];
+ pslot = path->slots[level + 1];
+
+ /*
+ * deal with the case where there is only one pointer in the root
+ * by promoting the node below to a root
+ */
+ if (!parent) {
+ struct extent_buffer *child;
+
+ if (btrfs_header_nritems(mid) != 1)
+ return 0;
+
+ /* promote the child to a root */
+ child = read_node_slot(root, mid, 0);
+ btrfs_tree_lock(child);
+ BUG_ON(!child);
+ ret = btrfs_cow_block(trans, root, child, mid, 0, &child, 0);
+ BUG_ON(ret);
+
+ spin_lock(&root->node_lock);
+ root->node = child;
+ spin_unlock(&root->node_lock);
+
+ ret = btrfs_update_extent_ref(trans, root, child->start,
+ mid->start, child->start,
+ root->root_key.objectid,
+ trans->transid, level - 1);
+ BUG_ON(ret);
+
+ add_root_to_dirty_list(root);
+ btrfs_tree_unlock(child);
+ path->locks[level] = 0;
+ path->nodes[level] = NULL;
+ clean_tree_block(trans, root, mid);
+ btrfs_tree_unlock(mid);
+ /* once for the path */
+ free_extent_buffer(mid);
+ ret = btrfs_free_extent(trans, root, mid->start, mid->len,
+ mid->start, root->root_key.objectid,
+ btrfs_header_generation(mid),
+ level, 1);
+ /* once for the root ptr */
+ free_extent_buffer(mid);
+ return ret;
+ }
+ if (btrfs_header_nritems(mid) >
+ BTRFS_NODEPTRS_PER_BLOCK(root) / 4)
+ return 0;
+
+ if (btrfs_header_nritems(mid) < 2)
+ err_on_enospc = 1;
+
+ left = read_node_slot(root, parent, pslot - 1);
+ if (left) {
+ btrfs_tree_lock(left);
+ wret = btrfs_cow_block(trans, root, left,
+ parent, pslot - 1, &left, 0);
+ if (wret) {
+ ret = wret;
+ goto enospc;
+ }
+ }
+ right = read_node_slot(root, parent, pslot + 1);
+ if (right) {
+ btrfs_tree_lock(right);
+ wret = btrfs_cow_block(trans, root, right,
+ parent, pslot + 1, &right, 0);
+ if (wret) {
+ ret = wret;
+ goto enospc;
+ }
+ }
+
+ /* first, try to make some room in the middle buffer */
+ if (left) {
+ orig_slot += btrfs_header_nritems(left);
+ wret = push_node_left(trans, root, left, mid, 1);
+ if (wret < 0)
+ ret = wret;
+ if (btrfs_header_nritems(mid) < 2)
+ err_on_enospc = 1;
+ }
+
+ /*
+ * then try to empty the right most buffer into the middle
+ */
+ if (right) {
+ wret = push_node_left(trans, root, mid, right, 1);
+ if (wret < 0 && wret != -ENOSPC)
+ ret = wret;
+ if (btrfs_header_nritems(right) == 0) {
+ u64 bytenr = right->start;
+ u64 generation = btrfs_header_generation(parent);
+ u32 blocksize = right->len;
+
+ clean_tree_block(trans, root, right);
+ btrfs_tree_unlock(right);
+ free_extent_buffer(right);
+ right = NULL;
+ wret = del_ptr(trans, root, path, level + 1, pslot +
+ 1);
+ if (wret)
+ ret = wret;
+ wret = btrfs_free_extent(trans, root, bytenr,
+ blocksize, parent->start,
+ btrfs_header_owner(parent),
+ generation, level, 1);
+ if (wret)
+ ret = wret;
+ } else {
+ struct btrfs_disk_key right_key;
+ btrfs_node_key(right, &right_key, 0);
+ btrfs_set_node_key(parent, &right_key, pslot + 1);
+ btrfs_mark_buffer_dirty(parent);
+ }
+ }
+ if (btrfs_header_nritems(mid) == 1) {
+ /*
+ * we're not allowed to leave a node with one item in the
+ * tree during a delete. A deletion from lower in the tree
+ * could try to delete the only pointer in this node.
+ * So, pull some keys from the left.
+ * There has to be a left pointer at this point because
+ * otherwise we would have pulled some pointers from the
+ * right
+ */
+ BUG_ON(!left);
+ wret = balance_node_right(trans, root, mid, left);
+ if (wret < 0) {
+ ret = wret;
+ goto enospc;
+ }
+ if (wret == 1) {
+ wret = push_node_left(trans, root, left, mid, 1);
+ if (wret < 0)
+ ret = wret;
+ }
+ BUG_ON(wret == 1);
+ }
+ if (btrfs_header_nritems(mid) == 0) {
+ /* we've managed to empty the middle node, drop it */
+ u64 root_gen = btrfs_header_generation(parent);
+ u64 bytenr = mid->start;
+ u32 blocksize = mid->len;
+
+ clean_tree_block(trans, root, mid);
+ btrfs_tree_unlock(mid);
+ free_extent_buffer(mid);
+ mid = NULL;
+ wret = del_ptr(trans, root, path, level + 1, pslot);
+ if (wret)
+ ret = wret;
+ wret = btrfs_free_extent(trans, root, bytenr, blocksize,
+ parent->start,
+ btrfs_header_owner(parent),
+ root_gen, level, 1);
+ if (wret)
+ ret = wret;
+ } else {
+ /* update the parent key to reflect our changes */
+ struct btrfs_disk_key mid_key;
+ btrfs_node_key(mid, &mid_key, 0);
+ btrfs_set_node_key(parent, &mid_key, pslot);
+ btrfs_mark_buffer_dirty(parent);
+ }
+
+ /* update the path */
+ if (left) {
+ if (btrfs_header_nritems(left) > orig_slot) {
+ extent_buffer_get(left);
+ /* left was locked after cow */
+ path->nodes[level] = left;
+ path->slots[level + 1] -= 1;
+ path->slots[level] = orig_slot;
+ if (mid) {
+ btrfs_tree_unlock(mid);
+ free_extent_buffer(mid);
+ }
+ } else {
+ orig_slot -= btrfs_header_nritems(left);
+ path->slots[level] = orig_slot;
+ }
+ }
+ /* double check we haven't messed things up */
+ check_block(root, path, level);
+ if (orig_ptr !=
+ btrfs_node_blockptr(path->nodes[level], path->slots[level]))
+ BUG();
+enospc:
+ if (right) {
+ btrfs_tree_unlock(right);
+ free_extent_buffer(right);
+ }
+ if (left) {
+ if (path->nodes[level] != left)
+ btrfs_tree_unlock(left);
+ free_extent_buffer(left);
+ }
+ return ret;
+}
+
+/* Node balancing for insertion. Here we only split or push nodes around
+ * when they are completely full. This is also done top down, so we
+ * have to be pessimistic.
+ */
+static noinline int push_nodes_for_insert(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, int level)
+{
+ struct extent_buffer *right = NULL;
+ struct extent_buffer *mid;
+ struct extent_buffer *left = NULL;
+ struct extent_buffer *parent = NULL;
+ int ret = 0;
+ int wret;
+ int pslot;
+ int orig_slot = path->slots[level];
+ u64 orig_ptr;
+
+ if (level == 0)
+ return 1;
+
+ mid = path->nodes[level];
+ WARN_ON(btrfs_header_generation(mid) != trans->transid);
+ orig_ptr = btrfs_node_blockptr(mid, orig_slot);
+
+ if (level < BTRFS_MAX_LEVEL - 1)
+ parent = path->nodes[level + 1];
+ pslot = path->slots[level + 1];
+
+ if (!parent)
+ return 1;
+
+ left = read_node_slot(root, parent, pslot - 1);
+
+ /* first, try to make some room in the middle buffer */
+ if (left) {
+ u32 left_nr;
+
+ btrfs_tree_lock(left);
+ left_nr = btrfs_header_nritems(left);
+ if (left_nr >= BTRFS_NODEPTRS_PER_BLOCK(root) - 1) {
+ wret = 1;
+ } else {
+ ret = btrfs_cow_block(trans, root, left, parent,
+ pslot - 1, &left, 0);
+ if (ret)
+ wret = 1;
+ else {
+ wret = push_node_left(trans, root,
+ left, mid, 0);
+ }
+ }
+ if (wret < 0)
+ ret = wret;
+ if (wret == 0) {
+ struct btrfs_disk_key disk_key;
+ orig_slot += left_nr;
+ btrfs_node_key(mid, &disk_key, 0);
+ btrfs_set_node_key(parent, &disk_key, pslot);
+ btrfs_mark_buffer_dirty(parent);
+ if (btrfs_header_nritems(left) > orig_slot) {
+ path->nodes[level] = left;
+ path->slots[level + 1] -= 1;
+ path->slots[level] = orig_slot;
+ btrfs_tree_unlock(mid);
+ free_extent_buffer(mid);
+ } else {
+ orig_slot -=
+ btrfs_header_nritems(left);
+ path->slots[level] = orig_slot;
+ btrfs_tree_unlock(left);
+ free_extent_buffer(left);
+ }
+ return 0;
+ }
+ btrfs_tree_unlock(left);
+ free_extent_buffer(left);
+ }
+ right = read_node_slot(root, parent, pslot + 1);
+
+ /*
+ * then try to empty the right most buffer into the middle
+ */
+ if (right) {
+ u32 right_nr;
+ btrfs_tree_lock(right);
+ right_nr = btrfs_header_nritems(right);
+ if (right_nr >= BTRFS_NODEPTRS_PER_BLOCK(root) - 1) {
+ wret = 1;
+ } else {
+ ret = btrfs_cow_block(trans, root, right,
+ parent, pslot + 1,
+ &right, 0);
+ if (ret)
+ wret = 1;
+ else {
+ wret = balance_node_right(trans, root,
+ right, mid);
+ }
+ }
+ if (wret < 0)
+ ret = wret;
+ if (wret == 0) {
+ struct btrfs_disk_key disk_key;
+
+ btrfs_node_key(right, &disk_key, 0);
+ btrfs_set_node_key(parent, &disk_key, pslot + 1);
+ btrfs_mark_buffer_dirty(parent);
+
+ if (btrfs_header_nritems(mid) <= orig_slot) {
+ path->nodes[level] = right;
+ path->slots[level + 1] += 1;
+ path->slots[level] = orig_slot -
+ btrfs_header_nritems(mid);
+ btrfs_tree_unlock(mid);
+ free_extent_buffer(mid);
+ } else {
+ btrfs_tree_unlock(right);
+ free_extent_buffer(right);
+ }
+ return 0;
+ }
+ btrfs_tree_unlock(right);
+ free_extent_buffer(right);
+ }
+ return 1;
+}
+
+/*
+ * readahead one full node of leaves, finding things that are close
+ * to the block in 'slot', and triggering ra on them.
+ */
+static noinline void reada_for_search(struct btrfs_root *root,
+ struct btrfs_path *path,
+ int level, int slot, u64 objectid)
+{
+ struct extent_buffer *node;
+ struct btrfs_disk_key disk_key;
+ u32 nritems;
+ u64 search;
+ u64 lowest_read;
+ u64 highest_read;
+ u64 nread = 0;
+ int direction = path->reada;
+ struct extent_buffer *eb;
+ u32 nr;
+ u32 blocksize;
+ u32 nscan = 0;
+
+ if (level != 1)
+ return;
+
+ if (!path->nodes[level])
+ return;
+
+ node = path->nodes[level];
+
+ search = btrfs_node_blockptr(node, slot);
+ blocksize = btrfs_level_size(root, level - 1);
+ eb = btrfs_find_tree_block(root, search, blocksize);
+ if (eb) {
+ free_extent_buffer(eb);
+ return;
+ }
+
+ highest_read = search;
+ lowest_read = search;
+
+ nritems = btrfs_header_nritems(node);
+ nr = slot;
+ while (1) {
+ if (direction < 0) {
+ if (nr == 0)
+ break;
+ nr--;
+ } else if (direction > 0) {
+ nr++;
+ if (nr >= nritems)
+ break;
+ }
+ if (path->reada < 0 && objectid) {
+ btrfs_node_key(node, &disk_key, nr);
+ if (btrfs_disk_key_objectid(&disk_key) != objectid)
+ break;
+ }
+ search = btrfs_node_blockptr(node, nr);
+ if ((search >= lowest_read && search <= highest_read) ||
+ (search < lowest_read && lowest_read - search <= 16384) ||
+ (search > highest_read && search - highest_read <= 16384)) {
+ readahead_tree_block(root, search, blocksize,
+ btrfs_node_ptr_generation(node, nr));
+ nread += blocksize;
+ }
+ nscan++;
+ if (path->reada < 2 && (nread > (64 * 1024) || nscan > 32))
+ break;
+
+ if (nread > (256 * 1024) || nscan > 128)
+ break;
+
+ if (search < lowest_read)
+ lowest_read = search;
+ if (search > highest_read)
+ highest_read = search;
+ }
+}
+
+/*
+ * when we walk down the tree, it is usually safe to unlock the higher layers
+ * in the tree. The exceptions are when our path goes through slot 0, because
+ * operations on the tree might require changing key pointers higher up in the
+ * tree.
+ *
+ * callers might also have set path->keep_locks, which tells this code to keep
+ * the lock if the path points to the last slot in the block. This is part of
+ * walking through the tree, and selecting the next slot in the higher block.
+ *
+ * lowest_unlock sets the lowest level in the tree we're allowed to unlock. so
+ * if lowest_unlock is 1, level 0 won't be unlocked
+ */
+static noinline void unlock_up(struct btrfs_path *path, int level,
+ int lowest_unlock)
+{
+ int i;
+ int skip_level = level;
+ int no_skips = 0;
+ struct extent_buffer *t;
+
+ for (i = level; i < BTRFS_MAX_LEVEL; i++) {
+ if (!path->nodes[i])
+ break;
+ if (!path->locks[i])
+ break;
+ if (!no_skips && path->slots[i] == 0) {
+ skip_level = i + 1;
+ continue;
+ }
+ if (!no_skips && path->keep_locks) {
+ u32 nritems;
+ t = path->nodes[i];
+ nritems = btrfs_header_nritems(t);
+ if (nritems < 1 || path->slots[i] >= nritems - 1) {
+ skip_level = i + 1;
+ continue;
+ }
+ }
+ if (skip_level < i && i >= lowest_unlock)
+ no_skips = 1;
+
+ t = path->nodes[i];
+ if (i >= lowest_unlock && i > skip_level && path->locks[i]) {
+ btrfs_tree_unlock(t);
+ path->locks[i] = 0;
+ }
+ }
+}
+
+/*
+ * look for key in the tree. path is filled in with nodes along the way
+ * if key is found, we return zero and you can find the item in the leaf
+ * level of the path (level 0)
+ *
+ * If the key isn't found, the path points to the slot where it should
+ * be inserted, and 1 is returned. If there are other errors during the
+ * search a negative error number is returned.
+ *
+ * if ins_len > 0, nodes and leaves will be split as we walk down the
+ * tree. if ins_len < 0, nodes will be merged as we walk down the tree (if
+ * possible)
+ */
+int btrfs_search_slot(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root, struct btrfs_key *key, struct btrfs_path *p, int
+ ins_len, int cow)
+{
+ struct extent_buffer *b;
+ struct extent_buffer *tmp;
+ int slot;
+ int ret;
+ int level;
+ int should_reada = p->reada;
+ int lowest_unlock = 1;
+ int blocksize;
+ u8 lowest_level = 0;
+ u64 blocknr;
+ u64 gen;
+ struct btrfs_key prealloc_block;
+
+ lowest_level = p->lowest_level;
+ WARN_ON(lowest_level && ins_len > 0);
+ WARN_ON(p->nodes[0] != NULL);
+
+ if (ins_len < 0)
+ lowest_unlock = 2;
+
+ prealloc_block.objectid = 0;
+
+again:
+ if (p->skip_locking)
+ b = btrfs_root_node(root);
+ else
+ b = btrfs_lock_root_node(root);
+
+ while (b) {
+ level = btrfs_header_level(b);
+
+ /*
+ * setup the path here so we can release it under lock
+ * contention with the cow code
+ */
+ p->nodes[level] = b;
+ if (!p->skip_locking)
+ p->locks[level] = 1;
+
+ if (cow) {
+ int wret;
+
+ /* is a cow on this block not required */
+ spin_lock(&root->fs_info->hash_lock);
+ if (btrfs_header_generation(b) == trans->transid &&
+ btrfs_header_owner(b) == root->root_key.objectid &&
+ !btrfs_header_flag(b, BTRFS_HEADER_FLAG_WRITTEN)) {
+ spin_unlock(&root->fs_info->hash_lock);
+ goto cow_done;
+ }
+ spin_unlock(&root->fs_info->hash_lock);
+
+ /* ok, we have to cow, is our old prealloc the right
+ * size?
+ */
+ if (prealloc_block.objectid &&
+ prealloc_block.offset != b->len) {
+ btrfs_free_reserved_extent(root,
+ prealloc_block.objectid,
+ prealloc_block.offset);
+ prealloc_block.objectid = 0;
+ }
+
+ /*
+ * for higher level blocks, try not to allocate blocks
+ * with the block and the parent locks held.
+ */
+ if (level > 1 && !prealloc_block.objectid &&
+ btrfs_path_lock_waiting(p, level)) {
+ u32 size = b->len;
+ u64 hint = b->start;
+
+ btrfs_release_path(root, p);
+ ret = btrfs_reserve_extent(trans, root,
+ size, size, 0,
+ hint, (u64)-1,
+ &prealloc_block, 0);
+ BUG_ON(ret);
+ goto again;
+ }
+
+ wret = btrfs_cow_block(trans, root, b,
+ p->nodes[level + 1],
+ p->slots[level + 1],
+ &b, prealloc_block.objectid);
+ prealloc_block.objectid = 0;
+ if (wret) {
+ free_extent_buffer(b);
+ ret = wret;
+ goto done;
+ }
+ }
+cow_done:
+ BUG_ON(!cow && ins_len);
+ if (level != btrfs_header_level(b))
+ WARN_ON(1);
+ level = btrfs_header_level(b);
+
+ p->nodes[level] = b;
+ if (!p->skip_locking)
+ p->locks[level] = 1;
+
+ ret = check_block(root, p, level);
+ if (ret) {
+ ret = -1;
+ goto done;
+ }
+
+ ret = bin_search(b, key, level, &slot);
+ if (level != 0) {
+ if (ret && slot > 0)
+ slot -= 1;
+ p->slots[level] = slot;
+ if ((p->search_for_split || ins_len > 0) &&
+ btrfs_header_nritems(b) >=
+ BTRFS_NODEPTRS_PER_BLOCK(root) - 3) {
+ int sret = split_node(trans, root, p, level);
+ BUG_ON(sret > 0);
+ if (sret) {
+ ret = sret;
+ goto done;
+ }
+ b = p->nodes[level];
+ slot = p->slots[level];
+ } else if (ins_len < 0) {
+ int sret = balance_level(trans, root, p,
+ level);
+ if (sret) {
+ ret = sret;
+ goto done;
+ }
+ b = p->nodes[level];
+ if (!b) {
+ btrfs_release_path(NULL, p);
+ goto again;
+ }
+ slot = p->slots[level];
+ BUG_ON(btrfs_header_nritems(b) == 1);
+ }
+ unlock_up(p, level, lowest_unlock);
+
+ /* this is only true while dropping a snapshot */
+ if (level == lowest_level) {
+ ret = 0;
+ goto done;
+ }
+
+ blocknr = btrfs_node_blockptr(b, slot);
+ gen = btrfs_node_ptr_generation(b, slot);
+ blocksize = btrfs_level_size(root, level - 1);
+
+ tmp = btrfs_find_tree_block(root, blocknr, blocksize);
+ if (tmp && btrfs_buffer_uptodate(tmp, gen)) {
+ b = tmp;
+ } else {
+ /*
+ * reduce lock contention at high levels
+ * of the btree by dropping locks before
+ * we read.
+ */
+ if (level > 1) {
+ btrfs_release_path(NULL, p);
+ if (tmp)
+ free_extent_buffer(tmp);
+ if (should_reada)
+ reada_for_search(root, p,
+ level, slot,
+ key->objectid);
+
+ tmp = read_tree_block(root, blocknr,
+ blocksize, gen);
+ if (tmp)
+ free_extent_buffer(tmp);
+ goto again;
+ } else {
+ if (tmp)
+ free_extent_buffer(tmp);
+ if (should_reada)
+ reada_for_search(root, p,
+ level, slot,
+ key->objectid);
+ b = read_node_slot(root, b, slot);
+ }
+ }
+ if (!p->skip_locking)
+ btrfs_tree_lock(b);
+ } else {
+ p->slots[level] = slot;
+ if (ins_len > 0 &&
+ btrfs_leaf_free_space(root, b) < ins_len) {
+ int sret = split_leaf(trans, root, key,
+ p, ins_len, ret == 0);
+ BUG_ON(sret > 0);
+ if (sret) {
+ ret = sret;
+ goto done;
+ }
+ }
+ if (!p->search_for_split)
+ unlock_up(p, level, lowest_unlock);
+ goto done;
+ }
+ }
+ ret = 1;
+done:
+ if (prealloc_block.objectid) {
+ btrfs_free_reserved_extent(root,
+ prealloc_block.objectid,
+ prealloc_block.offset);
+ }
+
+ return ret;
+}
+
+int btrfs_merge_path(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_key *node_keys,
+ u64 *nodes, int lowest_level)
+{
+ struct extent_buffer *eb;
+ struct extent_buffer *parent;
+ struct btrfs_key key;
+ u64 bytenr;
+ u64 generation;
+ u32 blocksize;
+ int level;
+ int slot;
+ int key_match;
+ int ret;
+
+ eb = btrfs_lock_root_node(root);
+ ret = btrfs_cow_block(trans, root, eb, NULL, 0, &eb, 0);
+ BUG_ON(ret);
+
+ parent = eb;
+ while (1) {
+ level = btrfs_header_level(parent);
+ if (level == 0 || level <= lowest_level)
+ break;
+
+ ret = bin_search(parent, &node_keys[lowest_level], level,
+ &slot);
+ if (ret && slot > 0)
+ slot--;
+
+ bytenr = btrfs_node_blockptr(parent, slot);
+ if (nodes[level - 1] == bytenr)
+ break;
+
+ blocksize = btrfs_level_size(root, level - 1);
+ generation = btrfs_node_ptr_generation(parent, slot);
+ btrfs_node_key_to_cpu(eb, &key, slot);
+ key_match = !memcmp(&key, &node_keys[level - 1], sizeof(key));
+
+ if (generation == trans->transid) {
+ eb = read_tree_block(root, bytenr, blocksize,
+ generation);
+ btrfs_tree_lock(eb);
+ }
+
+ /*
+ * if node keys match and node pointer hasn't been modified
+ * in the running transaction, we can merge the path. for
+ * blocks owened by reloc trees, the node pointer check is
+ * skipped, this is because these blocks are fully controlled
+ * by the space balance code, no one else can modify them.
+ */
+ if (!nodes[level - 1] || !key_match ||
+ (generation == trans->transid &&
+ btrfs_header_owner(eb) != BTRFS_TREE_RELOC_OBJECTID)) {
+ if (level == 1 || level == lowest_level + 1) {
+ if (generation == trans->transid) {
+ btrfs_tree_unlock(eb);
+ free_extent_buffer(eb);
+ }
+ break;
+ }
+
+ if (generation != trans->transid) {
+ eb = read_tree_block(root, bytenr, blocksize,
+ generation);
+ btrfs_tree_lock(eb);
+ }
+
+ ret = btrfs_cow_block(trans, root, eb, parent, slot,
+ &eb, 0);
+ BUG_ON(ret);
+
+ if (root->root_key.objectid ==
+ BTRFS_TREE_RELOC_OBJECTID) {
+ if (!nodes[level - 1]) {
+ nodes[level - 1] = eb->start;
+ memcpy(&node_keys[level - 1], &key,
+ sizeof(node_keys[0]));
+ } else {
+ WARN_ON(1);
+ }
+ }
+
+ btrfs_tree_unlock(parent);
+ free_extent_buffer(parent);
+ parent = eb;
+ continue;
+ }
+
+ btrfs_set_node_blockptr(parent, slot, nodes[level - 1]);
+ btrfs_set_node_ptr_generation(parent, slot, trans->transid);
+ btrfs_mark_buffer_dirty(parent);
+
+ ret = btrfs_inc_extent_ref(trans, root,
+ nodes[level - 1],
+ blocksize, parent->start,
+ btrfs_header_owner(parent),
+ btrfs_header_generation(parent),
+ level - 1);
+ BUG_ON(ret);
+
+ /*
+ * If the block was created in the running transaction,
+ * it's possible this is the last reference to it, so we
+ * should drop the subtree.
+ */
+ if (generation == trans->transid) {
+ ret = btrfs_drop_subtree(trans, root, eb, parent);
+ BUG_ON(ret);
+ btrfs_tree_unlock(eb);
+ free_extent_buffer(eb);
+ } else {
+ ret = btrfs_free_extent(trans, root, bytenr,
+ blocksize, parent->start,
+ btrfs_header_owner(parent),
+ btrfs_header_generation(parent),
+ level - 1, 1);
+ BUG_ON(ret);
+ }
+ break;
+ }
+ btrfs_tree_unlock(parent);
+ free_extent_buffer(parent);
+ return 0;
+}
+
+/*
+ * adjust the pointers going up the tree, starting at level
+ * making sure the right key of each node is points to 'key'.
+ * This is used after shifting pointers to the left, so it stops
+ * fixing up pointers when a given leaf/node is not in slot 0 of the
+ * higher levels
+ *
+ * If this fails to write a tree block, it returns -1, but continues
+ * fixing up the blocks in ram so the tree is consistent.
+ */
+static int fixup_low_keys(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct btrfs_path *path,
+ struct btrfs_disk_key *key, int level)
+{
+ int i;
+ int ret = 0;
+ struct extent_buffer *t;
+
+ for (i = level; i < BTRFS_MAX_LEVEL; i++) {
+ int tslot = path->slots[i];
+ if (!path->nodes[i])
+ break;
+ t = path->nodes[i];
+ btrfs_set_node_key(t, key, tslot);
+ btrfs_mark_buffer_dirty(path->nodes[i]);
+ if (tslot != 0)
+ break;
+ }
+ return ret;
+}
+
+/*
+ * update item key.
+ *
+ * This function isn't completely safe. It's the caller's responsibility
+ * that the new key won't break the order
+ */
+int btrfs_set_item_key_safe(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct btrfs_path *path,
+ struct btrfs_key *new_key)
+{
+ struct btrfs_disk_key disk_key;
+ struct extent_buffer *eb;
+ int slot;
+
+ eb = path->nodes[0];
+ slot = path->slots[0];
+ if (slot > 0) {
+ btrfs_item_key(eb, &disk_key, slot - 1);
+ if (comp_keys(&disk_key, new_key) >= 0)
+ return -1;
+ }
+ if (slot < btrfs_header_nritems(eb) - 1) {
+ btrfs_item_key(eb, &disk_key, slot + 1);
+ if (comp_keys(&disk_key, new_key) <= 0)
+ return -1;
+ }
+
+ btrfs_cpu_key_to_disk(&disk_key, new_key);
+ btrfs_set_item_key(eb, &disk_key, slot);
+ btrfs_mark_buffer_dirty(eb);
+ if (slot == 0)
+ fixup_low_keys(trans, root, path, &disk_key, 1);
+ return 0;
+}
+
+/*
+ * try to push data from one node into the next node left in the
+ * tree.
+ *
+ * returns 0 if some ptrs were pushed left, < 0 if there was some horrible
+ * error, and > 0 if there was no room in the left hand block.
+ */
+static int push_node_left(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct extent_buffer *dst,
+ struct extent_buffer *src, int empty)
+{
+ int push_items = 0;
+ int src_nritems;
+ int dst_nritems;
+ int ret = 0;
+
+ src_nritems = btrfs_header_nritems(src);
+ dst_nritems = btrfs_header_nritems(dst);
+ push_items = BTRFS_NODEPTRS_PER_BLOCK(root) - dst_nritems;
+ WARN_ON(btrfs_header_generation(src) != trans->transid);
+ WARN_ON(btrfs_header_generation(dst) != trans->transid);
+
+ if (!empty && src_nritems <= 8)
+ return 1;
+
+ if (push_items <= 0)
+ return 1;
+
+ if (empty) {
+ push_items = min(src_nritems, push_items);
+ if (push_items < src_nritems) {
+ /* leave at least 8 pointers in the node if
+ * we aren't going to empty it
+ */
+ if (src_nritems - push_items < 8) {
+ if (push_items <= 8)
+ return 1;
+ push_items -= 8;
+ }
+ }
+ } else
+ push_items = min(src_nritems - 8, push_items);
+
+ copy_extent_buffer(dst, src,
+ btrfs_node_key_ptr_offset(dst_nritems),
+ btrfs_node_key_ptr_offset(0),
+ push_items * sizeof(struct btrfs_key_ptr));
+
+ if (push_items < src_nritems) {
+ memmove_extent_buffer(src, btrfs_node_key_ptr_offset(0),
+ btrfs_node_key_ptr_offset(push_items),
+ (src_nritems - push_items) *
+ sizeof(struct btrfs_key_ptr));
+ }
+ btrfs_set_header_nritems(src, src_nritems - push_items);
+ btrfs_set_header_nritems(dst, dst_nritems + push_items);
+ btrfs_mark_buffer_dirty(src);
+ btrfs_mark_buffer_dirty(dst);
+
+ ret = btrfs_update_ref(trans, root, src, dst, dst_nritems, push_items);
+ BUG_ON(ret);
+
+ return ret;
+}
+
+/*
+ * try to push data from one node into the next node right in the
+ * tree.
+ *
+ * returns 0 if some ptrs were pushed, < 0 if there was some horrible
+ * error, and > 0 if there was no room in the right hand block.
+ *
+ * this will only push up to 1/2 the contents of the left node over
+ */
+static int balance_node_right(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct extent_buffer *dst,
+ struct extent_buffer *src)
+{
+ int push_items = 0;
+ int max_push;
+ int src_nritems;
+ int dst_nritems;
+ int ret = 0;
+
+ WARN_ON(btrfs_header_generation(src) != trans->transid);
+ WARN_ON(btrfs_header_generation(dst) != trans->transid);
+
+ src_nritems = btrfs_header_nritems(src);
+ dst_nritems = btrfs_header_nritems(dst);
+ push_items = BTRFS_NODEPTRS_PER_BLOCK(root) - dst_nritems;
+ if (push_items <= 0)
+ return 1;
+
+ if (src_nritems < 4)
+ return 1;
+
+ max_push = src_nritems / 2 + 1;
+ /* don't try to empty the node */
+ if (max_push >= src_nritems)
+ return 1;
+
+ if (max_push < push_items)
+ push_items = max_push;
+
+ memmove_extent_buffer(dst, btrfs_node_key_ptr_offset(push_items),
+ btrfs_node_key_ptr_offset(0),
+ (dst_nritems) *
+ sizeof(struct btrfs_key_ptr));
+
+ copy_extent_buffer(dst, src,
+ btrfs_node_key_ptr_offset(0),
+ btrfs_node_key_ptr_offset(src_nritems - push_items),
+ push_items * sizeof(struct btrfs_key_ptr));
+
+ btrfs_set_header_nritems(src, src_nritems - push_items);
+ btrfs_set_header_nritems(dst, dst_nritems + push_items);
+
+ btrfs_mark_buffer_dirty(src);
+ btrfs_mark_buffer_dirty(dst);
+
+ ret = btrfs_update_ref(trans, root, src, dst, 0, push_items);
+ BUG_ON(ret);
+
+ return ret;
+}
+
+/*
+ * helper function to insert a new root level in the tree.
+ * A new node is allocated, and a single item is inserted to
+ * point to the existing root
+ *
+ * returns zero on success or < 0 on failure.
+ */
+static noinline int insert_new_root(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, int level)
+{
+ u64 lower_gen;
+ struct extent_buffer *lower;
+ struct extent_buffer *c;
+ struct extent_buffer *old;
+ struct btrfs_disk_key lower_key;
+ int ret;
+
+ BUG_ON(path->nodes[level]);
+ BUG_ON(path->nodes[level-1] != root->node);
+
+ lower = path->nodes[level-1];
+ if (level == 1)
+ btrfs_item_key(lower, &lower_key, 0);
+ else
+ btrfs_node_key(lower, &lower_key, 0);
+
+ c = btrfs_alloc_free_block(trans, root, root->nodesize, 0,
+ root->root_key.objectid, trans->transid,
+ level, root->node->start, 0);
+ if (IS_ERR(c))
+ return PTR_ERR(c);
+
+ memset_extent_buffer(c, 0, 0, root->nodesize);
+ btrfs_set_header_nritems(c, 1);
+ btrfs_set_header_level(c, level);
+ btrfs_set_header_bytenr(c, c->start);
+ btrfs_set_header_generation(c, trans->transid);
+ btrfs_set_header_owner(c, root->root_key.objectid);
+
+ write_extent_buffer(c, root->fs_info->fsid,
+ (unsigned long)btrfs_header_fsid(c),
+ BTRFS_FSID_SIZE);
+
+ write_extent_buffer(c, root->fs_info->chunk_tree_uuid,
+ (unsigned long)btrfs_header_chunk_tree_uuid(c),
+ BTRFS_UUID_SIZE);
+
+ btrfs_set_node_key(c, &lower_key, 0);
+ btrfs_set_node_blockptr(c, 0, lower->start);
+ lower_gen = btrfs_header_generation(lower);
+ WARN_ON(lower_gen != trans->transid);
+
+ btrfs_set_node_ptr_generation(c, 0, lower_gen);
+
+ btrfs_mark_buffer_dirty(c);
+
+ spin_lock(&root->node_lock);
+ old = root->node;
+ root->node = c;
+ spin_unlock(&root->node_lock);
+
+ ret = btrfs_update_extent_ref(trans, root, lower->start,
+ lower->start, c->start,
+ root->root_key.objectid,
+ trans->transid, level - 1);
+ BUG_ON(ret);
+
+ /* the super has an extra ref to root->node */
+ free_extent_buffer(old);
+
+ add_root_to_dirty_list(root);
+ extent_buffer_get(c);
+ path->nodes[level] = c;
+ path->locks[level] = 1;
+ path->slots[level] = 0;
+ return 0;
+}
+
+/*
+ * worker function to insert a single pointer in a node.
+ * the node should have enough room for the pointer already
+ *
+ * slot and level indicate where you want the key to go, and
+ * blocknr is the block the key points to.
+ *
+ * returns zero on success and < 0 on any error
+ */
+static int insert_ptr(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root, struct btrfs_path *path, struct btrfs_disk_key
+ *key, u64 bytenr, int slot, int level)
+{
+ struct extent_buffer *lower;
+ int nritems;
+
+ BUG_ON(!path->nodes[level]);
+ lower = path->nodes[level];
+ nritems = btrfs_header_nritems(lower);
+ if (slot > nritems)
+ BUG();
+ if (nritems == BTRFS_NODEPTRS_PER_BLOCK(root))
+ BUG();
+ if (slot != nritems) {
+ memmove_extent_buffer(lower,
+ btrfs_node_key_ptr_offset(slot + 1),
+ btrfs_node_key_ptr_offset(slot),
+ (nritems - slot) * sizeof(struct btrfs_key_ptr));
+ }
+ btrfs_set_node_key(lower, key, slot);
+ btrfs_set_node_blockptr(lower, slot, bytenr);
+ WARN_ON(trans->transid == 0);
+ btrfs_set_node_ptr_generation(lower, slot, trans->transid);
+ btrfs_set_header_nritems(lower, nritems + 1);
+ btrfs_mark_buffer_dirty(lower);
+ return 0;
+}
+
+/*
+ * split the node at the specified level in path in two.
+ * The path is corrected to point to the appropriate node after the split
+ *
+ * Before splitting this tries to make some room in the node by pushing
+ * left and right, if either one works, it returns right away.
+ *
+ * returns 0 on success and < 0 on failure
+ */
+static noinline int split_node(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, int level)
+{
+ struct extent_buffer *c;
+ struct extent_buffer *split;
+ struct btrfs_disk_key disk_key;
+ int mid;
+ int ret;
+ int wret;
+ u32 c_nritems;
+
+ c = path->nodes[level];
+ WARN_ON(btrfs_header_generation(c) != trans->transid);
+ if (c == root->node) {
+ /* trying to split the root, lets make a new one */
+ ret = insert_new_root(trans, root, path, level + 1);
+ if (ret)
+ return ret;
+ } else {
+ ret = push_nodes_for_insert(trans, root, path, level);
+ c = path->nodes[level];
+ if (!ret && btrfs_header_nritems(c) <
+ BTRFS_NODEPTRS_PER_BLOCK(root) - 3)
+ return 0;
+ if (ret < 0)
+ return ret;
+ }
+
+ c_nritems = btrfs_header_nritems(c);
+
+ split = btrfs_alloc_free_block(trans, root, root->nodesize,
+ path->nodes[level + 1]->start,
+ root->root_key.objectid,
+ trans->transid, level, c->start, 0);
+ if (IS_ERR(split))
+ return PTR_ERR(split);
+
+ btrfs_set_header_flags(split, btrfs_header_flags(c));
+ btrfs_set_header_level(split, btrfs_header_level(c));
+ btrfs_set_header_bytenr(split, split->start);
+ btrfs_set_header_generation(split, trans->transid);
+ btrfs_set_header_owner(split, root->root_key.objectid);
+ btrfs_set_header_flags(split, 0);
+ write_extent_buffer(split, root->fs_info->fsid,
+ (unsigned long)btrfs_header_fsid(split),
+ BTRFS_FSID_SIZE);
+ write_extent_buffer(split, root->fs_info->chunk_tree_uuid,
+ (unsigned long)btrfs_header_chunk_tree_uuid(split),
+ BTRFS_UUID_SIZE);
+
+ mid = (c_nritems + 1) / 2;
+
+ copy_extent_buffer(split, c,
+ btrfs_node_key_ptr_offset(0),
+ btrfs_node_key_ptr_offset(mid),
+ (c_nritems - mid) * sizeof(struct btrfs_key_ptr));
+ btrfs_set_header_nritems(split, c_nritems - mid);
+ btrfs_set_header_nritems(c, mid);
+ ret = 0;
+
+ btrfs_mark_buffer_dirty(c);
+ btrfs_mark_buffer_dirty(split);
+
+ btrfs_node_key(split, &disk_key, 0);
+ wret = insert_ptr(trans, root, path, &disk_key, split->start,
+ path->slots[level + 1] + 1,
+ level + 1);
+ if (wret)
+ ret = wret;
+
+ ret = btrfs_update_ref(trans, root, c, split, 0, c_nritems - mid);
+ BUG_ON(ret);
+
+ if (path->slots[level] >= mid) {
+ path->slots[level] -= mid;
+ btrfs_tree_unlock(c);
+ free_extent_buffer(c);
+ path->nodes[level] = split;
+ path->slots[level + 1] += 1;
+ } else {
+ btrfs_tree_unlock(split);
+ free_extent_buffer(split);
+ }
+ return ret;
+}
+
+/*
+ * how many bytes are required to store the items in a leaf. start
+ * and nr indicate which items in the leaf to check. This totals up the
+ * space used both by the item structs and the item data
+ */
+static int leaf_space_used(struct extent_buffer *l, int start, int nr)
+{
+ int data_len;
+ int nritems = btrfs_header_nritems(l);
+ int end = min(nritems, start + nr) - 1;
+
+ if (!nr)
+ return 0;
+ data_len = btrfs_item_end_nr(l, start);
+ data_len = data_len - btrfs_item_offset_nr(l, end);
+ data_len += sizeof(struct btrfs_item) * nr;
+ WARN_ON(data_len < 0);
+ return data_len;
+}
+
+/*
+ * The space between the end of the leaf items and
+ * the start of the leaf data. IOW, how much room
+ * the leaf has left for both items and data
+ */
+noinline int btrfs_leaf_free_space(struct btrfs_root *root,
+ struct extent_buffer *leaf)
+{
+ int nritems = btrfs_header_nritems(leaf);
+ int ret;
+ ret = BTRFS_LEAF_DATA_SIZE(root) - leaf_space_used(leaf, 0, nritems);
+ if (ret < 0) {
+ printk(KERN_CRIT "leaf free space ret %d, leaf data size %lu, "
+ "used %d nritems %d\n",
+ ret, (unsigned long) BTRFS_LEAF_DATA_SIZE(root),
+ leaf_space_used(leaf, 0, nritems), nritems);
+ }
+ return ret;
+}
+
+/*
+ * push some data in the path leaf to the right, trying to free up at
+ * least data_size bytes. returns zero if the push worked, nonzero otherwise
+ *
+ * returns 1 if the push failed because the other node didn't have enough
+ * room, 0 if everything worked out and < 0 if there were major errors.
+ */
+static int push_leaf_right(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root, struct btrfs_path *path, int data_size,
+ int empty)
+{
+ struct extent_buffer *left = path->nodes[0];
+ struct extent_buffer *right;
+ struct extent_buffer *upper;
+ struct btrfs_disk_key disk_key;
+ int slot;
+ u32 i;
+ int free_space;
+ int push_space = 0;
+ int push_items = 0;
+ struct btrfs_item *item;
+ u32 left_nritems;
+ u32 nr;
+ u32 right_nritems;
+ u32 data_end;
+ u32 this_item_size;
+ int ret;
+
+ slot = path->slots[1];
+ if (!path->nodes[1])
+ return 1;
+
+ upper = path->nodes[1];
+ if (slot >= btrfs_header_nritems(upper) - 1)
+ return 1;
+
+ WARN_ON(!btrfs_tree_locked(path->nodes[1]));
+
+ right = read_node_slot(root, upper, slot + 1);
+ btrfs_tree_lock(right);
+ free_space = btrfs_leaf_free_space(root, right);
+ if (free_space < data_size)
+ goto out_unlock;
+
+ /* cow and double check */
+ ret = btrfs_cow_block(trans, root, right, upper,
+ slot + 1, &right, 0);
+ if (ret)
+ goto out_unlock;
+
+ free_space = btrfs_leaf_free_space(root, right);
+ if (free_space < data_size)
+ goto out_unlock;
+
+ left_nritems = btrfs_header_nritems(left);
+ if (left_nritems == 0)
+ goto out_unlock;
+
+ if (empty)
+ nr = 0;
+ else
+ nr = 1;
+
+ if (path->slots[0] >= left_nritems)
+ push_space += data_size;
+
+ i = left_nritems - 1;
+ while (i >= nr) {
+ item = btrfs_item_nr(left, i);
+
+ if (!empty && push_items > 0) {
+ if (path->slots[0] > i)
+ break;
+ if (path->slots[0] == i) {
+ int space = btrfs_leaf_free_space(root, left);
+ if (space + push_space * 2 > free_space)
+ break;
+ }
+ }
+
+ if (path->slots[0] == i)
+ push_space += data_size;
+
+ if (!left->map_token) {
+ map_extent_buffer(left, (unsigned long)item,
+ sizeof(struct btrfs_item),
+ &left->map_token, &left->kaddr,
+ &left->map_start, &left->map_len,
+ KM_USER1);
+ }
+
+ this_item_size = btrfs_item_size(left, item);
+ if (this_item_size + sizeof(*item) + push_space > free_space)
+ break;
+
+ push_items++;
+ push_space += this_item_size + sizeof(*item);
+ if (i == 0)
+ break;
+ i--;
+ }
+ if (left->map_token) {
+ unmap_extent_buffer(left, left->map_token, KM_USER1);
+ left->map_token = NULL;
+ }
+
+ if (push_items == 0)
+ goto out_unlock;
+
+ if (!empty && push_items == left_nritems)
+ WARN_ON(1);
+
+ /* push left to right */
+ right_nritems = btrfs_header_nritems(right);
+
+ push_space = btrfs_item_end_nr(left, left_nritems - push_items);
+ push_space -= leaf_data_end(root, left);
+
+ /* make room in the right data area */
+ data_end = leaf_data_end(root, right);
+ memmove_extent_buffer(right,
+ btrfs_leaf_data(right) + data_end - push_space,
+ btrfs_leaf_data(right) + data_end,
+ BTRFS_LEAF_DATA_SIZE(root) - data_end);
+
+ /* copy from the left data area */
+ copy_extent_buffer(right, left, btrfs_leaf_data(right) +
+ BTRFS_LEAF_DATA_SIZE(root) - push_space,
+ btrfs_leaf_data(left) + leaf_data_end(root, left),
+ push_space);
+
+ memmove_extent_buffer(right, btrfs_item_nr_offset(push_items),
+ btrfs_item_nr_offset(0),
+ right_nritems * sizeof(struct btrfs_item));
+
+ /* copy the items from left to right */
+ copy_extent_buffer(right, left, btrfs_item_nr_offset(0),
+ btrfs_item_nr_offset(left_nritems - push_items),
+ push_items * sizeof(struct btrfs_item));
+
+ /* update the item pointers */
+ right_nritems += push_items;
+ btrfs_set_header_nritems(right, right_nritems);
+ push_space = BTRFS_LEAF_DATA_SIZE(root);
+ for (i = 0; i < right_nritems; i++) {
+ item = btrfs_item_nr(right, i);
+ if (!right->map_token) {
+ map_extent_buffer(right, (unsigned long)item,
+ sizeof(struct btrfs_item),
+ &right->map_token, &right->kaddr,
+ &right->map_start, &right->map_len,
+ KM_USER1);
+ }
+ push_space -= btrfs_item_size(right, item);
+ btrfs_set_item_offset(right, item, push_space);
+ }
+
+ if (right->map_token) {
+ unmap_extent_buffer(right, right->map_token, KM_USER1);
+ right->map_token = NULL;
+ }
+ left_nritems -= push_items;
+ btrfs_set_header_nritems(left, left_nritems);
+
+ if (left_nritems)
+ btrfs_mark_buffer_dirty(left);
+ btrfs_mark_buffer_dirty(right);
+
+ ret = btrfs_update_ref(trans, root, left, right, 0, push_items);
+ BUG_ON(ret);
+
+ btrfs_item_key(right, &disk_key, 0);
+ btrfs_set_node_key(upper, &disk_key, slot + 1);
+ btrfs_mark_buffer_dirty(upper);
+
+ /* then fixup the leaf pointer in the path */
+ if (path->slots[0] >= left_nritems) {
+ path->slots[0] -= left_nritems;
+ if (btrfs_header_nritems(path->nodes[0]) == 0)
+ clean_tree_block(trans, root, path->nodes[0]);
+ btrfs_tree_unlock(path->nodes[0]);
+ free_extent_buffer(path->nodes[0]);
+ path->nodes[0] = right;
+ path->slots[1] += 1;
+ } else {
+ btrfs_tree_unlock(right);
+ free_extent_buffer(right);
+ }
+ return 0;
+
+out_unlock:
+ btrfs_tree_unlock(right);
+ free_extent_buffer(right);
+ return 1;
+}
+
+/*
+ * push some data in the path leaf to the left, trying to free up at
+ * least data_size bytes. returns zero if the push worked, nonzero otherwise
+ */
+static int push_leaf_left(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root, struct btrfs_path *path, int data_size,
+ int empty)
+{
+ struct btrfs_disk_key disk_key;
+ struct extent_buffer *right = path->nodes[0];
+ struct extent_buffer *left;
+ int slot;
+ int i;
+ int free_space;
+ int push_space = 0;
+ int push_items = 0;
+ struct btrfs_item *item;
+ u32 old_left_nritems;
+ u32 right_nritems;
+ u32 nr;
+ int ret = 0;
+ int wret;
+ u32 this_item_size;
+ u32 old_left_item_size;
+
+ slot = path->slots[1];
+ if (slot == 0)
+ return 1;
+ if (!path->nodes[1])
+ return 1;
+
+ right_nritems = btrfs_header_nritems(right);
+ if (right_nritems == 0)
+ return 1;
+
+ WARN_ON(!btrfs_tree_locked(path->nodes[1]));
+
+ left = read_node_slot(root, path->nodes[1], slot - 1);
+ btrfs_tree_lock(left);
+ free_space = btrfs_leaf_free_space(root, left);
+ if (free_space < data_size) {
+ ret = 1;
+ goto out;
+ }
+
+ /* cow and double check */
+ ret = btrfs_cow_block(trans, root, left,
+ path->nodes[1], slot - 1, &left, 0);
+ if (ret) {
+ /* we hit -ENOSPC, but it isn't fatal here */
+ ret = 1;
+ goto out;
+ }
+
+ free_space = btrfs_leaf_free_space(root, left);
+ if (free_space < data_size) {
+ ret = 1;
+ goto out;
+ }
+
+ if (empty)
+ nr = right_nritems;
+ else
+ nr = right_nritems - 1;
+
+ for (i = 0; i < nr; i++) {
+ item = btrfs_item_nr(right, i);
+ if (!right->map_token) {
+ map_extent_buffer(right, (unsigned long)item,
+ sizeof(struct btrfs_item),
+ &right->map_token, &right->kaddr,
+ &right->map_start, &right->map_len,
+ KM_USER1);
+ }
+
+ if (!empty && push_items > 0) {
+ if (path->slots[0] < i)
+ break;
+ if (path->slots[0] == i) {
+ int space = btrfs_leaf_free_space(root, right);
+ if (space + push_space * 2 > free_space)
+ break;
+ }
+ }
+
+ if (path->slots[0] == i)
+ push_space += data_size;
+
+ this_item_size = btrfs_item_size(right, item);
+ if (this_item_size + sizeof(*item) + push_space > free_space)
+ break;
+
+ push_items++;
+ push_space += this_item_size + sizeof(*item);
+ }
+
+ if (right->map_token) {
+ unmap_extent_buffer(right, right->map_token, KM_USER1);
+ right->map_token = NULL;
+ }
+
+ if (push_items == 0) {
+ ret = 1;
+ goto out;
+ }
+ if (!empty && push_items == btrfs_header_nritems(right))
+ WARN_ON(1);
+
+ /* push data from right to left */
+ copy_extent_buffer(left, right,
+ btrfs_item_nr_offset(btrfs_header_nritems(left)),
+ btrfs_item_nr_offset(0),
+ push_items * sizeof(struct btrfs_item));
+
+ push_space = BTRFS_LEAF_DATA_SIZE(root) -
+ btrfs_item_offset_nr(right, push_items - 1);
+
+ copy_extent_buffer(left, right, btrfs_leaf_data(left) +
+ leaf_data_end(root, left) - push_space,
+ btrfs_leaf_data(right) +
+ btrfs_item_offset_nr(right, push_items - 1),
+ push_space);
+ old_left_nritems = btrfs_header_nritems(left);
+ BUG_ON(old_left_nritems <= 0);
+
+ old_left_item_size = btrfs_item_offset_nr(left, old_left_nritems - 1);
+ for (i = old_left_nritems; i < old_left_nritems + push_items; i++) {
+ u32 ioff;
+
+ item = btrfs_item_nr(left, i);
+ if (!left->map_token) {
+ map_extent_buffer(left, (unsigned long)item,
+ sizeof(struct btrfs_item),
+ &left->map_token, &left->kaddr,
+ &left->map_start, &left->map_len,
+ KM_USER1);
+ }
+
+ ioff = btrfs_item_offset(left, item);
+ btrfs_set_item_offset(left, item,
+ ioff - (BTRFS_LEAF_DATA_SIZE(root) - old_left_item_size));
+ }
+ btrfs_set_header_nritems(left, old_left_nritems + push_items);
+ if (left->map_token) {
+ unmap_extent_buffer(left, left->map_token, KM_USER1);
+ left->map_token = NULL;
+ }
+
+ /* fixup right node */
+ if (push_items > right_nritems) {
+ printk(KERN_CRIT "push items %d nr %u\n", push_items,
+ right_nritems);
+ WARN_ON(1);
+ }
+
+ if (push_items < right_nritems) {
+ push_space = btrfs_item_offset_nr(right, push_items - 1) -
+ leaf_data_end(root, right);
+ memmove_extent_buffer(right, btrfs_leaf_data(right) +
+ BTRFS_LEAF_DATA_SIZE(root) - push_space,
+ btrfs_leaf_data(right) +
+ leaf_data_end(root, right), push_space);
+
+ memmove_extent_buffer(right, btrfs_item_nr_offset(0),
+ btrfs_item_nr_offset(push_items),
+ (btrfs_header_nritems(right) - push_items) *
+ sizeof(struct btrfs_item));
+ }
+ right_nritems -= push_items;
+ btrfs_set_header_nritems(right, right_nritems);
+ push_space = BTRFS_LEAF_DATA_SIZE(root);
+ for (i = 0; i < right_nritems; i++) {
+ item = btrfs_item_nr(right, i);
+
+ if (!right->map_token) {
+ map_extent_buffer(right, (unsigned long)item,
+ sizeof(struct btrfs_item),
+ &right->map_token, &right->kaddr,
+ &right->map_start, &right->map_len,
+ KM_USER1);
+ }
+
+ push_space = push_space - btrfs_item_size(right, item);
+ btrfs_set_item_offset(right, item, push_space);
+ }
+ if (right->map_token) {
+ unmap_extent_buffer(right, right->map_token, KM_USER1);
+ right->map_token = NULL;
+ }
+
+ btrfs_mark_buffer_dirty(left);
+ if (right_nritems)
+ btrfs_mark_buffer_dirty(right);
+
+ ret = btrfs_update_ref(trans, root, right, left,
+ old_left_nritems, push_items);
+ BUG_ON(ret);
+
+ btrfs_item_key(right, &disk_key, 0);
+ wret = fixup_low_keys(trans, root, path, &disk_key, 1);
+ if (wret)
+ ret = wret;
+
+ /* then fixup the leaf pointer in the path */
+ if (path->slots[0] < push_items) {
+ path->slots[0] += old_left_nritems;
+ if (btrfs_header_nritems(path->nodes[0]) == 0)
+ clean_tree_block(trans, root, path->nodes[0]);
+ btrfs_tree_unlock(path->nodes[0]);
+ free_extent_buffer(path->nodes[0]);
+ path->nodes[0] = left;
+ path->slots[1] -= 1;
+ } else {
+ btrfs_tree_unlock(left);
+ free_extent_buffer(left);
+ path->slots[0] -= push_items;
+ }
+ BUG_ON(path->slots[0] < 0);
+ return ret;
+out:
+ btrfs_tree_unlock(left);
+ free_extent_buffer(left);
+ return ret;
+}
+
+/*
+ * split the path's leaf in two, making sure there is at least data_size
+ * available for the resulting leaf level of the path.
+ *
+ * returns 0 if all went well and < 0 on failure.
+ */
+static noinline int split_leaf(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_key *ins_key,
+ struct btrfs_path *path, int data_size,
+ int extend)
+{
+ struct extent_buffer *l;
+ u32 nritems;
+ int mid;
+ int slot;
+ struct extent_buffer *right;
+ int data_copy_size;
+ int rt_data_off;
+ int i;
+ int ret = 0;
+ int wret;
+ int double_split;
+ int num_doubles = 0;
+ struct btrfs_disk_key disk_key;
+
+ /* first try to make some room by pushing left and right */
+ if (data_size && ins_key->type != BTRFS_DIR_ITEM_KEY) {
+ wret = push_leaf_right(trans, root, path, data_size, 0);
+ if (wret < 0)
+ return wret;
+ if (wret) {
+ wret = push_leaf_left(trans, root, path, data_size, 0);
+ if (wret < 0)
+ return wret;
+ }
+ l = path->nodes[0];
+
+ /* did the pushes work? */
+ if (btrfs_leaf_free_space(root, l) >= data_size)
+ return 0;
+ }
+
+ if (!path->nodes[1]) {
+ ret = insert_new_root(trans, root, path, 1);
+ if (ret)
+ return ret;
+ }
+again:
+ double_split = 0;
+ l = path->nodes[0];
+ slot = path->slots[0];
+ nritems = btrfs_header_nritems(l);
+ mid = (nritems + 1) / 2;
+
+ right = btrfs_alloc_free_block(trans, root, root->leafsize,
+ path->nodes[1]->start,
+ root->root_key.objectid,
+ trans->transid, 0, l->start, 0);
+ if (IS_ERR(right)) {
+ BUG_ON(1);
+ return PTR_ERR(right);
+ }
+
+ memset_extent_buffer(right, 0, 0, sizeof(struct btrfs_header));
+ btrfs_set_header_bytenr(right, right->start);
+ btrfs_set_header_generation(right, trans->transid);
+ btrfs_set_header_owner(right, root->root_key.objectid);
+ btrfs_set_header_level(right, 0);
+ write_extent_buffer(right, root->fs_info->fsid,
+ (unsigned long)btrfs_header_fsid(right),
+ BTRFS_FSID_SIZE);
+
+ write_extent_buffer(right, root->fs_info->chunk_tree_uuid,
+ (unsigned long)btrfs_header_chunk_tree_uuid(right),
+ BTRFS_UUID_SIZE);
+ if (mid <= slot) {
+ if (nritems == 1 ||
+ leaf_space_used(l, mid, nritems - mid) + data_size >
+ BTRFS_LEAF_DATA_SIZE(root)) {
+ if (slot >= nritems) {
+ btrfs_cpu_key_to_disk(&disk_key, ins_key);
+ btrfs_set_header_nritems(right, 0);
+ wret = insert_ptr(trans, root, path,
+ &disk_key, right->start,
+ path->slots[1] + 1, 1);
+ if (wret)
+ ret = wret;
+
+ btrfs_tree_unlock(path->nodes[0]);
+ free_extent_buffer(path->nodes[0]);
+ path->nodes[0] = right;
+ path->slots[0] = 0;
+ path->slots[1] += 1;
+ btrfs_mark_buffer_dirty(right);
+ return ret;
+ }
+ mid = slot;
+ if (mid != nritems &&
+ leaf_space_used(l, mid, nritems - mid) +
+ data_size > BTRFS_LEAF_DATA_SIZE(root)) {
+ double_split = 1;
+ }
+ }
+ } else {
+ if (leaf_space_used(l, 0, mid) + data_size >
+ BTRFS_LEAF_DATA_SIZE(root)) {
+ if (!extend && data_size && slot == 0) {
+ btrfs_cpu_key_to_disk(&disk_key, ins_key);
+ btrfs_set_header_nritems(right, 0);
+ wret = insert_ptr(trans, root, path,
+ &disk_key,
+ right->start,
+ path->slots[1], 1);
+ if (wret)
+ ret = wret;
+ btrfs_tree_unlock(path->nodes[0]);
+ free_extent_buffer(path->nodes[0]);
+ path->nodes[0] = right;
+ path->slots[0] = 0;
+ if (path->slots[1] == 0) {
+ wret = fixup_low_keys(trans, root,
+ path, &disk_key, 1);
+ if (wret)
+ ret = wret;
+ }
+ btrfs_mark_buffer_dirty(right);
+ return ret;
+ } else if ((extend || !data_size) && slot == 0) {
+ mid = 1;
+ } else {
+ mid = slot;
+ if (mid != nritems &&
+ leaf_space_used(l, mid, nritems - mid) +
+ data_size > BTRFS_LEAF_DATA_SIZE(root)) {
+ double_split = 1;
+ }
+ }
+ }
+ }
+ nritems = nritems - mid;
+ btrfs_set_header_nritems(right, nritems);
+ data_copy_size = btrfs_item_end_nr(l, mid) - leaf_data_end(root, l);
+
+ copy_extent_buffer(right, l, btrfs_item_nr_offset(0),
+ btrfs_item_nr_offset(mid),
+ nritems * sizeof(struct btrfs_item));
+
+ copy_extent_buffer(right, l,
+ btrfs_leaf_data(right) + BTRFS_LEAF_DATA_SIZE(root) -
+ data_copy_size, btrfs_leaf_data(l) +
+ leaf_data_end(root, l), data_copy_size);
+
+ rt_data_off = BTRFS_LEAF_DATA_SIZE(root) -
+ btrfs_item_end_nr(l, mid);
+
+ for (i = 0; i < nritems; i++) {
+ struct btrfs_item *item = btrfs_item_nr(right, i);
+ u32 ioff;
+
+ if (!right->map_token) {
+ map_extent_buffer(right, (unsigned long)item,
+ sizeof(struct btrfs_item),
+ &right->map_token, &right->kaddr,
+ &right->map_start, &right->map_len,
+ KM_USER1);
+ }
+
+ ioff = btrfs_item_offset(right, item);
+ btrfs_set_item_offset(right, item, ioff + rt_data_off);
+ }
+
+ if (right->map_token) {
+ unmap_extent_buffer(right, right->map_token, KM_USER1);
+ right->map_token = NULL;
+ }
+
+ btrfs_set_header_nritems(l, mid);
+ ret = 0;
+ btrfs_item_key(right, &disk_key, 0);
+ wret = insert_ptr(trans, root, path, &disk_key, right->start,
+ path->slots[1] + 1, 1);
+ if (wret)
+ ret = wret;
+
+ btrfs_mark_buffer_dirty(right);
+ btrfs_mark_buffer_dirty(l);
+ BUG_ON(path->slots[0] != slot);
+
+ ret = btrfs_update_ref(trans, root, l, right, 0, nritems);
+ BUG_ON(ret);
+
+ if (mid <= slot) {
+ btrfs_tree_unlock(path->nodes[0]);
+ free_extent_buffer(path->nodes[0]);
+ path->nodes[0] = right;
+ path->slots[0] -= mid;
+ path->slots[1] += 1;
+ } else {
+ btrfs_tree_unlock(right);
+ free_extent_buffer(right);
+ }
+
+ BUG_ON(path->slots[0] < 0);
+
+ if (double_split) {
+ BUG_ON(num_doubles != 0);
+ num_doubles++;
+ goto again;
+ }
+ return ret;
+}
+
+/*
+ * This function splits a single item into two items,
+ * giving 'new_key' to the new item and splitting the
+ * old one at split_offset (from the start of the item).
+ *
+ * The path may be released by this operation. After
+ * the split, the path is pointing to the old item. The
+ * new item is going to be in the same node as the old one.
+ *
+ * Note, the item being split must be smaller enough to live alone on
+ * a tree block with room for one extra struct btrfs_item
+ *
+ * This allows us to split the item in place, keeping a lock on the
+ * leaf the entire time.
+ */
+int btrfs_split_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct btrfs_key *new_key,
+ unsigned long split_offset)
+{
+ u32 item_size;
+ struct extent_buffer *leaf;
+ struct btrfs_key orig_key;
+ struct btrfs_item *item;
+ struct btrfs_item *new_item;
+ int ret = 0;
+ int slot;
+ u32 nritems;
+ u32 orig_offset;
+ struct btrfs_disk_key disk_key;
+ char *buf;
+
+ leaf = path->nodes[0];
+ btrfs_item_key_to_cpu(leaf, &orig_key, path->slots[0]);
+ if (btrfs_leaf_free_space(root, leaf) >= sizeof(struct btrfs_item))
+ goto split;
+
+ item_size = btrfs_item_size_nr(leaf, path->slots[0]);
+ btrfs_release_path(root, path);
+
+ path->search_for_split = 1;
+ path->keep_locks = 1;
+
+ ret = btrfs_search_slot(trans, root, &orig_key, path, 0, 1);
+ path->search_for_split = 0;
+
+ /* if our item isn't there or got smaller, return now */
+ if (ret != 0 || item_size != btrfs_item_size_nr(path->nodes[0],
+ path->slots[0])) {
+ path->keep_locks = 0;
+ return -EAGAIN;
+ }
+
+ ret = split_leaf(trans, root, &orig_key, path,
+ sizeof(struct btrfs_item), 1);
+ path->keep_locks = 0;
+ BUG_ON(ret);
+
+ leaf = path->nodes[0];
+ BUG_ON(btrfs_leaf_free_space(root, leaf) < sizeof(struct btrfs_item));
+
+split:
+ item = btrfs_item_nr(leaf, path->slots[0]);
+ orig_offset = btrfs_item_offset(leaf, item);
+ item_size = btrfs_item_size(leaf, item);
+
+
+ buf = kmalloc(item_size, GFP_NOFS);
+ read_extent_buffer(leaf, buf, btrfs_item_ptr_offset(leaf,
+ path->slots[0]), item_size);
+ slot = path->slots[0] + 1;
+ leaf = path->nodes[0];
+
+ nritems = btrfs_header_nritems(leaf);
+
+ if (slot != nritems) {
+ /* shift the items */
+ memmove_extent_buffer(leaf, btrfs_item_nr_offset(slot + 1),
+ btrfs_item_nr_offset(slot),
+ (nritems - slot) * sizeof(struct btrfs_item));
+
+ }
+
+ btrfs_cpu_key_to_disk(&disk_key, new_key);
+ btrfs_set_item_key(leaf, &disk_key, slot);
+
+ new_item = btrfs_item_nr(leaf, slot);
+
+ btrfs_set_item_offset(leaf, new_item, orig_offset);
+ btrfs_set_item_size(leaf, new_item, item_size - split_offset);
+
+ btrfs_set_item_offset(leaf, item,
+ orig_offset + item_size - split_offset);
+ btrfs_set_item_size(leaf, item, split_offset);
+
+ btrfs_set_header_nritems(leaf, nritems + 1);
+
+ /* write the data for the start of the original item */
+ write_extent_buffer(leaf, buf,
+ btrfs_item_ptr_offset(leaf, path->slots[0]),
+ split_offset);
+
+ /* write the data for the new item */
+ write_extent_buffer(leaf, buf + split_offset,
+ btrfs_item_ptr_offset(leaf, slot),
+ item_size - split_offset);
+ btrfs_mark_buffer_dirty(leaf);
+
+ ret = 0;
+ if (btrfs_leaf_free_space(root, leaf) < 0) {
+ btrfs_print_leaf(root, leaf);
+ BUG();
+ }
+ kfree(buf);
+ return ret;
+}
+
+/*
+ * make the item pointed to by the path smaller. new_size indicates
+ * how small to make it, and from_end tells us if we just chop bytes
+ * off the end of the item or if we shift the item to chop bytes off
+ * the front.
+ */
+int btrfs_truncate_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ u32 new_size, int from_end)
+{
+ int ret = 0;
+ int slot;
+ int slot_orig;
+ struct extent_buffer *leaf;
+ struct btrfs_item *item;
+ u32 nritems;
+ unsigned int data_end;
+ unsigned int old_data_start;
+ unsigned int old_size;
+ unsigned int size_diff;
+ int i;
+
+ slot_orig = path->slots[0];
+ leaf = path->nodes[0];
+ slot = path->slots[0];
+
+ old_size = btrfs_item_size_nr(leaf, slot);
+ if (old_size == new_size)
+ return 0;
+
+ nritems = btrfs_header_nritems(leaf);
+ data_end = leaf_data_end(root, leaf);
+
+ old_data_start = btrfs_item_offset_nr(leaf, slot);
+
+ size_diff = old_size - new_size;
+
+ BUG_ON(slot < 0);
+ BUG_ON(slot >= nritems);
+
+ /*
+ * item0..itemN ... dataN.offset..dataN.size .. data0.size
+ */
+ /* first correct the data pointers */
+ for (i = slot; i < nritems; i++) {
+ u32 ioff;
+ item = btrfs_item_nr(leaf, i);
+
+ if (!leaf->map_token) {
+ map_extent_buffer(leaf, (unsigned long)item,
+ sizeof(struct btrfs_item),
+ &leaf->map_token, &leaf->kaddr,
+ &leaf->map_start, &leaf->map_len,
+ KM_USER1);
+ }
+
+ ioff = btrfs_item_offset(leaf, item);
+ btrfs_set_item_offset(leaf, item, ioff + size_diff);
+ }
+
+ if (leaf->map_token) {
+ unmap_extent_buffer(leaf, leaf->map_token, KM_USER1);
+ leaf->map_token = NULL;
+ }
+
+ /* shift the data */
+ if (from_end) {
+ memmove_extent_buffer(leaf, btrfs_leaf_data(leaf) +
+ data_end + size_diff, btrfs_leaf_data(leaf) +
+ data_end, old_data_start + new_size - data_end);
+ } else {
+ struct btrfs_disk_key disk_key;
+ u64 offset;
+
+ btrfs_item_key(leaf, &disk_key, slot);
+
+ if (btrfs_disk_key_type(&disk_key) == BTRFS_EXTENT_DATA_KEY) {
+ unsigned long ptr;
+ struct btrfs_file_extent_item *fi;
+
+ fi = btrfs_item_ptr(leaf, slot,
+ struct btrfs_file_extent_item);
+ fi = (struct btrfs_file_extent_item *)(
+ (unsigned long)fi - size_diff);
+
+ if (btrfs_file_extent_type(leaf, fi) ==
+ BTRFS_FILE_EXTENT_INLINE) {
+ ptr = btrfs_item_ptr_offset(leaf, slot);
+ memmove_extent_buffer(leaf, ptr,
+ (unsigned long)fi,
+ offsetof(struct btrfs_file_extent_item,
+ disk_bytenr));
+ }
+ }
+
+ memmove_extent_buffer(leaf, btrfs_leaf_data(leaf) +
+ data_end + size_diff, btrfs_leaf_data(leaf) +
+ data_end, old_data_start - data_end);
+
+ offset = btrfs_disk_key_offset(&disk_key);
+ btrfs_set_disk_key_offset(&disk_key, offset + size_diff);
+ btrfs_set_item_key(leaf, &disk_key, slot);
+ if (slot == 0)
+ fixup_low_keys(trans, root, path, &disk_key, 1);
+ }
+
+ item = btrfs_item_nr(leaf, slot);
+ btrfs_set_item_size(leaf, item, new_size);
+ btrfs_mark_buffer_dirty(leaf);
+
+ ret = 0;
+ if (btrfs_leaf_free_space(root, leaf) < 0) {
+ btrfs_print_leaf(root, leaf);
+ BUG();
+ }
+ return ret;
+}
+
+/*
+ * make the item pointed to by the path bigger, data_size is the new size.
+ */
+int btrfs_extend_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct btrfs_path *path,
+ u32 data_size)
+{
+ int ret = 0;
+ int slot;
+ int slot_orig;
+ struct extent_buffer *leaf;
+ struct btrfs_item *item;
+ u32 nritems;
+ unsigned int data_end;
+ unsigned int old_data;
+ unsigned int old_size;
+ int i;
+
+ slot_orig = path->slots[0];
+ leaf = path->nodes[0];
+
+ nritems = btrfs_header_nritems(leaf);
+ data_end = leaf_data_end(root, leaf);
+
+ if (btrfs_leaf_free_space(root, leaf) < data_size) {
+ btrfs_print_leaf(root, leaf);
+ BUG();
+ }
+ slot = path->slots[0];
+ old_data = btrfs_item_end_nr(leaf, slot);
+
+ BUG_ON(slot < 0);
+ if (slot >= nritems) {
+ btrfs_print_leaf(root, leaf);
+ printk(KERN_CRIT "slot %d too large, nritems %d\n",
+ slot, nritems);
+ BUG_ON(1);
+ }
+
+ /*
+ * item0..itemN ... dataN.offset..dataN.size .. data0.size
+ */
+ /* first correct the data pointers */
+ for (i = slot; i < nritems; i++) {
+ u32 ioff;
+ item = btrfs_item_nr(leaf, i);
+
+ if (!leaf->map_token) {
+ map_extent_buffer(leaf, (unsigned long)item,
+ sizeof(struct btrfs_item),
+ &leaf->map_token, &leaf->kaddr,
+ &leaf->map_start, &leaf->map_len,
+ KM_USER1);
+ }
+ ioff = btrfs_item_offset(leaf, item);
+ btrfs_set_item_offset(leaf, item, ioff - data_size);
+ }
+
+ if (leaf->map_token) {
+ unmap_extent_buffer(leaf, leaf->map_token, KM_USER1);
+ leaf->map_token = NULL;
+ }
+
+ /* shift the data */
+ memmove_extent_buffer(leaf, btrfs_leaf_data(leaf) +
+ data_end - data_size, btrfs_leaf_data(leaf) +
+ data_end, old_data - data_end);
+
+ data_end = old_data;
+ old_size = btrfs_item_size_nr(leaf, slot);
+ item = btrfs_item_nr(leaf, slot);
+ btrfs_set_item_size(leaf, item, old_size + data_size);
+ btrfs_mark_buffer_dirty(leaf);
+
+ ret = 0;
+ if (btrfs_leaf_free_space(root, leaf) < 0) {
+ btrfs_print_leaf(root, leaf);
+ BUG();
+ }
+ return ret;
+}
+
+/*
+ * Given a key and some data, insert items into the tree.
+ * This does all the path init required, making room in the tree if needed.
+ * Returns the number of keys that were inserted.
+ */
+int btrfs_insert_some_items(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct btrfs_key *cpu_key, u32 *data_size,
+ int nr)
+{
+ struct extent_buffer *leaf;
+ struct btrfs_item *item;
+ int ret = 0;
+ int slot;
+ int i;
+ u32 nritems;
+ u32 total_data = 0;
+ u32 total_size = 0;
+ unsigned int data_end;
+ struct btrfs_disk_key disk_key;
+ struct btrfs_key found_key;
+
+ for (i = 0; i < nr; i++) {
+ if (total_size + data_size[i] + sizeof(struct btrfs_item) >
+ BTRFS_LEAF_DATA_SIZE(root)) {
+ break;
+ nr = i;
+ }
+ total_data += data_size[i];
+ total_size += data_size[i] + sizeof(struct btrfs_item);
+ }
+ BUG_ON(nr == 0);
+
+ ret = btrfs_search_slot(trans, root, cpu_key, path, total_size, 1);
+ if (ret == 0)
+ return -EEXIST;
+ if (ret < 0)
+ goto out;
+
+ leaf = path->nodes[0];
+
+ nritems = btrfs_header_nritems(leaf);
+ data_end = leaf_data_end(root, leaf);
+
+ if (btrfs_leaf_free_space(root, leaf) < total_size) {
+ for (i = nr; i >= 0; i--) {
+ total_data -= data_size[i];
+ total_size -= data_size[i] + sizeof(struct btrfs_item);
+ if (total_size < btrfs_leaf_free_space(root, leaf))
+ break;
+ }
+ nr = i;
+ }
+
+ slot = path->slots[0];
+ BUG_ON(slot < 0);
+
+ if (slot != nritems) {
+ unsigned int old_data = btrfs_item_end_nr(leaf, slot);
+
+ item = btrfs_item_nr(leaf, slot);
+ btrfs_item_key_to_cpu(leaf, &found_key, slot);
+
+ /* figure out how many keys we can insert in here */
+ total_data = data_size[0];
+ for (i = 1; i < nr; i++) {
+ if (comp_cpu_keys(&found_key, cpu_key + i) <= 0)
+ break;
+ total_data += data_size[i];
+ }
+ nr = i;
+
+ if (old_data < data_end) {
+ btrfs_print_leaf(root, leaf);
+ printk(KERN_CRIT "slot %d old_data %d data_end %d\n",
+ slot, old_data, data_end);
+ BUG_ON(1);
+ }
+ /*
+ * item0..itemN ... dataN.offset..dataN.size .. data0.size
+ */
+ /* first correct the data pointers */
+ WARN_ON(leaf->map_token);
+ for (i = slot; i < nritems; i++) {
+ u32 ioff;
+
+ item = btrfs_item_nr(leaf, i);
+ if (!leaf->map_token) {
+ map_extent_buffer(leaf, (unsigned long)item,
+ sizeof(struct btrfs_item),
+ &leaf->map_token, &leaf->kaddr,
+ &leaf->map_start, &leaf->map_len,
+ KM_USER1);
+ }
+
+ ioff = btrfs_item_offset(leaf, item);
+ btrfs_set_item_offset(leaf, item, ioff - total_data);
+ }
+ if (leaf->map_token) {
+ unmap_extent_buffer(leaf, leaf->map_token, KM_USER1);
+ leaf->map_token = NULL;
+ }
+
+ /* shift the items */
+ memmove_extent_buffer(leaf, btrfs_item_nr_offset(slot + nr),
+ btrfs_item_nr_offset(slot),
+ (nritems - slot) * sizeof(struct btrfs_item));
+
+ /* shift the data */
+ memmove_extent_buffer(leaf, btrfs_leaf_data(leaf) +
+ data_end - total_data, btrfs_leaf_data(leaf) +
+ data_end, old_data - data_end);
+ data_end = old_data;
+ } else {
+ /*
+ * this sucks but it has to be done, if we are inserting at
+ * the end of the leaf only insert 1 of the items, since we
+ * have no way of knowing whats on the next leaf and we'd have
+ * to drop our current locks to figure it out
+ */
+ nr = 1;
+ }
+
+ /* setup the item for the new data */
+ for (i = 0; i < nr; i++) {
+ btrfs_cpu_key_to_disk(&disk_key, cpu_key + i);
+ btrfs_set_item_key(leaf, &disk_key, slot + i);
+ item = btrfs_item_nr(leaf, slot + i);
+ btrfs_set_item_offset(leaf, item, data_end - data_size[i]);
+ data_end -= data_size[i];
+ btrfs_set_item_size(leaf, item, data_size[i]);
+ }
+ btrfs_set_header_nritems(leaf, nritems + nr);
+ btrfs_mark_buffer_dirty(leaf);
+
+ ret = 0;
+ if (slot == 0) {
+ btrfs_cpu_key_to_disk(&disk_key, cpu_key);
+ ret = fixup_low_keys(trans, root, path, &disk_key, 1);
+ }
+
+ if (btrfs_leaf_free_space(root, leaf) < 0) {
+ btrfs_print_leaf(root, leaf);
+ BUG();
+ }
+out:
+ if (!ret)
+ ret = nr;
+ return ret;
+}
+
+/*
+ * Given a key and some data, insert items into the tree.
+ * This does all the path init required, making room in the tree if needed.
+ */
+int btrfs_insert_empty_items(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct btrfs_key *cpu_key, u32 *data_size,
+ int nr)
+{
+ struct extent_buffer *leaf;
+ struct btrfs_item *item;
+ int ret = 0;
+ int slot;
+ int slot_orig;
+ int i;
+ u32 nritems;
+ u32 total_size = 0;
+ u32 total_data = 0;
+ unsigned int data_end;
+ struct btrfs_disk_key disk_key;
+
+ for (i = 0; i < nr; i++)
+ total_data += data_size[i];
+
+ total_size = total_data + (nr * sizeof(struct btrfs_item));
+ ret = btrfs_search_slot(trans, root, cpu_key, path, total_size, 1);
+ if (ret == 0)
+ return -EEXIST;
+ if (ret < 0)
+ goto out;
+
+ slot_orig = path->slots[0];
+ leaf = path->nodes[0];
+
+ nritems = btrfs_header_nritems(leaf);
+ data_end = leaf_data_end(root, leaf);
+
+ if (btrfs_leaf_free_space(root, leaf) < total_size) {
+ btrfs_print_leaf(root, leaf);
+ printk(KERN_CRIT "not enough freespace need %u have %d\n",
+ total_size, btrfs_leaf_free_space(root, leaf));
+ BUG();
+ }
+
+ slot = path->slots[0];
+ BUG_ON(slot < 0);
+
+ if (slot != nritems) {
+ unsigned int old_data = btrfs_item_end_nr(leaf, slot);
+
+ if (old_data < data_end) {
+ btrfs_print_leaf(root, leaf);
+ printk(KERN_CRIT "slot %d old_data %d data_end %d\n",
+ slot, old_data, data_end);
+ BUG_ON(1);
+ }
+ /*
+ * item0..itemN ... dataN.offset..dataN.size .. data0.size
+ */
+ /* first correct the data pointers */
+ WARN_ON(leaf->map_token);
+ for (i = slot; i < nritems; i++) {
+ u32 ioff;
+
+ item = btrfs_item_nr(leaf, i);
+ if (!leaf->map_token) {
+ map_extent_buffer(leaf, (unsigned long)item,
+ sizeof(struct btrfs_item),
+ &leaf->map_token, &leaf->kaddr,
+ &leaf->map_start, &leaf->map_len,
+ KM_USER1);
+ }
+
+ ioff = btrfs_item_offset(leaf, item);
+ btrfs_set_item_offset(leaf, item, ioff - total_data);
+ }
+ if (leaf->map_token) {
+ unmap_extent_buffer(leaf, leaf->map_token, KM_USER1);
+ leaf->map_token = NULL;
+ }
+
+ /* shift the items */
+ memmove_extent_buffer(leaf, btrfs_item_nr_offset(slot + nr),
+ btrfs_item_nr_offset(slot),
+ (nritems - slot) * sizeof(struct btrfs_item));
+
+ /* shift the data */
+ memmove_extent_buffer(leaf, btrfs_leaf_data(leaf) +
+ data_end - total_data, btrfs_leaf_data(leaf) +
+ data_end, old_data - data_end);
+ data_end = old_data;
+ }
+
+ /* setup the item for the new data */
+ for (i = 0; i < nr; i++) {
+ btrfs_cpu_key_to_disk(&disk_key, cpu_key + i);
+ btrfs_set_item_key(leaf, &disk_key, slot + i);
+ item = btrfs_item_nr(leaf, slot + i);
+ btrfs_set_item_offset(leaf, item, data_end - data_size[i]);
+ data_end -= data_size[i];
+ btrfs_set_item_size(leaf, item, data_size[i]);
+ }
+ btrfs_set_header_nritems(leaf, nritems + nr);
+ btrfs_mark_buffer_dirty(leaf);
+
+ ret = 0;
+ if (slot == 0) {
+ btrfs_cpu_key_to_disk(&disk_key, cpu_key);
+ ret = fixup_low_keys(trans, root, path, &disk_key, 1);
+ }
+
+ if (btrfs_leaf_free_space(root, leaf) < 0) {
+ btrfs_print_leaf(root, leaf);
+ BUG();
+ }
+out:
+ return ret;
+}
+
+/*
+ * Given a key and some data, insert an item into the tree.
+ * This does all the path init required, making room in the tree if needed.
+ */
+int btrfs_insert_item(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root, struct btrfs_key *cpu_key, void *data, u32
+ data_size)
+{
+ int ret = 0;
+ struct btrfs_path *path;
+ struct extent_buffer *leaf;
+ unsigned long ptr;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+ ret = btrfs_insert_empty_item(trans, root, path, cpu_key, data_size);
+ if (!ret) {
+ leaf = path->nodes[0];
+ ptr = btrfs_item_ptr_offset(leaf, path->slots[0]);
+ write_extent_buffer(leaf, data, ptr, data_size);
+ btrfs_mark_buffer_dirty(leaf);
+ }
+ btrfs_free_path(path);
+ return ret;
+}
+
+/*
+ * delete the pointer from a given node.
+ *
+ * the tree should have been previously balanced so the deletion does not
+ * empty a node.
+ */
+static int del_ptr(struct btrfs_trans_handle *trans, struct btrfs_root *root,
+ struct btrfs_path *path, int level, int slot)
+{
+ struct extent_buffer *parent = path->nodes[level];
+ u32 nritems;
+ int ret = 0;
+ int wret;
+
+ nritems = btrfs_header_nritems(parent);
+ if (slot != nritems - 1) {
+ memmove_extent_buffer(parent,
+ btrfs_node_key_ptr_offset(slot),
+ btrfs_node_key_ptr_offset(slot + 1),
+ sizeof(struct btrfs_key_ptr) *
+ (nritems - slot - 1));
+ }
+ nritems--;
+ btrfs_set_header_nritems(parent, nritems);
+ if (nritems == 0 && parent == root->node) {
+ BUG_ON(btrfs_header_level(root->node) != 1);
+ /* just turn the root into a leaf and break */
+ btrfs_set_header_level(root->node, 0);
+ } else if (slot == 0) {
+ struct btrfs_disk_key disk_key;
+
+ btrfs_node_key(parent, &disk_key, 0);
+ wret = fixup_low_keys(trans, root, path, &disk_key, level + 1);
+ if (wret)
+ ret = wret;
+ }
+ btrfs_mark_buffer_dirty(parent);
+ return ret;
+}
+
+/*
+ * a helper function to delete the leaf pointed to by path->slots[1] and
+ * path->nodes[1]. bytenr is the node block pointer, but since the callers
+ * already know it, it is faster to have them pass it down than to
+ * read it out of the node again.
+ *
+ * This deletes the pointer in path->nodes[1] and frees the leaf
+ * block extent. zero is returned if it all worked out, < 0 otherwise.
+ *
+ * The path must have already been setup for deleting the leaf, including
+ * all the proper balancing. path->nodes[1] must be locked.
+ */
+noinline int btrfs_del_leaf(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, u64 bytenr)
+{
+ int ret;
+ u64 root_gen = btrfs_header_generation(path->nodes[1]);
+
+ ret = del_ptr(trans, root, path, 1, path->slots[1]);
+ if (ret)
+ return ret;
+
+ ret = btrfs_free_extent(trans, root, bytenr,
+ btrfs_level_size(root, 0),
+ path->nodes[1]->start,
+ btrfs_header_owner(path->nodes[1]),
+ root_gen, 0, 1);
+ return ret;
+}
+/*
+ * delete the item at the leaf level in path. If that empties
+ * the leaf, remove it from the tree
+ */
+int btrfs_del_items(struct btrfs_trans_handle *trans, struct btrfs_root *root,
+ struct btrfs_path *path, int slot, int nr)
+{
+ struct extent_buffer *leaf;
+ struct btrfs_item *item;
+ int last_off;
+ int dsize = 0;
+ int ret = 0;
+ int wret;
+ int i;
+ u32 nritems;
+
+ leaf = path->nodes[0];
+ last_off = btrfs_item_offset_nr(leaf, slot + nr - 1);
+
+ for (i = 0; i < nr; i++)
+ dsize += btrfs_item_size_nr(leaf, slot + i);
+
+ nritems = btrfs_header_nritems(leaf);
+
+ if (slot + nr != nritems) {
+ int data_end = leaf_data_end(root, leaf);
+
+ memmove_extent_buffer(leaf, btrfs_leaf_data(leaf) +
+ data_end + dsize,
+ btrfs_leaf_data(leaf) + data_end,
+ last_off - data_end);
+
+ for (i = slot + nr; i < nritems; i++) {
+ u32 ioff;
+
+ item = btrfs_item_nr(leaf, i);
+ if (!leaf->map_token) {
+ map_extent_buffer(leaf, (unsigned long)item,
+ sizeof(struct btrfs_item),
+ &leaf->map_token, &leaf->kaddr,
+ &leaf->map_start, &leaf->map_len,
+ KM_USER1);
+ }
+ ioff = btrfs_item_offset(leaf, item);
+ btrfs_set_item_offset(leaf, item, ioff + dsize);
+ }
+
+ if (leaf->map_token) {
+ unmap_extent_buffer(leaf, leaf->map_token, KM_USER1);
+ leaf->map_token = NULL;
+ }
+
+ memmove_extent_buffer(leaf, btrfs_item_nr_offset(slot),
+ btrfs_item_nr_offset(slot + nr),
+ sizeof(struct btrfs_item) *
+ (nritems - slot - nr));
+ }
+ btrfs_set_header_nritems(leaf, nritems - nr);
+ nritems -= nr;
+
+ /* delete the leaf if we've emptied it */
+ if (nritems == 0) {
+ if (leaf == root->node) {
+ btrfs_set_header_level(leaf, 0);
+ } else {
+ ret = btrfs_del_leaf(trans, root, path, leaf->start);
+ BUG_ON(ret);
+ }
+ } else {
+ int used = leaf_space_used(leaf, 0, nritems);
+ if (slot == 0) {
+ struct btrfs_disk_key disk_key;
+
+ btrfs_item_key(leaf, &disk_key, 0);
+ wret = fixup_low_keys(trans, root, path,
+ &disk_key, 1);
+ if (wret)
+ ret = wret;
+ }
+
+ /* delete the leaf if it is mostly empty */
+ if (used < BTRFS_LEAF_DATA_SIZE(root) / 4) {
+ /* push_leaf_left fixes the path.
+ * make sure the path still points to our leaf
+ * for possible call to del_ptr below
+ */
+ slot = path->slots[1];
+ extent_buffer_get(leaf);
+
+ wret = push_leaf_left(trans, root, path, 1, 1);
+ if (wret < 0 && wret != -ENOSPC)
+ ret = wret;
+
+ if (path->nodes[0] == leaf &&
+ btrfs_header_nritems(leaf)) {
+ wret = push_leaf_right(trans, root, path, 1, 1);
+ if (wret < 0 && wret != -ENOSPC)
+ ret = wret;
+ }
+
+ if (btrfs_header_nritems(leaf) == 0) {
+ path->slots[1] = slot;
+ ret = btrfs_del_leaf(trans, root, path,
+ leaf->start);
+ BUG_ON(ret);
+ free_extent_buffer(leaf);
+ } else {
+ /* if we're still in the path, make sure
+ * we're dirty. Otherwise, one of the
+ * push_leaf functions must have already
+ * dirtied this buffer
+ */
+ if (path->nodes[0] == leaf)
+ btrfs_mark_buffer_dirty(leaf);
+ free_extent_buffer(leaf);
+ }
+ } else {
+ btrfs_mark_buffer_dirty(leaf);
+ }
+ }
+ return ret;
+}
+
+/*
+ * search the tree again to find a leaf with lesser keys
+ * returns 0 if it found something or 1 if there are no lesser leaves.
+ * returns < 0 on io errors.
+ *
+ * This may release the path, and so you may lose any locks held at the
+ * time you call it.
+ */
+int btrfs_prev_leaf(struct btrfs_root *root, struct btrfs_path *path)
+{
+ struct btrfs_key key;
+ struct btrfs_disk_key found_key;
+ int ret;
+
+ btrfs_item_key_to_cpu(path->nodes[0], &key, 0);
+
+ if (key.offset > 0)
+ key.offset--;
+ else if (key.type > 0)
+ key.type--;
+ else if (key.objectid > 0)
+ key.objectid--;
+ else
+ return 1;
+
+ btrfs_release_path(root, path);
+ ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ if (ret < 0)
+ return ret;
+ btrfs_item_key(path->nodes[0], &found_key, 0);
+ ret = comp_keys(&found_key, &key);
+ if (ret < 0)
+ return 0;
+ return 1;
+}
+
+/*
+ * A helper function to walk down the tree starting at min_key, and looking
+ * for nodes or leaves that are either in cache or have a minimum
+ * transaction id. This is used by the btree defrag code, and tree logging
+ *
+ * This does not cow, but it does stuff the starting key it finds back
+ * into min_key, so you can call btrfs_search_slot with cow=1 on the
+ * key and get a writable path.
+ *
+ * This does lock as it descends, and path->keep_locks should be set
+ * to 1 by the caller.
+ *
+ * This honors path->lowest_level to prevent descent past a given level
+ * of the tree.
+ *
+ * min_trans indicates the oldest transaction that you are interested
+ * in walking through. Any nodes or leaves older than min_trans are
+ * skipped over (without reading them).
+ *
+ * returns zero if something useful was found, < 0 on error and 1 if there
+ * was nothing in the tree that matched the search criteria.
+ */
+int btrfs_search_forward(struct btrfs_root *root, struct btrfs_key *min_key,
+ struct btrfs_key *max_key,
+ struct btrfs_path *path, int cache_only,
+ u64 min_trans)
+{
+ struct extent_buffer *cur;
+ struct btrfs_key found_key;
+ int slot;
+ int sret;
+ u32 nritems;
+ int level;
+ int ret = 1;
+
+ WARN_ON(!path->keep_locks);
+again:
+ cur = btrfs_lock_root_node(root);
+ level = btrfs_header_level(cur);
+ WARN_ON(path->nodes[level]);
+ path->nodes[level] = cur;
+ path->locks[level] = 1;
+
+ if (btrfs_header_generation(cur) < min_trans) {
+ ret = 1;
+ goto out;
+ }
+ while (1) {
+ nritems = btrfs_header_nritems(cur);
+ level = btrfs_header_level(cur);
+ sret = bin_search(cur, min_key, level, &slot);
+
+ /* at the lowest level, we're done, setup the path and exit */
+ if (level == path->lowest_level) {
+ if (slot >= nritems)
+ goto find_next_key;
+ ret = 0;
+ path->slots[level] = slot;
+ btrfs_item_key_to_cpu(cur, &found_key, slot);
+ goto out;
+ }
+ if (sret && slot > 0)
+ slot--;
+ /*
+ * check this node pointer against the cache_only and
+ * min_trans parameters. If it isn't in cache or is too
+ * old, skip to the next one.
+ */
+ while (slot < nritems) {
+ u64 blockptr;
+ u64 gen;
+ struct extent_buffer *tmp;
+ struct btrfs_disk_key disk_key;
+
+ blockptr = btrfs_node_blockptr(cur, slot);
+ gen = btrfs_node_ptr_generation(cur, slot);
+ if (gen < min_trans) {
+ slot++;
+ continue;
+ }
+ if (!cache_only)
+ break;
+
+ if (max_key) {
+ btrfs_node_key(cur, &disk_key, slot);
+ if (comp_keys(&disk_key, max_key) >= 0) {
+ ret = 1;
+ goto out;
+ }
+ }
+
+ tmp = btrfs_find_tree_block(root, blockptr,
+ btrfs_level_size(root, level - 1));
+
+ if (tmp && btrfs_buffer_uptodate(tmp, gen)) {
+ free_extent_buffer(tmp);
+ break;
+ }
+ if (tmp)
+ free_extent_buffer(tmp);
+ slot++;
+ }
+find_next_key:
+ /*
+ * we didn't find a candidate key in this node, walk forward
+ * and find another one
+ */
+ if (slot >= nritems) {
+ path->slots[level] = slot;
+ sret = btrfs_find_next_key(root, path, min_key, level,
+ cache_only, min_trans);
+ if (sret == 0) {
+ btrfs_release_path(root, path);
+ goto again;
+ } else {
+ goto out;
+ }
+ }
+ /* save our key for returning back */
+ btrfs_node_key_to_cpu(cur, &found_key, slot);
+ path->slots[level] = slot;
+ if (level == path->lowest_level) {
+ ret = 0;
+ unlock_up(path, level, 1);
+ goto out;
+ }
+ cur = read_node_slot(root, cur, slot);
+
+ btrfs_tree_lock(cur);
+ path->locks[level - 1] = 1;
+ path->nodes[level - 1] = cur;
+ unlock_up(path, level, 1);
+ }
+out:
+ if (ret == 0)
+ memcpy(min_key, &found_key, sizeof(found_key));
+ return ret;
+}
+
+/*
+ * this is similar to btrfs_next_leaf, but does not try to preserve
+ * and fixup the path. It looks for and returns the next key in the
+ * tree based on the current path and the cache_only and min_trans
+ * parameters.
+ *
+ * 0 is returned if another key is found, < 0 if there are any errors
+ * and 1 is returned if there are no higher keys in the tree
+ *
+ * path->keep_locks should be set to 1 on the search made before
+ * calling this function.
+ */
+int btrfs_find_next_key(struct btrfs_root *root, struct btrfs_path *path,
+ struct btrfs_key *key, int lowest_level,
+ int cache_only, u64 min_trans)
+{
+ int level = lowest_level;
+ int slot;
+ struct extent_buffer *c;
+
+ WARN_ON(!path->keep_locks);
+ while (level < BTRFS_MAX_LEVEL) {
+ if (!path->nodes[level])
+ return 1;
+
+ slot = path->slots[level] + 1;
+ c = path->nodes[level];
+next:
+ if (slot >= btrfs_header_nritems(c)) {
+ level++;
+ if (level == BTRFS_MAX_LEVEL)
+ return 1;
+ continue;
+ }
+ if (level == 0)
+ btrfs_item_key_to_cpu(c, key, slot);
+ else {
+ u64 blockptr = btrfs_node_blockptr(c, slot);
+ u64 gen = btrfs_node_ptr_generation(c, slot);
+
+ if (cache_only) {
+ struct extent_buffer *cur;
+ cur = btrfs_find_tree_block(root, blockptr,
+ btrfs_level_size(root, level - 1));
+ if (!cur || !btrfs_buffer_uptodate(cur, gen)) {
+ slot++;
+ if (cur)
+ free_extent_buffer(cur);
+ goto next;
+ }
+ free_extent_buffer(cur);
+ }
+ if (gen < min_trans) {
+ slot++;
+ goto next;
+ }
+ btrfs_node_key_to_cpu(c, key, slot);
+ }
+ return 0;
+ }
+ return 1;
+}
+
+/*
+ * search the tree again to find a leaf with greater keys
+ * returns 0 if it found something or 1 if there are no greater leaves.
+ * returns < 0 on io errors.
+ */
+int btrfs_next_leaf(struct btrfs_root *root, struct btrfs_path *path)
+{
+ int slot;
+ int level = 1;
+ struct extent_buffer *c;
+ struct extent_buffer *next = NULL;
+ struct btrfs_key key;
+ u32 nritems;
+ int ret;
+
+ nritems = btrfs_header_nritems(path->nodes[0]);
+ if (nritems == 0)
+ return 1;
+
+ btrfs_item_key_to_cpu(path->nodes[0], &key, nritems - 1);
+
+ btrfs_release_path(root, path);
+ path->keep_locks = 1;
+ ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ path->keep_locks = 0;
+
+ if (ret < 0)
+ return ret;
+
+ nritems = btrfs_header_nritems(path->nodes[0]);
+ /*
+ * by releasing the path above we dropped all our locks. A balance
+ * could have added more items next to the key that used to be
+ * at the very end of the block. So, check again here and
+ * advance the path if there are now more items available.
+ */
+ if (nritems > 0 && path->slots[0] < nritems - 1) {
+ path->slots[0]++;
+ goto done;
+ }
+
+ while (level < BTRFS_MAX_LEVEL) {
+ if (!path->nodes[level])
+ return 1;
+
+ slot = path->slots[level] + 1;
+ c = path->nodes[level];
+ if (slot >= btrfs_header_nritems(c)) {
+ level++;
+ if (level == BTRFS_MAX_LEVEL)
+ return 1;
+ continue;
+ }
+
+ if (next) {
+ btrfs_tree_unlock(next);
+ free_extent_buffer(next);
+ }
+
+ if (level == 1 && (path->locks[1] || path->skip_locking) &&
+ path->reada)
+ reada_for_search(root, path, level, slot, 0);
+
+ next = read_node_slot(root, c, slot);
+ if (!path->skip_locking) {
+ WARN_ON(!btrfs_tree_locked(c));
+ btrfs_tree_lock(next);
+ }
+ break;
+ }
+ path->slots[level] = slot;
+ while (1) {
+ level--;
+ c = path->nodes[level];
+ if (path->locks[level])
+ btrfs_tree_unlock(c);
+ free_extent_buffer(c);
+ path->nodes[level] = next;
+ path->slots[level] = 0;
+ if (!path->skip_locking)
+ path->locks[level] = 1;
+ if (!level)
+ break;
+ if (level == 1 && path->locks[1] && path->reada)
+ reada_for_search(root, path, level, slot, 0);
+ next = read_node_slot(root, next, 0);
+ if (!path->skip_locking) {
+ WARN_ON(!btrfs_tree_locked(path->nodes[level]));
+ btrfs_tree_lock(next);
+ }
+ }
+done:
+ unlock_up(path, 0, 1);
+ return 0;
+}
+
+/*
+ * this uses btrfs_prev_leaf to walk backwards in the tree, and keeps
+ * searching until it gets past min_objectid or finds an item of 'type'
+ *
+ * returns 0 if something is found, 1 if nothing was found and < 0 on error
+ */
+int btrfs_previous_item(struct btrfs_root *root,
+ struct btrfs_path *path, u64 min_objectid,
+ int type)
+{
+ struct btrfs_key found_key;
+ struct extent_buffer *leaf;
+ u32 nritems;
+ int ret;
+
+ while (1) {
+ if (path->slots[0] == 0) {
+ ret = btrfs_prev_leaf(root, path);
+ if (ret != 0)
+ return ret;
+ } else {
+ path->slots[0]--;
+ }
+ leaf = path->nodes[0];
+ nritems = btrfs_header_nritems(leaf);
+ if (nritems == 0)
+ return 1;
+ if (path->slots[0] == nritems)
+ path->slots[0]--;
+
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+ if (found_key.type == type)
+ return 0;
+ if (found_key.objectid < min_objectid)
+ break;
+ if (found_key.objectid == min_objectid &&
+ found_key.type < type)
+ break;
+ }
+ return 1;
+}
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
new file mode 100644
index 0000000..eee060f
--- /dev/null
+++ b/fs/btrfs/ctree.h
@@ -0,0 +1,2129 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __BTRFS_CTREE__
+#define __BTRFS_CTREE__
+
+#include <linux/version.h>
+#include <linux/mm.h>
+#include <linux/highmem.h>
+#include <linux/fs.h>
+#include <linux/completion.h>
+#include <linux/backing-dev.h>
+#include <linux/wait.h>
+#include <asm/kmap_types.h>
+#include "extent_io.h"
+#include "extent_map.h"
+#include "async-thread.h"
+
+struct btrfs_trans_handle;
+struct btrfs_transaction;
+extern struct kmem_cache *btrfs_trans_handle_cachep;
+extern struct kmem_cache *btrfs_transaction_cachep;
+extern struct kmem_cache *btrfs_bit_radix_cachep;
+extern struct kmem_cache *btrfs_path_cachep;
+struct btrfs_ordered_sum;
+
+#define BTRFS_MAGIC "_BHRfS_M"
+
+#define BTRFS_ACL_NOT_CACHED ((void *)-1)
+
+#ifdef CONFIG_LOCKDEP
+# define BTRFS_MAX_LEVEL 7
+#else
+# define BTRFS_MAX_LEVEL 8
+#endif
+
+/* holds pointers to all of the tree roots */
+#define BTRFS_ROOT_TREE_OBJECTID 1ULL
+
+/* stores information about which extents are in use, and reference counts */
+#define BTRFS_EXTENT_TREE_OBJECTID 2ULL
+
+/*
+ * chunk tree stores translations from logical -> physical block numbering
+ * the super block points to the chunk tree
+ */
+#define BTRFS_CHUNK_TREE_OBJECTID 3ULL
+
+/*
+ * stores information about which areas of a given device are in use.
+ * one per device. The tree of tree roots points to the device tree
+ */
+#define BTRFS_DEV_TREE_OBJECTID 4ULL
+
+/* one per subvolume, storing files and directories */
+#define BTRFS_FS_TREE_OBJECTID 5ULL
+
+/* directory objectid inside the root tree */
+#define BTRFS_ROOT_TREE_DIR_OBJECTID 6ULL
+
+/* holds checksums of all the data extents */
+#define BTRFS_CSUM_TREE_OBJECTID 7ULL
+
+/* orhpan objectid for tracking unlinked/truncated files */
+#define BTRFS_ORPHAN_OBJECTID -5ULL
+
+/* does write ahead logging to speed up fsyncs */
+#define BTRFS_TREE_LOG_OBJECTID -6ULL
+#define BTRFS_TREE_LOG_FIXUP_OBJECTID -7ULL
+
+/* for space balancing */
+#define BTRFS_TREE_RELOC_OBJECTID -8ULL
+#define BTRFS_DATA_RELOC_TREE_OBJECTID -9ULL
+
+/*
+ * extent checksums all have this objectid
+ * this allows them to share the logging tree
+ * for fsyncs
+ */
+#define BTRFS_EXTENT_CSUM_OBJECTID -10ULL
+
+/* dummy objectid represents multiple objectids */
+#define BTRFS_MULTIPLE_OBJECTIDS -255ULL
+
+/*
+ * All files have objectids in this range.
+ */
+#define BTRFS_FIRST_FREE_OBJECTID 256ULL
+#define BTRFS_LAST_FREE_OBJECTID -256ULL
+#define BTRFS_FIRST_CHUNK_TREE_OBJECTID 256ULL
+
+
+/*
+ * the device items go into the chunk tree. The key is in the form
+ * [ 1 BTRFS_DEV_ITEM_KEY device_id ]
+ */
+#define BTRFS_DEV_ITEMS_OBJECTID 1ULL
+
+/*
+ * we can actually store much bigger names, but lets not confuse the rest
+ * of linux
+ */
+#define BTRFS_NAME_LEN 255
+
+/* 32 bytes in various csum fields */
+#define BTRFS_CSUM_SIZE 32
+
+/* csum types */
+#define BTRFS_CSUM_TYPE_CRC32 0
+
+static int btrfs_csum_sizes[] = { 4, 0 };
+
+/* four bytes for CRC32 */
+#define BTRFS_EMPTY_DIR_SIZE 0
+
+#define BTRFS_FT_UNKNOWN 0
+#define BTRFS_FT_REG_FILE 1
+#define BTRFS_FT_DIR 2
+#define BTRFS_FT_CHRDEV 3
+#define BTRFS_FT_BLKDEV 4
+#define BTRFS_FT_FIFO 5
+#define BTRFS_FT_SOCK 6
+#define BTRFS_FT_SYMLINK 7
+#define BTRFS_FT_XATTR 8
+#define BTRFS_FT_MAX 9
+
+/*
+ * the key defines the order in the tree, and so it also defines (optimal)
+ * block layout. objectid corresonds to the inode number. The flags
+ * tells us things about the object, and is a kind of stream selector.
+ * so for a given inode, keys with flags of 1 might refer to the inode
+ * data, flags of 2 may point to file data in the btree and flags == 3
+ * may point to extents.
+ *
+ * offset is the starting byte offset for this key in the stream.
+ *
+ * btrfs_disk_key is in disk byte order. struct btrfs_key is always
+ * in cpu native order. Otherwise they are identical and their sizes
+ * should be the same (ie both packed)
+ */
+struct btrfs_disk_key {
+ __le64 objectid;
+ u8 type;
+ __le64 offset;
+} __attribute__ ((__packed__));
+
+struct btrfs_key {
+ u64 objectid;
+ u8 type;
+ u64 offset;
+} __attribute__ ((__packed__));
+
+struct btrfs_mapping_tree {
+ struct extent_map_tree map_tree;
+};
+
+#define BTRFS_UUID_SIZE 16
+struct btrfs_dev_item {
+ /* the internal btrfs device id */
+ __le64 devid;
+
+ /* size of the device */
+ __le64 total_bytes;
+
+ /* bytes used */
+ __le64 bytes_used;
+
+ /* optimal io alignment for this device */
+ __le32 io_align;
+
+ /* optimal io width for this device */
+ __le32 io_width;
+
+ /* minimal io size for this device */
+ __le32 sector_size;
+
+ /* type and info about this device */
+ __le64 type;
+
+ /* expected generation for this device */
+ __le64 generation;
+
+ /*
+ * starting byte of this partition on the device,
+ * to allowr for stripe alignment in the future
+ */
+ __le64 start_offset;
+
+ /* grouping information for allocation decisions */
+ __le32 dev_group;
+
+ /* seek speed 0-100 where 100 is fastest */
+ u8 seek_speed;
+
+ /* bandwidth 0-100 where 100 is fastest */
+ u8 bandwidth;
+
+ /* btrfs generated uuid for this device */
+ u8 uuid[BTRFS_UUID_SIZE];
+
+ /* uuid of FS who owns this device */
+ u8 fsid[BTRFS_UUID_SIZE];
+} __attribute__ ((__packed__));
+
+struct btrfs_stripe {
+ __le64 devid;
+ __le64 offset;
+ u8 dev_uuid[BTRFS_UUID_SIZE];
+} __attribute__ ((__packed__));
+
+struct btrfs_chunk {
+ /* size of this chunk in bytes */
+ __le64 length;
+
+ /* objectid of the root referencing this chunk */
+ __le64 owner;
+
+ __le64 stripe_len;
+ __le64 type;
+
+ /* optimal io alignment for this chunk */
+ __le32 io_align;
+
+ /* optimal io width for this chunk */
+ __le32 io_width;
+
+ /* minimal io size for this chunk */
+ __le32 sector_size;
+
+ /* 2^16 stripes is quite a lot, a second limit is the size of a single
+ * item in the btree
+ */
+ __le16 num_stripes;
+
+ /* sub stripes only matter for raid10 */
+ __le16 sub_stripes;
+ struct btrfs_stripe stripe;
+ /* additional stripes go here */
+} __attribute__ ((__packed__));
+
+static inline unsigned long btrfs_chunk_item_size(int num_stripes)
+{
+ BUG_ON(num_stripes == 0);
+ return sizeof(struct btrfs_chunk) +
+ sizeof(struct btrfs_stripe) * (num_stripes - 1);
+}
+
+#define BTRFS_FSID_SIZE 16
+#define BTRFS_HEADER_FLAG_WRITTEN (1 << 0)
+
+/*
+ * every tree block (leaf or node) starts with this header.
+ */
+struct btrfs_header {
+ /* these first four must match the super block */
+ u8 csum[BTRFS_CSUM_SIZE];
+ u8 fsid[BTRFS_FSID_SIZE]; /* FS specific uuid */
+ __le64 bytenr; /* which block this node is supposed to live in */
+ __le64 flags;
+
+ /* allowed to be different from the super from here on down */
+ u8 chunk_tree_uuid[BTRFS_UUID_SIZE];
+ __le64 generation;
+ __le64 owner;
+ __le32 nritems;
+ u8 level;
+} __attribute__ ((__packed__));
+
+#define BTRFS_NODEPTRS_PER_BLOCK(r) (((r)->nodesize - \
+ sizeof(struct btrfs_header)) / \
+ sizeof(struct btrfs_key_ptr))
+#define __BTRFS_LEAF_DATA_SIZE(bs) ((bs) - sizeof(struct btrfs_header))
+#define BTRFS_LEAF_DATA_SIZE(r) (__BTRFS_LEAF_DATA_SIZE(r->leafsize))
+#define BTRFS_MAX_INLINE_DATA_SIZE(r) (BTRFS_LEAF_DATA_SIZE(r) - \
+ sizeof(struct btrfs_item) - \
+ sizeof(struct btrfs_file_extent_item))
+
+#define BTRFS_SUPER_FLAG_SEEDING (1ULL << 32)
+
+/*
+ * this is a very generous portion of the super block, giving us
+ * room to translate 14 chunks with 3 stripes each.
+ */
+#define BTRFS_SYSTEM_CHUNK_ARRAY_SIZE 2048
+#define BTRFS_LABEL_SIZE 256
+
+/*
+ * the super block basically lists the main trees of the FS
+ * it currently lacks any block count etc etc
+ */
+struct btrfs_super_block {
+ u8 csum[BTRFS_CSUM_SIZE];
+ /* the first 4 fields must match struct btrfs_header */
+ u8 fsid[BTRFS_FSID_SIZE]; /* FS specific uuid */
+ __le64 bytenr; /* this block number */
+ __le64 flags;
+
+ /* allowed to be different from the btrfs_header from here own down */
+ __le64 magic;
+ __le64 generation;
+ __le64 root;
+ __le64 chunk_root;
+ __le64 log_root;
+
+ /* this will help find the new super based on the log root */
+ __le64 log_root_transid;
+ __le64 total_bytes;
+ __le64 bytes_used;
+ __le64 root_dir_objectid;
+ __le64 num_devices;
+ __le32 sectorsize;
+ __le32 nodesize;
+ __le32 leafsize;
+ __le32 stripesize;
+ __le32 sys_chunk_array_size;
+ __le64 chunk_root_generation;
+ __le64 compat_flags;
+ __le64 compat_ro_flags;
+ __le64 incompat_flags;
+ __le16 csum_type;
+ u8 root_level;
+ u8 chunk_root_level;
+ u8 log_root_level;
+ struct btrfs_dev_item dev_item;
+
+ char label[BTRFS_LABEL_SIZE];
+
+ /* future expansion */
+ __le64 reserved[32];
+ u8 sys_chunk_array[BTRFS_SYSTEM_CHUNK_ARRAY_SIZE];
+} __attribute__ ((__packed__));
+
+/*
+ * Compat flags that we support. If any incompat flags are set other than the
+ * ones specified below then we will fail to mount
+ */
+#define BTRFS_FEATURE_COMPAT_SUPP 0x0
+#define BTRFS_FEATURE_COMPAT_RO_SUPP 0x0
+#define BTRFS_FEATURE_INCOMPAT_SUPP 0x0
+
+/*
+ * A leaf is full of items. offset and size tell us where to find
+ * the item in the leaf (relative to the start of the data area)
+ */
+struct btrfs_item {
+ struct btrfs_disk_key key;
+ __le32 offset;
+ __le32 size;
+} __attribute__ ((__packed__));
+
+/*
+ * leaves have an item area and a data area:
+ * [item0, item1....itemN] [free space] [dataN...data1, data0]
+ *
+ * The data is separate from the items to get the keys closer together
+ * during searches.
+ */
+struct btrfs_leaf {
+ struct btrfs_header header;
+ struct btrfs_item items[];
+} __attribute__ ((__packed__));
+
+/*
+ * all non-leaf blocks are nodes, they hold only keys and pointers to
+ * other blocks
+ */
+struct btrfs_key_ptr {
+ struct btrfs_disk_key key;
+ __le64 blockptr;
+ __le64 generation;
+} __attribute__ ((__packed__));
+
+struct btrfs_node {
+ struct btrfs_header header;
+ struct btrfs_key_ptr ptrs[];
+} __attribute__ ((__packed__));
+
+/*
+ * btrfs_paths remember the path taken from the root down to the leaf.
+ * level 0 is always the leaf, and nodes[1...BTRFS_MAX_LEVEL] will point
+ * to any other levels that are present.
+ *
+ * The slots array records the index of the item or block pointer
+ * used while walking the tree.
+ */
+struct btrfs_path {
+ struct extent_buffer *nodes[BTRFS_MAX_LEVEL];
+ int slots[BTRFS_MAX_LEVEL];
+ /* if there is real range locking, this locks field will change */
+ int locks[BTRFS_MAX_LEVEL];
+ int reada;
+ /* keep some upper locks as we walk down */
+ int keep_locks;
+ int skip_locking;
+ int lowest_level;
+
+ /*
+ * set by btrfs_split_item, tells search_slot to keep all locks
+ * and to force calls to keep space in the nodes
+ */
+ int search_for_split;
+};
+
+/*
+ * items in the extent btree are used to record the objectid of the
+ * owner of the block and the number of references
+ */
+struct btrfs_extent_item {
+ __le32 refs;
+} __attribute__ ((__packed__));
+
+struct btrfs_extent_ref {
+ __le64 root;
+ __le64 generation;
+ __le64 objectid;
+ __le32 num_refs;
+} __attribute__ ((__packed__));
+
+/* dev extents record free space on individual devices. The owner
+ * field points back to the chunk allocation mapping tree that allocated
+ * the extent. The chunk tree uuid field is a way to double check the owner
+ */
+struct btrfs_dev_extent {
+ __le64 chunk_tree;
+ __le64 chunk_objectid;
+ __le64 chunk_offset;
+ __le64 length;
+ u8 chunk_tree_uuid[BTRFS_UUID_SIZE];
+} __attribute__ ((__packed__));
+
+struct btrfs_inode_ref {
+ __le64 index;
+ __le16 name_len;
+ /* name goes here */
+} __attribute__ ((__packed__));
+
+struct btrfs_timespec {
+ __le64 sec;
+ __le32 nsec;
+} __attribute__ ((__packed__));
+
+typedef enum {
+ BTRFS_COMPRESS_NONE = 0,
+ BTRFS_COMPRESS_ZLIB = 1,
+ BTRFS_COMPRESS_LAST = 2,
+} btrfs_compression_type;
+
+/* we don't understand any encryption methods right now */
+typedef enum {
+ BTRFS_ENCRYPTION_NONE = 0,
+ BTRFS_ENCRYPTION_LAST = 1,
+} btrfs_encryption_type;
+
+struct btrfs_inode_item {
+ /* nfs style generation number */
+ __le64 generation;
+ /* transid that last touched this inode */
+ __le64 transid;
+ __le64 size;
+ __le64 nbytes;
+ __le64 block_group;
+ __le32 nlink;
+ __le32 uid;
+ __le32 gid;
+ __le32 mode;
+ __le64 rdev;
+ __le64 flags;
+
+ /* modification sequence number for NFS */
+ __le64 sequence;
+
+ /*
+ * a little future expansion, for more than this we can
+ * just grow the inode item and version it
+ */
+ __le64 reserved[4];
+ struct btrfs_timespec atime;
+ struct btrfs_timespec ctime;
+ struct btrfs_timespec mtime;
+ struct btrfs_timespec otime;
+} __attribute__ ((__packed__));
+
+struct btrfs_dir_log_item {
+ __le64 end;
+} __attribute__ ((__packed__));
+
+struct btrfs_dir_item {
+ struct btrfs_disk_key location;
+ __le64 transid;
+ __le16 data_len;
+ __le16 name_len;
+ u8 type;
+} __attribute__ ((__packed__));
+
+struct btrfs_root_item {
+ struct btrfs_inode_item inode;
+ __le64 generation;
+ __le64 root_dirid;
+ __le64 bytenr;
+ __le64 byte_limit;
+ __le64 bytes_used;
+ __le64 last_snapshot;
+ __le64 flags;
+ __le32 refs;
+ struct btrfs_disk_key drop_progress;
+ u8 drop_level;
+ u8 level;
+} __attribute__ ((__packed__));
+
+/*
+ * this is used for both forward and backward root refs
+ */
+struct btrfs_root_ref {
+ __le64 dirid;
+ __le64 sequence;
+ __le16 name_len;
+} __attribute__ ((__packed__));
+
+#define BTRFS_FILE_EXTENT_INLINE 0
+#define BTRFS_FILE_EXTENT_REG 1
+#define BTRFS_FILE_EXTENT_PREALLOC 2
+
+struct btrfs_file_extent_item {
+ /*
+ * transaction id that created this extent
+ */
+ __le64 generation;
+ /*
+ * max number of bytes to hold this extent in ram
+ * when we split a compressed extent we can't know how big
+ * each of the resulting pieces will be. So, this is
+ * an upper limit on the size of the extent in ram instead of
+ * an exact limit.
+ */
+ __le64 ram_bytes;
+
+ /*
+ * 32 bits for the various ways we might encode the data,
+ * including compression and encryption. If any of these
+ * are set to something a given disk format doesn't understand
+ * it is treated like an incompat flag for reading and writing,
+ * but not for stat.
+ */
+ u8 compression;
+ u8 encryption;
+ __le16 other_encoding; /* spare for later use */
+
+ /* are we inline data or a real extent? */
+ u8 type;
+
+ /*
+ * disk space consumed by the extent, checksum blocks are included
+ * in these numbers
+ */
+ __le64 disk_bytenr;
+ __le64 disk_num_bytes;
+ /*
+ * the logical offset in file blocks (no csums)
+ * this extent record is for. This allows a file extent to point
+ * into the middle of an existing extent on disk, sharing it
+ * between two snapshots (useful if some bytes in the middle of the
+ * extent have changed
+ */
+ __le64 offset;
+ /*
+ * the logical number of file blocks (no csums included). This
+ * always reflects the size uncompressed and without encoding.
+ */
+ __le64 num_bytes;
+
+} __attribute__ ((__packed__));
+
+struct btrfs_csum_item {
+ u8 csum;
+} __attribute__ ((__packed__));
+
+/* different types of block groups (and chunks) */
+#define BTRFS_BLOCK_GROUP_DATA (1 << 0)
+#define BTRFS_BLOCK_GROUP_SYSTEM (1 << 1)
+#define BTRFS_BLOCK_GROUP_METADATA (1 << 2)
+#define BTRFS_BLOCK_GROUP_RAID0 (1 << 3)
+#define BTRFS_BLOCK_GROUP_RAID1 (1 << 4)
+#define BTRFS_BLOCK_GROUP_DUP (1 << 5)
+#define BTRFS_BLOCK_GROUP_RAID10 (1 << 6)
+
+struct btrfs_block_group_item {
+ __le64 used;
+ __le64 chunk_objectid;
+ __le64 flags;
+} __attribute__ ((__packed__));
+
+struct btrfs_space_info {
+ u64 flags;
+ u64 total_bytes;
+ u64 bytes_used;
+ u64 bytes_pinned;
+ u64 bytes_reserved;
+ u64 bytes_readonly;
+ int full;
+ int force_alloc;
+ struct list_head list;
+
+ /* for block groups in our same type */
+ struct list_head block_groups;
+ spinlock_t lock;
+ struct rw_semaphore groups_sem;
+};
+
+struct btrfs_free_space {
+ struct rb_node bytes_index;
+ struct rb_node offset_index;
+ u64 offset;
+ u64 bytes;
+};
+
+struct btrfs_block_group_cache {
+ struct btrfs_key key;
+ struct btrfs_block_group_item item;
+ spinlock_t lock;
+ struct mutex alloc_mutex;
+ struct mutex cache_mutex;
+ u64 pinned;
+ u64 reserved;
+ u64 flags;
+ int cached;
+ int ro;
+ int dirty;
+
+ struct btrfs_space_info *space_info;
+
+ /* free space cache stuff */
+ struct rb_root free_space_bytes;
+ struct rb_root free_space_offset;
+
+ /* block group cache stuff */
+ struct rb_node cache_node;
+
+ /* for block groups in the same raid type */
+ struct list_head list;
+
+ /* usage count */
+ atomic_t count;
+};
+
+struct btrfs_leaf_ref_tree {
+ struct rb_root root;
+ struct list_head list;
+ spinlock_t lock;
+};
+
+struct btrfs_device;
+struct btrfs_fs_devices;
+struct btrfs_fs_info {
+ u8 fsid[BTRFS_FSID_SIZE];
+ u8 chunk_tree_uuid[BTRFS_UUID_SIZE];
+ struct btrfs_root *extent_root;
+ struct btrfs_root *tree_root;
+ struct btrfs_root *chunk_root;
+ struct btrfs_root *dev_root;
+ struct btrfs_root *fs_root;
+ struct btrfs_root *csum_root;
+
+ /* the log root tree is a directory of all the other log roots */
+ struct btrfs_root *log_root_tree;
+ struct radix_tree_root fs_roots_radix;
+
+ /* block group cache stuff */
+ spinlock_t block_group_cache_lock;
+ struct rb_root block_group_cache_tree;
+
+ struct extent_io_tree pinned_extents;
+ struct extent_io_tree pending_del;
+ struct extent_io_tree extent_ins;
+
+ /* logical->physical extent mapping */
+ struct btrfs_mapping_tree mapping_tree;
+
+ u64 generation;
+ u64 last_trans_committed;
+ u64 last_trans_new_blockgroup;
+ u64 open_ioctl_trans;
+ unsigned long mount_opt;
+ u64 max_extent;
+ u64 max_inline;
+ u64 alloc_start;
+ struct btrfs_transaction *running_transaction;
+ wait_queue_head_t transaction_throttle;
+ wait_queue_head_t transaction_wait;
+
+ wait_queue_head_t async_submit_wait;
+ wait_queue_head_t tree_log_wait;
+
+ struct btrfs_super_block super_copy;
+ struct btrfs_super_block super_for_commit;
+ struct block_device *__bdev;
+ struct super_block *sb;
+ struct inode *btree_inode;
+ struct backing_dev_info bdi;
+ spinlock_t hash_lock;
+ struct mutex trans_mutex;
+ struct mutex tree_log_mutex;
+ struct mutex transaction_kthread_mutex;
+ struct mutex cleaner_mutex;
+ struct mutex extent_ins_mutex;
+ struct mutex pinned_mutex;
+ struct mutex chunk_mutex;
+ struct mutex drop_mutex;
+ struct mutex volume_mutex;
+ struct mutex tree_reloc_mutex;
+ struct list_head trans_list;
+ struct list_head hashers;
+ struct list_head dead_roots;
+
+ atomic_t nr_async_submits;
+ atomic_t async_submit_draining;
+ atomic_t nr_async_bios;
+ atomic_t async_delalloc_pages;
+ atomic_t tree_log_writers;
+ atomic_t tree_log_commit;
+ unsigned long tree_log_batch;
+ u64 tree_log_transid;
+
+ /*
+ * this is used by the balancing code to wait for all the pending
+ * ordered extents
+ */
+ spinlock_t ordered_extent_lock;
+ struct list_head ordered_extents;
+ struct list_head delalloc_inodes;
+
+ /*
+ * there is a pool of worker threads for checksumming during writes
+ * and a pool for checksumming after reads. This is because readers
+ * can run with FS locks held, and the writers may be waiting for
+ * those locks. We don't want ordering in the pending list to cause
+ * deadlocks, and so the two are serviced separately.
+ *
+ * A third pool does submit_bio to avoid deadlocking with the other
+ * two
+ */
+ struct btrfs_workers workers;
+ struct btrfs_workers delalloc_workers;
+ struct btrfs_workers endio_workers;
+ struct btrfs_workers endio_meta_workers;
+ struct btrfs_workers endio_meta_write_workers;
+ struct btrfs_workers endio_write_workers;
+ struct btrfs_workers submit_workers;
+ /*
+ * fixup workers take dirty pages that didn't properly go through
+ * the cow mechanism and make them safe to write. It happens
+ * for the sys_munmap function call path
+ */
+ struct btrfs_workers fixup_workers;
+ struct task_struct *transaction_kthread;
+ struct task_struct *cleaner_kthread;
+ int thread_pool_size;
+
+ /* tree relocation relocated fields */
+ struct list_head dead_reloc_roots;
+ struct btrfs_leaf_ref_tree reloc_ref_tree;
+ struct btrfs_leaf_ref_tree shared_ref_tree;
+
+ struct kobject super_kobj;
+ struct completion kobj_unregister;
+ int do_barriers;
+ int closing;
+ int log_root_recovering;
+ atomic_t throttles;
+ atomic_t throttle_gen;
+
+ u64 total_pinned;
+ struct list_head dirty_cowonly_roots;
+
+ struct btrfs_fs_devices *fs_devices;
+ struct list_head space_info;
+ spinlock_t delalloc_lock;
+ spinlock_t new_trans_lock;
+ u64 delalloc_bytes;
+ u64 last_alloc;
+ u64 last_data_alloc;
+
+ spinlock_t ref_cache_lock;
+ u64 total_ref_cache_size;
+
+ u64 avail_data_alloc_bits;
+ u64 avail_metadata_alloc_bits;
+ u64 avail_system_alloc_bits;
+ u64 data_alloc_profile;
+ u64 metadata_alloc_profile;
+ u64 system_alloc_profile;
+
+ void *bdev_holder;
+};
+
+/*
+ * in ram representation of the tree. extent_root is used for all allocations
+ * and for the extent tree extent_root root.
+ */
+struct btrfs_dirty_root;
+struct btrfs_root {
+ struct extent_buffer *node;
+
+ /* the node lock is held while changing the node pointer */
+ spinlock_t node_lock;
+
+ struct extent_buffer *commit_root;
+ struct btrfs_leaf_ref_tree *ref_tree;
+ struct btrfs_leaf_ref_tree ref_tree_struct;
+ struct btrfs_dirty_root *dirty_root;
+ struct btrfs_root *log_root;
+ struct btrfs_root *reloc_root;
+
+ struct btrfs_root_item root_item;
+ struct btrfs_key root_key;
+ struct btrfs_fs_info *fs_info;
+ struct extent_io_tree dirty_log_pages;
+
+ struct kobject root_kobj;
+ struct completion kobj_unregister;
+ struct mutex objectid_mutex;
+ struct mutex log_mutex;
+
+ u64 objectid;
+ u64 last_trans;
+
+ /* data allocations are done in sectorsize units */
+ u32 sectorsize;
+
+ /* node allocations are done in nodesize units */
+ u32 nodesize;
+
+ /* leaf allocations are done in leafsize units */
+ u32 leafsize;
+
+ u32 stripesize;
+
+ u32 type;
+ u64 highest_inode;
+ u64 last_inode_alloc;
+ int ref_cows;
+ int track_dirty;
+ u64 defrag_trans_start;
+ struct btrfs_key defrag_progress;
+ struct btrfs_key defrag_max;
+ int defrag_running;
+ int defrag_level;
+ char *name;
+ int in_sysfs;
+
+ /* the dirty list is only used by non-reference counted roots */
+ struct list_head dirty_list;
+
+ spinlock_t list_lock;
+ struct list_head dead_list;
+ struct list_head orphan_list;
+
+ /*
+ * right now this just gets used so that a root has its own devid
+ * for stat. It may be used for more later
+ */
+ struct super_block anon_super;
+};
+
+/*
+
+ * inode items have the data typically returned from stat and store other
+ * info about object characteristics. There is one for every file and dir in
+ * the FS
+ */
+#define BTRFS_INODE_ITEM_KEY 1
+#define BTRFS_INODE_REF_KEY 12
+#define BTRFS_XATTR_ITEM_KEY 24
+#define BTRFS_ORPHAN_ITEM_KEY 48
+/* reserve 2-15 close to the inode for later flexibility */
+
+/*
+ * dir items are the name -> inode pointers in a directory. There is one
+ * for every name in a directory.
+ */
+#define BTRFS_DIR_LOG_ITEM_KEY 60
+#define BTRFS_DIR_LOG_INDEX_KEY 72
+#define BTRFS_DIR_ITEM_KEY 84
+#define BTRFS_DIR_INDEX_KEY 96
+/*
+ * extent data is for file data
+ */
+#define BTRFS_EXTENT_DATA_KEY 108
+
+/*
+ * extent csums are stored in a separate tree and hold csums for
+ * an entire extent on disk.
+ */
+#define BTRFS_EXTENT_CSUM_KEY 128
+
+/*
+ * root items point to tree roots. There are typically in the root
+ * tree used by the super block to find all the other trees
+ */
+#define BTRFS_ROOT_ITEM_KEY 132
+
+/*
+ * root backrefs tie subvols and snapshots to the directory entries that
+ * reference them
+ */
+#define BTRFS_ROOT_BACKREF_KEY 144
+
+/*
+ * root refs make a fast index for listing all of the snapshots and
+ * subvolumes referenced by a given root. They point directly to the
+ * directory item in the root that references the subvol
+ */
+#define BTRFS_ROOT_REF_KEY 156
+
+/*
+ * extent items are in the extent map tree. These record which blocks
+ * are used, and how many references there are to each block
+ */
+#define BTRFS_EXTENT_ITEM_KEY 168
+#define BTRFS_EXTENT_REF_KEY 180
+
+/*
+ * block groups give us hints into the extent allocation trees. Which
+ * blocks are free etc etc
+ */
+#define BTRFS_BLOCK_GROUP_ITEM_KEY 192
+
+#define BTRFS_DEV_EXTENT_KEY 204
+#define BTRFS_DEV_ITEM_KEY 216
+#define BTRFS_CHUNK_ITEM_KEY 228
+
+/*
+ * string items are for debugging. They just store a short string of
+ * data in the FS
+ */
+#define BTRFS_STRING_ITEM_KEY 253
+
+#define BTRFS_MOUNT_NODATASUM (1 << 0)
+#define BTRFS_MOUNT_NODATACOW (1 << 1)
+#define BTRFS_MOUNT_NOBARRIER (1 << 2)
+#define BTRFS_MOUNT_SSD (1 << 3)
+#define BTRFS_MOUNT_DEGRADED (1 << 4)
+#define BTRFS_MOUNT_COMPRESS (1 << 5)
+
+#define btrfs_clear_opt(o, opt) ((o) &= ~BTRFS_MOUNT_##opt)
+#define btrfs_set_opt(o, opt) ((o) |= BTRFS_MOUNT_##opt)
+#define btrfs_test_opt(root, opt) ((root)->fs_info->mount_opt & \
+ BTRFS_MOUNT_##opt)
+/*
+ * Inode flags
+ */
+#define BTRFS_INODE_NODATASUM (1 << 0)
+#define BTRFS_INODE_NODATACOW (1 << 1)
+#define BTRFS_INODE_READONLY (1 << 2)
+#define BTRFS_INODE_NOCOMPRESS (1 << 3)
+#define BTRFS_INODE_PREALLOC (1 << 4)
+#define btrfs_clear_flag(inode, flag) (BTRFS_I(inode)->flags &= \
+ ~BTRFS_INODE_##flag)
+#define btrfs_set_flag(inode, flag) (BTRFS_I(inode)->flags |= \
+ BTRFS_INODE_##flag)
+#define btrfs_test_flag(inode, flag) (BTRFS_I(inode)->flags & \
+ BTRFS_INODE_##flag)
+/* some macros to generate set/get funcs for the struct fields. This
+ * assumes there is a lefoo_to_cpu for every type, so lets make a simple
+ * one for u8:
+ */
+#define le8_to_cpu(v) (v)
+#define cpu_to_le8(v) (v)
+#define __le8 u8
+
+#define read_eb_member(eb, ptr, type, member, result) ( \
+ read_extent_buffer(eb, (char *)(result), \
+ ((unsigned long)(ptr)) + \
+ offsetof(type, member), \
+ sizeof(((type *)0)->member)))
+
+#define write_eb_member(eb, ptr, type, member, result) ( \
+ write_extent_buffer(eb, (char *)(result), \
+ ((unsigned long)(ptr)) + \
+ offsetof(type, member), \
+ sizeof(((type *)0)->member)))
+
+#ifndef BTRFS_SETGET_FUNCS
+#define BTRFS_SETGET_FUNCS(name, type, member, bits) \
+u##bits btrfs_##name(struct extent_buffer *eb, type *s); \
+void btrfs_set_##name(struct extent_buffer *eb, type *s, u##bits val);
+#endif
+
+#define BTRFS_SETGET_HEADER_FUNCS(name, type, member, bits) \
+static inline u##bits btrfs_##name(struct extent_buffer *eb) \
+{ \
+ type *p = kmap_atomic(eb->first_page, KM_USER0); \
+ u##bits res = le##bits##_to_cpu(p->member); \
+ kunmap_atomic(p, KM_USER0); \
+ return res; \
+} \
+static inline void btrfs_set_##name(struct extent_buffer *eb, \
+ u##bits val) \
+{ \
+ type *p = kmap_atomic(eb->first_page, KM_USER0); \
+ p->member = cpu_to_le##bits(val); \
+ kunmap_atomic(p, KM_USER0); \
+}
+
+#define BTRFS_SETGET_STACK_FUNCS(name, type, member, bits) \
+static inline u##bits btrfs_##name(type *s) \
+{ \
+ return le##bits##_to_cpu(s->member); \
+} \
+static inline void btrfs_set_##name(type *s, u##bits val) \
+{ \
+ s->member = cpu_to_le##bits(val); \
+}
+
+BTRFS_SETGET_FUNCS(device_type, struct btrfs_dev_item, type, 64);
+BTRFS_SETGET_FUNCS(device_total_bytes, struct btrfs_dev_item, total_bytes, 64);
+BTRFS_SETGET_FUNCS(device_bytes_used, struct btrfs_dev_item, bytes_used, 64);
+BTRFS_SETGET_FUNCS(device_io_align, struct btrfs_dev_item, io_align, 32);
+BTRFS_SETGET_FUNCS(device_io_width, struct btrfs_dev_item, io_width, 32);
+BTRFS_SETGET_FUNCS(device_start_offset, struct btrfs_dev_item,
+ start_offset, 64);
+BTRFS_SETGET_FUNCS(device_sector_size, struct btrfs_dev_item, sector_size, 32);
+BTRFS_SETGET_FUNCS(device_id, struct btrfs_dev_item, devid, 64);
+BTRFS_SETGET_FUNCS(device_group, struct btrfs_dev_item, dev_group, 32);
+BTRFS_SETGET_FUNCS(device_seek_speed, struct btrfs_dev_item, seek_speed, 8);
+BTRFS_SETGET_FUNCS(device_bandwidth, struct btrfs_dev_item, bandwidth, 8);
+BTRFS_SETGET_FUNCS(device_generation, struct btrfs_dev_item, generation, 64);
+
+BTRFS_SETGET_STACK_FUNCS(stack_device_type, struct btrfs_dev_item, type, 64);
+BTRFS_SETGET_STACK_FUNCS(stack_device_total_bytes, struct btrfs_dev_item,
+ total_bytes, 64);
+BTRFS_SETGET_STACK_FUNCS(stack_device_bytes_used, struct btrfs_dev_item,
+ bytes_used, 64);
+BTRFS_SETGET_STACK_FUNCS(stack_device_io_align, struct btrfs_dev_item,
+ io_align, 32);
+BTRFS_SETGET_STACK_FUNCS(stack_device_io_width, struct btrfs_dev_item,
+ io_width, 32);
+BTRFS_SETGET_STACK_FUNCS(stack_device_sector_size, struct btrfs_dev_item,
+ sector_size, 32);
+BTRFS_SETGET_STACK_FUNCS(stack_device_id, struct btrfs_dev_item, devid, 64);
+BTRFS_SETGET_STACK_FUNCS(stack_device_group, struct btrfs_dev_item,
+ dev_group, 32);
+BTRFS_SETGET_STACK_FUNCS(stack_device_seek_speed, struct btrfs_dev_item,
+ seek_speed, 8);
+BTRFS_SETGET_STACK_FUNCS(stack_device_bandwidth, struct btrfs_dev_item,
+ bandwidth, 8);
+BTRFS_SETGET_STACK_FUNCS(stack_device_generation, struct btrfs_dev_item,
+ generation, 64);
+
+static inline char *btrfs_device_uuid(struct btrfs_dev_item *d)
+{
+ return (char *)d + offsetof(struct btrfs_dev_item, uuid);
+}
+
+static inline char *btrfs_device_fsid(struct btrfs_dev_item *d)
+{
+ return (char *)d + offsetof(struct btrfs_dev_item, fsid);
+}
+
+BTRFS_SETGET_FUNCS(chunk_length, struct btrfs_chunk, length, 64);
+BTRFS_SETGET_FUNCS(chunk_owner, struct btrfs_chunk, owner, 64);
+BTRFS_SETGET_FUNCS(chunk_stripe_len, struct btrfs_chunk, stripe_len, 64);
+BTRFS_SETGET_FUNCS(chunk_io_align, struct btrfs_chunk, io_align, 32);
+BTRFS_SETGET_FUNCS(chunk_io_width, struct btrfs_chunk, io_width, 32);
+BTRFS_SETGET_FUNCS(chunk_sector_size, struct btrfs_chunk, sector_size, 32);
+BTRFS_SETGET_FUNCS(chunk_type, struct btrfs_chunk, type, 64);
+BTRFS_SETGET_FUNCS(chunk_num_stripes, struct btrfs_chunk, num_stripes, 16);
+BTRFS_SETGET_FUNCS(chunk_sub_stripes, struct btrfs_chunk, sub_stripes, 16);
+BTRFS_SETGET_FUNCS(stripe_devid, struct btrfs_stripe, devid, 64);
+BTRFS_SETGET_FUNCS(stripe_offset, struct btrfs_stripe, offset, 64);
+
+static inline char *btrfs_stripe_dev_uuid(struct btrfs_stripe *s)
+{
+ return (char *)s + offsetof(struct btrfs_stripe, dev_uuid);
+}
+
+BTRFS_SETGET_STACK_FUNCS(stack_chunk_length, struct btrfs_chunk, length, 64);
+BTRFS_SETGET_STACK_FUNCS(stack_chunk_owner, struct btrfs_chunk, owner, 64);
+BTRFS_SETGET_STACK_FUNCS(stack_chunk_stripe_len, struct btrfs_chunk,
+ stripe_len, 64);
+BTRFS_SETGET_STACK_FUNCS(stack_chunk_io_align, struct btrfs_chunk,
+ io_align, 32);
+BTRFS_SETGET_STACK_FUNCS(stack_chunk_io_width, struct btrfs_chunk,
+ io_width, 32);
+BTRFS_SETGET_STACK_FUNCS(stack_chunk_sector_size, struct btrfs_chunk,
+ sector_size, 32);
+BTRFS_SETGET_STACK_FUNCS(stack_chunk_type, struct btrfs_chunk, type, 64);
+BTRFS_SETGET_STACK_FUNCS(stack_chunk_num_stripes, struct btrfs_chunk,
+ num_stripes, 16);
+BTRFS_SETGET_STACK_FUNCS(stack_chunk_sub_stripes, struct btrfs_chunk,
+ sub_stripes, 16);
+BTRFS_SETGET_STACK_FUNCS(stack_stripe_devid, struct btrfs_stripe, devid, 64);
+BTRFS_SETGET_STACK_FUNCS(stack_stripe_offset, struct btrfs_stripe, offset, 64);
+
+static inline struct btrfs_stripe *btrfs_stripe_nr(struct btrfs_chunk *c,
+ int nr)
+{
+ unsigned long offset = (unsigned long)c;
+ offset += offsetof(struct btrfs_chunk, stripe);
+ offset += nr * sizeof(struct btrfs_stripe);
+ return (struct btrfs_stripe *)offset;
+}
+
+static inline char *btrfs_stripe_dev_uuid_nr(struct btrfs_chunk *c, int nr)
+{
+ return btrfs_stripe_dev_uuid(btrfs_stripe_nr(c, nr));
+}
+
+static inline u64 btrfs_stripe_offset_nr(struct extent_buffer *eb,
+ struct btrfs_chunk *c, int nr)
+{
+ return btrfs_stripe_offset(eb, btrfs_stripe_nr(c, nr));
+}
+
+static inline void btrfs_set_stripe_offset_nr(struct extent_buffer *eb,
+ struct btrfs_chunk *c, int nr,
+ u64 val)
+{
+ btrfs_set_stripe_offset(eb, btrfs_stripe_nr(c, nr), val);
+}
+
+static inline u64 btrfs_stripe_devid_nr(struct extent_buffer *eb,
+ struct btrfs_chunk *c, int nr)
+{
+ return btrfs_stripe_devid(eb, btrfs_stripe_nr(c, nr));
+}
+
+static inline void btrfs_set_stripe_devid_nr(struct extent_buffer *eb,
+ struct btrfs_chunk *c, int nr,
+ u64 val)
+{
+ btrfs_set_stripe_devid(eb, btrfs_stripe_nr(c, nr), val);
+}
+
+/* struct btrfs_block_group_item */
+BTRFS_SETGET_STACK_FUNCS(block_group_used, struct btrfs_block_group_item,
+ used, 64);
+BTRFS_SETGET_FUNCS(disk_block_group_used, struct btrfs_block_group_item,
+ used, 64);
+BTRFS_SETGET_STACK_FUNCS(block_group_chunk_objectid,
+ struct btrfs_block_group_item, chunk_objectid, 64);
+
+BTRFS_SETGET_FUNCS(disk_block_group_chunk_objectid,
+ struct btrfs_block_group_item, chunk_objectid, 64);
+BTRFS_SETGET_FUNCS(disk_block_group_flags,
+ struct btrfs_block_group_item, flags, 64);
+BTRFS_SETGET_STACK_FUNCS(block_group_flags,
+ struct btrfs_block_group_item, flags, 64);
+
+/* struct btrfs_inode_ref */
+BTRFS_SETGET_FUNCS(inode_ref_name_len, struct btrfs_inode_ref, name_len, 16);
+BTRFS_SETGET_FUNCS(inode_ref_index, struct btrfs_inode_ref, index, 64);
+
+/* struct btrfs_inode_item */
+BTRFS_SETGET_FUNCS(inode_generation, struct btrfs_inode_item, generation, 64);
+BTRFS_SETGET_FUNCS(inode_sequence, struct btrfs_inode_item, sequence, 64);
+BTRFS_SETGET_FUNCS(inode_transid, struct btrfs_inode_item, transid, 64);
+BTRFS_SETGET_FUNCS(inode_size, struct btrfs_inode_item, size, 64);
+BTRFS_SETGET_FUNCS(inode_nbytes, struct btrfs_inode_item, nbytes, 64);
+BTRFS_SETGET_FUNCS(inode_block_group, struct btrfs_inode_item, block_group, 64);
+BTRFS_SETGET_FUNCS(inode_nlink, struct btrfs_inode_item, nlink, 32);
+BTRFS_SETGET_FUNCS(inode_uid, struct btrfs_inode_item, uid, 32);
+BTRFS_SETGET_FUNCS(inode_gid, struct btrfs_inode_item, gid, 32);
+BTRFS_SETGET_FUNCS(inode_mode, struct btrfs_inode_item, mode, 32);
+BTRFS_SETGET_FUNCS(inode_rdev, struct btrfs_inode_item, rdev, 64);
+BTRFS_SETGET_FUNCS(inode_flags, struct btrfs_inode_item, flags, 64);
+
+static inline struct btrfs_timespec *
+btrfs_inode_atime(struct btrfs_inode_item *inode_item)
+{
+ unsigned long ptr = (unsigned long)inode_item;
+ ptr += offsetof(struct btrfs_inode_item, atime);
+ return (struct btrfs_timespec *)ptr;
+}
+
+static inline struct btrfs_timespec *
+btrfs_inode_mtime(struct btrfs_inode_item *inode_item)
+{
+ unsigned long ptr = (unsigned long)inode_item;
+ ptr += offsetof(struct btrfs_inode_item, mtime);
+ return (struct btrfs_timespec *)ptr;
+}
+
+static inline struct btrfs_timespec *
+btrfs_inode_ctime(struct btrfs_inode_item *inode_item)
+{
+ unsigned long ptr = (unsigned long)inode_item;
+ ptr += offsetof(struct btrfs_inode_item, ctime);
+ return (struct btrfs_timespec *)ptr;
+}
+
+static inline struct btrfs_timespec *
+btrfs_inode_otime(struct btrfs_inode_item *inode_item)
+{
+ unsigned long ptr = (unsigned long)inode_item;
+ ptr += offsetof(struct btrfs_inode_item, otime);
+ return (struct btrfs_timespec *)ptr;
+}
+
+BTRFS_SETGET_FUNCS(timespec_sec, struct btrfs_timespec, sec, 64);
+BTRFS_SETGET_FUNCS(timespec_nsec, struct btrfs_timespec, nsec, 32);
+
+/* struct btrfs_dev_extent */
+BTRFS_SETGET_FUNCS(dev_extent_chunk_tree, struct btrfs_dev_extent,
+ chunk_tree, 64);
+BTRFS_SETGET_FUNCS(dev_extent_chunk_objectid, struct btrfs_dev_extent,
+ chunk_objectid, 64);
+BTRFS_SETGET_FUNCS(dev_extent_chunk_offset, struct btrfs_dev_extent,
+ chunk_offset, 64);
+BTRFS_SETGET_FUNCS(dev_extent_length, struct btrfs_dev_extent, length, 64);
+
+static inline u8 *btrfs_dev_extent_chunk_tree_uuid(struct btrfs_dev_extent *dev)
+{
+ unsigned long ptr = offsetof(struct btrfs_dev_extent, chunk_tree_uuid);
+ return (u8 *)((unsigned long)dev + ptr);
+}
+
+/* struct btrfs_extent_ref */
+BTRFS_SETGET_FUNCS(ref_root, struct btrfs_extent_ref, root, 64);
+BTRFS_SETGET_FUNCS(ref_generation, struct btrfs_extent_ref, generation, 64);
+BTRFS_SETGET_FUNCS(ref_objectid, struct btrfs_extent_ref, objectid, 64);
+BTRFS_SETGET_FUNCS(ref_num_refs, struct btrfs_extent_ref, num_refs, 32);
+
+BTRFS_SETGET_STACK_FUNCS(stack_ref_root, struct btrfs_extent_ref, root, 64);
+BTRFS_SETGET_STACK_FUNCS(stack_ref_generation, struct btrfs_extent_ref,
+ generation, 64);
+BTRFS_SETGET_STACK_FUNCS(stack_ref_objectid, struct btrfs_extent_ref,
+ objectid, 64);
+BTRFS_SETGET_STACK_FUNCS(stack_ref_num_refs, struct btrfs_extent_ref,
+ num_refs, 32);
+
+/* struct btrfs_extent_item */
+BTRFS_SETGET_FUNCS(extent_refs, struct btrfs_extent_item, refs, 32);
+BTRFS_SETGET_STACK_FUNCS(stack_extent_refs, struct btrfs_extent_item,
+ refs, 32);
+
+/* struct btrfs_node */
+BTRFS_SETGET_FUNCS(key_blockptr, struct btrfs_key_ptr, blockptr, 64);
+BTRFS_SETGET_FUNCS(key_generation, struct btrfs_key_ptr, generation, 64);
+
+static inline u64 btrfs_node_blockptr(struct extent_buffer *eb, int nr)
+{
+ unsigned long ptr;
+ ptr = offsetof(struct btrfs_node, ptrs) +
+ sizeof(struct btrfs_key_ptr) * nr;
+ return btrfs_key_blockptr(eb, (struct btrfs_key_ptr *)ptr);
+}
+
+static inline void btrfs_set_node_blockptr(struct extent_buffer *eb,
+ int nr, u64 val)
+{
+ unsigned long ptr;
+ ptr = offsetof(struct btrfs_node, ptrs) +
+ sizeof(struct btrfs_key_ptr) * nr;
+ btrfs_set_key_blockptr(eb, (struct btrfs_key_ptr *)ptr, val);
+}
+
+static inline u64 btrfs_node_ptr_generation(struct extent_buffer *eb, int nr)
+{
+ unsigned long ptr;
+ ptr = offsetof(struct btrfs_node, ptrs) +
+ sizeof(struct btrfs_key_ptr) * nr;
+ return btrfs_key_generation(eb, (struct btrfs_key_ptr *)ptr);
+}
+
+static inline void btrfs_set_node_ptr_generation(struct extent_buffer *eb,
+ int nr, u64 val)
+{
+ unsigned long ptr;
+ ptr = offsetof(struct btrfs_node, ptrs) +
+ sizeof(struct btrfs_key_ptr) * nr;
+ btrfs_set_key_generation(eb, (struct btrfs_key_ptr *)ptr, val);
+}
+
+static inline unsigned long btrfs_node_key_ptr_offset(int nr)
+{
+ return offsetof(struct btrfs_node, ptrs) +
+ sizeof(struct btrfs_key_ptr) * nr;
+}
+
+void btrfs_node_key(struct extent_buffer *eb,
+ struct btrfs_disk_key *disk_key, int nr);
+
+static inline void btrfs_set_node_key(struct extent_buffer *eb,
+ struct btrfs_disk_key *disk_key, int nr)
+{
+ unsigned long ptr;
+ ptr = btrfs_node_key_ptr_offset(nr);
+ write_eb_member(eb, (struct btrfs_key_ptr *)ptr,
+ struct btrfs_key_ptr, key, disk_key);
+}
+
+/* struct btrfs_item */
+BTRFS_SETGET_FUNCS(item_offset, struct btrfs_item, offset, 32);
+BTRFS_SETGET_FUNCS(item_size, struct btrfs_item, size, 32);
+
+static inline unsigned long btrfs_item_nr_offset(int nr)
+{
+ return offsetof(struct btrfs_leaf, items) +
+ sizeof(struct btrfs_item) * nr;
+}
+
+static inline struct btrfs_item *btrfs_item_nr(struct extent_buffer *eb,
+ int nr)
+{
+ return (struct btrfs_item *)btrfs_item_nr_offset(nr);
+}
+
+static inline u32 btrfs_item_end(struct extent_buffer *eb,
+ struct btrfs_item *item)
+{
+ return btrfs_item_offset(eb, item) + btrfs_item_size(eb, item);
+}
+
+static inline u32 btrfs_item_end_nr(struct extent_buffer *eb, int nr)
+{
+ return btrfs_item_end(eb, btrfs_item_nr(eb, nr));
+}
+
+static inline u32 btrfs_item_offset_nr(struct extent_buffer *eb, int nr)
+{
+ return btrfs_item_offset(eb, btrfs_item_nr(eb, nr));
+}
+
+static inline u32 btrfs_item_size_nr(struct extent_buffer *eb, int nr)
+{
+ return btrfs_item_size(eb, btrfs_item_nr(eb, nr));
+}
+
+static inline void btrfs_item_key(struct extent_buffer *eb,
+ struct btrfs_disk_key *disk_key, int nr)
+{
+ struct btrfs_item *item = btrfs_item_nr(eb, nr);
+ read_eb_member(eb, item, struct btrfs_item, key, disk_key);
+}
+
+static inline void btrfs_set_item_key(struct extent_buffer *eb,
+ struct btrfs_disk_key *disk_key, int nr)
+{
+ struct btrfs_item *item = btrfs_item_nr(eb, nr);
+ write_eb_member(eb, item, struct btrfs_item, key, disk_key);
+}
+
+BTRFS_SETGET_FUNCS(dir_log_end, struct btrfs_dir_log_item, end, 64);
+
+/*
+ * struct btrfs_root_ref
+ */
+BTRFS_SETGET_FUNCS(root_ref_dirid, struct btrfs_root_ref, dirid, 64);
+BTRFS_SETGET_FUNCS(root_ref_sequence, struct btrfs_root_ref, sequence, 64);
+BTRFS_SETGET_FUNCS(root_ref_name_len, struct btrfs_root_ref, name_len, 16);
+
+/* struct btrfs_dir_item */
+BTRFS_SETGET_FUNCS(dir_data_len, struct btrfs_dir_item, data_len, 16);
+BTRFS_SETGET_FUNCS(dir_type, struct btrfs_dir_item, type, 8);
+BTRFS_SETGET_FUNCS(dir_name_len, struct btrfs_dir_item, name_len, 16);
+BTRFS_SETGET_FUNCS(dir_transid, struct btrfs_dir_item, transid, 64);
+
+static inline void btrfs_dir_item_key(struct extent_buffer *eb,
+ struct btrfs_dir_item *item,
+ struct btrfs_disk_key *key)
+{
+ read_eb_member(eb, item, struct btrfs_dir_item, location, key);
+}
+
+static inline void btrfs_set_dir_item_key(struct extent_buffer *eb,
+ struct btrfs_dir_item *item,
+ struct btrfs_disk_key *key)
+{
+ write_eb_member(eb, item, struct btrfs_dir_item, location, key);
+}
+
+/* struct btrfs_disk_key */
+BTRFS_SETGET_STACK_FUNCS(disk_key_objectid, struct btrfs_disk_key,
+ objectid, 64);
+BTRFS_SETGET_STACK_FUNCS(disk_key_offset, struct btrfs_disk_key, offset, 64);
+BTRFS_SETGET_STACK_FUNCS(disk_key_type, struct btrfs_disk_key, type, 8);
+
+static inline void btrfs_disk_key_to_cpu(struct btrfs_key *cpu,
+ struct btrfs_disk_key *disk)
+{
+ cpu->offset = le64_to_cpu(disk->offset);
+ cpu->type = disk->type;
+ cpu->objectid = le64_to_cpu(disk->objectid);
+}
+
+static inline void btrfs_cpu_key_to_disk(struct btrfs_disk_key *disk,
+ struct btrfs_key *cpu)
+{
+ disk->offset = cpu_to_le64(cpu->offset);
+ disk->type = cpu->type;
+ disk->objectid = cpu_to_le64(cpu->objectid);
+}
+
+static inline void btrfs_node_key_to_cpu(struct extent_buffer *eb,
+ struct btrfs_key *key, int nr)
+{
+ struct btrfs_disk_key disk_key;
+ btrfs_node_key(eb, &disk_key, nr);
+ btrfs_disk_key_to_cpu(key, &disk_key);
+}
+
+static inline void btrfs_item_key_to_cpu(struct extent_buffer *eb,
+ struct btrfs_key *key, int nr)
+{
+ struct btrfs_disk_key disk_key;
+ btrfs_item_key(eb, &disk_key, nr);
+ btrfs_disk_key_to_cpu(key, &disk_key);
+}
+
+static inline void btrfs_dir_item_key_to_cpu(struct extent_buffer *eb,
+ struct btrfs_dir_item *item,
+ struct btrfs_key *key)
+{
+ struct btrfs_disk_key disk_key;
+ btrfs_dir_item_key(eb, item, &disk_key);
+ btrfs_disk_key_to_cpu(key, &disk_key);
+}
+
+
+static inline u8 btrfs_key_type(struct btrfs_key *key)
+{
+ return key->type;
+}
+
+static inline void btrfs_set_key_type(struct btrfs_key *key, u8 val)
+{
+ key->type = val;
+}
+
+/* struct btrfs_header */
+BTRFS_SETGET_HEADER_FUNCS(header_bytenr, struct btrfs_header, bytenr, 64);
+BTRFS_SETGET_HEADER_FUNCS(header_generation, struct btrfs_header,
+ generation, 64);
+BTRFS_SETGET_HEADER_FUNCS(header_owner, struct btrfs_header, owner, 64);
+BTRFS_SETGET_HEADER_FUNCS(header_nritems, struct btrfs_header, nritems, 32);
+BTRFS_SETGET_HEADER_FUNCS(header_flags, struct btrfs_header, flags, 64);
+BTRFS_SETGET_HEADER_FUNCS(header_level, struct btrfs_header, level, 8);
+
+static inline int btrfs_header_flag(struct extent_buffer *eb, u64 flag)
+{
+ return (btrfs_header_flags(eb) & flag) == flag;
+}
+
+static inline int btrfs_set_header_flag(struct extent_buffer *eb, u64 flag)
+{
+ u64 flags = btrfs_header_flags(eb);
+ btrfs_set_header_flags(eb, flags | flag);
+ return (flags & flag) == flag;
+}
+
+static inline int btrfs_clear_header_flag(struct extent_buffer *eb, u64 flag)
+{
+ u64 flags = btrfs_header_flags(eb);
+ btrfs_set_header_flags(eb, flags & ~flag);
+ return (flags & flag) == flag;
+}
+
+static inline u8 *btrfs_header_fsid(struct extent_buffer *eb)
+{
+ unsigned long ptr = offsetof(struct btrfs_header, fsid);
+ return (u8 *)ptr;
+}
+
+static inline u8 *btrfs_header_chunk_tree_uuid(struct extent_buffer *eb)
+{
+ unsigned long ptr = offsetof(struct btrfs_header, chunk_tree_uuid);
+ return (u8 *)ptr;
+}
+
+static inline u8 *btrfs_super_fsid(struct extent_buffer *eb)
+{
+ unsigned long ptr = offsetof(struct btrfs_super_block, fsid);
+ return (u8 *)ptr;
+}
+
+static inline u8 *btrfs_header_csum(struct extent_buffer *eb)
+{
+ unsigned long ptr = offsetof(struct btrfs_header, csum);
+ return (u8 *)ptr;
+}
+
+static inline struct btrfs_node *btrfs_buffer_node(struct extent_buffer *eb)
+{
+ return NULL;
+}
+
+static inline struct btrfs_leaf *btrfs_buffer_leaf(struct extent_buffer *eb)
+{
+ return NULL;
+}
+
+static inline struct btrfs_header *btrfs_buffer_header(struct extent_buffer *eb)
+{
+ return NULL;
+}
+
+static inline int btrfs_is_leaf(struct extent_buffer *eb)
+{
+ return btrfs_header_level(eb) == 0;
+}
+
+/* struct btrfs_root_item */
+BTRFS_SETGET_FUNCS(disk_root_generation, struct btrfs_root_item,
+ generation, 64);
+BTRFS_SETGET_FUNCS(disk_root_refs, struct btrfs_root_item, refs, 32);
+BTRFS_SETGET_FUNCS(disk_root_bytenr, struct btrfs_root_item, bytenr, 64);
+BTRFS_SETGET_FUNCS(disk_root_level, struct btrfs_root_item, level, 8);
+
+BTRFS_SETGET_STACK_FUNCS(root_generation, struct btrfs_root_item,
+ generation, 64);
+BTRFS_SETGET_STACK_FUNCS(root_bytenr, struct btrfs_root_item, bytenr, 64);
+BTRFS_SETGET_STACK_FUNCS(root_level, struct btrfs_root_item, level, 8);
+BTRFS_SETGET_STACK_FUNCS(root_dirid, struct btrfs_root_item, root_dirid, 64);
+BTRFS_SETGET_STACK_FUNCS(root_refs, struct btrfs_root_item, refs, 32);
+BTRFS_SETGET_STACK_FUNCS(root_flags, struct btrfs_root_item, flags, 64);
+BTRFS_SETGET_STACK_FUNCS(root_used, struct btrfs_root_item, bytes_used, 64);
+BTRFS_SETGET_STACK_FUNCS(root_limit, struct btrfs_root_item, byte_limit, 64);
+BTRFS_SETGET_STACK_FUNCS(root_last_snapshot, struct btrfs_root_item,
+ last_snapshot, 64);
+
+/* struct btrfs_super_block */
+
+BTRFS_SETGET_STACK_FUNCS(super_bytenr, struct btrfs_super_block, bytenr, 64);
+BTRFS_SETGET_STACK_FUNCS(super_flags, struct btrfs_super_block, flags, 64);
+BTRFS_SETGET_STACK_FUNCS(super_generation, struct btrfs_super_block,
+ generation, 64);
+BTRFS_SETGET_STACK_FUNCS(super_root, struct btrfs_super_block, root, 64);
+BTRFS_SETGET_STACK_FUNCS(super_sys_array_size,
+ struct btrfs_super_block, sys_chunk_array_size, 32);
+BTRFS_SETGET_STACK_FUNCS(super_chunk_root_generation,
+ struct btrfs_super_block, chunk_root_generation, 64);
+BTRFS_SETGET_STACK_FUNCS(super_root_level, struct btrfs_super_block,
+ root_level, 8);
+BTRFS_SETGET_STACK_FUNCS(super_chunk_root, struct btrfs_super_block,
+ chunk_root, 64);
+BTRFS_SETGET_STACK_FUNCS(super_chunk_root_level, struct btrfs_super_block,
+ chunk_root_level, 8);
+BTRFS_SETGET_STACK_FUNCS(super_log_root, struct btrfs_super_block,
+ log_root, 64);
+BTRFS_SETGET_STACK_FUNCS(super_log_root_transid, struct btrfs_super_block,
+ log_root_transid, 64);
+BTRFS_SETGET_STACK_FUNCS(super_log_root_level, struct btrfs_super_block,
+ log_root_level, 8);
+BTRFS_SETGET_STACK_FUNCS(super_total_bytes, struct btrfs_super_block,
+ total_bytes, 64);
+BTRFS_SETGET_STACK_FUNCS(super_bytes_used, struct btrfs_super_block,
+ bytes_used, 64);
+BTRFS_SETGET_STACK_FUNCS(super_sectorsize, struct btrfs_super_block,
+ sectorsize, 32);
+BTRFS_SETGET_STACK_FUNCS(super_nodesize, struct btrfs_super_block,
+ nodesize, 32);
+BTRFS_SETGET_STACK_FUNCS(super_leafsize, struct btrfs_super_block,
+ leafsize, 32);
+BTRFS_SETGET_STACK_FUNCS(super_stripesize, struct btrfs_super_block,
+ stripesize, 32);
+BTRFS_SETGET_STACK_FUNCS(super_root_dir, struct btrfs_super_block,
+ root_dir_objectid, 64);
+BTRFS_SETGET_STACK_FUNCS(super_num_devices, struct btrfs_super_block,
+ num_devices, 64);
+BTRFS_SETGET_STACK_FUNCS(super_compat_flags, struct btrfs_super_block,
+ compat_flags, 64);
+BTRFS_SETGET_STACK_FUNCS(super_compat_ro_flags, struct btrfs_super_block,
+ compat_flags, 64);
+BTRFS_SETGET_STACK_FUNCS(super_incompat_flags, struct btrfs_super_block,
+ incompat_flags, 64);
+BTRFS_SETGET_STACK_FUNCS(super_csum_type, struct btrfs_super_block,
+ csum_type, 16);
+
+static inline int btrfs_super_csum_size(struct btrfs_super_block *s)
+{
+ int t = btrfs_super_csum_type(s);
+ BUG_ON(t >= ARRAY_SIZE(btrfs_csum_sizes));
+ return btrfs_csum_sizes[t];
+}
+
+static inline unsigned long btrfs_leaf_data(struct extent_buffer *l)
+{
+ return offsetof(struct btrfs_leaf, items);
+}
+
+/* struct btrfs_file_extent_item */
+BTRFS_SETGET_FUNCS(file_extent_type, struct btrfs_file_extent_item, type, 8);
+
+static inline unsigned long
+btrfs_file_extent_inline_start(struct btrfs_file_extent_item *e)
+{
+ unsigned long offset = (unsigned long)e;
+ offset += offsetof(struct btrfs_file_extent_item, disk_bytenr);
+ return offset;
+}
+
+static inline u32 btrfs_file_extent_calc_inline_size(u32 datasize)
+{
+ return offsetof(struct btrfs_file_extent_item, disk_bytenr) + datasize;
+}
+
+BTRFS_SETGET_FUNCS(file_extent_disk_bytenr, struct btrfs_file_extent_item,
+ disk_bytenr, 64);
+BTRFS_SETGET_FUNCS(file_extent_generation, struct btrfs_file_extent_item,
+ generation, 64);
+BTRFS_SETGET_FUNCS(file_extent_disk_num_bytes, struct btrfs_file_extent_item,
+ disk_num_bytes, 64);
+BTRFS_SETGET_FUNCS(file_extent_offset, struct btrfs_file_extent_item,
+ offset, 64);
+BTRFS_SETGET_FUNCS(file_extent_num_bytes, struct btrfs_file_extent_item,
+ num_bytes, 64);
+BTRFS_SETGET_FUNCS(file_extent_ram_bytes, struct btrfs_file_extent_item,
+ ram_bytes, 64);
+BTRFS_SETGET_FUNCS(file_extent_compression, struct btrfs_file_extent_item,
+ compression, 8);
+BTRFS_SETGET_FUNCS(file_extent_encryption, struct btrfs_file_extent_item,
+ encryption, 8);
+BTRFS_SETGET_FUNCS(file_extent_other_encoding, struct btrfs_file_extent_item,
+ other_encoding, 16);
+
+/* this returns the number of file bytes represented by the inline item.
+ * If an item is compressed, this is the uncompressed size
+ */
+static inline u32 btrfs_file_extent_inline_len(struct extent_buffer *eb,
+ struct btrfs_file_extent_item *e)
+{
+ return btrfs_file_extent_ram_bytes(eb, e);
+}
+
+/*
+ * this returns the number of bytes used by the item on disk, minus the
+ * size of any extent headers. If a file is compressed on disk, this is
+ * the compressed size
+ */
+static inline u32 btrfs_file_extent_inline_item_len(struct extent_buffer *eb,
+ struct btrfs_item *e)
+{
+ unsigned long offset;
+ offset = offsetof(struct btrfs_file_extent_item, disk_bytenr);
+ return btrfs_item_size(eb, e) - offset;
+}
+
+static inline struct btrfs_root *btrfs_sb(struct super_block *sb)
+{
+ return sb->s_fs_info;
+}
+
+static inline int btrfs_set_root_name(struct btrfs_root *root,
+ const char *name, int len)
+{
+ /* if we already have a name just free it */
+ kfree(root->name);
+
+ root->name = kmalloc(len+1, GFP_KERNEL);
+ if (!root->name)
+ return -ENOMEM;
+
+ memcpy(root->name, name, len);
+ root->name[len] = '\0';
+
+ return 0;
+}
+
+static inline u32 btrfs_level_size(struct btrfs_root *root, int level)
+{
+ if (level == 0)
+ return root->leafsize;
+ return root->nodesize;
+}
+
+/* helper function to cast into the data area of the leaf. */
+#define btrfs_item_ptr(leaf, slot, type) \
+ ((type *)(btrfs_leaf_data(leaf) + \
+ btrfs_item_offset_nr(leaf, slot)))
+
+#define btrfs_item_ptr_offset(leaf, slot) \
+ ((unsigned long)(btrfs_leaf_data(leaf) + \
+ btrfs_item_offset_nr(leaf, slot)))
+
+static inline struct dentry *fdentry(struct file *file)
+{
+ return file->f_path.dentry;
+}
+
+/* extent-tree.c */
+int btrfs_lookup_extent(struct btrfs_root *root, u64 start, u64 len);
+int btrfs_lookup_extent_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 bytenr,
+ u64 num_bytes, u32 *refs);
+int btrfs_update_pinned_extents(struct btrfs_root *root,
+ u64 bytenr, u64 num, int pin);
+int btrfs_drop_leaf_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct extent_buffer *leaf);
+int btrfs_cross_ref_exist(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 objectid, u64 bytenr);
+int btrfs_extent_post_op(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root);
+int btrfs_copy_pinned(struct btrfs_root *root, struct extent_io_tree *copy);
+struct btrfs_block_group_cache *btrfs_lookup_block_group(
+ struct btrfs_fs_info *info,
+ u64 bytenr);
+u64 btrfs_find_block_group(struct btrfs_root *root,
+ u64 search_start, u64 search_hint, int owner);
+struct extent_buffer *btrfs_alloc_free_block(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u32 blocksize, u64 parent,
+ u64 root_objectid,
+ u64 ref_generation,
+ int level,
+ u64 hint,
+ u64 empty_size);
+struct extent_buffer *btrfs_init_new_buffer(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 bytenr, u32 blocksize);
+int btrfs_alloc_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 num_bytes, u64 parent, u64 min_bytes,
+ u64 root_objectid, u64 ref_generation,
+ u64 owner, u64 empty_size, u64 hint_byte,
+ u64 search_end, struct btrfs_key *ins, u64 data);
+int btrfs_alloc_reserved_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 parent,
+ u64 root_objectid, u64 ref_generation,
+ u64 owner, struct btrfs_key *ins);
+int btrfs_alloc_logged_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 parent,
+ u64 root_objectid, u64 ref_generation,
+ u64 owner, struct btrfs_key *ins);
+int btrfs_reserve_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 num_bytes, u64 min_alloc_size,
+ u64 empty_size, u64 hint_byte,
+ u64 search_end, struct btrfs_key *ins,
+ u64 data);
+int btrfs_inc_ref(struct btrfs_trans_handle *trans, struct btrfs_root *root,
+ struct extent_buffer *orig_buf, struct extent_buffer *buf,
+ u32 *nr_extents);
+int btrfs_cache_ref(struct btrfs_trans_handle *trans, struct btrfs_root *root,
+ struct extent_buffer *buf, u32 nr_extents);
+int btrfs_update_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct extent_buffer *orig_buf,
+ struct extent_buffer *buf, int start_slot, int nr);
+int btrfs_free_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 bytenr, u64 num_bytes, u64 parent,
+ u64 root_objectid, u64 ref_generation,
+ u64 owner_objectid, int pin);
+int btrfs_free_reserved_extent(struct btrfs_root *root, u64 start, u64 len);
+int btrfs_finish_extent_commit(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct extent_io_tree *unpin);
+int btrfs_inc_extent_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 bytenr, u64 num_bytes, u64 parent,
+ u64 root_objectid, u64 ref_generation,
+ u64 owner_objectid);
+int btrfs_update_extent_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 bytenr,
+ u64 orig_parent, u64 parent,
+ u64 root_objectid, u64 ref_generation,
+ u64 owner_objectid);
+int btrfs_write_dirty_block_groups(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root);
+int btrfs_extent_readonly(struct btrfs_root *root, u64 bytenr);
+int btrfs_free_block_groups(struct btrfs_fs_info *info);
+int btrfs_read_block_groups(struct btrfs_root *root);
+int btrfs_make_block_group(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 bytes_used,
+ u64 type, u64 chunk_objectid, u64 chunk_offset,
+ u64 size);
+int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 group_start);
+int btrfs_relocate_block_group(struct btrfs_root *root, u64 group_start);
+int btrfs_free_reloc_root(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root);
+int btrfs_drop_dead_reloc_roots(struct btrfs_root *root);
+int btrfs_reloc_tree_cache_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct extent_buffer *buf, u64 orig_start);
+int btrfs_add_dead_reloc_root(struct btrfs_root *root);
+int btrfs_cleanup_reloc_trees(struct btrfs_root *root);
+int btrfs_reloc_clone_csums(struct inode *inode, u64 file_pos, u64 len);
+u64 btrfs_reduce_alloc_profile(struct btrfs_root *root, u64 flags);
+/* ctree.c */
+int btrfs_previous_item(struct btrfs_root *root,
+ struct btrfs_path *path, u64 min_objectid,
+ int type);
+int btrfs_merge_path(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_key *node_keys,
+ u64 *nodes, int lowest_level);
+int btrfs_set_item_key_safe(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct btrfs_path *path,
+ struct btrfs_key *new_key);
+struct extent_buffer *btrfs_root_node(struct btrfs_root *root);
+struct extent_buffer *btrfs_lock_root_node(struct btrfs_root *root);
+int btrfs_find_next_key(struct btrfs_root *root, struct btrfs_path *path,
+ struct btrfs_key *key, int lowest_level,
+ int cache_only, u64 min_trans);
+int btrfs_search_forward(struct btrfs_root *root, struct btrfs_key *min_key,
+ struct btrfs_key *max_key,
+ struct btrfs_path *path, int cache_only,
+ u64 min_trans);
+int btrfs_cow_block(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct extent_buffer *buf,
+ struct extent_buffer *parent, int parent_slot,
+ struct extent_buffer **cow_ret, u64 prealloc_dest);
+int btrfs_copy_root(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct extent_buffer *buf,
+ struct extent_buffer **cow_ret, u64 new_root_objectid);
+int btrfs_extend_item(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root, struct btrfs_path *path, u32 data_size);
+int btrfs_truncate_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ u32 new_size, int from_end);
+int btrfs_split_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct btrfs_key *new_key,
+ unsigned long split_offset);
+int btrfs_search_slot(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root, struct btrfs_key *key, struct btrfs_path *p, int
+ ins_len, int cow);
+int btrfs_realloc_node(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct extent_buffer *parent,
+ int start_slot, int cache_only, u64 *last_ret,
+ struct btrfs_key *progress);
+void btrfs_release_path(struct btrfs_root *root, struct btrfs_path *p);
+struct btrfs_path *btrfs_alloc_path(void);
+void btrfs_free_path(struct btrfs_path *p);
+void btrfs_init_path(struct btrfs_path *p);
+int btrfs_del_items(struct btrfs_trans_handle *trans, struct btrfs_root *root,
+ struct btrfs_path *path, int slot, int nr);
+int btrfs_del_leaf(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, u64 bytenr);
+static inline int btrfs_del_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path)
+{
+ return btrfs_del_items(trans, root, path, path->slots[0], 1);
+}
+
+int btrfs_insert_item(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root, struct btrfs_key *key, void *data, u32 data_size);
+int btrfs_insert_some_items(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct btrfs_key *cpu_key, u32 *data_size,
+ int nr);
+int btrfs_insert_empty_items(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct btrfs_key *cpu_key, u32 *data_size, int nr);
+
+static inline int btrfs_insert_empty_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct btrfs_key *key,
+ u32 data_size)
+{
+ return btrfs_insert_empty_items(trans, root, path, key, &data_size, 1);
+}
+
+int btrfs_next_leaf(struct btrfs_root *root, struct btrfs_path *path);
+int btrfs_prev_leaf(struct btrfs_root *root, struct btrfs_path *path);
+int btrfs_leaf_free_space(struct btrfs_root *root, struct extent_buffer *leaf);
+int btrfs_drop_snapshot(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root);
+int btrfs_drop_subtree(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct extent_buffer *node,
+ struct extent_buffer *parent);
+/* root-item.c */
+int btrfs_find_root_ref(struct btrfs_root *tree_root,
+ struct btrfs_path *path,
+ u64 root_id, u64 ref_id);
+int btrfs_add_root_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *tree_root,
+ u64 root_id, u8 type, u64 ref_id,
+ u64 dirid, u64 sequence,
+ const char *name, int name_len);
+int btrfs_del_root(struct btrfs_trans_handle *trans, struct btrfs_root *root,
+ struct btrfs_key *key);
+int btrfs_insert_root(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root, struct btrfs_key *key, struct btrfs_root_item
+ *item);
+int btrfs_update_root(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root, struct btrfs_key *key, struct btrfs_root_item
+ *item);
+int btrfs_find_last_root(struct btrfs_root *root, u64 objectid, struct
+ btrfs_root_item *item, struct btrfs_key *key);
+int btrfs_search_root(struct btrfs_root *root, u64 search_start,
+ u64 *found_objectid);
+int btrfs_find_dead_roots(struct btrfs_root *root, u64 objectid,
+ struct btrfs_root *latest_root);
+/* dir-item.c */
+int btrfs_insert_dir_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, const char *name,
+ int name_len, u64 dir,
+ struct btrfs_key *location, u8 type, u64 index);
+struct btrfs_dir_item *btrfs_lookup_dir_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, u64 dir,
+ const char *name, int name_len,
+ int mod);
+struct btrfs_dir_item *
+btrfs_lookup_dir_index_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, u64 dir,
+ u64 objectid, const char *name, int name_len,
+ int mod);
+struct btrfs_dir_item *btrfs_match_dir_item_name(struct btrfs_root *root,
+ struct btrfs_path *path,
+ const char *name, int name_len);
+int btrfs_delete_one_dir_name(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct btrfs_dir_item *di);
+int btrfs_insert_xattr_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, const char *name,
+ u16 name_len, const void *data, u16 data_len,
+ u64 dir);
+struct btrfs_dir_item *btrfs_lookup_xattr(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, u64 dir,
+ const char *name, u16 name_len,
+ int mod);
+
+/* orphan.c */
+int btrfs_insert_orphan_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 offset);
+int btrfs_del_orphan_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 offset);
+
+/* inode-map.c */
+int btrfs_find_free_objectid(struct btrfs_trans_handle *trans,
+ struct btrfs_root *fs_root,
+ u64 dirid, u64 *objectid);
+int btrfs_find_highest_inode(struct btrfs_root *fs_root, u64 *objectid);
+
+/* inode-item.c */
+int btrfs_insert_inode_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ const char *name, int name_len,
+ u64 inode_objectid, u64 ref_objectid, u64 index);
+int btrfs_del_inode_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ const char *name, int name_len,
+ u64 inode_objectid, u64 ref_objectid, u64 *index);
+int btrfs_insert_empty_inode(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, u64 objectid);
+int btrfs_lookup_inode(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root, struct btrfs_path *path,
+ struct btrfs_key *location, int mod);
+
+/* file-item.c */
+int btrfs_del_csums(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 bytenr, u64 len);
+int btrfs_lookup_bio_sums(struct btrfs_root *root, struct inode *inode,
+ struct bio *bio, u32 *dst);
+int btrfs_insert_file_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 objectid, u64 pos,
+ u64 disk_offset, u64 disk_num_bytes,
+ u64 num_bytes, u64 offset, u64 ram_bytes,
+ u8 compression, u8 encryption, u16 other_encoding);
+int btrfs_lookup_file_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, u64 objectid,
+ u64 bytenr, int mod);
+int btrfs_csum_file_blocks(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_ordered_sum *sums);
+int btrfs_csum_one_bio(struct btrfs_root *root, struct inode *inode,
+ struct bio *bio, u64 file_start, int contig);
+int btrfs_csum_file_bytes(struct btrfs_root *root, struct inode *inode,
+ u64 start, unsigned long len);
+struct btrfs_csum_item *btrfs_lookup_csum(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ u64 bytenr, int cow);
+int btrfs_csum_truncate(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct btrfs_path *path,
+ u64 isize);
+int btrfs_lookup_csums_range(struct btrfs_root *root, u64 start,
+ u64 end, struct list_head *list);
+/* inode.c */
+
+/* RHEL and EL kernels have a patch that renames PG_checked to FsMisc */
+#if defined(ClearPageFsMisc) && !defined(ClearPageChecked)
+#define ClearPageChecked ClearPageFsMisc
+#define SetPageChecked SetPageFsMisc
+#define PageChecked PageFsMisc
+#endif
+
+struct inode *btrfs_lookup_dentry(struct inode *dir, struct dentry *dentry);
+int btrfs_set_inode_index(struct inode *dir, u64 *index);
+int btrfs_unlink_inode(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct inode *dir, struct inode *inode,
+ const char *name, int name_len);
+int btrfs_add_link(struct btrfs_trans_handle *trans,
+ struct inode *parent_inode, struct inode *inode,
+ const char *name, int name_len, int add_backref, u64 index);
+int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct inode *inode, u64 new_size,
+ u32 min_type);
+
+int btrfs_start_delalloc_inodes(struct btrfs_root *root);
+int btrfs_set_extent_delalloc(struct inode *inode, u64 start, u64 end);
+int btrfs_writepages(struct address_space *mapping,
+ struct writeback_control *wbc);
+int btrfs_create_subvol_root(struct btrfs_trans_handle *trans,
+ struct btrfs_root *new_root, struct dentry *dentry,
+ u64 new_dirid, u64 alloc_hint);
+int btrfs_merge_bio_hook(struct page *page, unsigned long offset,
+ size_t size, struct bio *bio, unsigned long bio_flags);
+
+unsigned long btrfs_force_ra(struct address_space *mapping,
+ struct file_ra_state *ra, struct file *file,
+ pgoff_t offset, pgoff_t last_index);
+int btrfs_check_free_space(struct btrfs_root *root, u64 num_required,
+ int for_del);
+int btrfs_page_mkwrite(struct vm_area_struct *vma, struct page *page);
+int btrfs_readpage(struct file *file, struct page *page);
+void btrfs_delete_inode(struct inode *inode);
+void btrfs_put_inode(struct inode *inode);
+void btrfs_read_locked_inode(struct inode *inode);
+int btrfs_write_inode(struct inode *inode, int wait);
+void btrfs_dirty_inode(struct inode *inode);
+struct inode *btrfs_alloc_inode(struct super_block *sb);
+void btrfs_destroy_inode(struct inode *inode);
+int btrfs_init_cachep(void);
+void btrfs_destroy_cachep(void);
+long btrfs_ioctl_trans_end(struct file *file);
+struct inode *btrfs_ilookup(struct super_block *s, u64 objectid,
+ struct btrfs_root *root, int wait);
+struct inode *btrfs_iget_locked(struct super_block *s, u64 objectid,
+ struct btrfs_root *root);
+struct inode *btrfs_iget(struct super_block *s, struct btrfs_key *location,
+ struct btrfs_root *root, int *is_new);
+int btrfs_commit_write(struct file *file, struct page *page,
+ unsigned from, unsigned to);
+struct extent_map *btrfs_get_extent(struct inode *inode, struct page *page,
+ size_t page_offset, u64 start, u64 end,
+ int create);
+int btrfs_update_inode(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct inode *inode);
+int btrfs_orphan_add(struct btrfs_trans_handle *trans, struct inode *inode);
+int btrfs_orphan_del(struct btrfs_trans_handle *trans, struct inode *inode);
+void btrfs_orphan_cleanup(struct btrfs_root *root);
+int btrfs_cont_expand(struct inode *inode, loff_t size);
+
+/* ioctl.c */
+long btrfs_ioctl(struct file *file, unsigned int cmd, unsigned long arg);
+
+/* file.c */
+int btrfs_sync_file(struct file *file, struct dentry *dentry, int datasync);
+int btrfs_drop_extent_cache(struct inode *inode, u64 start, u64 end,
+ int skip_pinned);
+int btrfs_check_file(struct btrfs_root *root, struct inode *inode);
+extern struct file_operations btrfs_file_operations;
+int btrfs_drop_extents(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct inode *inode,
+ u64 start, u64 end, u64 inline_limit, u64 *hint_block);
+int btrfs_mark_extent_written(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct inode *inode, u64 start, u64 end);
+int btrfs_release_file(struct inode *inode, struct file *file);
+
+/* tree-defrag.c */
+int btrfs_defrag_leaves(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, int cache_only);
+
+/* sysfs.c */
+int btrfs_init_sysfs(void);
+void btrfs_exit_sysfs(void);
+int btrfs_sysfs_add_super(struct btrfs_fs_info *fs);
+int btrfs_sysfs_add_root(struct btrfs_root *root);
+void btrfs_sysfs_del_root(struct btrfs_root *root);
+void btrfs_sysfs_del_super(struct btrfs_fs_info *root);
+
+/* xattr.c */
+ssize_t btrfs_listxattr(struct dentry *dentry, char *buffer, size_t size);
+
+/* super.c */
+u64 btrfs_parse_size(char *str);
+int btrfs_parse_options(struct btrfs_root *root, char *options);
+int btrfs_sync_fs(struct super_block *sb, int wait);
+
+/* acl.c */
+int btrfs_check_acl(struct inode *inode, int mask);
+int btrfs_init_acl(struct inode *inode, struct inode *dir);
+int btrfs_acl_chmod(struct inode *inode);
+
+/* free-space-cache.c */
+int btrfs_add_free_space(struct btrfs_block_group_cache *block_group,
+ u64 bytenr, u64 size);
+int btrfs_add_free_space_lock(struct btrfs_block_group_cache *block_group,
+ u64 offset, u64 bytes);
+int btrfs_remove_free_space(struct btrfs_block_group_cache *block_group,
+ u64 bytenr, u64 size);
+int btrfs_remove_free_space_lock(struct btrfs_block_group_cache *block_group,
+ u64 offset, u64 bytes);
+void btrfs_remove_free_space_cache(struct btrfs_block_group_cache
+ *block_group);
+struct btrfs_free_space *btrfs_find_free_space(struct btrfs_block_group_cache
+ *block_group, u64 offset,
+ u64 bytes);
+void btrfs_dump_free_space(struct btrfs_block_group_cache *block_group,
+ u64 bytes);
+u64 btrfs_block_group_free_space(struct btrfs_block_group_cache *block_group);
+#endif
diff --git a/fs/btrfs/dir-item.c b/fs/btrfs/dir-item.c
new file mode 100644
index 0000000..926a0b2
--- /dev/null
+++ b/fs/btrfs/dir-item.c
@@ -0,0 +1,386 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include "ctree.h"
+#include "disk-io.h"
+#include "hash.h"
+#include "transaction.h"
+
+/*
+ * insert a name into a directory, doing overflow properly if there is a hash
+ * collision. data_size indicates how big the item inserted should be. On
+ * success a struct btrfs_dir_item pointer is returned, otherwise it is
+ * an ERR_PTR.
+ *
+ * The name is not copied into the dir item, you have to do that yourself.
+ */
+static struct btrfs_dir_item *insert_with_overflow(struct btrfs_trans_handle
+ *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct btrfs_key *cpu_key,
+ u32 data_size,
+ const char *name,
+ int name_len)
+{
+ int ret;
+ char *ptr;
+ struct btrfs_item *item;
+ struct extent_buffer *leaf;
+
+ ret = btrfs_insert_empty_item(trans, root, path, cpu_key, data_size);
+ if (ret == -EEXIST) {
+ struct btrfs_dir_item *di;
+ di = btrfs_match_dir_item_name(root, path, name, name_len);
+ if (di)
+ return ERR_PTR(-EEXIST);
+ ret = btrfs_extend_item(trans, root, path, data_size);
+ WARN_ON(ret > 0);
+ }
+ if (ret < 0)
+ return ERR_PTR(ret);
+ WARN_ON(ret > 0);
+ leaf = path->nodes[0];
+ item = btrfs_item_nr(leaf, path->slots[0]);
+ ptr = btrfs_item_ptr(leaf, path->slots[0], char);
+ BUG_ON(data_size > btrfs_item_size(leaf, item));
+ ptr += btrfs_item_size(leaf, item) - data_size;
+ return (struct btrfs_dir_item *)ptr;
+}
+
+/*
+ * xattrs work a lot like directories, this inserts an xattr item
+ * into the tree
+ */
+int btrfs_insert_xattr_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, const char *name,
+ u16 name_len, const void *data, u16 data_len,
+ u64 dir)
+{
+ int ret = 0;
+ struct btrfs_path *path;
+ struct btrfs_dir_item *dir_item;
+ unsigned long name_ptr, data_ptr;
+ struct btrfs_key key, location;
+ struct btrfs_disk_key disk_key;
+ struct extent_buffer *leaf;
+ u32 data_size;
+
+ key.objectid = dir;
+ btrfs_set_key_type(&key, BTRFS_XATTR_ITEM_KEY);
+ key.offset = btrfs_name_hash(name, name_len);
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+ if (name_len + data_len + sizeof(struct btrfs_dir_item) >
+ BTRFS_LEAF_DATA_SIZE(root) - sizeof(struct btrfs_item))
+ return -ENOSPC;
+
+ data_size = sizeof(*dir_item) + name_len + data_len;
+ dir_item = insert_with_overflow(trans, root, path, &key, data_size,
+ name, name_len);
+ /*
+ * FIXME: at some point we should handle xattr's that are larger than
+ * what we can fit in our leaf. We set location to NULL b/c we arent
+ * pointing at anything else, that will change if we store the xattr
+ * data in a separate inode.
+ */
+ BUG_ON(IS_ERR(dir_item));
+ memset(&location, 0, sizeof(location));
+
+ leaf = path->nodes[0];
+ btrfs_cpu_key_to_disk(&disk_key, &location);
+ btrfs_set_dir_item_key(leaf, dir_item, &disk_key);
+ btrfs_set_dir_type(leaf, dir_item, BTRFS_FT_XATTR);
+ btrfs_set_dir_name_len(leaf, dir_item, name_len);
+ btrfs_set_dir_transid(leaf, dir_item, trans->transid);
+ btrfs_set_dir_data_len(leaf, dir_item, data_len);
+ name_ptr = (unsigned long)(dir_item + 1);
+ data_ptr = (unsigned long)((char *)name_ptr + name_len);
+
+ write_extent_buffer(leaf, name, name_ptr, name_len);
+ write_extent_buffer(leaf, data, data_ptr, data_len);
+ btrfs_mark_buffer_dirty(path->nodes[0]);
+
+ btrfs_free_path(path);
+ return ret;
+}
+
+/*
+ * insert a directory item in the tree, doing all the magic for
+ * both indexes. 'dir' indicates which objectid to insert it into,
+ * 'location' is the key to stuff into the directory item, 'type' is the
+ * type of the inode we're pointing to, and 'index' is the sequence number
+ * to use for the second index (if one is created).
+ */
+int btrfs_insert_dir_item(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root, const char *name, int name_len, u64 dir,
+ struct btrfs_key *location, u8 type, u64 index)
+{
+ int ret = 0;
+ int ret2 = 0;
+ struct btrfs_path *path;
+ struct btrfs_dir_item *dir_item;
+ struct extent_buffer *leaf;
+ unsigned long name_ptr;
+ struct btrfs_key key;
+ struct btrfs_disk_key disk_key;
+ u32 data_size;
+
+ key.objectid = dir;
+ btrfs_set_key_type(&key, BTRFS_DIR_ITEM_KEY);
+ key.offset = btrfs_name_hash(name, name_len);
+ path = btrfs_alloc_path();
+ data_size = sizeof(*dir_item) + name_len;
+ dir_item = insert_with_overflow(trans, root, path, &key, data_size,
+ name, name_len);
+ if (IS_ERR(dir_item)) {
+ ret = PTR_ERR(dir_item);
+ if (ret == -EEXIST)
+ goto second_insert;
+ goto out;
+ }
+
+ leaf = path->nodes[0];
+ btrfs_cpu_key_to_disk(&disk_key, location);
+ btrfs_set_dir_item_key(leaf, dir_item, &disk_key);
+ btrfs_set_dir_type(leaf, dir_item, type);
+ btrfs_set_dir_data_len(leaf, dir_item, 0);
+ btrfs_set_dir_name_len(leaf, dir_item, name_len);
+ btrfs_set_dir_transid(leaf, dir_item, trans->transid);
+ name_ptr = (unsigned long)(dir_item + 1);
+
+ write_extent_buffer(leaf, name, name_ptr, name_len);
+ btrfs_mark_buffer_dirty(leaf);
+
+second_insert:
+ /* FIXME, use some real flag for selecting the extra index */
+ if (root == root->fs_info->tree_root) {
+ ret = 0;
+ goto out;
+ }
+ btrfs_release_path(root, path);
+
+ btrfs_set_key_type(&key, BTRFS_DIR_INDEX_KEY);
+ key.offset = index;
+ dir_item = insert_with_overflow(trans, root, path, &key, data_size,
+ name, name_len);
+ if (IS_ERR(dir_item)) {
+ ret2 = PTR_ERR(dir_item);
+ goto out;
+ }
+ leaf = path->nodes[0];
+ btrfs_cpu_key_to_disk(&disk_key, location);
+ btrfs_set_dir_item_key(leaf, dir_item, &disk_key);
+ btrfs_set_dir_type(leaf, dir_item, type);
+ btrfs_set_dir_data_len(leaf, dir_item, 0);
+ btrfs_set_dir_name_len(leaf, dir_item, name_len);
+ btrfs_set_dir_transid(leaf, dir_item, trans->transid);
+ name_ptr = (unsigned long)(dir_item + 1);
+ write_extent_buffer(leaf, name, name_ptr, name_len);
+ btrfs_mark_buffer_dirty(leaf);
+out:
+ btrfs_free_path(path);
+ if (ret)
+ return ret;
+ if (ret2)
+ return ret2;
+ return 0;
+}
+
+/*
+ * lookup a directory item based on name. 'dir' is the objectid
+ * we're searching in, and 'mod' tells us if you plan on deleting the
+ * item (use mod < 0) or changing the options (use mod > 0)
+ */
+struct btrfs_dir_item *btrfs_lookup_dir_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, u64 dir,
+ const char *name, int name_len,
+ int mod)
+{
+ int ret;
+ struct btrfs_key key;
+ int ins_len = mod < 0 ? -1 : 0;
+ int cow = mod != 0;
+ struct btrfs_key found_key;
+ struct extent_buffer *leaf;
+
+ key.objectid = dir;
+ btrfs_set_key_type(&key, BTRFS_DIR_ITEM_KEY);
+
+ key.offset = btrfs_name_hash(name, name_len);
+
+ ret = btrfs_search_slot(trans, root, &key, path, ins_len, cow);
+ if (ret < 0)
+ return ERR_PTR(ret);
+ if (ret > 0) {
+ if (path->slots[0] == 0)
+ return NULL;
+ path->slots[0]--;
+ }
+
+ leaf = path->nodes[0];
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+
+ if (found_key.objectid != dir ||
+ btrfs_key_type(&found_key) != BTRFS_DIR_ITEM_KEY ||
+ found_key.offset != key.offset)
+ return NULL;
+
+ return btrfs_match_dir_item_name(root, path, name, name_len);
+}
+
+/*
+ * lookup a directory item based on index. 'dir' is the objectid
+ * we're searching in, and 'mod' tells us if you plan on deleting the
+ * item (use mod < 0) or changing the options (use mod > 0)
+ *
+ * The name is used to make sure the index really points to the name you were
+ * looking for.
+ */
+struct btrfs_dir_item *
+btrfs_lookup_dir_index_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, u64 dir,
+ u64 objectid, const char *name, int name_len,
+ int mod)
+{
+ int ret;
+ struct btrfs_key key;
+ int ins_len = mod < 0 ? -1 : 0;
+ int cow = mod != 0;
+
+ key.objectid = dir;
+ btrfs_set_key_type(&key, BTRFS_DIR_INDEX_KEY);
+ key.offset = objectid;
+
+ ret = btrfs_search_slot(trans, root, &key, path, ins_len, cow);
+ if (ret < 0)
+ return ERR_PTR(ret);
+ if (ret > 0)
+ return ERR_PTR(-ENOENT);
+ return btrfs_match_dir_item_name(root, path, name, name_len);
+}
+
+struct btrfs_dir_item *btrfs_lookup_xattr(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, u64 dir,
+ const char *name, u16 name_len,
+ int mod)
+{
+ int ret;
+ struct btrfs_key key;
+ int ins_len = mod < 0 ? -1 : 0;
+ int cow = mod != 0;
+ struct btrfs_key found_key;
+ struct extent_buffer *leaf;
+
+ key.objectid = dir;
+ btrfs_set_key_type(&key, BTRFS_XATTR_ITEM_KEY);
+ key.offset = btrfs_name_hash(name, name_len);
+ ret = btrfs_search_slot(trans, root, &key, path, ins_len, cow);
+ if (ret < 0)
+ return ERR_PTR(ret);
+ if (ret > 0) {
+ if (path->slots[0] == 0)
+ return NULL;
+ path->slots[0]--;
+ }
+
+ leaf = path->nodes[0];
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+
+ if (found_key.objectid != dir ||
+ btrfs_key_type(&found_key) != BTRFS_XATTR_ITEM_KEY ||
+ found_key.offset != key.offset)
+ return NULL;
+
+ return btrfs_match_dir_item_name(root, path, name, name_len);
+}
+
+/*
+ * helper function to look at the directory item pointed to by 'path'
+ * this walks through all the entries in a dir item and finds one
+ * for a specific name.
+ */
+struct btrfs_dir_item *btrfs_match_dir_item_name(struct btrfs_root *root,
+ struct btrfs_path *path,
+ const char *name, int name_len)
+{
+ struct btrfs_dir_item *dir_item;
+ unsigned long name_ptr;
+ u32 total_len;
+ u32 cur = 0;
+ u32 this_len;
+ struct extent_buffer *leaf;
+
+ leaf = path->nodes[0];
+ dir_item = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_dir_item);
+ total_len = btrfs_item_size_nr(leaf, path->slots[0]);
+ while (cur < total_len) {
+ this_len = sizeof(*dir_item) +
+ btrfs_dir_name_len(leaf, dir_item) +
+ btrfs_dir_data_len(leaf, dir_item);
+ name_ptr = (unsigned long)(dir_item + 1);
+
+ if (btrfs_dir_name_len(leaf, dir_item) == name_len &&
+ memcmp_extent_buffer(leaf, name, name_ptr, name_len) == 0)
+ return dir_item;
+
+ cur += this_len;
+ dir_item = (struct btrfs_dir_item *)((char *)dir_item +
+ this_len);
+ }
+ return NULL;
+}
+
+/*
+ * given a pointer into a directory item, delete it. This
+ * handles items that have more than one entry in them.
+ */
+int btrfs_delete_one_dir_name(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct btrfs_dir_item *di)
+{
+
+ struct extent_buffer *leaf;
+ u32 sub_item_len;
+ u32 item_len;
+ int ret = 0;
+
+ leaf = path->nodes[0];
+ sub_item_len = sizeof(*di) + btrfs_dir_name_len(leaf, di) +
+ btrfs_dir_data_len(leaf, di);
+ item_len = btrfs_item_size_nr(leaf, path->slots[0]);
+ if (sub_item_len == item_len) {
+ ret = btrfs_del_item(trans, root, path);
+ } else {
+ /* MARKER */
+ unsigned long ptr = (unsigned long)di;
+ unsigned long start;
+
+ start = btrfs_item_ptr_offset(leaf, path->slots[0]);
+ memmove_extent_buffer(leaf, ptr, ptr + sub_item_len,
+ item_len - (ptr + sub_item_len - start));
+ ret = btrfs_truncate_item(trans, root, path,
+ item_len - sub_item_len, 1);
+ }
+ return 0;
+}
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
new file mode 100644
index 0000000..81a3138
--- /dev/null
+++ b/fs/btrfs/disk-io.c
@@ -0,0 +1,2343 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/version.h>
+#include <linux/fs.h>
+#include <linux/blkdev.h>
+#include <linux/scatterlist.h>
+#include <linux/swap.h>
+#include <linux/radix-tree.h>
+#include <linux/writeback.h>
+#include <linux/buffer_head.h>
+#include <linux/workqueue.h>
+#include <linux/kthread.h>
+#include <linux/freezer.h>
+#include "compat.h"
+#include "crc32c.h"
+#include "ctree.h"
+#include "disk-io.h"
+#include "transaction.h"
+#include "btrfs_inode.h"
+#include "volumes.h"
+#include "print-tree.h"
+#include "async-thread.h"
+#include "locking.h"
+#include "ref-cache.h"
+#include "tree-log.h"
+
+static struct extent_io_ops btree_extent_io_ops;
+static void end_workqueue_fn(struct btrfs_work *work);
+
+/*
+ * end_io_wq structs are used to do processing in task context when an IO is
+ * complete. This is used during reads to verify checksums, and it is used
+ * by writes to insert metadata for new file extents after IO is complete.
+ */
+struct end_io_wq {
+ struct bio *bio;
+ bio_end_io_t *end_io;
+ void *private;
+ struct btrfs_fs_info *info;
+ int error;
+ int metadata;
+ struct list_head list;
+ struct btrfs_work work;
+};
+
+/*
+ * async submit bios are used to offload expensive checksumming
+ * onto the worker threads. They checksum file and metadata bios
+ * just before they are sent down the IO stack.
+ */
+struct async_submit_bio {
+ struct inode *inode;
+ struct bio *bio;
+ struct list_head list;
+ extent_submit_bio_hook_t *submit_bio_start;
+ extent_submit_bio_hook_t *submit_bio_done;
+ int rw;
+ int mirror_num;
+ unsigned long bio_flags;
+ struct btrfs_work work;
+};
+
+/*
+ * extents on the btree inode are pretty simple, there's one extent
+ * that covers the entire device
+ */
+static struct extent_map *btree_get_extent(struct inode *inode,
+ struct page *page, size_t page_offset, u64 start, u64 len,
+ int create)
+{
+ struct extent_map_tree *em_tree = &BTRFS_I(inode)->extent_tree;
+ struct extent_map *em;
+ int ret;
+
+ spin_lock(&em_tree->lock);
+ em = lookup_extent_mapping(em_tree, start, len);
+ if (em) {
+ em->bdev =
+ BTRFS_I(inode)->root->fs_info->fs_devices->latest_bdev;
+ spin_unlock(&em_tree->lock);
+ goto out;
+ }
+ spin_unlock(&em_tree->lock);
+
+ em = alloc_extent_map(GFP_NOFS);
+ if (!em) {
+ em = ERR_PTR(-ENOMEM);
+ goto out;
+ }
+ em->start = 0;
+ em->len = (u64)-1;
+ em->block_len = (u64)-1;
+ em->block_start = 0;
+ em->bdev = BTRFS_I(inode)->root->fs_info->fs_devices->latest_bdev;
+
+ spin_lock(&em_tree->lock);
+ ret = add_extent_mapping(em_tree, em);
+ if (ret == -EEXIST) {
+ u64 failed_start = em->start;
+ u64 failed_len = em->len;
+
+ free_extent_map(em);
+ em = lookup_extent_mapping(em_tree, start, len);
+ if (em) {
+ ret = 0;
+ } else {
+ em = lookup_extent_mapping(em_tree, failed_start,
+ failed_len);
+ ret = -EIO;
+ }
+ } else if (ret) {
+ free_extent_map(em);
+ em = NULL;
+ }
+ spin_unlock(&em_tree->lock);
+
+ if (ret)
+ em = ERR_PTR(ret);
+out:
+ return em;
+}
+
+u32 btrfs_csum_data(struct btrfs_root *root, char *data, u32 seed, size_t len)
+{
+ return btrfs_crc32c(seed, data, len);
+}
+
+void btrfs_csum_final(u32 crc, char *result)
+{
+ *(__le32 *)result = ~cpu_to_le32(crc);
+}
+
+/*
+ * compute the csum for a btree block, and either verify it or write it
+ * into the csum field of the block.
+ */
+static int csum_tree_block(struct btrfs_root *root, struct extent_buffer *buf,
+ int verify)
+{
+ u16 csum_size =
+ btrfs_super_csum_size(&root->fs_info->super_copy);
+ char *result = NULL;
+ unsigned long len;
+ unsigned long cur_len;
+ unsigned long offset = BTRFS_CSUM_SIZE;
+ char *map_token = NULL;
+ char *kaddr;
+ unsigned long map_start;
+ unsigned long map_len;
+ int err;
+ u32 crc = ~(u32)0;
+ unsigned long inline_result;
+
+ len = buf->len - offset;
+ while (len > 0) {
+ err = map_private_extent_buffer(buf, offset, 32,
+ &map_token, &kaddr,
+ &map_start, &map_len, KM_USER0);
+ if (err)
+ return 1;
+ cur_len = min(len, map_len - (offset - map_start));
+ crc = btrfs_csum_data(root, kaddr + offset - map_start,
+ crc, cur_len);
+ len -= cur_len;
+ offset += cur_len;
+ unmap_extent_buffer(buf, map_token, KM_USER0);
+ }
+ if (csum_size > sizeof(inline_result)) {
+ result = kzalloc(csum_size * sizeof(char), GFP_NOFS);
+ if (!result)
+ return 1;
+ } else {
+ result = (char *)&inline_result;
+ }
+
+ btrfs_csum_final(crc, result);
+
+ if (verify) {
+ if (memcmp_extent_buffer(buf, result, 0, csum_size)) {
+ u32 val;
+ u32 found = 0;
+ memcpy(&found, result, csum_size);
+
+ read_extent_buffer(buf, &val, 0, csum_size);
+ printk(KERN_INFO "btrfs: %s checksum verify failed "
+ "on %llu wanted %X found %X level %d\n",
+ root->fs_info->sb->s_id,
+ buf->start, val, found, btrfs_header_level(buf));
+ if (result != (char *)&inline_result)
+ kfree(result);
+ return 1;
+ }
+ } else {
+ write_extent_buffer(buf, result, 0, csum_size);
+ }
+ if (result != (char *)&inline_result)
+ kfree(result);
+ return 0;
+}
+
+/*
+ * we can't consider a given block up to date unless the transid of the
+ * block matches the transid in the parent node's pointer. This is how we
+ * detect blocks that either didn't get written at all or got written
+ * in the wrong place.
+ */
+static int verify_parent_transid(struct extent_io_tree *io_tree,
+ struct extent_buffer *eb, u64 parent_transid)
+{
+ int ret;
+
+ if (!parent_transid || btrfs_header_generation(eb) == parent_transid)
+ return 0;
+
+ lock_extent(io_tree, eb->start, eb->start + eb->len - 1, GFP_NOFS);
+ if (extent_buffer_uptodate(io_tree, eb) &&
+ btrfs_header_generation(eb) == parent_transid) {
+ ret = 0;
+ goto out;
+ }
+ printk("parent transid verify failed on %llu wanted %llu found %llu\n",
+ (unsigned long long)eb->start,
+ (unsigned long long)parent_transid,
+ (unsigned long long)btrfs_header_generation(eb));
+ ret = 1;
+ clear_extent_buffer_uptodate(io_tree, eb);
+out:
+ unlock_extent(io_tree, eb->start, eb->start + eb->len - 1,
+ GFP_NOFS);
+ return ret;
+}
+
+/*
+ * helper to read a given tree block, doing retries as required when
+ * the checksums don't match and we have alternate mirrors to try.
+ */
+static int btree_read_extent_buffer_pages(struct btrfs_root *root,
+ struct extent_buffer *eb,
+ u64 start, u64 parent_transid)
+{
+ struct extent_io_tree *io_tree;
+ int ret;
+ int num_copies = 0;
+ int mirror_num = 0;
+
+ io_tree = &BTRFS_I(root->fs_info->btree_inode)->io_tree;
+ while (1) {
+ ret = read_extent_buffer_pages(io_tree, eb, start, 1,
+ btree_get_extent, mirror_num);
+ if (!ret &&
+ !verify_parent_transid(io_tree, eb, parent_transid))
+ return ret;
+
+ num_copies = btrfs_num_copies(&root->fs_info->mapping_tree,
+ eb->start, eb->len);
+ if (num_copies == 1)
+ return ret;
+
+ mirror_num++;
+ if (mirror_num > num_copies)
+ return ret;
+ }
+ return -EIO;
+}
+
+/*
+ * checksum a dirty tree block before IO. This has extra checks to make sure
+ * we only fill in the checksum field in the first page of a multi-page block
+ */
+
+static int csum_dirty_buffer(struct btrfs_root *root, struct page *page)
+{
+ struct extent_io_tree *tree;
+ u64 start = (u64)page->index << PAGE_CACHE_SHIFT;
+ u64 found_start;
+ int found_level;
+ unsigned long len;
+ struct extent_buffer *eb;
+ int ret;
+
+ tree = &BTRFS_I(page->mapping->host)->io_tree;
+
+ if (page->private == EXTENT_PAGE_PRIVATE)
+ goto out;
+ if (!page->private)
+ goto out;
+ len = page->private >> 2;
+ WARN_ON(len == 0);
+
+ eb = alloc_extent_buffer(tree, start, len, page, GFP_NOFS);
+ ret = btree_read_extent_buffer_pages(root, eb, start + PAGE_CACHE_SIZE,
+ btrfs_header_generation(eb));
+ BUG_ON(ret);
+ found_start = btrfs_header_bytenr(eb);
+ if (found_start != start) {
+ WARN_ON(1);
+ goto err;
+ }
+ if (eb->first_page != page) {
+ WARN_ON(1);
+ goto err;
+ }
+ if (!PageUptodate(page)) {
+ WARN_ON(1);
+ goto err;
+ }
+ found_level = btrfs_header_level(eb);
+
+ csum_tree_block(root, eb, 0);
+err:
+ free_extent_buffer(eb);
+out:
+ return 0;
+}
+
+static int check_tree_block_fsid(struct btrfs_root *root,
+ struct extent_buffer *eb)
+{
+ struct btrfs_fs_devices *fs_devices = root->fs_info->fs_devices;
+ u8 fsid[BTRFS_UUID_SIZE];
+ int ret = 1;
+
+ read_extent_buffer(eb, fsid, (unsigned long)btrfs_header_fsid(eb),
+ BTRFS_FSID_SIZE);
+ while (fs_devices) {
+ if (!memcmp(fsid, fs_devices->fsid, BTRFS_FSID_SIZE)) {
+ ret = 0;
+ break;
+ }
+ fs_devices = fs_devices->seed;
+ }
+ return ret;
+}
+
+static int btree_readpage_end_io_hook(struct page *page, u64 start, u64 end,
+ struct extent_state *state)
+{
+ struct extent_io_tree *tree;
+ u64 found_start;
+ int found_level;
+ unsigned long len;
+ struct extent_buffer *eb;
+ struct btrfs_root *root = BTRFS_I(page->mapping->host)->root;
+ int ret = 0;
+
+ tree = &BTRFS_I(page->mapping->host)->io_tree;
+ if (page->private == EXTENT_PAGE_PRIVATE)
+ goto out;
+ if (!page->private)
+ goto out;
+
+ len = page->private >> 2;
+ WARN_ON(len == 0);
+
+ eb = alloc_extent_buffer(tree, start, len, page, GFP_NOFS);
+
+ found_start = btrfs_header_bytenr(eb);
+ if (found_start != start) {
+ printk(KERN_INFO "btrfs bad tree block start %llu %llu\n",
+ (unsigned long long)found_start,
+ (unsigned long long)eb->start);
+ ret = -EIO;
+ goto err;
+ }
+ if (eb->first_page != page) {
+ printk(KERN_INFO "btrfs bad first page %lu %lu\n",
+ eb->first_page->index, page->index);
+ WARN_ON(1);
+ ret = -EIO;
+ goto err;
+ }
+ if (check_tree_block_fsid(root, eb)) {
+ printk(KERN_INFO "btrfs bad fsid on block %llu\n",
+ (unsigned long long)eb->start);
+ ret = -EIO;
+ goto err;
+ }
+ found_level = btrfs_header_level(eb);
+
+ ret = csum_tree_block(root, eb, 1);
+ if (ret)
+ ret = -EIO;
+
+ end = min_t(u64, eb->len, PAGE_CACHE_SIZE);
+ end = eb->start + end - 1;
+err:
+ free_extent_buffer(eb);
+out:
+ return ret;
+}
+
+static void end_workqueue_bio(struct bio *bio, int err)
+{
+ struct end_io_wq *end_io_wq = bio->bi_private;
+ struct btrfs_fs_info *fs_info;
+
+ fs_info = end_io_wq->info;
+ end_io_wq->error = err;
+ end_io_wq->work.func = end_workqueue_fn;
+ end_io_wq->work.flags = 0;
+
+ if (bio->bi_rw & (1 << BIO_RW)) {
+ if (end_io_wq->metadata)
+ btrfs_queue_worker(&fs_info->endio_meta_write_workers,
+ &end_io_wq->work);
+ else
+ btrfs_queue_worker(&fs_info->endio_write_workers,
+ &end_io_wq->work);
+ } else {
+ if (end_io_wq->metadata)
+ btrfs_queue_worker(&fs_info->endio_meta_workers,
+ &end_io_wq->work);
+ else
+ btrfs_queue_worker(&fs_info->endio_workers,
+ &end_io_wq->work);
+ }
+}
+
+int btrfs_bio_wq_end_io(struct btrfs_fs_info *info, struct bio *bio,
+ int metadata)
+{
+ struct end_io_wq *end_io_wq;
+ end_io_wq = kmalloc(sizeof(*end_io_wq), GFP_NOFS);
+ if (!end_io_wq)
+ return -ENOMEM;
+
+ end_io_wq->private = bio->bi_private;
+ end_io_wq->end_io = bio->bi_end_io;
+ end_io_wq->info = info;
+ end_io_wq->error = 0;
+ end_io_wq->bio = bio;
+ end_io_wq->metadata = metadata;
+
+ bio->bi_private = end_io_wq;
+ bio->bi_end_io = end_workqueue_bio;
+ return 0;
+}
+
+unsigned long btrfs_async_submit_limit(struct btrfs_fs_info *info)
+{
+ unsigned long limit = min_t(unsigned long,
+ info->workers.max_workers,
+ info->fs_devices->open_devices);
+ return 256 * limit;
+}
+
+int btrfs_congested_async(struct btrfs_fs_info *info, int iodone)
+{
+ return atomic_read(&info->nr_async_bios) >
+ btrfs_async_submit_limit(info);
+}
+
+static void run_one_async_start(struct btrfs_work *work)
+{
+ struct btrfs_fs_info *fs_info;
+ struct async_submit_bio *async;
+
+ async = container_of(work, struct async_submit_bio, work);
+ fs_info = BTRFS_I(async->inode)->root->fs_info;
+ async->submit_bio_start(async->inode, async->rw, async->bio,
+ async->mirror_num, async->bio_flags);
+}
+
+static void run_one_async_done(struct btrfs_work *work)
+{
+ struct btrfs_fs_info *fs_info;
+ struct async_submit_bio *async;
+ int limit;
+
+ async = container_of(work, struct async_submit_bio, work);
+ fs_info = BTRFS_I(async->inode)->root->fs_info;
+
+ limit = btrfs_async_submit_limit(fs_info);
+ limit = limit * 2 / 3;
+
+ atomic_dec(&fs_info->nr_async_submits);
+
+ if (atomic_read(&fs_info->nr_async_submits) < limit &&
+ waitqueue_active(&fs_info->async_submit_wait))
+ wake_up(&fs_info->async_submit_wait);
+
+ async->submit_bio_done(async->inode, async->rw, async->bio,
+ async->mirror_num, async->bio_flags);
+}
+
+static void run_one_async_free(struct btrfs_work *work)
+{
+ struct async_submit_bio *async;
+
+ async = container_of(work, struct async_submit_bio, work);
+ kfree(async);
+}
+
+int btrfs_wq_submit_bio(struct btrfs_fs_info *fs_info, struct inode *inode,
+ int rw, struct bio *bio, int mirror_num,
+ unsigned long bio_flags,
+ extent_submit_bio_hook_t *submit_bio_start,
+ extent_submit_bio_hook_t *submit_bio_done)
+{
+ struct async_submit_bio *async;
+
+ async = kmalloc(sizeof(*async), GFP_NOFS);
+ if (!async)
+ return -ENOMEM;
+
+ async->inode = inode;
+ async->rw = rw;
+ async->bio = bio;
+ async->mirror_num = mirror_num;
+ async->submit_bio_start = submit_bio_start;
+ async->submit_bio_done = submit_bio_done;
+
+ async->work.func = run_one_async_start;
+ async->work.ordered_func = run_one_async_done;
+ async->work.ordered_free = run_one_async_free;
+
+ async->work.flags = 0;
+ async->bio_flags = bio_flags;
+
+ atomic_inc(&fs_info->nr_async_submits);
+ btrfs_queue_worker(&fs_info->workers, &async->work);
+#if 0
+ int limit = btrfs_async_submit_limit(fs_info);
+ if (atomic_read(&fs_info->nr_async_submits) > limit) {
+ wait_event_timeout(fs_info->async_submit_wait,
+ (atomic_read(&fs_info->nr_async_submits) < limit),
+ HZ/10);
+
+ wait_event_timeout(fs_info->async_submit_wait,
+ (atomic_read(&fs_info->nr_async_bios) < limit),
+ HZ/10);
+ }
+#endif
+ while (atomic_read(&fs_info->async_submit_draining) &&
+ atomic_read(&fs_info->nr_async_submits)) {
+ wait_event(fs_info->async_submit_wait,
+ (atomic_read(&fs_info->nr_async_submits) == 0));
+ }
+
+ return 0;
+}
+
+static int btree_csum_one_bio(struct bio *bio)
+{
+ struct bio_vec *bvec = bio->bi_io_vec;
+ int bio_index = 0;
+ struct btrfs_root *root;
+
+ WARN_ON(bio->bi_vcnt <= 0);
+ while (bio_index < bio->bi_vcnt) {
+ root = BTRFS_I(bvec->bv_page->mapping->host)->root;
+ csum_dirty_buffer(root, bvec->bv_page);
+ bio_index++;
+ bvec++;
+ }
+ return 0;
+}
+
+static int __btree_submit_bio_start(struct inode *inode, int rw,
+ struct bio *bio, int mirror_num,
+ unsigned long bio_flags)
+{
+ /*
+ * when we're called for a write, we're already in the async
+ * submission context. Just jump into btrfs_map_bio
+ */
+ btree_csum_one_bio(bio);
+ return 0;
+}
+
+static int __btree_submit_bio_done(struct inode *inode, int rw, struct bio *bio,
+ int mirror_num, unsigned long bio_flags)
+{
+ /*
+ * when we're called for a write, we're already in the async
+ * submission context. Just jump into btrfs_map_bio
+ */
+ return btrfs_map_bio(BTRFS_I(inode)->root, rw, bio, mirror_num, 1);
+}
+
+static int btree_submit_bio_hook(struct inode *inode, int rw, struct bio *bio,
+ int mirror_num, unsigned long bio_flags)
+{
+ int ret;
+
+ ret = btrfs_bio_wq_end_io(BTRFS_I(inode)->root->fs_info,
+ bio, 1);
+ BUG_ON(ret);
+
+ if (!(rw & (1 << BIO_RW))) {
+ /*
+ * called for a read, do the setup so that checksum validation
+ * can happen in the async kernel threads
+ */
+ return btrfs_map_bio(BTRFS_I(inode)->root, rw, bio,
+ mirror_num, 0);
+ }
+ /*
+ * kthread helpers are used to submit writes so that checksumming
+ * can happen in parallel across all CPUs
+ */
+ return btrfs_wq_submit_bio(BTRFS_I(inode)->root->fs_info,
+ inode, rw, bio, mirror_num, 0,
+ __btree_submit_bio_start,
+ __btree_submit_bio_done);
+}
+
+static int btree_writepage(struct page *page, struct writeback_control *wbc)
+{
+ struct extent_io_tree *tree;
+ tree = &BTRFS_I(page->mapping->host)->io_tree;
+
+ if (current->flags & PF_MEMALLOC) {
+ redirty_page_for_writepage(wbc, page);
+ unlock_page(page);
+ return 0;
+ }
+ return extent_write_full_page(tree, page, btree_get_extent, wbc);
+}
+
+static int btree_writepages(struct address_space *mapping,
+ struct writeback_control *wbc)
+{
+ struct extent_io_tree *tree;
+ tree = &BTRFS_I(mapping->host)->io_tree;
+ if (wbc->sync_mode == WB_SYNC_NONE) {
+ u64 num_dirty;
+ u64 start = 0;
+ unsigned long thresh = 32 * 1024 * 1024;
+
+ if (wbc->for_kupdate)
+ return 0;
+
+ num_dirty = count_range_bits(tree, &start, (u64)-1,
+ thresh, EXTENT_DIRTY);
+ if (num_dirty < thresh)
+ return 0;
+ }
+ return extent_writepages(tree, mapping, btree_get_extent, wbc);
+}
+
+static int btree_readpage(struct file *file, struct page *page)
+{
+ struct extent_io_tree *tree;
+ tree = &BTRFS_I(page->mapping->host)->io_tree;
+ return extent_read_full_page(tree, page, btree_get_extent);
+}
+
+static int btree_releasepage(struct page *page, gfp_t gfp_flags)
+{
+ struct extent_io_tree *tree;
+ struct extent_map_tree *map;
+ int ret;
+
+ if (PageWriteback(page) || PageDirty(page))
+ return 0;
+
+ tree = &BTRFS_I(page->mapping->host)->io_tree;
+ map = &BTRFS_I(page->mapping->host)->extent_tree;
+
+ ret = try_release_extent_state(map, tree, page, gfp_flags);
+ if (!ret)
+ return 0;
+
+ ret = try_release_extent_buffer(tree, page);
+ if (ret == 1) {
+ ClearPagePrivate(page);
+ set_page_private(page, 0);
+ page_cache_release(page);
+ }
+
+ return ret;
+}
+
+static void btree_invalidatepage(struct page *page, unsigned long offset)
+{
+ struct extent_io_tree *tree;
+ tree = &BTRFS_I(page->mapping->host)->io_tree;
+ extent_invalidatepage(tree, page, offset);
+ btree_releasepage(page, GFP_NOFS);
+ if (PagePrivate(page)) {
+ printk(KERN_WARNING "btrfs warning page private not zero "
+ "on page %llu\n", (unsigned long long)page_offset(page));
+ ClearPagePrivate(page);
+ set_page_private(page, 0);
+ page_cache_release(page);
+ }
+}
+
+#if 0
+static int btree_writepage(struct page *page, struct writeback_control *wbc)
+{
+ struct buffer_head *bh;
+ struct btrfs_root *root = BTRFS_I(page->mapping->host)->root;
+ struct buffer_head *head;
+ if (!page_has_buffers(page)) {
+ create_empty_buffers(page, root->fs_info->sb->s_blocksize,
+ (1 << BH_Dirty)|(1 << BH_Uptodate));
+ }
+ head = page_buffers(page);
+ bh = head;
+ do {
+ if (buffer_dirty(bh))
+ csum_tree_block(root, bh, 0);
+ bh = bh->b_this_page;
+ } while (bh != head);
+ return block_write_full_page(page, btree_get_block, wbc);
+}
+#endif
+
+static struct address_space_operations btree_aops = {
+ .readpage = btree_readpage,
+ .writepage = btree_writepage,
+ .writepages = btree_writepages,
+ .releasepage = btree_releasepage,
+ .invalidatepage = btree_invalidatepage,
+ .sync_page = block_sync_page,
+};
+
+int readahead_tree_block(struct btrfs_root *root, u64 bytenr, u32 blocksize,
+ u64 parent_transid)
+{
+ struct extent_buffer *buf = NULL;
+ struct inode *btree_inode = root->fs_info->btree_inode;
+ int ret = 0;
+
+ buf = btrfs_find_create_tree_block(root, bytenr, blocksize);
+ if (!buf)
+ return 0;
+ read_extent_buffer_pages(&BTRFS_I(btree_inode)->io_tree,
+ buf, 0, 0, btree_get_extent, 0);
+ free_extent_buffer(buf);
+ return ret;
+}
+
+struct extent_buffer *btrfs_find_tree_block(struct btrfs_root *root,
+ u64 bytenr, u32 blocksize)
+{
+ struct inode *btree_inode = root->fs_info->btree_inode;
+ struct extent_buffer *eb;
+ eb = find_extent_buffer(&BTRFS_I(btree_inode)->io_tree,
+ bytenr, blocksize, GFP_NOFS);
+ return eb;
+}
+
+struct extent_buffer *btrfs_find_create_tree_block(struct btrfs_root *root,
+ u64 bytenr, u32 blocksize)
+{
+ struct inode *btree_inode = root->fs_info->btree_inode;
+ struct extent_buffer *eb;
+
+ eb = alloc_extent_buffer(&BTRFS_I(btree_inode)->io_tree,
+ bytenr, blocksize, NULL, GFP_NOFS);
+ return eb;
+}
+
+
+int btrfs_write_tree_block(struct extent_buffer *buf)
+{
+ return btrfs_fdatawrite_range(buf->first_page->mapping, buf->start,
+ buf->start + buf->len - 1, WB_SYNC_ALL);
+}
+
+int btrfs_wait_tree_block_writeback(struct extent_buffer *buf)
+{
+ return btrfs_wait_on_page_writeback_range(buf->first_page->mapping,
+ buf->start, buf->start + buf->len - 1);
+}
+
+struct extent_buffer *read_tree_block(struct btrfs_root *root, u64 bytenr,
+ u32 blocksize, u64 parent_transid)
+{
+ struct extent_buffer *buf = NULL;
+ struct inode *btree_inode = root->fs_info->btree_inode;
+ struct extent_io_tree *io_tree;
+ int ret;
+
+ io_tree = &BTRFS_I(btree_inode)->io_tree;
+
+ buf = btrfs_find_create_tree_block(root, bytenr, blocksize);
+ if (!buf)
+ return NULL;
+
+ ret = btree_read_extent_buffer_pages(root, buf, 0, parent_transid);
+
+ if (ret == 0)
+ buf->flags |= EXTENT_UPTODATE;
+ else
+ WARN_ON(1);
+ return buf;
+
+}
+
+int clean_tree_block(struct btrfs_trans_handle *trans, struct btrfs_root *root,
+ struct extent_buffer *buf)
+{
+ struct inode *btree_inode = root->fs_info->btree_inode;
+ if (btrfs_header_generation(buf) ==
+ root->fs_info->running_transaction->transid) {
+ WARN_ON(!btrfs_tree_locked(buf));
+ clear_extent_buffer_dirty(&BTRFS_I(btree_inode)->io_tree,
+ buf);
+ }
+ return 0;
+}
+
+static int __setup_root(u32 nodesize, u32 leafsize, u32 sectorsize,
+ u32 stripesize, struct btrfs_root *root,
+ struct btrfs_fs_info *fs_info,
+ u64 objectid)
+{
+ root->node = NULL;
+ root->commit_root = NULL;
+ root->ref_tree = NULL;
+ root->sectorsize = sectorsize;
+ root->nodesize = nodesize;
+ root->leafsize = leafsize;
+ root->stripesize = stripesize;
+ root->ref_cows = 0;
+ root->track_dirty = 0;
+
+ root->fs_info = fs_info;
+ root->objectid = objectid;
+ root->last_trans = 0;
+ root->highest_inode = 0;
+ root->last_inode_alloc = 0;
+ root->name = NULL;
+ root->in_sysfs = 0;
+
+ INIT_LIST_HEAD(&root->dirty_list);
+ INIT_LIST_HEAD(&root->orphan_list);
+ INIT_LIST_HEAD(&root->dead_list);
+ spin_lock_init(&root->node_lock);
+ spin_lock_init(&root->list_lock);
+ mutex_init(&root->objectid_mutex);
+ mutex_init(&root->log_mutex);
+ extent_io_tree_init(&root->dirty_log_pages,
+ fs_info->btree_inode->i_mapping, GFP_NOFS);
+
+ btrfs_leaf_ref_tree_init(&root->ref_tree_struct);
+ root->ref_tree = &root->ref_tree_struct;
+
+ memset(&root->root_key, 0, sizeof(root->root_key));
+ memset(&root->root_item, 0, sizeof(root->root_item));
+ memset(&root->defrag_progress, 0, sizeof(root->defrag_progress));
+ memset(&root->root_kobj, 0, sizeof(root->root_kobj));
+ root->defrag_trans_start = fs_info->generation;
+ init_completion(&root->kobj_unregister);
+ root->defrag_running = 0;
+ root->defrag_level = 0;
+ root->root_key.objectid = objectid;
+ root->anon_super.s_root = NULL;
+ root->anon_super.s_dev = 0;
+ INIT_LIST_HEAD(&root->anon_super.s_list);
+ INIT_LIST_HEAD(&root->anon_super.s_instances);
+ init_rwsem(&root->anon_super.s_umount);
+
+ return 0;
+}
+
+static int find_and_setup_root(struct btrfs_root *tree_root,
+ struct btrfs_fs_info *fs_info,
+ u64 objectid,
+ struct btrfs_root *root)
+{
+ int ret;
+ u32 blocksize;
+ u64 generation;
+
+ __setup_root(tree_root->nodesize, tree_root->leafsize,
+ tree_root->sectorsize, tree_root->stripesize,
+ root, fs_info, objectid);
+ ret = btrfs_find_last_root(tree_root, objectid,
+ &root->root_item, &root->root_key);
+ BUG_ON(ret);
+
+ generation = btrfs_root_generation(&root->root_item);
+ blocksize = btrfs_level_size(root, btrfs_root_level(&root->root_item));
+ root->node = read_tree_block(root, btrfs_root_bytenr(&root->root_item),
+ blocksize, generation);
+ BUG_ON(!root->node);
+ return 0;
+}
+
+int btrfs_free_log_root_tree(struct btrfs_trans_handle *trans,
+ struct btrfs_fs_info *fs_info)
+{
+ struct extent_buffer *eb;
+ struct btrfs_root *log_root_tree = fs_info->log_root_tree;
+ u64 start = 0;
+ u64 end = 0;
+ int ret;
+
+ if (!log_root_tree)
+ return 0;
+
+ while (1) {
+ ret = find_first_extent_bit(&log_root_tree->dirty_log_pages,
+ 0, &start, &end, EXTENT_DIRTY);
+ if (ret)
+ break;
+
+ clear_extent_dirty(&log_root_tree->dirty_log_pages,
+ start, end, GFP_NOFS);
+ }
+ eb = fs_info->log_root_tree->node;
+
+ WARN_ON(btrfs_header_level(eb) != 0);
+ WARN_ON(btrfs_header_nritems(eb) != 0);
+
+ ret = btrfs_free_reserved_extent(fs_info->tree_root,
+ eb->start, eb->len);
+ BUG_ON(ret);
+
+ free_extent_buffer(eb);
+ kfree(fs_info->log_root_tree);
+ fs_info->log_root_tree = NULL;
+ return 0;
+}
+
+int btrfs_init_log_root_tree(struct btrfs_trans_handle *trans,
+ struct btrfs_fs_info *fs_info)
+{
+ struct btrfs_root *root;
+ struct btrfs_root *tree_root = fs_info->tree_root;
+
+ root = kzalloc(sizeof(*root), GFP_NOFS);
+ if (!root)
+ return -ENOMEM;
+
+ __setup_root(tree_root->nodesize, tree_root->leafsize,
+ tree_root->sectorsize, tree_root->stripesize,
+ root, fs_info, BTRFS_TREE_LOG_OBJECTID);
+
+ root->root_key.objectid = BTRFS_TREE_LOG_OBJECTID;
+ root->root_key.type = BTRFS_ROOT_ITEM_KEY;
+ root->root_key.offset = BTRFS_TREE_LOG_OBJECTID;
+ root->ref_cows = 0;
+
+ root->node = btrfs_alloc_free_block(trans, root, root->leafsize,
+ 0, BTRFS_TREE_LOG_OBJECTID,
+ trans->transid, 0, 0, 0);
+
+ btrfs_set_header_nritems(root->node, 0);
+ btrfs_set_header_level(root->node, 0);
+ btrfs_set_header_bytenr(root->node, root->node->start);
+ btrfs_set_header_generation(root->node, trans->transid);
+ btrfs_set_header_owner(root->node, BTRFS_TREE_LOG_OBJECTID);
+
+ write_extent_buffer(root->node, root->fs_info->fsid,
+ (unsigned long)btrfs_header_fsid(root->node),
+ BTRFS_FSID_SIZE);
+ btrfs_mark_buffer_dirty(root->node);
+ btrfs_tree_unlock(root->node);
+ fs_info->log_root_tree = root;
+ return 0;
+}
+
+struct btrfs_root *btrfs_read_fs_root_no_radix(struct btrfs_root *tree_root,
+ struct btrfs_key *location)
+{
+ struct btrfs_root *root;
+ struct btrfs_fs_info *fs_info = tree_root->fs_info;
+ struct btrfs_path *path;
+ struct extent_buffer *l;
+ u64 highest_inode;
+ u64 generation;
+ u32 blocksize;
+ int ret = 0;
+
+ root = kzalloc(sizeof(*root), GFP_NOFS);
+ if (!root)
+ return ERR_PTR(-ENOMEM);
+ if (location->offset == (u64)-1) {
+ ret = find_and_setup_root(tree_root, fs_info,
+ location->objectid, root);
+ if (ret) {
+ kfree(root);
+ return ERR_PTR(ret);
+ }
+ goto insert;
+ }
+
+ __setup_root(tree_root->nodesize, tree_root->leafsize,
+ tree_root->sectorsize, tree_root->stripesize,
+ root, fs_info, location->objectid);
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+ ret = btrfs_search_slot(NULL, tree_root, location, path, 0, 0);
+ if (ret != 0) {
+ if (ret > 0)
+ ret = -ENOENT;
+ goto out;
+ }
+ l = path->nodes[0];
+ read_extent_buffer(l, &root->root_item,
+ btrfs_item_ptr_offset(l, path->slots[0]),
+ sizeof(root->root_item));
+ memcpy(&root->root_key, location, sizeof(*location));
+ ret = 0;
+out:
+ btrfs_release_path(root, path);
+ btrfs_free_path(path);
+ if (ret) {
+ kfree(root);
+ return ERR_PTR(ret);
+ }
+ generation = btrfs_root_generation(&root->root_item);
+ blocksize = btrfs_level_size(root, btrfs_root_level(&root->root_item));
+ root->node = read_tree_block(root, btrfs_root_bytenr(&root->root_item),
+ blocksize, generation);
+ BUG_ON(!root->node);
+insert:
+ if (location->objectid != BTRFS_TREE_LOG_OBJECTID) {
+ root->ref_cows = 1;
+ ret = btrfs_find_highest_inode(root, &highest_inode);
+ if (ret == 0) {
+ root->highest_inode = highest_inode;
+ root->last_inode_alloc = highest_inode;
+ }
+ }
+ return root;
+}
+
+struct btrfs_root *btrfs_lookup_fs_root(struct btrfs_fs_info *fs_info,
+ u64 root_objectid)
+{
+ struct btrfs_root *root;
+
+ if (root_objectid == BTRFS_ROOT_TREE_OBJECTID)
+ return fs_info->tree_root;
+ if (root_objectid == BTRFS_EXTENT_TREE_OBJECTID)
+ return fs_info->extent_root;
+
+ root = radix_tree_lookup(&fs_info->fs_roots_radix,
+ (unsigned long)root_objectid);
+ return root;
+}
+
+struct btrfs_root *btrfs_read_fs_root_no_name(struct btrfs_fs_info *fs_info,
+ struct btrfs_key *location)
+{
+ struct btrfs_root *root;
+ int ret;
+
+ if (location->objectid == BTRFS_ROOT_TREE_OBJECTID)
+ return fs_info->tree_root;
+ if (location->objectid == BTRFS_EXTENT_TREE_OBJECTID)
+ return fs_info->extent_root;
+ if (location->objectid == BTRFS_CHUNK_TREE_OBJECTID)
+ return fs_info->chunk_root;
+ if (location->objectid == BTRFS_DEV_TREE_OBJECTID)
+ return fs_info->dev_root;
+ if (location->objectid == BTRFS_CSUM_TREE_OBJECTID)
+ return fs_info->csum_root;
+
+ root = radix_tree_lookup(&fs_info->fs_roots_radix,
+ (unsigned long)location->objectid);
+ if (root)
+ return root;
+
+ root = btrfs_read_fs_root_no_radix(fs_info->tree_root, location);
+ if (IS_ERR(root))
+ return root;
+
+ set_anon_super(&root->anon_super, NULL);
+
+ ret = radix_tree_insert(&fs_info->fs_roots_radix,
+ (unsigned long)root->root_key.objectid,
+ root);
+ if (ret) {
+ free_extent_buffer(root->node);
+ kfree(root);
+ return ERR_PTR(ret);
+ }
+ if (!(fs_info->sb->s_flags & MS_RDONLY)) {
+ ret = btrfs_find_dead_roots(fs_info->tree_root,
+ root->root_key.objectid, root);
+ BUG_ON(ret);
+ btrfs_orphan_cleanup(root);
+ }
+ return root;
+}
+
+struct btrfs_root *btrfs_read_fs_root(struct btrfs_fs_info *fs_info,
+ struct btrfs_key *location,
+ const char *name, int namelen)
+{
+ struct btrfs_root *root;
+ int ret;
+
+ root = btrfs_read_fs_root_no_name(fs_info, location);
+ if (!root)
+ return NULL;
+
+ if (root->in_sysfs)
+ return root;
+
+ ret = btrfs_set_root_name(root, name, namelen);
+ if (ret) {
+ free_extent_buffer(root->node);
+ kfree(root);
+ return ERR_PTR(ret);
+ }
+#if 0
+ ret = btrfs_sysfs_add_root(root);
+ if (ret) {
+ free_extent_buffer(root->node);
+ kfree(root->name);
+ kfree(root);
+ return ERR_PTR(ret);
+ }
+#endif
+ root->in_sysfs = 1;
+ return root;
+}
+
+static int btrfs_congested_fn(void *congested_data, int bdi_bits)
+{
+ struct btrfs_fs_info *info = (struct btrfs_fs_info *)congested_data;
+ int ret = 0;
+ struct list_head *cur;
+ struct btrfs_device *device;
+ struct backing_dev_info *bdi;
+#if 0
+ if ((bdi_bits & (1 << BDI_write_congested)) &&
+ btrfs_congested_async(info, 0))
+ return 1;
+#endif
+ list_for_each(cur, &info->fs_devices->devices) {
+ device = list_entry(cur, struct btrfs_device, dev_list);
+ if (!device->bdev)
+ continue;
+ bdi = blk_get_backing_dev_info(device->bdev);
+ if (bdi && bdi_congested(bdi, bdi_bits)) {
+ ret = 1;
+ break;
+ }
+ }
+ return ret;
+}
+
+/*
+ * this unplugs every device on the box, and it is only used when page
+ * is null
+ */
+static void __unplug_io_fn(struct backing_dev_info *bdi, struct page *page)
+{
+ struct list_head *cur;
+ struct btrfs_device *device;
+ struct btrfs_fs_info *info;
+
+ info = (struct btrfs_fs_info *)bdi->unplug_io_data;
+ list_for_each(cur, &info->fs_devices->devices) {
+ device = list_entry(cur, struct btrfs_device, dev_list);
+ if (!device->bdev)
+ continue;
+
+ bdi = blk_get_backing_dev_info(device->bdev);
+ if (bdi->unplug_io_fn)
+ bdi->unplug_io_fn(bdi, page);
+ }
+}
+
+static void btrfs_unplug_io_fn(struct backing_dev_info *bdi, struct page *page)
+{
+ struct inode *inode;
+ struct extent_map_tree *em_tree;
+ struct extent_map *em;
+ struct address_space *mapping;
+ u64 offset;
+
+ /* the generic O_DIRECT read code does this */
+ if (1 || !page) {
+ __unplug_io_fn(bdi, page);
+ return;
+ }
+
+ /*
+ * page->mapping may change at any time. Get a consistent copy
+ * and use that for everything below
+ */
+ smp_mb();
+ mapping = page->mapping;
+ if (!mapping)
+ return;
+
+ inode = mapping->host;
+
+ /*
+ * don't do the expensive searching for a small number of
+ * devices
+ */
+ if (BTRFS_I(inode)->root->fs_info->fs_devices->open_devices <= 2) {
+ __unplug_io_fn(bdi, page);
+ return;
+ }
+
+ offset = page_offset(page);
+
+ em_tree = &BTRFS_I(inode)->extent_tree;
+ spin_lock(&em_tree->lock);
+ em = lookup_extent_mapping(em_tree, offset, PAGE_CACHE_SIZE);
+ spin_unlock(&em_tree->lock);
+ if (!em) {
+ __unplug_io_fn(bdi, page);
+ return;
+ }
+
+ if (em->block_start >= EXTENT_MAP_LAST_BYTE) {
+ free_extent_map(em);
+ __unplug_io_fn(bdi, page);
+ return;
+ }
+ offset = offset - em->start;
+ btrfs_unplug_page(&BTRFS_I(inode)->root->fs_info->mapping_tree,
+ em->block_start + offset, page);
+ free_extent_map(em);
+}
+
+static int setup_bdi(struct btrfs_fs_info *info, struct backing_dev_info *bdi)
+{
+ bdi_init(bdi);
+ bdi->ra_pages = default_backing_dev_info.ra_pages;
+ bdi->state = 0;
+ bdi->capabilities = default_backing_dev_info.capabilities;
+ bdi->unplug_io_fn = btrfs_unplug_io_fn;
+ bdi->unplug_io_data = info;
+ bdi->congested_fn = btrfs_congested_fn;
+ bdi->congested_data = info;
+ return 0;
+}
+
+static int bio_ready_for_csum(struct bio *bio)
+{
+ u64 length = 0;
+ u64 buf_len = 0;
+ u64 start = 0;
+ struct page *page;
+ struct extent_io_tree *io_tree = NULL;
+ struct btrfs_fs_info *info = NULL;
+ struct bio_vec *bvec;
+ int i;
+ int ret;
+
+ bio_for_each_segment(bvec, bio, i) {
+ page = bvec->bv_page;
+ if (page->private == EXTENT_PAGE_PRIVATE) {
+ length += bvec->bv_len;
+ continue;
+ }
+ if (!page->private) {
+ length += bvec->bv_len;
+ continue;
+ }
+ length = bvec->bv_len;
+ buf_len = page->private >> 2;
+ start = page_offset(page) + bvec->bv_offset;
+ io_tree = &BTRFS_I(page->mapping->host)->io_tree;
+ info = BTRFS_I(page->mapping->host)->root->fs_info;
+ }
+ /* are we fully contained in this bio? */
+ if (buf_len <= length)
+ return 1;
+
+ ret = extent_range_uptodate(io_tree, start + length,
+ start + buf_len - 1);
+ if (ret == 1)
+ return ret;
+ return ret;
+}
+
+/*
+ * called by the kthread helper functions to finally call the bio end_io
+ * functions. This is where read checksum verification actually happens
+ */
+static void end_workqueue_fn(struct btrfs_work *work)
+{
+ struct bio *bio;
+ struct end_io_wq *end_io_wq;
+ struct btrfs_fs_info *fs_info;
+ int error;
+
+ end_io_wq = container_of(work, struct end_io_wq, work);
+ bio = end_io_wq->bio;
+ fs_info = end_io_wq->info;
+
+ /* metadata bio reads are special because the whole tree block must
+ * be checksummed at once. This makes sure the entire block is in
+ * ram and up to date before trying to verify things. For
+ * blocksize <= pagesize, it is basically a noop
+ */
+ if (!(bio->bi_rw & (1 << BIO_RW)) && end_io_wq->metadata &&
+ !bio_ready_for_csum(bio)) {
+ btrfs_queue_worker(&fs_info->endio_meta_workers,
+ &end_io_wq->work);
+ return;
+ }
+ error = end_io_wq->error;
+ bio->bi_private = end_io_wq->private;
+ bio->bi_end_io = end_io_wq->end_io;
+ kfree(end_io_wq);
+ bio_endio(bio, error);
+}
+
+static int cleaner_kthread(void *arg)
+{
+ struct btrfs_root *root = arg;
+
+ do {
+ smp_mb();
+ if (root->fs_info->closing)
+ break;
+
+ vfs_check_frozen(root->fs_info->sb, SB_FREEZE_WRITE);
+ mutex_lock(&root->fs_info->cleaner_mutex);
+ btrfs_clean_old_snapshots(root);
+ mutex_unlock(&root->fs_info->cleaner_mutex);
+
+ if (freezing(current)) {
+ refrigerator();
+ } else {
+ smp_mb();
+ if (root->fs_info->closing)
+ break;
+ set_current_state(TASK_INTERRUPTIBLE);
+ schedule();
+ __set_current_state(TASK_RUNNING);
+ }
+ } while (!kthread_should_stop());
+ return 0;
+}
+
+static int transaction_kthread(void *arg)
+{
+ struct btrfs_root *root = arg;
+ struct btrfs_trans_handle *trans;
+ struct btrfs_transaction *cur;
+ unsigned long now;
+ unsigned long delay;
+ int ret;
+
+ do {
+ smp_mb();
+ if (root->fs_info->closing)
+ break;
+
+ delay = HZ * 30;
+ vfs_check_frozen(root->fs_info->sb, SB_FREEZE_WRITE);
+ mutex_lock(&root->fs_info->transaction_kthread_mutex);
+
+ if (root->fs_info->total_ref_cache_size > 20 * 1024 * 1024) {
+ printk(KERN_INFO "btrfs: total reference cache "
+ "size %llu\n",
+ root->fs_info->total_ref_cache_size);
+ }
+
+ mutex_lock(&root->fs_info->trans_mutex);
+ cur = root->fs_info->running_transaction;
+ if (!cur) {
+ mutex_unlock(&root->fs_info->trans_mutex);
+ goto sleep;
+ }
+
+ now = get_seconds();
+ if (now < cur->start_time || now - cur->start_time < 30) {
+ mutex_unlock(&root->fs_info->trans_mutex);
+ delay = HZ * 5;
+ goto sleep;
+ }
+ mutex_unlock(&root->fs_info->trans_mutex);
+ trans = btrfs_start_transaction(root, 1);
+ ret = btrfs_commit_transaction(trans, root);
+sleep:
+ wake_up_process(root->fs_info->cleaner_kthread);
+ mutex_unlock(&root->fs_info->transaction_kthread_mutex);
+
+ if (freezing(current)) {
+ refrigerator();
+ } else {
+ if (root->fs_info->closing)
+ break;
+ set_current_state(TASK_INTERRUPTIBLE);
+ schedule_timeout(delay);
+ __set_current_state(TASK_RUNNING);
+ }
+ } while (!kthread_should_stop());
+ return 0;
+}
+
+struct btrfs_root *open_ctree(struct super_block *sb,
+ struct btrfs_fs_devices *fs_devices,
+ char *options)
+{
+ u32 sectorsize;
+ u32 nodesize;
+ u32 leafsize;
+ u32 blocksize;
+ u32 stripesize;
+ u64 generation;
+ u64 features;
+ struct btrfs_key location;
+ struct buffer_head *bh;
+ struct btrfs_root *extent_root = kzalloc(sizeof(struct btrfs_root),
+ GFP_NOFS);
+ struct btrfs_root *csum_root = kzalloc(sizeof(struct btrfs_root),
+ GFP_NOFS);
+ struct btrfs_root *tree_root = kzalloc(sizeof(struct btrfs_root),
+ GFP_NOFS);
+ struct btrfs_fs_info *fs_info = kzalloc(sizeof(*fs_info),
+ GFP_NOFS);
+ struct btrfs_root *chunk_root = kzalloc(sizeof(struct btrfs_root),
+ GFP_NOFS);
+ struct btrfs_root *dev_root = kzalloc(sizeof(struct btrfs_root),
+ GFP_NOFS);
+ struct btrfs_root *log_tree_root;
+
+ int ret;
+ int err = -EINVAL;
+
+ struct btrfs_super_block *disk_super;
+
+ if (!extent_root || !tree_root || !fs_info ||
+ !chunk_root || !dev_root || !csum_root) {
+ err = -ENOMEM;
+ goto fail;
+ }
+ INIT_RADIX_TREE(&fs_info->fs_roots_radix, GFP_NOFS);
+ INIT_LIST_HEAD(&fs_info->trans_list);
+ INIT_LIST_HEAD(&fs_info->dead_roots);
+ INIT_LIST_HEAD(&fs_info->hashers);
+ INIT_LIST_HEAD(&fs_info->delalloc_inodes);
+ spin_lock_init(&fs_info->hash_lock);
+ spin_lock_init(&fs_info->delalloc_lock);
+ spin_lock_init(&fs_info->new_trans_lock);
+ spin_lock_init(&fs_info->ref_cache_lock);
+
+ init_completion(&fs_info->kobj_unregister);
+ fs_info->tree_root = tree_root;
+ fs_info->extent_root = extent_root;
+ fs_info->csum_root = csum_root;
+ fs_info->chunk_root = chunk_root;
+ fs_info->dev_root = dev_root;
+ fs_info->fs_devices = fs_devices;
+ INIT_LIST_HEAD(&fs_info->dirty_cowonly_roots);
+ INIT_LIST_HEAD(&fs_info->space_info);
+ btrfs_mapping_init(&fs_info->mapping_tree);
+ atomic_set(&fs_info->nr_async_submits, 0);
+ atomic_set(&fs_info->async_delalloc_pages, 0);
+ atomic_set(&fs_info->async_submit_draining, 0);
+ atomic_set(&fs_info->nr_async_bios, 0);
+ atomic_set(&fs_info->throttles, 0);
+ atomic_set(&fs_info->throttle_gen, 0);
+ fs_info->sb = sb;
+ fs_info->max_extent = (u64)-1;
+ fs_info->max_inline = 8192 * 1024;
+ setup_bdi(fs_info, &fs_info->bdi);
+ fs_info->btree_inode = new_inode(sb);
+ fs_info->btree_inode->i_ino = 1;
+ fs_info->btree_inode->i_nlink = 1;
+
+ fs_info->thread_pool_size = min_t(unsigned long,
+ num_online_cpus() + 2, 8);
+
+ INIT_LIST_HEAD(&fs_info->ordered_extents);
+ spin_lock_init(&fs_info->ordered_extent_lock);
+
+ sb->s_blocksize = 4096;
+ sb->s_blocksize_bits = blksize_bits(4096);
+
+ /*
+ * we set the i_size on the btree inode to the max possible int.
+ * the real end of the address space is determined by all of
+ * the devices in the system
+ */
+ fs_info->btree_inode->i_size = OFFSET_MAX;
+ fs_info->btree_inode->i_mapping->a_ops = &btree_aops;
+ fs_info->btree_inode->i_mapping->backing_dev_info = &fs_info->bdi;
+
+ extent_io_tree_init(&BTRFS_I(fs_info->btree_inode)->io_tree,
+ fs_info->btree_inode->i_mapping,
+ GFP_NOFS);
+ extent_map_tree_init(&BTRFS_I(fs_info->btree_inode)->extent_tree,
+ GFP_NOFS);
+
+ BTRFS_I(fs_info->btree_inode)->io_tree.ops = &btree_extent_io_ops;
+
+ spin_lock_init(&fs_info->block_group_cache_lock);
+ fs_info->block_group_cache_tree.rb_node = NULL;
+
+ extent_io_tree_init(&fs_info->pinned_extents,
+ fs_info->btree_inode->i_mapping, GFP_NOFS);
+ extent_io_tree_init(&fs_info->pending_del,
+ fs_info->btree_inode->i_mapping, GFP_NOFS);
+ extent_io_tree_init(&fs_info->extent_ins,
+ fs_info->btree_inode->i_mapping, GFP_NOFS);
+ fs_info->do_barriers = 1;
+
+ INIT_LIST_HEAD(&fs_info->dead_reloc_roots);
+ btrfs_leaf_ref_tree_init(&fs_info->reloc_ref_tree);
+ btrfs_leaf_ref_tree_init(&fs_info->shared_ref_tree);
+
+ BTRFS_I(fs_info->btree_inode)->root = tree_root;
+ memset(&BTRFS_I(fs_info->btree_inode)->location, 0,
+ sizeof(struct btrfs_key));
+ insert_inode_hash(fs_info->btree_inode);
+
+ mutex_init(&fs_info->trans_mutex);
+ mutex_init(&fs_info->tree_log_mutex);
+ mutex_init(&fs_info->drop_mutex);
+ mutex_init(&fs_info->extent_ins_mutex);
+ mutex_init(&fs_info->pinned_mutex);
+ mutex_init(&fs_info->chunk_mutex);
+ mutex_init(&fs_info->transaction_kthread_mutex);
+ mutex_init(&fs_info->cleaner_mutex);
+ mutex_init(&fs_info->volume_mutex);
+ mutex_init(&fs_info->tree_reloc_mutex);
+ init_waitqueue_head(&fs_info->transaction_throttle);
+ init_waitqueue_head(&fs_info->transaction_wait);
+ init_waitqueue_head(&fs_info->async_submit_wait);
+ init_waitqueue_head(&fs_info->tree_log_wait);
+ atomic_set(&fs_info->tree_log_commit, 0);
+ atomic_set(&fs_info->tree_log_writers, 0);
+ fs_info->tree_log_transid = 0;
+
+ __setup_root(4096, 4096, 4096, 4096, tree_root,
+ fs_info, BTRFS_ROOT_TREE_OBJECTID);
+
+
+ bh = btrfs_read_dev_super(fs_devices->latest_bdev);
+ if (!bh)
+ goto fail_iput;
+
+ memcpy(&fs_info->super_copy, bh->b_data, sizeof(fs_info->super_copy));
+ memcpy(&fs_info->super_for_commit, &fs_info->super_copy,
+ sizeof(fs_info->super_for_commit));
+ brelse(bh);
+
+ memcpy(fs_info->fsid, fs_info->super_copy.fsid, BTRFS_FSID_SIZE);
+
+ disk_super = &fs_info->super_copy;
+ if (!btrfs_super_root(disk_super))
+ goto fail_iput;
+
+ ret = btrfs_parse_options(tree_root, options);
+ if (ret) {
+ err = ret;
+ goto fail_iput;
+ }
+
+ features = btrfs_super_incompat_flags(disk_super) &
+ ~BTRFS_FEATURE_INCOMPAT_SUPP;
+ if (features) {
+ printk(KERN_ERR "BTRFS: couldn't mount because of "
+ "unsupported optional features (%Lx).\n",
+ features);
+ err = -EINVAL;
+ goto fail_iput;
+ }
+
+ features = btrfs_super_compat_ro_flags(disk_super) &
+ ~BTRFS_FEATURE_COMPAT_RO_SUPP;
+ if (!(sb->s_flags & MS_RDONLY) && features) {
+ printk(KERN_ERR "BTRFS: couldn't mount RDWR because of "
+ "unsupported option features (%Lx).\n",
+ features);
+ err = -EINVAL;
+ goto fail_iput;
+ }
+
+ /*
+ * we need to start all the end_io workers up front because the
+ * queue work function gets called at interrupt time, and so it
+ * cannot dynamically grow.
+ */
+ btrfs_init_workers(&fs_info->workers, "worker",
+ fs_info->thread_pool_size);
+
+ btrfs_init_workers(&fs_info->delalloc_workers, "delalloc",
+ fs_info->thread_pool_size);
+
+ btrfs_init_workers(&fs_info->submit_workers, "submit",
+ min_t(u64, fs_devices->num_devices,
+ fs_info->thread_pool_size));
+
+ /* a higher idle thresh on the submit workers makes it much more
+ * likely that bios will be send down in a sane order to the
+ * devices
+ */
+ fs_info->submit_workers.idle_thresh = 64;
+
+ fs_info->workers.idle_thresh = 16;
+ fs_info->workers.ordered = 1;
+
+ fs_info->delalloc_workers.idle_thresh = 2;
+ fs_info->delalloc_workers.ordered = 1;
+
+ btrfs_init_workers(&fs_info->fixup_workers, "fixup", 1);
+ btrfs_init_workers(&fs_info->endio_workers, "endio",
+ fs_info->thread_pool_size);
+ btrfs_init_workers(&fs_info->endio_meta_workers, "endio-meta",
+ fs_info->thread_pool_size);
+ btrfs_init_workers(&fs_info->endio_meta_write_workers,
+ "endio-meta-write", fs_info->thread_pool_size);
+ btrfs_init_workers(&fs_info->endio_write_workers, "endio-write",
+ fs_info->thread_pool_size);
+
+ /*
+ * endios are largely parallel and should have a very
+ * low idle thresh
+ */
+ fs_info->endio_workers.idle_thresh = 4;
+ fs_info->endio_write_workers.idle_thresh = 64;
+ fs_info->endio_meta_write_workers.idle_thresh = 64;
+
+ btrfs_start_workers(&fs_info->workers, 1);
+ btrfs_start_workers(&fs_info->submit_workers, 1);
+ btrfs_start_workers(&fs_info->delalloc_workers, 1);
+ btrfs_start_workers(&fs_info->fixup_workers, 1);
+ btrfs_start_workers(&fs_info->endio_workers, fs_info->thread_pool_size);
+ btrfs_start_workers(&fs_info->endio_meta_workers,
+ fs_info->thread_pool_size);
+ btrfs_start_workers(&fs_info->endio_meta_write_workers,
+ fs_info->thread_pool_size);
+ btrfs_start_workers(&fs_info->endio_write_workers,
+ fs_info->thread_pool_size);
+
+ fs_info->bdi.ra_pages *= btrfs_super_num_devices(disk_super);
+ fs_info->bdi.ra_pages = max(fs_info->bdi.ra_pages,
+ 4 * 1024 * 1024 / PAGE_CACHE_SIZE);
+
+ nodesize = btrfs_super_nodesize(disk_super);
+ leafsize = btrfs_super_leafsize(disk_super);
+ sectorsize = btrfs_super_sectorsize(disk_super);
+ stripesize = btrfs_super_stripesize(disk_super);
+ tree_root->nodesize = nodesize;
+ tree_root->leafsize = leafsize;
+ tree_root->sectorsize = sectorsize;
+ tree_root->stripesize = stripesize;
+
+ sb->s_blocksize = sectorsize;
+ sb->s_blocksize_bits = blksize_bits(sectorsize);
+
+ if (strncmp((char *)(&disk_super->magic), BTRFS_MAGIC,
+ sizeof(disk_super->magic))) {
+ printk(KERN_INFO "btrfs: valid FS not found on %s\n", sb->s_id);
+ goto fail_sb_buffer;
+ }
+
+ mutex_lock(&fs_info->chunk_mutex);
+ ret = btrfs_read_sys_array(tree_root);
+ mutex_unlock(&fs_info->chunk_mutex);
+ if (ret) {
+ printk(KERN_WARNING "btrfs: failed to read the system "
+ "array on %s\n", sb->s_id);
+ goto fail_sys_array;
+ }
+
+ blocksize = btrfs_level_size(tree_root,
+ btrfs_super_chunk_root_level(disk_super));
+ generation = btrfs_super_chunk_root_generation(disk_super);
+
+ __setup_root(nodesize, leafsize, sectorsize, stripesize,
+ chunk_root, fs_info, BTRFS_CHUNK_TREE_OBJECTID);
+
+ chunk_root->node = read_tree_block(chunk_root,
+ btrfs_super_chunk_root(disk_super),
+ blocksize, generation);
+ BUG_ON(!chunk_root->node);
+
+ read_extent_buffer(chunk_root->node, fs_info->chunk_tree_uuid,
+ (unsigned long)btrfs_header_chunk_tree_uuid(chunk_root->node),
+ BTRFS_UUID_SIZE);
+
+ mutex_lock(&fs_info->chunk_mutex);
+ ret = btrfs_read_chunk_tree(chunk_root);
+ mutex_unlock(&fs_info->chunk_mutex);
+ if (ret) {
+ printk(KERN_WARNING "btrfs: failed to read chunk tree on %s\n",
+ sb->s_id);
+ goto fail_chunk_root;
+ }
+
+ btrfs_close_extra_devices(fs_devices);
+
+ blocksize = btrfs_level_size(tree_root,
+ btrfs_super_root_level(disk_super));
+ generation = btrfs_super_generation(disk_super);
+
+ tree_root->node = read_tree_block(tree_root,
+ btrfs_super_root(disk_super),
+ blocksize, generation);
+ if (!tree_root->node)
+ goto fail_chunk_root;
+
+
+ ret = find_and_setup_root(tree_root, fs_info,
+ BTRFS_EXTENT_TREE_OBJECTID, extent_root);
+ if (ret)
+ goto fail_tree_root;
+ extent_root->track_dirty = 1;
+
+ ret = find_and_setup_root(tree_root, fs_info,
+ BTRFS_DEV_TREE_OBJECTID, dev_root);
+ dev_root->track_dirty = 1;
+
+ if (ret)
+ goto fail_extent_root;
+
+ ret = find_and_setup_root(tree_root, fs_info,
+ BTRFS_CSUM_TREE_OBJECTID, csum_root);
+ if (ret)
+ goto fail_extent_root;
+
+ csum_root->track_dirty = 1;
+
+ btrfs_read_block_groups(extent_root);
+
+ fs_info->generation = generation;
+ fs_info->last_trans_committed = generation;
+ fs_info->data_alloc_profile = (u64)-1;
+ fs_info->metadata_alloc_profile = (u64)-1;
+ fs_info->system_alloc_profile = fs_info->metadata_alloc_profile;
+ fs_info->cleaner_kthread = kthread_run(cleaner_kthread, tree_root,
+ "btrfs-cleaner");
+ if (!fs_info->cleaner_kthread)
+ goto fail_csum_root;
+
+ fs_info->transaction_kthread = kthread_run(transaction_kthread,
+ tree_root,
+ "btrfs-transaction");
+ if (!fs_info->transaction_kthread)
+ goto fail_cleaner;
+
+ if (btrfs_super_log_root(disk_super) != 0) {
+ u64 bytenr = btrfs_super_log_root(disk_super);
+
+ if (fs_devices->rw_devices == 0) {
+ printk(KERN_WARNING "Btrfs log replay required "
+ "on RO media\n");
+ err = -EIO;
+ goto fail_trans_kthread;
+ }
+ blocksize =
+ btrfs_level_size(tree_root,
+ btrfs_super_log_root_level(disk_super));
+
+ log_tree_root = kzalloc(sizeof(struct btrfs_root),
+ GFP_NOFS);
+
+ __setup_root(nodesize, leafsize, sectorsize, stripesize,
+ log_tree_root, fs_info, BTRFS_TREE_LOG_OBJECTID);
+
+ log_tree_root->node = read_tree_block(tree_root, bytenr,
+ blocksize,
+ generation + 1);
+ ret = btrfs_recover_log_trees(log_tree_root);
+ BUG_ON(ret);
+
+ if (sb->s_flags & MS_RDONLY) {
+ ret = btrfs_commit_super(tree_root);
+ BUG_ON(ret);
+ }
+ }
+
+ if (!(sb->s_flags & MS_RDONLY)) {
+ ret = btrfs_cleanup_reloc_trees(tree_root);
+ BUG_ON(ret);
+ }
+
+ location.objectid = BTRFS_FS_TREE_OBJECTID;
+ location.type = BTRFS_ROOT_ITEM_KEY;
+ location.offset = (u64)-1;
+
+ fs_info->fs_root = btrfs_read_fs_root_no_name(fs_info, &location);
+ if (!fs_info->fs_root)
+ goto fail_trans_kthread;
+ return tree_root;
+
+fail_trans_kthread:
+ kthread_stop(fs_info->transaction_kthread);
+fail_cleaner:
+ kthread_stop(fs_info->cleaner_kthread);
+
+ /*
+ * make sure we're done with the btree inode before we stop our
+ * kthreads
+ */
+ filemap_write_and_wait(fs_info->btree_inode->i_mapping);
+ invalidate_inode_pages2(fs_info->btree_inode->i_mapping);
+
+fail_csum_root:
+ free_extent_buffer(csum_root->node);
+fail_extent_root:
+ free_extent_buffer(extent_root->node);
+fail_tree_root:
+ free_extent_buffer(tree_root->node);
+fail_chunk_root:
+ free_extent_buffer(chunk_root->node);
+fail_sys_array:
+ free_extent_buffer(dev_root->node);
+fail_sb_buffer:
+ btrfs_stop_workers(&fs_info->fixup_workers);
+ btrfs_stop_workers(&fs_info->delalloc_workers);
+ btrfs_stop_workers(&fs_info->workers);
+ btrfs_stop_workers(&fs_info->endio_workers);
+ btrfs_stop_workers(&fs_info->endio_meta_workers);
+ btrfs_stop_workers(&fs_info->endio_meta_write_workers);
+ btrfs_stop_workers(&fs_info->endio_write_workers);
+ btrfs_stop_workers(&fs_info->submit_workers);
+fail_iput:
+ invalidate_inode_pages2(fs_info->btree_inode->i_mapping);
+ iput(fs_info->btree_inode);
+fail:
+ btrfs_close_devices(fs_info->fs_devices);
+ btrfs_mapping_tree_free(&fs_info->mapping_tree);
+
+ kfree(extent_root);
+ kfree(tree_root);
+ bdi_destroy(&fs_info->bdi);
+ kfree(fs_info);
+ kfree(chunk_root);
+ kfree(dev_root);
+ kfree(csum_root);
+ return ERR_PTR(err);
+}
+
+static void btrfs_end_buffer_write_sync(struct buffer_head *bh, int uptodate)
+{
+ char b[BDEVNAME_SIZE];
+
+ if (uptodate) {
+ set_buffer_uptodate(bh);
+ } else {
+ if (!buffer_eopnotsupp(bh) && printk_ratelimit()) {
+ printk(KERN_WARNING "lost page write due to "
+ "I/O error on %s\n",
+ bdevname(bh->b_bdev, b));
+ }
+ /* note, we dont' set_buffer_write_io_error because we have
+ * our own ways of dealing with the IO errors
+ */
+ clear_buffer_uptodate(bh);
+ }
+ unlock_buffer(bh);
+ put_bh(bh);
+}
+
+struct buffer_head *btrfs_read_dev_super(struct block_device *bdev)
+{
+ struct buffer_head *bh;
+ struct buffer_head *latest = NULL;
+ struct btrfs_super_block *super;
+ int i;
+ u64 transid = 0;
+ u64 bytenr;
+
+ /* we would like to check all the supers, but that would make
+ * a btrfs mount succeed after a mkfs from a different FS.
+ * So, we need to add a special mount option to scan for
+ * later supers, using BTRFS_SUPER_MIRROR_MAX instead
+ */
+ for (i = 0; i < 1; i++) {
+ bytenr = btrfs_sb_offset(i);
+ if (bytenr + 4096 >= i_size_read(bdev->bd_inode))
+ break;
+ bh = __bread(bdev, bytenr / 4096, 4096);
+ if (!bh)
+ continue;
+
+ super = (struct btrfs_super_block *)bh->b_data;
+ if (btrfs_super_bytenr(super) != bytenr ||
+ strncmp((char *)(&super->magic), BTRFS_MAGIC,
+ sizeof(super->magic))) {
+ brelse(bh);
+ continue;
+ }
+
+ if (!latest || btrfs_super_generation(super) > transid) {
+ brelse(latest);
+ latest = bh;
+ transid = btrfs_super_generation(super);
+ } else {
+ brelse(bh);
+ }
+ }
+ return latest;
+}
+
+static int write_dev_supers(struct btrfs_device *device,
+ struct btrfs_super_block *sb,
+ int do_barriers, int wait, int max_mirrors)
+{
+ struct buffer_head *bh;
+ int i;
+ int ret;
+ int errors = 0;
+ u32 crc;
+ u64 bytenr;
+ int last_barrier = 0;
+
+ if (max_mirrors == 0)
+ max_mirrors = BTRFS_SUPER_MIRROR_MAX;
+
+ /* make sure only the last submit_bh does a barrier */
+ if (do_barriers) {
+ for (i = 0; i < max_mirrors; i++) {
+ bytenr = btrfs_sb_offset(i);
+ if (bytenr + BTRFS_SUPER_INFO_SIZE >=
+ device->total_bytes)
+ break;
+ last_barrier = i;
+ }
+ }
+
+ for (i = 0; i < max_mirrors; i++) {
+ bytenr = btrfs_sb_offset(i);
+ if (bytenr + BTRFS_SUPER_INFO_SIZE >= device->total_bytes)
+ break;
+
+ if (wait) {
+ bh = __find_get_block(device->bdev, bytenr / 4096,
+ BTRFS_SUPER_INFO_SIZE);
+ BUG_ON(!bh);
+ brelse(bh);
+ wait_on_buffer(bh);
+ if (buffer_uptodate(bh)) {
+ brelse(bh);
+ continue;
+ }
+ } else {
+ btrfs_set_super_bytenr(sb, bytenr);
+
+ crc = ~(u32)0;
+ crc = btrfs_csum_data(NULL, (char *)sb +
+ BTRFS_CSUM_SIZE, crc,
+ BTRFS_SUPER_INFO_SIZE -
+ BTRFS_CSUM_SIZE);
+ btrfs_csum_final(crc, sb->csum);
+
+ bh = __getblk(device->bdev, bytenr / 4096,
+ BTRFS_SUPER_INFO_SIZE);
+ memcpy(bh->b_data, sb, BTRFS_SUPER_INFO_SIZE);
+
+ set_buffer_uptodate(bh);
+ get_bh(bh);
+ lock_buffer(bh);
+ bh->b_end_io = btrfs_end_buffer_write_sync;
+ }
+
+ if (i == last_barrier && do_barriers && device->barriers) {
+ ret = submit_bh(WRITE_BARRIER, bh);
+ if (ret == -EOPNOTSUPP) {
+ printk("btrfs: disabling barriers on dev %s\n",
+ device->name);
+ set_buffer_uptodate(bh);
+ device->barriers = 0;
+ get_bh(bh);
+ lock_buffer(bh);
+ ret = submit_bh(WRITE, bh);
+ }
+ } else {
+ ret = submit_bh(WRITE, bh);
+ }
+
+ if (!ret && wait) {
+ wait_on_buffer(bh);
+ if (!buffer_uptodate(bh))
+ errors++;
+ } else if (ret) {
+ errors++;
+ }
+ if (wait)
+ brelse(bh);
+ }
+ return errors < i ? 0 : -1;
+}
+
+int write_all_supers(struct btrfs_root *root, int max_mirrors)
+{
+ struct list_head *cur;
+ struct list_head *head = &root->fs_info->fs_devices->devices;
+ struct btrfs_device *dev;
+ struct btrfs_super_block *sb;
+ struct btrfs_dev_item *dev_item;
+ int ret;
+ int do_barriers;
+ int max_errors;
+ int total_errors = 0;
+ u64 flags;
+
+ max_errors = btrfs_super_num_devices(&root->fs_info->super_copy) - 1;
+ do_barriers = !btrfs_test_opt(root, NOBARRIER);
+
+ sb = &root->fs_info->super_for_commit;
+ dev_item = &sb->dev_item;
+ list_for_each(cur, head) {
+ dev = list_entry(cur, struct btrfs_device, dev_list);
+ if (!dev->bdev) {
+ total_errors++;
+ continue;
+ }
+ if (!dev->in_fs_metadata || !dev->writeable)
+ continue;
+
+ btrfs_set_stack_device_generation(dev_item, 0);
+ btrfs_set_stack_device_type(dev_item, dev->type);
+ btrfs_set_stack_device_id(dev_item, dev->devid);
+ btrfs_set_stack_device_total_bytes(dev_item, dev->total_bytes);
+ btrfs_set_stack_device_bytes_used(dev_item, dev->bytes_used);
+ btrfs_set_stack_device_io_align(dev_item, dev->io_align);
+ btrfs_set_stack_device_io_width(dev_item, dev->io_width);
+ btrfs_set_stack_device_sector_size(dev_item, dev->sector_size);
+ memcpy(dev_item->uuid, dev->uuid, BTRFS_UUID_SIZE);
+ memcpy(dev_item->fsid, dev->fs_devices->fsid, BTRFS_UUID_SIZE);
+
+ flags = btrfs_super_flags(sb);
+ btrfs_set_super_flags(sb, flags | BTRFS_HEADER_FLAG_WRITTEN);
+
+ ret = write_dev_supers(dev, sb, do_barriers, 0, max_mirrors);
+ if (ret)
+ total_errors++;
+ }
+ if (total_errors > max_errors) {
+ printk(KERN_ERR "btrfs: %d errors while writing supers\n",
+ total_errors);
+ BUG();
+ }
+
+ total_errors = 0;
+ list_for_each(cur, head) {
+ dev = list_entry(cur, struct btrfs_device, dev_list);
+ if (!dev->bdev)
+ continue;
+ if (!dev->in_fs_metadata || !dev->writeable)
+ continue;
+
+ ret = write_dev_supers(dev, sb, do_barriers, 1, max_mirrors);
+ if (ret)
+ total_errors++;
+ }
+ if (total_errors > max_errors) {
+ printk(KERN_ERR "btrfs: %d errors while writing supers\n",
+ total_errors);
+ BUG();
+ }
+ return 0;
+}
+
+int write_ctree_super(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, int max_mirrors)
+{
+ int ret;
+
+ ret = write_all_supers(root, max_mirrors);
+ return ret;
+}
+
+int btrfs_free_fs_root(struct btrfs_fs_info *fs_info, struct btrfs_root *root)
+{
+ radix_tree_delete(&fs_info->fs_roots_radix,
+ (unsigned long)root->root_key.objectid);
+ if (root->anon_super.s_dev) {
+ down_write(&root->anon_super.s_umount);
+ kill_anon_super(&root->anon_super);
+ }
+ if (root->node)
+ free_extent_buffer(root->node);
+ if (root->commit_root)
+ free_extent_buffer(root->commit_root);
+ kfree(root->name);
+ kfree(root);
+ return 0;
+}
+
+static int del_fs_roots(struct btrfs_fs_info *fs_info)
+{
+ int ret;
+ struct btrfs_root *gang[8];
+ int i;
+
+ while (1) {
+ ret = radix_tree_gang_lookup(&fs_info->fs_roots_radix,
+ (void **)gang, 0,
+ ARRAY_SIZE(gang));
+ if (!ret)
+ break;
+ for (i = 0; i < ret; i++)
+ btrfs_free_fs_root(fs_info, gang[i]);
+ }
+ return 0;
+}
+
+int btrfs_cleanup_fs_roots(struct btrfs_fs_info *fs_info)
+{
+ u64 root_objectid = 0;
+ struct btrfs_root *gang[8];
+ int i;
+ int ret;
+
+ while (1) {
+ ret = radix_tree_gang_lookup(&fs_info->fs_roots_radix,
+ (void **)gang, root_objectid,
+ ARRAY_SIZE(gang));
+ if (!ret)
+ break;
+ for (i = 0; i < ret; i++) {
+ root_objectid = gang[i]->root_key.objectid;
+ ret = btrfs_find_dead_roots(fs_info->tree_root,
+ root_objectid, gang[i]);
+ BUG_ON(ret);
+ btrfs_orphan_cleanup(gang[i]);
+ }
+ root_objectid++;
+ }
+ return 0;
+}
+
+int btrfs_commit_super(struct btrfs_root *root)
+{
+ struct btrfs_trans_handle *trans;
+ int ret;
+
+ mutex_lock(&root->fs_info->cleaner_mutex);
+ btrfs_clean_old_snapshots(root);
+ mutex_unlock(&root->fs_info->cleaner_mutex);
+ trans = btrfs_start_transaction(root, 1);
+ ret = btrfs_commit_transaction(trans, root);
+ BUG_ON(ret);
+ /* run commit again to drop the original snapshot */
+ trans = btrfs_start_transaction(root, 1);
+ btrfs_commit_transaction(trans, root);
+ ret = btrfs_write_and_wait_transaction(NULL, root);
+ BUG_ON(ret);
+
+ ret = write_ctree_super(NULL, root, 0);
+ return ret;
+}
+
+int close_ctree(struct btrfs_root *root)
+{
+ struct btrfs_fs_info *fs_info = root->fs_info;
+ int ret;
+
+ fs_info->closing = 1;
+ smp_mb();
+
+ kthread_stop(root->fs_info->transaction_kthread);
+ kthread_stop(root->fs_info->cleaner_kthread);
+
+ if (!(fs_info->sb->s_flags & MS_RDONLY)) {
+ ret = btrfs_commit_super(root);
+ if (ret)
+ printk(KERN_ERR "btrfs: commit super ret %d\n", ret);
+ }
+
+ if (fs_info->delalloc_bytes) {
+ printk(KERN_INFO "btrfs: at unmount delalloc count %llu\n",
+ fs_info->delalloc_bytes);
+ }
+ if (fs_info->total_ref_cache_size) {
+ printk(KERN_INFO "btrfs: at umount reference cache size %llu\n",
+ (unsigned long long)fs_info->total_ref_cache_size);
+ }
+
+ if (fs_info->extent_root->node)
+ free_extent_buffer(fs_info->extent_root->node);
+
+ if (fs_info->tree_root->node)
+ free_extent_buffer(fs_info->tree_root->node);
+
+ if (root->fs_info->chunk_root->node)
+ free_extent_buffer(root->fs_info->chunk_root->node);
+
+ if (root->fs_info->dev_root->node)
+ free_extent_buffer(root->fs_info->dev_root->node);
+
+ if (root->fs_info->csum_root->node)
+ free_extent_buffer(root->fs_info->csum_root->node);
+
+ btrfs_free_block_groups(root->fs_info);
+
+ del_fs_roots(fs_info);
+
+ iput(fs_info->btree_inode);
+
+ btrfs_stop_workers(&fs_info->fixup_workers);
+ btrfs_stop_workers(&fs_info->delalloc_workers);
+ btrfs_stop_workers(&fs_info->workers);
+ btrfs_stop_workers(&fs_info->endio_workers);
+ btrfs_stop_workers(&fs_info->endio_meta_workers);
+ btrfs_stop_workers(&fs_info->endio_meta_write_workers);
+ btrfs_stop_workers(&fs_info->endio_write_workers);
+ btrfs_stop_workers(&fs_info->submit_workers);
+
+#if 0
+ while (!list_empty(&fs_info->hashers)) {
+ struct btrfs_hasher *hasher;
+ hasher = list_entry(fs_info->hashers.next, struct btrfs_hasher,
+ hashers);
+ list_del(&hasher->hashers);
+ crypto_free_hash(&fs_info->hash_tfm);
+ kfree(hasher);
+ }
+#endif
+ btrfs_close_devices(fs_info->fs_devices);
+ btrfs_mapping_tree_free(&fs_info->mapping_tree);
+
+ bdi_destroy(&fs_info->bdi);
+
+ kfree(fs_info->extent_root);
+ kfree(fs_info->tree_root);
+ kfree(fs_info->chunk_root);
+ kfree(fs_info->dev_root);
+ kfree(fs_info->csum_root);
+ return 0;
+}
+
+int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid)
+{
+ int ret;
+ struct inode *btree_inode = buf->first_page->mapping->host;
+
+ ret = extent_buffer_uptodate(&BTRFS_I(btree_inode)->io_tree, buf);
+ if (!ret)
+ return ret;
+
+ ret = verify_parent_transid(&BTRFS_I(btree_inode)->io_tree, buf,
+ parent_transid);
+ return !ret;
+}
+
+int btrfs_set_buffer_uptodate(struct extent_buffer *buf)
+{
+ struct inode *btree_inode = buf->first_page->mapping->host;
+ return set_extent_buffer_uptodate(&BTRFS_I(btree_inode)->io_tree,
+ buf);
+}
+
+void btrfs_mark_buffer_dirty(struct extent_buffer *buf)
+{
+ struct btrfs_root *root = BTRFS_I(buf->first_page->mapping->host)->root;
+ u64 transid = btrfs_header_generation(buf);
+ struct inode *btree_inode = root->fs_info->btree_inode;
+
+ WARN_ON(!btrfs_tree_locked(buf));
+ if (transid != root->fs_info->generation) {
+ printk(KERN_CRIT "btrfs transid mismatch buffer %llu, "
+ "found %llu running %llu\n",
+ (unsigned long long)buf->start,
+ (unsigned long long)transid,
+ (unsigned long long)root->fs_info->generation);
+ WARN_ON(1);
+ }
+ set_extent_buffer_dirty(&BTRFS_I(btree_inode)->io_tree, buf);
+}
+
+void btrfs_btree_balance_dirty(struct btrfs_root *root, unsigned long nr)
+{
+ /*
+ * looks as though older kernels can get into trouble with
+ * this code, they end up stuck in balance_dirty_pages forever
+ */
+ struct extent_io_tree *tree;
+ u64 num_dirty;
+ u64 start = 0;
+ unsigned long thresh = 32 * 1024 * 1024;
+ tree = &BTRFS_I(root->fs_info->btree_inode)->io_tree;
+
+ if (current_is_pdflush() || current->flags & PF_MEMALLOC)
+ return;
+
+ num_dirty = count_range_bits(tree, &start, (u64)-1,
+ thresh, EXTENT_DIRTY);
+ if (num_dirty > thresh) {
+ balance_dirty_pages_ratelimited_nr(
+ root->fs_info->btree_inode->i_mapping, 1);
+ }
+ return;
+}
+
+int btrfs_read_buffer(struct extent_buffer *buf, u64 parent_transid)
+{
+ struct btrfs_root *root = BTRFS_I(buf->first_page->mapping->host)->root;
+ int ret;
+ ret = btree_read_extent_buffer_pages(root, buf, 0, parent_transid);
+ if (ret == 0)
+ buf->flags |= EXTENT_UPTODATE;
+ return ret;
+}
+
+int btree_lock_page_hook(struct page *page)
+{
+ struct inode *inode = page->mapping->host;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct extent_io_tree *io_tree = &BTRFS_I(inode)->io_tree;
+ struct extent_buffer *eb;
+ unsigned long len;
+ u64 bytenr = page_offset(page);
+
+ if (page->private == EXTENT_PAGE_PRIVATE)
+ goto out;
+
+ len = page->private >> 2;
+ eb = find_extent_buffer(io_tree, bytenr, len, GFP_NOFS);
+ if (!eb)
+ goto out;
+
+ btrfs_tree_lock(eb);
+ spin_lock(&root->fs_info->hash_lock);
+ btrfs_set_header_flag(eb, BTRFS_HEADER_FLAG_WRITTEN);
+ spin_unlock(&root->fs_info->hash_lock);
+ btrfs_tree_unlock(eb);
+ free_extent_buffer(eb);
+out:
+ lock_page(page);
+ return 0;
+}
+
+static struct extent_io_ops btree_extent_io_ops = {
+ .write_cache_pages_lock_hook = btree_lock_page_hook,
+ .readpage_end_io_hook = btree_readpage_end_io_hook,
+ .submit_bio_hook = btree_submit_bio_hook,
+ /* note we're sharing with inode.c for the merge bio hook */
+ .merge_bio_hook = btrfs_merge_bio_hook,
+};
diff --git a/fs/btrfs/disk-io.h b/fs/btrfs/disk-io.h
new file mode 100644
index 0000000..c0ff404
--- /dev/null
+++ b/fs/btrfs/disk-io.h
@@ -0,0 +1,102 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __DISKIO__
+#define __DISKIO__
+
+#define BTRFS_SUPER_INFO_OFFSET (64 * 1024)
+#define BTRFS_SUPER_INFO_SIZE 4096
+
+#define BTRFS_SUPER_MIRROR_MAX 3
+#define BTRFS_SUPER_MIRROR_SHIFT 12
+
+static inline u64 btrfs_sb_offset(int mirror)
+{
+ u64 start = 16 * 1024;
+ if (mirror)
+ return start << (BTRFS_SUPER_MIRROR_SHIFT * mirror);
+ return BTRFS_SUPER_INFO_OFFSET;
+}
+
+struct btrfs_device;
+struct btrfs_fs_devices;
+
+struct extent_buffer *read_tree_block(struct btrfs_root *root, u64 bytenr,
+ u32 blocksize, u64 parent_transid);
+int readahead_tree_block(struct btrfs_root *root, u64 bytenr, u32 blocksize,
+ u64 parent_transid);
+struct extent_buffer *btrfs_find_create_tree_block(struct btrfs_root *root,
+ u64 bytenr, u32 blocksize);
+int clean_tree_block(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct extent_buffer *buf);
+struct btrfs_root *open_ctree(struct super_block *sb,
+ struct btrfs_fs_devices *fs_devices,
+ char *options);
+int close_ctree(struct btrfs_root *root);
+int write_ctree_super(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, int max_mirrors);
+struct buffer_head *btrfs_read_dev_super(struct block_device *bdev);
+int btrfs_commit_super(struct btrfs_root *root);
+struct extent_buffer *btrfs_find_tree_block(struct btrfs_root *root,
+ u64 bytenr, u32 blocksize);
+struct btrfs_root *btrfs_lookup_fs_root(struct btrfs_fs_info *fs_info,
+ u64 root_objectid);
+struct btrfs_root *btrfs_read_fs_root(struct btrfs_fs_info *fs_info,
+ struct btrfs_key *location,
+ const char *name, int namelen);
+struct btrfs_root *btrfs_read_fs_root_no_radix(struct btrfs_root *tree_root,
+ struct btrfs_key *location);
+struct btrfs_root *btrfs_read_fs_root_no_name(struct btrfs_fs_info *fs_info,
+ struct btrfs_key *location);
+int btrfs_cleanup_fs_roots(struct btrfs_fs_info *fs_info);
+int btrfs_insert_dev_radix(struct btrfs_root *root,
+ struct block_device *bdev,
+ u64 device_id,
+ u64 block_start,
+ u64 num_blocks);
+void btrfs_btree_balance_dirty(struct btrfs_root *root, unsigned long nr);
+int btrfs_free_fs_root(struct btrfs_fs_info *fs_info, struct btrfs_root *root);
+void btrfs_mark_buffer_dirty(struct extent_buffer *buf);
+int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid);
+int btrfs_set_buffer_uptodate(struct extent_buffer *buf);
+int wait_on_tree_block_writeback(struct btrfs_root *root,
+ struct extent_buffer *buf);
+int btrfs_read_buffer(struct extent_buffer *buf, u64 parent_transid);
+u32 btrfs_csum_data(struct btrfs_root *root, char *data, u32 seed, size_t len);
+void btrfs_csum_final(u32 crc, char *result);
+int btrfs_open_device(struct btrfs_device *dev);
+int btrfs_verify_block_csum(struct btrfs_root *root,
+ struct extent_buffer *buf);
+int btrfs_bio_wq_end_io(struct btrfs_fs_info *info, struct bio *bio,
+ int metadata);
+int btrfs_wq_submit_bio(struct btrfs_fs_info *fs_info, struct inode *inode,
+ int rw, struct bio *bio, int mirror_num,
+ unsigned long bio_flags,
+ extent_submit_bio_hook_t *submit_bio_start,
+ extent_submit_bio_hook_t *submit_bio_done);
+
+int btrfs_congested_async(struct btrfs_fs_info *info, int iodone);
+unsigned long btrfs_async_submit_limit(struct btrfs_fs_info *info);
+int btrfs_write_tree_block(struct extent_buffer *buf);
+int btrfs_wait_tree_block_writeback(struct extent_buffer *buf);
+int btrfs_free_log_root_tree(struct btrfs_trans_handle *trans,
+ struct btrfs_fs_info *fs_info);
+int btrfs_init_log_root_tree(struct btrfs_trans_handle *trans,
+ struct btrfs_fs_info *fs_info);
+int btree_lock_page_hook(struct page *page);
+#endif
diff --git a/fs/btrfs/export.c b/fs/btrfs/export.c
new file mode 100644
index 0000000..85315d2
--- /dev/null
+++ b/fs/btrfs/export.c
@@ -0,0 +1,203 @@
+#include <linux/fs.h>
+#include <linux/types.h>
+#include "ctree.h"
+#include "disk-io.h"
+#include "btrfs_inode.h"
+#include "print-tree.h"
+#include "export.h"
+#include "compat.h"
+
+#define BTRFS_FID_SIZE_NON_CONNECTABLE (offsetof(struct btrfs_fid, \
+ parent_objectid) / 4)
+#define BTRFS_FID_SIZE_CONNECTABLE (offsetof(struct btrfs_fid, \
+ parent_root_objectid) / 4)
+#define BTRFS_FID_SIZE_CONNECTABLE_ROOT (sizeof(struct btrfs_fid) / 4)
+
+static int btrfs_encode_fh(struct dentry *dentry, u32 *fh, int *max_len,
+ int connectable)
+{
+ struct btrfs_fid *fid = (struct btrfs_fid *)fh;
+ struct inode *inode = dentry->d_inode;
+ int len = *max_len;
+ int type;
+
+ if ((len < BTRFS_FID_SIZE_NON_CONNECTABLE) ||
+ (connectable && len < BTRFS_FID_SIZE_CONNECTABLE))
+ return 255;
+
+ len = BTRFS_FID_SIZE_NON_CONNECTABLE;
+ type = FILEID_BTRFS_WITHOUT_PARENT;
+
+ fid->objectid = BTRFS_I(inode)->location.objectid;
+ fid->root_objectid = BTRFS_I(inode)->root->objectid;
+ fid->gen = inode->i_generation;
+
+ if (connectable && !S_ISDIR(inode->i_mode)) {
+ struct inode *parent;
+ u64 parent_root_id;
+
+ spin_lock(&dentry->d_lock);
+
+ parent = dentry->d_parent->d_inode;
+ fid->parent_objectid = BTRFS_I(parent)->location.objectid;
+ fid->parent_gen = parent->i_generation;
+ parent_root_id = BTRFS_I(parent)->root->objectid;
+
+ spin_unlock(&dentry->d_lock);
+
+ if (parent_root_id != fid->root_objectid) {
+ fid->parent_root_objectid = parent_root_id;
+ len = BTRFS_FID_SIZE_CONNECTABLE_ROOT;
+ type = FILEID_BTRFS_WITH_PARENT_ROOT;
+ } else {
+ len = BTRFS_FID_SIZE_CONNECTABLE;
+ type = FILEID_BTRFS_WITH_PARENT;
+ }
+ }
+
+ *max_len = len;
+ return type;
+}
+
+static struct dentry *btrfs_get_dentry(struct super_block *sb, u64 objectid,
+ u64 root_objectid, u32 generation)
+{
+ struct btrfs_root *root;
+ struct inode *inode;
+ struct btrfs_key key;
+
+ key.objectid = root_objectid;
+ btrfs_set_key_type(&key, BTRFS_ROOT_ITEM_KEY);
+ key.offset = (u64)-1;
+
+ root = btrfs_read_fs_root_no_name(btrfs_sb(sb)->fs_info, &key);
+ if (IS_ERR(root))
+ return ERR_CAST(root);
+
+ key.objectid = objectid;
+ btrfs_set_key_type(&key, BTRFS_INODE_ITEM_KEY);
+ key.offset = 0;
+
+ inode = btrfs_iget(sb, &key, root, NULL);
+ if (IS_ERR(inode))
+ return (void *)inode;
+
+ if (generation != inode->i_generation) {
+ iput(inode);
+ return ERR_PTR(-ESTALE);
+ }
+
+ return d_obtain_alias(inode);
+}
+
+static struct dentry *btrfs_fh_to_parent(struct super_block *sb, struct fid *fh,
+ int fh_len, int fh_type)
+{
+ struct btrfs_fid *fid = (struct btrfs_fid *) fh;
+ u64 objectid, root_objectid;
+ u32 generation;
+
+ if (fh_type == FILEID_BTRFS_WITH_PARENT) {
+ if (fh_len != BTRFS_FID_SIZE_CONNECTABLE)
+ return NULL;
+ root_objectid = fid->root_objectid;
+ } else if (fh_type == FILEID_BTRFS_WITH_PARENT_ROOT) {
+ if (fh_len != BTRFS_FID_SIZE_CONNECTABLE_ROOT)
+ return NULL;
+ root_objectid = fid->parent_root_objectid;
+ } else
+ return NULL;
+
+ objectid = fid->parent_objectid;
+ generation = fid->parent_gen;
+
+ return btrfs_get_dentry(sb, objectid, root_objectid, generation);
+}
+
+static struct dentry *btrfs_fh_to_dentry(struct super_block *sb, struct fid *fh,
+ int fh_len, int fh_type)
+{
+ struct btrfs_fid *fid = (struct btrfs_fid *) fh;
+ u64 objectid, root_objectid;
+ u32 generation;
+
+ if ((fh_type != FILEID_BTRFS_WITH_PARENT ||
+ fh_len != BTRFS_FID_SIZE_CONNECTABLE) &&
+ (fh_type != FILEID_BTRFS_WITH_PARENT_ROOT ||
+ fh_len != BTRFS_FID_SIZE_CONNECTABLE_ROOT) &&
+ (fh_type != FILEID_BTRFS_WITHOUT_PARENT ||
+ fh_len != BTRFS_FID_SIZE_NON_CONNECTABLE))
+ return NULL;
+
+ objectid = fid->objectid;
+ root_objectid = fid->root_objectid;
+ generation = fid->gen;
+
+ return btrfs_get_dentry(sb, objectid, root_objectid, generation);
+}
+
+static struct dentry *btrfs_get_parent(struct dentry *child)
+{
+ struct inode *dir = child->d_inode;
+ struct btrfs_root *root = BTRFS_I(dir)->root;
+ struct btrfs_key key;
+ struct btrfs_path *path;
+ struct extent_buffer *leaf;
+ int slot;
+ u64 objectid;
+ int ret;
+
+ path = btrfs_alloc_path();
+
+ key.objectid = dir->i_ino;
+ btrfs_set_key_type(&key, BTRFS_INODE_REF_KEY);
+ key.offset = (u64)-1;
+
+ ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ if (ret < 0) {
+ /* Error */
+ btrfs_free_path(path);
+ return ERR_PTR(ret);
+ }
+ leaf = path->nodes[0];
+ slot = path->slots[0];
+ if (ret) {
+ /* btrfs_search_slot() returns the slot where we'd want to
+ insert a backref for parent inode #0xFFFFFFFFFFFFFFFF.
+ The _real_ backref, telling us what the parent inode
+ _actually_ is, will be in the slot _before_ the one
+ that btrfs_search_slot() returns. */
+ if (!slot) {
+ /* Unless there is _no_ key in the tree before... */
+ btrfs_free_path(path);
+ return ERR_PTR(-EIO);
+ }
+ slot--;
+ }
+
+ btrfs_item_key_to_cpu(leaf, &key, slot);
+ btrfs_free_path(path);
+
+ if (key.objectid != dir->i_ino || key.type != BTRFS_INODE_REF_KEY)
+ return ERR_PTR(-EINVAL);
+
+ objectid = key.offset;
+
+ /* If we are already at the root of a subvol, return the real root */
+ if (objectid == dir->i_ino)
+ return dget(dir->i_sb->s_root);
+
+ /* Build a new key for the inode item */
+ key.objectid = objectid;
+ btrfs_set_key_type(&key, BTRFS_INODE_ITEM_KEY);
+ key.offset = 0;
+
+ return d_obtain_alias(btrfs_iget(root->fs_info->sb, &key, root, NULL));
+}
+
+const struct export_operations btrfs_export_ops = {
+ .encode_fh = btrfs_encode_fh,
+ .fh_to_dentry = btrfs_fh_to_dentry,
+ .fh_to_parent = btrfs_fh_to_parent,
+ .get_parent = btrfs_get_parent,
+};
diff --git a/fs/btrfs/export.h b/fs/btrfs/export.h
new file mode 100644
index 0000000..074348a
--- /dev/null
+++ b/fs/btrfs/export.h
@@ -0,0 +1,19 @@
+#ifndef BTRFS_EXPORT_H
+#define BTRFS_EXPORT_H
+
+#include <linux/exportfs.h>
+
+extern const struct export_operations btrfs_export_ops;
+
+struct btrfs_fid {
+ u64 objectid;
+ u64 root_objectid;
+ u32 gen;
+
+ u64 parent_objectid;
+ u32 parent_gen;
+
+ u64 parent_root_objectid;
+} __attribute__ ((packed));
+
+#endif
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
new file mode 100644
index 0000000..293da65
--- /dev/null
+++ b/fs/btrfs/extent-tree.c
@@ -0,0 +1,5986 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+#include <linux/sched.h>
+#include <linux/pagemap.h>
+#include <linux/writeback.h>
+#include <linux/blkdev.h>
+#include <linux/version.h>
+#include "compat.h"
+#include "hash.h"
+#include "crc32c.h"
+#include "ctree.h"
+#include "disk-io.h"
+#include "print-tree.h"
+#include "transaction.h"
+#include "volumes.h"
+#include "locking.h"
+#include "ref-cache.h"
+#include "compat.h"
+
+#define PENDING_EXTENT_INSERT 0
+#define PENDING_EXTENT_DELETE 1
+#define PENDING_BACKREF_UPDATE 2
+
+struct pending_extent_op {
+ int type;
+ u64 bytenr;
+ u64 num_bytes;
+ u64 parent;
+ u64 orig_parent;
+ u64 generation;
+ u64 orig_generation;
+ int level;
+ struct list_head list;
+ int del;
+};
+
+static int finish_current_insert(struct btrfs_trans_handle *trans,
+ struct btrfs_root *extent_root, int all);
+static int del_pending_extents(struct btrfs_trans_handle *trans,
+ struct btrfs_root *extent_root, int all);
+static int pin_down_bytes(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 bytenr, u64 num_bytes, int is_data);
+static int update_block_group(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 bytenr, u64 num_bytes, int alloc,
+ int mark_free);
+
+static int block_group_bits(struct btrfs_block_group_cache *cache, u64 bits)
+{
+ return (cache->flags & bits) == bits;
+}
+
+/*
+ * this adds the block group to the fs_info rb tree for the block group
+ * cache
+ */
+static int btrfs_add_block_group_cache(struct btrfs_fs_info *info,
+ struct btrfs_block_group_cache *block_group)
+{
+ struct rb_node **p;
+ struct rb_node *parent = NULL;
+ struct btrfs_block_group_cache *cache;
+
+ spin_lock(&info->block_group_cache_lock);
+ p = &info->block_group_cache_tree.rb_node;
+
+ while (*p) {
+ parent = *p;
+ cache = rb_entry(parent, struct btrfs_block_group_cache,
+ cache_node);
+ if (block_group->key.objectid < cache->key.objectid) {
+ p = &(*p)->rb_left;
+ } else if (block_group->key.objectid > cache->key.objectid) {
+ p = &(*p)->rb_right;
+ } else {
+ spin_unlock(&info->block_group_cache_lock);
+ return -EEXIST;
+ }
+ }
+
+ rb_link_node(&block_group->cache_node, parent, p);
+ rb_insert_color(&block_group->cache_node,
+ &info->block_group_cache_tree);
+ spin_unlock(&info->block_group_cache_lock);
+
+ return 0;
+}
+
+/*
+ * This will return the block group at or after bytenr if contains is 0, else
+ * it will return the block group that contains the bytenr
+ */
+static struct btrfs_block_group_cache *
+block_group_cache_tree_search(struct btrfs_fs_info *info, u64 bytenr,
+ int contains)
+{
+ struct btrfs_block_group_cache *cache, *ret = NULL;
+ struct rb_node *n;
+ u64 end, start;
+
+ spin_lock(&info->block_group_cache_lock);
+ n = info->block_group_cache_tree.rb_node;
+
+ while (n) {
+ cache = rb_entry(n, struct btrfs_block_group_cache,
+ cache_node);
+ end = cache->key.objectid + cache->key.offset - 1;
+ start = cache->key.objectid;
+
+ if (bytenr < start) {
+ if (!contains && (!ret || start < ret->key.objectid))
+ ret = cache;
+ n = n->rb_left;
+ } else if (bytenr > start) {
+ if (contains && bytenr <= end) {
+ ret = cache;
+ break;
+ }
+ n = n->rb_right;
+ } else {
+ ret = cache;
+ break;
+ }
+ }
+ if (ret)
+ atomic_inc(&ret->count);
+ spin_unlock(&info->block_group_cache_lock);
+
+ return ret;
+}
+
+/*
+ * this is only called by cache_block_group, since we could have freed extents
+ * we need to check the pinned_extents for any extents that can't be used yet
+ * since their free space will be released as soon as the transaction commits.
+ */
+static int add_new_free_space(struct btrfs_block_group_cache *block_group,
+ struct btrfs_fs_info *info, u64 start, u64 end)
+{
+ u64 extent_start, extent_end, size;
+ int ret;
+
+ mutex_lock(&info->pinned_mutex);
+ while (start < end) {
+ ret = find_first_extent_bit(&info->pinned_extents, start,
+ &extent_start, &extent_end,
+ EXTENT_DIRTY);
+ if (ret)
+ break;
+
+ if (extent_start == start) {
+ start = extent_end + 1;
+ } else if (extent_start > start && extent_start < end) {
+ size = extent_start - start;
+ ret = btrfs_add_free_space(block_group, start,
+ size);
+ BUG_ON(ret);
+ start = extent_end + 1;
+ } else {
+ break;
+ }
+ }
+
+ if (start < end) {
+ size = end - start;
+ ret = btrfs_add_free_space(block_group, start, size);
+ BUG_ON(ret);
+ }
+ mutex_unlock(&info->pinned_mutex);
+
+ return 0;
+}
+
+static int remove_sb_from_cache(struct btrfs_root *root,
+ struct btrfs_block_group_cache *cache)
+{
+ u64 bytenr;
+ u64 *logical;
+ int stripe_len;
+ int i, nr, ret;
+
+ for (i = 0; i < BTRFS_SUPER_MIRROR_MAX; i++) {
+ bytenr = btrfs_sb_offset(i);
+ ret = btrfs_rmap_block(&root->fs_info->mapping_tree,
+ cache->key.objectid, bytenr, 0,
+ &logical, &nr, &stripe_len);
+ BUG_ON(ret);
+ while (nr--) {
+ btrfs_remove_free_space(cache, logical[nr],
+ stripe_len);
+ }
+ kfree(logical);
+ }
+ return 0;
+}
+
+static int cache_block_group(struct btrfs_root *root,
+ struct btrfs_block_group_cache *block_group)
+{
+ struct btrfs_path *path;
+ int ret = 0;
+ struct btrfs_key key;
+ struct extent_buffer *leaf;
+ int slot;
+ u64 last;
+
+ if (!block_group)
+ return 0;
+
+ root = root->fs_info->extent_root;
+
+ if (block_group->cached)
+ return 0;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ path->reada = 2;
+ /*
+ * we get into deadlocks with paths held by callers of this function.
+ * since the alloc_mutex is protecting things right now, just
+ * skip the locking here
+ */
+ path->skip_locking = 1;
+ last = max_t(u64, block_group->key.objectid, BTRFS_SUPER_INFO_OFFSET);
+ key.objectid = last;
+ key.offset = 0;
+ btrfs_set_key_type(&key, BTRFS_EXTENT_ITEM_KEY);
+ ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ if (ret < 0)
+ goto err;
+
+ while (1) {
+ leaf = path->nodes[0];
+ slot = path->slots[0];
+ if (slot >= btrfs_header_nritems(leaf)) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret < 0)
+ goto err;
+ if (ret == 0)
+ continue;
+ else
+ break;
+ }
+ btrfs_item_key_to_cpu(leaf, &key, slot);
+ if (key.objectid < block_group->key.objectid)
+ goto next;
+
+ if (key.objectid >= block_group->key.objectid +
+ block_group->key.offset)
+ break;
+
+ if (btrfs_key_type(&key) == BTRFS_EXTENT_ITEM_KEY) {
+ add_new_free_space(block_group, root->fs_info, last,
+ key.objectid);
+
+ last = key.objectid + key.offset;
+ }
+next:
+ path->slots[0]++;
+ }
+
+ add_new_free_space(block_group, root->fs_info, last,
+ block_group->key.objectid +
+ block_group->key.offset);
+
+ remove_sb_from_cache(root, block_group);
+ block_group->cached = 1;
+ ret = 0;
+err:
+ btrfs_free_path(path);
+ return ret;
+}
+
+/*
+ * return the block group that starts at or after bytenr
+ */
+static struct btrfs_block_group_cache *
+btrfs_lookup_first_block_group(struct btrfs_fs_info *info, u64 bytenr)
+{
+ struct btrfs_block_group_cache *cache;
+
+ cache = block_group_cache_tree_search(info, bytenr, 0);
+
+ return cache;
+}
+
+/*
+ * return the block group that contains teh given bytenr
+ */
+struct btrfs_block_group_cache *btrfs_lookup_block_group(
+ struct btrfs_fs_info *info,
+ u64 bytenr)
+{
+ struct btrfs_block_group_cache *cache;
+
+ cache = block_group_cache_tree_search(info, bytenr, 1);
+
+ return cache;
+}
+
+static inline void put_block_group(struct btrfs_block_group_cache *cache)
+{
+ if (atomic_dec_and_test(&cache->count))
+ kfree(cache);
+}
+
+static struct btrfs_space_info *__find_space_info(struct btrfs_fs_info *info,
+ u64 flags)
+{
+ struct list_head *head = &info->space_info;
+ struct list_head *cur;
+ struct btrfs_space_info *found;
+ list_for_each(cur, head) {
+ found = list_entry(cur, struct btrfs_space_info, list);
+ if (found->flags == flags)
+ return found;
+ }
+ return NULL;
+}
+
+static u64 div_factor(u64 num, int factor)
+{
+ if (factor == 10)
+ return num;
+ num *= factor;
+ do_div(num, 10);
+ return num;
+}
+
+u64 btrfs_find_block_group(struct btrfs_root *root,
+ u64 search_start, u64 search_hint, int owner)
+{
+ struct btrfs_block_group_cache *cache;
+ u64 used;
+ u64 last = max(search_hint, search_start);
+ u64 group_start = 0;
+ int full_search = 0;
+ int factor = 9;
+ int wrapped = 0;
+again:
+ while (1) {
+ cache = btrfs_lookup_first_block_group(root->fs_info, last);
+ if (!cache)
+ break;
+
+ spin_lock(&cache->lock);
+ last = cache->key.objectid + cache->key.offset;
+ used = btrfs_block_group_used(&cache->item);
+
+ if ((full_search || !cache->ro) &&
+ block_group_bits(cache, BTRFS_BLOCK_GROUP_METADATA)) {
+ if (used + cache->pinned + cache->reserved <
+ div_factor(cache->key.offset, factor)) {
+ group_start = cache->key.objectid;
+ spin_unlock(&cache->lock);
+ put_block_group(cache);
+ goto found;
+ }
+ }
+ spin_unlock(&cache->lock);
+ put_block_group(cache);
+ cond_resched();
+ }
+ if (!wrapped) {
+ last = search_start;
+ wrapped = 1;
+ goto again;
+ }
+ if (!full_search && factor < 10) {
+ last = search_start;
+ full_search = 1;
+ factor = 10;
+ goto again;
+ }
+found:
+ return group_start;
+}
+
+/* simple helper to search for an existing extent at a given offset */
+int btrfs_lookup_extent(struct btrfs_root *root, u64 start, u64 len)
+{
+ int ret;
+ struct btrfs_key key;
+ struct btrfs_path *path;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+ key.objectid = start;
+ key.offset = len;
+ btrfs_set_key_type(&key, BTRFS_EXTENT_ITEM_KEY);
+ ret = btrfs_search_slot(NULL, root->fs_info->extent_root, &key, path,
+ 0, 0);
+ btrfs_free_path(path);
+ return ret;
+}
+
+/*
+ * Back reference rules. Back refs have three main goals:
+ *
+ * 1) differentiate between all holders of references to an extent so that
+ * when a reference is dropped we can make sure it was a valid reference
+ * before freeing the extent.
+ *
+ * 2) Provide enough information to quickly find the holders of an extent
+ * if we notice a given block is corrupted or bad.
+ *
+ * 3) Make it easy to migrate blocks for FS shrinking or storage pool
+ * maintenance. This is actually the same as #2, but with a slightly
+ * different use case.
+ *
+ * File extents can be referenced by:
+ *
+ * - multiple snapshots, subvolumes, or different generations in one subvol
+ * - different files inside a single subvolume
+ * - different offsets inside a file (bookend extents in file.c)
+ *
+ * The extent ref structure has fields for:
+ *
+ * - Objectid of the subvolume root
+ * - Generation number of the tree holding the reference
+ * - objectid of the file holding the reference
+ * - number of references holding by parent node (alway 1 for tree blocks)
+ *
+ * Btree leaf may hold multiple references to a file extent. In most cases,
+ * these references are from same file and the corresponding offsets inside
+ * the file are close together.
+ *
+ * When a file extent is allocated the fields are filled in:
+ * (root_key.objectid, trans->transid, inode objectid, 1)
+ *
+ * When a leaf is cow'd new references are added for every file extent found
+ * in the leaf. It looks similar to the create case, but trans->transid will
+ * be different when the block is cow'd.
+ *
+ * (root_key.objectid, trans->transid, inode objectid,
+ * number of references in the leaf)
+ *
+ * When a file extent is removed either during snapshot deletion or
+ * file truncation, we find the corresponding back reference and check
+ * the following fields:
+ *
+ * (btrfs_header_owner(leaf), btrfs_header_generation(leaf),
+ * inode objectid)
+ *
+ * Btree extents can be referenced by:
+ *
+ * - Different subvolumes
+ * - Different generations of the same subvolume
+ *
+ * When a tree block is created, back references are inserted:
+ *
+ * (root->root_key.objectid, trans->transid, level, 1)
+ *
+ * When a tree block is cow'd, new back references are added for all the
+ * blocks it points to. If the tree block isn't in reference counted root,
+ * the old back references are removed. These new back references are of
+ * the form (trans->transid will have increased since creation):
+ *
+ * (root->root_key.objectid, trans->transid, level, 1)
+ *
+ * When a backref is in deleting, the following fields are checked:
+ *
+ * if backref was for a tree root:
+ * (btrfs_header_owner(itself), btrfs_header_generation(itself), level)
+ * else
+ * (btrfs_header_owner(parent), btrfs_header_generation(parent), level)
+ *
+ * Back Reference Key composing:
+ *
+ * The key objectid corresponds to the first byte in the extent, the key
+ * type is set to BTRFS_EXTENT_REF_KEY, and the key offset is the first
+ * byte of parent extent. If a extent is tree root, the key offset is set
+ * to the key objectid.
+ */
+
+static noinline int lookup_extent_backref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ u64 bytenr, u64 parent,
+ u64 ref_root, u64 ref_generation,
+ u64 owner_objectid, int del)
+{
+ struct btrfs_key key;
+ struct btrfs_extent_ref *ref;
+ struct extent_buffer *leaf;
+ u64 ref_objectid;
+ int ret;
+
+ key.objectid = bytenr;
+ key.type = BTRFS_EXTENT_REF_KEY;
+ key.offset = parent;
+
+ ret = btrfs_search_slot(trans, root, &key, path, del ? -1 : 0, 1);
+ if (ret < 0)
+ goto out;
+ if (ret > 0) {
+ ret = -ENOENT;
+ goto out;
+ }
+
+ leaf = path->nodes[0];
+ ref = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_extent_ref);
+ ref_objectid = btrfs_ref_objectid(leaf, ref);
+ if (btrfs_ref_root(leaf, ref) != ref_root ||
+ btrfs_ref_generation(leaf, ref) != ref_generation ||
+ (ref_objectid != owner_objectid &&
+ ref_objectid != BTRFS_MULTIPLE_OBJECTIDS)) {
+ ret = -EIO;
+ WARN_ON(1);
+ goto out;
+ }
+ ret = 0;
+out:
+ return ret;
+}
+
+/*
+ * updates all the backrefs that are pending on update_list for the
+ * extent_root
+ */
+static noinline int update_backrefs(struct btrfs_trans_handle *trans,
+ struct btrfs_root *extent_root,
+ struct btrfs_path *path,
+ struct list_head *update_list)
+{
+ struct btrfs_key key;
+ struct btrfs_extent_ref *ref;
+ struct btrfs_fs_info *info = extent_root->fs_info;
+ struct pending_extent_op *op;
+ struct extent_buffer *leaf;
+ int ret = 0;
+ struct list_head *cur = update_list->next;
+ u64 ref_objectid;
+ u64 ref_root = extent_root->root_key.objectid;
+
+ op = list_entry(cur, struct pending_extent_op, list);
+
+search:
+ key.objectid = op->bytenr;
+ key.type = BTRFS_EXTENT_REF_KEY;
+ key.offset = op->orig_parent;
+
+ ret = btrfs_search_slot(trans, extent_root, &key, path, 0, 1);
+ BUG_ON(ret);
+
+ leaf = path->nodes[0];
+
+loop:
+ ref = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_extent_ref);
+
+ ref_objectid = btrfs_ref_objectid(leaf, ref);
+
+ if (btrfs_ref_root(leaf, ref) != ref_root ||
+ btrfs_ref_generation(leaf, ref) != op->orig_generation ||
+ (ref_objectid != op->level &&
+ ref_objectid != BTRFS_MULTIPLE_OBJECTIDS)) {
+ printk(KERN_ERR "btrfs couldn't find %llu, parent %llu, "
+ "root %llu, owner %u\n",
+ (unsigned long long)op->bytenr,
+ (unsigned long long)op->orig_parent,
+ (unsigned long long)ref_root, op->level);
+ btrfs_print_leaf(extent_root, leaf);
+ BUG();
+ }
+
+ key.objectid = op->bytenr;
+ key.offset = op->parent;
+ key.type = BTRFS_EXTENT_REF_KEY;
+ ret = btrfs_set_item_key_safe(trans, extent_root, path, &key);
+ BUG_ON(ret);
+ ref = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_extent_ref);
+ btrfs_set_ref_generation(leaf, ref, op->generation);
+
+ cur = cur->next;
+
+ list_del_init(&op->list);
+ unlock_extent(&info->extent_ins, op->bytenr,
+ op->bytenr + op->num_bytes - 1, GFP_NOFS);
+ kfree(op);
+
+ if (cur == update_list) {
+ btrfs_mark_buffer_dirty(path->nodes[0]);
+ btrfs_release_path(extent_root, path);
+ goto out;
+ }
+
+ op = list_entry(cur, struct pending_extent_op, list);
+
+ path->slots[0]++;
+ while (path->slots[0] < btrfs_header_nritems(leaf)) {
+ btrfs_item_key_to_cpu(leaf, &key, path->slots[0]);
+ if (key.objectid == op->bytenr &&
+ key.type == BTRFS_EXTENT_REF_KEY)
+ goto loop;
+ path->slots[0]++;
+ }
+
+ btrfs_mark_buffer_dirty(path->nodes[0]);
+ btrfs_release_path(extent_root, path);
+ goto search;
+
+out:
+ return 0;
+}
+
+static noinline int insert_extents(struct btrfs_trans_handle *trans,
+ struct btrfs_root *extent_root,
+ struct btrfs_path *path,
+ struct list_head *insert_list, int nr)
+{
+ struct btrfs_key *keys;
+ u32 *data_size;
+ struct pending_extent_op *op;
+ struct extent_buffer *leaf;
+ struct list_head *cur = insert_list->next;
+ struct btrfs_fs_info *info = extent_root->fs_info;
+ u64 ref_root = extent_root->root_key.objectid;
+ int i = 0, last = 0, ret;
+ int total = nr * 2;
+
+ if (!nr)
+ return 0;
+
+ keys = kzalloc(total * sizeof(struct btrfs_key), GFP_NOFS);
+ if (!keys)
+ return -ENOMEM;
+
+ data_size = kzalloc(total * sizeof(u32), GFP_NOFS);
+ if (!data_size) {
+ kfree(keys);
+ return -ENOMEM;
+ }
+
+ list_for_each_entry(op, insert_list, list) {
+ keys[i].objectid = op->bytenr;
+ keys[i].offset = op->num_bytes;
+ keys[i].type = BTRFS_EXTENT_ITEM_KEY;
+ data_size[i] = sizeof(struct btrfs_extent_item);
+ i++;
+
+ keys[i].objectid = op->bytenr;
+ keys[i].offset = op->parent;
+ keys[i].type = BTRFS_EXTENT_REF_KEY;
+ data_size[i] = sizeof(struct btrfs_extent_ref);
+ i++;
+ }
+
+ op = list_entry(cur, struct pending_extent_op, list);
+ i = 0;
+ while (i < total) {
+ int c;
+ ret = btrfs_insert_some_items(trans, extent_root, path,
+ keys+i, data_size+i, total-i);
+ BUG_ON(ret < 0);
+
+ if (last && ret > 1)
+ BUG();
+
+ leaf = path->nodes[0];
+ for (c = 0; c < ret; c++) {
+ int ref_first = keys[i].type == BTRFS_EXTENT_REF_KEY;
+
+ /*
+ * if the first item we inserted was a backref, then
+ * the EXTENT_ITEM will be the odd c's, else it will
+ * be the even c's
+ */
+ if ((ref_first && (c % 2)) ||
+ (!ref_first && !(c % 2))) {
+ struct btrfs_extent_item *itm;
+
+ itm = btrfs_item_ptr(leaf, path->slots[0] + c,
+ struct btrfs_extent_item);
+ btrfs_set_extent_refs(path->nodes[0], itm, 1);
+ op->del++;
+ } else {
+ struct btrfs_extent_ref *ref;
+
+ ref = btrfs_item_ptr(leaf, path->slots[0] + c,
+ struct btrfs_extent_ref);
+ btrfs_set_ref_root(leaf, ref, ref_root);
+ btrfs_set_ref_generation(leaf, ref,
+ op->generation);
+ btrfs_set_ref_objectid(leaf, ref, op->level);
+ btrfs_set_ref_num_refs(leaf, ref, 1);
+ op->del++;
+ }
+
+ /*
+ * using del to see when its ok to free up the
+ * pending_extent_op. In the case where we insert the
+ * last item on the list in order to help do batching
+ * we need to not free the extent op until we actually
+ * insert the extent_item
+ */
+ if (op->del == 2) {
+ unlock_extent(&info->extent_ins, op->bytenr,
+ op->bytenr + op->num_bytes - 1,
+ GFP_NOFS);
+ cur = cur->next;
+ list_del_init(&op->list);
+ kfree(op);
+ if (cur != insert_list)
+ op = list_entry(cur,
+ struct pending_extent_op,
+ list);
+ }
+ }
+ btrfs_mark_buffer_dirty(leaf);
+ btrfs_release_path(extent_root, path);
+
+ /*
+ * Ok backref's and items usually go right next to eachother,
+ * but if we could only insert 1 item that means that we
+ * inserted on the end of a leaf, and we have no idea what may
+ * be on the next leaf so we just play it safe. In order to
+ * try and help this case we insert the last thing on our
+ * insert list so hopefully it will end up being the last
+ * thing on the leaf and everything else will be before it,
+ * which will let us insert a whole bunch of items at the same
+ * time.
+ */
+ if (ret == 1 && !last && (i + ret < total)) {
+ /*
+ * last: where we will pick up the next time around
+ * i: our current key to insert, will be total - 1
+ * cur: the current op we are screwing with
+ * op: duh
+ */
+ last = i + ret;
+ i = total - 1;
+ cur = insert_list->prev;
+ op = list_entry(cur, struct pending_extent_op, list);
+ } else if (last) {
+ /*
+ * ok we successfully inserted the last item on the
+ * list, lets reset everything
+ *
+ * i: our current key to insert, so where we left off
+ * last time
+ * last: done with this
+ * cur: the op we are messing with
+ * op: duh
+ * total: since we inserted the last key, we need to
+ * decrement total so we dont overflow
+ */
+ i = last;
+ last = 0;
+ total--;
+ if (i < total) {
+ cur = insert_list->next;
+ op = list_entry(cur, struct pending_extent_op,
+ list);
+ }
+ } else {
+ i += ret;
+ }
+
+ cond_resched();
+ }
+ ret = 0;
+ kfree(keys);
+ kfree(data_size);
+ return ret;
+}
+
+static noinline int insert_extent_backref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ u64 bytenr, u64 parent,
+ u64 ref_root, u64 ref_generation,
+ u64 owner_objectid)
+{
+ struct btrfs_key key;
+ struct extent_buffer *leaf;
+ struct btrfs_extent_ref *ref;
+ u32 num_refs;
+ int ret;
+
+ key.objectid = bytenr;
+ key.type = BTRFS_EXTENT_REF_KEY;
+ key.offset = parent;
+
+ ret = btrfs_insert_empty_item(trans, root, path, &key, sizeof(*ref));
+ if (ret == 0) {
+ leaf = path->nodes[0];
+ ref = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_extent_ref);
+ btrfs_set_ref_root(leaf, ref, ref_root);
+ btrfs_set_ref_generation(leaf, ref, ref_generation);
+ btrfs_set_ref_objectid(leaf, ref, owner_objectid);
+ btrfs_set_ref_num_refs(leaf, ref, 1);
+ } else if (ret == -EEXIST) {
+ u64 existing_owner;
+ BUG_ON(owner_objectid < BTRFS_FIRST_FREE_OBJECTID);
+ leaf = path->nodes[0];
+ ref = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_extent_ref);
+ if (btrfs_ref_root(leaf, ref) != ref_root ||
+ btrfs_ref_generation(leaf, ref) != ref_generation) {
+ ret = -EIO;
+ WARN_ON(1);
+ goto out;
+ }
+
+ num_refs = btrfs_ref_num_refs(leaf, ref);
+ BUG_ON(num_refs == 0);
+ btrfs_set_ref_num_refs(leaf, ref, num_refs + 1);
+
+ existing_owner = btrfs_ref_objectid(leaf, ref);
+ if (existing_owner != owner_objectid &&
+ existing_owner != BTRFS_MULTIPLE_OBJECTIDS) {
+ btrfs_set_ref_objectid(leaf, ref,
+ BTRFS_MULTIPLE_OBJECTIDS);
+ }
+ ret = 0;
+ } else {
+ goto out;
+ }
+ btrfs_mark_buffer_dirty(path->nodes[0]);
+out:
+ btrfs_release_path(root, path);
+ return ret;
+}
+
+static noinline int remove_extent_backref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path)
+{
+ struct extent_buffer *leaf;
+ struct btrfs_extent_ref *ref;
+ u32 num_refs;
+ int ret = 0;
+
+ leaf = path->nodes[0];
+ ref = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_extent_ref);
+ num_refs = btrfs_ref_num_refs(leaf, ref);
+ BUG_ON(num_refs == 0);
+ num_refs -= 1;
+ if (num_refs == 0) {
+ ret = btrfs_del_item(trans, root, path);
+ } else {
+ btrfs_set_ref_num_refs(leaf, ref, num_refs);
+ btrfs_mark_buffer_dirty(leaf);
+ }
+ btrfs_release_path(root, path);
+ return ret;
+}
+
+#ifdef BIO_RW_DISCARD
+static void btrfs_issue_discard(struct block_device *bdev,
+ u64 start, u64 len)
+{
+ blkdev_issue_discard(bdev, start >> 9, len >> 9, GFP_KERNEL);
+}
+#endif
+
+static int btrfs_discard_extent(struct btrfs_root *root, u64 bytenr,
+ u64 num_bytes)
+{
+#ifdef BIO_RW_DISCARD
+ int ret;
+ u64 map_length = num_bytes;
+ struct btrfs_multi_bio *multi = NULL;
+
+ /* Tell the block device(s) that the sectors can be discarded */
+ ret = btrfs_map_block(&root->fs_info->mapping_tree, READ,
+ bytenr, &map_length, &multi, 0);
+ if (!ret) {
+ struct btrfs_bio_stripe *stripe = multi->stripes;
+ int i;
+
+ if (map_length > num_bytes)
+ map_length = num_bytes;
+
+ for (i = 0; i < multi->num_stripes; i++, stripe++) {
+ btrfs_issue_discard(stripe->dev->bdev,
+ stripe->physical,
+ map_length);
+ }
+ kfree(multi);
+ }
+
+ return ret;
+#else
+ return 0;
+#endif
+}
+
+static noinline int free_extents(struct btrfs_trans_handle *trans,
+ struct btrfs_root *extent_root,
+ struct list_head *del_list)
+{
+ struct btrfs_fs_info *info = extent_root->fs_info;
+ struct btrfs_path *path;
+ struct btrfs_key key, found_key;
+ struct extent_buffer *leaf;
+ struct list_head *cur;
+ struct pending_extent_op *op;
+ struct btrfs_extent_item *ei;
+ int ret, num_to_del, extent_slot = 0, found_extent = 0;
+ u32 refs;
+ u64 bytes_freed = 0;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+ path->reada = 1;
+
+search:
+ /* search for the backref for the current ref we want to delete */
+ cur = del_list->next;
+ op = list_entry(cur, struct pending_extent_op, list);
+ ret = lookup_extent_backref(trans, extent_root, path, op->bytenr,
+ op->orig_parent,
+ extent_root->root_key.objectid,
+ op->orig_generation, op->level, 1);
+ if (ret) {
+ printk(KERN_ERR "btrfs unable to find backref byte nr %llu "
+ "root %llu gen %llu owner %u\n",
+ (unsigned long long)op->bytenr,
+ (unsigned long long)extent_root->root_key.objectid,
+ (unsigned long long)op->orig_generation, op->level);
+ btrfs_print_leaf(extent_root, path->nodes[0]);
+ WARN_ON(1);
+ goto out;
+ }
+
+ extent_slot = path->slots[0];
+ num_to_del = 1;
+ found_extent = 0;
+
+ /*
+ * if we aren't the first item on the leaf we can move back one and see
+ * if our ref is right next to our extent item
+ */
+ if (likely(extent_slot)) {
+ extent_slot--;
+ btrfs_item_key_to_cpu(path->nodes[0], &found_key,
+ extent_slot);
+ if (found_key.objectid == op->bytenr &&
+ found_key.type == BTRFS_EXTENT_ITEM_KEY &&
+ found_key.offset == op->num_bytes) {
+ num_to_del++;
+ found_extent = 1;
+ }
+ }
+
+ /*
+ * if we didn't find the extent we need to delete the backref and then
+ * search for the extent item key so we can update its ref count
+ */
+ if (!found_extent) {
+ key.objectid = op->bytenr;
+ key.type = BTRFS_EXTENT_ITEM_KEY;
+ key.offset = op->num_bytes;
+
+ ret = remove_extent_backref(trans, extent_root, path);
+ BUG_ON(ret);
+ btrfs_release_path(extent_root, path);
+ ret = btrfs_search_slot(trans, extent_root, &key, path, -1, 1);
+ BUG_ON(ret);
+ extent_slot = path->slots[0];
+ }
+
+ /* this is where we update the ref count for the extent */
+ leaf = path->nodes[0];
+ ei = btrfs_item_ptr(leaf, extent_slot, struct btrfs_extent_item);
+ refs = btrfs_extent_refs(leaf, ei);
+ BUG_ON(refs == 0);
+ refs--;
+ btrfs_set_extent_refs(leaf, ei, refs);
+
+ btrfs_mark_buffer_dirty(leaf);
+
+ /*
+ * This extent needs deleting. The reason cur_slot is extent_slot +
+ * num_to_del is because extent_slot points to the slot where the extent
+ * is, and if the backref was not right next to the extent we will be
+ * deleting at least 1 item, and will want to start searching at the
+ * slot directly next to extent_slot. However if we did find the
+ * backref next to the extent item them we will be deleting at least 2
+ * items and will want to start searching directly after the ref slot
+ */
+ if (!refs) {
+ struct list_head *pos, *n, *end;
+ int cur_slot = extent_slot+num_to_del;
+ u64 super_used;
+ u64 root_used;
+
+ path->slots[0] = extent_slot;
+ bytes_freed = op->num_bytes;
+
+ mutex_lock(&info->pinned_mutex);
+ ret = pin_down_bytes(trans, extent_root, op->bytenr,
+ op->num_bytes, op->level >=
+ BTRFS_FIRST_FREE_OBJECTID);
+ mutex_unlock(&info->pinned_mutex);
+ BUG_ON(ret < 0);
+ op->del = ret;
+
+ /*
+ * we need to see if we can delete multiple things at once, so
+ * start looping through the list of extents we are wanting to
+ * delete and see if their extent/backref's are right next to
+ * eachother and the extents only have 1 ref
+ */
+ for (pos = cur->next; pos != del_list; pos = pos->next) {
+ struct pending_extent_op *tmp;
+
+ tmp = list_entry(pos, struct pending_extent_op, list);
+
+ /* we only want to delete extent+ref at this stage */
+ if (cur_slot >= btrfs_header_nritems(leaf) - 1)
+ break;
+
+ btrfs_item_key_to_cpu(leaf, &found_key, cur_slot);
+ if (found_key.objectid != tmp->bytenr ||
+ found_key.type != BTRFS_EXTENT_ITEM_KEY ||
+ found_key.offset != tmp->num_bytes)
+ break;
+
+ /* check to make sure this extent only has one ref */
+ ei = btrfs_item_ptr(leaf, cur_slot,
+ struct btrfs_extent_item);
+ if (btrfs_extent_refs(leaf, ei) != 1)
+ break;
+
+ btrfs_item_key_to_cpu(leaf, &found_key, cur_slot+1);
+ if (found_key.objectid != tmp->bytenr ||
+ found_key.type != BTRFS_EXTENT_REF_KEY ||
+ found_key.offset != tmp->orig_parent)
+ break;
+
+ /*
+ * the ref is right next to the extent, we can set the
+ * ref count to 0 since we will delete them both now
+ */
+ btrfs_set_extent_refs(leaf, ei, 0);
+
+ /* pin down the bytes for this extent */
+ mutex_lock(&info->pinned_mutex);
+ ret = pin_down_bytes(trans, extent_root, tmp->bytenr,
+ tmp->num_bytes, tmp->level >=
+ BTRFS_FIRST_FREE_OBJECTID);
+ mutex_unlock(&info->pinned_mutex);
+ BUG_ON(ret < 0);
+
+ /*
+ * use the del field to tell if we need to go ahead and
+ * free up the extent when we delete the item or not.
+ */
+ tmp->del = ret;
+ bytes_freed += tmp->num_bytes;
+
+ num_to_del += 2;
+ cur_slot += 2;
+ }
+ end = pos;
+
+ /* update the free space counters */
+ spin_lock(&info->delalloc_lock);
+ super_used = btrfs_super_bytes_used(&info->super_copy);
+ btrfs_set_super_bytes_used(&info->super_copy,
+ super_used - bytes_freed);
+
+ root_used = btrfs_root_used(&extent_root->root_item);
+ btrfs_set_root_used(&extent_root->root_item,
+ root_used - bytes_freed);
+ spin_unlock(&info->delalloc_lock);
+
+ /* delete the items */
+ ret = btrfs_del_items(trans, extent_root, path,
+ path->slots[0], num_to_del);
+ BUG_ON(ret);
+
+ /*
+ * loop through the extents we deleted and do the cleanup work
+ * on them
+ */
+ for (pos = cur, n = pos->next; pos != end;
+ pos = n, n = pos->next) {
+ struct pending_extent_op *tmp;
+ tmp = list_entry(pos, struct pending_extent_op, list);
+
+ /*
+ * remember tmp->del tells us wether or not we pinned
+ * down the extent
+ */
+ ret = update_block_group(trans, extent_root,
+ tmp->bytenr, tmp->num_bytes, 0,
+ tmp->del);
+ BUG_ON(ret);
+
+ list_del_init(&tmp->list);
+ unlock_extent(&info->extent_ins, tmp->bytenr,
+ tmp->bytenr + tmp->num_bytes - 1,
+ GFP_NOFS);
+ kfree(tmp);
+ }
+ } else if (refs && found_extent) {
+ /*
+ * the ref and extent were right next to eachother, but the
+ * extent still has a ref, so just free the backref and keep
+ * going
+ */
+ ret = remove_extent_backref(trans, extent_root, path);
+ BUG_ON(ret);
+
+ list_del_init(&op->list);
+ unlock_extent(&info->extent_ins, op->bytenr,
+ op->bytenr + op->num_bytes - 1, GFP_NOFS);
+ kfree(op);
+ } else {
+ /*
+ * the extent has multiple refs and the backref we were looking
+ * for was not right next to it, so just unlock and go next,
+ * we're good to go
+ */
+ list_del_init(&op->list);
+ unlock_extent(&info->extent_ins, op->bytenr,
+ op->bytenr + op->num_bytes - 1, GFP_NOFS);
+ kfree(op);
+ }
+
+ btrfs_release_path(extent_root, path);
+ if (!list_empty(del_list))
+ goto search;
+
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
+static int __btrfs_update_extent_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 bytenr,
+ u64 orig_parent, u64 parent,
+ u64 orig_root, u64 ref_root,
+ u64 orig_generation, u64 ref_generation,
+ u64 owner_objectid)
+{
+ int ret;
+ struct btrfs_root *extent_root = root->fs_info->extent_root;
+ struct btrfs_path *path;
+
+ if (root == root->fs_info->extent_root) {
+ struct pending_extent_op *extent_op;
+ u64 num_bytes;
+
+ BUG_ON(owner_objectid >= BTRFS_MAX_LEVEL);
+ num_bytes = btrfs_level_size(root, (int)owner_objectid);
+ mutex_lock(&root->fs_info->extent_ins_mutex);
+ if (test_range_bit(&root->fs_info->extent_ins, bytenr,
+ bytenr + num_bytes - 1, EXTENT_WRITEBACK, 0)) {
+ u64 priv;
+ ret = get_state_private(&root->fs_info->extent_ins,
+ bytenr, &priv);
+ BUG_ON(ret);
+ extent_op = (struct pending_extent_op *)
+ (unsigned long)priv;
+ BUG_ON(extent_op->parent != orig_parent);
+ BUG_ON(extent_op->generation != orig_generation);
+
+ extent_op->parent = parent;
+ extent_op->generation = ref_generation;
+ } else {
+ extent_op = kmalloc(sizeof(*extent_op), GFP_NOFS);
+ BUG_ON(!extent_op);
+
+ extent_op->type = PENDING_BACKREF_UPDATE;
+ extent_op->bytenr = bytenr;
+ extent_op->num_bytes = num_bytes;
+ extent_op->parent = parent;
+ extent_op->orig_parent = orig_parent;
+ extent_op->generation = ref_generation;
+ extent_op->orig_generation = orig_generation;
+ extent_op->level = (int)owner_objectid;
+ INIT_LIST_HEAD(&extent_op->list);
+ extent_op->del = 0;
+
+ set_extent_bits(&root->fs_info->extent_ins,
+ bytenr, bytenr + num_bytes - 1,
+ EXTENT_WRITEBACK, GFP_NOFS);
+ set_state_private(&root->fs_info->extent_ins,
+ bytenr, (unsigned long)extent_op);
+ }
+ mutex_unlock(&root->fs_info->extent_ins_mutex);
+ return 0;
+ }
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+ ret = lookup_extent_backref(trans, extent_root, path,
+ bytenr, orig_parent, orig_root,
+ orig_generation, owner_objectid, 1);
+ if (ret)
+ goto out;
+ ret = remove_extent_backref(trans, extent_root, path);
+ if (ret)
+ goto out;
+ ret = insert_extent_backref(trans, extent_root, path, bytenr,
+ parent, ref_root, ref_generation,
+ owner_objectid);
+ BUG_ON(ret);
+ finish_current_insert(trans, extent_root, 0);
+ del_pending_extents(trans, extent_root, 0);
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
+int btrfs_update_extent_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 bytenr,
+ u64 orig_parent, u64 parent,
+ u64 ref_root, u64 ref_generation,
+ u64 owner_objectid)
+{
+ int ret;
+ if (ref_root == BTRFS_TREE_LOG_OBJECTID &&
+ owner_objectid < BTRFS_FIRST_FREE_OBJECTID)
+ return 0;
+ ret = __btrfs_update_extent_ref(trans, root, bytenr, orig_parent,
+ parent, ref_root, ref_root,
+ ref_generation, ref_generation,
+ owner_objectid);
+ return ret;
+}
+
+static int __btrfs_inc_extent_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 bytenr,
+ u64 orig_parent, u64 parent,
+ u64 orig_root, u64 ref_root,
+ u64 orig_generation, u64 ref_generation,
+ u64 owner_objectid)
+{
+ struct btrfs_path *path;
+ int ret;
+ struct btrfs_key key;
+ struct extent_buffer *l;
+ struct btrfs_extent_item *item;
+ u32 refs;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ path->reada = 1;
+ key.objectid = bytenr;
+ key.type = BTRFS_EXTENT_ITEM_KEY;
+ key.offset = (u64)-1;
+
+ ret = btrfs_search_slot(trans, root->fs_info->extent_root, &key, path,
+ 0, 1);
+ if (ret < 0)
+ return ret;
+ BUG_ON(ret == 0 || path->slots[0] == 0);
+
+ path->slots[0]--;
+ l = path->nodes[0];
+
+ btrfs_item_key_to_cpu(l, &key, path->slots[0]);
+ if (key.objectid != bytenr) {
+ btrfs_print_leaf(root->fs_info->extent_root, path->nodes[0]);
+ printk(KERN_ERR "btrfs wanted %llu found %llu\n",
+ (unsigned long long)bytenr,
+ (unsigned long long)key.objectid);
+ BUG();
+ }
+ BUG_ON(key.type != BTRFS_EXTENT_ITEM_KEY);
+
+ item = btrfs_item_ptr(l, path->slots[0], struct btrfs_extent_item);
+ refs = btrfs_extent_refs(l, item);
+ btrfs_set_extent_refs(l, item, refs + 1);
+ btrfs_mark_buffer_dirty(path->nodes[0]);
+
+ btrfs_release_path(root->fs_info->extent_root, path);
+
+ path->reada = 1;
+ ret = insert_extent_backref(trans, root->fs_info->extent_root,
+ path, bytenr, parent,
+ ref_root, ref_generation,
+ owner_objectid);
+ BUG_ON(ret);
+ finish_current_insert(trans, root->fs_info->extent_root, 0);
+ del_pending_extents(trans, root->fs_info->extent_root, 0);
+
+ btrfs_free_path(path);
+ return 0;
+}
+
+int btrfs_inc_extent_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 bytenr, u64 num_bytes, u64 parent,
+ u64 ref_root, u64 ref_generation,
+ u64 owner_objectid)
+{
+ int ret;
+ if (ref_root == BTRFS_TREE_LOG_OBJECTID &&
+ owner_objectid < BTRFS_FIRST_FREE_OBJECTID)
+ return 0;
+ ret = __btrfs_inc_extent_ref(trans, root, bytenr, 0, parent,
+ 0, ref_root, 0, ref_generation,
+ owner_objectid);
+ return ret;
+}
+
+int btrfs_extent_post_op(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root)
+{
+ finish_current_insert(trans, root->fs_info->extent_root, 1);
+ del_pending_extents(trans, root->fs_info->extent_root, 1);
+ return 0;
+}
+
+int btrfs_lookup_extent_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 bytenr,
+ u64 num_bytes, u32 *refs)
+{
+ struct btrfs_path *path;
+ int ret;
+ struct btrfs_key key;
+ struct extent_buffer *l;
+ struct btrfs_extent_item *item;
+
+ WARN_ON(num_bytes < root->sectorsize);
+ path = btrfs_alloc_path();
+ path->reada = 1;
+ key.objectid = bytenr;
+ key.offset = num_bytes;
+ btrfs_set_key_type(&key, BTRFS_EXTENT_ITEM_KEY);
+ ret = btrfs_search_slot(trans, root->fs_info->extent_root, &key, path,
+ 0, 0);
+ if (ret < 0)
+ goto out;
+ if (ret != 0) {
+ btrfs_print_leaf(root, path->nodes[0]);
+ printk(KERN_INFO "btrfs failed to find block number %llu\n",
+ (unsigned long long)bytenr);
+ BUG();
+ }
+ l = path->nodes[0];
+ item = btrfs_item_ptr(l, path->slots[0], struct btrfs_extent_item);
+ *refs = btrfs_extent_refs(l, item);
+out:
+ btrfs_free_path(path);
+ return 0;
+}
+
+int btrfs_cross_ref_exist(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 objectid, u64 bytenr)
+{
+ struct btrfs_root *extent_root = root->fs_info->extent_root;
+ struct btrfs_path *path;
+ struct extent_buffer *leaf;
+ struct btrfs_extent_ref *ref_item;
+ struct btrfs_key key;
+ struct btrfs_key found_key;
+ u64 ref_root;
+ u64 last_snapshot;
+ u32 nritems;
+ int ret;
+
+ key.objectid = bytenr;
+ key.offset = (u64)-1;
+ key.type = BTRFS_EXTENT_ITEM_KEY;
+
+ path = btrfs_alloc_path();
+ ret = btrfs_search_slot(NULL, extent_root, &key, path, 0, 0);
+ if (ret < 0)
+ goto out;
+ BUG_ON(ret == 0);
+
+ ret = -ENOENT;
+ if (path->slots[0] == 0)
+ goto out;
+
+ path->slots[0]--;
+ leaf = path->nodes[0];
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+
+ if (found_key.objectid != bytenr ||
+ found_key.type != BTRFS_EXTENT_ITEM_KEY)
+ goto out;
+
+ last_snapshot = btrfs_root_last_snapshot(&root->root_item);
+ while (1) {
+ leaf = path->nodes[0];
+ nritems = btrfs_header_nritems(leaf);
+ if (path->slots[0] >= nritems) {
+ ret = btrfs_next_leaf(extent_root, path);
+ if (ret < 0)
+ goto out;
+ if (ret == 0)
+ continue;
+ break;
+ }
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+ if (found_key.objectid != bytenr)
+ break;
+
+ if (found_key.type != BTRFS_EXTENT_REF_KEY) {
+ path->slots[0]++;
+ continue;
+ }
+
+ ref_item = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_extent_ref);
+ ref_root = btrfs_ref_root(leaf, ref_item);
+ if ((ref_root != root->root_key.objectid &&
+ ref_root != BTRFS_TREE_LOG_OBJECTID) ||
+ objectid != btrfs_ref_objectid(leaf, ref_item)) {
+ ret = 1;
+ goto out;
+ }
+ if (btrfs_ref_generation(leaf, ref_item) <= last_snapshot) {
+ ret = 1;
+ goto out;
+ }
+
+ path->slots[0]++;
+ }
+ ret = 0;
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
+int btrfs_cache_ref(struct btrfs_trans_handle *trans, struct btrfs_root *root,
+ struct extent_buffer *buf, u32 nr_extents)
+{
+ struct btrfs_key key;
+ struct btrfs_file_extent_item *fi;
+ u64 root_gen;
+ u32 nritems;
+ int i;
+ int level;
+ int ret = 0;
+ int shared = 0;
+
+ if (!root->ref_cows)
+ return 0;
+
+ if (root->root_key.objectid != BTRFS_TREE_RELOC_OBJECTID) {
+ shared = 0;
+ root_gen = root->root_key.offset;
+ } else {
+ shared = 1;
+ root_gen = trans->transid - 1;
+ }
+
+ level = btrfs_header_level(buf);
+ nritems = btrfs_header_nritems(buf);
+
+ if (level == 0) {
+ struct btrfs_leaf_ref *ref;
+ struct btrfs_extent_info *info;
+
+ ref = btrfs_alloc_leaf_ref(root, nr_extents);
+ if (!ref) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ ref->root_gen = root_gen;
+ ref->bytenr = buf->start;
+ ref->owner = btrfs_header_owner(buf);
+ ref->generation = btrfs_header_generation(buf);
+ ref->nritems = nr_extents;
+ info = ref->extents;
+
+ for (i = 0; nr_extents > 0 && i < nritems; i++) {
+ u64 disk_bytenr;
+ btrfs_item_key_to_cpu(buf, &key, i);
+ if (btrfs_key_type(&key) != BTRFS_EXTENT_DATA_KEY)
+ continue;
+ fi = btrfs_item_ptr(buf, i,
+ struct btrfs_file_extent_item);
+ if (btrfs_file_extent_type(buf, fi) ==
+ BTRFS_FILE_EXTENT_INLINE)
+ continue;
+ disk_bytenr = btrfs_file_extent_disk_bytenr(buf, fi);
+ if (disk_bytenr == 0)
+ continue;
+
+ info->bytenr = disk_bytenr;
+ info->num_bytes =
+ btrfs_file_extent_disk_num_bytes(buf, fi);
+ info->objectid = key.objectid;
+ info->offset = key.offset;
+ info++;
+ }
+
+ ret = btrfs_add_leaf_ref(root, ref, shared);
+ if (ret == -EEXIST && shared) {
+ struct btrfs_leaf_ref *old;
+ old = btrfs_lookup_leaf_ref(root, ref->bytenr);
+ BUG_ON(!old);
+ btrfs_remove_leaf_ref(root, old);
+ btrfs_free_leaf_ref(root, old);
+ ret = btrfs_add_leaf_ref(root, ref, shared);
+ }
+ WARN_ON(ret);
+ btrfs_free_leaf_ref(root, ref);
+ }
+out:
+ return ret;
+}
+
+int btrfs_inc_ref(struct btrfs_trans_handle *trans, struct btrfs_root *root,
+ struct extent_buffer *orig_buf, struct extent_buffer *buf,
+ u32 *nr_extents)
+{
+ u64 bytenr;
+ u64 ref_root;
+ u64 orig_root;
+ u64 ref_generation;
+ u64 orig_generation;
+ u32 nritems;
+ u32 nr_file_extents = 0;
+ struct btrfs_key key;
+ struct btrfs_file_extent_item *fi;
+ int i;
+ int level;
+ int ret = 0;
+ int faili = 0;
+ int (*process_func)(struct btrfs_trans_handle *, struct btrfs_root *,
+ u64, u64, u64, u64, u64, u64, u64, u64);
+
+ ref_root = btrfs_header_owner(buf);
+ ref_generation = btrfs_header_generation(buf);
+ orig_root = btrfs_header_owner(orig_buf);
+ orig_generation = btrfs_header_generation(orig_buf);
+
+ nritems = btrfs_header_nritems(buf);
+ level = btrfs_header_level(buf);
+
+ if (root->ref_cows) {
+ process_func = __btrfs_inc_extent_ref;
+ } else {
+ if (level == 0 &&
+ root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID)
+ goto out;
+ if (level != 0 &&
+ root->root_key.objectid == BTRFS_TREE_LOG_OBJECTID)
+ goto out;
+ process_func = __btrfs_update_extent_ref;
+ }
+
+ for (i = 0; i < nritems; i++) {
+ cond_resched();
+ if (level == 0) {
+ btrfs_item_key_to_cpu(buf, &key, i);
+ if (btrfs_key_type(&key) != BTRFS_EXTENT_DATA_KEY)
+ continue;
+ fi = btrfs_item_ptr(buf, i,
+ struct btrfs_file_extent_item);
+ if (btrfs_file_extent_type(buf, fi) ==
+ BTRFS_FILE_EXTENT_INLINE)
+ continue;
+ bytenr = btrfs_file_extent_disk_bytenr(buf, fi);
+ if (bytenr == 0)
+ continue;
+
+ nr_file_extents++;
+
+ ret = process_func(trans, root, bytenr,
+ orig_buf->start, buf->start,
+ orig_root, ref_root,
+ orig_generation, ref_generation,
+ key.objectid);
+
+ if (ret) {
+ faili = i;
+ WARN_ON(1);
+ goto fail;
+ }
+ } else {
+ bytenr = btrfs_node_blockptr(buf, i);
+ ret = process_func(trans, root, bytenr,
+ orig_buf->start, buf->start,
+ orig_root, ref_root,
+ orig_generation, ref_generation,
+ level - 1);
+ if (ret) {
+ faili = i;
+ WARN_ON(1);
+ goto fail;
+ }
+ }
+ }
+out:
+ if (nr_extents) {
+ if (level == 0)
+ *nr_extents = nr_file_extents;
+ else
+ *nr_extents = nritems;
+ }
+ return 0;
+fail:
+ WARN_ON(1);
+ return ret;
+}
+
+int btrfs_update_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct extent_buffer *orig_buf,
+ struct extent_buffer *buf, int start_slot, int nr)
+
+{
+ u64 bytenr;
+ u64 ref_root;
+ u64 orig_root;
+ u64 ref_generation;
+ u64 orig_generation;
+ struct btrfs_key key;
+ struct btrfs_file_extent_item *fi;
+ int i;
+ int ret;
+ int slot;
+ int level;
+
+ BUG_ON(start_slot < 0);
+ BUG_ON(start_slot + nr > btrfs_header_nritems(buf));
+
+ ref_root = btrfs_header_owner(buf);
+ ref_generation = btrfs_header_generation(buf);
+ orig_root = btrfs_header_owner(orig_buf);
+ orig_generation = btrfs_header_generation(orig_buf);
+ level = btrfs_header_level(buf);
+
+ if (!root->ref_cows) {
+ if (level == 0 &&
+ root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID)
+ return 0;
+ if (level != 0 &&
+ root->root_key.objectid == BTRFS_TREE_LOG_OBJECTID)
+ return 0;
+ }
+
+ for (i = 0, slot = start_slot; i < nr; i++, slot++) {
+ cond_resched();
+ if (level == 0) {
+ btrfs_item_key_to_cpu(buf, &key, slot);
+ if (btrfs_key_type(&key) != BTRFS_EXTENT_DATA_KEY)
+ continue;
+ fi = btrfs_item_ptr(buf, slot,
+ struct btrfs_file_extent_item);
+ if (btrfs_file_extent_type(buf, fi) ==
+ BTRFS_FILE_EXTENT_INLINE)
+ continue;
+ bytenr = btrfs_file_extent_disk_bytenr(buf, fi);
+ if (bytenr == 0)
+ continue;
+ ret = __btrfs_update_extent_ref(trans, root, bytenr,
+ orig_buf->start, buf->start,
+ orig_root, ref_root,
+ orig_generation, ref_generation,
+ key.objectid);
+ if (ret)
+ goto fail;
+ } else {
+ bytenr = btrfs_node_blockptr(buf, slot);
+ ret = __btrfs_update_extent_ref(trans, root, bytenr,
+ orig_buf->start, buf->start,
+ orig_root, ref_root,
+ orig_generation, ref_generation,
+ level - 1);
+ if (ret)
+ goto fail;
+ }
+ }
+ return 0;
+fail:
+ WARN_ON(1);
+ return -1;
+}
+
+static int write_one_cache_group(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct btrfs_block_group_cache *cache)
+{
+ int ret;
+ int pending_ret;
+ struct btrfs_root *extent_root = root->fs_info->extent_root;
+ unsigned long bi;
+ struct extent_buffer *leaf;
+
+ ret = btrfs_search_slot(trans, extent_root, &cache->key, path, 0, 1);
+ if (ret < 0)
+ goto fail;
+ BUG_ON(ret);
+
+ leaf = path->nodes[0];
+ bi = btrfs_item_ptr_offset(leaf, path->slots[0]);
+ write_extent_buffer(leaf, &cache->item, bi, sizeof(cache->item));
+ btrfs_mark_buffer_dirty(leaf);
+ btrfs_release_path(extent_root, path);
+fail:
+ finish_current_insert(trans, extent_root, 0);
+ pending_ret = del_pending_extents(trans, extent_root, 0);
+ if (ret)
+ return ret;
+ if (pending_ret)
+ return pending_ret;
+ return 0;
+
+}
+
+int btrfs_write_dirty_block_groups(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root)
+{
+ struct btrfs_block_group_cache *cache, *entry;
+ struct rb_node *n;
+ int err = 0;
+ int werr = 0;
+ struct btrfs_path *path;
+ u64 last = 0;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ while (1) {
+ cache = NULL;
+ spin_lock(&root->fs_info->block_group_cache_lock);
+ for (n = rb_first(&root->fs_info->block_group_cache_tree);
+ n; n = rb_next(n)) {
+ entry = rb_entry(n, struct btrfs_block_group_cache,
+ cache_node);
+ if (entry->dirty) {
+ cache = entry;
+ break;
+ }
+ }
+ spin_unlock(&root->fs_info->block_group_cache_lock);
+
+ if (!cache)
+ break;
+
+ cache->dirty = 0;
+ last += cache->key.offset;
+
+ err = write_one_cache_group(trans, root,
+ path, cache);
+ /*
+ * if we fail to write the cache group, we want
+ * to keep it marked dirty in hopes that a later
+ * write will work
+ */
+ if (err) {
+ werr = err;
+ continue;
+ }
+ }
+ btrfs_free_path(path);
+ return werr;
+}
+
+int btrfs_extent_readonly(struct btrfs_root *root, u64 bytenr)
+{
+ struct btrfs_block_group_cache *block_group;
+ int readonly = 0;
+
+ block_group = btrfs_lookup_block_group(root->fs_info, bytenr);
+ if (!block_group || block_group->ro)
+ readonly = 1;
+ if (block_group)
+ put_block_group(block_group);
+ return readonly;
+}
+
+static int update_space_info(struct btrfs_fs_info *info, u64 flags,
+ u64 total_bytes, u64 bytes_used,
+ struct btrfs_space_info **space_info)
+{
+ struct btrfs_space_info *found;
+
+ found = __find_space_info(info, flags);
+ if (found) {
+ spin_lock(&found->lock);
+ found->total_bytes += total_bytes;
+ found->bytes_used += bytes_used;
+ found->full = 0;
+ spin_unlock(&found->lock);
+ *space_info = found;
+ return 0;
+ }
+ found = kzalloc(sizeof(*found), GFP_NOFS);
+ if (!found)
+ return -ENOMEM;
+
+ list_add(&found->list, &info->space_info);
+ INIT_LIST_HEAD(&found->block_groups);
+ init_rwsem(&found->groups_sem);
+ spin_lock_init(&found->lock);
+ found->flags = flags;
+ found->total_bytes = total_bytes;
+ found->bytes_used = bytes_used;
+ found->bytes_pinned = 0;
+ found->bytes_reserved = 0;
+ found->bytes_readonly = 0;
+ found->full = 0;
+ found->force_alloc = 0;
+ *space_info = found;
+ return 0;
+}
+
+static void set_avail_alloc_bits(struct btrfs_fs_info *fs_info, u64 flags)
+{
+ u64 extra_flags = flags & (BTRFS_BLOCK_GROUP_RAID0 |
+ BTRFS_BLOCK_GROUP_RAID1 |
+ BTRFS_BLOCK_GROUP_RAID10 |
+ BTRFS_BLOCK_GROUP_DUP);
+ if (extra_flags) {
+ if (flags & BTRFS_BLOCK_GROUP_DATA)
+ fs_info->avail_data_alloc_bits |= extra_flags;
+ if (flags & BTRFS_BLOCK_GROUP_METADATA)
+ fs_info->avail_metadata_alloc_bits |= extra_flags;
+ if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
+ fs_info->avail_system_alloc_bits |= extra_flags;
+ }
+}
+
+static void set_block_group_readonly(struct btrfs_block_group_cache *cache)
+{
+ spin_lock(&cache->space_info->lock);
+ spin_lock(&cache->lock);
+ if (!cache->ro) {
+ cache->space_info->bytes_readonly += cache->key.offset -
+ btrfs_block_group_used(&cache->item);
+ cache->ro = 1;
+ }
+ spin_unlock(&cache->lock);
+ spin_unlock(&cache->space_info->lock);
+}
+
+u64 btrfs_reduce_alloc_profile(struct btrfs_root *root, u64 flags)
+{
+ u64 num_devices = root->fs_info->fs_devices->rw_devices;
+
+ if (num_devices == 1)
+ flags &= ~(BTRFS_BLOCK_GROUP_RAID1 | BTRFS_BLOCK_GROUP_RAID0);
+ if (num_devices < 4)
+ flags &= ~BTRFS_BLOCK_GROUP_RAID10;
+
+ if ((flags & BTRFS_BLOCK_GROUP_DUP) &&
+ (flags & (BTRFS_BLOCK_GROUP_RAID1 |
+ BTRFS_BLOCK_GROUP_RAID10))) {
+ flags &= ~BTRFS_BLOCK_GROUP_DUP;
+ }
+
+ if ((flags & BTRFS_BLOCK_GROUP_RAID1) &&
+ (flags & BTRFS_BLOCK_GROUP_RAID10)) {
+ flags &= ~BTRFS_BLOCK_GROUP_RAID1;
+ }
+
+ if ((flags & BTRFS_BLOCK_GROUP_RAID0) &&
+ ((flags & BTRFS_BLOCK_GROUP_RAID1) |
+ (flags & BTRFS_BLOCK_GROUP_RAID10) |
+ (flags & BTRFS_BLOCK_GROUP_DUP)))
+ flags &= ~BTRFS_BLOCK_GROUP_RAID0;
+ return flags;
+}
+
+static int do_chunk_alloc(struct btrfs_trans_handle *trans,
+ struct btrfs_root *extent_root, u64 alloc_bytes,
+ u64 flags, int force)
+{
+ struct btrfs_space_info *space_info;
+ u64 thresh;
+ int ret = 0;
+
+ mutex_lock(&extent_root->fs_info->chunk_mutex);
+
+ flags = btrfs_reduce_alloc_profile(extent_root, flags);
+
+ space_info = __find_space_info(extent_root->fs_info, flags);
+ if (!space_info) {
+ ret = update_space_info(extent_root->fs_info, flags,
+ 0, 0, &space_info);
+ BUG_ON(ret);
+ }
+ BUG_ON(!space_info);
+
+ spin_lock(&space_info->lock);
+ if (space_info->force_alloc) {
+ force = 1;
+ space_info->force_alloc = 0;
+ }
+ if (space_info->full) {
+ spin_unlock(&space_info->lock);
+ goto out;
+ }
+
+ thresh = space_info->total_bytes - space_info->bytes_readonly;
+ thresh = div_factor(thresh, 6);
+ if (!force &&
+ (space_info->bytes_used + space_info->bytes_pinned +
+ space_info->bytes_reserved + alloc_bytes) < thresh) {
+ spin_unlock(&space_info->lock);
+ goto out;
+ }
+ spin_unlock(&space_info->lock);
+
+ ret = btrfs_alloc_chunk(trans, extent_root, flags);
+ if (ret)
+ space_info->full = 1;
+out:
+ mutex_unlock(&extent_root->fs_info->chunk_mutex);
+ return ret;
+}
+
+static int update_block_group(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 bytenr, u64 num_bytes, int alloc,
+ int mark_free)
+{
+ struct btrfs_block_group_cache *cache;
+ struct btrfs_fs_info *info = root->fs_info;
+ u64 total = num_bytes;
+ u64 old_val;
+ u64 byte_in_group;
+
+ while (total) {
+ cache = btrfs_lookup_block_group(info, bytenr);
+ if (!cache)
+ return -1;
+ byte_in_group = bytenr - cache->key.objectid;
+ WARN_ON(byte_in_group > cache->key.offset);
+
+ spin_lock(&cache->space_info->lock);
+ spin_lock(&cache->lock);
+ cache->dirty = 1;
+ old_val = btrfs_block_group_used(&cache->item);
+ num_bytes = min(total, cache->key.offset - byte_in_group);
+ if (alloc) {
+ old_val += num_bytes;
+ cache->space_info->bytes_used += num_bytes;
+ if (cache->ro)
+ cache->space_info->bytes_readonly -= num_bytes;
+ btrfs_set_block_group_used(&cache->item, old_val);
+ spin_unlock(&cache->lock);
+ spin_unlock(&cache->space_info->lock);
+ } else {
+ old_val -= num_bytes;
+ cache->space_info->bytes_used -= num_bytes;
+ if (cache->ro)
+ cache->space_info->bytes_readonly += num_bytes;
+ btrfs_set_block_group_used(&cache->item, old_val);
+ spin_unlock(&cache->lock);
+ spin_unlock(&cache->space_info->lock);
+ if (mark_free) {
+ int ret;
+
+ ret = btrfs_discard_extent(root, bytenr,
+ num_bytes);
+ WARN_ON(ret);
+
+ ret = btrfs_add_free_space(cache, bytenr,
+ num_bytes);
+ WARN_ON(ret);
+ }
+ }
+ put_block_group(cache);
+ total -= num_bytes;
+ bytenr += num_bytes;
+ }
+ return 0;
+}
+
+static u64 first_logical_byte(struct btrfs_root *root, u64 search_start)
+{
+ struct btrfs_block_group_cache *cache;
+ u64 bytenr;
+
+ cache = btrfs_lookup_first_block_group(root->fs_info, search_start);
+ if (!cache)
+ return 0;
+
+ bytenr = cache->key.objectid;
+ put_block_group(cache);
+
+ return bytenr;
+}
+
+int btrfs_update_pinned_extents(struct btrfs_root *root,
+ u64 bytenr, u64 num, int pin)
+{
+ u64 len;
+ struct btrfs_block_group_cache *cache;
+ struct btrfs_fs_info *fs_info = root->fs_info;
+
+ WARN_ON(!mutex_is_locked(&root->fs_info->pinned_mutex));
+ if (pin) {
+ set_extent_dirty(&fs_info->pinned_extents,
+ bytenr, bytenr + num - 1, GFP_NOFS);
+ } else {
+ clear_extent_dirty(&fs_info->pinned_extents,
+ bytenr, bytenr + num - 1, GFP_NOFS);
+ }
+ while (num > 0) {
+ cache = btrfs_lookup_block_group(fs_info, bytenr);
+ BUG_ON(!cache);
+ len = min(num, cache->key.offset -
+ (bytenr - cache->key.objectid));
+ if (pin) {
+ spin_lock(&cache->space_info->lock);
+ spin_lock(&cache->lock);
+ cache->pinned += len;
+ cache->space_info->bytes_pinned += len;
+ spin_unlock(&cache->lock);
+ spin_unlock(&cache->space_info->lock);
+ fs_info->total_pinned += len;
+ } else {
+ spin_lock(&cache->space_info->lock);
+ spin_lock(&cache->lock);
+ cache->pinned -= len;
+ cache->space_info->bytes_pinned -= len;
+ spin_unlock(&cache->lock);
+ spin_unlock(&cache->space_info->lock);
+ fs_info->total_pinned -= len;
+ if (cache->cached)
+ btrfs_add_free_space(cache, bytenr, len);
+ }
+ put_block_group(cache);
+ bytenr += len;
+ num -= len;
+ }
+ return 0;
+}
+
+static int update_reserved_extents(struct btrfs_root *root,
+ u64 bytenr, u64 num, int reserve)
+{
+ u64 len;
+ struct btrfs_block_group_cache *cache;
+ struct btrfs_fs_info *fs_info = root->fs_info;
+
+ while (num > 0) {
+ cache = btrfs_lookup_block_group(fs_info, bytenr);
+ BUG_ON(!cache);
+ len = min(num, cache->key.offset -
+ (bytenr - cache->key.objectid));
+
+ spin_lock(&cache->space_info->lock);
+ spin_lock(&cache->lock);
+ if (reserve) {
+ cache->reserved += len;
+ cache->space_info->bytes_reserved += len;
+ } else {
+ cache->reserved -= len;
+ cache->space_info->bytes_reserved -= len;
+ }
+ spin_unlock(&cache->lock);
+ spin_unlock(&cache->space_info->lock);
+ put_block_group(cache);
+ bytenr += len;
+ num -= len;
+ }
+ return 0;
+}
+
+int btrfs_copy_pinned(struct btrfs_root *root, struct extent_io_tree *copy)
+{
+ u64 last = 0;
+ u64 start;
+ u64 end;
+ struct extent_io_tree *pinned_extents = &root->fs_info->pinned_extents;
+ int ret;
+
+ mutex_lock(&root->fs_info->pinned_mutex);
+ while (1) {
+ ret = find_first_extent_bit(pinned_extents, last,
+ &start, &end, EXTENT_DIRTY);
+ if (ret)
+ break;
+ set_extent_dirty(copy, start, end, GFP_NOFS);
+ last = end + 1;
+ }
+ mutex_unlock(&root->fs_info->pinned_mutex);
+ return 0;
+}
+
+int btrfs_finish_extent_commit(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct extent_io_tree *unpin)
+{
+ u64 start;
+ u64 end;
+ int ret;
+
+ mutex_lock(&root->fs_info->pinned_mutex);
+ while (1) {
+ ret = find_first_extent_bit(unpin, 0, &start, &end,
+ EXTENT_DIRTY);
+ if (ret)
+ break;
+
+ ret = btrfs_discard_extent(root, start, end + 1 - start);
+
+ btrfs_update_pinned_extents(root, start, end + 1 - start, 0);
+ clear_extent_dirty(unpin, start, end, GFP_NOFS);
+
+ if (need_resched()) {
+ mutex_unlock(&root->fs_info->pinned_mutex);
+ cond_resched();
+ mutex_lock(&root->fs_info->pinned_mutex);
+ }
+ }
+ mutex_unlock(&root->fs_info->pinned_mutex);
+ return ret;
+}
+
+static int finish_current_insert(struct btrfs_trans_handle *trans,
+ struct btrfs_root *extent_root, int all)
+{
+ u64 start;
+ u64 end;
+ u64 priv;
+ u64 search = 0;
+ u64 skipped = 0;
+ struct btrfs_fs_info *info = extent_root->fs_info;
+ struct btrfs_path *path;
+ struct pending_extent_op *extent_op, *tmp;
+ struct list_head insert_list, update_list;
+ int ret;
+ int num_inserts = 0, max_inserts;
+
+ path = btrfs_alloc_path();
+ INIT_LIST_HEAD(&insert_list);
+ INIT_LIST_HEAD(&update_list);
+
+ max_inserts = extent_root->leafsize /
+ (2 * sizeof(struct btrfs_key) + 2 * sizeof(struct btrfs_item) +
+ sizeof(struct btrfs_extent_ref) +
+ sizeof(struct btrfs_extent_item));
+again:
+ mutex_lock(&info->extent_ins_mutex);
+ while (1) {
+ ret = find_first_extent_bit(&info->extent_ins, search, &start,
+ &end, EXTENT_WRITEBACK);
+ if (ret) {
+ if (skipped && all && !num_inserts) {
+ skipped = 0;
+ search = 0;
+ continue;
+ }
+ mutex_unlock(&info->extent_ins_mutex);
+ break;
+ }
+
+ ret = try_lock_extent(&info->extent_ins, start, end, GFP_NOFS);
+ if (!ret) {
+ skipped = 1;
+ search = end + 1;
+ if (need_resched()) {
+ mutex_unlock(&info->extent_ins_mutex);
+ cond_resched();
+ mutex_lock(&info->extent_ins_mutex);
+ }
+ continue;
+ }
+
+ ret = get_state_private(&info->extent_ins, start, &priv);
+ BUG_ON(ret);
+ extent_op = (struct pending_extent_op *)(unsigned long) priv;
+
+ if (extent_op->type == PENDING_EXTENT_INSERT) {
+ num_inserts++;
+ list_add_tail(&extent_op->list, &insert_list);
+ search = end + 1;
+ if (num_inserts == max_inserts) {
+ mutex_unlock(&info->extent_ins_mutex);
+ break;
+ }
+ } else if (extent_op->type == PENDING_BACKREF_UPDATE) {
+ list_add_tail(&extent_op->list, &update_list);
+ search = end + 1;
+ } else {
+ BUG();
+ }
+ }
+
+ /*
+ * process the update list, clear the writeback bit for it, and if
+ * somebody marked this thing for deletion then just unlock it and be
+ * done, the free_extents will handle it
+ */
+ mutex_lock(&info->extent_ins_mutex);
+ list_for_each_entry_safe(extent_op, tmp, &update_list, list) {
+ clear_extent_bits(&info->extent_ins, extent_op->bytenr,
+ extent_op->bytenr + extent_op->num_bytes - 1,
+ EXTENT_WRITEBACK, GFP_NOFS);
+ if (extent_op->del) {
+ list_del_init(&extent_op->list);
+ unlock_extent(&info->extent_ins, extent_op->bytenr,
+ extent_op->bytenr + extent_op->num_bytes
+ - 1, GFP_NOFS);
+ kfree(extent_op);
+ }
+ }
+ mutex_unlock(&info->extent_ins_mutex);
+
+ /*
+ * still have things left on the update list, go ahead an update
+ * everything
+ */
+ if (!list_empty(&update_list)) {
+ ret = update_backrefs(trans, extent_root, path, &update_list);
+ BUG_ON(ret);
+ }
+
+ /*
+ * if no inserts need to be done, but we skipped some extents and we
+ * need to make sure everything is cleaned then reset everything and
+ * go back to the beginning
+ */
+ if (!num_inserts && all && skipped) {
+ search = 0;
+ skipped = 0;
+ INIT_LIST_HEAD(&update_list);
+ INIT_LIST_HEAD(&insert_list);
+ goto again;
+ } else if (!num_inserts) {
+ goto out;
+ }
+
+ /*
+ * process the insert extents list. Again if we are deleting this
+ * extent, then just unlock it, pin down the bytes if need be, and be
+ * done with it. Saves us from having to actually insert the extent
+ * into the tree and then subsequently come along and delete it
+ */
+ mutex_lock(&info->extent_ins_mutex);
+ list_for_each_entry_safe(extent_op, tmp, &insert_list, list) {
+ clear_extent_bits(&info->extent_ins, extent_op->bytenr,
+ extent_op->bytenr + extent_op->num_bytes - 1,
+ EXTENT_WRITEBACK, GFP_NOFS);
+ if (extent_op->del) {
+ u64 used;
+ list_del_init(&extent_op->list);
+ unlock_extent(&info->extent_ins, extent_op->bytenr,
+ extent_op->bytenr + extent_op->num_bytes
+ - 1, GFP_NOFS);
+
+ mutex_lock(&extent_root->fs_info->pinned_mutex);
+ ret = pin_down_bytes(trans, extent_root,
+ extent_op->bytenr,
+ extent_op->num_bytes, 0);
+ mutex_unlock(&extent_root->fs_info->pinned_mutex);
+
+ spin_lock(&info->delalloc_lock);
+ used = btrfs_super_bytes_used(&info->super_copy);
+ btrfs_set_super_bytes_used(&info->super_copy,
+ used - extent_op->num_bytes);
+ used = btrfs_root_used(&extent_root->root_item);
+ btrfs_set_root_used(&extent_root->root_item,
+ used - extent_op->num_bytes);
+ spin_unlock(&info->delalloc_lock);
+
+ ret = update_block_group(trans, extent_root,
+ extent_op->bytenr,
+ extent_op->num_bytes,
+ 0, ret > 0);
+ BUG_ON(ret);
+ kfree(extent_op);
+ num_inserts--;
+ }
+ }
+ mutex_unlock(&info->extent_ins_mutex);
+
+ ret = insert_extents(trans, extent_root, path, &insert_list,
+ num_inserts);
+ BUG_ON(ret);
+
+ /*
+ * if we broke out of the loop in order to insert stuff because we hit
+ * the maximum number of inserts at a time we can handle, then loop
+ * back and pick up where we left off
+ */
+ if (num_inserts == max_inserts) {
+ INIT_LIST_HEAD(&insert_list);
+ INIT_LIST_HEAD(&update_list);
+ num_inserts = 0;
+ goto again;
+ }
+
+ /*
+ * again, if we need to make absolutely sure there are no more pending
+ * extent operations left and we know that we skipped some, go back to
+ * the beginning and do it all again
+ */
+ if (all && skipped) {
+ INIT_LIST_HEAD(&insert_list);
+ INIT_LIST_HEAD(&update_list);
+ search = 0;
+ skipped = 0;
+ num_inserts = 0;
+ goto again;
+ }
+out:
+ btrfs_free_path(path);
+ return 0;
+}
+
+static int pin_down_bytes(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 bytenr, u64 num_bytes, int is_data)
+{
+ int err = 0;
+ struct extent_buffer *buf;
+
+ if (is_data)
+ goto pinit;
+
+ buf = btrfs_find_tree_block(root, bytenr, num_bytes);
+ if (!buf)
+ goto pinit;
+
+ /* we can reuse a block if it hasn't been written
+ * and it is from this transaction. We can't
+ * reuse anything from the tree log root because
+ * it has tiny sub-transactions.
+ */
+ if (btrfs_buffer_uptodate(buf, 0) &&
+ btrfs_try_tree_lock(buf)) {
+ u64 header_owner = btrfs_header_owner(buf);
+ u64 header_transid = btrfs_header_generation(buf);
+ if (header_owner != BTRFS_TREE_LOG_OBJECTID &&
+ header_owner != BTRFS_TREE_RELOC_OBJECTID &&
+ header_transid == trans->transid &&
+ !btrfs_header_flag(buf, BTRFS_HEADER_FLAG_WRITTEN)) {
+ clean_tree_block(NULL, root, buf);
+ btrfs_tree_unlock(buf);
+ free_extent_buffer(buf);
+ return 1;
+ }
+ btrfs_tree_unlock(buf);
+ }
+ free_extent_buffer(buf);
+pinit:
+ btrfs_update_pinned_extents(root, bytenr, num_bytes, 1);
+
+ BUG_ON(err < 0);
+ return 0;
+}
+
+/*
+ * remove an extent from the root, returns 0 on success
+ */
+static int __free_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 bytenr, u64 num_bytes, u64 parent,
+ u64 root_objectid, u64 ref_generation,
+ u64 owner_objectid, int pin, int mark_free)
+{
+ struct btrfs_path *path;
+ struct btrfs_key key;
+ struct btrfs_fs_info *info = root->fs_info;
+ struct btrfs_root *extent_root = info->extent_root;
+ struct extent_buffer *leaf;
+ int ret;
+ int extent_slot = 0;
+ int found_extent = 0;
+ int num_to_del = 1;
+ struct btrfs_extent_item *ei;
+ u32 refs;
+
+ key.objectid = bytenr;
+ btrfs_set_key_type(&key, BTRFS_EXTENT_ITEM_KEY);
+ key.offset = num_bytes;
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ path->reada = 1;
+ ret = lookup_extent_backref(trans, extent_root, path,
+ bytenr, parent, root_objectid,
+ ref_generation, owner_objectid, 1);
+ if (ret == 0) {
+ struct btrfs_key found_key;
+ extent_slot = path->slots[0];
+ while (extent_slot > 0) {
+ extent_slot--;
+ btrfs_item_key_to_cpu(path->nodes[0], &found_key,
+ extent_slot);
+ if (found_key.objectid != bytenr)
+ break;
+ if (found_key.type == BTRFS_EXTENT_ITEM_KEY &&
+ found_key.offset == num_bytes) {
+ found_extent = 1;
+ break;
+ }
+ if (path->slots[0] - extent_slot > 5)
+ break;
+ }
+ if (!found_extent) {
+ ret = remove_extent_backref(trans, extent_root, path);
+ BUG_ON(ret);
+ btrfs_release_path(extent_root, path);
+ ret = btrfs_search_slot(trans, extent_root,
+ &key, path, -1, 1);
+ if (ret) {
+ printk(KERN_ERR "umm, got %d back from search"
+ ", was looking for %llu\n", ret,
+ (unsigned long long)bytenr);
+ btrfs_print_leaf(extent_root, path->nodes[0]);
+ }
+ BUG_ON(ret);
+ extent_slot = path->slots[0];
+ }
+ } else {
+ btrfs_print_leaf(extent_root, path->nodes[0]);
+ WARN_ON(1);
+ printk(KERN_ERR "btrfs unable to find ref byte nr %llu "
+ "root %llu gen %llu owner %llu\n",
+ (unsigned long long)bytenr,
+ (unsigned long long)root_objectid,
+ (unsigned long long)ref_generation,
+ (unsigned long long)owner_objectid);
+ }
+
+ leaf = path->nodes[0];
+ ei = btrfs_item_ptr(leaf, extent_slot,
+ struct btrfs_extent_item);
+ refs = btrfs_extent_refs(leaf, ei);
+ BUG_ON(refs == 0);
+ refs -= 1;
+ btrfs_set_extent_refs(leaf, ei, refs);
+
+ btrfs_mark_buffer_dirty(leaf);
+
+ if (refs == 0 && found_extent && path->slots[0] == extent_slot + 1) {
+ struct btrfs_extent_ref *ref;
+ ref = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_extent_ref);
+ BUG_ON(btrfs_ref_num_refs(leaf, ref) != 1);
+ /* if the back ref and the extent are next to each other
+ * they get deleted below in one shot
+ */
+ path->slots[0] = extent_slot;
+ num_to_del = 2;
+ } else if (found_extent) {
+ /* otherwise delete the extent back ref */
+ ret = remove_extent_backref(trans, extent_root, path);
+ BUG_ON(ret);
+ /* if refs are 0, we need to setup the path for deletion */
+ if (refs == 0) {
+ btrfs_release_path(extent_root, path);
+ ret = btrfs_search_slot(trans, extent_root, &key, path,
+ -1, 1);
+ BUG_ON(ret);
+ }
+ }
+
+ if (refs == 0) {
+ u64 super_used;
+ u64 root_used;
+
+ if (pin) {
+ mutex_lock(&root->fs_info->pinned_mutex);
+ ret = pin_down_bytes(trans, root, bytenr, num_bytes,
+ owner_objectid >= BTRFS_FIRST_FREE_OBJECTID);
+ mutex_unlock(&root->fs_info->pinned_mutex);
+ if (ret > 0)
+ mark_free = 1;
+ BUG_ON(ret < 0);
+ }
+ /* block accounting for super block */
+ spin_lock(&info->delalloc_lock);
+ super_used = btrfs_super_bytes_used(&info->super_copy);
+ btrfs_set_super_bytes_used(&info->super_copy,
+ super_used - num_bytes);
+
+ /* block accounting for root item */
+ root_used = btrfs_root_used(&root->root_item);
+ btrfs_set_root_used(&root->root_item,
+ root_used - num_bytes);
+ spin_unlock(&info->delalloc_lock);
+ ret = btrfs_del_items(trans, extent_root, path, path->slots[0],
+ num_to_del);
+ BUG_ON(ret);
+ btrfs_release_path(extent_root, path);
+
+ if (owner_objectid >= BTRFS_FIRST_FREE_OBJECTID) {
+ ret = btrfs_del_csums(trans, root, bytenr, num_bytes);
+ BUG_ON(ret);
+ }
+
+ ret = update_block_group(trans, root, bytenr, num_bytes, 0,
+ mark_free);
+ BUG_ON(ret);
+ }
+ btrfs_free_path(path);
+ finish_current_insert(trans, extent_root, 0);
+ return ret;
+}
+
+/*
+ * find all the blocks marked as pending in the radix tree and remove
+ * them from the extent map
+ */
+static int del_pending_extents(struct btrfs_trans_handle *trans,
+ struct btrfs_root *extent_root, int all)
+{
+ int ret;
+ int err = 0;
+ u64 start;
+ u64 end;
+ u64 priv;
+ u64 search = 0;
+ int nr = 0, skipped = 0;
+ struct extent_io_tree *pending_del;
+ struct extent_io_tree *extent_ins;
+ struct pending_extent_op *extent_op;
+ struct btrfs_fs_info *info = extent_root->fs_info;
+ struct list_head delete_list;
+
+ INIT_LIST_HEAD(&delete_list);
+ extent_ins = &extent_root->fs_info->extent_ins;
+ pending_del = &extent_root->fs_info->pending_del;
+
+again:
+ mutex_lock(&info->extent_ins_mutex);
+ while (1) {
+ ret = find_first_extent_bit(pending_del, search, &start, &end,
+ EXTENT_WRITEBACK);
+ if (ret) {
+ if (all && skipped && !nr) {
+ search = 0;
+ continue;
+ }
+ mutex_unlock(&info->extent_ins_mutex);
+ break;
+ }
+
+ ret = try_lock_extent(extent_ins, start, end, GFP_NOFS);
+ if (!ret) {
+ search = end+1;
+ skipped = 1;
+
+ if (need_resched()) {
+ mutex_unlock(&info->extent_ins_mutex);
+ cond_resched();
+ mutex_lock(&info->extent_ins_mutex);
+ }
+
+ continue;
+ }
+ BUG_ON(ret < 0);
+
+ ret = get_state_private(pending_del, start, &priv);
+ BUG_ON(ret);
+ extent_op = (struct pending_extent_op *)(unsigned long)priv;
+
+ clear_extent_bits(pending_del, start, end, EXTENT_WRITEBACK,
+ GFP_NOFS);
+ if (!test_range_bit(extent_ins, start, end,
+ EXTENT_WRITEBACK, 0)) {
+ list_add_tail(&extent_op->list, &delete_list);
+ nr++;
+ } else {
+ kfree(extent_op);
+
+ ret = get_state_private(&info->extent_ins, start,
+ &priv);
+ BUG_ON(ret);
+ extent_op = (struct pending_extent_op *)
+ (unsigned long)priv;
+
+ clear_extent_bits(&info->extent_ins, start, end,
+ EXTENT_WRITEBACK, GFP_NOFS);
+
+ if (extent_op->type == PENDING_BACKREF_UPDATE) {
+ list_add_tail(&extent_op->list, &delete_list);
+ search = end + 1;
+ nr++;
+ continue;
+ }
+
+ mutex_lock(&extent_root->fs_info->pinned_mutex);
+ ret = pin_down_bytes(trans, extent_root, start,
+ end + 1 - start, 0);
+ mutex_unlock(&extent_root->fs_info->pinned_mutex);
+
+ ret = update_block_group(trans, extent_root, start,
+ end + 1 - start, 0, ret > 0);
+
+ unlock_extent(extent_ins, start, end, GFP_NOFS);
+ BUG_ON(ret);
+ kfree(extent_op);
+ }
+ if (ret)
+ err = ret;
+
+ search = end + 1;
+
+ if (need_resched()) {
+ mutex_unlock(&info->extent_ins_mutex);
+ cond_resched();
+ mutex_lock(&info->extent_ins_mutex);
+ }
+ }
+
+ if (nr) {
+ ret = free_extents(trans, extent_root, &delete_list);
+ BUG_ON(ret);
+ }
+
+ if (all && skipped) {
+ INIT_LIST_HEAD(&delete_list);
+ search = 0;
+ nr = 0;
+ goto again;
+ }
+
+ return err;
+}
+
+/*
+ * remove an extent from the root, returns 0 on success
+ */
+static int __btrfs_free_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 bytenr, u64 num_bytes, u64 parent,
+ u64 root_objectid, u64 ref_generation,
+ u64 owner_objectid, int pin)
+{
+ struct btrfs_root *extent_root = root->fs_info->extent_root;
+ int pending_ret;
+ int ret;
+
+ WARN_ON(num_bytes < root->sectorsize);
+ if (root == extent_root) {
+ struct pending_extent_op *extent_op = NULL;
+
+ mutex_lock(&root->fs_info->extent_ins_mutex);
+ if (test_range_bit(&root->fs_info->extent_ins, bytenr,
+ bytenr + num_bytes - 1, EXTENT_WRITEBACK, 0)) {
+ u64 priv;
+ ret = get_state_private(&root->fs_info->extent_ins,
+ bytenr, &priv);
+ BUG_ON(ret);
+ extent_op = (struct pending_extent_op *)
+ (unsigned long)priv;
+
+ extent_op->del = 1;
+ if (extent_op->type == PENDING_EXTENT_INSERT) {
+ mutex_unlock(&root->fs_info->extent_ins_mutex);
+ return 0;
+ }
+ }
+
+ if (extent_op) {
+ ref_generation = extent_op->orig_generation;
+ parent = extent_op->orig_parent;
+ }
+
+ extent_op = kmalloc(sizeof(*extent_op), GFP_NOFS);
+ BUG_ON(!extent_op);
+
+ extent_op->type = PENDING_EXTENT_DELETE;
+ extent_op->bytenr = bytenr;
+ extent_op->num_bytes = num_bytes;
+ extent_op->parent = parent;
+ extent_op->orig_parent = parent;
+ extent_op->generation = ref_generation;
+ extent_op->orig_generation = ref_generation;
+ extent_op->level = (int)owner_objectid;
+ INIT_LIST_HEAD(&extent_op->list);
+ extent_op->del = 0;
+
+ set_extent_bits(&root->fs_info->pending_del,
+ bytenr, bytenr + num_bytes - 1,
+ EXTENT_WRITEBACK, GFP_NOFS);
+ set_state_private(&root->fs_info->pending_del,
+ bytenr, (unsigned long)extent_op);
+ mutex_unlock(&root->fs_info->extent_ins_mutex);
+ return 0;
+ }
+ /* if metadata always pin */
+ if (owner_objectid < BTRFS_FIRST_FREE_OBJECTID) {
+ if (root->root_key.objectid == BTRFS_TREE_LOG_OBJECTID) {
+ struct btrfs_block_group_cache *cache;
+
+ /* btrfs_free_reserved_extent */
+ cache = btrfs_lookup_block_group(root->fs_info, bytenr);
+ BUG_ON(!cache);
+ btrfs_add_free_space(cache, bytenr, num_bytes);
+ put_block_group(cache);
+ update_reserved_extents(root, bytenr, num_bytes, 0);
+ return 0;
+ }
+ pin = 1;
+ }
+
+ /* if data pin when any transaction has committed this */
+ if (ref_generation != trans->transid)
+ pin = 1;
+
+ ret = __free_extent(trans, root, bytenr, num_bytes, parent,
+ root_objectid, ref_generation,
+ owner_objectid, pin, pin == 0);
+
+ finish_current_insert(trans, root->fs_info->extent_root, 0);
+ pending_ret = del_pending_extents(trans, root->fs_info->extent_root, 0);
+ return ret ? ret : pending_ret;
+}
+
+int btrfs_free_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 bytenr, u64 num_bytes, u64 parent,
+ u64 root_objectid, u64 ref_generation,
+ u64 owner_objectid, int pin)
+{
+ int ret;
+
+ ret = __btrfs_free_extent(trans, root, bytenr, num_bytes, parent,
+ root_objectid, ref_generation,
+ owner_objectid, pin);
+ return ret;
+}
+
+static u64 stripe_align(struct btrfs_root *root, u64 val)
+{
+ u64 mask = ((u64)root->stripesize - 1);
+ u64 ret = (val + mask) & ~mask;
+ return ret;
+}
+
+/*
+ * walks the btree of allocated extents and find a hole of a given size.
+ * The key ins is changed to record the hole:
+ * ins->objectid == block start
+ * ins->flags = BTRFS_EXTENT_ITEM_KEY
+ * ins->offset == number of blocks
+ * Any available blocks before search_start are skipped.
+ */
+static noinline int find_free_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *orig_root,
+ u64 num_bytes, u64 empty_size,
+ u64 search_start, u64 search_end,
+ u64 hint_byte, struct btrfs_key *ins,
+ u64 exclude_start, u64 exclude_nr,
+ int data)
+{
+ int ret = 0;
+ struct btrfs_root *root = orig_root->fs_info->extent_root;
+ u64 total_needed = num_bytes;
+ u64 *last_ptr = NULL;
+ u64 last_wanted = 0;
+ struct btrfs_block_group_cache *block_group = NULL;
+ int chunk_alloc_done = 0;
+ int empty_cluster = 2 * 1024 * 1024;
+ int allowed_chunk_alloc = 0;
+ struct list_head *head = NULL, *cur = NULL;
+ int loop = 0;
+ int extra_loop = 0;
+ struct btrfs_space_info *space_info;
+
+ WARN_ON(num_bytes < root->sectorsize);
+ btrfs_set_key_type(ins, BTRFS_EXTENT_ITEM_KEY);
+ ins->objectid = 0;
+ ins->offset = 0;
+
+ if (orig_root->ref_cows || empty_size)
+ allowed_chunk_alloc = 1;
+
+ if (data & BTRFS_BLOCK_GROUP_METADATA) {
+ last_ptr = &root->fs_info->last_alloc;
+ empty_cluster = 64 * 1024;
+ }
+
+ if ((data & BTRFS_BLOCK_GROUP_DATA) && btrfs_test_opt(root, SSD))
+ last_ptr = &root->fs_info->last_data_alloc;
+
+ if (last_ptr) {
+ if (*last_ptr) {
+ hint_byte = *last_ptr;
+ last_wanted = *last_ptr;
+ } else
+ empty_size += empty_cluster;
+ } else {
+ empty_cluster = 0;
+ }
+ search_start = max(search_start, first_logical_byte(root, 0));
+ search_start = max(search_start, hint_byte);
+
+ if (last_wanted && search_start != last_wanted) {
+ last_wanted = 0;
+ empty_size += empty_cluster;
+ }
+
+ total_needed += empty_size;
+ block_group = btrfs_lookup_block_group(root->fs_info, search_start);
+ if (!block_group)
+ block_group = btrfs_lookup_first_block_group(root->fs_info,
+ search_start);
+ space_info = __find_space_info(root->fs_info, data);
+
+ down_read(&space_info->groups_sem);
+ while (1) {
+ struct btrfs_free_space *free_space;
+ /*
+ * the only way this happens if our hint points to a block
+ * group thats not of the proper type, while looping this
+ * should never happen
+ */
+ if (empty_size)
+ extra_loop = 1;
+
+ if (!block_group)
+ goto new_group_no_lock;
+
+ if (unlikely(!block_group->cached)) {
+ mutex_lock(&block_group->cache_mutex);
+ ret = cache_block_group(root, block_group);
+ mutex_unlock(&block_group->cache_mutex);
+ if (ret)
+ break;
+ }
+
+ mutex_lock(&block_group->alloc_mutex);
+ if (unlikely(!block_group_bits(block_group, data)))
+ goto new_group;
+
+ if (unlikely(block_group->ro))
+ goto new_group;
+
+ free_space = btrfs_find_free_space(block_group, search_start,
+ total_needed);
+ if (free_space) {
+ u64 start = block_group->key.objectid;
+ u64 end = block_group->key.objectid +
+ block_group->key.offset;
+
+ search_start = stripe_align(root, free_space->offset);
+
+ /* move on to the next group */
+ if (search_start + num_bytes >= search_end)
+ goto new_group;
+
+ /* move on to the next group */
+ if (search_start + num_bytes > end)
+ goto new_group;
+
+ if (last_wanted && search_start != last_wanted) {
+ total_needed += empty_cluster;
+ empty_size += empty_cluster;
+ last_wanted = 0;
+ /*
+ * if search_start is still in this block group
+ * then we just re-search this block group
+ */
+ if (search_start >= start &&
+ search_start < end) {
+ mutex_unlock(&block_group->alloc_mutex);
+ continue;
+ }
+
+ /* else we go to the next block group */
+ goto new_group;
+ }
+
+ if (exclude_nr > 0 &&
+ (search_start + num_bytes > exclude_start &&
+ search_start < exclude_start + exclude_nr)) {
+ search_start = exclude_start + exclude_nr;
+ /*
+ * if search_start is still in this block group
+ * then we just re-search this block group
+ */
+ if (search_start >= start &&
+ search_start < end) {
+ mutex_unlock(&block_group->alloc_mutex);
+ last_wanted = 0;
+ continue;
+ }
+
+ /* else we go to the next block group */
+ goto new_group;
+ }
+
+ ins->objectid = search_start;
+ ins->offset = num_bytes;
+
+ btrfs_remove_free_space_lock(block_group, search_start,
+ num_bytes);
+ /* we are all good, lets return */
+ mutex_unlock(&block_group->alloc_mutex);
+ break;
+ }
+new_group:
+ mutex_unlock(&block_group->alloc_mutex);
+ put_block_group(block_group);
+ block_group = NULL;
+new_group_no_lock:
+ /* don't try to compare new allocations against the
+ * last allocation any more
+ */
+ last_wanted = 0;
+
+ /*
+ * Here's how this works.
+ * loop == 0: we were searching a block group via a hint
+ * and didn't find anything, so we start at
+ * the head of the block groups and keep searching
+ * loop == 1: we're searching through all of the block groups
+ * if we hit the head again we have searched
+ * all of the block groups for this space and we
+ * need to try and allocate, if we cant error out.
+ * loop == 2: we allocated more space and are looping through
+ * all of the block groups again.
+ */
+ if (loop == 0) {
+ head = &space_info->block_groups;
+ cur = head->next;
+ loop++;
+ } else if (loop == 1 && cur == head) {
+ int keep_going;
+
+ /* at this point we give up on the empty_size
+ * allocations and just try to allocate the min
+ * space.
+ *
+ * The extra_loop field was set if an empty_size
+ * allocation was attempted above, and if this
+ * is try we need to try the loop again without
+ * the additional empty_size.
+ */
+ total_needed -= empty_size;
+ empty_size = 0;
+ keep_going = extra_loop;
+ loop++;
+
+ if (allowed_chunk_alloc && !chunk_alloc_done) {
+ up_read(&space_info->groups_sem);
+ ret = do_chunk_alloc(trans, root, num_bytes +
+ 2 * 1024 * 1024, data, 1);
+ down_read(&space_info->groups_sem);
+ if (ret < 0)
+ goto loop_check;
+ head = &space_info->block_groups;
+ /*
+ * we've allocated a new chunk, keep
+ * trying
+ */
+ keep_going = 1;
+ chunk_alloc_done = 1;
+ } else if (!allowed_chunk_alloc) {
+ space_info->force_alloc = 1;
+ }
+loop_check:
+ if (keep_going) {
+ cur = head->next;
+ extra_loop = 0;
+ } else {
+ break;
+ }
+ } else if (cur == head) {
+ break;
+ }
+
+ block_group = list_entry(cur, struct btrfs_block_group_cache,
+ list);
+ atomic_inc(&block_group->count);
+
+ search_start = block_group->key.objectid;
+ cur = cur->next;
+ }
+
+ /* we found what we needed */
+ if (ins->objectid) {
+ if (!(data & BTRFS_BLOCK_GROUP_DATA))
+ trans->block_group = block_group->key.objectid;
+
+ if (last_ptr)
+ *last_ptr = ins->objectid + ins->offset;
+ ret = 0;
+ } else if (!ret) {
+ printk(KERN_ERR "btrfs searching for %llu bytes, "
+ "num_bytes %llu, loop %d, allowed_alloc %d\n",
+ (unsigned long long)total_needed,
+ (unsigned long long)num_bytes,
+ loop, allowed_chunk_alloc);
+ ret = -ENOSPC;
+ }
+ if (block_group)
+ put_block_group(block_group);
+
+ up_read(&space_info->groups_sem);
+ return ret;
+}
+
+static void dump_space_info(struct btrfs_space_info *info, u64 bytes)
+{
+ struct btrfs_block_group_cache *cache;
+ struct list_head *l;
+
+ printk(KERN_INFO "space_info has %llu free, is %sfull\n",
+ (unsigned long long)(info->total_bytes - info->bytes_used -
+ info->bytes_pinned - info->bytes_reserved),
+ (info->full) ? "" : "not ");
+
+ down_read(&info->groups_sem);
+ list_for_each(l, &info->block_groups) {
+ cache = list_entry(l, struct btrfs_block_group_cache, list);
+ spin_lock(&cache->lock);
+ printk(KERN_INFO "block group %llu has %llu bytes, %llu used "
+ "%llu pinned %llu reserved\n",
+ (unsigned long long)cache->key.objectid,
+ (unsigned long long)cache->key.offset,
+ (unsigned long long)btrfs_block_group_used(&cache->item),
+ (unsigned long long)cache->pinned,
+ (unsigned long long)cache->reserved);
+ btrfs_dump_free_space(cache, bytes);
+ spin_unlock(&cache->lock);
+ }
+ up_read(&info->groups_sem);
+}
+
+static int __btrfs_reserve_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 num_bytes, u64 min_alloc_size,
+ u64 empty_size, u64 hint_byte,
+ u64 search_end, struct btrfs_key *ins,
+ u64 data)
+{
+ int ret;
+ u64 search_start = 0;
+ u64 alloc_profile;
+ struct btrfs_fs_info *info = root->fs_info;
+
+ if (data) {
+ alloc_profile = info->avail_data_alloc_bits &
+ info->data_alloc_profile;
+ data = BTRFS_BLOCK_GROUP_DATA | alloc_profile;
+ } else if (root == root->fs_info->chunk_root) {
+ alloc_profile = info->avail_system_alloc_bits &
+ info->system_alloc_profile;
+ data = BTRFS_BLOCK_GROUP_SYSTEM | alloc_profile;
+ } else {
+ alloc_profile = info->avail_metadata_alloc_bits &
+ info->metadata_alloc_profile;
+ data = BTRFS_BLOCK_GROUP_METADATA | alloc_profile;
+ }
+again:
+ data = btrfs_reduce_alloc_profile(root, data);
+ /*
+ * the only place that sets empty_size is btrfs_realloc_node, which
+ * is not called recursively on allocations
+ */
+ if (empty_size || root->ref_cows) {
+ if (!(data & BTRFS_BLOCK_GROUP_METADATA)) {
+ ret = do_chunk_alloc(trans, root->fs_info->extent_root,
+ 2 * 1024 * 1024,
+ BTRFS_BLOCK_GROUP_METADATA |
+ (info->metadata_alloc_profile &
+ info->avail_metadata_alloc_bits), 0);
+ }
+ ret = do_chunk_alloc(trans, root->fs_info->extent_root,
+ num_bytes + 2 * 1024 * 1024, data, 0);
+ }
+
+ WARN_ON(num_bytes < root->sectorsize);
+ ret = find_free_extent(trans, root, num_bytes, empty_size,
+ search_start, search_end, hint_byte, ins,
+ trans->alloc_exclude_start,
+ trans->alloc_exclude_nr, data);
+
+ if (ret == -ENOSPC && num_bytes > min_alloc_size) {
+ num_bytes = num_bytes >> 1;
+ num_bytes = num_bytes & ~(root->sectorsize - 1);
+ num_bytes = max(num_bytes, min_alloc_size);
+ do_chunk_alloc(trans, root->fs_info->extent_root,
+ num_bytes, data, 1);
+ goto again;
+ }
+ if (ret) {
+ struct btrfs_space_info *sinfo;
+
+ sinfo = __find_space_info(root->fs_info, data);
+ printk(KERN_ERR "btrfs allocation failed flags %llu, "
+ "wanted %llu\n", (unsigned long long)data,
+ (unsigned long long)num_bytes);
+ dump_space_info(sinfo, num_bytes);
+ BUG();
+ }
+
+ return ret;
+}
+
+int btrfs_free_reserved_extent(struct btrfs_root *root, u64 start, u64 len)
+{
+ struct btrfs_block_group_cache *cache;
+ int ret = 0;
+
+ cache = btrfs_lookup_block_group(root->fs_info, start);
+ if (!cache) {
+ printk(KERN_ERR "Unable to find block group for %llu\n",
+ (unsigned long long)start);
+ return -ENOSPC;
+ }
+
+ ret = btrfs_discard_extent(root, start, len);
+
+ btrfs_add_free_space(cache, start, len);
+ put_block_group(cache);
+ update_reserved_extents(root, start, len, 0);
+
+ return ret;
+}
+
+int btrfs_reserve_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 num_bytes, u64 min_alloc_size,
+ u64 empty_size, u64 hint_byte,
+ u64 search_end, struct btrfs_key *ins,
+ u64 data)
+{
+ int ret;
+ ret = __btrfs_reserve_extent(trans, root, num_bytes, min_alloc_size,
+ empty_size, hint_byte, search_end, ins,
+ data);
+ update_reserved_extents(root, ins->objectid, ins->offset, 1);
+ return ret;
+}
+
+static int __btrfs_alloc_reserved_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 parent,
+ u64 root_objectid, u64 ref_generation,
+ u64 owner, struct btrfs_key *ins)
+{
+ int ret;
+ int pending_ret;
+ u64 super_used;
+ u64 root_used;
+ u64 num_bytes = ins->offset;
+ u32 sizes[2];
+ struct btrfs_fs_info *info = root->fs_info;
+ struct btrfs_root *extent_root = info->extent_root;
+ struct btrfs_extent_item *extent_item;
+ struct btrfs_extent_ref *ref;
+ struct btrfs_path *path;
+ struct btrfs_key keys[2];
+
+ if (parent == 0)
+ parent = ins->objectid;
+
+ /* block accounting for super block */
+ spin_lock(&info->delalloc_lock);
+ super_used = btrfs_super_bytes_used(&info->super_copy);
+ btrfs_set_super_bytes_used(&info->super_copy, super_used + num_bytes);
+
+ /* block accounting for root item */
+ root_used = btrfs_root_used(&root->root_item);
+ btrfs_set_root_used(&root->root_item, root_used + num_bytes);
+ spin_unlock(&info->delalloc_lock);
+
+ if (root == extent_root) {
+ struct pending_extent_op *extent_op;
+
+ extent_op = kmalloc(sizeof(*extent_op), GFP_NOFS);
+ BUG_ON(!extent_op);
+
+ extent_op->type = PENDING_EXTENT_INSERT;
+ extent_op->bytenr = ins->objectid;
+ extent_op->num_bytes = ins->offset;
+ extent_op->parent = parent;
+ extent_op->orig_parent = 0;
+ extent_op->generation = ref_generation;
+ extent_op->orig_generation = 0;
+ extent_op->level = (int)owner;
+ INIT_LIST_HEAD(&extent_op->list);
+ extent_op->del = 0;
+
+ mutex_lock(&root->fs_info->extent_ins_mutex);
+ set_extent_bits(&root->fs_info->extent_ins, ins->objectid,
+ ins->objectid + ins->offset - 1,
+ EXTENT_WRITEBACK, GFP_NOFS);
+ set_state_private(&root->fs_info->extent_ins,
+ ins->objectid, (unsigned long)extent_op);
+ mutex_unlock(&root->fs_info->extent_ins_mutex);
+ goto update_block;
+ }
+
+ memcpy(&keys[0], ins, sizeof(*ins));
+ keys[1].objectid = ins->objectid;
+ keys[1].type = BTRFS_EXTENT_REF_KEY;
+ keys[1].offset = parent;
+ sizes[0] = sizeof(*extent_item);
+ sizes[1] = sizeof(*ref);
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+
+ ret = btrfs_insert_empty_items(trans, extent_root, path, keys,
+ sizes, 2);
+ BUG_ON(ret);
+
+ extent_item = btrfs_item_ptr(path->nodes[0], path->slots[0],
+ struct btrfs_extent_item);
+ btrfs_set_extent_refs(path->nodes[0], extent_item, 1);
+ ref = btrfs_item_ptr(path->nodes[0], path->slots[0] + 1,
+ struct btrfs_extent_ref);
+
+ btrfs_set_ref_root(path->nodes[0], ref, root_objectid);
+ btrfs_set_ref_generation(path->nodes[0], ref, ref_generation);
+ btrfs_set_ref_objectid(path->nodes[0], ref, owner);
+ btrfs_set_ref_num_refs(path->nodes[0], ref, 1);
+
+ btrfs_mark_buffer_dirty(path->nodes[0]);
+
+ trans->alloc_exclude_start = 0;
+ trans->alloc_exclude_nr = 0;
+ btrfs_free_path(path);
+ finish_current_insert(trans, extent_root, 0);
+ pending_ret = del_pending_extents(trans, extent_root, 0);
+
+ if (ret)
+ goto out;
+ if (pending_ret) {
+ ret = pending_ret;
+ goto out;
+ }
+
+update_block:
+ ret = update_block_group(trans, root, ins->objectid,
+ ins->offset, 1, 0);
+ if (ret) {
+ printk(KERN_ERR "btrfs update block group failed for %llu "
+ "%llu\n", (unsigned long long)ins->objectid,
+ (unsigned long long)ins->offset);
+ BUG();
+ }
+out:
+ return ret;
+}
+
+int btrfs_alloc_reserved_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 parent,
+ u64 root_objectid, u64 ref_generation,
+ u64 owner, struct btrfs_key *ins)
+{
+ int ret;
+
+ if (root_objectid == BTRFS_TREE_LOG_OBJECTID)
+ return 0;
+ ret = __btrfs_alloc_reserved_extent(trans, root, parent, root_objectid,
+ ref_generation, owner, ins);
+ update_reserved_extents(root, ins->objectid, ins->offset, 0);
+ return ret;
+}
+
+/*
+ * this is used by the tree logging recovery code. It records that
+ * an extent has been allocated and makes sure to clear the free
+ * space cache bits as well
+ */
+int btrfs_alloc_logged_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 parent,
+ u64 root_objectid, u64 ref_generation,
+ u64 owner, struct btrfs_key *ins)
+{
+ int ret;
+ struct btrfs_block_group_cache *block_group;
+
+ block_group = btrfs_lookup_block_group(root->fs_info, ins->objectid);
+ mutex_lock(&block_group->cache_mutex);
+ cache_block_group(root, block_group);
+ mutex_unlock(&block_group->cache_mutex);
+
+ ret = btrfs_remove_free_space(block_group, ins->objectid,
+ ins->offset);
+ BUG_ON(ret);
+ put_block_group(block_group);
+ ret = __btrfs_alloc_reserved_extent(trans, root, parent, root_objectid,
+ ref_generation, owner, ins);
+ return ret;
+}
+
+/*
+ * finds a free extent and does all the dirty work required for allocation
+ * returns the key for the extent through ins, and a tree buffer for
+ * the first block of the extent through buf.
+ *
+ * returns 0 if everything worked, non-zero otherwise.
+ */
+int btrfs_alloc_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 num_bytes, u64 parent, u64 min_alloc_size,
+ u64 root_objectid, u64 ref_generation,
+ u64 owner_objectid, u64 empty_size, u64 hint_byte,
+ u64 search_end, struct btrfs_key *ins, u64 data)
+{
+ int ret;
+
+ ret = __btrfs_reserve_extent(trans, root, num_bytes,
+ min_alloc_size, empty_size, hint_byte,
+ search_end, ins, data);
+ BUG_ON(ret);
+ if (root_objectid != BTRFS_TREE_LOG_OBJECTID) {
+ ret = __btrfs_alloc_reserved_extent(trans, root, parent,
+ root_objectid, ref_generation,
+ owner_objectid, ins);
+ BUG_ON(ret);
+
+ } else {
+ update_reserved_extents(root, ins->objectid, ins->offset, 1);
+ }
+ return ret;
+}
+
+struct extent_buffer *btrfs_init_new_buffer(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 bytenr, u32 blocksize)
+{
+ struct extent_buffer *buf;
+
+ buf = btrfs_find_create_tree_block(root, bytenr, blocksize);
+ if (!buf)
+ return ERR_PTR(-ENOMEM);
+ btrfs_set_header_generation(buf, trans->transid);
+ btrfs_tree_lock(buf);
+ clean_tree_block(trans, root, buf);
+ btrfs_set_buffer_uptodate(buf);
+ if (root->root_key.objectid == BTRFS_TREE_LOG_OBJECTID) {
+ set_extent_dirty(&root->dirty_log_pages, buf->start,
+ buf->start + buf->len - 1, GFP_NOFS);
+ } else {
+ set_extent_dirty(&trans->transaction->dirty_pages, buf->start,
+ buf->start + buf->len - 1, GFP_NOFS);
+ }
+ trans->blocks_used++;
+ return buf;
+}
+
+/*
+ * helper function to allocate a block for a given tree
+ * returns the tree buffer or NULL.
+ */
+struct extent_buffer *btrfs_alloc_free_block(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u32 blocksize, u64 parent,
+ u64 root_objectid,
+ u64 ref_generation,
+ int level,
+ u64 hint,
+ u64 empty_size)
+{
+ struct btrfs_key ins;
+ int ret;
+ struct extent_buffer *buf;
+
+ ret = btrfs_alloc_extent(trans, root, blocksize, parent, blocksize,
+ root_objectid, ref_generation, level,
+ empty_size, hint, (u64)-1, &ins, 0);
+ if (ret) {
+ BUG_ON(ret > 0);
+ return ERR_PTR(ret);
+ }
+
+ buf = btrfs_init_new_buffer(trans, root, ins.objectid, blocksize);
+ return buf;
+}
+
+int btrfs_drop_leaf_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct extent_buffer *leaf)
+{
+ u64 leaf_owner;
+ u64 leaf_generation;
+ struct btrfs_key key;
+ struct btrfs_file_extent_item *fi;
+ int i;
+ int nritems;
+ int ret;
+
+ BUG_ON(!btrfs_is_leaf(leaf));
+ nritems = btrfs_header_nritems(leaf);
+ leaf_owner = btrfs_header_owner(leaf);
+ leaf_generation = btrfs_header_generation(leaf);
+
+ for (i = 0; i < nritems; i++) {
+ u64 disk_bytenr;
+ cond_resched();
+
+ btrfs_item_key_to_cpu(leaf, &key, i);
+ if (btrfs_key_type(&key) != BTRFS_EXTENT_DATA_KEY)
+ continue;
+ fi = btrfs_item_ptr(leaf, i, struct btrfs_file_extent_item);
+ if (btrfs_file_extent_type(leaf, fi) ==
+ BTRFS_FILE_EXTENT_INLINE)
+ continue;
+ /*
+ * FIXME make sure to insert a trans record that
+ * repeats the snapshot del on crash
+ */
+ disk_bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
+ if (disk_bytenr == 0)
+ continue;
+
+ ret = __btrfs_free_extent(trans, root, disk_bytenr,
+ btrfs_file_extent_disk_num_bytes(leaf, fi),
+ leaf->start, leaf_owner, leaf_generation,
+ key.objectid, 0);
+ BUG_ON(ret);
+
+ atomic_inc(&root->fs_info->throttle_gen);
+ wake_up(&root->fs_info->transaction_throttle);
+ cond_resched();
+ }
+ return 0;
+}
+
+static noinline int cache_drop_leaf_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_leaf_ref *ref)
+{
+ int i;
+ int ret;
+ struct btrfs_extent_info *info = ref->extents;
+
+ for (i = 0; i < ref->nritems; i++) {
+ ret = __btrfs_free_extent(trans, root, info->bytenr,
+ info->num_bytes, ref->bytenr,
+ ref->owner, ref->generation,
+ info->objectid, 0);
+
+ atomic_inc(&root->fs_info->throttle_gen);
+ wake_up(&root->fs_info->transaction_throttle);
+ cond_resched();
+
+ BUG_ON(ret);
+ info++;
+ }
+
+ return 0;
+}
+
+static int drop_snap_lookup_refcount(struct btrfs_root *root, u64 start,
+ u64 len, u32 *refs)
+{
+ int ret;
+
+ ret = btrfs_lookup_extent_ref(NULL, root, start, len, refs);
+ BUG_ON(ret);
+
+#if 0 /* some debugging code in case we see problems here */
+ /* if the refs count is one, it won't get increased again. But
+ * if the ref count is > 1, someone may be decreasing it at
+ * the same time we are.
+ */
+ if (*refs != 1) {
+ struct extent_buffer *eb = NULL;
+ eb = btrfs_find_create_tree_block(root, start, len);
+ if (eb)
+ btrfs_tree_lock(eb);
+
+ mutex_lock(&root->fs_info->alloc_mutex);
+ ret = lookup_extent_ref(NULL, root, start, len, refs);
+ BUG_ON(ret);
+ mutex_unlock(&root->fs_info->alloc_mutex);
+
+ if (eb) {
+ btrfs_tree_unlock(eb);
+ free_extent_buffer(eb);
+ }
+ if (*refs == 1) {
+ printk(KERN_ERR "btrfs block %llu went down to one "
+ "during drop_snap\n", (unsigned long long)start);
+ }
+
+ }
+#endif
+
+ cond_resched();
+ return ret;
+}
+
+/*
+ * helper function for drop_snapshot, this walks down the tree dropping ref
+ * counts as it goes.
+ */
+static noinline int walk_down_tree(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, int *level)
+{
+ u64 root_owner;
+ u64 root_gen;
+ u64 bytenr;
+ u64 ptr_gen;
+ struct extent_buffer *next;
+ struct extent_buffer *cur;
+ struct extent_buffer *parent;
+ struct btrfs_leaf_ref *ref;
+ u32 blocksize;
+ int ret;
+ u32 refs;
+
+ WARN_ON(*level < 0);
+ WARN_ON(*level >= BTRFS_MAX_LEVEL);
+ ret = drop_snap_lookup_refcount(root, path->nodes[*level]->start,
+ path->nodes[*level]->len, &refs);
+ BUG_ON(ret);
+ if (refs > 1)
+ goto out;
+
+ /*
+ * walk down to the last node level and free all the leaves
+ */
+ while (*level >= 0) {
+ WARN_ON(*level < 0);
+ WARN_ON(*level >= BTRFS_MAX_LEVEL);
+ cur = path->nodes[*level];
+
+ if (btrfs_header_level(cur) != *level)
+ WARN_ON(1);
+
+ if (path->slots[*level] >=
+ btrfs_header_nritems(cur))
+ break;
+ if (*level == 0) {
+ ret = btrfs_drop_leaf_ref(trans, root, cur);
+ BUG_ON(ret);
+ break;
+ }
+ bytenr = btrfs_node_blockptr(cur, path->slots[*level]);
+ ptr_gen = btrfs_node_ptr_generation(cur, path->slots[*level]);
+ blocksize = btrfs_level_size(root, *level - 1);
+
+ ret = drop_snap_lookup_refcount(root, bytenr, blocksize, &refs);
+ BUG_ON(ret);
+ if (refs != 1) {
+ parent = path->nodes[*level];
+ root_owner = btrfs_header_owner(parent);
+ root_gen = btrfs_header_generation(parent);
+ path->slots[*level]++;
+
+ ret = __btrfs_free_extent(trans, root, bytenr,
+ blocksize, parent->start,
+ root_owner, root_gen,
+ *level - 1, 1);
+ BUG_ON(ret);
+
+ atomic_inc(&root->fs_info->throttle_gen);
+ wake_up(&root->fs_info->transaction_throttle);
+ cond_resched();
+
+ continue;
+ }
+ /*
+ * at this point, we have a single ref, and since the
+ * only place referencing this extent is a dead root
+ * the reference count should never go higher.
+ * So, we don't need to check it again
+ */
+ if (*level == 1) {
+ ref = btrfs_lookup_leaf_ref(root, bytenr);
+ if (ref && ref->generation != ptr_gen) {
+ btrfs_free_leaf_ref(root, ref);
+ ref = NULL;
+ }
+ if (ref) {
+ ret = cache_drop_leaf_ref(trans, root, ref);
+ BUG_ON(ret);
+ btrfs_remove_leaf_ref(root, ref);
+ btrfs_free_leaf_ref(root, ref);
+ *level = 0;
+ break;
+ }
+ }
+ next = btrfs_find_tree_block(root, bytenr, blocksize);
+ if (!next || !btrfs_buffer_uptodate(next, ptr_gen)) {
+ free_extent_buffer(next);
+
+ next = read_tree_block(root, bytenr, blocksize,
+ ptr_gen);
+ cond_resched();
+#if 0
+ /*
+ * this is a debugging check and can go away
+ * the ref should never go all the way down to 1
+ * at this point
+ */
+ ret = lookup_extent_ref(NULL, root, bytenr, blocksize,
+ &refs);
+ BUG_ON(ret);
+ WARN_ON(refs != 1);
+#endif
+ }
+ WARN_ON(*level <= 0);
+ if (path->nodes[*level-1])
+ free_extent_buffer(path->nodes[*level-1]);
+ path->nodes[*level-1] = next;
+ *level = btrfs_header_level(next);
+ path->slots[*level] = 0;
+ cond_resched();
+ }
+out:
+ WARN_ON(*level < 0);
+ WARN_ON(*level >= BTRFS_MAX_LEVEL);
+
+ if (path->nodes[*level] == root->node) {
+ parent = path->nodes[*level];
+ bytenr = path->nodes[*level]->start;
+ } else {
+ parent = path->nodes[*level + 1];
+ bytenr = btrfs_node_blockptr(parent, path->slots[*level + 1]);
+ }
+
+ blocksize = btrfs_level_size(root, *level);
+ root_owner = btrfs_header_owner(parent);
+ root_gen = btrfs_header_generation(parent);
+
+ ret = __btrfs_free_extent(trans, root, bytenr, blocksize,
+ parent->start, root_owner, root_gen,
+ *level, 1);
+ free_extent_buffer(path->nodes[*level]);
+ path->nodes[*level] = NULL;
+ *level += 1;
+ BUG_ON(ret);
+
+ cond_resched();
+ return 0;
+}
+
+/*
+ * helper function for drop_subtree, this function is similar to
+ * walk_down_tree. The main difference is that it checks reference
+ * counts while tree blocks are locked.
+ */
+static noinline int walk_down_subtree(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, int *level)
+{
+ struct extent_buffer *next;
+ struct extent_buffer *cur;
+ struct extent_buffer *parent;
+ u64 bytenr;
+ u64 ptr_gen;
+ u32 blocksize;
+ u32 refs;
+ int ret;
+
+ cur = path->nodes[*level];
+ ret = btrfs_lookup_extent_ref(trans, root, cur->start, cur->len,
+ &refs);
+ BUG_ON(ret);
+ if (refs > 1)
+ goto out;
+
+ while (*level >= 0) {
+ cur = path->nodes[*level];
+ if (*level == 0) {
+ ret = btrfs_drop_leaf_ref(trans, root, cur);
+ BUG_ON(ret);
+ clean_tree_block(trans, root, cur);
+ break;
+ }
+ if (path->slots[*level] >= btrfs_header_nritems(cur)) {
+ clean_tree_block(trans, root, cur);
+ break;
+ }
+
+ bytenr = btrfs_node_blockptr(cur, path->slots[*level]);
+ blocksize = btrfs_level_size(root, *level - 1);
+ ptr_gen = btrfs_node_ptr_generation(cur, path->slots[*level]);
+
+ next = read_tree_block(root, bytenr, blocksize, ptr_gen);
+ btrfs_tree_lock(next);
+
+ ret = btrfs_lookup_extent_ref(trans, root, bytenr, blocksize,
+ &refs);
+ BUG_ON(ret);
+ if (refs > 1) {
+ parent = path->nodes[*level];
+ ret = btrfs_free_extent(trans, root, bytenr,
+ blocksize, parent->start,
+ btrfs_header_owner(parent),
+ btrfs_header_generation(parent),
+ *level - 1, 1);
+ BUG_ON(ret);
+ path->slots[*level]++;
+ btrfs_tree_unlock(next);
+ free_extent_buffer(next);
+ continue;
+ }
+
+ *level = btrfs_header_level(next);
+ path->nodes[*level] = next;
+ path->slots[*level] = 0;
+ path->locks[*level] = 1;
+ cond_resched();
+ }
+out:
+ parent = path->nodes[*level + 1];
+ bytenr = path->nodes[*level]->start;
+ blocksize = path->nodes[*level]->len;
+
+ ret = btrfs_free_extent(trans, root, bytenr, blocksize,
+ parent->start, btrfs_header_owner(parent),
+ btrfs_header_generation(parent), *level, 1);
+ BUG_ON(ret);
+
+ if (path->locks[*level]) {
+ btrfs_tree_unlock(path->nodes[*level]);
+ path->locks[*level] = 0;
+ }
+ free_extent_buffer(path->nodes[*level]);
+ path->nodes[*level] = NULL;
+ *level += 1;
+ cond_resched();
+ return 0;
+}
+
+/*
+ * helper for dropping snapshots. This walks back up the tree in the path
+ * to find the first node higher up where we haven't yet gone through
+ * all the slots
+ */
+static noinline int walk_up_tree(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ int *level, int max_level)
+{
+ u64 root_owner;
+ u64 root_gen;
+ struct btrfs_root_item *root_item = &root->root_item;
+ int i;
+ int slot;
+ int ret;
+
+ for (i = *level; i < max_level && path->nodes[i]; i++) {
+ slot = path->slots[i];
+ if (slot < btrfs_header_nritems(path->nodes[i]) - 1) {
+ struct extent_buffer *node;
+ struct btrfs_disk_key disk_key;
+ node = path->nodes[i];
+ path->slots[i]++;
+ *level = i;
+ WARN_ON(*level == 0);
+ btrfs_node_key(node, &disk_key, path->slots[i]);
+ memcpy(&root_item->drop_progress,
+ &disk_key, sizeof(disk_key));
+ root_item->drop_level = i;
+ return 0;
+ } else {
+ struct extent_buffer *parent;
+ if (path->nodes[*level] == root->node)
+ parent = path->nodes[*level];
+ else
+ parent = path->nodes[*level + 1];
+
+ root_owner = btrfs_header_owner(parent);
+ root_gen = btrfs_header_generation(parent);
+
+ clean_tree_block(trans, root, path->nodes[*level]);
+ ret = btrfs_free_extent(trans, root,
+ path->nodes[*level]->start,
+ path->nodes[*level]->len,
+ parent->start, root_owner,
+ root_gen, *level, 1);
+ BUG_ON(ret);
+ if (path->locks[*level]) {
+ btrfs_tree_unlock(path->nodes[*level]);
+ path->locks[*level] = 0;
+ }
+ free_extent_buffer(path->nodes[*level]);
+ path->nodes[*level] = NULL;
+ *level = i + 1;
+ }
+ }
+ return 1;
+}
+
+/*
+ * drop the reference count on the tree rooted at 'snap'. This traverses
+ * the tree freeing any blocks that have a ref count of zero after being
+ * decremented.
+ */
+int btrfs_drop_snapshot(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root)
+{
+ int ret = 0;
+ int wret;
+ int level;
+ struct btrfs_path *path;
+ int i;
+ int orig_level;
+ struct btrfs_root_item *root_item = &root->root_item;
+
+ WARN_ON(!mutex_is_locked(&root->fs_info->drop_mutex));
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+
+ level = btrfs_header_level(root->node);
+ orig_level = level;
+ if (btrfs_disk_key_objectid(&root_item->drop_progress) == 0) {
+ path->nodes[level] = root->node;
+ extent_buffer_get(root->node);
+ path->slots[level] = 0;
+ } else {
+ struct btrfs_key key;
+ struct btrfs_disk_key found_key;
+ struct extent_buffer *node;
+
+ btrfs_disk_key_to_cpu(&key, &root_item->drop_progress);
+ level = root_item->drop_level;
+ path->lowest_level = level;
+ wret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ if (wret < 0) {
+ ret = wret;
+ goto out;
+ }
+ node = path->nodes[level];
+ btrfs_node_key(node, &found_key, path->slots[level]);
+ WARN_ON(memcmp(&found_key, &root_item->drop_progress,
+ sizeof(found_key)));
+ /*
+ * unlock our path, this is safe because only this
+ * function is allowed to delete this snapshot
+ */
+ for (i = 0; i < BTRFS_MAX_LEVEL; i++) {
+ if (path->nodes[i] && path->locks[i]) {
+ path->locks[i] = 0;
+ btrfs_tree_unlock(path->nodes[i]);
+ }
+ }
+ }
+ while (1) {
+ wret = walk_down_tree(trans, root, path, &level);
+ if (wret > 0)
+ break;
+ if (wret < 0)
+ ret = wret;
+
+ wret = walk_up_tree(trans, root, path, &level,
+ BTRFS_MAX_LEVEL);
+ if (wret > 0)
+ break;
+ if (wret < 0)
+ ret = wret;
+ if (trans->transaction->in_commit) {
+ ret = -EAGAIN;
+ break;
+ }
+ atomic_inc(&root->fs_info->throttle_gen);
+ wake_up(&root->fs_info->transaction_throttle);
+ }
+ for (i = 0; i <= orig_level; i++) {
+ if (path->nodes[i]) {
+ free_extent_buffer(path->nodes[i]);
+ path->nodes[i] = NULL;
+ }
+ }
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
+int btrfs_drop_subtree(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct extent_buffer *node,
+ struct extent_buffer *parent)
+{
+ struct btrfs_path *path;
+ int level;
+ int parent_level;
+ int ret = 0;
+ int wret;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+
+ BUG_ON(!btrfs_tree_locked(parent));
+ parent_level = btrfs_header_level(parent);
+ extent_buffer_get(parent);
+ path->nodes[parent_level] = parent;
+ path->slots[parent_level] = btrfs_header_nritems(parent);
+
+ BUG_ON(!btrfs_tree_locked(node));
+ level = btrfs_header_level(node);
+ extent_buffer_get(node);
+ path->nodes[level] = node;
+ path->slots[level] = 0;
+
+ while (1) {
+ wret = walk_down_subtree(trans, root, path, &level);
+ if (wret < 0)
+ ret = wret;
+ if (wret != 0)
+ break;
+
+ wret = walk_up_tree(trans, root, path, &level, parent_level);
+ if (wret < 0)
+ ret = wret;
+ if (wret != 0)
+ break;
+ }
+
+ btrfs_free_path(path);
+ return ret;
+}
+
+static unsigned long calc_ra(unsigned long start, unsigned long last,
+ unsigned long nr)
+{
+ return min(last, start + nr - 1);
+}
+
+static noinline int relocate_inode_pages(struct inode *inode, u64 start,
+ u64 len)
+{
+ u64 page_start;
+ u64 page_end;
+ unsigned long first_index;
+ unsigned long last_index;
+ unsigned long i;
+ struct page *page;
+ struct extent_io_tree *io_tree = &BTRFS_I(inode)->io_tree;
+ struct file_ra_state *ra;
+ struct btrfs_ordered_extent *ordered;
+ unsigned int total_read = 0;
+ unsigned int total_dirty = 0;
+ int ret = 0;
+
+ ra = kzalloc(sizeof(*ra), GFP_NOFS);
+
+ mutex_lock(&inode->i_mutex);
+ first_index = start >> PAGE_CACHE_SHIFT;
+ last_index = (start + len - 1) >> PAGE_CACHE_SHIFT;
+
+ /* make sure the dirty trick played by the caller work */
+ ret = invalidate_inode_pages2_range(inode->i_mapping,
+ first_index, last_index);
+ if (ret)
+ goto out_unlock;
+
+ file_ra_state_init(ra, inode->i_mapping);
+
+ for (i = first_index ; i <= last_index; i++) {
+ if (total_read % ra->ra_pages == 0) {
+ btrfs_force_ra(inode->i_mapping, ra, NULL, i,
+ calc_ra(i, last_index, ra->ra_pages));
+ }
+ total_read++;
+again:
+ if (((u64)i << PAGE_CACHE_SHIFT) > i_size_read(inode))
+ BUG_ON(1);
+ page = grab_cache_page(inode->i_mapping, i);
+ if (!page) {
+ ret = -ENOMEM;
+ goto out_unlock;
+ }
+ if (!PageUptodate(page)) {
+ btrfs_readpage(NULL, page);
+ lock_page(page);
+ if (!PageUptodate(page)) {
+ unlock_page(page);
+ page_cache_release(page);
+ ret = -EIO;
+ goto out_unlock;
+ }
+ }
+ wait_on_page_writeback(page);
+
+ page_start = (u64)page->index << PAGE_CACHE_SHIFT;
+ page_end = page_start + PAGE_CACHE_SIZE - 1;
+ lock_extent(io_tree, page_start, page_end, GFP_NOFS);
+
+ ordered = btrfs_lookup_ordered_extent(inode, page_start);
+ if (ordered) {
+ unlock_extent(io_tree, page_start, page_end, GFP_NOFS);
+ unlock_page(page);
+ page_cache_release(page);
+ btrfs_start_ordered_extent(inode, ordered, 1);
+ btrfs_put_ordered_extent(ordered);
+ goto again;
+ }
+ set_page_extent_mapped(page);
+
+ if (i == first_index)
+ set_extent_bits(io_tree, page_start, page_end,
+ EXTENT_BOUNDARY, GFP_NOFS);
+ btrfs_set_extent_delalloc(inode, page_start, page_end);
+
+ set_page_dirty(page);
+ total_dirty++;
+
+ unlock_extent(io_tree, page_start, page_end, GFP_NOFS);
+ unlock_page(page);
+ page_cache_release(page);
+ }
+
+out_unlock:
+ kfree(ra);
+ mutex_unlock(&inode->i_mutex);
+ balance_dirty_pages_ratelimited_nr(inode->i_mapping, total_dirty);
+ return ret;
+}
+
+static noinline int relocate_data_extent(struct inode *reloc_inode,
+ struct btrfs_key *extent_key,
+ u64 offset)
+{
+ struct btrfs_root *root = BTRFS_I(reloc_inode)->root;
+ struct extent_map_tree *em_tree = &BTRFS_I(reloc_inode)->extent_tree;
+ struct extent_map *em;
+ u64 start = extent_key->objectid - offset;
+ u64 end = start + extent_key->offset - 1;
+
+ em = alloc_extent_map(GFP_NOFS);
+ BUG_ON(!em || IS_ERR(em));
+
+ em->start = start;
+ em->len = extent_key->offset;
+ em->block_len = extent_key->offset;
+ em->block_start = extent_key->objectid;
+ em->bdev = root->fs_info->fs_devices->latest_bdev;
+ set_bit(EXTENT_FLAG_PINNED, &em->flags);
+
+ /* setup extent map to cheat btrfs_readpage */
+ lock_extent(&BTRFS_I(reloc_inode)->io_tree, start, end, GFP_NOFS);
+ while (1) {
+ int ret;
+ spin_lock(&em_tree->lock);
+ ret = add_extent_mapping(em_tree, em);
+ spin_unlock(&em_tree->lock);
+ if (ret != -EEXIST) {
+ free_extent_map(em);
+ break;
+ }
+ btrfs_drop_extent_cache(reloc_inode, start, end, 0);
+ }
+ unlock_extent(&BTRFS_I(reloc_inode)->io_tree, start, end, GFP_NOFS);
+
+ return relocate_inode_pages(reloc_inode, start, extent_key->offset);
+}
+
+struct btrfs_ref_path {
+ u64 extent_start;
+ u64 nodes[BTRFS_MAX_LEVEL];
+ u64 root_objectid;
+ u64 root_generation;
+ u64 owner_objectid;
+ u32 num_refs;
+ int lowest_level;
+ int current_level;
+ int shared_level;
+
+ struct btrfs_key node_keys[BTRFS_MAX_LEVEL];
+ u64 new_nodes[BTRFS_MAX_LEVEL];
+};
+
+struct disk_extent {
+ u64 ram_bytes;
+ u64 disk_bytenr;
+ u64 disk_num_bytes;
+ u64 offset;
+ u64 num_bytes;
+ u8 compression;
+ u8 encryption;
+ u16 other_encoding;
+};
+
+static int is_cowonly_root(u64 root_objectid)
+{
+ if (root_objectid == BTRFS_ROOT_TREE_OBJECTID ||
+ root_objectid == BTRFS_EXTENT_TREE_OBJECTID ||
+ root_objectid == BTRFS_CHUNK_TREE_OBJECTID ||
+ root_objectid == BTRFS_DEV_TREE_OBJECTID ||
+ root_objectid == BTRFS_TREE_LOG_OBJECTID ||
+ root_objectid == BTRFS_CSUM_TREE_OBJECTID)
+ return 1;
+ return 0;
+}
+
+static noinline int __next_ref_path(struct btrfs_trans_handle *trans,
+ struct btrfs_root *extent_root,
+ struct btrfs_ref_path *ref_path,
+ int first_time)
+{
+ struct extent_buffer *leaf;
+ struct btrfs_path *path;
+ struct btrfs_extent_ref *ref;
+ struct btrfs_key key;
+ struct btrfs_key found_key;
+ u64 bytenr;
+ u32 nritems;
+ int level;
+ int ret = 1;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ if (first_time) {
+ ref_path->lowest_level = -1;
+ ref_path->current_level = -1;
+ ref_path->shared_level = -1;
+ goto walk_up;
+ }
+walk_down:
+ level = ref_path->current_level - 1;
+ while (level >= -1) {
+ u64 parent;
+ if (level < ref_path->lowest_level)
+ break;
+
+ if (level >= 0)
+ bytenr = ref_path->nodes[level];
+ else
+ bytenr = ref_path->extent_start;
+ BUG_ON(bytenr == 0);
+
+ parent = ref_path->nodes[level + 1];
+ ref_path->nodes[level + 1] = 0;
+ ref_path->current_level = level;
+ BUG_ON(parent == 0);
+
+ key.objectid = bytenr;
+ key.offset = parent + 1;
+ key.type = BTRFS_EXTENT_REF_KEY;
+
+ ret = btrfs_search_slot(trans, extent_root, &key, path, 0, 0);
+ if (ret < 0)
+ goto out;
+ BUG_ON(ret == 0);
+
+ leaf = path->nodes[0];
+ nritems = btrfs_header_nritems(leaf);
+ if (path->slots[0] >= nritems) {
+ ret = btrfs_next_leaf(extent_root, path);
+ if (ret < 0)
+ goto out;
+ if (ret > 0)
+ goto next;
+ leaf = path->nodes[0];
+ }
+
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+ if (found_key.objectid == bytenr &&
+ found_key.type == BTRFS_EXTENT_REF_KEY) {
+ if (level < ref_path->shared_level)
+ ref_path->shared_level = level;
+ goto found;
+ }
+next:
+ level--;
+ btrfs_release_path(extent_root, path);
+ cond_resched();
+ }
+ /* reached lowest level */
+ ret = 1;
+ goto out;
+walk_up:
+ level = ref_path->current_level;
+ while (level < BTRFS_MAX_LEVEL - 1) {
+ u64 ref_objectid;
+
+ if (level >= 0)
+ bytenr = ref_path->nodes[level];
+ else
+ bytenr = ref_path->extent_start;
+
+ BUG_ON(bytenr == 0);
+
+ key.objectid = bytenr;
+ key.offset = 0;
+ key.type = BTRFS_EXTENT_REF_KEY;
+
+ ret = btrfs_search_slot(trans, extent_root, &key, path, 0, 0);
+ if (ret < 0)
+ goto out;
+
+ leaf = path->nodes[0];
+ nritems = btrfs_header_nritems(leaf);
+ if (path->slots[0] >= nritems) {
+ ret = btrfs_next_leaf(extent_root, path);
+ if (ret < 0)
+ goto out;
+ if (ret > 0) {
+ /* the extent was freed by someone */
+ if (ref_path->lowest_level == level)
+ goto out;
+ btrfs_release_path(extent_root, path);
+ goto walk_down;
+ }
+ leaf = path->nodes[0];
+ }
+
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+ if (found_key.objectid != bytenr ||
+ found_key.type != BTRFS_EXTENT_REF_KEY) {
+ /* the extent was freed by someone */
+ if (ref_path->lowest_level == level) {
+ ret = 1;
+ goto out;
+ }
+ btrfs_release_path(extent_root, path);
+ goto walk_down;
+ }
+found:
+ ref = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_extent_ref);
+ ref_objectid = btrfs_ref_objectid(leaf, ref);
+ if (ref_objectid < BTRFS_FIRST_FREE_OBJECTID) {
+ if (first_time) {
+ level = (int)ref_objectid;
+ BUG_ON(level >= BTRFS_MAX_LEVEL);
+ ref_path->lowest_level = level;
+ ref_path->current_level = level;
+ ref_path->nodes[level] = bytenr;
+ } else {
+ WARN_ON(ref_objectid != level);
+ }
+ } else {
+ WARN_ON(level != -1);
+ }
+ first_time = 0;
+
+ if (ref_path->lowest_level == level) {
+ ref_path->owner_objectid = ref_objectid;
+ ref_path->num_refs = btrfs_ref_num_refs(leaf, ref);
+ }
+
+ /*
+ * the block is tree root or the block isn't in reference
+ * counted tree.
+ */
+ if (found_key.objectid == found_key.offset ||
+ is_cowonly_root(btrfs_ref_root(leaf, ref))) {
+ ref_path->root_objectid = btrfs_ref_root(leaf, ref);
+ ref_path->root_generation =
+ btrfs_ref_generation(leaf, ref);
+ if (level < 0) {
+ /* special reference from the tree log */
+ ref_path->nodes[0] = found_key.offset;
+ ref_path->current_level = 0;
+ }
+ ret = 0;
+ goto out;
+ }
+
+ level++;
+ BUG_ON(ref_path->nodes[level] != 0);
+ ref_path->nodes[level] = found_key.offset;
+ ref_path->current_level = level;
+
+ /*
+ * the reference was created in the running transaction,
+ * no need to continue walking up.
+ */
+ if (btrfs_ref_generation(leaf, ref) == trans->transid) {
+ ref_path->root_objectid = btrfs_ref_root(leaf, ref);
+ ref_path->root_generation =
+ btrfs_ref_generation(leaf, ref);
+ ret = 0;
+ goto out;
+ }
+
+ btrfs_release_path(extent_root, path);
+ cond_resched();
+ }
+ /* reached max tree level, but no tree root found. */
+ BUG();
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
+static int btrfs_first_ref_path(struct btrfs_trans_handle *trans,
+ struct btrfs_root *extent_root,
+ struct btrfs_ref_path *ref_path,
+ u64 extent_start)
+{
+ memset(ref_path, 0, sizeof(*ref_path));
+ ref_path->extent_start = extent_start;
+
+ return __next_ref_path(trans, extent_root, ref_path, 1);
+}
+
+static int btrfs_next_ref_path(struct btrfs_trans_handle *trans,
+ struct btrfs_root *extent_root,
+ struct btrfs_ref_path *ref_path)
+{
+ return __next_ref_path(trans, extent_root, ref_path, 0);
+}
+
+static noinline int get_new_locations(struct inode *reloc_inode,
+ struct btrfs_key *extent_key,
+ u64 offset, int no_fragment,
+ struct disk_extent **extents,
+ int *nr_extents)
+{
+ struct btrfs_root *root = BTRFS_I(reloc_inode)->root;
+ struct btrfs_path *path;
+ struct btrfs_file_extent_item *fi;
+ struct extent_buffer *leaf;
+ struct disk_extent *exts = *extents;
+ struct btrfs_key found_key;
+ u64 cur_pos;
+ u64 last_byte;
+ u32 nritems;
+ int nr = 0;
+ int max = *nr_extents;
+ int ret;
+
+ WARN_ON(!no_fragment && *extents);
+ if (!exts) {
+ max = 1;
+ exts = kmalloc(sizeof(*exts) * max, GFP_NOFS);
+ if (!exts)
+ return -ENOMEM;
+ }
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+
+ cur_pos = extent_key->objectid - offset;
+ last_byte = extent_key->objectid + extent_key->offset;
+ ret = btrfs_lookup_file_extent(NULL, root, path, reloc_inode->i_ino,
+ cur_pos, 0);
+ if (ret < 0)
+ goto out;
+ if (ret > 0) {
+ ret = -ENOENT;
+ goto out;
+ }
+
+ while (1) {
+ leaf = path->nodes[0];
+ nritems = btrfs_header_nritems(leaf);
+ if (path->slots[0] >= nritems) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret < 0)
+ goto out;
+ if (ret > 0)
+ break;
+ leaf = path->nodes[0];
+ }
+
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+ if (found_key.offset != cur_pos ||
+ found_key.type != BTRFS_EXTENT_DATA_KEY ||
+ found_key.objectid != reloc_inode->i_ino)
+ break;
+
+ fi = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_file_extent_item);
+ if (btrfs_file_extent_type(leaf, fi) !=
+ BTRFS_FILE_EXTENT_REG ||
+ btrfs_file_extent_disk_bytenr(leaf, fi) == 0)
+ break;
+
+ if (nr == max) {
+ struct disk_extent *old = exts;
+ max *= 2;
+ exts = kzalloc(sizeof(*exts) * max, GFP_NOFS);
+ memcpy(exts, old, sizeof(*exts) * nr);
+ if (old != *extents)
+ kfree(old);
+ }
+
+ exts[nr].disk_bytenr =
+ btrfs_file_extent_disk_bytenr(leaf, fi);
+ exts[nr].disk_num_bytes =
+ btrfs_file_extent_disk_num_bytes(leaf, fi);
+ exts[nr].offset = btrfs_file_extent_offset(leaf, fi);
+ exts[nr].num_bytes = btrfs_file_extent_num_bytes(leaf, fi);
+ exts[nr].ram_bytes = btrfs_file_extent_ram_bytes(leaf, fi);
+ exts[nr].compression = btrfs_file_extent_compression(leaf, fi);
+ exts[nr].encryption = btrfs_file_extent_encryption(leaf, fi);
+ exts[nr].other_encoding = btrfs_file_extent_other_encoding(leaf,
+ fi);
+ BUG_ON(exts[nr].offset > 0);
+ BUG_ON(exts[nr].compression || exts[nr].encryption);
+ BUG_ON(exts[nr].num_bytes != exts[nr].disk_num_bytes);
+
+ cur_pos += exts[nr].num_bytes;
+ nr++;
+
+ if (cur_pos + offset >= last_byte)
+ break;
+
+ if (no_fragment) {
+ ret = 1;
+ goto out;
+ }
+ path->slots[0]++;
+ }
+
+ BUG_ON(cur_pos + offset > last_byte);
+ if (cur_pos + offset < last_byte) {
+ ret = -ENOENT;
+ goto out;
+ }
+ ret = 0;
+out:
+ btrfs_free_path(path);
+ if (ret) {
+ if (exts != *extents)
+ kfree(exts);
+ } else {
+ *extents = exts;
+ *nr_extents = nr;
+ }
+ return ret;
+}
+
+static noinline int replace_one_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct btrfs_key *extent_key,
+ struct btrfs_key *leaf_key,
+ struct btrfs_ref_path *ref_path,
+ struct disk_extent *new_extents,
+ int nr_extents)
+{
+ struct extent_buffer *leaf;
+ struct btrfs_file_extent_item *fi;
+ struct inode *inode = NULL;
+ struct btrfs_key key;
+ u64 lock_start = 0;
+ u64 lock_end = 0;
+ u64 num_bytes;
+ u64 ext_offset;
+ u64 first_pos;
+ u32 nritems;
+ int nr_scaned = 0;
+ int extent_locked = 0;
+ int extent_type;
+ int ret;
+
+ memcpy(&key, leaf_key, sizeof(key));
+ first_pos = INT_LIMIT(loff_t) - extent_key->offset;
+ if (ref_path->owner_objectid != BTRFS_MULTIPLE_OBJECTIDS) {
+ if (key.objectid < ref_path->owner_objectid ||
+ (key.objectid == ref_path->owner_objectid &&
+ key.type < BTRFS_EXTENT_DATA_KEY)) {
+ key.objectid = ref_path->owner_objectid;
+ key.type = BTRFS_EXTENT_DATA_KEY;
+ key.offset = 0;
+ }
+ }
+
+ while (1) {
+ ret = btrfs_search_slot(trans, root, &key, path, 0, 1);
+ if (ret < 0)
+ goto out;
+
+ leaf = path->nodes[0];
+ nritems = btrfs_header_nritems(leaf);
+next:
+ if (extent_locked && ret > 0) {
+ /*
+ * the file extent item was modified by someone
+ * before the extent got locked.
+ */
+ unlock_extent(&BTRFS_I(inode)->io_tree, lock_start,
+ lock_end, GFP_NOFS);
+ extent_locked = 0;
+ }
+
+ if (path->slots[0] >= nritems) {
+ if (++nr_scaned > 2)
+ break;
+
+ BUG_ON(extent_locked);
+ ret = btrfs_next_leaf(root, path);
+ if (ret < 0)
+ goto out;
+ if (ret > 0)
+ break;
+ leaf = path->nodes[0];
+ nritems = btrfs_header_nritems(leaf);
+ }
+
+ btrfs_item_key_to_cpu(leaf, &key, path->slots[0]);
+
+ if (ref_path->owner_objectid != BTRFS_MULTIPLE_OBJECTIDS) {
+ if ((key.objectid > ref_path->owner_objectid) ||
+ (key.objectid == ref_path->owner_objectid &&
+ key.type > BTRFS_EXTENT_DATA_KEY) ||
+ (key.offset >= first_pos + extent_key->offset))
+ break;
+ }
+
+ if (inode && key.objectid != inode->i_ino) {
+ BUG_ON(extent_locked);
+ btrfs_release_path(root, path);
+ mutex_unlock(&inode->i_mutex);
+ iput(inode);
+ inode = NULL;
+ continue;
+ }
+
+ if (key.type != BTRFS_EXTENT_DATA_KEY) {
+ path->slots[0]++;
+ ret = 1;
+ goto next;
+ }
+ fi = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_file_extent_item);
+ extent_type = btrfs_file_extent_type(leaf, fi);
+ if ((extent_type != BTRFS_FILE_EXTENT_REG &&
+ extent_type != BTRFS_FILE_EXTENT_PREALLOC) ||
+ (btrfs_file_extent_disk_bytenr(leaf, fi) !=
+ extent_key->objectid)) {
+ path->slots[0]++;
+ ret = 1;
+ goto next;
+ }
+
+ num_bytes = btrfs_file_extent_num_bytes(leaf, fi);
+ ext_offset = btrfs_file_extent_offset(leaf, fi);
+
+ if (first_pos > key.offset - ext_offset)
+ first_pos = key.offset - ext_offset;
+
+ if (!extent_locked) {
+ lock_start = key.offset;
+ lock_end = lock_start + num_bytes - 1;
+ } else {
+ if (lock_start > key.offset ||
+ lock_end + 1 < key.offset + num_bytes) {
+ unlock_extent(&BTRFS_I(inode)->io_tree,
+ lock_start, lock_end, GFP_NOFS);
+ extent_locked = 0;
+ }
+ }
+
+ if (!inode) {
+ btrfs_release_path(root, path);
+
+ inode = btrfs_iget_locked(root->fs_info->sb,
+ key.objectid, root);
+ if (inode->i_state & I_NEW) {
+ BTRFS_I(inode)->root = root;
+ BTRFS_I(inode)->location.objectid =
+ key.objectid;
+ BTRFS_I(inode)->location.type =
+ BTRFS_INODE_ITEM_KEY;
+ BTRFS_I(inode)->location.offset = 0;
+ btrfs_read_locked_inode(inode);
+ unlock_new_inode(inode);
+ }
+ /*
+ * some code call btrfs_commit_transaction while
+ * holding the i_mutex, so we can't use mutex_lock
+ * here.
+ */
+ if (is_bad_inode(inode) ||
+ !mutex_trylock(&inode->i_mutex)) {
+ iput(inode);
+ inode = NULL;
+ key.offset = (u64)-1;
+ goto skip;
+ }
+ }
+
+ if (!extent_locked) {
+ struct btrfs_ordered_extent *ordered;
+
+ btrfs_release_path(root, path);
+
+ lock_extent(&BTRFS_I(inode)->io_tree, lock_start,
+ lock_end, GFP_NOFS);
+ ordered = btrfs_lookup_first_ordered_extent(inode,
+ lock_end);
+ if (ordered &&
+ ordered->file_offset <= lock_end &&
+ ordered->file_offset + ordered->len > lock_start) {
+ unlock_extent(&BTRFS_I(inode)->io_tree,
+ lock_start, lock_end, GFP_NOFS);
+ btrfs_start_ordered_extent(inode, ordered, 1);
+ btrfs_put_ordered_extent(ordered);
+ key.offset += num_bytes;
+ goto skip;
+ }
+ if (ordered)
+ btrfs_put_ordered_extent(ordered);
+
+ extent_locked = 1;
+ continue;
+ }
+
+ if (nr_extents == 1) {
+ /* update extent pointer in place */
+ btrfs_set_file_extent_disk_bytenr(leaf, fi,
+ new_extents[0].disk_bytenr);
+ btrfs_set_file_extent_disk_num_bytes(leaf, fi,
+ new_extents[0].disk_num_bytes);
+ btrfs_mark_buffer_dirty(leaf);
+
+ btrfs_drop_extent_cache(inode, key.offset,
+ key.offset + num_bytes - 1, 0);
+
+ ret = btrfs_inc_extent_ref(trans, root,
+ new_extents[0].disk_bytenr,
+ new_extents[0].disk_num_bytes,
+ leaf->start,
+ root->root_key.objectid,
+ trans->transid,
+ key.objectid);
+ BUG_ON(ret);
+
+ ret = btrfs_free_extent(trans, root,
+ extent_key->objectid,
+ extent_key->offset,
+ leaf->start,
+ btrfs_header_owner(leaf),
+ btrfs_header_generation(leaf),
+ key.objectid, 0);
+ BUG_ON(ret);
+
+ btrfs_release_path(root, path);
+ key.offset += num_bytes;
+ } else {
+ BUG_ON(1);
+#if 0
+ u64 alloc_hint;
+ u64 extent_len;
+ int i;
+ /*
+ * drop old extent pointer at first, then insert the
+ * new pointers one bye one
+ */
+ btrfs_release_path(root, path);
+ ret = btrfs_drop_extents(trans, root, inode, key.offset,
+ key.offset + num_bytes,
+ key.offset, &alloc_hint);
+ BUG_ON(ret);
+
+ for (i = 0; i < nr_extents; i++) {
+ if (ext_offset >= new_extents[i].num_bytes) {
+ ext_offset -= new_extents[i].num_bytes;
+ continue;
+ }
+ extent_len = min(new_extents[i].num_bytes -
+ ext_offset, num_bytes);
+
+ ret = btrfs_insert_empty_item(trans, root,
+ path, &key,
+ sizeof(*fi));
+ BUG_ON(ret);
+
+ leaf = path->nodes[0];
+ fi = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_file_extent_item);
+ btrfs_set_file_extent_generation(leaf, fi,
+ trans->transid);
+ btrfs_set_file_extent_type(leaf, fi,
+ BTRFS_FILE_EXTENT_REG);
+ btrfs_set_file_extent_disk_bytenr(leaf, fi,
+ new_extents[i].disk_bytenr);
+ btrfs_set_file_extent_disk_num_bytes(leaf, fi,
+ new_extents[i].disk_num_bytes);
+ btrfs_set_file_extent_ram_bytes(leaf, fi,
+ new_extents[i].ram_bytes);
+
+ btrfs_set_file_extent_compression(leaf, fi,
+ new_extents[i].compression);
+ btrfs_set_file_extent_encryption(leaf, fi,
+ new_extents[i].encryption);
+ btrfs_set_file_extent_other_encoding(leaf, fi,
+ new_extents[i].other_encoding);
+
+ btrfs_set_file_extent_num_bytes(leaf, fi,
+ extent_len);
+ ext_offset += new_extents[i].offset;
+ btrfs_set_file_extent_offset(leaf, fi,
+ ext_offset);
+ btrfs_mark_buffer_dirty(leaf);
+
+ btrfs_drop_extent_cache(inode, key.offset,
+ key.offset + extent_len - 1, 0);
+
+ ret = btrfs_inc_extent_ref(trans, root,
+ new_extents[i].disk_bytenr,
+ new_extents[i].disk_num_bytes,
+ leaf->start,
+ root->root_key.objectid,
+ trans->transid, key.objectid);
+ BUG_ON(ret);
+ btrfs_release_path(root, path);
+
+ inode_add_bytes(inode, extent_len);
+
+ ext_offset = 0;
+ num_bytes -= extent_len;
+ key.offset += extent_len;
+
+ if (num_bytes == 0)
+ break;
+ }
+ BUG_ON(i >= nr_extents);
+#endif
+ }
+
+ if (extent_locked) {
+ unlock_extent(&BTRFS_I(inode)->io_tree, lock_start,
+ lock_end, GFP_NOFS);
+ extent_locked = 0;
+ }
+skip:
+ if (ref_path->owner_objectid != BTRFS_MULTIPLE_OBJECTIDS &&
+ key.offset >= first_pos + extent_key->offset)
+ break;
+
+ cond_resched();
+ }
+ ret = 0;
+out:
+ btrfs_release_path(root, path);
+ if (inode) {
+ mutex_unlock(&inode->i_mutex);
+ if (extent_locked) {
+ unlock_extent(&BTRFS_I(inode)->io_tree, lock_start,
+ lock_end, GFP_NOFS);
+ }
+ iput(inode);
+ }
+ return ret;
+}
+
+int btrfs_reloc_tree_cache_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct extent_buffer *buf, u64 orig_start)
+{
+ int level;
+ int ret;
+
+ BUG_ON(btrfs_header_generation(buf) != trans->transid);
+ BUG_ON(root->root_key.objectid != BTRFS_TREE_RELOC_OBJECTID);
+
+ level = btrfs_header_level(buf);
+ if (level == 0) {
+ struct btrfs_leaf_ref *ref;
+ struct btrfs_leaf_ref *orig_ref;
+
+ orig_ref = btrfs_lookup_leaf_ref(root, orig_start);
+ if (!orig_ref)
+ return -ENOENT;
+
+ ref = btrfs_alloc_leaf_ref(root, orig_ref->nritems);
+ if (!ref) {
+ btrfs_free_leaf_ref(root, orig_ref);
+ return -ENOMEM;
+ }
+
+ ref->nritems = orig_ref->nritems;
+ memcpy(ref->extents, orig_ref->extents,
+ sizeof(ref->extents[0]) * ref->nritems);
+
+ btrfs_free_leaf_ref(root, orig_ref);
+
+ ref->root_gen = trans->transid;
+ ref->bytenr = buf->start;
+ ref->owner = btrfs_header_owner(buf);
+ ref->generation = btrfs_header_generation(buf);
+ ret = btrfs_add_leaf_ref(root, ref, 0);
+ WARN_ON(ret);
+ btrfs_free_leaf_ref(root, ref);
+ }
+ return 0;
+}
+
+static noinline int invalidate_extent_cache(struct btrfs_root *root,
+ struct extent_buffer *leaf,
+ struct btrfs_block_group_cache *group,
+ struct btrfs_root *target_root)
+{
+ struct btrfs_key key;
+ struct inode *inode = NULL;
+ struct btrfs_file_extent_item *fi;
+ u64 num_bytes;
+ u64 skip_objectid = 0;
+ u32 nritems;
+ u32 i;
+
+ nritems = btrfs_header_nritems(leaf);
+ for (i = 0; i < nritems; i++) {
+ btrfs_item_key_to_cpu(leaf, &key, i);
+ if (key.objectid == skip_objectid ||
+ key.type != BTRFS_EXTENT_DATA_KEY)
+ continue;
+ fi = btrfs_item_ptr(leaf, i, struct btrfs_file_extent_item);
+ if (btrfs_file_extent_type(leaf, fi) ==
+ BTRFS_FILE_EXTENT_INLINE)
+ continue;
+ if (btrfs_file_extent_disk_bytenr(leaf, fi) == 0)
+ continue;
+ if (!inode || inode->i_ino != key.objectid) {
+ iput(inode);
+ inode = btrfs_ilookup(target_root->fs_info->sb,
+ key.objectid, target_root, 1);
+ }
+ if (!inode) {
+ skip_objectid = key.objectid;
+ continue;
+ }
+ num_bytes = btrfs_file_extent_num_bytes(leaf, fi);
+
+ lock_extent(&BTRFS_I(inode)->io_tree, key.offset,
+ key.offset + num_bytes - 1, GFP_NOFS);
+ btrfs_drop_extent_cache(inode, key.offset,
+ key.offset + num_bytes - 1, 1);
+ unlock_extent(&BTRFS_I(inode)->io_tree, key.offset,
+ key.offset + num_bytes - 1, GFP_NOFS);
+ cond_resched();
+ }
+ iput(inode);
+ return 0;
+}
+
+static noinline int replace_extents_in_leaf(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct extent_buffer *leaf,
+ struct btrfs_block_group_cache *group,
+ struct inode *reloc_inode)
+{
+ struct btrfs_key key;
+ struct btrfs_key extent_key;
+ struct btrfs_file_extent_item *fi;
+ struct btrfs_leaf_ref *ref;
+ struct disk_extent *new_extent;
+ u64 bytenr;
+ u64 num_bytes;
+ u32 nritems;
+ u32 i;
+ int ext_index;
+ int nr_extent;
+ int ret;
+
+ new_extent = kmalloc(sizeof(*new_extent), GFP_NOFS);
+ BUG_ON(!new_extent);
+
+ ref = btrfs_lookup_leaf_ref(root, leaf->start);
+ BUG_ON(!ref);
+
+ ext_index = -1;
+ nritems = btrfs_header_nritems(leaf);
+ for (i = 0; i < nritems; i++) {
+ btrfs_item_key_to_cpu(leaf, &key, i);
+ if (btrfs_key_type(&key) != BTRFS_EXTENT_DATA_KEY)
+ continue;
+ fi = btrfs_item_ptr(leaf, i, struct btrfs_file_extent_item);
+ if (btrfs_file_extent_type(leaf, fi) ==
+ BTRFS_FILE_EXTENT_INLINE)
+ continue;
+ bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
+ num_bytes = btrfs_file_extent_disk_num_bytes(leaf, fi);
+ if (bytenr == 0)
+ continue;
+
+ ext_index++;
+ if (bytenr >= group->key.objectid + group->key.offset ||
+ bytenr + num_bytes <= group->key.objectid)
+ continue;
+
+ extent_key.objectid = bytenr;
+ extent_key.offset = num_bytes;
+ extent_key.type = BTRFS_EXTENT_ITEM_KEY;
+ nr_extent = 1;
+ ret = get_new_locations(reloc_inode, &extent_key,
+ group->key.objectid, 1,
+ &new_extent, &nr_extent);
+ if (ret > 0)
+ continue;
+ BUG_ON(ret < 0);
+
+ BUG_ON(ref->extents[ext_index].bytenr != bytenr);
+ BUG_ON(ref->extents[ext_index].num_bytes != num_bytes);
+ ref->extents[ext_index].bytenr = new_extent->disk_bytenr;
+ ref->extents[ext_index].num_bytes = new_extent->disk_num_bytes;
+
+ btrfs_set_file_extent_disk_bytenr(leaf, fi,
+ new_extent->disk_bytenr);
+ btrfs_set_file_extent_disk_num_bytes(leaf, fi,
+ new_extent->disk_num_bytes);
+ btrfs_mark_buffer_dirty(leaf);
+
+ ret = btrfs_inc_extent_ref(trans, root,
+ new_extent->disk_bytenr,
+ new_extent->disk_num_bytes,
+ leaf->start,
+ root->root_key.objectid,
+ trans->transid, key.objectid);
+ BUG_ON(ret);
+ ret = btrfs_free_extent(trans, root,
+ bytenr, num_bytes, leaf->start,
+ btrfs_header_owner(leaf),
+ btrfs_header_generation(leaf),
+ key.objectid, 0);
+ BUG_ON(ret);
+ cond_resched();
+ }
+ kfree(new_extent);
+ BUG_ON(ext_index + 1 != ref->nritems);
+ btrfs_free_leaf_ref(root, ref);
+ return 0;
+}
+
+int btrfs_free_reloc_root(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root)
+{
+ struct btrfs_root *reloc_root;
+ int ret;
+
+ if (root->reloc_root) {
+ reloc_root = root->reloc_root;
+ root->reloc_root = NULL;
+ list_add(&reloc_root->dead_list,
+ &root->fs_info->dead_reloc_roots);
+
+ btrfs_set_root_bytenr(&reloc_root->root_item,
+ reloc_root->node->start);
+ btrfs_set_root_level(&root->root_item,
+ btrfs_header_level(reloc_root->node));
+ memset(&reloc_root->root_item.drop_progress, 0,
+ sizeof(struct btrfs_disk_key));
+ reloc_root->root_item.drop_level = 0;
+
+ ret = btrfs_update_root(trans, root->fs_info->tree_root,
+ &reloc_root->root_key,
+ &reloc_root->root_item);
+ BUG_ON(ret);
+ }
+ return 0;
+}
+
+int btrfs_drop_dead_reloc_roots(struct btrfs_root *root)
+{
+ struct btrfs_trans_handle *trans;
+ struct btrfs_root *reloc_root;
+ struct btrfs_root *prev_root = NULL;
+ struct list_head dead_roots;
+ int ret;
+ unsigned long nr;
+
+ INIT_LIST_HEAD(&dead_roots);
+ list_splice_init(&root->fs_info->dead_reloc_roots, &dead_roots);
+
+ while (!list_empty(&dead_roots)) {
+ reloc_root = list_entry(dead_roots.prev,
+ struct btrfs_root, dead_list);
+ list_del_init(&reloc_root->dead_list);
+
+ BUG_ON(reloc_root->commit_root != NULL);
+ while (1) {
+ trans = btrfs_join_transaction(root, 1);
+ BUG_ON(!trans);
+
+ mutex_lock(&root->fs_info->drop_mutex);
+ ret = btrfs_drop_snapshot(trans, reloc_root);
+ if (ret != -EAGAIN)
+ break;
+ mutex_unlock(&root->fs_info->drop_mutex);
+
+ nr = trans->blocks_used;
+ ret = btrfs_end_transaction(trans, root);
+ BUG_ON(ret);
+ btrfs_btree_balance_dirty(root, nr);
+ }
+
+ free_extent_buffer(reloc_root->node);
+
+ ret = btrfs_del_root(trans, root->fs_info->tree_root,
+ &reloc_root->root_key);
+ BUG_ON(ret);
+ mutex_unlock(&root->fs_info->drop_mutex);
+
+ nr = trans->blocks_used;
+ ret = btrfs_end_transaction(trans, root);
+ BUG_ON(ret);
+ btrfs_btree_balance_dirty(root, nr);
+
+ kfree(prev_root);
+ prev_root = reloc_root;
+ }
+ if (prev_root) {
+ btrfs_remove_leaf_refs(prev_root, (u64)-1, 0);
+ kfree(prev_root);
+ }
+ return 0;
+}
+
+int btrfs_add_dead_reloc_root(struct btrfs_root *root)
+{
+ list_add(&root->dead_list, &root->fs_info->dead_reloc_roots);
+ return 0;
+}
+
+int btrfs_cleanup_reloc_trees(struct btrfs_root *root)
+{
+ struct btrfs_root *reloc_root;
+ struct btrfs_trans_handle *trans;
+ struct btrfs_key location;
+ int found;
+ int ret;
+
+ mutex_lock(&root->fs_info->tree_reloc_mutex);
+ ret = btrfs_find_dead_roots(root, BTRFS_TREE_RELOC_OBJECTID, NULL);
+ BUG_ON(ret);
+ found = !list_empty(&root->fs_info->dead_reloc_roots);
+ mutex_unlock(&root->fs_info->tree_reloc_mutex);
+
+ if (found) {
+ trans = btrfs_start_transaction(root, 1);
+ BUG_ON(!trans);
+ ret = btrfs_commit_transaction(trans, root);
+ BUG_ON(ret);
+ }
+
+ location.objectid = BTRFS_DATA_RELOC_TREE_OBJECTID;
+ location.offset = (u64)-1;
+ location.type = BTRFS_ROOT_ITEM_KEY;
+
+ reloc_root = btrfs_read_fs_root_no_name(root->fs_info, &location);
+ BUG_ON(!reloc_root);
+ btrfs_orphan_cleanup(reloc_root);
+ return 0;
+}
+
+static noinline int init_reloc_tree(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root)
+{
+ struct btrfs_root *reloc_root;
+ struct extent_buffer *eb;
+ struct btrfs_root_item *root_item;
+ struct btrfs_key root_key;
+ int ret;
+
+ BUG_ON(!root->ref_cows);
+ if (root->reloc_root)
+ return 0;
+
+ root_item = kmalloc(sizeof(*root_item), GFP_NOFS);
+ BUG_ON(!root_item);
+
+ ret = btrfs_copy_root(trans, root, root->commit_root,
+ &eb, BTRFS_TREE_RELOC_OBJECTID);
+ BUG_ON(ret);
+
+ root_key.objectid = BTRFS_TREE_RELOC_OBJECTID;
+ root_key.offset = root->root_key.objectid;
+ root_key.type = BTRFS_ROOT_ITEM_KEY;
+
+ memcpy(root_item, &root->root_item, sizeof(root_item));
+ btrfs_set_root_refs(root_item, 0);
+ btrfs_set_root_bytenr(root_item, eb->start);
+ btrfs_set_root_level(root_item, btrfs_header_level(eb));
+ btrfs_set_root_generation(root_item, trans->transid);
+
+ btrfs_tree_unlock(eb);
+ free_extent_buffer(eb);
+
+ ret = btrfs_insert_root(trans, root->fs_info->tree_root,
+ &root_key, root_item);
+ BUG_ON(ret);
+ kfree(root_item);
+
+ reloc_root = btrfs_read_fs_root_no_radix(root->fs_info->tree_root,
+ &root_key);
+ BUG_ON(!reloc_root);
+ reloc_root->last_trans = trans->transid;
+ reloc_root->commit_root = NULL;
+ reloc_root->ref_tree = &root->fs_info->reloc_ref_tree;
+
+ root->reloc_root = reloc_root;
+ return 0;
+}
+
+/*
+ * Core function of space balance.
+ *
+ * The idea is using reloc trees to relocate tree blocks in reference
+ * counted roots. There is one reloc tree for each subvol, and all
+ * reloc trees share same root key objectid. Reloc trees are snapshots
+ * of the latest committed roots of subvols (root->commit_root).
+ *
+ * To relocate a tree block referenced by a subvol, there are two steps.
+ * COW the block through subvol's reloc tree, then update block pointer
+ * in the subvol to point to the new block. Since all reloc trees share
+ * same root key objectid, doing special handing for tree blocks owned
+ * by them is easy. Once a tree block has been COWed in one reloc tree,
+ * we can use the resulting new block directly when the same block is
+ * required to COW again through other reloc trees. By this way, relocated
+ * tree blocks are shared between reloc trees, so they are also shared
+ * between subvols.
+ */
+static noinline int relocate_one_path(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct btrfs_key *first_key,
+ struct btrfs_ref_path *ref_path,
+ struct btrfs_block_group_cache *group,
+ struct inode *reloc_inode)
+{
+ struct btrfs_root *reloc_root;
+ struct extent_buffer *eb = NULL;
+ struct btrfs_key *keys;
+ u64 *nodes;
+ int level;
+ int shared_level;
+ int lowest_level = 0;
+ int ret;
+
+ if (ref_path->owner_objectid < BTRFS_FIRST_FREE_OBJECTID)
+ lowest_level = ref_path->owner_objectid;
+
+ if (!root->ref_cows) {
+ path->lowest_level = lowest_level;
+ ret = btrfs_search_slot(trans, root, first_key, path, 0, 1);
+ BUG_ON(ret < 0);
+ path->lowest_level = 0;
+ btrfs_release_path(root, path);
+ return 0;
+ }
+
+ mutex_lock(&root->fs_info->tree_reloc_mutex);
+ ret = init_reloc_tree(trans, root);
+ BUG_ON(ret);
+ reloc_root = root->reloc_root;
+
+ shared_level = ref_path->shared_level;
+ ref_path->shared_level = BTRFS_MAX_LEVEL - 1;
+
+ keys = ref_path->node_keys;
+ nodes = ref_path->new_nodes;
+ memset(&keys[shared_level + 1], 0,
+ sizeof(*keys) * (BTRFS_MAX_LEVEL - shared_level - 1));
+ memset(&nodes[shared_level + 1], 0,
+ sizeof(*nodes) * (BTRFS_MAX_LEVEL - shared_level - 1));
+
+ if (nodes[lowest_level] == 0) {
+ path->lowest_level = lowest_level;
+ ret = btrfs_search_slot(trans, reloc_root, first_key, path,
+ 0, 1);
+ BUG_ON(ret);
+ for (level = lowest_level; level < BTRFS_MAX_LEVEL; level++) {
+ eb = path->nodes[level];
+ if (!eb || eb == reloc_root->node)
+ break;
+ nodes[level] = eb->start;
+ if (level == 0)
+ btrfs_item_key_to_cpu(eb, &keys[level], 0);
+ else
+ btrfs_node_key_to_cpu(eb, &keys[level], 0);
+ }
+ if (nodes[0] &&
+ ref_path->owner_objectid >= BTRFS_FIRST_FREE_OBJECTID) {
+ eb = path->nodes[0];
+ ret = replace_extents_in_leaf(trans, reloc_root, eb,
+ group, reloc_inode);
+ BUG_ON(ret);
+ }
+ btrfs_release_path(reloc_root, path);
+ } else {
+ ret = btrfs_merge_path(trans, reloc_root, keys, nodes,
+ lowest_level);
+ BUG_ON(ret);
+ }
+
+ /*
+ * replace tree blocks in the fs tree with tree blocks in
+ * the reloc tree.
+ */
+ ret = btrfs_merge_path(trans, root, keys, nodes, lowest_level);
+ BUG_ON(ret < 0);
+
+ if (ref_path->owner_objectid >= BTRFS_FIRST_FREE_OBJECTID) {
+ ret = btrfs_search_slot(trans, reloc_root, first_key, path,
+ 0, 0);
+ BUG_ON(ret);
+ extent_buffer_get(path->nodes[0]);
+ eb = path->nodes[0];
+ btrfs_release_path(reloc_root, path);
+ ret = invalidate_extent_cache(reloc_root, eb, group, root);
+ BUG_ON(ret);
+ free_extent_buffer(eb);
+ }
+
+ mutex_unlock(&root->fs_info->tree_reloc_mutex);
+ path->lowest_level = 0;
+ return 0;
+}
+
+static noinline int relocate_tree_block(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct btrfs_key *first_key,
+ struct btrfs_ref_path *ref_path)
+{
+ int ret;
+
+ ret = relocate_one_path(trans, root, path, first_key,
+ ref_path, NULL, NULL);
+ BUG_ON(ret);
+
+ if (root == root->fs_info->extent_root)
+ btrfs_extent_post_op(trans, root);
+
+ return 0;
+}
+
+static noinline int del_extent_zero(struct btrfs_trans_handle *trans,
+ struct btrfs_root *extent_root,
+ struct btrfs_path *path,
+ struct btrfs_key *extent_key)
+{
+ int ret;
+
+ ret = btrfs_search_slot(trans, extent_root, extent_key, path, -1, 1);
+ if (ret)
+ goto out;
+ ret = btrfs_del_item(trans, extent_root, path);
+out:
+ btrfs_release_path(extent_root, path);
+ return ret;
+}
+
+static noinline struct btrfs_root *read_ref_root(struct btrfs_fs_info *fs_info,
+ struct btrfs_ref_path *ref_path)
+{
+ struct btrfs_key root_key;
+
+ root_key.objectid = ref_path->root_objectid;
+ root_key.type = BTRFS_ROOT_ITEM_KEY;
+ if (is_cowonly_root(ref_path->root_objectid))
+ root_key.offset = 0;
+ else
+ root_key.offset = (u64)-1;
+
+ return btrfs_read_fs_root_no_name(fs_info, &root_key);
+}
+
+static noinline int relocate_one_extent(struct btrfs_root *extent_root,
+ struct btrfs_path *path,
+ struct btrfs_key *extent_key,
+ struct btrfs_block_group_cache *group,
+ struct inode *reloc_inode, int pass)
+{
+ struct btrfs_trans_handle *trans;
+ struct btrfs_root *found_root;
+ struct btrfs_ref_path *ref_path = NULL;
+ struct disk_extent *new_extents = NULL;
+ int nr_extents = 0;
+ int loops;
+ int ret;
+ int level;
+ struct btrfs_key first_key;
+ u64 prev_block = 0;
+
+
+ trans = btrfs_start_transaction(extent_root, 1);
+ BUG_ON(!trans);
+
+ if (extent_key->objectid == 0) {
+ ret = del_extent_zero(trans, extent_root, path, extent_key);
+ goto out;
+ }
+
+ ref_path = kmalloc(sizeof(*ref_path), GFP_NOFS);
+ if (!ref_path) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ for (loops = 0; ; loops++) {
+ if (loops == 0) {
+ ret = btrfs_first_ref_path(trans, extent_root, ref_path,
+ extent_key->objectid);
+ } else {
+ ret = btrfs_next_ref_path(trans, extent_root, ref_path);
+ }
+ if (ret < 0)
+ goto out;
+ if (ret > 0)
+ break;
+
+ if (ref_path->root_objectid == BTRFS_TREE_LOG_OBJECTID ||
+ ref_path->root_objectid == BTRFS_TREE_RELOC_OBJECTID)
+ continue;
+
+ found_root = read_ref_root(extent_root->fs_info, ref_path);
+ BUG_ON(!found_root);
+ /*
+ * for reference counted tree, only process reference paths
+ * rooted at the latest committed root.
+ */
+ if (found_root->ref_cows &&
+ ref_path->root_generation != found_root->root_key.offset)
+ continue;
+
+ if (ref_path->owner_objectid >= BTRFS_FIRST_FREE_OBJECTID) {
+ if (pass == 0) {
+ /*
+ * copy data extents to new locations
+ */
+ u64 group_start = group->key.objectid;
+ ret = relocate_data_extent(reloc_inode,
+ extent_key,
+ group_start);
+ if (ret < 0)
+ goto out;
+ break;
+ }
+ level = 0;
+ } else {
+ level = ref_path->owner_objectid;
+ }
+
+ if (prev_block != ref_path->nodes[level]) {
+ struct extent_buffer *eb;
+ u64 block_start = ref_path->nodes[level];
+ u64 block_size = btrfs_level_size(found_root, level);
+
+ eb = read_tree_block(found_root, block_start,
+ block_size, 0);
+ btrfs_tree_lock(eb);
+ BUG_ON(level != btrfs_header_level(eb));
+
+ if (level == 0)
+ btrfs_item_key_to_cpu(eb, &first_key, 0);
+ else
+ btrfs_node_key_to_cpu(eb, &first_key, 0);
+
+ btrfs_tree_unlock(eb);
+ free_extent_buffer(eb);
+ prev_block = block_start;
+ }
+
+ btrfs_record_root_in_trans(found_root);
+ if (ref_path->owner_objectid >= BTRFS_FIRST_FREE_OBJECTID) {
+ /*
+ * try to update data extent references while
+ * keeping metadata shared between snapshots.
+ */
+ if (pass == 1) {
+ ret = relocate_one_path(trans, found_root,
+ path, &first_key, ref_path,
+ group, reloc_inode);
+ if (ret < 0)
+ goto out;
+ continue;
+ }
+ /*
+ * use fallback method to process the remaining
+ * references.
+ */
+ if (!new_extents) {
+ u64 group_start = group->key.objectid;
+ new_extents = kmalloc(sizeof(*new_extents),
+ GFP_NOFS);
+ nr_extents = 1;
+ ret = get_new_locations(reloc_inode,
+ extent_key,
+ group_start, 1,
+ &new_extents,
+ &nr_extents);
+ if (ret)
+ goto out;
+ }
+ ret = replace_one_extent(trans, found_root,
+ path, extent_key,
+ &first_key, ref_path,
+ new_extents, nr_extents);
+ } else {
+ ret = relocate_tree_block(trans, found_root, path,
+ &first_key, ref_path);
+ }
+ if (ret < 0)
+ goto out;
+ }
+ ret = 0;
+out:
+ btrfs_end_transaction(trans, extent_root);
+ kfree(new_extents);
+ kfree(ref_path);
+ return ret;
+}
+
+static u64 update_block_group_flags(struct btrfs_root *root, u64 flags)
+{
+ u64 num_devices;
+ u64 stripped = BTRFS_BLOCK_GROUP_RAID0 |
+ BTRFS_BLOCK_GROUP_RAID1 | BTRFS_BLOCK_GROUP_RAID10;
+
+ num_devices = root->fs_info->fs_devices->rw_devices;
+ if (num_devices == 1) {
+ stripped |= BTRFS_BLOCK_GROUP_DUP;
+ stripped = flags & ~stripped;
+
+ /* turn raid0 into single device chunks */
+ if (flags & BTRFS_BLOCK_GROUP_RAID0)
+ return stripped;
+
+ /* turn mirroring into duplication */
+ if (flags & (BTRFS_BLOCK_GROUP_RAID1 |
+ BTRFS_BLOCK_GROUP_RAID10))
+ return stripped | BTRFS_BLOCK_GROUP_DUP;
+ return flags;
+ } else {
+ /* they already had raid on here, just return */
+ if (flags & stripped)
+ return flags;
+
+ stripped |= BTRFS_BLOCK_GROUP_DUP;
+ stripped = flags & ~stripped;
+
+ /* switch duplicated blocks with raid1 */
+ if (flags & BTRFS_BLOCK_GROUP_DUP)
+ return stripped | BTRFS_BLOCK_GROUP_RAID1;
+
+ /* turn single device chunks into raid0 */
+ return stripped | BTRFS_BLOCK_GROUP_RAID0;
+ }
+ return flags;
+}
+
+static int __alloc_chunk_for_shrink(struct btrfs_root *root,
+ struct btrfs_block_group_cache *shrink_block_group,
+ int force)
+{
+ struct btrfs_trans_handle *trans;
+ u64 new_alloc_flags;
+ u64 calc;
+
+ spin_lock(&shrink_block_group->lock);
+ if (btrfs_block_group_used(&shrink_block_group->item) > 0) {
+ spin_unlock(&shrink_block_group->lock);
+
+ trans = btrfs_start_transaction(root, 1);
+ spin_lock(&shrink_block_group->lock);
+
+ new_alloc_flags = update_block_group_flags(root,
+ shrink_block_group->flags);
+ if (new_alloc_flags != shrink_block_group->flags) {
+ calc =
+ btrfs_block_group_used(&shrink_block_group->item);
+ } else {
+ calc = shrink_block_group->key.offset;
+ }
+ spin_unlock(&shrink_block_group->lock);
+
+ do_chunk_alloc(trans, root->fs_info->extent_root,
+ calc + 2 * 1024 * 1024, new_alloc_flags, force);
+
+ btrfs_end_transaction(trans, root);
+ } else
+ spin_unlock(&shrink_block_group->lock);
+ return 0;
+}
+
+static int __insert_orphan_inode(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 objectid, u64 size)
+{
+ struct btrfs_path *path;
+ struct btrfs_inode_item *item;
+ struct extent_buffer *leaf;
+ int ret;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ ret = btrfs_insert_empty_inode(trans, root, path, objectid);
+ if (ret)
+ goto out;
+
+ leaf = path->nodes[0];
+ item = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_inode_item);
+ memset_extent_buffer(leaf, 0, (unsigned long)item, sizeof(*item));
+ btrfs_set_inode_generation(leaf, item, 1);
+ btrfs_set_inode_size(leaf, item, size);
+ btrfs_set_inode_mode(leaf, item, S_IFREG | 0600);
+ btrfs_set_inode_flags(leaf, item, BTRFS_INODE_NOCOMPRESS);
+ btrfs_mark_buffer_dirty(leaf);
+ btrfs_release_path(root, path);
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
+static noinline struct inode *create_reloc_inode(struct btrfs_fs_info *fs_info,
+ struct btrfs_block_group_cache *group)
+{
+ struct inode *inode = NULL;
+ struct btrfs_trans_handle *trans;
+ struct btrfs_root *root;
+ struct btrfs_key root_key;
+ u64 objectid = BTRFS_FIRST_FREE_OBJECTID;
+ int err = 0;
+
+ root_key.objectid = BTRFS_DATA_RELOC_TREE_OBJECTID;
+ root_key.type = BTRFS_ROOT_ITEM_KEY;
+ root_key.offset = (u64)-1;
+ root = btrfs_read_fs_root_no_name(fs_info, &root_key);
+ if (IS_ERR(root))
+ return ERR_CAST(root);
+
+ trans = btrfs_start_transaction(root, 1);
+ BUG_ON(!trans);
+
+ err = btrfs_find_free_objectid(trans, root, objectid, &objectid);
+ if (err)
+ goto out;
+
+ err = __insert_orphan_inode(trans, root, objectid, group->key.offset);
+ BUG_ON(err);
+
+ err = btrfs_insert_file_extent(trans, root, objectid, 0, 0, 0,
+ group->key.offset, 0, group->key.offset,
+ 0, 0, 0);
+ BUG_ON(err);
+
+ inode = btrfs_iget_locked(root->fs_info->sb, objectid, root);
+ if (inode->i_state & I_NEW) {
+ BTRFS_I(inode)->root = root;
+ BTRFS_I(inode)->location.objectid = objectid;
+ BTRFS_I(inode)->location.type = BTRFS_INODE_ITEM_KEY;
+ BTRFS_I(inode)->location.offset = 0;
+ btrfs_read_locked_inode(inode);
+ unlock_new_inode(inode);
+ BUG_ON(is_bad_inode(inode));
+ } else {
+ BUG_ON(1);
+ }
+ BTRFS_I(inode)->index_cnt = group->key.objectid;
+
+ err = btrfs_orphan_add(trans, inode);
+out:
+ btrfs_end_transaction(trans, root);
+ if (err) {
+ if (inode)
+ iput(inode);
+ inode = ERR_PTR(err);
+ }
+ return inode;
+}
+
+int btrfs_reloc_clone_csums(struct inode *inode, u64 file_pos, u64 len)
+{
+
+ struct btrfs_ordered_sum *sums;
+ struct btrfs_sector_sum *sector_sum;
+ struct btrfs_ordered_extent *ordered;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct list_head list;
+ size_t offset;
+ int ret;
+ u64 disk_bytenr;
+
+ INIT_LIST_HEAD(&list);
+
+ ordered = btrfs_lookup_ordered_extent(inode, file_pos);
+ BUG_ON(ordered->file_offset != file_pos || ordered->len != len);
+
+ disk_bytenr = file_pos + BTRFS_I(inode)->index_cnt;
+ ret = btrfs_lookup_csums_range(root->fs_info->csum_root, disk_bytenr,
+ disk_bytenr + len - 1, &list);
+
+ while (!list_empty(&list)) {
+ sums = list_entry(list.next, struct btrfs_ordered_sum, list);
+ list_del_init(&sums->list);
+
+ sector_sum = sums->sums;
+ sums->bytenr = ordered->start;
+
+ offset = 0;
+ while (offset < sums->len) {
+ sector_sum->bytenr += ordered->start - disk_bytenr;
+ sector_sum++;
+ offset += root->sectorsize;
+ }
+
+ btrfs_add_ordered_sum(inode, ordered, sums);
+ }
+ btrfs_put_ordered_extent(ordered);
+ return 0;
+}
+
+int btrfs_relocate_block_group(struct btrfs_root *root, u64 group_start)
+{
+ struct btrfs_trans_handle *trans;
+ struct btrfs_path *path;
+ struct btrfs_fs_info *info = root->fs_info;
+ struct extent_buffer *leaf;
+ struct inode *reloc_inode;
+ struct btrfs_block_group_cache *block_group;
+ struct btrfs_key key;
+ u64 skipped;
+ u64 cur_byte;
+ u64 total_found;
+ u32 nritems;
+ int ret;
+ int progress;
+ int pass = 0;
+
+ root = root->fs_info->extent_root;
+
+ block_group = btrfs_lookup_block_group(info, group_start);
+ BUG_ON(!block_group);
+
+ printk(KERN_INFO "btrfs relocating block group %llu flags %llu\n",
+ (unsigned long long)block_group->key.objectid,
+ (unsigned long long)block_group->flags);
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+
+ reloc_inode = create_reloc_inode(info, block_group);
+ BUG_ON(IS_ERR(reloc_inode));
+
+ __alloc_chunk_for_shrink(root, block_group, 1);
+ set_block_group_readonly(block_group);
+
+ btrfs_start_delalloc_inodes(info->tree_root);
+ btrfs_wait_ordered_extents(info->tree_root, 0);
+again:
+ skipped = 0;
+ total_found = 0;
+ progress = 0;
+ key.objectid = block_group->key.objectid;
+ key.offset = 0;
+ key.type = 0;
+ cur_byte = key.objectid;
+
+ trans = btrfs_start_transaction(info->tree_root, 1);
+ btrfs_commit_transaction(trans, info->tree_root);
+
+ mutex_lock(&root->fs_info->cleaner_mutex);
+ btrfs_clean_old_snapshots(info->tree_root);
+ btrfs_remove_leaf_refs(info->tree_root, (u64)-1, 1);
+ mutex_unlock(&root->fs_info->cleaner_mutex);
+
+ while (1) {
+ ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ if (ret < 0)
+ goto out;
+next:
+ leaf = path->nodes[0];
+ nritems = btrfs_header_nritems(leaf);
+ if (path->slots[0] >= nritems) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret < 0)
+ goto out;
+ if (ret == 1) {
+ ret = 0;
+ break;
+ }
+ leaf = path->nodes[0];
+ nritems = btrfs_header_nritems(leaf);
+ }
+
+ btrfs_item_key_to_cpu(leaf, &key, path->slots[0]);
+
+ if (key.objectid >= block_group->key.objectid +
+ block_group->key.offset)
+ break;
+
+ if (progress && need_resched()) {
+ btrfs_release_path(root, path);
+ cond_resched();
+ progress = 0;
+ continue;
+ }
+ progress = 1;
+
+ if (btrfs_key_type(&key) != BTRFS_EXTENT_ITEM_KEY ||
+ key.objectid + key.offset <= cur_byte) {
+ path->slots[0]++;
+ goto next;
+ }
+
+ total_found++;
+ cur_byte = key.objectid + key.offset;
+ btrfs_release_path(root, path);
+
+ __alloc_chunk_for_shrink(root, block_group, 0);
+ ret = relocate_one_extent(root, path, &key, block_group,
+ reloc_inode, pass);
+ BUG_ON(ret < 0);
+ if (ret > 0)
+ skipped++;
+
+ key.objectid = cur_byte;
+ key.type = 0;
+ key.offset = 0;
+ }
+
+ btrfs_release_path(root, path);
+
+ if (pass == 0) {
+ btrfs_wait_ordered_range(reloc_inode, 0, (u64)-1);
+ invalidate_mapping_pages(reloc_inode->i_mapping, 0, -1);
+ }
+
+ if (total_found > 0) {
+ printk(KERN_INFO "btrfs found %llu extents in pass %d\n",
+ (unsigned long long)total_found, pass);
+ pass++;
+ if (total_found == skipped && pass > 2) {
+ iput(reloc_inode);
+ reloc_inode = create_reloc_inode(info, block_group);
+ pass = 0;
+ }
+ goto again;
+ }
+
+ /* delete reloc_inode */
+ iput(reloc_inode);
+
+ /* unpin extents in this range */
+ trans = btrfs_start_transaction(info->tree_root, 1);
+ btrfs_commit_transaction(trans, info->tree_root);
+
+ spin_lock(&block_group->lock);
+ WARN_ON(block_group->pinned > 0);
+ WARN_ON(block_group->reserved > 0);
+ WARN_ON(btrfs_block_group_used(&block_group->item) > 0);
+ spin_unlock(&block_group->lock);
+ put_block_group(block_group);
+ ret = 0;
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
+static int find_first_block_group(struct btrfs_root *root,
+ struct btrfs_path *path, struct btrfs_key *key)
+{
+ int ret = 0;
+ struct btrfs_key found_key;
+ struct extent_buffer *leaf;
+ int slot;
+
+ ret = btrfs_search_slot(NULL, root, key, path, 0, 0);
+ if (ret < 0)
+ goto out;
+
+ while (1) {
+ slot = path->slots[0];
+ leaf = path->nodes[0];
+ if (slot >= btrfs_header_nritems(leaf)) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret == 0)
+ continue;
+ if (ret < 0)
+ goto out;
+ break;
+ }
+ btrfs_item_key_to_cpu(leaf, &found_key, slot);
+
+ if (found_key.objectid >= key->objectid &&
+ found_key.type == BTRFS_BLOCK_GROUP_ITEM_KEY) {
+ ret = 0;
+ goto out;
+ }
+ path->slots[0]++;
+ }
+ ret = -ENOENT;
+out:
+ return ret;
+}
+
+int btrfs_free_block_groups(struct btrfs_fs_info *info)
+{
+ struct btrfs_block_group_cache *block_group;
+ struct rb_node *n;
+
+ spin_lock(&info->block_group_cache_lock);
+ while ((n = rb_last(&info->block_group_cache_tree)) != NULL) {
+ block_group = rb_entry(n, struct btrfs_block_group_cache,
+ cache_node);
+ rb_erase(&block_group->cache_node,
+ &info->block_group_cache_tree);
+ spin_unlock(&info->block_group_cache_lock);
+
+ btrfs_remove_free_space_cache(block_group);
+ down_write(&block_group->space_info->groups_sem);
+ list_del(&block_group->list);
+ up_write(&block_group->space_info->groups_sem);
+
+ WARN_ON(atomic_read(&block_group->count) != 1);
+ kfree(block_group);
+
+ spin_lock(&info->block_group_cache_lock);
+ }
+ spin_unlock(&info->block_group_cache_lock);
+ return 0;
+}
+
+int btrfs_read_block_groups(struct btrfs_root *root)
+{
+ struct btrfs_path *path;
+ int ret;
+ struct btrfs_block_group_cache *cache;
+ struct btrfs_fs_info *info = root->fs_info;
+ struct btrfs_space_info *space_info;
+ struct btrfs_key key;
+ struct btrfs_key found_key;
+ struct extent_buffer *leaf;
+
+ root = info->extent_root;
+ key.objectid = 0;
+ key.offset = 0;
+ btrfs_set_key_type(&key, BTRFS_BLOCK_GROUP_ITEM_KEY);
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ while (1) {
+ ret = find_first_block_group(root, path, &key);
+ if (ret > 0) {
+ ret = 0;
+ goto error;
+ }
+ if (ret != 0)
+ goto error;
+
+ leaf = path->nodes[0];
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+ cache = kzalloc(sizeof(*cache), GFP_NOFS);
+ if (!cache) {
+ ret = -ENOMEM;
+ break;
+ }
+
+ atomic_set(&cache->count, 1);
+ spin_lock_init(&cache->lock);
+ mutex_init(&cache->alloc_mutex);
+ mutex_init(&cache->cache_mutex);
+ INIT_LIST_HEAD(&cache->list);
+ read_extent_buffer(leaf, &cache->item,
+ btrfs_item_ptr_offset(leaf, path->slots[0]),
+ sizeof(cache->item));
+ memcpy(&cache->key, &found_key, sizeof(found_key));
+
+ key.objectid = found_key.objectid + found_key.offset;
+ btrfs_release_path(root, path);
+ cache->flags = btrfs_block_group_flags(&cache->item);
+
+ ret = update_space_info(info, cache->flags, found_key.offset,
+ btrfs_block_group_used(&cache->item),
+ &space_info);
+ BUG_ON(ret);
+ cache->space_info = space_info;
+ down_write(&space_info->groups_sem);
+ list_add_tail(&cache->list, &space_info->block_groups);
+ up_write(&space_info->groups_sem);
+
+ ret = btrfs_add_block_group_cache(root->fs_info, cache);
+ BUG_ON(ret);
+
+ set_avail_alloc_bits(root->fs_info, cache->flags);
+ if (btrfs_chunk_readonly(root, cache->key.objectid))
+ set_block_group_readonly(cache);
+ }
+ ret = 0;
+error:
+ btrfs_free_path(path);
+ return ret;
+}
+
+int btrfs_make_block_group(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 bytes_used,
+ u64 type, u64 chunk_objectid, u64 chunk_offset,
+ u64 size)
+{
+ int ret;
+ struct btrfs_root *extent_root;
+ struct btrfs_block_group_cache *cache;
+
+ extent_root = root->fs_info->extent_root;
+
+ root->fs_info->last_trans_new_blockgroup = trans->transid;
+
+ cache = kzalloc(sizeof(*cache), GFP_NOFS);
+ if (!cache)
+ return -ENOMEM;
+
+ cache->key.objectid = chunk_offset;
+ cache->key.offset = size;
+ cache->key.type = BTRFS_BLOCK_GROUP_ITEM_KEY;
+ atomic_set(&cache->count, 1);
+ spin_lock_init(&cache->lock);
+ mutex_init(&cache->alloc_mutex);
+ mutex_init(&cache->cache_mutex);
+ INIT_LIST_HEAD(&cache->list);
+
+ btrfs_set_block_group_used(&cache->item, bytes_used);
+ btrfs_set_block_group_chunk_objectid(&cache->item, chunk_objectid);
+ cache->flags = type;
+ btrfs_set_block_group_flags(&cache->item, type);
+
+ ret = update_space_info(root->fs_info, cache->flags, size, bytes_used,
+ &cache->space_info);
+ BUG_ON(ret);
+ down_write(&cache->space_info->groups_sem);
+ list_add_tail(&cache->list, &cache->space_info->block_groups);
+ up_write(&cache->space_info->groups_sem);
+
+ ret = btrfs_add_block_group_cache(root->fs_info, cache);
+ BUG_ON(ret);
+
+ ret = btrfs_insert_item(trans, extent_root, &cache->key, &cache->item,
+ sizeof(cache->item));
+ BUG_ON(ret);
+
+ finish_current_insert(trans, extent_root, 0);
+ ret = del_pending_extents(trans, extent_root, 0);
+ BUG_ON(ret);
+ set_avail_alloc_bits(extent_root->fs_info, type);
+
+ return 0;
+}
+
+int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 group_start)
+{
+ struct btrfs_path *path;
+ struct btrfs_block_group_cache *block_group;
+ struct btrfs_key key;
+ int ret;
+
+ root = root->fs_info->extent_root;
+
+ block_group = btrfs_lookup_block_group(root->fs_info, group_start);
+ BUG_ON(!block_group);
+ BUG_ON(!block_group->ro);
+
+ memcpy(&key, &block_group->key, sizeof(key));
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+
+ btrfs_remove_free_space_cache(block_group);
+ rb_erase(&block_group->cache_node,
+ &root->fs_info->block_group_cache_tree);
+ down_write(&block_group->space_info->groups_sem);
+ list_del(&block_group->list);
+ up_write(&block_group->space_info->groups_sem);
+
+ spin_lock(&block_group->space_info->lock);
+ block_group->space_info->total_bytes -= block_group->key.offset;
+ block_group->space_info->bytes_readonly -= block_group->key.offset;
+ spin_unlock(&block_group->space_info->lock);
+ block_group->space_info->full = 0;
+
+ put_block_group(block_group);
+ put_block_group(block_group);
+
+ ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
+ if (ret > 0)
+ ret = -EIO;
+ if (ret < 0)
+ goto out;
+
+ ret = btrfs_del_item(trans, root, path);
+out:
+ btrfs_free_path(path);
+ return ret;
+}
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
new file mode 100644
index 0000000..e086d40
--- /dev/null
+++ b/fs/btrfs/extent_io.c
@@ -0,0 +1,3717 @@
+#include <linux/bitops.h>
+#include <linux/slab.h>
+#include <linux/bio.h>
+#include <linux/mm.h>
+#include <linux/gfp.h>
+#include <linux/pagemap.h>
+#include <linux/page-flags.h>
+#include <linux/module.h>
+#include <linux/spinlock.h>
+#include <linux/blkdev.h>
+#include <linux/swap.h>
+#include <linux/version.h>
+#include <linux/writeback.h>
+#include <linux/pagevec.h>
+#include "extent_io.h"
+#include "extent_map.h"
+#include "compat.h"
+#include "ctree.h"
+#include "btrfs_inode.h"
+
+/* temporary define until extent_map moves out of btrfs */
+struct kmem_cache *btrfs_cache_create(const char *name, size_t size,
+ unsigned long extra_flags,
+ void (*ctor)(void *, struct kmem_cache *,
+ unsigned long));
+
+static struct kmem_cache *extent_state_cache;
+static struct kmem_cache *extent_buffer_cache;
+
+static LIST_HEAD(buffers);
+static LIST_HEAD(states);
+
+#define LEAK_DEBUG 0
+#ifdef LEAK_DEBUG
+static DEFINE_SPINLOCK(leak_lock);
+#endif
+
+#define BUFFER_LRU_MAX 64
+
+struct tree_entry {
+ u64 start;
+ u64 end;
+ struct rb_node rb_node;
+};
+
+struct extent_page_data {
+ struct bio *bio;
+ struct extent_io_tree *tree;
+ get_extent_t *get_extent;
+
+ /* tells writepage not to lock the state bits for this range
+ * it still does the unlocking
+ */
+ int extent_locked;
+};
+
+int __init extent_io_init(void)
+{
+ extent_state_cache = btrfs_cache_create("extent_state",
+ sizeof(struct extent_state), 0,
+ NULL);
+ if (!extent_state_cache)
+ return -ENOMEM;
+
+ extent_buffer_cache = btrfs_cache_create("extent_buffers",
+ sizeof(struct extent_buffer), 0,
+ NULL);
+ if (!extent_buffer_cache)
+ goto free_state_cache;
+ return 0;
+
+free_state_cache:
+ kmem_cache_destroy(extent_state_cache);
+ return -ENOMEM;
+}
+
+void extent_io_exit(void)
+{
+ struct extent_state *state;
+ struct extent_buffer *eb;
+
+ while (!list_empty(&states)) {
+ state = list_entry(states.next, struct extent_state, leak_list);
+ printk(KERN_ERR "btrfs state leak: start %llu end %llu "
+ "state %lu in tree %p refs %d\n",
+ (unsigned long long)state->start,
+ (unsigned long long)state->end,
+ state->state, state->tree, atomic_read(&state->refs));
+ list_del(&state->leak_list);
+ kmem_cache_free(extent_state_cache, state);
+
+ }
+
+ while (!list_empty(&buffers)) {
+ eb = list_entry(buffers.next, struct extent_buffer, leak_list);
+ printk(KERN_ERR "btrfs buffer leak start %llu len %lu "
+ "refs %d\n", (unsigned long long)eb->start,
+ eb->len, atomic_read(&eb->refs));
+ list_del(&eb->leak_list);
+ kmem_cache_free(extent_buffer_cache, eb);
+ }
+ if (extent_state_cache)
+ kmem_cache_destroy(extent_state_cache);
+ if (extent_buffer_cache)
+ kmem_cache_destroy(extent_buffer_cache);
+}
+
+void extent_io_tree_init(struct extent_io_tree *tree,
+ struct address_space *mapping, gfp_t mask)
+{
+ tree->state.rb_node = NULL;
+ tree->buffer.rb_node = NULL;
+ tree->ops = NULL;
+ tree->dirty_bytes = 0;
+ spin_lock_init(&tree->lock);
+ spin_lock_init(&tree->buffer_lock);
+ tree->mapping = mapping;
+}
+
+static struct extent_state *alloc_extent_state(gfp_t mask)
+{
+ struct extent_state *state;
+#ifdef LEAK_DEBUG
+ unsigned long flags;
+#endif
+
+ state = kmem_cache_alloc(extent_state_cache, mask);
+ if (!state)
+ return state;
+ state->state = 0;
+ state->private = 0;
+ state->tree = NULL;
+#ifdef LEAK_DEBUG
+ spin_lock_irqsave(&leak_lock, flags);
+ list_add(&state->leak_list, &states);
+ spin_unlock_irqrestore(&leak_lock, flags);
+#endif
+ atomic_set(&state->refs, 1);
+ init_waitqueue_head(&state->wq);
+ return state;
+}
+
+static void free_extent_state(struct extent_state *state)
+{
+ if (!state)
+ return;
+ if (atomic_dec_and_test(&state->refs)) {
+#ifdef LEAK_DEBUG
+ unsigned long flags;
+#endif
+ WARN_ON(state->tree);
+#ifdef LEAK_DEBUG
+ spin_lock_irqsave(&leak_lock, flags);
+ list_del(&state->leak_list);
+ spin_unlock_irqrestore(&leak_lock, flags);
+#endif
+ kmem_cache_free(extent_state_cache, state);
+ }
+}
+
+static struct rb_node *tree_insert(struct rb_root *root, u64 offset,
+ struct rb_node *node)
+{
+ struct rb_node **p = &root->rb_node;
+ struct rb_node *parent = NULL;
+ struct tree_entry *entry;
+
+ while (*p) {
+ parent = *p;
+ entry = rb_entry(parent, struct tree_entry, rb_node);
+
+ if (offset < entry->start)
+ p = &(*p)->rb_left;
+ else if (offset > entry->end)
+ p = &(*p)->rb_right;
+ else
+ return parent;
+ }
+
+ entry = rb_entry(node, struct tree_entry, rb_node);
+ rb_link_node(node, parent, p);
+ rb_insert_color(node, root);
+ return NULL;
+}
+
+static struct rb_node *__etree_search(struct extent_io_tree *tree, u64 offset,
+ struct rb_node **prev_ret,
+ struct rb_node **next_ret)
+{
+ struct rb_root *root = &tree->state;
+ struct rb_node *n = root->rb_node;
+ struct rb_node *prev = NULL;
+ struct rb_node *orig_prev = NULL;
+ struct tree_entry *entry;
+ struct tree_entry *prev_entry = NULL;
+
+ while (n) {
+ entry = rb_entry(n, struct tree_entry, rb_node);
+ prev = n;
+ prev_entry = entry;
+
+ if (offset < entry->start)
+ n = n->rb_left;
+ else if (offset > entry->end)
+ n = n->rb_right;
+ else
+ return n;
+ }
+
+ if (prev_ret) {
+ orig_prev = prev;
+ while (prev && offset > prev_entry->end) {
+ prev = rb_next(prev);
+ prev_entry = rb_entry(prev, struct tree_entry, rb_node);
+ }
+ *prev_ret = prev;
+ prev = orig_prev;
+ }
+
+ if (next_ret) {
+ prev_entry = rb_entry(prev, struct tree_entry, rb_node);
+ while (prev && offset < prev_entry->start) {
+ prev = rb_prev(prev);
+ prev_entry = rb_entry(prev, struct tree_entry, rb_node);
+ }
+ *next_ret = prev;
+ }
+ return NULL;
+}
+
+static inline struct rb_node *tree_search(struct extent_io_tree *tree,
+ u64 offset)
+{
+ struct rb_node *prev = NULL;
+ struct rb_node *ret;
+
+ ret = __etree_search(tree, offset, &prev, NULL);
+ if (!ret)
+ return prev;
+ return ret;
+}
+
+static struct extent_buffer *buffer_tree_insert(struct extent_io_tree *tree,
+ u64 offset, struct rb_node *node)
+{
+ struct rb_root *root = &tree->buffer;
+ struct rb_node **p = &root->rb_node;
+ struct rb_node *parent = NULL;
+ struct extent_buffer *eb;
+
+ while (*p) {
+ parent = *p;
+ eb = rb_entry(parent, struct extent_buffer, rb_node);
+
+ if (offset < eb->start)
+ p = &(*p)->rb_left;
+ else if (offset > eb->start)
+ p = &(*p)->rb_right;
+ else
+ return eb;
+ }
+
+ rb_link_node(node, parent, p);
+ rb_insert_color(node, root);
+ return NULL;
+}
+
+static struct extent_buffer *buffer_search(struct extent_io_tree *tree,
+ u64 offset)
+{
+ struct rb_root *root = &tree->buffer;
+ struct rb_node *n = root->rb_node;
+ struct extent_buffer *eb;
+
+ while (n) {
+ eb = rb_entry(n, struct extent_buffer, rb_node);
+ if (offset < eb->start)
+ n = n->rb_left;
+ else if (offset > eb->start)
+ n = n->rb_right;
+ else
+ return eb;
+ }
+ return NULL;
+}
+
+/*
+ * utility function to look for merge candidates inside a given range.
+ * Any extents with matching state are merged together into a single
+ * extent in the tree. Extents with EXTENT_IO in their state field
+ * are not merged because the end_io handlers need to be able to do
+ * operations on them without sleeping (or doing allocations/splits).
+ *
+ * This should be called with the tree lock held.
+ */
+static int merge_state(struct extent_io_tree *tree,
+ struct extent_state *state)
+{
+ struct extent_state *other;
+ struct rb_node *other_node;
+
+ if (state->state & (EXTENT_IOBITS | EXTENT_BOUNDARY))
+ return 0;
+
+ other_node = rb_prev(&state->rb_node);
+ if (other_node) {
+ other = rb_entry(other_node, struct extent_state, rb_node);
+ if (other->end == state->start - 1 &&
+ other->state == state->state) {
+ state->start = other->start;
+ other->tree = NULL;
+ rb_erase(&other->rb_node, &tree->state);
+ free_extent_state(other);
+ }
+ }
+ other_node = rb_next(&state->rb_node);
+ if (other_node) {
+ other = rb_entry(other_node, struct extent_state, rb_node);
+ if (other->start == state->end + 1 &&
+ other->state == state->state) {
+ other->start = state->start;
+ state->tree = NULL;
+ rb_erase(&state->rb_node, &tree->state);
+ free_extent_state(state);
+ }
+ }
+ return 0;
+}
+
+static void set_state_cb(struct extent_io_tree *tree,
+ struct extent_state *state,
+ unsigned long bits)
+{
+ if (tree->ops && tree->ops->set_bit_hook) {
+ tree->ops->set_bit_hook(tree->mapping->host, state->start,
+ state->end, state->state, bits);
+ }
+}
+
+static void clear_state_cb(struct extent_io_tree *tree,
+ struct extent_state *state,
+ unsigned long bits)
+{
+ if (tree->ops && tree->ops->clear_bit_hook) {
+ tree->ops->clear_bit_hook(tree->mapping->host, state->start,
+ state->end, state->state, bits);
+ }
+}
+
+/*
+ * insert an extent_state struct into the tree. 'bits' are set on the
+ * struct before it is inserted.
+ *
+ * This may return -EEXIST if the extent is already there, in which case the
+ * state struct is freed.
+ *
+ * The tree lock is not taken internally. This is a utility function and
+ * probably isn't what you want to call (see set/clear_extent_bit).
+ */
+static int insert_state(struct extent_io_tree *tree,
+ struct extent_state *state, u64 start, u64 end,
+ int bits)
+{
+ struct rb_node *node;
+
+ if (end < start) {
+ printk(KERN_ERR "btrfs end < start %llu %llu\n",
+ (unsigned long long)end,
+ (unsigned long long)start);
+ WARN_ON(1);
+ }
+ if (bits & EXTENT_DIRTY)
+ tree->dirty_bytes += end - start + 1;
+ set_state_cb(tree, state, bits);
+ state->state |= bits;
+ state->start = start;
+ state->end = end;
+ node = tree_insert(&tree->state, end, &state->rb_node);
+ if (node) {
+ struct extent_state *found;
+ found = rb_entry(node, struct extent_state, rb_node);
+ printk(KERN_ERR "btrfs found node %llu %llu on insert of "
+ "%llu %llu\n", (unsigned long long)found->start,
+ (unsigned long long)found->end,
+ (unsigned long long)start, (unsigned long long)end);
+ free_extent_state(state);
+ return -EEXIST;
+ }
+ state->tree = tree;
+ merge_state(tree, state);
+ return 0;
+}
+
+/*
+ * split a given extent state struct in two, inserting the preallocated
+ * struct 'prealloc' as the newly created second half. 'split' indicates an
+ * offset inside 'orig' where it should be split.
+ *
+ * Before calling,
+ * the tree has 'orig' at [orig->start, orig->end]. After calling, there
+ * are two extent state structs in the tree:
+ * prealloc: [orig->start, split - 1]
+ * orig: [ split, orig->end ]
+ *
+ * The tree locks are not taken by this function. They need to be held
+ * by the caller.
+ */
+static int split_state(struct extent_io_tree *tree, struct extent_state *orig,
+ struct extent_state *prealloc, u64 split)
+{
+ struct rb_node *node;
+ prealloc->start = orig->start;
+ prealloc->end = split - 1;
+ prealloc->state = orig->state;
+ orig->start = split;
+
+ node = tree_insert(&tree->state, prealloc->end, &prealloc->rb_node);
+ if (node) {
+ struct extent_state *found;
+ found = rb_entry(node, struct extent_state, rb_node);
+ free_extent_state(prealloc);
+ return -EEXIST;
+ }
+ prealloc->tree = tree;
+ return 0;
+}
+
+/*
+ * utility function to clear some bits in an extent state struct.
+ * it will optionally wake up any one waiting on this state (wake == 1), or
+ * forcibly remove the state from the tree (delete == 1).
+ *
+ * If no bits are set on the state struct after clearing things, the
+ * struct is freed and removed from the tree
+ */
+static int clear_state_bit(struct extent_io_tree *tree,
+ struct extent_state *state, int bits, int wake,
+ int delete)
+{
+ int ret = state->state & bits;
+
+ if ((bits & EXTENT_DIRTY) && (state->state & EXTENT_DIRTY)) {
+ u64 range = state->end - state->start + 1;
+ WARN_ON(range > tree->dirty_bytes);
+ tree->dirty_bytes -= range;
+ }
+ clear_state_cb(tree, state, bits);
+ state->state &= ~bits;
+ if (wake)
+ wake_up(&state->wq);
+ if (delete || state->state == 0) {
+ if (state->tree) {
+ clear_state_cb(tree, state, state->state);
+ rb_erase(&state->rb_node, &tree->state);
+ state->tree = NULL;
+ free_extent_state(state);
+ } else {
+ WARN_ON(1);
+ }
+ } else {
+ merge_state(tree, state);
+ }
+ return ret;
+}
+
+/*
+ * clear some bits on a range in the tree. This may require splitting
+ * or inserting elements in the tree, so the gfp mask is used to
+ * indicate which allocations or sleeping are allowed.
+ *
+ * pass 'wake' == 1 to kick any sleepers, and 'delete' == 1 to remove
+ * the given range from the tree regardless of state (ie for truncate).
+ *
+ * the range [start, end] is inclusive.
+ *
+ * This takes the tree lock, and returns < 0 on error, > 0 if any of the
+ * bits were already set, or zero if none of the bits were already set.
+ */
+int clear_extent_bit(struct extent_io_tree *tree, u64 start, u64 end,
+ int bits, int wake, int delete, gfp_t mask)
+{
+ struct extent_state *state;
+ struct extent_state *prealloc = NULL;
+ struct rb_node *node;
+ int err;
+ int set = 0;
+
+again:
+ if (!prealloc && (mask & __GFP_WAIT)) {
+ prealloc = alloc_extent_state(mask);
+ if (!prealloc)
+ return -ENOMEM;
+ }
+
+ spin_lock(&tree->lock);
+ /*
+ * this search will find the extents that end after
+ * our range starts
+ */
+ node = tree_search(tree, start);
+ if (!node)
+ goto out;
+ state = rb_entry(node, struct extent_state, rb_node);
+ if (state->start > end)
+ goto out;
+ WARN_ON(state->end < start);
+
+ /*
+ * | ---- desired range ---- |
+ * | state | or
+ * | ------------- state -------------- |
+ *
+ * We need to split the extent we found, and may flip
+ * bits on second half.
+ *
+ * If the extent we found extends past our range, we
+ * just split and search again. It'll get split again
+ * the next time though.
+ *
+ * If the extent we found is inside our range, we clear
+ * the desired bit on it.
+ */
+
+ if (state->start < start) {
+ if (!prealloc)
+ prealloc = alloc_extent_state(GFP_ATOMIC);
+ err = split_state(tree, state, prealloc, start);
+ BUG_ON(err == -EEXIST);
+ prealloc = NULL;
+ if (err)
+ goto out;
+ if (state->end <= end) {
+ start = state->end + 1;
+ set |= clear_state_bit(tree, state, bits,
+ wake, delete);
+ } else {
+ start = state->start;
+ }
+ goto search_again;
+ }
+ /*
+ * | ---- desired range ---- |
+ * | state |
+ * We need to split the extent, and clear the bit
+ * on the first half
+ */
+ if (state->start <= end && state->end > end) {
+ if (!prealloc)
+ prealloc = alloc_extent_state(GFP_ATOMIC);
+ err = split_state(tree, state, prealloc, end + 1);
+ BUG_ON(err == -EEXIST);
+
+ if (wake)
+ wake_up(&state->wq);
+ set |= clear_state_bit(tree, prealloc, bits,
+ wake, delete);
+ prealloc = NULL;
+ goto out;
+ }
+
+ start = state->end + 1;
+ set |= clear_state_bit(tree, state, bits, wake, delete);
+ goto search_again;
+
+out:
+ spin_unlock(&tree->lock);
+ if (prealloc)
+ free_extent_state(prealloc);
+
+ return set;
+
+search_again:
+ if (start > end)
+ goto out;
+ spin_unlock(&tree->lock);
+ if (mask & __GFP_WAIT)
+ cond_resched();
+ goto again;
+}
+
+static int wait_on_state(struct extent_io_tree *tree,
+ struct extent_state *state)
+ __releases(tree->lock)
+ __acquires(tree->lock)
+{
+ DEFINE_WAIT(wait);
+ prepare_to_wait(&state->wq, &wait, TASK_UNINTERRUPTIBLE);
+ spin_unlock(&tree->lock);
+ schedule();
+ spin_lock(&tree->lock);
+ finish_wait(&state->wq, &wait);
+ return 0;
+}
+
+/*
+ * waits for one or more bits to clear on a range in the state tree.
+ * The range [start, end] is inclusive.
+ * The tree lock is taken by this function
+ */
+int wait_extent_bit(struct extent_io_tree *tree, u64 start, u64 end, int bits)
+{
+ struct extent_state *state;
+ struct rb_node *node;
+
+ spin_lock(&tree->lock);
+again:
+ while (1) {
+ /*
+ * this search will find all the extents that end after
+ * our range starts
+ */
+ node = tree_search(tree, start);
+ if (!node)
+ break;
+
+ state = rb_entry(node, struct extent_state, rb_node);
+
+ if (state->start > end)
+ goto out;
+
+ if (state->state & bits) {
+ start = state->start;
+ atomic_inc(&state->refs);
+ wait_on_state(tree, state);
+ free_extent_state(state);
+ goto again;
+ }
+ start = state->end + 1;
+
+ if (start > end)
+ break;
+
+ if (need_resched()) {
+ spin_unlock(&tree->lock);
+ cond_resched();
+ spin_lock(&tree->lock);
+ }
+ }
+out:
+ spin_unlock(&tree->lock);
+ return 0;
+}
+
+static void set_state_bits(struct extent_io_tree *tree,
+ struct extent_state *state,
+ int bits)
+{
+ if ((bits & EXTENT_DIRTY) && !(state->state & EXTENT_DIRTY)) {
+ u64 range = state->end - state->start + 1;
+ tree->dirty_bytes += range;
+ }
+ set_state_cb(tree, state, bits);
+ state->state |= bits;
+}
+
+/*
+ * set some bits on a range in the tree. This may require allocations
+ * or sleeping, so the gfp mask is used to indicate what is allowed.
+ *
+ * If 'exclusive' == 1, this will fail with -EEXIST if some part of the
+ * range already has the desired bits set. The start of the existing
+ * range is returned in failed_start in this case.
+ *
+ * [start, end] is inclusive
+ * This takes the tree lock.
+ */
+static int set_extent_bit(struct extent_io_tree *tree, u64 start, u64 end,
+ int bits, int exclusive, u64 *failed_start,
+ gfp_t mask)
+{
+ struct extent_state *state;
+ struct extent_state *prealloc = NULL;
+ struct rb_node *node;
+ int err = 0;
+ int set;
+ u64 last_start;
+ u64 last_end;
+again:
+ if (!prealloc && (mask & __GFP_WAIT)) {
+ prealloc = alloc_extent_state(mask);
+ if (!prealloc)
+ return -ENOMEM;
+ }
+
+ spin_lock(&tree->lock);
+ /*
+ * this search will find all the extents that end after
+ * our range starts.
+ */
+ node = tree_search(tree, start);
+ if (!node) {
+ err = insert_state(tree, prealloc, start, end, bits);
+ prealloc = NULL;
+ BUG_ON(err == -EEXIST);
+ goto out;
+ }
+
+ state = rb_entry(node, struct extent_state, rb_node);
+ last_start = state->start;
+ last_end = state->end;
+
+ /*
+ * | ---- desired range ---- |
+ * | state |
+ *
+ * Just lock what we found and keep going
+ */
+ if (state->start == start && state->end <= end) {
+ set = state->state & bits;
+ if (set && exclusive) {
+ *failed_start = state->start;
+ err = -EEXIST;
+ goto out;
+ }
+ set_state_bits(tree, state, bits);
+ start = state->end + 1;
+ merge_state(tree, state);
+ goto search_again;
+ }
+
+ /*
+ * | ---- desired range ---- |
+ * | state |
+ * or
+ * | ------------- state -------------- |
+ *
+ * We need to split the extent we found, and may flip bits on
+ * second half.
+ *
+ * If the extent we found extends past our
+ * range, we just split and search again. It'll get split
+ * again the next time though.
+ *
+ * If the extent we found is inside our range, we set the
+ * desired bit on it.
+ */
+ if (state->start < start) {
+ set = state->state & bits;
+ if (exclusive && set) {
+ *failed_start = start;
+ err = -EEXIST;
+ goto out;
+ }
+ err = split_state(tree, state, prealloc, start);
+ BUG_ON(err == -EEXIST);
+ prealloc = NULL;
+ if (err)
+ goto out;
+ if (state->end <= end) {
+ set_state_bits(tree, state, bits);
+ start = state->end + 1;
+ merge_state(tree, state);
+ } else {
+ start = state->start;
+ }
+ goto search_again;
+ }
+ /*
+ * | ---- desired range ---- |
+ * | state | or | state |
+ *
+ * There's a hole, we need to insert something in it and
+ * ignore the extent we found.
+ */
+ if (state->start > start) {
+ u64 this_end;
+ if (end < last_start)
+ this_end = end;
+ else
+ this_end = last_start - 1;
+ err = insert_state(tree, prealloc, start, this_end,
+ bits);
+ prealloc = NULL;
+ BUG_ON(err == -EEXIST);
+ if (err)
+ goto out;
+ start = this_end + 1;
+ goto search_again;
+ }
+ /*
+ * | ---- desired range ---- |
+ * | state |
+ * We need to split the extent, and set the bit
+ * on the first half
+ */
+ if (state->start <= end && state->end > end) {
+ set = state->state & bits;
+ if (exclusive && set) {
+ *failed_start = start;
+ err = -EEXIST;
+ goto out;
+ }
+ err = split_state(tree, state, prealloc, end + 1);
+ BUG_ON(err == -EEXIST);
+
+ set_state_bits(tree, prealloc, bits);
+ merge_state(tree, prealloc);
+ prealloc = NULL;
+ goto out;
+ }
+
+ goto search_again;
+
+out:
+ spin_unlock(&tree->lock);
+ if (prealloc)
+ free_extent_state(prealloc);
+
+ return err;
+
+search_again:
+ if (start > end)
+ goto out;
+ spin_unlock(&tree->lock);
+ if (mask & __GFP_WAIT)
+ cond_resched();
+ goto again;
+}
+
+/* wrappers around set/clear extent bit */
+int set_extent_dirty(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask)
+{
+ return set_extent_bit(tree, start, end, EXTENT_DIRTY, 0, NULL,
+ mask);
+}
+
+int set_extent_ordered(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask)
+{
+ return set_extent_bit(tree, start, end, EXTENT_ORDERED, 0, NULL, mask);
+}
+
+int set_extent_bits(struct extent_io_tree *tree, u64 start, u64 end,
+ int bits, gfp_t mask)
+{
+ return set_extent_bit(tree, start, end, bits, 0, NULL,
+ mask);
+}
+
+int clear_extent_bits(struct extent_io_tree *tree, u64 start, u64 end,
+ int bits, gfp_t mask)
+{
+ return clear_extent_bit(tree, start, end, bits, 0, 0, mask);
+}
+
+int set_extent_delalloc(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask)
+{
+ return set_extent_bit(tree, start, end,
+ EXTENT_DELALLOC | EXTENT_DIRTY,
+ 0, NULL, mask);
+}
+
+int clear_extent_dirty(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask)
+{
+ return clear_extent_bit(tree, start, end,
+ EXTENT_DIRTY | EXTENT_DELALLOC, 0, 0, mask);
+}
+
+int clear_extent_ordered(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask)
+{
+ return clear_extent_bit(tree, start, end, EXTENT_ORDERED, 1, 0, mask);
+}
+
+int set_extent_new(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask)
+{
+ return set_extent_bit(tree, start, end, EXTENT_NEW, 0, NULL,
+ mask);
+}
+
+static int clear_extent_new(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask)
+{
+ return clear_extent_bit(tree, start, end, EXTENT_NEW, 0, 0, mask);
+}
+
+int set_extent_uptodate(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask)
+{
+ return set_extent_bit(tree, start, end, EXTENT_UPTODATE, 0, NULL,
+ mask);
+}
+
+static int clear_extent_uptodate(struct extent_io_tree *tree, u64 start,
+ u64 end, gfp_t mask)
+{
+ return clear_extent_bit(tree, start, end, EXTENT_UPTODATE, 0, 0, mask);
+}
+
+static int set_extent_writeback(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask)
+{
+ return set_extent_bit(tree, start, end, EXTENT_WRITEBACK,
+ 0, NULL, mask);
+}
+
+static int clear_extent_writeback(struct extent_io_tree *tree, u64 start,
+ u64 end, gfp_t mask)
+{
+ return clear_extent_bit(tree, start, end, EXTENT_WRITEBACK, 1, 0, mask);
+}
+
+int wait_on_extent_writeback(struct extent_io_tree *tree, u64 start, u64 end)
+{
+ return wait_extent_bit(tree, start, end, EXTENT_WRITEBACK);
+}
+
+/*
+ * either insert or lock state struct between start and end use mask to tell
+ * us if waiting is desired.
+ */
+int lock_extent(struct extent_io_tree *tree, u64 start, u64 end, gfp_t mask)
+{
+ int err;
+ u64 failed_start;
+ while (1) {
+ err = set_extent_bit(tree, start, end, EXTENT_LOCKED, 1,
+ &failed_start, mask);
+ if (err == -EEXIST && (mask & __GFP_WAIT)) {
+ wait_extent_bit(tree, failed_start, end, EXTENT_LOCKED);
+ start = failed_start;
+ } else {
+ break;
+ }
+ WARN_ON(start > end);
+ }
+ return err;
+}
+
+int try_lock_extent(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask)
+{
+ int err;
+ u64 failed_start;
+
+ err = set_extent_bit(tree, start, end, EXTENT_LOCKED, 1,
+ &failed_start, mask);
+ if (err == -EEXIST) {
+ if (failed_start > start)
+ clear_extent_bit(tree, start, failed_start - 1,
+ EXTENT_LOCKED, 1, 0, mask);
+ return 0;
+ }
+ return 1;
+}
+
+int unlock_extent(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask)
+{
+ return clear_extent_bit(tree, start, end, EXTENT_LOCKED, 1, 0, mask);
+}
+
+/*
+ * helper function to set pages and extents in the tree dirty
+ */
+int set_range_dirty(struct extent_io_tree *tree, u64 start, u64 end)
+{
+ unsigned long index = start >> PAGE_CACHE_SHIFT;
+ unsigned long end_index = end >> PAGE_CACHE_SHIFT;
+ struct page *page;
+
+ while (index <= end_index) {
+ page = find_get_page(tree->mapping, index);
+ BUG_ON(!page);
+ __set_page_dirty_nobuffers(page);
+ page_cache_release(page);
+ index++;
+ }
+ set_extent_dirty(tree, start, end, GFP_NOFS);
+ return 0;
+}
+
+/*
+ * helper function to set both pages and extents in the tree writeback
+ */
+static int set_range_writeback(struct extent_io_tree *tree, u64 start, u64 end)
+{
+ unsigned long index = start >> PAGE_CACHE_SHIFT;
+ unsigned long end_index = end >> PAGE_CACHE_SHIFT;
+ struct page *page;
+
+ while (index <= end_index) {
+ page = find_get_page(tree->mapping, index);
+ BUG_ON(!page);
+ set_page_writeback(page);
+ page_cache_release(page);
+ index++;
+ }
+ set_extent_writeback(tree, start, end, GFP_NOFS);
+ return 0;
+}
+
+/*
+ * find the first offset in the io tree with 'bits' set. zero is
+ * returned if we find something, and *start_ret and *end_ret are
+ * set to reflect the state struct that was found.
+ *
+ * If nothing was found, 1 is returned, < 0 on error
+ */
+int find_first_extent_bit(struct extent_io_tree *tree, u64 start,
+ u64 *start_ret, u64 *end_ret, int bits)
+{
+ struct rb_node *node;
+ struct extent_state *state;
+ int ret = 1;
+
+ spin_lock(&tree->lock);
+ /*
+ * this search will find all the extents that end after
+ * our range starts.
+ */
+ node = tree_search(tree, start);
+ if (!node)
+ goto out;
+
+ while (1) {
+ state = rb_entry(node, struct extent_state, rb_node);
+ if (state->end >= start && (state->state & bits)) {
+ *start_ret = state->start;
+ *end_ret = state->end;
+ ret = 0;
+ break;
+ }
+ node = rb_next(node);
+ if (!node)
+ break;
+ }
+out:
+ spin_unlock(&tree->lock);
+ return ret;
+}
+
+/* find the first state struct with 'bits' set after 'start', and
+ * return it. tree->lock must be held. NULL will returned if
+ * nothing was found after 'start'
+ */
+struct extent_state *find_first_extent_bit_state(struct extent_io_tree *tree,
+ u64 start, int bits)
+{
+ struct rb_node *node;
+ struct extent_state *state;
+
+ /*
+ * this search will find all the extents that end after
+ * our range starts.
+ */
+ node = tree_search(tree, start);
+ if (!node)
+ goto out;
+
+ while (1) {
+ state = rb_entry(node, struct extent_state, rb_node);
+ if (state->end >= start && (state->state & bits))
+ return state;
+
+ node = rb_next(node);
+ if (!node)
+ break;
+ }
+out:
+ return NULL;
+}
+
+/*
+ * find a contiguous range of bytes in the file marked as delalloc, not
+ * more than 'max_bytes'. start and end are used to return the range,
+ *
+ * 1 is returned if we find something, 0 if nothing was in the tree
+ */
+static noinline u64 find_delalloc_range(struct extent_io_tree *tree,
+ u64 *start, u64 *end, u64 max_bytes)
+{
+ struct rb_node *node;
+ struct extent_state *state;
+ u64 cur_start = *start;
+ u64 found = 0;
+ u64 total_bytes = 0;
+
+ spin_lock(&tree->lock);
+
+ /*
+ * this search will find all the extents that end after
+ * our range starts.
+ */
+ node = tree_search(tree, cur_start);
+ if (!node) {
+ if (!found)
+ *end = (u64)-1;
+ goto out;
+ }
+
+ while (1) {
+ state = rb_entry(node, struct extent_state, rb_node);
+ if (found && (state->start != cur_start ||
+ (state->state & EXTENT_BOUNDARY))) {
+ goto out;
+ }
+ if (!(state->state & EXTENT_DELALLOC)) {
+ if (!found)
+ *end = state->end;
+ goto out;
+ }
+ if (!found)
+ *start = state->start;
+ found++;
+ *end = state->end;
+ cur_start = state->end + 1;
+ node = rb_next(node);
+ if (!node)
+ break;
+ total_bytes += state->end - state->start + 1;
+ if (total_bytes >= max_bytes)
+ break;
+ }
+out:
+ spin_unlock(&tree->lock);
+ return found;
+}
+
+static noinline int __unlock_for_delalloc(struct inode *inode,
+ struct page *locked_page,
+ u64 start, u64 end)
+{
+ int ret;
+ struct page *pages[16];
+ unsigned long index = start >> PAGE_CACHE_SHIFT;
+ unsigned long end_index = end >> PAGE_CACHE_SHIFT;
+ unsigned long nr_pages = end_index - index + 1;
+ int i;
+
+ if (index == locked_page->index && end_index == index)
+ return 0;
+
+ while (nr_pages > 0) {
+ ret = find_get_pages_contig(inode->i_mapping, index,
+ min_t(unsigned long, nr_pages,
+ ARRAY_SIZE(pages)), pages);
+ for (i = 0; i < ret; i++) {
+ if (pages[i] != locked_page)
+ unlock_page(pages[i]);
+ page_cache_release(pages[i]);
+ }
+ nr_pages -= ret;
+ index += ret;
+ cond_resched();
+ }
+ return 0;
+}
+
+static noinline int lock_delalloc_pages(struct inode *inode,
+ struct page *locked_page,
+ u64 delalloc_start,
+ u64 delalloc_end)
+{
+ unsigned long index = delalloc_start >> PAGE_CACHE_SHIFT;
+ unsigned long start_index = index;
+ unsigned long end_index = delalloc_end >> PAGE_CACHE_SHIFT;
+ unsigned long pages_locked = 0;
+ struct page *pages[16];
+ unsigned long nrpages;
+ int ret;
+ int i;
+
+ /* the caller is responsible for locking the start index */
+ if (index == locked_page->index && index == end_index)
+ return 0;
+
+ /* skip the page at the start index */
+ nrpages = end_index - index + 1;
+ while (nrpages > 0) {
+ ret = find_get_pages_contig(inode->i_mapping, index,
+ min_t(unsigned long,
+ nrpages, ARRAY_SIZE(pages)), pages);
+ if (ret == 0) {
+ ret = -EAGAIN;
+ goto done;
+ }
+ /* now we have an array of pages, lock them all */
+ for (i = 0; i < ret; i++) {
+ /*
+ * the caller is taking responsibility for
+ * locked_page
+ */
+ if (pages[i] != locked_page) {
+ lock_page(pages[i]);
+ if (!PageDirty(pages[i]) ||
+ pages[i]->mapping != inode->i_mapping) {
+ ret = -EAGAIN;
+ unlock_page(pages[i]);
+ page_cache_release(pages[i]);
+ goto done;
+ }
+ }
+ page_cache_release(pages[i]);
+ pages_locked++;
+ }
+ nrpages -= ret;
+ index += ret;
+ cond_resched();
+ }
+ ret = 0;
+done:
+ if (ret && pages_locked) {
+ __unlock_for_delalloc(inode, locked_page,
+ delalloc_start,
+ ((u64)(start_index + pages_locked - 1)) <<
+ PAGE_CACHE_SHIFT);
+ }
+ return ret;
+}
+
+/*
+ * find a contiguous range of bytes in the file marked as delalloc, not
+ * more than 'max_bytes'. start and end are used to return the range,
+ *
+ * 1 is returned if we find something, 0 if nothing was in the tree
+ */
+static noinline u64 find_lock_delalloc_range(struct inode *inode,
+ struct extent_io_tree *tree,
+ struct page *locked_page,
+ u64 *start, u64 *end,
+ u64 max_bytes)
+{
+ u64 delalloc_start;
+ u64 delalloc_end;
+ u64 found;
+ int ret;
+ int loops = 0;
+
+again:
+ /* step one, find a bunch of delalloc bytes starting at start */
+ delalloc_start = *start;
+ delalloc_end = 0;
+ found = find_delalloc_range(tree, &delalloc_start, &delalloc_end,
+ max_bytes);
+ if (!found || delalloc_end <= *start) {
+ *start = delalloc_start;
+ *end = delalloc_end;
+ return found;
+ }
+
+ /*
+ * start comes from the offset of locked_page. We have to lock
+ * pages in order, so we can't process delalloc bytes before
+ * locked_page
+ */
+ if (delalloc_start < *start)
+ delalloc_start = *start;
+
+ /*
+ * make sure to limit the number of pages we try to lock down
+ * if we're looping.
+ */
+ if (delalloc_end + 1 - delalloc_start > max_bytes && loops)
+ delalloc_end = delalloc_start + PAGE_CACHE_SIZE - 1;
+
+ /* step two, lock all the pages after the page that has start */
+ ret = lock_delalloc_pages(inode, locked_page,
+ delalloc_start, delalloc_end);
+ if (ret == -EAGAIN) {
+ /* some of the pages are gone, lets avoid looping by
+ * shortening the size of the delalloc range we're searching
+ */
+ if (!loops) {
+ unsigned long offset = (*start) & (PAGE_CACHE_SIZE - 1);
+ max_bytes = PAGE_CACHE_SIZE - offset;
+ loops = 1;
+ goto again;
+ } else {
+ found = 0;
+ goto out_failed;
+ }
+ }
+ BUG_ON(ret);
+
+ /* step three, lock the state bits for the whole range */
+ lock_extent(tree, delalloc_start, delalloc_end, GFP_NOFS);
+
+ /* then test to make sure it is all still delalloc */
+ ret = test_range_bit(tree, delalloc_start, delalloc_end,
+ EXTENT_DELALLOC, 1);
+ if (!ret) {
+ unlock_extent(tree, delalloc_start, delalloc_end, GFP_NOFS);
+ __unlock_for_delalloc(inode, locked_page,
+ delalloc_start, delalloc_end);
+ cond_resched();
+ goto again;
+ }
+ *start = delalloc_start;
+ *end = delalloc_end;
+out_failed:
+ return found;
+}
+
+int extent_clear_unlock_delalloc(struct inode *inode,
+ struct extent_io_tree *tree,
+ u64 start, u64 end, struct page *locked_page,
+ int unlock_pages,
+ int clear_unlock,
+ int clear_delalloc, int clear_dirty,
+ int set_writeback,
+ int end_writeback)
+{
+ int ret;
+ struct page *pages[16];
+ unsigned long index = start >> PAGE_CACHE_SHIFT;
+ unsigned long end_index = end >> PAGE_CACHE_SHIFT;
+ unsigned long nr_pages = end_index - index + 1;
+ int i;
+ int clear_bits = 0;
+
+ if (clear_unlock)
+ clear_bits |= EXTENT_LOCKED;
+ if (clear_dirty)
+ clear_bits |= EXTENT_DIRTY;
+
+ if (clear_delalloc)
+ clear_bits |= EXTENT_DELALLOC;
+
+ clear_extent_bit(tree, start, end, clear_bits, 1, 0, GFP_NOFS);
+ if (!(unlock_pages || clear_dirty || set_writeback || end_writeback))
+ return 0;
+
+ while (nr_pages > 0) {
+ ret = find_get_pages_contig(inode->i_mapping, index,
+ min_t(unsigned long,
+ nr_pages, ARRAY_SIZE(pages)), pages);
+ for (i = 0; i < ret; i++) {
+ if (pages[i] == locked_page) {
+ page_cache_release(pages[i]);
+ continue;
+ }
+ if (clear_dirty)
+ clear_page_dirty_for_io(pages[i]);
+ if (set_writeback)
+ set_page_writeback(pages[i]);
+ if (end_writeback)
+ end_page_writeback(pages[i]);
+ if (unlock_pages)
+ unlock_page(pages[i]);
+ page_cache_release(pages[i]);
+ }
+ nr_pages -= ret;
+ index += ret;
+ cond_resched();
+ }
+ return 0;
+}
+
+/*
+ * count the number of bytes in the tree that have a given bit(s)
+ * set. This can be fairly slow, except for EXTENT_DIRTY which is
+ * cached. The total number found is returned.
+ */
+u64 count_range_bits(struct extent_io_tree *tree,
+ u64 *start, u64 search_end, u64 max_bytes,
+ unsigned long bits)
+{
+ struct rb_node *node;
+ struct extent_state *state;
+ u64 cur_start = *start;
+ u64 total_bytes = 0;
+ int found = 0;
+
+ if (search_end <= cur_start) {
+ WARN_ON(1);
+ return 0;
+ }
+
+ spin_lock(&tree->lock);
+ if (cur_start == 0 && bits == EXTENT_DIRTY) {
+ total_bytes = tree->dirty_bytes;
+ goto out;
+ }
+ /*
+ * this search will find all the extents that end after
+ * our range starts.
+ */
+ node = tree_search(tree, cur_start);
+ if (!node)
+ goto out;
+
+ while (1) {
+ state = rb_entry(node, struct extent_state, rb_node);
+ if (state->start > search_end)
+ break;
+ if (state->end >= cur_start && (state->state & bits)) {
+ total_bytes += min(search_end, state->end) + 1 -
+ max(cur_start, state->start);
+ if (total_bytes >= max_bytes)
+ break;
+ if (!found) {
+ *start = state->start;
+ found = 1;
+ }
+ }
+ node = rb_next(node);
+ if (!node)
+ break;
+ }
+out:
+ spin_unlock(&tree->lock);
+ return total_bytes;
+}
+
+#if 0
+/*
+ * helper function to lock both pages and extents in the tree.
+ * pages must be locked first.
+ */
+static int lock_range(struct extent_io_tree *tree, u64 start, u64 end)
+{
+ unsigned long index = start >> PAGE_CACHE_SHIFT;
+ unsigned long end_index = end >> PAGE_CACHE_SHIFT;
+ struct page *page;
+ int err;
+
+ while (index <= end_index) {
+ page = grab_cache_page(tree->mapping, index);
+ if (!page) {
+ err = -ENOMEM;
+ goto failed;
+ }
+ if (IS_ERR(page)) {
+ err = PTR_ERR(page);
+ goto failed;
+ }
+ index++;
+ }
+ lock_extent(tree, start, end, GFP_NOFS);
+ return 0;
+
+failed:
+ /*
+ * we failed above in getting the page at 'index', so we undo here
+ * up to but not including the page at 'index'
+ */
+ end_index = index;
+ index = start >> PAGE_CACHE_SHIFT;
+ while (index < end_index) {
+ page = find_get_page(tree->mapping, index);
+ unlock_page(page);
+ page_cache_release(page);
+ index++;
+ }
+ return err;
+}
+
+/*
+ * helper function to unlock both pages and extents in the tree.
+ */
+static int unlock_range(struct extent_io_tree *tree, u64 start, u64 end)
+{
+ unsigned long index = start >> PAGE_CACHE_SHIFT;
+ unsigned long end_index = end >> PAGE_CACHE_SHIFT;
+ struct page *page;
+
+ while (index <= end_index) {
+ page = find_get_page(tree->mapping, index);
+ unlock_page(page);
+ page_cache_release(page);
+ index++;
+ }
+ unlock_extent(tree, start, end, GFP_NOFS);
+ return 0;
+}
+#endif
+
+/*
+ * set the private field for a given byte offset in the tree. If there isn't
+ * an extent_state there already, this does nothing.
+ */
+int set_state_private(struct extent_io_tree *tree, u64 start, u64 private)
+{
+ struct rb_node *node;
+ struct extent_state *state;
+ int ret = 0;
+
+ spin_lock(&tree->lock);
+ /*
+ * this search will find all the extents that end after
+ * our range starts.
+ */
+ node = tree_search(tree, start);
+ if (!node) {
+ ret = -ENOENT;
+ goto out;
+ }
+ state = rb_entry(node, struct extent_state, rb_node);
+ if (state->start != start) {
+ ret = -ENOENT;
+ goto out;
+ }
+ state->private = private;
+out:
+ spin_unlock(&tree->lock);
+ return ret;
+}
+
+int get_state_private(struct extent_io_tree *tree, u64 start, u64 *private)
+{
+ struct rb_node *node;
+ struct extent_state *state;
+ int ret = 0;
+
+ spin_lock(&tree->lock);
+ /*
+ * this search will find all the extents that end after
+ * our range starts.
+ */
+ node = tree_search(tree, start);
+ if (!node) {
+ ret = -ENOENT;
+ goto out;
+ }
+ state = rb_entry(node, struct extent_state, rb_node);
+ if (state->start != start) {
+ ret = -ENOENT;
+ goto out;
+ }
+ *private = state->private;
+out:
+ spin_unlock(&tree->lock);
+ return ret;
+}
+
+/*
+ * searches a range in the state tree for a given mask.
+ * If 'filled' == 1, this returns 1 only if every extent in the tree
+ * has the bits set. Otherwise, 1 is returned if any bit in the
+ * range is found set.
+ */
+int test_range_bit(struct extent_io_tree *tree, u64 start, u64 end,
+ int bits, int filled)
+{
+ struct extent_state *state = NULL;
+ struct rb_node *node;
+ int bitset = 0;
+
+ spin_lock(&tree->lock);
+ node = tree_search(tree, start);
+ while (node && start <= end) {
+ state = rb_entry(node, struct extent_state, rb_node);
+
+ if (filled && state->start > start) {
+ bitset = 0;
+ break;
+ }
+
+ if (state->start > end)
+ break;
+
+ if (state->state & bits) {
+ bitset = 1;
+ if (!filled)
+ break;
+ } else if (filled) {
+ bitset = 0;
+ break;
+ }
+ start = state->end + 1;
+ if (start > end)
+ break;
+ node = rb_next(node);
+ if (!node) {
+ if (filled)
+ bitset = 0;
+ break;
+ }
+ }
+ spin_unlock(&tree->lock);
+ return bitset;
+}
+
+/*
+ * helper function to set a given page up to date if all the
+ * extents in the tree for that page are up to date
+ */
+static int check_page_uptodate(struct extent_io_tree *tree,
+ struct page *page)
+{
+ u64 start = (u64)page->index << PAGE_CACHE_SHIFT;
+ u64 end = start + PAGE_CACHE_SIZE - 1;
+ if (test_range_bit(tree, start, end, EXTENT_UPTODATE, 1))
+ SetPageUptodate(page);
+ return 0;
+}
+
+/*
+ * helper function to unlock a page if all the extents in the tree
+ * for that page are unlocked
+ */
+static int check_page_locked(struct extent_io_tree *tree,
+ struct page *page)
+{
+ u64 start = (u64)page->index << PAGE_CACHE_SHIFT;
+ u64 end = start + PAGE_CACHE_SIZE - 1;
+ if (!test_range_bit(tree, start, end, EXTENT_LOCKED, 0))
+ unlock_page(page);
+ return 0;
+}
+
+/*
+ * helper function to end page writeback if all the extents
+ * in the tree for that page are done with writeback
+ */
+static int check_page_writeback(struct extent_io_tree *tree,
+ struct page *page)
+{
+ u64 start = (u64)page->index << PAGE_CACHE_SHIFT;
+ u64 end = start + PAGE_CACHE_SIZE - 1;
+ if (!test_range_bit(tree, start, end, EXTENT_WRITEBACK, 0))
+ end_page_writeback(page);
+ return 0;
+}
+
+/* lots and lots of room for performance fixes in the end_bio funcs */
+
+/*
+ * after a writepage IO is done, we need to:
+ * clear the uptodate bits on error
+ * clear the writeback bits in the extent tree for this IO
+ * end_page_writeback if the page has no more pending IO
+ *
+ * Scheduling is not allowed, so the extent state tree is expected
+ * to have one and only one object corresponding to this IO.
+ */
+static void end_bio_extent_writepage(struct bio *bio, int err)
+{
+ int uptodate = err == 0;
+ struct bio_vec *bvec = bio->bi_io_vec + bio->bi_vcnt - 1;
+ struct extent_io_tree *tree;
+ u64 start;
+ u64 end;
+ int whole_page;
+ int ret;
+
+ do {
+ struct page *page = bvec->bv_page;
+ tree = &BTRFS_I(page->mapping->host)->io_tree;
+
+ start = ((u64)page->index << PAGE_CACHE_SHIFT) +
+ bvec->bv_offset;
+ end = start + bvec->bv_len - 1;
+
+ if (bvec->bv_offset == 0 && bvec->bv_len == PAGE_CACHE_SIZE)
+ whole_page = 1;
+ else
+ whole_page = 0;
+
+ if (--bvec >= bio->bi_io_vec)
+ prefetchw(&bvec->bv_page->flags);
+ if (tree->ops && tree->ops->writepage_end_io_hook) {
+ ret = tree->ops->writepage_end_io_hook(page, start,
+ end, NULL, uptodate);
+ if (ret)
+ uptodate = 0;
+ }
+
+ if (!uptodate && tree->ops &&
+ tree->ops->writepage_io_failed_hook) {
+ ret = tree->ops->writepage_io_failed_hook(bio, page,
+ start, end, NULL);
+ if (ret == 0) {
+ uptodate = (err == 0);
+ continue;
+ }
+ }
+
+ if (!uptodate) {
+ clear_extent_uptodate(tree, start, end, GFP_ATOMIC);
+ ClearPageUptodate(page);
+ SetPageError(page);
+ }
+
+ clear_extent_writeback(tree, start, end, GFP_ATOMIC);
+
+ if (whole_page)
+ end_page_writeback(page);
+ else
+ check_page_writeback(tree, page);
+ } while (bvec >= bio->bi_io_vec);
+
+ bio_put(bio);
+}
+
+/*
+ * after a readpage IO is done, we need to:
+ * clear the uptodate bits on error
+ * set the uptodate bits if things worked
+ * set the page up to date if all extents in the tree are uptodate
+ * clear the lock bit in the extent tree
+ * unlock the page if there are no other extents locked for it
+ *
+ * Scheduling is not allowed, so the extent state tree is expected
+ * to have one and only one object corresponding to this IO.
+ */
+static void end_bio_extent_readpage(struct bio *bio, int err)
+{
+ int uptodate = test_bit(BIO_UPTODATE, &bio->bi_flags);
+ struct bio_vec *bvec = bio->bi_io_vec + bio->bi_vcnt - 1;
+ struct extent_io_tree *tree;
+ u64 start;
+ u64 end;
+ int whole_page;
+ int ret;
+
+ if (err)
+ uptodate = 0;
+
+ do {
+ struct page *page = bvec->bv_page;
+ tree = &BTRFS_I(page->mapping->host)->io_tree;
+
+ start = ((u64)page->index << PAGE_CACHE_SHIFT) +
+ bvec->bv_offset;
+ end = start + bvec->bv_len - 1;
+
+ if (bvec->bv_offset == 0 && bvec->bv_len == PAGE_CACHE_SIZE)
+ whole_page = 1;
+ else
+ whole_page = 0;
+
+ if (--bvec >= bio->bi_io_vec)
+ prefetchw(&bvec->bv_page->flags);
+
+ if (uptodate && tree->ops && tree->ops->readpage_end_io_hook) {
+ ret = tree->ops->readpage_end_io_hook(page, start, end,
+ NULL);
+ if (ret)
+ uptodate = 0;
+ }
+ if (!uptodate && tree->ops &&
+ tree->ops->readpage_io_failed_hook) {
+ ret = tree->ops->readpage_io_failed_hook(bio, page,
+ start, end, NULL);
+ if (ret == 0) {
+ uptodate =
+ test_bit(BIO_UPTODATE, &bio->bi_flags);
+ if (err)
+ uptodate = 0;
+ continue;
+ }
+ }
+
+ if (uptodate) {
+ set_extent_uptodate(tree, start, end,
+ GFP_ATOMIC);
+ }
+ unlock_extent(tree, start, end, GFP_ATOMIC);
+
+ if (whole_page) {
+ if (uptodate) {
+ SetPageUptodate(page);
+ } else {
+ ClearPageUptodate(page);
+ SetPageError(page);
+ }
+ unlock_page(page);
+ } else {
+ if (uptodate) {
+ check_page_uptodate(tree, page);
+ } else {
+ ClearPageUptodate(page);
+ SetPageError(page);
+ }
+ check_page_locked(tree, page);
+ }
+ } while (bvec >= bio->bi_io_vec);
+
+ bio_put(bio);
+}
+
+/*
+ * IO done from prepare_write is pretty simple, we just unlock
+ * the structs in the extent tree when done, and set the uptodate bits
+ * as appropriate.
+ */
+static void end_bio_extent_preparewrite(struct bio *bio, int err)
+{
+ const int uptodate = test_bit(BIO_UPTODATE, &bio->bi_flags);
+ struct bio_vec *bvec = bio->bi_io_vec + bio->bi_vcnt - 1;
+ struct extent_io_tree *tree;
+ u64 start;
+ u64 end;
+
+ do {
+ struct page *page = bvec->bv_page;
+ tree = &BTRFS_I(page->mapping->host)->io_tree;
+
+ start = ((u64)page->index << PAGE_CACHE_SHIFT) +
+ bvec->bv_offset;
+ end = start + bvec->bv_len - 1;
+
+ if (--bvec >= bio->bi_io_vec)
+ prefetchw(&bvec->bv_page->flags);
+
+ if (uptodate) {
+ set_extent_uptodate(tree, start, end, GFP_ATOMIC);
+ } else {
+ ClearPageUptodate(page);
+ SetPageError(page);
+ }
+
+ unlock_extent(tree, start, end, GFP_ATOMIC);
+
+ } while (bvec >= bio->bi_io_vec);
+
+ bio_put(bio);
+}
+
+static struct bio *
+extent_bio_alloc(struct block_device *bdev, u64 first_sector, int nr_vecs,
+ gfp_t gfp_flags)
+{
+ struct bio *bio;
+
+ bio = bio_alloc(gfp_flags, nr_vecs);
+
+ if (bio == NULL && (current->flags & PF_MEMALLOC)) {
+ while (!bio && (nr_vecs /= 2))
+ bio = bio_alloc(gfp_flags, nr_vecs);
+ }
+
+ if (bio) {
+ bio->bi_size = 0;
+ bio->bi_bdev = bdev;
+ bio->bi_sector = first_sector;
+ }
+ return bio;
+}
+
+static int submit_one_bio(int rw, struct bio *bio, int mirror_num,
+ unsigned long bio_flags)
+{
+ int ret = 0;
+ struct bio_vec *bvec = bio->bi_io_vec + bio->bi_vcnt - 1;
+ struct page *page = bvec->bv_page;
+ struct extent_io_tree *tree = bio->bi_private;
+ u64 start;
+ u64 end;
+
+ start = ((u64)page->index << PAGE_CACHE_SHIFT) + bvec->bv_offset;
+ end = start + bvec->bv_len - 1;
+
+ bio->bi_private = NULL;
+
+ bio_get(bio);
+
+ if (tree->ops && tree->ops->submit_bio_hook)
+ tree->ops->submit_bio_hook(page->mapping->host, rw, bio,
+ mirror_num, bio_flags);
+ else
+ submit_bio(rw, bio);
+ if (bio_flagged(bio, BIO_EOPNOTSUPP))
+ ret = -EOPNOTSUPP;
+ bio_put(bio);
+ return ret;
+}
+
+static int submit_extent_page(int rw, struct extent_io_tree *tree,
+ struct page *page, sector_t sector,
+ size_t size, unsigned long offset,
+ struct block_device *bdev,
+ struct bio **bio_ret,
+ unsigned long max_pages,
+ bio_end_io_t end_io_func,
+ int mirror_num,
+ unsigned long prev_bio_flags,
+ unsigned long bio_flags)
+{
+ int ret = 0;
+ struct bio *bio;
+ int nr;
+ int contig = 0;
+ int this_compressed = bio_flags & EXTENT_BIO_COMPRESSED;
+ int old_compressed = prev_bio_flags & EXTENT_BIO_COMPRESSED;
+ size_t page_size = min_t(size_t, size, PAGE_CACHE_SIZE);
+
+ if (bio_ret && *bio_ret) {
+ bio = *bio_ret;
+ if (old_compressed)
+ contig = bio->bi_sector == sector;
+ else
+ contig = bio->bi_sector + (bio->bi_size >> 9) ==
+ sector;
+
+ if (prev_bio_flags != bio_flags || !contig ||
+ (tree->ops && tree->ops->merge_bio_hook &&
+ tree->ops->merge_bio_hook(page, offset, page_size, bio,
+ bio_flags)) ||
+ bio_add_page(bio, page, page_size, offset) < page_size) {
+ ret = submit_one_bio(rw, bio, mirror_num,
+ prev_bio_flags);
+ bio = NULL;
+ } else {
+ return 0;
+ }
+ }
+ if (this_compressed)
+ nr = BIO_MAX_PAGES;
+ else
+ nr = bio_get_nr_vecs(bdev);
+
+ bio = extent_bio_alloc(bdev, sector, nr, GFP_NOFS | __GFP_HIGH);
+
+ bio_add_page(bio, page, page_size, offset);
+ bio->bi_end_io = end_io_func;
+ bio->bi_private = tree;
+
+ if (bio_ret)
+ *bio_ret = bio;
+ else
+ ret = submit_one_bio(rw, bio, mirror_num, bio_flags);
+
+ return ret;
+}
+
+void set_page_extent_mapped(struct page *page)
+{
+ if (!PagePrivate(page)) {
+ SetPagePrivate(page);
+ page_cache_get(page);
+ set_page_private(page, EXTENT_PAGE_PRIVATE);
+ }
+}
+
+static void set_page_extent_head(struct page *page, unsigned long len)
+{
+ set_page_private(page, EXTENT_PAGE_PRIVATE_FIRST_PAGE | len << 2);
+}
+
+/*
+ * basic readpage implementation. Locked extent state structs are inserted
+ * into the tree that are removed when the IO is done (by the end_io
+ * handlers)
+ */
+static int __extent_read_full_page(struct extent_io_tree *tree,
+ struct page *page,
+ get_extent_t *get_extent,
+ struct bio **bio, int mirror_num,
+ unsigned long *bio_flags)
+{
+ struct inode *inode = page->mapping->host;
+ u64 start = (u64)page->index << PAGE_CACHE_SHIFT;
+ u64 page_end = start + PAGE_CACHE_SIZE - 1;
+ u64 end;
+ u64 cur = start;
+ u64 extent_offset;
+ u64 last_byte = i_size_read(inode);
+ u64 block_start;
+ u64 cur_end;
+ sector_t sector;
+ struct extent_map *em;
+ struct block_device *bdev;
+ int ret;
+ int nr = 0;
+ size_t page_offset = 0;
+ size_t iosize;
+ size_t disk_io_size;
+ size_t blocksize = inode->i_sb->s_blocksize;
+ unsigned long this_bio_flag = 0;
+
+ set_page_extent_mapped(page);
+
+ end = page_end;
+ lock_extent(tree, start, end, GFP_NOFS);
+
+ if (page->index == last_byte >> PAGE_CACHE_SHIFT) {
+ char *userpage;
+ size_t zero_offset = last_byte & (PAGE_CACHE_SIZE - 1);
+
+ if (zero_offset) {
+ iosize = PAGE_CACHE_SIZE - zero_offset;
+ userpage = kmap_atomic(page, KM_USER0);
+ memset(userpage + zero_offset, 0, iosize);
+ flush_dcache_page(page);
+ kunmap_atomic(userpage, KM_USER0);
+ }
+ }
+ while (cur <= end) {
+ if (cur >= last_byte) {
+ char *userpage;
+ iosize = PAGE_CACHE_SIZE - page_offset;
+ userpage = kmap_atomic(page, KM_USER0);
+ memset(userpage + page_offset, 0, iosize);
+ flush_dcache_page(page);
+ kunmap_atomic(userpage, KM_USER0);
+ set_extent_uptodate(tree, cur, cur + iosize - 1,
+ GFP_NOFS);
+ unlock_extent(tree, cur, cur + iosize - 1, GFP_NOFS);
+ break;
+ }
+ em = get_extent(inode, page, page_offset, cur,
+ end - cur + 1, 0);
+ if (IS_ERR(em) || !em) {
+ SetPageError(page);
+ unlock_extent(tree, cur, end, GFP_NOFS);
+ break;
+ }
+ extent_offset = cur - em->start;
+ BUG_ON(extent_map_end(em) <= cur);
+ BUG_ON(end < cur);
+
+ if (test_bit(EXTENT_FLAG_COMPRESSED, &em->flags))
+ this_bio_flag = EXTENT_BIO_COMPRESSED;
+
+ iosize = min(extent_map_end(em) - cur, end - cur + 1);
+ cur_end = min(extent_map_end(em) - 1, end);
+ iosize = (iosize + blocksize - 1) & ~((u64)blocksize - 1);
+ if (this_bio_flag & EXTENT_BIO_COMPRESSED) {
+ disk_io_size = em->block_len;
+ sector = em->block_start >> 9;
+ } else {
+ sector = (em->block_start + extent_offset) >> 9;
+ disk_io_size = iosize;
+ }
+ bdev = em->bdev;
+ block_start = em->block_start;
+ if (test_bit(EXTENT_FLAG_PREALLOC, &em->flags))
+ block_start = EXTENT_MAP_HOLE;
+ free_extent_map(em);
+ em = NULL;
+
+ /* we've found a hole, just zero and go on */
+ if (block_start == EXTENT_MAP_HOLE) {
+ char *userpage;
+ userpage = kmap_atomic(page, KM_USER0);
+ memset(userpage + page_offset, 0, iosize);
+ flush_dcache_page(page);
+ kunmap_atomic(userpage, KM_USER0);
+
+ set_extent_uptodate(tree, cur, cur + iosize - 1,
+ GFP_NOFS);
+ unlock_extent(tree, cur, cur + iosize - 1, GFP_NOFS);
+ cur = cur + iosize;
+ page_offset += iosize;
+ continue;
+ }
+ /* the get_extent function already copied into the page */
+ if (test_range_bit(tree, cur, cur_end, EXTENT_UPTODATE, 1)) {
+ check_page_uptodate(tree, page);
+ unlock_extent(tree, cur, cur + iosize - 1, GFP_NOFS);
+ cur = cur + iosize;
+ page_offset += iosize;
+ continue;
+ }
+ /* we have an inline extent but it didn't get marked up
+ * to date. Error out
+ */
+ if (block_start == EXTENT_MAP_INLINE) {
+ SetPageError(page);
+ unlock_extent(tree, cur, cur + iosize - 1, GFP_NOFS);
+ cur = cur + iosize;
+ page_offset += iosize;
+ continue;
+ }
+
+ ret = 0;
+ if (tree->ops && tree->ops->readpage_io_hook) {
+ ret = tree->ops->readpage_io_hook(page, cur,
+ cur + iosize - 1);
+ }
+ if (!ret) {
+ unsigned long pnr = (last_byte >> PAGE_CACHE_SHIFT) + 1;
+ pnr -= page->index;
+ ret = submit_extent_page(READ, tree, page,
+ sector, disk_io_size, page_offset,
+ bdev, bio, pnr,
+ end_bio_extent_readpage, mirror_num,
+ *bio_flags,
+ this_bio_flag);
+ nr++;
+ *bio_flags = this_bio_flag;
+ }
+ if (ret)
+ SetPageError(page);
+ cur = cur + iosize;
+ page_offset += iosize;
+ }
+ if (!nr) {
+ if (!PageError(page))
+ SetPageUptodate(page);
+ unlock_page(page);
+ }
+ return 0;
+}
+
+int extent_read_full_page(struct extent_io_tree *tree, struct page *page,
+ get_extent_t *get_extent)
+{
+ struct bio *bio = NULL;
+ unsigned long bio_flags = 0;
+ int ret;
+
+ ret = __extent_read_full_page(tree, page, get_extent, &bio, 0,
+ &bio_flags);
+ if (bio)
+ submit_one_bio(READ, bio, 0, bio_flags);
+ return ret;
+}
+
+/*
+ * the writepage semantics are similar to regular writepage. extent
+ * records are inserted to lock ranges in the tree, and as dirty areas
+ * are found, they are marked writeback. Then the lock bits are removed
+ * and the end_io handler clears the writeback ranges
+ */
+static int __extent_writepage(struct page *page, struct writeback_control *wbc,
+ void *data)
+{
+ struct inode *inode = page->mapping->host;
+ struct extent_page_data *epd = data;
+ struct extent_io_tree *tree = epd->tree;
+ u64 start = (u64)page->index << PAGE_CACHE_SHIFT;
+ u64 delalloc_start;
+ u64 page_end = start + PAGE_CACHE_SIZE - 1;
+ u64 end;
+ u64 cur = start;
+ u64 extent_offset;
+ u64 last_byte = i_size_read(inode);
+ u64 block_start;
+ u64 iosize;
+ u64 unlock_start;
+ sector_t sector;
+ struct extent_map *em;
+ struct block_device *bdev;
+ int ret;
+ int nr = 0;
+ size_t pg_offset = 0;
+ size_t blocksize;
+ loff_t i_size = i_size_read(inode);
+ unsigned long end_index = i_size >> PAGE_CACHE_SHIFT;
+ u64 nr_delalloc;
+ u64 delalloc_end;
+ int page_started;
+ int compressed;
+ unsigned long nr_written = 0;
+
+ WARN_ON(!PageLocked(page));
+ pg_offset = i_size & (PAGE_CACHE_SIZE - 1);
+ if (page->index > end_index ||
+ (page->index == end_index && !pg_offset)) {
+ page->mapping->a_ops->invalidatepage(page, 0);
+ unlock_page(page);
+ return 0;
+ }
+
+ if (page->index == end_index) {
+ char *userpage;
+
+ userpage = kmap_atomic(page, KM_USER0);
+ memset(userpage + pg_offset, 0,
+ PAGE_CACHE_SIZE - pg_offset);
+ kunmap_atomic(userpage, KM_USER0);
+ flush_dcache_page(page);
+ }
+ pg_offset = 0;
+
+ set_page_extent_mapped(page);
+
+ delalloc_start = start;
+ delalloc_end = 0;
+ page_started = 0;
+ if (!epd->extent_locked) {
+ while (delalloc_end < page_end) {
+ nr_delalloc = find_lock_delalloc_range(inode, tree,
+ page,
+ &delalloc_start,
+ &delalloc_end,
+ 128 * 1024 * 1024);
+ if (nr_delalloc == 0) {
+ delalloc_start = delalloc_end + 1;
+ continue;
+ }
+ tree->ops->fill_delalloc(inode, page, delalloc_start,
+ delalloc_end, &page_started,
+ &nr_written);
+ delalloc_start = delalloc_end + 1;
+ }
+
+ /* did the fill delalloc function already unlock and start
+ * the IO?
+ */
+ if (page_started) {
+ ret = 0;
+ goto update_nr_written;
+ }
+ }
+ lock_extent(tree, start, page_end, GFP_NOFS);
+
+ unlock_start = start;
+
+ if (tree->ops && tree->ops->writepage_start_hook) {
+ ret = tree->ops->writepage_start_hook(page, start,
+ page_end);
+ if (ret == -EAGAIN) {
+ unlock_extent(tree, start, page_end, GFP_NOFS);
+ redirty_page_for_writepage(wbc, page);
+ unlock_page(page);
+ ret = 0;
+ goto update_nr_written;
+ }
+ }
+
+ nr_written++;
+
+ end = page_end;
+ if (test_range_bit(tree, start, page_end, EXTENT_DELALLOC, 0))
+ printk(KERN_ERR "btrfs delalloc bits after lock_extent\n");
+
+ if (last_byte <= start) {
+ clear_extent_dirty(tree, start, page_end, GFP_NOFS);
+ unlock_extent(tree, start, page_end, GFP_NOFS);
+ if (tree->ops && tree->ops->writepage_end_io_hook)
+ tree->ops->writepage_end_io_hook(page, start,
+ page_end, NULL, 1);
+ unlock_start = page_end + 1;
+ goto done;
+ }
+
+ set_extent_uptodate(tree, start, page_end, GFP_NOFS);
+ blocksize = inode->i_sb->s_blocksize;
+
+ while (cur <= end) {
+ if (cur >= last_byte) {
+ clear_extent_dirty(tree, cur, page_end, GFP_NOFS);
+ unlock_extent(tree, unlock_start, page_end, GFP_NOFS);
+ if (tree->ops && tree->ops->writepage_end_io_hook)
+ tree->ops->writepage_end_io_hook(page, cur,
+ page_end, NULL, 1);
+ unlock_start = page_end + 1;
+ break;
+ }
+ em = epd->get_extent(inode, page, pg_offset, cur,
+ end - cur + 1, 1);
+ if (IS_ERR(em) || !em) {
+ SetPageError(page);
+ break;
+ }
+
+ extent_offset = cur - em->start;
+ BUG_ON(extent_map_end(em) <= cur);
+ BUG_ON(end < cur);
+ iosize = min(extent_map_end(em) - cur, end - cur + 1);
+ iosize = (iosize + blocksize - 1) & ~((u64)blocksize - 1);
+ sector = (em->block_start + extent_offset) >> 9;
+ bdev = em->bdev;
+ block_start = em->block_start;
+ compressed = test_bit(EXTENT_FLAG_COMPRESSED, &em->flags);
+ free_extent_map(em);
+ em = NULL;
+
+ /*
+ * compressed and inline extents are written through other
+ * paths in the FS
+ */
+ if (compressed || block_start == EXTENT_MAP_HOLE ||
+ block_start == EXTENT_MAP_INLINE) {
+ clear_extent_dirty(tree, cur,
+ cur + iosize - 1, GFP_NOFS);
+
+ unlock_extent(tree, unlock_start, cur + iosize - 1,
+ GFP_NOFS);
+
+ /*
+ * end_io notification does not happen here for
+ * compressed extents
+ */
+ if (!compressed && tree->ops &&
+ tree->ops->writepage_end_io_hook)
+ tree->ops->writepage_end_io_hook(page, cur,
+ cur + iosize - 1,
+ NULL, 1);
+ else if (compressed) {
+ /* we don't want to end_page_writeback on
+ * a compressed extent. this happens
+ * elsewhere
+ */
+ nr++;
+ }
+
+ cur += iosize;
+ pg_offset += iosize;
+ unlock_start = cur;
+ continue;
+ }
+ /* leave this out until we have a page_mkwrite call */
+ if (0 && !test_range_bit(tree, cur, cur + iosize - 1,
+ EXTENT_DIRTY, 0)) {
+ cur = cur + iosize;
+ pg_offset += iosize;
+ continue;
+ }
+
+ clear_extent_dirty(tree, cur, cur + iosize - 1, GFP_NOFS);
+ if (tree->ops && tree->ops->writepage_io_hook) {
+ ret = tree->ops->writepage_io_hook(page, cur,
+ cur + iosize - 1);
+ } else {
+ ret = 0;
+ }
+ if (ret) {
+ SetPageError(page);
+ } else {
+ unsigned long max_nr = end_index + 1;
+
+ set_range_writeback(tree, cur, cur + iosize - 1);
+ if (!PageWriteback(page)) {
+ printk(KERN_ERR "btrfs warning page %lu not "
+ "writeback, cur %llu end %llu\n",
+ page->index, (unsigned long long)cur,
+ (unsigned long long)end);
+ }
+
+ ret = submit_extent_page(WRITE, tree, page, sector,
+ iosize, pg_offset, bdev,
+ &epd->bio, max_nr,
+ end_bio_extent_writepage,
+ 0, 0, 0);
+ if (ret)
+ SetPageError(page);
+ }
+ cur = cur + iosize;
+ pg_offset += iosize;
+ nr++;
+ }
+done:
+ if (nr == 0) {
+ /* make sure the mapping tag for page dirty gets cleared */
+ set_page_writeback(page);
+ end_page_writeback(page);
+ }
+ if (unlock_start <= page_end)
+ unlock_extent(tree, unlock_start, page_end, GFP_NOFS);
+ unlock_page(page);
+
+update_nr_written:
+ wbc->nr_to_write -= nr_written;
+ if (wbc->range_cyclic || (wbc->nr_to_write > 0 &&
+ wbc->range_start == 0 && wbc->range_end == LLONG_MAX))
+ page->mapping->writeback_index = page->index + nr_written;
+ return 0;
+}
+
+/**
+ * write_cache_pages - walk the list of dirty pages of the given address space and write all of them.
+ * @mapping: address space structure to write
+ * @wbc: subtract the number of written pages from *@wbc->nr_to_write
+ * @writepage: function called for each page
+ * @data: data passed to writepage function
+ *
+ * If a page is already under I/O, write_cache_pages() skips it, even
+ * if it's dirty. This is desirable behaviour for memory-cleaning writeback,
+ * but it is INCORRECT for data-integrity system calls such as fsync(). fsync()
+ * and msync() need to guarantee that all the data which was dirty at the time
+ * the call was made get new I/O started against them. If wbc->sync_mode is
+ * WB_SYNC_ALL then we were called for data integrity and we must wait for
+ * existing IO to complete.
+ */
+static int extent_write_cache_pages(struct extent_io_tree *tree,
+ struct address_space *mapping,
+ struct writeback_control *wbc,
+ writepage_t writepage, void *data,
+ void (*flush_fn)(void *))
+{
+ struct backing_dev_info *bdi = mapping->backing_dev_info;
+ int ret = 0;
+ int done = 0;
+ struct pagevec pvec;
+ int nr_pages;
+ pgoff_t index;
+ pgoff_t end; /* Inclusive */
+ int scanned = 0;
+ int range_whole = 0;
+
+ if (wbc->nonblocking && bdi_write_congested(bdi)) {
+ wbc->encountered_congestion = 1;
+ return 0;
+ }
+
+ pagevec_init(&pvec, 0);
+ if (wbc->range_cyclic) {
+ index = mapping->writeback_index; /* Start from prev offset */
+ end = -1;
+ } else {
+ index = wbc->range_start >> PAGE_CACHE_SHIFT;
+ end = wbc->range_end >> PAGE_CACHE_SHIFT;
+ if (wbc->range_start == 0 && wbc->range_end == LLONG_MAX)
+ range_whole = 1;
+ scanned = 1;
+ }
+retry:
+ while (!done && (index <= end) &&
+ (nr_pages = pagevec_lookup_tag(&pvec, mapping, &index,
+ PAGECACHE_TAG_DIRTY, min(end - index,
+ (pgoff_t)PAGEVEC_SIZE-1) + 1))) {
+ unsigned i;
+
+ scanned = 1;
+ for (i = 0; i < nr_pages; i++) {
+ struct page *page = pvec.pages[i];
+
+ /*
+ * At this point we hold neither mapping->tree_lock nor
+ * lock on the page itself: the page may be truncated or
+ * invalidated (changing page->mapping to NULL), or even
+ * swizzled back from swapper_space to tmpfs file
+ * mapping
+ */
+ if (tree->ops && tree->ops->write_cache_pages_lock_hook)
+ tree->ops->write_cache_pages_lock_hook(page);
+ else
+ lock_page(page);
+
+ if (unlikely(page->mapping != mapping)) {
+ unlock_page(page);
+ continue;
+ }
+
+ if (!wbc->range_cyclic && page->index > end) {
+ done = 1;
+ unlock_page(page);
+ continue;
+ }
+
+ if (wbc->sync_mode != WB_SYNC_NONE) {
+ if (PageWriteback(page))
+ flush_fn(data);
+ wait_on_page_writeback(page);
+ }
+
+ if (PageWriteback(page) ||
+ !clear_page_dirty_for_io(page)) {
+ unlock_page(page);
+ continue;
+ }
+
+ ret = (*writepage)(page, wbc, data);
+
+ if (unlikely(ret == AOP_WRITEPAGE_ACTIVATE)) {
+ unlock_page(page);
+ ret = 0;
+ }
+ if (ret || wbc->nr_to_write <= 0)
+ done = 1;
+ if (wbc->nonblocking && bdi_write_congested(bdi)) {
+ wbc->encountered_congestion = 1;
+ done = 1;
+ }
+ }
+ pagevec_release(&pvec);
+ cond_resched();
+ }
+ if (!scanned && !done) {
+ /*
+ * We hit the last page and there is more work to be done: wrap
+ * back to the start of the file
+ */
+ scanned = 1;
+ index = 0;
+ goto retry;
+ }
+ return ret;
+}
+
+static noinline void flush_write_bio(void *data)
+{
+ struct extent_page_data *epd = data;
+ if (epd->bio) {
+ submit_one_bio(WRITE, epd->bio, 0, 0);
+ epd->bio = NULL;
+ }
+}
+
+int extent_write_full_page(struct extent_io_tree *tree, struct page *page,
+ get_extent_t *get_extent,
+ struct writeback_control *wbc)
+{
+ int ret;
+ struct address_space *mapping = page->mapping;
+ struct extent_page_data epd = {
+ .bio = NULL,
+ .tree = tree,
+ .get_extent = get_extent,
+ .extent_locked = 0,
+ };
+ struct writeback_control wbc_writepages = {
+ .bdi = wbc->bdi,
+ .sync_mode = WB_SYNC_NONE,
+ .older_than_this = NULL,
+ .nr_to_write = 64,
+ .range_start = page_offset(page) + PAGE_CACHE_SIZE,
+ .range_end = (loff_t)-1,
+ };
+
+
+ ret = __extent_writepage(page, wbc, &epd);
+
+ extent_write_cache_pages(tree, mapping, &wbc_writepages,
+ __extent_writepage, &epd, flush_write_bio);
+ if (epd.bio)
+ submit_one_bio(WRITE, epd.bio, 0, 0);
+ return ret;
+}
+
+int extent_write_locked_range(struct extent_io_tree *tree, struct inode *inode,
+ u64 start, u64 end, get_extent_t *get_extent,
+ int mode)
+{
+ int ret = 0;
+ struct address_space *mapping = inode->i_mapping;
+ struct page *page;
+ unsigned long nr_pages = (end - start + PAGE_CACHE_SIZE) >>
+ PAGE_CACHE_SHIFT;
+
+ struct extent_page_data epd = {
+ .bio = NULL,
+ .tree = tree,
+ .get_extent = get_extent,
+ .extent_locked = 1,
+ };
+ struct writeback_control wbc_writepages = {
+ .bdi = inode->i_mapping->backing_dev_info,
+ .sync_mode = mode,
+ .older_than_this = NULL,
+ .nr_to_write = nr_pages * 2,
+ .range_start = start,
+ .range_end = end + 1,
+ };
+
+ while (start <= end) {
+ page = find_get_page(mapping, start >> PAGE_CACHE_SHIFT);
+ if (clear_page_dirty_for_io(page))
+ ret = __extent_writepage(page, &wbc_writepages, &epd);
+ else {
+ if (tree->ops && tree->ops->writepage_end_io_hook)
+ tree->ops->writepage_end_io_hook(page, start,
+ start + PAGE_CACHE_SIZE - 1,
+ NULL, 1);
+ unlock_page(page);
+ }
+ page_cache_release(page);
+ start += PAGE_CACHE_SIZE;
+ }
+
+ if (epd.bio)
+ submit_one_bio(WRITE, epd.bio, 0, 0);
+ return ret;
+}
+
+int extent_writepages(struct extent_io_tree *tree,
+ struct address_space *mapping,
+ get_extent_t *get_extent,
+ struct writeback_control *wbc)
+{
+ int ret = 0;
+ struct extent_page_data epd = {
+ .bio = NULL,
+ .tree = tree,
+ .get_extent = get_extent,
+ .extent_locked = 0,
+ };
+
+ ret = extent_write_cache_pages(tree, mapping, wbc,
+ __extent_writepage, &epd,
+ flush_write_bio);
+ if (epd.bio)
+ submit_one_bio(WRITE, epd.bio, 0, 0);
+ return ret;
+}
+
+int extent_readpages(struct extent_io_tree *tree,
+ struct address_space *mapping,
+ struct list_head *pages, unsigned nr_pages,
+ get_extent_t get_extent)
+{
+ struct bio *bio = NULL;
+ unsigned page_idx;
+ struct pagevec pvec;
+ unsigned long bio_flags = 0;
+
+ pagevec_init(&pvec, 0);
+ for (page_idx = 0; page_idx < nr_pages; page_idx++) {
+ struct page *page = list_entry(pages->prev, struct page, lru);
+
+ prefetchw(&page->flags);
+ list_del(&page->lru);
+ /*
+ * what we want to do here is call add_to_page_cache_lru,
+ * but that isn't exported, so we reproduce it here
+ */
+ if (!add_to_page_cache(page, mapping,
+ page->index, GFP_KERNEL)) {
+
+ /* open coding of lru_cache_add, also not exported */
+ page_cache_get(page);
+ if (!pagevec_add(&pvec, page))
+ __pagevec_lru_add_file(&pvec);
+ __extent_read_full_page(tree, page, get_extent,
+ &bio, 0, &bio_flags);
+ }
+ page_cache_release(page);
+ }
+ if (pagevec_count(&pvec))
+ __pagevec_lru_add_file(&pvec);
+ BUG_ON(!list_empty(pages));
+ if (bio)
+ submit_one_bio(READ, bio, 0, bio_flags);
+ return 0;
+}
+
+/*
+ * basic invalidatepage code, this waits on any locked or writeback
+ * ranges corresponding to the page, and then deletes any extent state
+ * records from the tree
+ */
+int extent_invalidatepage(struct extent_io_tree *tree,
+ struct page *page, unsigned long offset)
+{
+ u64 start = ((u64)page->index << PAGE_CACHE_SHIFT);
+ u64 end = start + PAGE_CACHE_SIZE - 1;
+ size_t blocksize = page->mapping->host->i_sb->s_blocksize;
+
+ start += (offset + blocksize - 1) & ~(blocksize - 1);
+ if (start > end)
+ return 0;
+
+ lock_extent(tree, start, end, GFP_NOFS);
+ wait_on_extent_writeback(tree, start, end);
+ clear_extent_bit(tree, start, end,
+ EXTENT_LOCKED | EXTENT_DIRTY | EXTENT_DELALLOC,
+ 1, 1, GFP_NOFS);
+ return 0;
+}
+
+/*
+ * simple commit_write call, set_range_dirty is used to mark both
+ * the pages and the extent records as dirty
+ */
+int extent_commit_write(struct extent_io_tree *tree,
+ struct inode *inode, struct page *page,
+ unsigned from, unsigned to)
+{
+ loff_t pos = ((loff_t)page->index << PAGE_CACHE_SHIFT) + to;
+
+ set_page_extent_mapped(page);
+ set_page_dirty(page);
+
+ if (pos > inode->i_size) {
+ i_size_write(inode, pos);
+ mark_inode_dirty(inode);
+ }
+ return 0;
+}
+
+int extent_prepare_write(struct extent_io_tree *tree,
+ struct inode *inode, struct page *page,
+ unsigned from, unsigned to, get_extent_t *get_extent)
+{
+ u64 page_start = (u64)page->index << PAGE_CACHE_SHIFT;
+ u64 page_end = page_start + PAGE_CACHE_SIZE - 1;
+ u64 block_start;
+ u64 orig_block_start;
+ u64 block_end;
+ u64 cur_end;
+ struct extent_map *em;
+ unsigned blocksize = 1 << inode->i_blkbits;
+ size_t page_offset = 0;
+ size_t block_off_start;
+ size_t block_off_end;
+ int err = 0;
+ int iocount = 0;
+ int ret = 0;
+ int isnew;
+
+ set_page_extent_mapped(page);
+
+ block_start = (page_start + from) & ~((u64)blocksize - 1);
+ block_end = (page_start + to - 1) | (blocksize - 1);
+ orig_block_start = block_start;
+
+ lock_extent(tree, page_start, page_end, GFP_NOFS);
+ while (block_start <= block_end) {
+ em = get_extent(inode, page, page_offset, block_start,
+ block_end - block_start + 1, 1);
+ if (IS_ERR(em) || !em)
+ goto err;
+
+ cur_end = min(block_end, extent_map_end(em) - 1);
+ block_off_start = block_start & (PAGE_CACHE_SIZE - 1);
+ block_off_end = block_off_start + blocksize;
+ isnew = clear_extent_new(tree, block_start, cur_end, GFP_NOFS);
+
+ if (!PageUptodate(page) && isnew &&
+ (block_off_end > to || block_off_start < from)) {
+ void *kaddr;
+
+ kaddr = kmap_atomic(page, KM_USER0);
+ if (block_off_end > to)
+ memset(kaddr + to, 0, block_off_end - to);
+ if (block_off_start < from)
+ memset(kaddr + block_off_start, 0,
+ from - block_off_start);
+ flush_dcache_page(page);
+ kunmap_atomic(kaddr, KM_USER0);
+ }
+ if ((em->block_start != EXTENT_MAP_HOLE &&
+ em->block_start != EXTENT_MAP_INLINE) &&
+ !isnew && !PageUptodate(page) &&
+ (block_off_end > to || block_off_start < from) &&
+ !test_range_bit(tree, block_start, cur_end,
+ EXTENT_UPTODATE, 1)) {
+ u64 sector;
+ u64 extent_offset = block_start - em->start;
+ size_t iosize;
+ sector = (em->block_start + extent_offset) >> 9;
+ iosize = (cur_end - block_start + blocksize) &
+ ~((u64)blocksize - 1);
+ /*
+ * we've already got the extent locked, but we
+ * need to split the state such that our end_bio
+ * handler can clear the lock.
+ */
+ set_extent_bit(tree, block_start,
+ block_start + iosize - 1,
+ EXTENT_LOCKED, 0, NULL, GFP_NOFS);
+ ret = submit_extent_page(READ, tree, page,
+ sector, iosize, page_offset, em->bdev,
+ NULL, 1,
+ end_bio_extent_preparewrite, 0,
+ 0, 0);
+ iocount++;
+ block_start = block_start + iosize;
+ } else {
+ set_extent_uptodate(tree, block_start, cur_end,
+ GFP_NOFS);
+ unlock_extent(tree, block_start, cur_end, GFP_NOFS);
+ block_start = cur_end + 1;
+ }
+ page_offset = block_start & (PAGE_CACHE_SIZE - 1);
+ free_extent_map(em);
+ }
+ if (iocount) {
+ wait_extent_bit(tree, orig_block_start,
+ block_end, EXTENT_LOCKED);
+ }
+ check_page_uptodate(tree, page);
+err:
+ /* FIXME, zero out newly allocated blocks on error */
+ return err;
+}
+
+/*
+ * a helper for releasepage, this tests for areas of the page that
+ * are locked or under IO and drops the related state bits if it is safe
+ * to drop the page.
+ */
+int try_release_extent_state(struct extent_map_tree *map,
+ struct extent_io_tree *tree, struct page *page,
+ gfp_t mask)
+{
+ u64 start = (u64)page->index << PAGE_CACHE_SHIFT;
+ u64 end = start + PAGE_CACHE_SIZE - 1;
+ int ret = 1;
+
+ if (test_range_bit(tree, start, end,
+ EXTENT_IOBITS | EXTENT_ORDERED, 0))
+ ret = 0;
+ else {
+ if ((mask & GFP_NOFS) == GFP_NOFS)
+ mask = GFP_NOFS;
+ clear_extent_bit(tree, start, end, EXTENT_UPTODATE,
+ 1, 1, mask);
+ }
+ return ret;
+}
+
+/*
+ * a helper for releasepage. As long as there are no locked extents
+ * in the range corresponding to the page, both state records and extent
+ * map records are removed
+ */
+int try_release_extent_mapping(struct extent_map_tree *map,
+ struct extent_io_tree *tree, struct page *page,
+ gfp_t mask)
+{
+ struct extent_map *em;
+ u64 start = (u64)page->index << PAGE_CACHE_SHIFT;
+ u64 end = start + PAGE_CACHE_SIZE - 1;
+
+ if ((mask & __GFP_WAIT) &&
+ page->mapping->host->i_size > 16 * 1024 * 1024) {
+ u64 len;
+ while (start <= end) {
+ len = end - start + 1;
+ spin_lock(&map->lock);
+ em = lookup_extent_mapping(map, start, len);
+ if (!em || IS_ERR(em)) {
+ spin_unlock(&map->lock);
+ break;
+ }
+ if (test_bit(EXTENT_FLAG_PINNED, &em->flags) ||
+ em->start != start) {
+ spin_unlock(&map->lock);
+ free_extent_map(em);
+ break;
+ }
+ if (!test_range_bit(tree, em->start,
+ extent_map_end(em) - 1,
+ EXTENT_LOCKED | EXTENT_WRITEBACK |
+ EXTENT_ORDERED,
+ 0)) {
+ remove_extent_mapping(map, em);
+ /* once for the rb tree */
+ free_extent_map(em);
+ }
+ start = extent_map_end(em);
+ spin_unlock(&map->lock);
+
+ /* once for us */
+ free_extent_map(em);
+ }
+ }
+ return try_release_extent_state(map, tree, page, mask);
+}
+
+sector_t extent_bmap(struct address_space *mapping, sector_t iblock,
+ get_extent_t *get_extent)
+{
+ struct inode *inode = mapping->host;
+ u64 start = iblock << inode->i_blkbits;
+ sector_t sector = 0;
+ size_t blksize = (1 << inode->i_blkbits);
+ struct extent_map *em;
+
+ lock_extent(&BTRFS_I(inode)->io_tree, start, start + blksize - 1,
+ GFP_NOFS);
+ em = get_extent(inode, NULL, 0, start, blksize, 0);
+ unlock_extent(&BTRFS_I(inode)->io_tree, start, start + blksize - 1,
+ GFP_NOFS);
+ if (!em || IS_ERR(em))
+ return 0;
+
+ if (em->block_start > EXTENT_MAP_LAST_BYTE)
+ goto out;
+
+ sector = (em->block_start + start - em->start) >> inode->i_blkbits;
+out:
+ free_extent_map(em);
+ return sector;
+}
+
+static inline struct page *extent_buffer_page(struct extent_buffer *eb,
+ unsigned long i)
+{
+ struct page *p;
+ struct address_space *mapping;
+
+ if (i == 0)
+ return eb->first_page;
+ i += eb->start >> PAGE_CACHE_SHIFT;
+ mapping = eb->first_page->mapping;
+ if (!mapping)
+ return NULL;
+
+ /*
+ * extent_buffer_page is only called after pinning the page
+ * by increasing the reference count. So we know the page must
+ * be in the radix tree.
+ */
+ rcu_read_lock();
+ p = radix_tree_lookup(&mapping->page_tree, i);
+ rcu_read_unlock();
+
+ return p;
+}
+
+static inline unsigned long num_extent_pages(u64 start, u64 len)
+{
+ return ((start + len + PAGE_CACHE_SIZE - 1) >> PAGE_CACHE_SHIFT) -
+ (start >> PAGE_CACHE_SHIFT);
+}
+
+static struct extent_buffer *__alloc_extent_buffer(struct extent_io_tree *tree,
+ u64 start,
+ unsigned long len,
+ gfp_t mask)
+{
+ struct extent_buffer *eb = NULL;
+#ifdef LEAK_DEBUG
+ unsigned long flags;
+#endif
+
+ eb = kmem_cache_zalloc(extent_buffer_cache, mask);
+ eb->start = start;
+ eb->len = len;
+ mutex_init(&eb->mutex);
+#ifdef LEAK_DEBUG
+ spin_lock_irqsave(&leak_lock, flags);
+ list_add(&eb->leak_list, &buffers);
+ spin_unlock_irqrestore(&leak_lock, flags);
+#endif
+ atomic_set(&eb->refs, 1);
+
+ return eb;
+}
+
+static void __free_extent_buffer(struct extent_buffer *eb)
+{
+#ifdef LEAK_DEBUG
+ unsigned long flags;
+ spin_lock_irqsave(&leak_lock, flags);
+ list_del(&eb->leak_list);
+ spin_unlock_irqrestore(&leak_lock, flags);
+#endif
+ kmem_cache_free(extent_buffer_cache, eb);
+}
+
+struct extent_buffer *alloc_extent_buffer(struct extent_io_tree *tree,
+ u64 start, unsigned long len,
+ struct page *page0,
+ gfp_t mask)
+{
+ unsigned long num_pages = num_extent_pages(start, len);
+ unsigned long i;
+ unsigned long index = start >> PAGE_CACHE_SHIFT;
+ struct extent_buffer *eb;
+ struct extent_buffer *exists = NULL;
+ struct page *p;
+ struct address_space *mapping = tree->mapping;
+ int uptodate = 1;
+
+ spin_lock(&tree->buffer_lock);
+ eb = buffer_search(tree, start);
+ if (eb) {
+ atomic_inc(&eb->refs);
+ spin_unlock(&tree->buffer_lock);
+ mark_page_accessed(eb->first_page);
+ return eb;
+ }
+ spin_unlock(&tree->buffer_lock);
+
+ eb = __alloc_extent_buffer(tree, start, len, mask);
+ if (!eb)
+ return NULL;
+
+ if (page0) {
+ eb->first_page = page0;
+ i = 1;
+ index++;
+ page_cache_get(page0);
+ mark_page_accessed(page0);
+ set_page_extent_mapped(page0);
+ set_page_extent_head(page0, len);
+ uptodate = PageUptodate(page0);
+ } else {
+ i = 0;
+ }
+ for (; i < num_pages; i++, index++) {
+ p = find_or_create_page(mapping, index, mask | __GFP_HIGHMEM);
+ if (!p) {
+ WARN_ON(1);
+ goto free_eb;
+ }
+ set_page_extent_mapped(p);
+ mark_page_accessed(p);
+ if (i == 0) {
+ eb->first_page = p;
+ set_page_extent_head(p, len);
+ } else {
+ set_page_private(p, EXTENT_PAGE_PRIVATE);
+ }
+ if (!PageUptodate(p))
+ uptodate = 0;
+ unlock_page(p);
+ }
+ if (uptodate)
+ eb->flags |= EXTENT_UPTODATE;
+ eb->flags |= EXTENT_BUFFER_FILLED;
+
+ spin_lock(&tree->buffer_lock);
+ exists = buffer_tree_insert(tree, start, &eb->rb_node);
+ if (exists) {
+ /* add one reference for the caller */
+ atomic_inc(&exists->refs);
+ spin_unlock(&tree->buffer_lock);
+ goto free_eb;
+ }
+ spin_unlock(&tree->buffer_lock);
+
+ /* add one reference for the tree */
+ atomic_inc(&eb->refs);
+ return eb;
+
+free_eb:
+ if (!atomic_dec_and_test(&eb->refs))
+ return exists;
+ for (index = 1; index < i; index++)
+ page_cache_release(extent_buffer_page(eb, index));
+ page_cache_release(extent_buffer_page(eb, 0));
+ __free_extent_buffer(eb);
+ return exists;
+}
+
+struct extent_buffer *find_extent_buffer(struct extent_io_tree *tree,
+ u64 start, unsigned long len,
+ gfp_t mask)
+{
+ struct extent_buffer *eb;
+
+ spin_lock(&tree->buffer_lock);
+ eb = buffer_search(tree, start);
+ if (eb)
+ atomic_inc(&eb->refs);
+ spin_unlock(&tree->buffer_lock);
+
+ if (eb)
+ mark_page_accessed(eb->first_page);
+
+ return eb;
+}
+
+void free_extent_buffer(struct extent_buffer *eb)
+{
+ if (!eb)
+ return;
+
+ if (!atomic_dec_and_test(&eb->refs))
+ return;
+
+ WARN_ON(1);
+}
+
+int clear_extent_buffer_dirty(struct extent_io_tree *tree,
+ struct extent_buffer *eb)
+{
+ int set;
+ unsigned long i;
+ unsigned long num_pages;
+ struct page *page;
+
+ u64 start = eb->start;
+ u64 end = start + eb->len - 1;
+
+ set = clear_extent_dirty(tree, start, end, GFP_NOFS);
+ num_pages = num_extent_pages(eb->start, eb->len);
+
+ for (i = 0; i < num_pages; i++) {
+ page = extent_buffer_page(eb, i);
+ if (!set && !PageDirty(page))
+ continue;
+
+ lock_page(page);
+ if (i == 0)
+ set_page_extent_head(page, eb->len);
+ else
+ set_page_private(page, EXTENT_PAGE_PRIVATE);
+
+ /*
+ * if we're on the last page or the first page and the
+ * block isn't aligned on a page boundary, do extra checks
+ * to make sure we don't clean page that is partially dirty
+ */
+ if ((i == 0 && (eb->start & (PAGE_CACHE_SIZE - 1))) ||
+ ((i == num_pages - 1) &&
+ ((eb->start + eb->len) & (PAGE_CACHE_SIZE - 1)))) {
+ start = (u64)page->index << PAGE_CACHE_SHIFT;
+ end = start + PAGE_CACHE_SIZE - 1;
+ if (test_range_bit(tree, start, end,
+ EXTENT_DIRTY, 0)) {
+ unlock_page(page);
+ continue;
+ }
+ }
+ clear_page_dirty_for_io(page);
+ spin_lock_irq(&page->mapping->tree_lock);
+ if (!PageDirty(page)) {
+ radix_tree_tag_clear(&page->mapping->page_tree,
+ page_index(page),
+ PAGECACHE_TAG_DIRTY);
+ }
+ spin_unlock_irq(&page->mapping->tree_lock);
+ unlock_page(page);
+ }
+ return 0;
+}
+
+int wait_on_extent_buffer_writeback(struct extent_io_tree *tree,
+ struct extent_buffer *eb)
+{
+ return wait_on_extent_writeback(tree, eb->start,
+ eb->start + eb->len - 1);
+}
+
+int set_extent_buffer_dirty(struct extent_io_tree *tree,
+ struct extent_buffer *eb)
+{
+ unsigned long i;
+ unsigned long num_pages;
+
+ num_pages = num_extent_pages(eb->start, eb->len);
+ for (i = 0; i < num_pages; i++) {
+ struct page *page = extent_buffer_page(eb, i);
+ /* writepage may need to do something special for the
+ * first page, we have to make sure page->private is
+ * properly set. releasepage may drop page->private
+ * on us if the page isn't already dirty.
+ */
+ lock_page(page);
+ if (i == 0) {
+ set_page_extent_head(page, eb->len);
+ } else if (PagePrivate(page) &&
+ page->private != EXTENT_PAGE_PRIVATE) {
+ set_page_extent_mapped(page);
+ }
+ __set_page_dirty_nobuffers(extent_buffer_page(eb, i));
+ set_extent_dirty(tree, page_offset(page),
+ page_offset(page) + PAGE_CACHE_SIZE - 1,
+ GFP_NOFS);
+ unlock_page(page);
+ }
+ return 0;
+}
+
+int clear_extent_buffer_uptodate(struct extent_io_tree *tree,
+ struct extent_buffer *eb)
+{
+ unsigned long i;
+ struct page *page;
+ unsigned long num_pages;
+
+ num_pages = num_extent_pages(eb->start, eb->len);
+ eb->flags &= ~EXTENT_UPTODATE;
+
+ clear_extent_uptodate(tree, eb->start, eb->start + eb->len - 1,
+ GFP_NOFS);
+ for (i = 0; i < num_pages; i++) {
+ page = extent_buffer_page(eb, i);
+ if (page)
+ ClearPageUptodate(page);
+ }
+ return 0;
+}
+
+int set_extent_buffer_uptodate(struct extent_io_tree *tree,
+ struct extent_buffer *eb)
+{
+ unsigned long i;
+ struct page *page;
+ unsigned long num_pages;
+
+ num_pages = num_extent_pages(eb->start, eb->len);
+
+ set_extent_uptodate(tree, eb->start, eb->start + eb->len - 1,
+ GFP_NOFS);
+ for (i = 0; i < num_pages; i++) {
+ page = extent_buffer_page(eb, i);
+ if ((i == 0 && (eb->start & (PAGE_CACHE_SIZE - 1))) ||
+ ((i == num_pages - 1) &&
+ ((eb->start + eb->len) & (PAGE_CACHE_SIZE - 1)))) {
+ check_page_uptodate(tree, page);
+ continue;
+ }
+ SetPageUptodate(page);
+ }
+ return 0;
+}
+
+int extent_range_uptodate(struct extent_io_tree *tree,
+ u64 start, u64 end)
+{
+ struct page *page;
+ int ret;
+ int pg_uptodate = 1;
+ int uptodate;
+ unsigned long index;
+
+ ret = test_range_bit(tree, start, end, EXTENT_UPTODATE, 1);
+ if (ret)
+ return 1;
+ while (start <= end) {
+ index = start >> PAGE_CACHE_SHIFT;
+ page = find_get_page(tree->mapping, index);
+ uptodate = PageUptodate(page);
+ page_cache_release(page);
+ if (!uptodate) {
+ pg_uptodate = 0;
+ break;
+ }
+ start += PAGE_CACHE_SIZE;
+ }
+ return pg_uptodate;
+}
+
+int extent_buffer_uptodate(struct extent_io_tree *tree,
+ struct extent_buffer *eb)
+{
+ int ret = 0;
+ unsigned long num_pages;
+ unsigned long i;
+ struct page *page;
+ int pg_uptodate = 1;
+
+ if (eb->flags & EXTENT_UPTODATE)
+ return 1;
+
+ ret = test_range_bit(tree, eb->start, eb->start + eb->len - 1,
+ EXTENT_UPTODATE, 1);
+ if (ret)
+ return ret;
+
+ num_pages = num_extent_pages(eb->start, eb->len);
+ for (i = 0; i < num_pages; i++) {
+ page = extent_buffer_page(eb, i);
+ if (!PageUptodate(page)) {
+ pg_uptodate = 0;
+ break;
+ }
+ }
+ return pg_uptodate;
+}
+
+int read_extent_buffer_pages(struct extent_io_tree *tree,
+ struct extent_buffer *eb,
+ u64 start, int wait,
+ get_extent_t *get_extent, int mirror_num)
+{
+ unsigned long i;
+ unsigned long start_i;
+ struct page *page;
+ int err;
+ int ret = 0;
+ int locked_pages = 0;
+ int all_uptodate = 1;
+ int inc_all_pages = 0;
+ unsigned long num_pages;
+ struct bio *bio = NULL;
+ unsigned long bio_flags = 0;
+
+ if (eb->flags & EXTENT_UPTODATE)
+ return 0;
+
+ if (test_range_bit(tree, eb->start, eb->start + eb->len - 1,
+ EXTENT_UPTODATE, 1)) {
+ return 0;
+ }
+
+ if (start) {
+ WARN_ON(start < eb->start);
+ start_i = (start >> PAGE_CACHE_SHIFT) -
+ (eb->start >> PAGE_CACHE_SHIFT);
+ } else {
+ start_i = 0;
+ }
+
+ num_pages = num_extent_pages(eb->start, eb->len);
+ for (i = start_i; i < num_pages; i++) {
+ page = extent_buffer_page(eb, i);
+ if (!wait) {
+ if (!trylock_page(page))
+ goto unlock_exit;
+ } else {
+ lock_page(page);
+ }
+ locked_pages++;
+ if (!PageUptodate(page))
+ all_uptodate = 0;
+ }
+ if (all_uptodate) {
+ if (start_i == 0)
+ eb->flags |= EXTENT_UPTODATE;
+ goto unlock_exit;
+ }
+
+ for (i = start_i; i < num_pages; i++) {
+ page = extent_buffer_page(eb, i);
+ if (inc_all_pages)
+ page_cache_get(page);
+ if (!PageUptodate(page)) {
+ if (start_i == 0)
+ inc_all_pages = 1;
+ ClearPageError(page);
+ err = __extent_read_full_page(tree, page,
+ get_extent, &bio,
+ mirror_num, &bio_flags);
+ if (err)
+ ret = err;
+ } else {
+ unlock_page(page);
+ }
+ }
+
+ if (bio)
+ submit_one_bio(READ, bio, mirror_num, bio_flags);
+
+ if (ret || !wait)
+ return ret;
+
+ for (i = start_i; i < num_pages; i++) {
+ page = extent_buffer_page(eb, i);
+ wait_on_page_locked(page);
+ if (!PageUptodate(page))
+ ret = -EIO;
+ }
+
+ if (!ret)
+ eb->flags |= EXTENT_UPTODATE;
+ return ret;
+
+unlock_exit:
+ i = start_i;
+ while (locked_pages > 0) {
+ page = extent_buffer_page(eb, i);
+ i++;
+ unlock_page(page);
+ locked_pages--;
+ }
+ return ret;
+}
+
+void read_extent_buffer(struct extent_buffer *eb, void *dstv,
+ unsigned long start,
+ unsigned long len)
+{
+ size_t cur;
+ size_t offset;
+ struct page *page;
+ char *kaddr;
+ char *dst = (char *)dstv;
+ size_t start_offset = eb->start & ((u64)PAGE_CACHE_SIZE - 1);
+ unsigned long i = (start_offset + start) >> PAGE_CACHE_SHIFT;
+
+ WARN_ON(start > eb->len);
+ WARN_ON(start + len > eb->start + eb->len);
+
+ offset = (start_offset + start) & ((unsigned long)PAGE_CACHE_SIZE - 1);
+
+ while (len > 0) {
+ page = extent_buffer_page(eb, i);
+
+ cur = min(len, (PAGE_CACHE_SIZE - offset));
+ kaddr = kmap_atomic(page, KM_USER1);
+ memcpy(dst, kaddr + offset, cur);
+ kunmap_atomic(kaddr, KM_USER1);
+
+ dst += cur;
+ len -= cur;
+ offset = 0;
+ i++;
+ }
+}
+
+int map_private_extent_buffer(struct extent_buffer *eb, unsigned long start,
+ unsigned long min_len, char **token, char **map,
+ unsigned long *map_start,
+ unsigned long *map_len, int km)
+{
+ size_t offset = start & (PAGE_CACHE_SIZE - 1);
+ char *kaddr;
+ struct page *p;
+ size_t start_offset = eb->start & ((u64)PAGE_CACHE_SIZE - 1);
+ unsigned long i = (start_offset + start) >> PAGE_CACHE_SHIFT;
+ unsigned long end_i = (start_offset + start + min_len - 1) >>
+ PAGE_CACHE_SHIFT;
+
+ if (i != end_i)
+ return -EINVAL;
+
+ if (i == 0) {
+ offset = start_offset;
+ *map_start = 0;
+ } else {
+ offset = 0;
+ *map_start = ((u64)i << PAGE_CACHE_SHIFT) - start_offset;
+ }
+
+ if (start + min_len > eb->len) {
+ printk(KERN_ERR "btrfs bad mapping eb start %llu len %lu, "
+ "wanted %lu %lu\n", (unsigned long long)eb->start,
+ eb->len, start, min_len);
+ WARN_ON(1);
+ }
+
+ p = extent_buffer_page(eb, i);
+ kaddr = kmap_atomic(p, km);
+ *token = kaddr;
+ *map = kaddr + offset;
+ *map_len = PAGE_CACHE_SIZE - offset;
+ return 0;
+}
+
+int map_extent_buffer(struct extent_buffer *eb, unsigned long start,
+ unsigned long min_len,
+ char **token, char **map,
+ unsigned long *map_start,
+ unsigned long *map_len, int km)
+{
+ int err;
+ int save = 0;
+ if (eb->map_token) {
+ unmap_extent_buffer(eb, eb->map_token, km);
+ eb->map_token = NULL;
+ save = 1;
+ WARN_ON(!mutex_is_locked(&eb->mutex));
+ }
+ err = map_private_extent_buffer(eb, start, min_len, token, map,
+ map_start, map_len, km);
+ if (!err && save) {
+ eb->map_token = *token;
+ eb->kaddr = *map;
+ eb->map_start = *map_start;
+ eb->map_len = *map_len;
+ }
+ return err;
+}
+
+void unmap_extent_buffer(struct extent_buffer *eb, char *token, int km)
+{
+ kunmap_atomic(token, km);
+}
+
+int memcmp_extent_buffer(struct extent_buffer *eb, const void *ptrv,
+ unsigned long start,
+ unsigned long len)
+{
+ size_t cur;
+ size_t offset;
+ struct page *page;
+ char *kaddr;
+ char *ptr = (char *)ptrv;
+ size_t start_offset = eb->start & ((u64)PAGE_CACHE_SIZE - 1);
+ unsigned long i = (start_offset + start) >> PAGE_CACHE_SHIFT;
+ int ret = 0;
+
+ WARN_ON(start > eb->len);
+ WARN_ON(start + len > eb->start + eb->len);
+
+ offset = (start_offset + start) & ((unsigned long)PAGE_CACHE_SIZE - 1);
+
+ while (len > 0) {
+ page = extent_buffer_page(eb, i);
+
+ cur = min(len, (PAGE_CACHE_SIZE - offset));
+
+ kaddr = kmap_atomic(page, KM_USER0);
+ ret = memcmp(ptr, kaddr + offset, cur);
+ kunmap_atomic(kaddr, KM_USER0);
+ if (ret)
+ break;
+
+ ptr += cur;
+ len -= cur;
+ offset = 0;
+ i++;
+ }
+ return ret;
+}
+
+void write_extent_buffer(struct extent_buffer *eb, const void *srcv,
+ unsigned long start, unsigned long len)
+{
+ size_t cur;
+ size_t offset;
+ struct page *page;
+ char *kaddr;
+ char *src = (char *)srcv;
+ size_t start_offset = eb->start & ((u64)PAGE_CACHE_SIZE - 1);
+ unsigned long i = (start_offset + start) >> PAGE_CACHE_SHIFT;
+
+ WARN_ON(start > eb->len);
+ WARN_ON(start + len > eb->start + eb->len);
+
+ offset = (start_offset + start) & ((unsigned long)PAGE_CACHE_SIZE - 1);
+
+ while (len > 0) {
+ page = extent_buffer_page(eb, i);
+ WARN_ON(!PageUptodate(page));
+
+ cur = min(len, PAGE_CACHE_SIZE - offset);
+ kaddr = kmap_atomic(page, KM_USER1);
+ memcpy(kaddr + offset, src, cur);
+ kunmap_atomic(kaddr, KM_USER1);
+
+ src += cur;
+ len -= cur;
+ offset = 0;
+ i++;
+ }
+}
+
+void memset_extent_buffer(struct extent_buffer *eb, char c,
+ unsigned long start, unsigned long len)
+{
+ size_t cur;
+ size_t offset;
+ struct page *page;
+ char *kaddr;
+ size_t start_offset = eb->start & ((u64)PAGE_CACHE_SIZE - 1);
+ unsigned long i = (start_offset + start) >> PAGE_CACHE_SHIFT;
+
+ WARN_ON(start > eb->len);
+ WARN_ON(start + len > eb->start + eb->len);
+
+ offset = (start_offset + start) & ((unsigned long)PAGE_CACHE_SIZE - 1);
+
+ while (len > 0) {
+ page = extent_buffer_page(eb, i);
+ WARN_ON(!PageUptodate(page));
+
+ cur = min(len, PAGE_CACHE_SIZE - offset);
+ kaddr = kmap_atomic(page, KM_USER0);
+ memset(kaddr + offset, c, cur);
+ kunmap_atomic(kaddr, KM_USER0);
+
+ len -= cur;
+ offset = 0;
+ i++;
+ }
+}
+
+void copy_extent_buffer(struct extent_buffer *dst, struct extent_buffer *src,
+ unsigned long dst_offset, unsigned long src_offset,
+ unsigned long len)
+{
+ u64 dst_len = dst->len;
+ size_t cur;
+ size_t offset;
+ struct page *page;
+ char *kaddr;
+ size_t start_offset = dst->start & ((u64)PAGE_CACHE_SIZE - 1);
+ unsigned long i = (start_offset + dst_offset) >> PAGE_CACHE_SHIFT;
+
+ WARN_ON(src->len != dst_len);
+
+ offset = (start_offset + dst_offset) &
+ ((unsigned long)PAGE_CACHE_SIZE - 1);
+
+ while (len > 0) {
+ page = extent_buffer_page(dst, i);
+ WARN_ON(!PageUptodate(page));
+
+ cur = min(len, (unsigned long)(PAGE_CACHE_SIZE - offset));
+
+ kaddr = kmap_atomic(page, KM_USER0);
+ read_extent_buffer(src, kaddr + offset, src_offset, cur);
+ kunmap_atomic(kaddr, KM_USER0);
+
+ src_offset += cur;
+ len -= cur;
+ offset = 0;
+ i++;
+ }
+}
+
+static void move_pages(struct page *dst_page, struct page *src_page,
+ unsigned long dst_off, unsigned long src_off,
+ unsigned long len)
+{
+ char *dst_kaddr = kmap_atomic(dst_page, KM_USER0);
+ if (dst_page == src_page) {
+ memmove(dst_kaddr + dst_off, dst_kaddr + src_off, len);
+ } else {
+ char *src_kaddr = kmap_atomic(src_page, KM_USER1);
+ char *p = dst_kaddr + dst_off + len;
+ char *s = src_kaddr + src_off + len;
+
+ while (len--)
+ *--p = *--s;
+
+ kunmap_atomic(src_kaddr, KM_USER1);
+ }
+ kunmap_atomic(dst_kaddr, KM_USER0);
+}
+
+static void copy_pages(struct page *dst_page, struct page *src_page,
+ unsigned long dst_off, unsigned long src_off,
+ unsigned long len)
+{
+ char *dst_kaddr = kmap_atomic(dst_page, KM_USER0);
+ char *src_kaddr;
+
+ if (dst_page != src_page)
+ src_kaddr = kmap_atomic(src_page, KM_USER1);
+ else
+ src_kaddr = dst_kaddr;
+
+ memcpy(dst_kaddr + dst_off, src_kaddr + src_off, len);
+ kunmap_atomic(dst_kaddr, KM_USER0);
+ if (dst_page != src_page)
+ kunmap_atomic(src_kaddr, KM_USER1);
+}
+
+void memcpy_extent_buffer(struct extent_buffer *dst, unsigned long dst_offset,
+ unsigned long src_offset, unsigned long len)
+{
+ size_t cur;
+ size_t dst_off_in_page;
+ size_t src_off_in_page;
+ size_t start_offset = dst->start & ((u64)PAGE_CACHE_SIZE - 1);
+ unsigned long dst_i;
+ unsigned long src_i;
+
+ if (src_offset + len > dst->len) {
+ printk(KERN_ERR "btrfs memmove bogus src_offset %lu move "
+ "len %lu dst len %lu\n", src_offset, len, dst->len);
+ BUG_ON(1);
+ }
+ if (dst_offset + len > dst->len) {
+ printk(KERN_ERR "btrfs memmove bogus dst_offset %lu move "
+ "len %lu dst len %lu\n", dst_offset, len, dst->len);
+ BUG_ON(1);
+ }
+
+ while (len > 0) {
+ dst_off_in_page = (start_offset + dst_offset) &
+ ((unsigned long)PAGE_CACHE_SIZE - 1);
+ src_off_in_page = (start_offset + src_offset) &
+ ((unsigned long)PAGE_CACHE_SIZE - 1);
+
+ dst_i = (start_offset + dst_offset) >> PAGE_CACHE_SHIFT;
+ src_i = (start_offset + src_offset) >> PAGE_CACHE_SHIFT;
+
+ cur = min(len, (unsigned long)(PAGE_CACHE_SIZE -
+ src_off_in_page));
+ cur = min_t(unsigned long, cur,
+ (unsigned long)(PAGE_CACHE_SIZE - dst_off_in_page));
+
+ copy_pages(extent_buffer_page(dst, dst_i),
+ extent_buffer_page(dst, src_i),
+ dst_off_in_page, src_off_in_page, cur);
+
+ src_offset += cur;
+ dst_offset += cur;
+ len -= cur;
+ }
+}
+
+void memmove_extent_buffer(struct extent_buffer *dst, unsigned long dst_offset,
+ unsigned long src_offset, unsigned long len)
+{
+ size_t cur;
+ size_t dst_off_in_page;
+ size_t src_off_in_page;
+ unsigned long dst_end = dst_offset + len - 1;
+ unsigned long src_end = src_offset + len - 1;
+ size_t start_offset = dst->start & ((u64)PAGE_CACHE_SIZE - 1);
+ unsigned long dst_i;
+ unsigned long src_i;
+
+ if (src_offset + len > dst->len) {
+ printk(KERN_ERR "btrfs memmove bogus src_offset %lu move "
+ "len %lu len %lu\n", src_offset, len, dst->len);
+ BUG_ON(1);
+ }
+ if (dst_offset + len > dst->len) {
+ printk(KERN_ERR "btrfs memmove bogus dst_offset %lu move "
+ "len %lu len %lu\n", dst_offset, len, dst->len);
+ BUG_ON(1);
+ }
+ if (dst_offset < src_offset) {
+ memcpy_extent_buffer(dst, dst_offset, src_offset, len);
+ return;
+ }
+ while (len > 0) {
+ dst_i = (start_offset + dst_end) >> PAGE_CACHE_SHIFT;
+ src_i = (start_offset + src_end) >> PAGE_CACHE_SHIFT;
+
+ dst_off_in_page = (start_offset + dst_end) &
+ ((unsigned long)PAGE_CACHE_SIZE - 1);
+ src_off_in_page = (start_offset + src_end) &
+ ((unsigned long)PAGE_CACHE_SIZE - 1);
+
+ cur = min_t(unsigned long, len, src_off_in_page + 1);
+ cur = min(cur, dst_off_in_page + 1);
+ move_pages(extent_buffer_page(dst, dst_i),
+ extent_buffer_page(dst, src_i),
+ dst_off_in_page - cur + 1,
+ src_off_in_page - cur + 1, cur);
+
+ dst_end -= cur;
+ src_end -= cur;
+ len -= cur;
+ }
+}
+
+int try_release_extent_buffer(struct extent_io_tree *tree, struct page *page)
+{
+ u64 start = page_offset(page);
+ struct extent_buffer *eb;
+ int ret = 1;
+ unsigned long i;
+ unsigned long num_pages;
+
+ spin_lock(&tree->buffer_lock);
+ eb = buffer_search(tree, start);
+ if (!eb)
+ goto out;
+
+ if (atomic_read(&eb->refs) > 1) {
+ ret = 0;
+ goto out;
+ }
+ /* at this point we can safely release the extent buffer */
+ num_pages = num_extent_pages(eb->start, eb->len);
+ for (i = 0; i < num_pages; i++)
+ page_cache_release(extent_buffer_page(eb, i));
+ rb_erase(&eb->rb_node, &tree->buffer);
+ __free_extent_buffer(eb);
+out:
+ spin_unlock(&tree->buffer_lock);
+ return ret;
+}
diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h
new file mode 100644
index 0000000..c5b483a
--- /dev/null
+++ b/fs/btrfs/extent_io.h
@@ -0,0 +1,269 @@
+#ifndef __EXTENTIO__
+#define __EXTENTIO__
+
+#include <linux/rbtree.h>
+
+/* bits for the extent state */
+#define EXTENT_DIRTY 1
+#define EXTENT_WRITEBACK (1 << 1)
+#define EXTENT_UPTODATE (1 << 2)
+#define EXTENT_LOCKED (1 << 3)
+#define EXTENT_NEW (1 << 4)
+#define EXTENT_DELALLOC (1 << 5)
+#define EXTENT_DEFRAG (1 << 6)
+#define EXTENT_DEFRAG_DONE (1 << 7)
+#define EXTENT_BUFFER_FILLED (1 << 8)
+#define EXTENT_ORDERED (1 << 9)
+#define EXTENT_ORDERED_METADATA (1 << 10)
+#define EXTENT_BOUNDARY (1 << 11)
+#define EXTENT_NODATASUM (1 << 12)
+#define EXTENT_IOBITS (EXTENT_LOCKED | EXTENT_WRITEBACK)
+
+/* flags for bio submission */
+#define EXTENT_BIO_COMPRESSED 1
+
+/*
+ * page->private values. Every page that is controlled by the extent
+ * map has page->private set to one.
+ */
+#define EXTENT_PAGE_PRIVATE 1
+#define EXTENT_PAGE_PRIVATE_FIRST_PAGE 3
+
+struct extent_state;
+
+typedef int (extent_submit_bio_hook_t)(struct inode *inode, int rw,
+ struct bio *bio, int mirror_num,
+ unsigned long bio_flags);
+struct extent_io_ops {
+ int (*fill_delalloc)(struct inode *inode, struct page *locked_page,
+ u64 start, u64 end, int *page_started,
+ unsigned long *nr_written);
+ int (*writepage_start_hook)(struct page *page, u64 start, u64 end);
+ int (*writepage_io_hook)(struct page *page, u64 start, u64 end);
+ extent_submit_bio_hook_t *submit_bio_hook;
+ int (*merge_bio_hook)(struct page *page, unsigned long offset,
+ size_t size, struct bio *bio,
+ unsigned long bio_flags);
+ int (*readpage_io_hook)(struct page *page, u64 start, u64 end);
+ int (*readpage_io_failed_hook)(struct bio *bio, struct page *page,
+ u64 start, u64 end,
+ struct extent_state *state);
+ int (*writepage_io_failed_hook)(struct bio *bio, struct page *page,
+ u64 start, u64 end,
+ struct extent_state *state);
+ int (*readpage_end_io_hook)(struct page *page, u64 start, u64 end,
+ struct extent_state *state);
+ int (*writepage_end_io_hook)(struct page *page, u64 start, u64 end,
+ struct extent_state *state, int uptodate);
+ int (*set_bit_hook)(struct inode *inode, u64 start, u64 end,
+ unsigned long old, unsigned long bits);
+ int (*clear_bit_hook)(struct inode *inode, u64 start, u64 end,
+ unsigned long old, unsigned long bits);
+ int (*write_cache_pages_lock_hook)(struct page *page);
+};
+
+struct extent_io_tree {
+ struct rb_root state;
+ struct rb_root buffer;
+ struct address_space *mapping;
+ u64 dirty_bytes;
+ spinlock_t lock;
+ spinlock_t buffer_lock;
+ struct extent_io_ops *ops;
+};
+
+struct extent_state {
+ u64 start;
+ u64 end; /* inclusive */
+ struct rb_node rb_node;
+ struct extent_io_tree *tree;
+ wait_queue_head_t wq;
+ atomic_t refs;
+ unsigned long state;
+
+ /* for use by the FS */
+ u64 private;
+
+ struct list_head leak_list;
+};
+
+struct extent_buffer {
+ u64 start;
+ unsigned long len;
+ char *map_token;
+ char *kaddr;
+ unsigned long map_start;
+ unsigned long map_len;
+ struct page *first_page;
+ atomic_t refs;
+ int flags;
+ struct list_head leak_list;
+ struct rb_node rb_node;
+ struct mutex mutex;
+};
+
+struct extent_map_tree;
+
+static inline struct extent_state *extent_state_next(struct extent_state *state)
+{
+ struct rb_node *node;
+ node = rb_next(&state->rb_node);
+ if (!node)
+ return NULL;
+ return rb_entry(node, struct extent_state, rb_node);
+}
+
+typedef struct extent_map *(get_extent_t)(struct inode *inode,
+ struct page *page,
+ size_t page_offset,
+ u64 start, u64 len,
+ int create);
+
+void extent_io_tree_init(struct extent_io_tree *tree,
+ struct address_space *mapping, gfp_t mask);
+int try_release_extent_mapping(struct extent_map_tree *map,
+ struct extent_io_tree *tree, struct page *page,
+ gfp_t mask);
+int try_release_extent_buffer(struct extent_io_tree *tree, struct page *page);
+int try_release_extent_state(struct extent_map_tree *map,
+ struct extent_io_tree *tree, struct page *page,
+ gfp_t mask);
+int lock_extent(struct extent_io_tree *tree, u64 start, u64 end, gfp_t mask);
+int unlock_extent(struct extent_io_tree *tree, u64 start, u64 end, gfp_t mask);
+int try_lock_extent(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask);
+int extent_read_full_page(struct extent_io_tree *tree, struct page *page,
+ get_extent_t *get_extent);
+int __init extent_io_init(void);
+void extent_io_exit(void);
+
+u64 count_range_bits(struct extent_io_tree *tree,
+ u64 *start, u64 search_end,
+ u64 max_bytes, unsigned long bits);
+
+int test_range_bit(struct extent_io_tree *tree, u64 start, u64 end,
+ int bits, int filled);
+int clear_extent_bits(struct extent_io_tree *tree, u64 start, u64 end,
+ int bits, gfp_t mask);
+int clear_extent_bit(struct extent_io_tree *tree, u64 start, u64 end,
+ int bits, int wake, int delete, gfp_t mask);
+int set_extent_bits(struct extent_io_tree *tree, u64 start, u64 end,
+ int bits, gfp_t mask);
+int set_extent_uptodate(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask);
+int set_extent_new(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask);
+int set_extent_dirty(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask);
+int clear_extent_dirty(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask);
+int clear_extent_ordered(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask);
+int clear_extent_ordered_metadata(struct extent_io_tree *tree, u64 start,
+ u64 end, gfp_t mask);
+int set_extent_delalloc(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask);
+int set_extent_ordered(struct extent_io_tree *tree, u64 start, u64 end,
+ gfp_t mask);
+int find_first_extent_bit(struct extent_io_tree *tree, u64 start,
+ u64 *start_ret, u64 *end_ret, int bits);
+struct extent_state *find_first_extent_bit_state(struct extent_io_tree *tree,
+ u64 start, int bits);
+int extent_invalidatepage(struct extent_io_tree *tree,
+ struct page *page, unsigned long offset);
+int extent_write_full_page(struct extent_io_tree *tree, struct page *page,
+ get_extent_t *get_extent,
+ struct writeback_control *wbc);
+int extent_write_locked_range(struct extent_io_tree *tree, struct inode *inode,
+ u64 start, u64 end, get_extent_t *get_extent,
+ int mode);
+int extent_writepages(struct extent_io_tree *tree,
+ struct address_space *mapping,
+ get_extent_t *get_extent,
+ struct writeback_control *wbc);
+int extent_readpages(struct extent_io_tree *tree,
+ struct address_space *mapping,
+ struct list_head *pages, unsigned nr_pages,
+ get_extent_t get_extent);
+int extent_prepare_write(struct extent_io_tree *tree,
+ struct inode *inode, struct page *page,
+ unsigned from, unsigned to, get_extent_t *get_extent);
+int extent_commit_write(struct extent_io_tree *tree,
+ struct inode *inode, struct page *page,
+ unsigned from, unsigned to);
+sector_t extent_bmap(struct address_space *mapping, sector_t iblock,
+ get_extent_t *get_extent);
+int set_range_dirty(struct extent_io_tree *tree, u64 start, u64 end);
+int set_state_private(struct extent_io_tree *tree, u64 start, u64 private);
+int get_state_private(struct extent_io_tree *tree, u64 start, u64 *private);
+void set_page_extent_mapped(struct page *page);
+
+struct extent_buffer *alloc_extent_buffer(struct extent_io_tree *tree,
+ u64 start, unsigned long len,
+ struct page *page0,
+ gfp_t mask);
+struct extent_buffer *find_extent_buffer(struct extent_io_tree *tree,
+ u64 start, unsigned long len,
+ gfp_t mask);
+void free_extent_buffer(struct extent_buffer *eb);
+int read_extent_buffer_pages(struct extent_io_tree *tree,
+ struct extent_buffer *eb, u64 start, int wait,
+ get_extent_t *get_extent, int mirror_num);
+
+static inline void extent_buffer_get(struct extent_buffer *eb)
+{
+ atomic_inc(&eb->refs);
+}
+
+int memcmp_extent_buffer(struct extent_buffer *eb, const void *ptrv,
+ unsigned long start,
+ unsigned long len);
+void read_extent_buffer(struct extent_buffer *eb, void *dst,
+ unsigned long start,
+ unsigned long len);
+void write_extent_buffer(struct extent_buffer *eb, const void *src,
+ unsigned long start, unsigned long len);
+void copy_extent_buffer(struct extent_buffer *dst, struct extent_buffer *src,
+ unsigned long dst_offset, unsigned long src_offset,
+ unsigned long len);
+void memcpy_extent_buffer(struct extent_buffer *dst, unsigned long dst_offset,
+ unsigned long src_offset, unsigned long len);
+void memmove_extent_buffer(struct extent_buffer *dst, unsigned long dst_offset,
+ unsigned long src_offset, unsigned long len);
+void memset_extent_buffer(struct extent_buffer *eb, char c,
+ unsigned long start, unsigned long len);
+int wait_on_extent_buffer_writeback(struct extent_io_tree *tree,
+ struct extent_buffer *eb);
+int wait_on_extent_writeback(struct extent_io_tree *tree, u64 start, u64 end);
+int wait_extent_bit(struct extent_io_tree *tree, u64 start, u64 end, int bits);
+int clear_extent_buffer_dirty(struct extent_io_tree *tree,
+ struct extent_buffer *eb);
+int set_extent_buffer_dirty(struct extent_io_tree *tree,
+ struct extent_buffer *eb);
+int set_extent_buffer_uptodate(struct extent_io_tree *tree,
+ struct extent_buffer *eb);
+int clear_extent_buffer_uptodate(struct extent_io_tree *tree,
+ struct extent_buffer *eb);
+int extent_buffer_uptodate(struct extent_io_tree *tree,
+ struct extent_buffer *eb);
+int map_extent_buffer(struct extent_buffer *eb, unsigned long offset,
+ unsigned long min_len, char **token, char **map,
+ unsigned long *map_start,
+ unsigned long *map_len, int km);
+int map_private_extent_buffer(struct extent_buffer *eb, unsigned long offset,
+ unsigned long min_len, char **token, char **map,
+ unsigned long *map_start,
+ unsigned long *map_len, int km);
+void unmap_extent_buffer(struct extent_buffer *eb, char *token, int km);
+int release_extent_buffer_tail_pages(struct extent_buffer *eb);
+int extent_range_uptodate(struct extent_io_tree *tree,
+ u64 start, u64 end);
+int extent_clear_unlock_delalloc(struct inode *inode,
+ struct extent_io_tree *tree,
+ u64 start, u64 end, struct page *locked_page,
+ int unlock_page,
+ int clear_unlock,
+ int clear_delalloc, int clear_dirty,
+ int set_writeback,
+ int end_writeback);
+#endif
diff --git a/fs/btrfs/extent_map.c b/fs/btrfs/extent_map.c
new file mode 100644
index 0000000..4a83e33
--- /dev/null
+++ b/fs/btrfs/extent_map.c
@@ -0,0 +1,351 @@
+#include <linux/err.h>
+#include <linux/gfp.h>
+#include <linux/slab.h>
+#include <linux/module.h>
+#include <linux/spinlock.h>
+#include <linux/version.h>
+#include <linux/hardirq.h>
+#include "extent_map.h"
+
+/* temporary define until extent_map moves out of btrfs */
+struct kmem_cache *btrfs_cache_create(const char *name, size_t size,
+ unsigned long extra_flags,
+ void (*ctor)(void *, struct kmem_cache *,
+ unsigned long));
+
+static struct kmem_cache *extent_map_cache;
+
+int __init extent_map_init(void)
+{
+ extent_map_cache = btrfs_cache_create("extent_map",
+ sizeof(struct extent_map), 0,
+ NULL);
+ if (!extent_map_cache)
+ return -ENOMEM;
+ return 0;
+}
+
+void extent_map_exit(void)
+{
+ if (extent_map_cache)
+ kmem_cache_destroy(extent_map_cache);
+}
+
+/**
+ * extent_map_tree_init - initialize extent map tree
+ * @tree: tree to initialize
+ * @mask: flags for memory allocations during tree operations
+ *
+ * Initialize the extent tree @tree. Should be called for each new inode
+ * or other user of the extent_map interface.
+ */
+void extent_map_tree_init(struct extent_map_tree *tree, gfp_t mask)
+{
+ tree->map.rb_node = NULL;
+ spin_lock_init(&tree->lock);
+}
+EXPORT_SYMBOL(extent_map_tree_init);
+
+/**
+ * alloc_extent_map - allocate new extent map structure
+ * @mask: memory allocation flags
+ *
+ * Allocate a new extent_map structure. The new structure is
+ * returned with a reference count of one and needs to be
+ * freed using free_extent_map()
+ */
+struct extent_map *alloc_extent_map(gfp_t mask)
+{
+ struct extent_map *em;
+ em = kmem_cache_alloc(extent_map_cache, mask);
+ if (!em || IS_ERR(em))
+ return em;
+ em->in_tree = 0;
+ em->flags = 0;
+ atomic_set(&em->refs, 1);
+ return em;
+}
+EXPORT_SYMBOL(alloc_extent_map);
+
+/**
+ * free_extent_map - drop reference count of an extent_map
+ * @em: extent map beeing releasead
+ *
+ * Drops the reference out on @em by one and free the structure
+ * if the reference count hits zero.
+ */
+void free_extent_map(struct extent_map *em)
+{
+ if (!em)
+ return;
+ WARN_ON(atomic_read(&em->refs) == 0);
+ if (atomic_dec_and_test(&em->refs)) {
+ WARN_ON(em->in_tree);
+ kmem_cache_free(extent_map_cache, em);
+ }
+}
+EXPORT_SYMBOL(free_extent_map);
+
+static struct rb_node *tree_insert(struct rb_root *root, u64 offset,
+ struct rb_node *node)
+{
+ struct rb_node **p = &root->rb_node;
+ struct rb_node *parent = NULL;
+ struct extent_map *entry;
+
+ while (*p) {
+ parent = *p;
+ entry = rb_entry(parent, struct extent_map, rb_node);
+
+ WARN_ON(!entry->in_tree);
+
+ if (offset < entry->start)
+ p = &(*p)->rb_left;
+ else if (offset >= extent_map_end(entry))
+ p = &(*p)->rb_right;
+ else
+ return parent;
+ }
+
+ entry = rb_entry(node, struct extent_map, rb_node);
+ entry->in_tree = 1;
+ rb_link_node(node, parent, p);
+ rb_insert_color(node, root);
+ return NULL;
+}
+
+/*
+ * search through the tree for an extent_map with a given offset. If
+ * it can't be found, try to find some neighboring extents
+ */
+static struct rb_node *__tree_search(struct rb_root *root, u64 offset,
+ struct rb_node **prev_ret,
+ struct rb_node **next_ret)
+{
+ struct rb_node *n = root->rb_node;
+ struct rb_node *prev = NULL;
+ struct rb_node *orig_prev = NULL;
+ struct extent_map *entry;
+ struct extent_map *prev_entry = NULL;
+
+ while (n) {
+ entry = rb_entry(n, struct extent_map, rb_node);
+ prev = n;
+ prev_entry = entry;
+
+ WARN_ON(!entry->in_tree);
+
+ if (offset < entry->start)
+ n = n->rb_left;
+ else if (offset >= extent_map_end(entry))
+ n = n->rb_right;
+ else
+ return n;
+ }
+
+ if (prev_ret) {
+ orig_prev = prev;
+ while (prev && offset >= extent_map_end(prev_entry)) {
+ prev = rb_next(prev);
+ prev_entry = rb_entry(prev, struct extent_map, rb_node);
+ }
+ *prev_ret = prev;
+ prev = orig_prev;
+ }
+
+ if (next_ret) {
+ prev_entry = rb_entry(prev, struct extent_map, rb_node);
+ while (prev && offset < prev_entry->start) {
+ prev = rb_prev(prev);
+ prev_entry = rb_entry(prev, struct extent_map, rb_node);
+ }
+ *next_ret = prev;
+ }
+ return NULL;
+}
+
+/*
+ * look for an offset in the tree, and if it can't be found, return
+ * the first offset we can find smaller than 'offset'.
+ */
+static inline struct rb_node *tree_search(struct rb_root *root, u64 offset)
+{
+ struct rb_node *prev;
+ struct rb_node *ret;
+ ret = __tree_search(root, offset, &prev, NULL);
+ if (!ret)
+ return prev;
+ return ret;
+}
+
+/* check to see if two extent_map structs are adjacent and safe to merge */
+static int mergable_maps(struct extent_map *prev, struct extent_map *next)
+{
+ if (test_bit(EXTENT_FLAG_PINNED, &prev->flags))
+ return 0;
+
+ /*
+ * don't merge compressed extents, we need to know their
+ * actual size
+ */
+ if (test_bit(EXTENT_FLAG_COMPRESSED, &prev->flags))
+ return 0;
+
+ if (extent_map_end(prev) == next->start &&
+ prev->flags == next->flags &&
+ prev->bdev == next->bdev &&
+ ((next->block_start == EXTENT_MAP_HOLE &&
+ prev->block_start == EXTENT_MAP_HOLE) ||
+ (next->block_start == EXTENT_MAP_INLINE &&
+ prev->block_start == EXTENT_MAP_INLINE) ||
+ (next->block_start == EXTENT_MAP_DELALLOC &&
+ prev->block_start == EXTENT_MAP_DELALLOC) ||
+ (next->block_start < EXTENT_MAP_LAST_BYTE - 1 &&
+ next->block_start == extent_map_block_end(prev)))) {
+ return 1;
+ }
+ return 0;
+}
+
+/**
+ * add_extent_mapping - add new extent map to the extent tree
+ * @tree: tree to insert new map in
+ * @em: map to insert
+ *
+ * Insert @em into @tree or perform a simple forward/backward merge with
+ * existing mappings. The extent_map struct passed in will be inserted
+ * into the tree directly, with an additional reference taken, or a
+ * reference dropped if the merge attempt was sucessfull.
+ */
+int add_extent_mapping(struct extent_map_tree *tree,
+ struct extent_map *em)
+{
+ int ret = 0;
+ struct extent_map *merge = NULL;
+ struct rb_node *rb;
+ struct extent_map *exist;
+
+ exist = lookup_extent_mapping(tree, em->start, em->len);
+ if (exist) {
+ free_extent_map(exist);
+ ret = -EEXIST;
+ goto out;
+ }
+ assert_spin_locked(&tree->lock);
+ rb = tree_insert(&tree->map, em->start, &em->rb_node);
+ if (rb) {
+ ret = -EEXIST;
+ free_extent_map(merge);
+ goto out;
+ }
+ atomic_inc(&em->refs);
+ if (em->start != 0) {
+ rb = rb_prev(&em->rb_node);
+ if (rb)
+ merge = rb_entry(rb, struct extent_map, rb_node);
+ if (rb && mergable_maps(merge, em)) {
+ em->start = merge->start;
+ em->len += merge->len;
+ em->block_len += merge->block_len;
+ em->block_start = merge->block_start;
+ merge->in_tree = 0;
+ rb_erase(&merge->rb_node, &tree->map);
+ free_extent_map(merge);
+ }
+ }
+ rb = rb_next(&em->rb_node);
+ if (rb)
+ merge = rb_entry(rb, struct extent_map, rb_node);
+ if (rb && mergable_maps(em, merge)) {
+ em->len += merge->len;
+ em->block_len += merge->len;
+ rb_erase(&merge->rb_node, &tree->map);
+ merge->in_tree = 0;
+ free_extent_map(merge);
+ }
+out:
+ return ret;
+}
+EXPORT_SYMBOL(add_extent_mapping);
+
+/* simple helper to do math around the end of an extent, handling wrap */
+static u64 range_end(u64 start, u64 len)
+{
+ if (start + len < start)
+ return (u64)-1;
+ return start + len;
+}
+
+/**
+ * lookup_extent_mapping - lookup extent_map
+ * @tree: tree to lookup in
+ * @start: byte offset to start the search
+ * @len: length of the lookup range
+ *
+ * Find and return the first extent_map struct in @tree that intersects the
+ * [start, len] range. There may be additional objects in the tree that
+ * intersect, so check the object returned carefully to make sure that no
+ * additional lookups are needed.
+ */
+struct extent_map *lookup_extent_mapping(struct extent_map_tree *tree,
+ u64 start, u64 len)
+{
+ struct extent_map *em;
+ struct rb_node *rb_node;
+ struct rb_node *prev = NULL;
+ struct rb_node *next = NULL;
+ u64 end = range_end(start, len);
+
+ assert_spin_locked(&tree->lock);
+ rb_node = __tree_search(&tree->map, start, &prev, &next);
+ if (!rb_node && prev) {
+ em = rb_entry(prev, struct extent_map, rb_node);
+ if (end > em->start && start < extent_map_end(em))
+ goto found;
+ }
+ if (!rb_node && next) {
+ em = rb_entry(next, struct extent_map, rb_node);
+ if (end > em->start && start < extent_map_end(em))
+ goto found;
+ }
+ if (!rb_node) {
+ em = NULL;
+ goto out;
+ }
+ if (IS_ERR(rb_node)) {
+ em = ERR_PTR(PTR_ERR(rb_node));
+ goto out;
+ }
+ em = rb_entry(rb_node, struct extent_map, rb_node);
+ if (end > em->start && start < extent_map_end(em))
+ goto found;
+
+ em = NULL;
+ goto out;
+
+found:
+ atomic_inc(&em->refs);
+out:
+ return em;
+}
+EXPORT_SYMBOL(lookup_extent_mapping);
+
+/**
+ * remove_extent_mapping - removes an extent_map from the extent tree
+ * @tree: extent tree to remove from
+ * @em: extent map beeing removed
+ *
+ * Removes @em from @tree. No reference counts are dropped, and no checks
+ * are done to see if the range is in use
+ */
+int remove_extent_mapping(struct extent_map_tree *tree, struct extent_map *em)
+{
+ int ret = 0;
+
+ WARN_ON(test_bit(EXTENT_FLAG_PINNED, &em->flags));
+ assert_spin_locked(&tree->lock);
+ rb_erase(&em->rb_node, &tree->map);
+ em->in_tree = 0;
+ return ret;
+}
+EXPORT_SYMBOL(remove_extent_mapping);
diff --git a/fs/btrfs/extent_map.h b/fs/btrfs/extent_map.h
new file mode 100644
index 0000000..fb6eeef
--- /dev/null
+++ b/fs/btrfs/extent_map.h
@@ -0,0 +1,62 @@
+#ifndef __EXTENTMAP__
+#define __EXTENTMAP__
+
+#include <linux/rbtree.h>
+
+#define EXTENT_MAP_LAST_BYTE (u64)-4
+#define EXTENT_MAP_HOLE (u64)-3
+#define EXTENT_MAP_INLINE (u64)-2
+#define EXTENT_MAP_DELALLOC (u64)-1
+
+/* bits for the flags field */
+#define EXTENT_FLAG_PINNED 0 /* this entry not yet on disk, don't free it */
+#define EXTENT_FLAG_COMPRESSED 1
+#define EXTENT_FLAG_VACANCY 2 /* no file extent item found */
+#define EXTENT_FLAG_PREALLOC 3 /* pre-allocated extent */
+
+struct extent_map {
+ struct rb_node rb_node;
+
+ /* all of these are in bytes */
+ u64 start;
+ u64 len;
+ u64 orig_start;
+ u64 block_start;
+ u64 block_len;
+ unsigned long flags;
+ struct block_device *bdev;
+ atomic_t refs;
+ int in_tree;
+};
+
+struct extent_map_tree {
+ struct rb_root map;
+ spinlock_t lock;
+};
+
+static inline u64 extent_map_end(struct extent_map *em)
+{
+ if (em->start + em->len < em->start)
+ return (u64)-1;
+ return em->start + em->len;
+}
+
+static inline u64 extent_map_block_end(struct extent_map *em)
+{
+ if (em->block_start + em->block_len < em->block_start)
+ return (u64)-1;
+ return em->block_start + em->block_len;
+}
+
+void extent_map_tree_init(struct extent_map_tree *tree, gfp_t mask);
+struct extent_map *lookup_extent_mapping(struct extent_map_tree *tree,
+ u64 start, u64 len);
+int add_extent_mapping(struct extent_map_tree *tree,
+ struct extent_map *em);
+int remove_extent_mapping(struct extent_map_tree *tree, struct extent_map *em);
+
+struct extent_map *alloc_extent_map(gfp_t mask);
+void free_extent_map(struct extent_map *em);
+int __init extent_map_init(void);
+void extent_map_exit(void);
+#endif
diff --git a/fs/btrfs/file-item.c b/fs/btrfs/file-item.c
new file mode 100644
index 0000000..9646524
--- /dev/null
+++ b/fs/btrfs/file-item.c
@@ -0,0 +1,831 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/bio.h>
+#include <linux/pagemap.h>
+#include <linux/highmem.h>
+#include "ctree.h"
+#include "disk-io.h"
+#include "transaction.h"
+#include "print-tree.h"
+
+#define MAX_CSUM_ITEMS(r, size) ((((BTRFS_LEAF_DATA_SIZE(r) - \
+ sizeof(struct btrfs_item) * 2) / \
+ size) - 1))
+
+#define MAX_ORDERED_SUM_BYTES(r) ((PAGE_SIZE - \
+ sizeof(struct btrfs_ordered_sum)) / \
+ sizeof(struct btrfs_sector_sum) * \
+ (r)->sectorsize - (r)->sectorsize)
+
+int btrfs_insert_file_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 objectid, u64 pos,
+ u64 disk_offset, u64 disk_num_bytes,
+ u64 num_bytes, u64 offset, u64 ram_bytes,
+ u8 compression, u8 encryption, u16 other_encoding)
+{
+ int ret = 0;
+ struct btrfs_file_extent_item *item;
+ struct btrfs_key file_key;
+ struct btrfs_path *path;
+ struct extent_buffer *leaf;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+ file_key.objectid = objectid;
+ file_key.offset = pos;
+ btrfs_set_key_type(&file_key, BTRFS_EXTENT_DATA_KEY);
+
+ ret = btrfs_insert_empty_item(trans, root, path, &file_key,
+ sizeof(*item));
+ if (ret < 0)
+ goto out;
+ BUG_ON(ret);
+ leaf = path->nodes[0];
+ item = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_file_extent_item);
+ btrfs_set_file_extent_disk_bytenr(leaf, item, disk_offset);
+ btrfs_set_file_extent_disk_num_bytes(leaf, item, disk_num_bytes);
+ btrfs_set_file_extent_offset(leaf, item, offset);
+ btrfs_set_file_extent_num_bytes(leaf, item, num_bytes);
+ btrfs_set_file_extent_ram_bytes(leaf, item, ram_bytes);
+ btrfs_set_file_extent_generation(leaf, item, trans->transid);
+ btrfs_set_file_extent_type(leaf, item, BTRFS_FILE_EXTENT_REG);
+ btrfs_set_file_extent_compression(leaf, item, compression);
+ btrfs_set_file_extent_encryption(leaf, item, encryption);
+ btrfs_set_file_extent_other_encoding(leaf, item, other_encoding);
+
+ btrfs_mark_buffer_dirty(leaf);
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
+struct btrfs_csum_item *btrfs_lookup_csum(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ u64 bytenr, int cow)
+{
+ int ret;
+ struct btrfs_key file_key;
+ struct btrfs_key found_key;
+ struct btrfs_csum_item *item;
+ struct extent_buffer *leaf;
+ u64 csum_offset = 0;
+ u16 csum_size =
+ btrfs_super_csum_size(&root->fs_info->super_copy);
+ int csums_in_item;
+
+ file_key.objectid = BTRFS_EXTENT_CSUM_OBJECTID;
+ file_key.offset = bytenr;
+ btrfs_set_key_type(&file_key, BTRFS_EXTENT_CSUM_KEY);
+ ret = btrfs_search_slot(trans, root, &file_key, path, 0, cow);
+ if (ret < 0)
+ goto fail;
+ leaf = path->nodes[0];
+ if (ret > 0) {
+ ret = 1;
+ if (path->slots[0] == 0)
+ goto fail;
+ path->slots[0]--;
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+ if (btrfs_key_type(&found_key) != BTRFS_EXTENT_CSUM_KEY)
+ goto fail;
+
+ csum_offset = (bytenr - found_key.offset) >>
+ root->fs_info->sb->s_blocksize_bits;
+ csums_in_item = btrfs_item_size_nr(leaf, path->slots[0]);
+ csums_in_item /= csum_size;
+
+ if (csum_offset >= csums_in_item) {
+ ret = -EFBIG;
+ goto fail;
+ }
+ }
+ item = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_csum_item);
+ item = (struct btrfs_csum_item *)((unsigned char *)item +
+ csum_offset * csum_size);
+ return item;
+fail:
+ if (ret > 0)
+ ret = -ENOENT;
+ return ERR_PTR(ret);
+}
+
+
+int btrfs_lookup_file_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, u64 objectid,
+ u64 offset, int mod)
+{
+ int ret;
+ struct btrfs_key file_key;
+ int ins_len = mod < 0 ? -1 : 0;
+ int cow = mod != 0;
+
+ file_key.objectid = objectid;
+ file_key.offset = offset;
+ btrfs_set_key_type(&file_key, BTRFS_EXTENT_DATA_KEY);
+ ret = btrfs_search_slot(trans, root, &file_key, path, ins_len, cow);
+ return ret;
+}
+
+
+int btrfs_lookup_bio_sums(struct btrfs_root *root, struct inode *inode,
+ struct bio *bio, u32 *dst)
+{
+ u32 sum;
+ struct bio_vec *bvec = bio->bi_io_vec;
+ int bio_index = 0;
+ u64 offset;
+ u64 item_start_offset = 0;
+ u64 item_last_offset = 0;
+ u64 disk_bytenr;
+ u32 diff;
+ u16 csum_size =
+ btrfs_super_csum_size(&root->fs_info->super_copy);
+ int ret;
+ struct btrfs_path *path;
+ struct btrfs_csum_item *item = NULL;
+ struct extent_io_tree *io_tree = &BTRFS_I(inode)->io_tree;
+
+ path = btrfs_alloc_path();
+ if (bio->bi_size > PAGE_CACHE_SIZE * 8)
+ path->reada = 2;
+
+ WARN_ON(bio->bi_vcnt <= 0);
+
+ disk_bytenr = (u64)bio->bi_sector << 9;
+ while (bio_index < bio->bi_vcnt) {
+ offset = page_offset(bvec->bv_page) + bvec->bv_offset;
+ ret = btrfs_find_ordered_sum(inode, offset, disk_bytenr, &sum);
+ if (ret == 0)
+ goto found;
+
+ if (!item || disk_bytenr < item_start_offset ||
+ disk_bytenr >= item_last_offset) {
+ struct btrfs_key found_key;
+ u32 item_size;
+
+ if (item)
+ btrfs_release_path(root, path);
+ item = btrfs_lookup_csum(NULL, root->fs_info->csum_root,
+ path, disk_bytenr, 0);
+ if (IS_ERR(item)) {
+ ret = PTR_ERR(item);
+ if (ret == -ENOENT || ret == -EFBIG)
+ ret = 0;
+ sum = 0;
+ if (BTRFS_I(inode)->root->root_key.objectid ==
+ BTRFS_DATA_RELOC_TREE_OBJECTID) {
+ set_extent_bits(io_tree, offset,
+ offset + bvec->bv_len - 1,
+ EXTENT_NODATASUM, GFP_NOFS);
+ } else {
+ printk(KERN_INFO "btrfs no csum found "
+ "for inode %lu start %llu\n",
+ inode->i_ino,
+ (unsigned long long)offset);
+ }
+ item = NULL;
+ btrfs_release_path(root, path);
+ goto found;
+ }
+ btrfs_item_key_to_cpu(path->nodes[0], &found_key,
+ path->slots[0]);
+
+ item_start_offset = found_key.offset;
+ item_size = btrfs_item_size_nr(path->nodes[0],
+ path->slots[0]);
+ item_last_offset = item_start_offset +
+ (item_size / csum_size) *
+ root->sectorsize;
+ item = btrfs_item_ptr(path->nodes[0], path->slots[0],
+ struct btrfs_csum_item);
+ }
+ /*
+ * this byte range must be able to fit inside
+ * a single leaf so it will also fit inside a u32
+ */
+ diff = disk_bytenr - item_start_offset;
+ diff = diff / root->sectorsize;
+ diff = diff * csum_size;
+
+ read_extent_buffer(path->nodes[0], &sum,
+ ((unsigned long)item) + diff,
+ csum_size);
+found:
+ if (dst)
+ *dst++ = sum;
+ else
+ set_state_private(io_tree, offset, sum);
+ disk_bytenr += bvec->bv_len;
+ bio_index++;
+ bvec++;
+ }
+ btrfs_free_path(path);
+ return 0;
+}
+
+int btrfs_lookup_csums_range(struct btrfs_root *root, u64 start, u64 end,
+ struct list_head *list)
+{
+ struct btrfs_key key;
+ struct btrfs_path *path;
+ struct extent_buffer *leaf;
+ struct btrfs_ordered_sum *sums;
+ struct btrfs_sector_sum *sector_sum;
+ struct btrfs_csum_item *item;
+ unsigned long offset;
+ int ret;
+ size_t size;
+ u64 csum_end;
+ u16 csum_size = btrfs_super_csum_size(&root->fs_info->super_copy);
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+
+ key.objectid = BTRFS_EXTENT_CSUM_OBJECTID;
+ key.offset = start;
+ key.type = BTRFS_EXTENT_CSUM_KEY;
+
+ ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ if (ret < 0)
+ goto fail;
+ if (ret > 0 && path->slots[0] > 0) {
+ leaf = path->nodes[0];
+ btrfs_item_key_to_cpu(leaf, &key, path->slots[0] - 1);
+ if (key.objectid == BTRFS_EXTENT_CSUM_OBJECTID &&
+ key.type == BTRFS_EXTENT_CSUM_KEY) {
+ offset = (start - key.offset) >>
+ root->fs_info->sb->s_blocksize_bits;
+ if (offset * csum_size <
+ btrfs_item_size_nr(leaf, path->slots[0] - 1))
+ path->slots[0]--;
+ }
+ }
+
+ while (start <= end) {
+ leaf = path->nodes[0];
+ if (path->slots[0] >= btrfs_header_nritems(leaf)) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret < 0)
+ goto fail;
+ if (ret > 0)
+ break;
+ leaf = path->nodes[0];
+ }
+
+ btrfs_item_key_to_cpu(leaf, &key, path->slots[0]);
+ if (key.objectid != BTRFS_EXTENT_CSUM_OBJECTID ||
+ key.type != BTRFS_EXTENT_CSUM_KEY)
+ break;
+
+ btrfs_item_key_to_cpu(leaf, &key, path->slots[0]);
+ if (key.offset > end)
+ break;
+
+ if (key.offset > start)
+ start = key.offset;
+
+ size = btrfs_item_size_nr(leaf, path->slots[0]);
+ csum_end = key.offset + (size / csum_size) * root->sectorsize;
+ if (csum_end <= start) {
+ path->slots[0]++;
+ continue;
+ }
+
+ csum_end = min(csum_end, end + 1);
+ item = btrfs_item_ptr(path->nodes[0], path->slots[0],
+ struct btrfs_csum_item);
+ while (start < csum_end) {
+ size = min_t(size_t, csum_end - start,
+ MAX_ORDERED_SUM_BYTES(root));
+ sums = kzalloc(btrfs_ordered_sum_size(root, size),
+ GFP_NOFS);
+ BUG_ON(!sums);
+
+ sector_sum = sums->sums;
+ sums->bytenr = start;
+ sums->len = size;
+
+ offset = (start - key.offset) >>
+ root->fs_info->sb->s_blocksize_bits;
+ offset *= csum_size;
+
+ while (size > 0) {
+ read_extent_buffer(path->nodes[0],
+ §or_sum->sum,
+ ((unsigned long)item) +
+ offset, csum_size);
+ sector_sum->bytenr = start;
+
+ size -= root->sectorsize;
+ start += root->sectorsize;
+ offset += csum_size;
+ sector_sum++;
+ }
+ list_add_tail(&sums->list, list);
+ }
+ path->slots[0]++;
+ }
+ ret = 0;
+fail:
+ btrfs_free_path(path);
+ return ret;
+}
+
+int btrfs_csum_one_bio(struct btrfs_root *root, struct inode *inode,
+ struct bio *bio, u64 file_start, int contig)
+{
+ struct btrfs_ordered_sum *sums;
+ struct btrfs_sector_sum *sector_sum;
+ struct btrfs_ordered_extent *ordered;
+ char *data;
+ struct bio_vec *bvec = bio->bi_io_vec;
+ int bio_index = 0;
+ unsigned long total_bytes = 0;
+ unsigned long this_sum_bytes = 0;
+ u64 offset;
+ u64 disk_bytenr;
+
+ WARN_ON(bio->bi_vcnt <= 0);
+ sums = kzalloc(btrfs_ordered_sum_size(root, bio->bi_size), GFP_NOFS);
+ if (!sums)
+ return -ENOMEM;
+
+ sector_sum = sums->sums;
+ disk_bytenr = (u64)bio->bi_sector << 9;
+ sums->len = bio->bi_size;
+ INIT_LIST_HEAD(&sums->list);
+
+ if (contig)
+ offset = file_start;
+ else
+ offset = page_offset(bvec->bv_page) + bvec->bv_offset;
+
+ ordered = btrfs_lookup_ordered_extent(inode, offset);
+ BUG_ON(!ordered);
+ sums->bytenr = ordered->start;
+
+ while (bio_index < bio->bi_vcnt) {
+ if (!contig)
+ offset = page_offset(bvec->bv_page) + bvec->bv_offset;
+
+ if (!contig && (offset >= ordered->file_offset + ordered->len ||
+ offset < ordered->file_offset)) {
+ unsigned long bytes_left;
+ sums->len = this_sum_bytes;
+ this_sum_bytes = 0;
+ btrfs_add_ordered_sum(inode, ordered, sums);
+ btrfs_put_ordered_extent(ordered);
+
+ bytes_left = bio->bi_size - total_bytes;
+
+ sums = kzalloc(btrfs_ordered_sum_size(root, bytes_left),
+ GFP_NOFS);
+ BUG_ON(!sums);
+ sector_sum = sums->sums;
+ sums->len = bytes_left;
+ ordered = btrfs_lookup_ordered_extent(inode, offset);
+ BUG_ON(!ordered);
+ sums->bytenr = ordered->start;
+ }
+
+ data = kmap_atomic(bvec->bv_page, KM_USER0);
+ sector_sum->sum = ~(u32)0;
+ sector_sum->sum = btrfs_csum_data(root,
+ data + bvec->bv_offset,
+ sector_sum->sum,
+ bvec->bv_len);
+ kunmap_atomic(data, KM_USER0);
+ btrfs_csum_final(sector_sum->sum,
+ (char *)§or_sum->sum);
+ sector_sum->bytenr = disk_bytenr;
+
+ sector_sum++;
+ bio_index++;
+ total_bytes += bvec->bv_len;
+ this_sum_bytes += bvec->bv_len;
+ disk_bytenr += bvec->bv_len;
+ offset += bvec->bv_len;
+ bvec++;
+ }
+ this_sum_bytes = 0;
+ btrfs_add_ordered_sum(inode, ordered, sums);
+ btrfs_put_ordered_extent(ordered);
+ return 0;
+}
+
+/*
+ * helper function for csum removal, this expects the
+ * key to describe the csum pointed to by the path, and it expects
+ * the csum to overlap the range [bytenr, len]
+ *
+ * The csum should not be entirely contained in the range and the
+ * range should not be entirely contained in the csum.
+ *
+ * This calls btrfs_truncate_item with the correct args based on the
+ * overlap, and fixes up the key as required.
+ */
+static noinline int truncate_one_csum(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct btrfs_key *key,
+ u64 bytenr, u64 len)
+{
+ struct extent_buffer *leaf;
+ u16 csum_size =
+ btrfs_super_csum_size(&root->fs_info->super_copy);
+ u64 csum_end;
+ u64 end_byte = bytenr + len;
+ u32 blocksize_bits = root->fs_info->sb->s_blocksize_bits;
+ int ret;
+
+ leaf = path->nodes[0];
+ csum_end = btrfs_item_size_nr(leaf, path->slots[0]) / csum_size;
+ csum_end <<= root->fs_info->sb->s_blocksize_bits;
+ csum_end += key->offset;
+
+ if (key->offset < bytenr && csum_end <= end_byte) {
+ /*
+ * [ bytenr - len ]
+ * [ ]
+ * [csum ]
+ * A simple truncate off the end of the item
+ */
+ u32 new_size = (bytenr - key->offset) >> blocksize_bits;
+ new_size *= csum_size;
+ ret = btrfs_truncate_item(trans, root, path, new_size, 1);
+ BUG_ON(ret);
+ } else if (key->offset >= bytenr && csum_end > end_byte &&
+ end_byte > key->offset) {
+ /*
+ * [ bytenr - len ]
+ * [ ]
+ * [csum ]
+ * we need to truncate from the beginning of the csum
+ */
+ u32 new_size = (csum_end - end_byte) >> blocksize_bits;
+ new_size *= csum_size;
+
+ ret = btrfs_truncate_item(trans, root, path, new_size, 0);
+ BUG_ON(ret);
+
+ key->offset = end_byte;
+ ret = btrfs_set_item_key_safe(trans, root, path, key);
+ BUG_ON(ret);
+ } else {
+ BUG();
+ }
+ return 0;
+}
+
+/*
+ * deletes the csum items from the csum tree for a given
+ * range of bytes.
+ */
+int btrfs_del_csums(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 bytenr, u64 len)
+{
+ struct btrfs_path *path;
+ struct btrfs_key key;
+ u64 end_byte = bytenr + len;
+ u64 csum_end;
+ struct extent_buffer *leaf;
+ int ret;
+ u16 csum_size =
+ btrfs_super_csum_size(&root->fs_info->super_copy);
+ int blocksize_bits = root->fs_info->sb->s_blocksize_bits;
+
+ root = root->fs_info->csum_root;
+
+ path = btrfs_alloc_path();
+
+ while (1) {
+ key.objectid = BTRFS_EXTENT_CSUM_OBJECTID;
+ key.offset = end_byte - 1;
+ key.type = BTRFS_EXTENT_CSUM_KEY;
+
+ ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
+ if (ret > 0) {
+ if (path->slots[0] == 0)
+ goto out;
+ path->slots[0]--;
+ }
+ leaf = path->nodes[0];
+ btrfs_item_key_to_cpu(leaf, &key, path->slots[0]);
+
+ if (key.objectid != BTRFS_EXTENT_CSUM_OBJECTID ||
+ key.type != BTRFS_EXTENT_CSUM_KEY) {
+ break;
+ }
+
+ if (key.offset >= end_byte)
+ break;
+
+ csum_end = btrfs_item_size_nr(leaf, path->slots[0]) / csum_size;
+ csum_end <<= blocksize_bits;
+ csum_end += key.offset;
+
+ /* this csum ends before we start, we're done */
+ if (csum_end <= bytenr)
+ break;
+
+ /* delete the entire item, it is inside our range */
+ if (key.offset >= bytenr && csum_end <= end_byte) {
+ ret = btrfs_del_item(trans, root, path);
+ BUG_ON(ret);
+ if (key.offset == bytenr)
+ break;
+ } else if (key.offset < bytenr && csum_end > end_byte) {
+ unsigned long offset;
+ unsigned long shift_len;
+ unsigned long item_offset;
+ /*
+ * [ bytenr - len ]
+ * [csum ]
+ *
+ * Our bytes are in the middle of the csum,
+ * we need to split this item and insert a new one.
+ *
+ * But we can't drop the path because the
+ * csum could change, get removed, extended etc.
+ *
+ * The trick here is the max size of a csum item leaves
+ * enough room in the tree block for a single
+ * item header. So, we split the item in place,
+ * adding a new header pointing to the existing
+ * bytes. Then we loop around again and we have
+ * a nicely formed csum item that we can neatly
+ * truncate.
+ */
+ offset = (bytenr - key.offset) >> blocksize_bits;
+ offset *= csum_size;
+
+ shift_len = (len >> blocksize_bits) * csum_size;
+
+ item_offset = btrfs_item_ptr_offset(leaf,
+ path->slots[0]);
+
+ memset_extent_buffer(leaf, 0, item_offset + offset,
+ shift_len);
+ key.offset = bytenr;
+
+ /*
+ * btrfs_split_item returns -EAGAIN when the
+ * item changed size or key
+ */
+ ret = btrfs_split_item(trans, root, path, &key, offset);
+ BUG_ON(ret && ret != -EAGAIN);
+
+ key.offset = end_byte - 1;
+ } else {
+ ret = truncate_one_csum(trans, root, path,
+ &key, bytenr, len);
+ BUG_ON(ret);
+ if (key.offset < bytenr)
+ break;
+ }
+ btrfs_release_path(root, path);
+ }
+out:
+ btrfs_free_path(path);
+ return 0;
+}
+
+int btrfs_csum_file_blocks(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_ordered_sum *sums)
+{
+ u64 bytenr;
+ int ret;
+ struct btrfs_key file_key;
+ struct btrfs_key found_key;
+ u64 next_offset;
+ u64 total_bytes = 0;
+ int found_next;
+ struct btrfs_path *path;
+ struct btrfs_csum_item *item;
+ struct btrfs_csum_item *item_end;
+ struct extent_buffer *leaf = NULL;
+ u64 csum_offset;
+ struct btrfs_sector_sum *sector_sum;
+ u32 nritems;
+ u32 ins_size;
+ char *eb_map;
+ char *eb_token;
+ unsigned long map_len;
+ unsigned long map_start;
+ u16 csum_size =
+ btrfs_super_csum_size(&root->fs_info->super_copy);
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+ sector_sum = sums->sums;
+again:
+ next_offset = (u64)-1;
+ found_next = 0;
+ file_key.objectid = BTRFS_EXTENT_CSUM_OBJECTID;
+ file_key.offset = sector_sum->bytenr;
+ bytenr = sector_sum->bytenr;
+ btrfs_set_key_type(&file_key, BTRFS_EXTENT_CSUM_KEY);
+
+ item = btrfs_lookup_csum(trans, root, path, sector_sum->bytenr, 1);
+ if (!IS_ERR(item)) {
+ leaf = path->nodes[0];
+ ret = 0;
+ goto found;
+ }
+ ret = PTR_ERR(item);
+ if (ret == -EFBIG) {
+ u32 item_size;
+ /* we found one, but it isn't big enough yet */
+ leaf = path->nodes[0];
+ item_size = btrfs_item_size_nr(leaf, path->slots[0]);
+ if ((item_size / csum_size) >=
+ MAX_CSUM_ITEMS(root, csum_size)) {
+ /* already at max size, make a new one */
+ goto insert;
+ }
+ } else {
+ int slot = path->slots[0] + 1;
+ /* we didn't find a csum item, insert one */
+ nritems = btrfs_header_nritems(path->nodes[0]);
+ if (path->slots[0] >= nritems - 1) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret == 1)
+ found_next = 1;
+ if (ret != 0)
+ goto insert;
+ slot = 0;
+ }
+ btrfs_item_key_to_cpu(path->nodes[0], &found_key, slot);
+ if (found_key.objectid != BTRFS_EXTENT_CSUM_OBJECTID ||
+ found_key.type != BTRFS_EXTENT_CSUM_KEY) {
+ found_next = 1;
+ goto insert;
+ }
+ next_offset = found_key.offset;
+ found_next = 1;
+ goto insert;
+ }
+
+ /*
+ * at this point, we know the tree has an item, but it isn't big
+ * enough yet to put our csum in. Grow it
+ */
+ btrfs_release_path(root, path);
+ ret = btrfs_search_slot(trans, root, &file_key, path,
+ csum_size, 1);
+ if (ret < 0)
+ goto fail_unlock;
+
+ if (ret > 0) {
+ if (path->slots[0] == 0)
+ goto insert;
+ path->slots[0]--;
+ }
+
+ leaf = path->nodes[0];
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+ csum_offset = (bytenr - found_key.offset) >>
+ root->fs_info->sb->s_blocksize_bits;
+
+ if (btrfs_key_type(&found_key) != BTRFS_EXTENT_CSUM_KEY ||
+ found_key.objectid != BTRFS_EXTENT_CSUM_OBJECTID ||
+ csum_offset >= MAX_CSUM_ITEMS(root, csum_size)) {
+ goto insert;
+ }
+
+ if (csum_offset >= btrfs_item_size_nr(leaf, path->slots[0]) /
+ csum_size) {
+ u32 diff = (csum_offset + 1) * csum_size;
+
+ /*
+ * is the item big enough already? we dropped our lock
+ * before and need to recheck
+ */
+ if (diff < btrfs_item_size_nr(leaf, path->slots[0]))
+ goto csum;
+
+ diff = diff - btrfs_item_size_nr(leaf, path->slots[0]);
+ if (diff != csum_size)
+ goto insert;
+
+ ret = btrfs_extend_item(trans, root, path, diff);
+ BUG_ON(ret);
+ goto csum;
+ }
+
+insert:
+ btrfs_release_path(root, path);
+ csum_offset = 0;
+ if (found_next) {
+ u64 tmp = total_bytes + root->sectorsize;
+ u64 next_sector = sector_sum->bytenr;
+ struct btrfs_sector_sum *next = sector_sum + 1;
+
+ while (tmp < sums->len) {
+ if (next_sector + root->sectorsize != next->bytenr)
+ break;
+ tmp += root->sectorsize;
+ next_sector = next->bytenr;
+ next++;
+ }
+ tmp = min(tmp, next_offset - file_key.offset);
+ tmp >>= root->fs_info->sb->s_blocksize_bits;
+ tmp = max((u64)1, tmp);
+ tmp = min(tmp, (u64)MAX_CSUM_ITEMS(root, csum_size));
+ ins_size = csum_size * tmp;
+ } else {
+ ins_size = csum_size;
+ }
+ ret = btrfs_insert_empty_item(trans, root, path, &file_key,
+ ins_size);
+ if (ret < 0)
+ goto fail_unlock;
+ if (ret != 0) {
+ WARN_ON(1);
+ goto fail_unlock;
+ }
+csum:
+ leaf = path->nodes[0];
+ item = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_csum_item);
+ ret = 0;
+ item = (struct btrfs_csum_item *)((unsigned char *)item +
+ csum_offset * csum_size);
+found:
+ item_end = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_csum_item);
+ item_end = (struct btrfs_csum_item *)((unsigned char *)item_end +
+ btrfs_item_size_nr(leaf, path->slots[0]));
+ eb_token = NULL;
+ cond_resched();
+next_sector:
+
+ if (!eb_token ||
+ (unsigned long)item + csum_size >= map_start + map_len) {
+ int err;
+
+ if (eb_token)
+ unmap_extent_buffer(leaf, eb_token, KM_USER1);
+ eb_token = NULL;
+ err = map_private_extent_buffer(leaf, (unsigned long)item,
+ csum_size,
+ &eb_token, &eb_map,
+ &map_start, &map_len, KM_USER1);
+ if (err)
+ eb_token = NULL;
+ }
+ if (eb_token) {
+ memcpy(eb_token + ((unsigned long)item & (PAGE_CACHE_SIZE - 1)),
+ §or_sum->sum, csum_size);
+ } else {
+ write_extent_buffer(leaf, §or_sum->sum,
+ (unsigned long)item, csum_size);
+ }
+
+ total_bytes += root->sectorsize;
+ sector_sum++;
+ if (total_bytes < sums->len) {
+ item = (struct btrfs_csum_item *)((char *)item +
+ csum_size);
+ if (item < item_end && bytenr + PAGE_CACHE_SIZE ==
+ sector_sum->bytenr) {
+ bytenr = sector_sum->bytenr;
+ goto next_sector;
+ }
+ }
+ if (eb_token) {
+ unmap_extent_buffer(leaf, eb_token, KM_USER1);
+ eb_token = NULL;
+ }
+ btrfs_mark_buffer_dirty(path->nodes[0]);
+ cond_resched();
+ if (total_bytes < sums->len) {
+ btrfs_release_path(root, path);
+ goto again;
+ }
+out:
+ btrfs_free_path(path);
+ return ret;
+
+fail_unlock:
+ goto out;
+}
diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c
new file mode 100644
index 0000000..9026833
--- /dev/null
+++ b/fs/btrfs/file.c
@@ -0,0 +1,1288 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/fs.h>
+#include <linux/pagemap.h>
+#include <linux/highmem.h>
+#include <linux/time.h>
+#include <linux/init.h>
+#include <linux/string.h>
+#include <linux/smp_lock.h>
+#include <linux/backing-dev.h>
+#include <linux/mpage.h>
+#include <linux/swap.h>
+#include <linux/writeback.h>
+#include <linux/statfs.h>
+#include <linux/compat.h>
+#include <linux/version.h>
+#include "ctree.h"
+#include "disk-io.h"
+#include "transaction.h"
+#include "btrfs_inode.h"
+#include "ioctl.h"
+#include "print-tree.h"
+#include "tree-log.h"
+#include "locking.h"
+#include "compat.h"
+
+
+/* simple helper to fault in pages and copy. This should go away
+ * and be replaced with calls into generic code.
+ */
+static noinline int btrfs_copy_from_user(loff_t pos, int num_pages,
+ int write_bytes,
+ struct page **prepared_pages,
+ const char __user *buf)
+{
+ long page_fault = 0;
+ int i;
+ int offset = pos & (PAGE_CACHE_SIZE - 1);
+
+ for (i = 0; i < num_pages && write_bytes > 0; i++, offset = 0) {
+ size_t count = min_t(size_t,
+ PAGE_CACHE_SIZE - offset, write_bytes);
+ struct page *page = prepared_pages[i];
+ fault_in_pages_readable(buf, count);
+
+ /* Copy data from userspace to the current page */
+ kmap(page);
+ page_fault = __copy_from_user(page_address(page) + offset,
+ buf, count);
+ /* Flush processor's dcache for this page */
+ flush_dcache_page(page);
+ kunmap(page);
+ buf += count;
+ write_bytes -= count;
+
+ if (page_fault)
+ break;
+ }
+ return page_fault ? -EFAULT : 0;
+}
+
+/*
+ * unlocks pages after btrfs_file_write is done with them
+ */
+static noinline void btrfs_drop_pages(struct page **pages, size_t num_pages)
+{
+ size_t i;
+ for (i = 0; i < num_pages; i++) {
+ if (!pages[i])
+ break;
+ /* page checked is some magic around finding pages that
+ * have been modified without going through btrfs_set_page_dirty
+ * clear it here
+ */
+ ClearPageChecked(pages[i]);
+ unlock_page(pages[i]);
+ mark_page_accessed(pages[i]);
+ page_cache_release(pages[i]);
+ }
+}
+
+/*
+ * after copy_from_user, pages need to be dirtied and we need to make
+ * sure holes are created between the current EOF and the start of
+ * any next extents (if required).
+ *
+ * this also makes the decision about creating an inline extent vs
+ * doing real data extents, marking pages dirty and delalloc as required.
+ */
+static noinline int dirty_and_release_pages(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct file *file,
+ struct page **pages,
+ size_t num_pages,
+ loff_t pos,
+ size_t write_bytes)
+{
+ int err = 0;
+ int i;
+ struct inode *inode = fdentry(file)->d_inode;
+ struct extent_io_tree *io_tree = &BTRFS_I(inode)->io_tree;
+ u64 hint_byte;
+ u64 num_bytes;
+ u64 start_pos;
+ u64 end_of_last_block;
+ u64 end_pos = pos + write_bytes;
+ loff_t isize = i_size_read(inode);
+
+ start_pos = pos & ~((u64)root->sectorsize - 1);
+ num_bytes = (write_bytes + pos - start_pos +
+ root->sectorsize - 1) & ~((u64)root->sectorsize - 1);
+
+ end_of_last_block = start_pos + num_bytes - 1;
+
+ lock_extent(io_tree, start_pos, end_of_last_block, GFP_NOFS);
+ trans = btrfs_join_transaction(root, 1);
+ if (!trans) {
+ err = -ENOMEM;
+ goto out_unlock;
+ }
+ btrfs_set_trans_block_group(trans, inode);
+ hint_byte = 0;
+
+ set_extent_uptodate(io_tree, start_pos, end_of_last_block, GFP_NOFS);
+
+ /* check for reserved extents on each page, we don't want
+ * to reset the delalloc bit on things that already have
+ * extents reserved.
+ */
+ btrfs_set_extent_delalloc(inode, start_pos, end_of_last_block);
+ for (i = 0; i < num_pages; i++) {
+ struct page *p = pages[i];
+ SetPageUptodate(p);
+ ClearPageChecked(p);
+ set_page_dirty(p);
+ }
+ if (end_pos > isize) {
+ i_size_write(inode, end_pos);
+ btrfs_update_inode(trans, root, inode);
+ }
+ err = btrfs_end_transaction(trans, root);
+out_unlock:
+ unlock_extent(io_tree, start_pos, end_of_last_block, GFP_NOFS);
+ return err;
+}
+
+/*
+ * this drops all the extents in the cache that intersect the range
+ * [start, end]. Existing extents are split as required.
+ */
+int btrfs_drop_extent_cache(struct inode *inode, u64 start, u64 end,
+ int skip_pinned)
+{
+ struct extent_map *em;
+ struct extent_map *split = NULL;
+ struct extent_map *split2 = NULL;
+ struct extent_map_tree *em_tree = &BTRFS_I(inode)->extent_tree;
+ u64 len = end - start + 1;
+ int ret;
+ int testend = 1;
+ unsigned long flags;
+ int compressed = 0;
+
+ WARN_ON(end < start);
+ if (end == (u64)-1) {
+ len = (u64)-1;
+ testend = 0;
+ }
+ while (1) {
+ if (!split)
+ split = alloc_extent_map(GFP_NOFS);
+ if (!split2)
+ split2 = alloc_extent_map(GFP_NOFS);
+
+ spin_lock(&em_tree->lock);
+ em = lookup_extent_mapping(em_tree, start, len);
+ if (!em) {
+ spin_unlock(&em_tree->lock);
+ break;
+ }
+ flags = em->flags;
+ if (skip_pinned && test_bit(EXTENT_FLAG_PINNED, &em->flags)) {
+ spin_unlock(&em_tree->lock);
+ if (em->start <= start &&
+ (!testend || em->start + em->len >= start + len)) {
+ free_extent_map(em);
+ break;
+ }
+ if (start < em->start) {
+ len = em->start - start;
+ } else {
+ len = start + len - (em->start + em->len);
+ start = em->start + em->len;
+ }
+ free_extent_map(em);
+ continue;
+ }
+ compressed = test_bit(EXTENT_FLAG_COMPRESSED, &em->flags);
+ clear_bit(EXTENT_FLAG_PINNED, &em->flags);
+ remove_extent_mapping(em_tree, em);
+
+ if (em->block_start < EXTENT_MAP_LAST_BYTE &&
+ em->start < start) {
+ split->start = em->start;
+ split->len = start - em->start;
+ split->orig_start = em->orig_start;
+ split->block_start = em->block_start;
+
+ if (compressed)
+ split->block_len = em->block_len;
+ else
+ split->block_len = split->len;
+
+ split->bdev = em->bdev;
+ split->flags = flags;
+ ret = add_extent_mapping(em_tree, split);
+ BUG_ON(ret);
+ free_extent_map(split);
+ split = split2;
+ split2 = NULL;
+ }
+ if (em->block_start < EXTENT_MAP_LAST_BYTE &&
+ testend && em->start + em->len > start + len) {
+ u64 diff = start + len - em->start;
+
+ split->start = start + len;
+ split->len = em->start + em->len - (start + len);
+ split->bdev = em->bdev;
+ split->flags = flags;
+
+ if (compressed) {
+ split->block_len = em->block_len;
+ split->block_start = em->block_start;
+ split->orig_start = em->orig_start;
+ } else {
+ split->block_len = split->len;
+ split->block_start = em->block_start + diff;
+ split->orig_start = split->start;
+ }
+
+ ret = add_extent_mapping(em_tree, split);
+ BUG_ON(ret);
+ free_extent_map(split);
+ split = NULL;
+ }
+ spin_unlock(&em_tree->lock);
+
+ /* once for us */
+ free_extent_map(em);
+ /* once for the tree*/
+ free_extent_map(em);
+ }
+ if (split)
+ free_extent_map(split);
+ if (split2)
+ free_extent_map(split2);
+ return 0;
+}
+
+int btrfs_check_file(struct btrfs_root *root, struct inode *inode)
+{
+ return 0;
+#if 0
+ struct btrfs_path *path;
+ struct btrfs_key found_key;
+ struct extent_buffer *leaf;
+ struct btrfs_file_extent_item *extent;
+ u64 last_offset = 0;
+ int nritems;
+ int slot;
+ int found_type;
+ int ret;
+ int err = 0;
+ u64 extent_end = 0;
+
+ path = btrfs_alloc_path();
+ ret = btrfs_lookup_file_extent(NULL, root, path, inode->i_ino,
+ last_offset, 0);
+ while (1) {
+ nritems = btrfs_header_nritems(path->nodes[0]);
+ if (path->slots[0] >= nritems) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret)
+ goto out;
+ nritems = btrfs_header_nritems(path->nodes[0]);
+ }
+ slot = path->slots[0];
+ leaf = path->nodes[0];
+ btrfs_item_key_to_cpu(leaf, &found_key, slot);
+ if (found_key.objectid != inode->i_ino)
+ break;
+ if (found_key.type != BTRFS_EXTENT_DATA_KEY)
+ goto out;
+
+ if (found_key.offset < last_offset) {
+ WARN_ON(1);
+ btrfs_print_leaf(root, leaf);
+ printk(KERN_ERR "inode %lu found offset %llu "
+ "expected %llu\n", inode->i_ino,
+ (unsigned long long)found_key.offset,
+ (unsigned long long)last_offset);
+ err = 1;
+ goto out;
+ }
+ extent = btrfs_item_ptr(leaf, slot,
+ struct btrfs_file_extent_item);
+ found_type = btrfs_file_extent_type(leaf, extent);
+ if (found_type == BTRFS_FILE_EXTENT_REG) {
+ extent_end = found_key.offset +
+ btrfs_file_extent_num_bytes(leaf, extent);
+ } else if (found_type == BTRFS_FILE_EXTENT_INLINE) {
+ struct btrfs_item *item;
+ item = btrfs_item_nr(leaf, slot);
+ extent_end = found_key.offset +
+ btrfs_file_extent_inline_len(leaf, extent);
+ extent_end = (extent_end + root->sectorsize - 1) &
+ ~((u64)root->sectorsize - 1);
+ }
+ last_offset = extent_end;
+ path->slots[0]++;
+ }
+ if (0 && last_offset < inode->i_size) {
+ WARN_ON(1);
+ btrfs_print_leaf(root, leaf);
+ printk(KERN_ERR "inode %lu found offset %llu size %llu\n",
+ inode->i_ino, (unsigned long long)last_offset,
+ (unsigned long long)inode->i_size);
+ err = 1;
+
+ }
+out:
+ btrfs_free_path(path);
+ return err;
+#endif
+}
+
+/*
+ * this is very complex, but the basic idea is to drop all extents
+ * in the range start - end. hint_block is filled in with a block number
+ * that would be a good hint to the block allocator for this file.
+ *
+ * If an extent intersects the range but is not entirely inside the range
+ * it is either truncated or split. Anything entirely inside the range
+ * is deleted from the tree.
+ *
+ * inline_limit is used to tell this code which offsets in the file to keep
+ * if they contain inline extents.
+ */
+noinline int btrfs_drop_extents(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct inode *inode,
+ u64 start, u64 end, u64 inline_limit, u64 *hint_byte)
+{
+ u64 extent_end = 0;
+ u64 locked_end = end;
+ u64 search_start = start;
+ u64 leaf_start;
+ u64 ram_bytes = 0;
+ u64 orig_parent = 0;
+ u64 disk_bytenr = 0;
+ u8 compression;
+ u8 encryption;
+ u16 other_encoding = 0;
+ u64 root_gen;
+ u64 root_owner;
+ struct extent_buffer *leaf;
+ struct btrfs_file_extent_item *extent;
+ struct btrfs_path *path;
+ struct btrfs_key key;
+ struct btrfs_file_extent_item old;
+ int keep;
+ int slot;
+ int bookend;
+ int found_type = 0;
+ int found_extent;
+ int found_inline;
+ int recow;
+ int ret;
+
+ inline_limit = 0;
+ btrfs_drop_extent_cache(inode, start, end - 1, 0);
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+ while (1) {
+ recow = 0;
+ btrfs_release_path(root, path);
+ ret = btrfs_lookup_file_extent(trans, root, path, inode->i_ino,
+ search_start, -1);
+ if (ret < 0)
+ goto out;
+ if (ret > 0) {
+ if (path->slots[0] == 0) {
+ ret = 0;
+ goto out;
+ }
+ path->slots[0]--;
+ }
+next_slot:
+ keep = 0;
+ bookend = 0;
+ found_extent = 0;
+ found_inline = 0;
+ leaf_start = 0;
+ root_gen = 0;
+ root_owner = 0;
+ compression = 0;
+ encryption = 0;
+ extent = NULL;
+ leaf = path->nodes[0];
+ slot = path->slots[0];
+ ret = 0;
+ btrfs_item_key_to_cpu(leaf, &key, slot);
+ if (btrfs_key_type(&key) == BTRFS_EXTENT_DATA_KEY &&
+ key.offset >= end) {
+ goto out;
+ }
+ if (btrfs_key_type(&key) > BTRFS_EXTENT_DATA_KEY ||
+ key.objectid != inode->i_ino) {
+ goto out;
+ }
+ if (recow) {
+ search_start = max(key.offset, start);
+ continue;
+ }
+ if (btrfs_key_type(&key) == BTRFS_EXTENT_DATA_KEY) {
+ extent = btrfs_item_ptr(leaf, slot,
+ struct btrfs_file_extent_item);
+ found_type = btrfs_file_extent_type(leaf, extent);
+ compression = btrfs_file_extent_compression(leaf,
+ extent);
+ encryption = btrfs_file_extent_encryption(leaf,
+ extent);
+ other_encoding = btrfs_file_extent_other_encoding(leaf,
+ extent);
+ if (found_type == BTRFS_FILE_EXTENT_REG ||
+ found_type == BTRFS_FILE_EXTENT_PREALLOC) {
+ extent_end =
+ btrfs_file_extent_disk_bytenr(leaf,
+ extent);
+ if (extent_end)
+ *hint_byte = extent_end;
+
+ extent_end = key.offset +
+ btrfs_file_extent_num_bytes(leaf, extent);
+ ram_bytes = btrfs_file_extent_ram_bytes(leaf,
+ extent);
+ found_extent = 1;
+ } else if (found_type == BTRFS_FILE_EXTENT_INLINE) {
+ found_inline = 1;
+ extent_end = key.offset +
+ btrfs_file_extent_inline_len(leaf, extent);
+ }
+ } else {
+ extent_end = search_start;
+ }
+
+ /* we found nothing we can drop */
+ if ((!found_extent && !found_inline) ||
+ search_start >= extent_end) {
+ int nextret;
+ u32 nritems;
+ nritems = btrfs_header_nritems(leaf);
+ if (slot >= nritems - 1) {
+ nextret = btrfs_next_leaf(root, path);
+ if (nextret)
+ goto out;
+ recow = 1;
+ } else {
+ path->slots[0]++;
+ }
+ goto next_slot;
+ }
+
+ if (end <= extent_end && start >= key.offset && found_inline)
+ *hint_byte = EXTENT_MAP_INLINE;
+
+ if (found_extent) {
+ read_extent_buffer(leaf, &old, (unsigned long)extent,
+ sizeof(old));
+ root_gen = btrfs_header_generation(leaf);
+ root_owner = btrfs_header_owner(leaf);
+ leaf_start = leaf->start;
+ }
+
+ if (end < extent_end && end >= key.offset) {
+ bookend = 1;
+ if (found_inline && start <= key.offset)
+ keep = 1;
+ }
+
+ if (bookend && found_extent) {
+ if (locked_end < extent_end) {
+ ret = try_lock_extent(&BTRFS_I(inode)->io_tree,
+ locked_end, extent_end - 1,
+ GFP_NOFS);
+ if (!ret) {
+ btrfs_release_path(root, path);
+ lock_extent(&BTRFS_I(inode)->io_tree,
+ locked_end, extent_end - 1,
+ GFP_NOFS);
+ locked_end = extent_end;
+ continue;
+ }
+ locked_end = extent_end;
+ }
+ orig_parent = path->nodes[0]->start;
+ disk_bytenr = le64_to_cpu(old.disk_bytenr);
+ if (disk_bytenr != 0) {
+ ret = btrfs_inc_extent_ref(trans, root,
+ disk_bytenr,
+ le64_to_cpu(old.disk_num_bytes),
+ orig_parent, root->root_key.objectid,
+ trans->transid, inode->i_ino);
+ BUG_ON(ret);
+ }
+ }
+
+ if (found_inline) {
+ u64 mask = root->sectorsize - 1;
+ search_start = (extent_end + mask) & ~mask;
+ } else
+ search_start = extent_end;
+
+ /* truncate existing extent */
+ if (start > key.offset) {
+ u64 new_num;
+ u64 old_num;
+ keep = 1;
+ WARN_ON(start & (root->sectorsize - 1));
+ if (found_extent) {
+ new_num = start - key.offset;
+ old_num = btrfs_file_extent_num_bytes(leaf,
+ extent);
+ *hint_byte =
+ btrfs_file_extent_disk_bytenr(leaf,
+ extent);
+ if (btrfs_file_extent_disk_bytenr(leaf,
+ extent)) {
+ inode_sub_bytes(inode, old_num -
+ new_num);
+ }
+ btrfs_set_file_extent_num_bytes(leaf,
+ extent, new_num);
+ btrfs_mark_buffer_dirty(leaf);
+ } else if (key.offset < inline_limit &&
+ (end > extent_end) &&
+ (inline_limit < extent_end)) {
+ u32 new_size;
+ new_size = btrfs_file_extent_calc_inline_size(
+ inline_limit - key.offset);
+ inode_sub_bytes(inode, extent_end -
+ inline_limit);
+ btrfs_set_file_extent_ram_bytes(leaf, extent,
+ new_size);
+ if (!compression && !encryption) {
+ btrfs_truncate_item(trans, root, path,
+ new_size, 1);
+ }
+ }
+ }
+ /* delete the entire extent */
+ if (!keep) {
+ if (found_inline)
+ inode_sub_bytes(inode, extent_end -
+ key.offset);
+ ret = btrfs_del_item(trans, root, path);
+ /* TODO update progress marker and return */
+ BUG_ON(ret);
+ extent = NULL;
+ btrfs_release_path(root, path);
+ /* the extent will be freed later */
+ }
+ if (bookend && found_inline && start <= key.offset) {
+ u32 new_size;
+ new_size = btrfs_file_extent_calc_inline_size(
+ extent_end - end);
+ inode_sub_bytes(inode, end - key.offset);
+ btrfs_set_file_extent_ram_bytes(leaf, extent,
+ new_size);
+ if (!compression && !encryption)
+ ret = btrfs_truncate_item(trans, root, path,
+ new_size, 0);
+ BUG_ON(ret);
+ }
+ /* create bookend, splitting the extent in two */
+ if (bookend && found_extent) {
+ struct btrfs_key ins;
+ ins.objectid = inode->i_ino;
+ ins.offset = end;
+ btrfs_set_key_type(&ins, BTRFS_EXTENT_DATA_KEY);
+
+ btrfs_release_path(root, path);
+ ret = btrfs_insert_empty_item(trans, root, path, &ins,
+ sizeof(*extent));
+ BUG_ON(ret);
+
+ leaf = path->nodes[0];
+ extent = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_file_extent_item);
+ write_extent_buffer(leaf, &old,
+ (unsigned long)extent, sizeof(old));
+
+ btrfs_set_file_extent_compression(leaf, extent,
+ compression);
+ btrfs_set_file_extent_encryption(leaf, extent,
+ encryption);
+ btrfs_set_file_extent_other_encoding(leaf, extent,
+ other_encoding);
+ btrfs_set_file_extent_offset(leaf, extent,
+ le64_to_cpu(old.offset) + end - key.offset);
+ WARN_ON(le64_to_cpu(old.num_bytes) <
+ (extent_end - end));
+ btrfs_set_file_extent_num_bytes(leaf, extent,
+ extent_end - end);
+
+ /*
+ * set the ram bytes to the size of the full extent
+ * before splitting. This is a worst case flag,
+ * but its the best we can do because we don't know
+ * how splitting affects compression
+ */
+ btrfs_set_file_extent_ram_bytes(leaf, extent,
+ ram_bytes);
+ btrfs_set_file_extent_type(leaf, extent, found_type);
+
+ btrfs_mark_buffer_dirty(path->nodes[0]);
+
+ if (disk_bytenr != 0) {
+ ret = btrfs_update_extent_ref(trans, root,
+ disk_bytenr, orig_parent,
+ leaf->start,
+ root->root_key.objectid,
+ trans->transid, ins.objectid);
+
+ BUG_ON(ret);
+ }
+ btrfs_release_path(root, path);
+ if (disk_bytenr != 0)
+ inode_add_bytes(inode, extent_end - end);
+ }
+
+ if (found_extent && !keep) {
+ u64 old_disk_bytenr = le64_to_cpu(old.disk_bytenr);
+
+ if (old_disk_bytenr != 0) {
+ inode_sub_bytes(inode,
+ le64_to_cpu(old.num_bytes));
+ ret = btrfs_free_extent(trans, root,
+ old_disk_bytenr,
+ le64_to_cpu(old.disk_num_bytes),
+ leaf_start, root_owner,
+ root_gen, key.objectid, 0);
+ BUG_ON(ret);
+ *hint_byte = old_disk_bytenr;
+ }
+ }
+
+ if (search_start >= end) {
+ ret = 0;
+ goto out;
+ }
+ }
+out:
+ btrfs_free_path(path);
+ if (locked_end > end) {
+ unlock_extent(&BTRFS_I(inode)->io_tree, end, locked_end - 1,
+ GFP_NOFS);
+ }
+ btrfs_check_file(root, inode);
+ return ret;
+}
+
+static int extent_mergeable(struct extent_buffer *leaf, int slot,
+ u64 objectid, u64 bytenr, u64 *start, u64 *end)
+{
+ struct btrfs_file_extent_item *fi;
+ struct btrfs_key key;
+ u64 extent_end;
+
+ if (slot < 0 || slot >= btrfs_header_nritems(leaf))
+ return 0;
+
+ btrfs_item_key_to_cpu(leaf, &key, slot);
+ if (key.objectid != objectid || key.type != BTRFS_EXTENT_DATA_KEY)
+ return 0;
+
+ fi = btrfs_item_ptr(leaf, slot, struct btrfs_file_extent_item);
+ if (btrfs_file_extent_type(leaf, fi) != BTRFS_FILE_EXTENT_REG ||
+ btrfs_file_extent_disk_bytenr(leaf, fi) != bytenr ||
+ btrfs_file_extent_compression(leaf, fi) ||
+ btrfs_file_extent_encryption(leaf, fi) ||
+ btrfs_file_extent_other_encoding(leaf, fi))
+ return 0;
+
+ extent_end = key.offset + btrfs_file_extent_num_bytes(leaf, fi);
+ if ((*start && *start != key.offset) || (*end && *end != extent_end))
+ return 0;
+
+ *start = key.offset;
+ *end = extent_end;
+ return 1;
+}
+
+/*
+ * Mark extent in the range start - end as written.
+ *
+ * This changes extent type from 'pre-allocated' to 'regular'. If only
+ * part of extent is marked as written, the extent will be split into
+ * two or three.
+ */
+int btrfs_mark_extent_written(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct inode *inode, u64 start, u64 end)
+{
+ struct extent_buffer *leaf;
+ struct btrfs_path *path;
+ struct btrfs_file_extent_item *fi;
+ struct btrfs_key key;
+ u64 bytenr;
+ u64 num_bytes;
+ u64 extent_end;
+ u64 extent_offset;
+ u64 other_start;
+ u64 other_end;
+ u64 split = start;
+ u64 locked_end = end;
+ u64 orig_parent;
+ int extent_type;
+ int split_end = 1;
+ int ret;
+
+ btrfs_drop_extent_cache(inode, start, end - 1, 0);
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+again:
+ key.objectid = inode->i_ino;
+ key.type = BTRFS_EXTENT_DATA_KEY;
+ if (split == start)
+ key.offset = split;
+ else
+ key.offset = split - 1;
+
+ ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
+ if (ret > 0 && path->slots[0] > 0)
+ path->slots[0]--;
+
+ leaf = path->nodes[0];
+ btrfs_item_key_to_cpu(leaf, &key, path->slots[0]);
+ BUG_ON(key.objectid != inode->i_ino ||
+ key.type != BTRFS_EXTENT_DATA_KEY);
+ fi = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_file_extent_item);
+ extent_type = btrfs_file_extent_type(leaf, fi);
+ BUG_ON(extent_type != BTRFS_FILE_EXTENT_PREALLOC);
+ extent_end = key.offset + btrfs_file_extent_num_bytes(leaf, fi);
+ BUG_ON(key.offset > start || extent_end < end);
+
+ bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
+ num_bytes = btrfs_file_extent_disk_num_bytes(leaf, fi);
+ extent_offset = btrfs_file_extent_offset(leaf, fi);
+
+ if (key.offset == start)
+ split = end;
+
+ if (key.offset == start && extent_end == end) {
+ int del_nr = 0;
+ int del_slot = 0;
+ u64 leaf_owner = btrfs_header_owner(leaf);
+ u64 leaf_gen = btrfs_header_generation(leaf);
+ other_start = end;
+ other_end = 0;
+ if (extent_mergeable(leaf, path->slots[0] + 1, inode->i_ino,
+ bytenr, &other_start, &other_end)) {
+ extent_end = other_end;
+ del_slot = path->slots[0] + 1;
+ del_nr++;
+ ret = btrfs_free_extent(trans, root, bytenr, num_bytes,
+ leaf->start, leaf_owner,
+ leaf_gen, inode->i_ino, 0);
+ BUG_ON(ret);
+ }
+ other_start = 0;
+ other_end = start;
+ if (extent_mergeable(leaf, path->slots[0] - 1, inode->i_ino,
+ bytenr, &other_start, &other_end)) {
+ key.offset = other_start;
+ del_slot = path->slots[0];
+ del_nr++;
+ ret = btrfs_free_extent(trans, root, bytenr, num_bytes,
+ leaf->start, leaf_owner,
+ leaf_gen, inode->i_ino, 0);
+ BUG_ON(ret);
+ }
+ split_end = 0;
+ if (del_nr == 0) {
+ btrfs_set_file_extent_type(leaf, fi,
+ BTRFS_FILE_EXTENT_REG);
+ goto done;
+ }
+
+ fi = btrfs_item_ptr(leaf, del_slot - 1,
+ struct btrfs_file_extent_item);
+ btrfs_set_file_extent_type(leaf, fi, BTRFS_FILE_EXTENT_REG);
+ btrfs_set_file_extent_num_bytes(leaf, fi,
+ extent_end - key.offset);
+ btrfs_mark_buffer_dirty(leaf);
+
+ ret = btrfs_del_items(trans, root, path, del_slot, del_nr);
+ BUG_ON(ret);
+ goto done;
+ } else if (split == start) {
+ if (locked_end < extent_end) {
+ ret = try_lock_extent(&BTRFS_I(inode)->io_tree,
+ locked_end, extent_end - 1, GFP_NOFS);
+ if (!ret) {
+ btrfs_release_path(root, path);
+ lock_extent(&BTRFS_I(inode)->io_tree,
+ locked_end, extent_end - 1, GFP_NOFS);
+ locked_end = extent_end;
+ goto again;
+ }
+ locked_end = extent_end;
+ }
+ btrfs_set_file_extent_num_bytes(leaf, fi, split - key.offset);
+ extent_offset += split - key.offset;
+ } else {
+ BUG_ON(key.offset != start);
+ btrfs_set_file_extent_offset(leaf, fi, extent_offset +
+ split - key.offset);
+ btrfs_set_file_extent_num_bytes(leaf, fi, extent_end - split);
+ key.offset = split;
+ btrfs_set_item_key_safe(trans, root, path, &key);
+ extent_end = split;
+ }
+
+ if (extent_end == end) {
+ split_end = 0;
+ extent_type = BTRFS_FILE_EXTENT_REG;
+ }
+ if (extent_end == end && split == start) {
+ other_start = end;
+ other_end = 0;
+ if (extent_mergeable(leaf, path->slots[0] + 1, inode->i_ino,
+ bytenr, &other_start, &other_end)) {
+ path->slots[0]++;
+ fi = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_file_extent_item);
+ key.offset = split;
+ btrfs_set_item_key_safe(trans, root, path, &key);
+ btrfs_set_file_extent_offset(leaf, fi, extent_offset);
+ btrfs_set_file_extent_num_bytes(leaf, fi,
+ other_end - split);
+ goto done;
+ }
+ }
+ if (extent_end == end && split == end) {
+ other_start = 0;
+ other_end = start;
+ if (extent_mergeable(leaf, path->slots[0] - 1 , inode->i_ino,
+ bytenr, &other_start, &other_end)) {
+ path->slots[0]--;
+ fi = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_file_extent_item);
+ btrfs_set_file_extent_num_bytes(leaf, fi, extent_end -
+ other_start);
+ goto done;
+ }
+ }
+
+ btrfs_mark_buffer_dirty(leaf);
+
+ orig_parent = leaf->start;
+ ret = btrfs_inc_extent_ref(trans, root, bytenr, num_bytes,
+ orig_parent, root->root_key.objectid,
+ trans->transid, inode->i_ino);
+ BUG_ON(ret);
+ btrfs_release_path(root, path);
+
+ key.offset = start;
+ ret = btrfs_insert_empty_item(trans, root, path, &key, sizeof(*fi));
+ BUG_ON(ret);
+
+ leaf = path->nodes[0];
+ fi = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_file_extent_item);
+ btrfs_set_file_extent_generation(leaf, fi, trans->transid);
+ btrfs_set_file_extent_type(leaf, fi, extent_type);
+ btrfs_set_file_extent_disk_bytenr(leaf, fi, bytenr);
+ btrfs_set_file_extent_disk_num_bytes(leaf, fi, num_bytes);
+ btrfs_set_file_extent_offset(leaf, fi, extent_offset);
+ btrfs_set_file_extent_num_bytes(leaf, fi, extent_end - key.offset);
+ btrfs_set_file_extent_ram_bytes(leaf, fi, num_bytes);
+ btrfs_set_file_extent_compression(leaf, fi, 0);
+ btrfs_set_file_extent_encryption(leaf, fi, 0);
+ btrfs_set_file_extent_other_encoding(leaf, fi, 0);
+
+ if (orig_parent != leaf->start) {
+ ret = btrfs_update_extent_ref(trans, root, bytenr,
+ orig_parent, leaf->start,
+ root->root_key.objectid,
+ trans->transid, inode->i_ino);
+ BUG_ON(ret);
+ }
+done:
+ btrfs_mark_buffer_dirty(leaf);
+ btrfs_release_path(root, path);
+ if (split_end && split == start) {
+ split = end;
+ goto again;
+ }
+ if (locked_end > end) {
+ unlock_extent(&BTRFS_I(inode)->io_tree, end, locked_end - 1,
+ GFP_NOFS);
+ }
+ btrfs_free_path(path);
+ return 0;
+}
+
+/*
+ * this gets pages into the page cache and locks them down, it also properly
+ * waits for data=ordered extents to finish before allowing the pages to be
+ * modified.
+ */
+static noinline int prepare_pages(struct btrfs_root *root, struct file *file,
+ struct page **pages, size_t num_pages,
+ loff_t pos, unsigned long first_index,
+ unsigned long last_index, size_t write_bytes)
+{
+ int i;
+ unsigned long index = pos >> PAGE_CACHE_SHIFT;
+ struct inode *inode = fdentry(file)->d_inode;
+ int err = 0;
+ u64 start_pos;
+ u64 last_pos;
+
+ start_pos = pos & ~((u64)root->sectorsize - 1);
+ last_pos = ((u64)index + num_pages) << PAGE_CACHE_SHIFT;
+
+ if (start_pos > inode->i_size) {
+ err = btrfs_cont_expand(inode, start_pos);
+ if (err)
+ return err;
+ }
+
+ memset(pages, 0, num_pages * sizeof(struct page *));
+again:
+ for (i = 0; i < num_pages; i++) {
+ pages[i] = grab_cache_page(inode->i_mapping, index + i);
+ if (!pages[i]) {
+ err = -ENOMEM;
+ BUG_ON(1);
+ }
+ wait_on_page_writeback(pages[i]);
+ }
+ if (start_pos < inode->i_size) {
+ struct btrfs_ordered_extent *ordered;
+ lock_extent(&BTRFS_I(inode)->io_tree,
+ start_pos, last_pos - 1, GFP_NOFS);
+ ordered = btrfs_lookup_first_ordered_extent(inode,
+ last_pos - 1);
+ if (ordered &&
+ ordered->file_offset + ordered->len > start_pos &&
+ ordered->file_offset < last_pos) {
+ btrfs_put_ordered_extent(ordered);
+ unlock_extent(&BTRFS_I(inode)->io_tree,
+ start_pos, last_pos - 1, GFP_NOFS);
+ for (i = 0; i < num_pages; i++) {
+ unlock_page(pages[i]);
+ page_cache_release(pages[i]);
+ }
+ btrfs_wait_ordered_range(inode, start_pos,
+ last_pos - start_pos);
+ goto again;
+ }
+ if (ordered)
+ btrfs_put_ordered_extent(ordered);
+
+ clear_extent_bits(&BTRFS_I(inode)->io_tree, start_pos,
+ last_pos - 1, EXTENT_DIRTY | EXTENT_DELALLOC,
+ GFP_NOFS);
+ unlock_extent(&BTRFS_I(inode)->io_tree,
+ start_pos, last_pos - 1, GFP_NOFS);
+ }
+ for (i = 0; i < num_pages; i++) {
+ clear_page_dirty_for_io(pages[i]);
+ set_page_extent_mapped(pages[i]);
+ WARN_ON(!PageLocked(pages[i]));
+ }
+ return 0;
+}
+
+static ssize_t btrfs_file_write(struct file *file, const char __user *buf,
+ size_t count, loff_t *ppos)
+{
+ loff_t pos;
+ loff_t start_pos;
+ ssize_t num_written = 0;
+ ssize_t err = 0;
+ int ret = 0;
+ struct inode *inode = fdentry(file)->d_inode;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct page **pages = NULL;
+ int nrptrs;
+ struct page *pinned[2];
+ unsigned long first_index;
+ unsigned long last_index;
+ int will_write;
+
+ will_write = ((file->f_flags & O_SYNC) || IS_SYNC(inode) ||
+ (file->f_flags & O_DIRECT));
+
+ nrptrs = min((count + PAGE_CACHE_SIZE - 1) / PAGE_CACHE_SIZE,
+ PAGE_CACHE_SIZE / (sizeof(struct page *)));
+ pinned[0] = NULL;
+ pinned[1] = NULL;
+
+ pos = *ppos;
+ start_pos = pos;
+
+ vfs_check_frozen(inode->i_sb, SB_FREEZE_WRITE);
+ current->backing_dev_info = inode->i_mapping->backing_dev_info;
+ err = generic_write_checks(file, &pos, &count, S_ISBLK(inode->i_mode));
+ if (err)
+ goto out_nolock;
+ if (count == 0)
+ goto out_nolock;
+
+ err = file_remove_suid(file);
+ if (err)
+ goto out_nolock;
+ file_update_time(file);
+
+ pages = kmalloc(nrptrs * sizeof(struct page *), GFP_KERNEL);
+
+ mutex_lock(&inode->i_mutex);
+ BTRFS_I(inode)->sequence++;
+ first_index = pos >> PAGE_CACHE_SHIFT;
+ last_index = (pos + count) >> PAGE_CACHE_SHIFT;
+
+ /*
+ * there are lots of better ways to do this, but this code
+ * makes sure the first and last page in the file range are
+ * up to date and ready for cow
+ */
+ if ((pos & (PAGE_CACHE_SIZE - 1))) {
+ pinned[0] = grab_cache_page(inode->i_mapping, first_index);
+ if (!PageUptodate(pinned[0])) {
+ ret = btrfs_readpage(NULL, pinned[0]);
+ BUG_ON(ret);
+ wait_on_page_locked(pinned[0]);
+ } else {
+ unlock_page(pinned[0]);
+ }
+ }
+ if ((pos + count) & (PAGE_CACHE_SIZE - 1)) {
+ pinned[1] = grab_cache_page(inode->i_mapping, last_index);
+ if (!PageUptodate(pinned[1])) {
+ ret = btrfs_readpage(NULL, pinned[1]);
+ BUG_ON(ret);
+ wait_on_page_locked(pinned[1]);
+ } else {
+ unlock_page(pinned[1]);
+ }
+ }
+
+ while (count > 0) {
+ size_t offset = pos & (PAGE_CACHE_SIZE - 1);
+ size_t write_bytes = min(count, nrptrs *
+ (size_t)PAGE_CACHE_SIZE -
+ offset);
+ size_t num_pages = (write_bytes + PAGE_CACHE_SIZE - 1) >>
+ PAGE_CACHE_SHIFT;
+
+ WARN_ON(num_pages > nrptrs);
+ memset(pages, 0, sizeof(struct page *) * nrptrs);
+
+ ret = btrfs_check_free_space(root, write_bytes, 0);
+ if (ret)
+ goto out;
+
+ ret = prepare_pages(root, file, pages, num_pages,
+ pos, first_index, last_index,
+ write_bytes);
+ if (ret)
+ goto out;
+
+ ret = btrfs_copy_from_user(pos, num_pages,
+ write_bytes, pages, buf);
+ if (ret) {
+ btrfs_drop_pages(pages, num_pages);
+ goto out;
+ }
+
+ ret = dirty_and_release_pages(NULL, root, file, pages,
+ num_pages, pos, write_bytes);
+ btrfs_drop_pages(pages, num_pages);
+ if (ret)
+ goto out;
+
+ if (will_write) {
+ btrfs_fdatawrite_range(inode->i_mapping, pos,
+ pos + write_bytes - 1,
+ WB_SYNC_NONE);
+ } else {
+ balance_dirty_pages_ratelimited_nr(inode->i_mapping,
+ num_pages);
+ if (num_pages <
+ (root->leafsize >> PAGE_CACHE_SHIFT) + 1)
+ btrfs_btree_balance_dirty(root, 1);
+ btrfs_throttle(root);
+ }
+
+ buf += write_bytes;
+ count -= write_bytes;
+ pos += write_bytes;
+ num_written += write_bytes;
+
+ cond_resched();
+ }
+out:
+ mutex_unlock(&inode->i_mutex);
+
+out_nolock:
+ kfree(pages);
+ if (pinned[0])
+ page_cache_release(pinned[0]);
+ if (pinned[1])
+ page_cache_release(pinned[1]);
+ *ppos = pos;
+
+ if (num_written > 0 && will_write) {
+ struct btrfs_trans_handle *trans;
+
+ err = btrfs_wait_ordered_range(inode, start_pos, num_written);
+ if (err)
+ num_written = err;
+
+ if ((file->f_flags & O_SYNC) || IS_SYNC(inode)) {
+ trans = btrfs_start_transaction(root, 1);
+ ret = btrfs_log_dentry_safe(trans, root,
+ file->f_dentry);
+ if (ret == 0) {
+ btrfs_sync_log(trans, root);
+ btrfs_end_transaction(trans, root);
+ } else {
+ btrfs_commit_transaction(trans, root);
+ }
+ }
+ if (file->f_flags & O_DIRECT) {
+ invalidate_mapping_pages(inode->i_mapping,
+ start_pos >> PAGE_CACHE_SHIFT,
+ (start_pos + num_written - 1) >> PAGE_CACHE_SHIFT);
+ }
+ }
+ current->backing_dev_info = NULL;
+ return num_written ? num_written : err;
+}
+
+int btrfs_release_file(struct inode *inode, struct file *filp)
+{
+ if (filp->private_data)
+ btrfs_ioctl_trans_end(filp);
+ return 0;
+}
+
+/*
+ * fsync call for both files and directories. This logs the inode into
+ * the tree log instead of forcing full commits whenever possible.
+ *
+ * It needs to call filemap_fdatawait so that all ordered extent updates are
+ * in the metadata btree are up to date for copying to the log.
+ *
+ * It drops the inode mutex before doing the tree log commit. This is an
+ * important optimization for directories because holding the mutex prevents
+ * new operations on the dir while we write to disk.
+ */
+int btrfs_sync_file(struct file *file, struct dentry *dentry, int datasync)
+{
+ struct inode *inode = dentry->d_inode;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ int ret = 0;
+ struct btrfs_trans_handle *trans;
+
+ /*
+ * check the transaction that last modified this inode
+ * and see if its already been committed
+ */
+ if (!BTRFS_I(inode)->last_trans)
+ goto out;
+
+ mutex_lock(&root->fs_info->trans_mutex);
+ if (BTRFS_I(inode)->last_trans <=
+ root->fs_info->last_trans_committed) {
+ BTRFS_I(inode)->last_trans = 0;
+ mutex_unlock(&root->fs_info->trans_mutex);
+ goto out;
+ }
+ mutex_unlock(&root->fs_info->trans_mutex);
+
+ root->fs_info->tree_log_batch++;
+ filemap_fdatawrite(inode->i_mapping);
+ btrfs_wait_ordered_range(inode, 0, (u64)-1);
+ root->fs_info->tree_log_batch++;
+
+ /*
+ * ok we haven't committed the transaction yet, lets do a commit
+ */
+ if (file->private_data)
+ btrfs_ioctl_trans_end(file);
+
+ trans = btrfs_start_transaction(root, 1);
+ if (!trans) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ ret = btrfs_log_dentry_safe(trans, root, file->f_dentry);
+ if (ret < 0)
+ goto out;
+
+ /* we've logged all the items and now have a consistent
+ * version of the file in the log. It is possible that
+ * someone will come in and modify the file, but that's
+ * fine because the log is consistent on disk, and we
+ * have references to all of the file's extents
+ *
+ * It is possible that someone will come in and log the
+ * file again, but that will end up using the synchronization
+ * inside btrfs_sync_log to keep things safe.
+ */
+ mutex_unlock(&file->f_dentry->d_inode->i_mutex);
+
+ if (ret > 0) {
+ ret = btrfs_commit_transaction(trans, root);
+ } else {
+ btrfs_sync_log(trans, root);
+ ret = btrfs_end_transaction(trans, root);
+ }
+ mutex_lock(&file->f_dentry->d_inode->i_mutex);
+out:
+ return ret > 0 ? EIO : ret;
+}
+
+static struct vm_operations_struct btrfs_file_vm_ops = {
+ .fault = filemap_fault,
+ .page_mkwrite = btrfs_page_mkwrite,
+};
+
+static int btrfs_file_mmap(struct file *filp, struct vm_area_struct *vma)
+{
+ vma->vm_ops = &btrfs_file_vm_ops;
+ file_accessed(filp);
+ return 0;
+}
+
+struct file_operations btrfs_file_operations = {
+ .llseek = generic_file_llseek,
+ .read = do_sync_read,
+ .aio_read = generic_file_aio_read,
+ .splice_read = generic_file_splice_read,
+ .write = btrfs_file_write,
+ .mmap = btrfs_file_mmap,
+ .open = generic_file_open,
+ .release = btrfs_release_file,
+ .fsync = btrfs_sync_file,
+ .unlocked_ioctl = btrfs_ioctl,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = btrfs_ioctl,
+#endif
+};
diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
new file mode 100644
index 0000000..d1e5f0e
--- /dev/null
+++ b/fs/btrfs/free-space-cache.c
@@ -0,0 +1,495 @@
+/*
+ * Copyright (C) 2008 Red Hat. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/sched.h>
+#include "ctree.h"
+
+static int tree_insert_offset(struct rb_root *root, u64 offset,
+ struct rb_node *node)
+{
+ struct rb_node **p = &root->rb_node;
+ struct rb_node *parent = NULL;
+ struct btrfs_free_space *info;
+
+ while (*p) {
+ parent = *p;
+ info = rb_entry(parent, struct btrfs_free_space, offset_index);
+
+ if (offset < info->offset)
+ p = &(*p)->rb_left;
+ else if (offset > info->offset)
+ p = &(*p)->rb_right;
+ else
+ return -EEXIST;
+ }
+
+ rb_link_node(node, parent, p);
+ rb_insert_color(node, root);
+
+ return 0;
+}
+
+static int tree_insert_bytes(struct rb_root *root, u64 bytes,
+ struct rb_node *node)
+{
+ struct rb_node **p = &root->rb_node;
+ struct rb_node *parent = NULL;
+ struct btrfs_free_space *info;
+
+ while (*p) {
+ parent = *p;
+ info = rb_entry(parent, struct btrfs_free_space, bytes_index);
+
+ if (bytes < info->bytes)
+ p = &(*p)->rb_left;
+ else
+ p = &(*p)->rb_right;
+ }
+
+ rb_link_node(node, parent, p);
+ rb_insert_color(node, root);
+
+ return 0;
+}
+
+/*
+ * searches the tree for the given offset. If contains is set we will return
+ * the free space that contains the given offset. If contains is not set we
+ * will return the free space that starts at or after the given offset and is
+ * at least bytes long.
+ */
+static struct btrfs_free_space *tree_search_offset(struct rb_root *root,
+ u64 offset, u64 bytes,
+ int contains)
+{
+ struct rb_node *n = root->rb_node;
+ struct btrfs_free_space *entry, *ret = NULL;
+
+ while (n) {
+ entry = rb_entry(n, struct btrfs_free_space, offset_index);
+
+ if (offset < entry->offset) {
+ if (!contains &&
+ (!ret || entry->offset < ret->offset) &&
+ (bytes <= entry->bytes))
+ ret = entry;
+ n = n->rb_left;
+ } else if (offset > entry->offset) {
+ if ((entry->offset + entry->bytes - 1) >= offset &&
+ bytes <= entry->bytes) {
+ ret = entry;
+ break;
+ }
+ n = n->rb_right;
+ } else {
+ if (bytes > entry->bytes) {
+ n = n->rb_right;
+ continue;
+ }
+ ret = entry;
+ break;
+ }
+ }
+
+ return ret;
+}
+
+/*
+ * return a chunk at least bytes size, as close to offset that we can get.
+ */
+static struct btrfs_free_space *tree_search_bytes(struct rb_root *root,
+ u64 offset, u64 bytes)
+{
+ struct rb_node *n = root->rb_node;
+ struct btrfs_free_space *entry, *ret = NULL;
+
+ while (n) {
+ entry = rb_entry(n, struct btrfs_free_space, bytes_index);
+
+ if (bytes < entry->bytes) {
+ /*
+ * We prefer to get a hole size as close to the size we
+ * are asking for so we don't take small slivers out of
+ * huge holes, but we also want to get as close to the
+ * offset as possible so we don't have a whole lot of
+ * fragmentation.
+ */
+ if (offset <= entry->offset) {
+ if (!ret)
+ ret = entry;
+ else if (entry->bytes < ret->bytes)
+ ret = entry;
+ else if (entry->offset < ret->offset)
+ ret = entry;
+ }
+ n = n->rb_left;
+ } else if (bytes > entry->bytes) {
+ n = n->rb_right;
+ } else {
+ /*
+ * Ok we may have multiple chunks of the wanted size,
+ * so we don't want to take the first one we find, we
+ * want to take the one closest to our given offset, so
+ * keep searching just in case theres a better match.
+ */
+ n = n->rb_right;
+ if (offset > entry->offset)
+ continue;
+ else if (!ret || entry->offset < ret->offset)
+ ret = entry;
+ }
+ }
+
+ return ret;
+}
+
+static void unlink_free_space(struct btrfs_block_group_cache *block_group,
+ struct btrfs_free_space *info)
+{
+ rb_erase(&info->offset_index, &block_group->free_space_offset);
+ rb_erase(&info->bytes_index, &block_group->free_space_bytes);
+}
+
+static int link_free_space(struct btrfs_block_group_cache *block_group,
+ struct btrfs_free_space *info)
+{
+ int ret = 0;
+
+
+ ret = tree_insert_offset(&block_group->free_space_offset, info->offset,
+ &info->offset_index);
+ if (ret)
+ return ret;
+
+ ret = tree_insert_bytes(&block_group->free_space_bytes, info->bytes,
+ &info->bytes_index);
+ if (ret)
+ return ret;
+
+ return ret;
+}
+
+static int __btrfs_add_free_space(struct btrfs_block_group_cache *block_group,
+ u64 offset, u64 bytes)
+{
+ struct btrfs_free_space *right_info;
+ struct btrfs_free_space *left_info;
+ struct btrfs_free_space *info = NULL;
+ struct btrfs_free_space *alloc_info;
+ int ret = 0;
+
+ alloc_info = kzalloc(sizeof(struct btrfs_free_space), GFP_NOFS);
+ if (!alloc_info)
+ return -ENOMEM;
+
+ /*
+ * first we want to see if there is free space adjacent to the range we
+ * are adding, if there is remove that struct and add a new one to
+ * cover the entire range
+ */
+ right_info = tree_search_offset(&block_group->free_space_offset,
+ offset+bytes, 0, 1);
+ left_info = tree_search_offset(&block_group->free_space_offset,
+ offset-1, 0, 1);
+
+ if (right_info && right_info->offset == offset+bytes) {
+ unlink_free_space(block_group, right_info);
+ info = right_info;
+ info->offset = offset;
+ info->bytes += bytes;
+ } else if (right_info && right_info->offset != offset+bytes) {
+ printk(KERN_ERR "btrfs adding space in the middle of an "
+ "existing free space area. existing: "
+ "offset=%llu, bytes=%llu. new: offset=%llu, "
+ "bytes=%llu\n", (unsigned long long)right_info->offset,
+ (unsigned long long)right_info->bytes,
+ (unsigned long long)offset,
+ (unsigned long long)bytes);
+ BUG();
+ }
+
+ if (left_info) {
+ unlink_free_space(block_group, left_info);
+
+ if (unlikely((left_info->offset + left_info->bytes) !=
+ offset)) {
+ printk(KERN_ERR "btrfs free space to the left "
+ "of new free space isn't "
+ "quite right. existing: offset=%llu, "
+ "bytes=%llu. new: offset=%llu, bytes=%llu\n",
+ (unsigned long long)left_info->offset,
+ (unsigned long long)left_info->bytes,
+ (unsigned long long)offset,
+ (unsigned long long)bytes);
+ BUG();
+ }
+
+ if (info) {
+ info->offset = left_info->offset;
+ info->bytes += left_info->bytes;
+ kfree(left_info);
+ } else {
+ info = left_info;
+ info->bytes += bytes;
+ }
+ }
+
+ if (info) {
+ ret = link_free_space(block_group, info);
+ if (!ret)
+ info = NULL;
+ goto out;
+ }
+
+ info = alloc_info;
+ alloc_info = NULL;
+ info->offset = offset;
+ info->bytes = bytes;
+
+ ret = link_free_space(block_group, info);
+ if (ret)
+ kfree(info);
+out:
+ if (ret) {
+ printk(KERN_ERR "btrfs: unable to add free space :%d\n", ret);
+ if (ret == -EEXIST)
+ BUG();
+ }
+
+ kfree(alloc_info);
+
+ return ret;
+}
+
+static int
+__btrfs_remove_free_space(struct btrfs_block_group_cache *block_group,
+ u64 offset, u64 bytes)
+{
+ struct btrfs_free_space *info;
+ int ret = 0;
+
+ info = tree_search_offset(&block_group->free_space_offset, offset, 0,
+ 1);
+
+ if (info && info->offset == offset) {
+ if (info->bytes < bytes) {
+ printk(KERN_ERR "Found free space at %llu, size %llu,"
+ "trying to use %llu\n",
+ (unsigned long long)info->offset,
+ (unsigned long long)info->bytes,
+ (unsigned long long)bytes);
+ WARN_ON(1);
+ ret = -EINVAL;
+ goto out;
+ }
+ unlink_free_space(block_group, info);
+
+ if (info->bytes == bytes) {
+ kfree(info);
+ goto out;
+ }
+
+ info->offset += bytes;
+ info->bytes -= bytes;
+
+ ret = link_free_space(block_group, info);
+ BUG_ON(ret);
+ } else if (info && info->offset < offset &&
+ info->offset + info->bytes >= offset + bytes) {
+ u64 old_start = info->offset;
+ /*
+ * we're freeing space in the middle of the info,
+ * this can happen during tree log replay
+ *
+ * first unlink the old info and then
+ * insert it again after the hole we're creating
+ */
+ unlink_free_space(block_group, info);
+ if (offset + bytes < info->offset + info->bytes) {
+ u64 old_end = info->offset + info->bytes;
+
+ info->offset = offset + bytes;
+ info->bytes = old_end - info->offset;
+ ret = link_free_space(block_group, info);
+ BUG_ON(ret);
+ } else {
+ /* the hole we're creating ends at the end
+ * of the info struct, just free the info
+ */
+ kfree(info);
+ }
+
+ /* step two, insert a new info struct to cover anything
+ * before the hole
+ */
+ ret = __btrfs_add_free_space(block_group, old_start,
+ offset - old_start);
+ BUG_ON(ret);
+ } else {
+ WARN_ON(1);
+ }
+out:
+ return ret;
+}
+
+int btrfs_add_free_space(struct btrfs_block_group_cache *block_group,
+ u64 offset, u64 bytes)
+{
+ int ret;
+ struct btrfs_free_space *sp;
+
+ mutex_lock(&block_group->alloc_mutex);
+ ret = __btrfs_add_free_space(block_group, offset, bytes);
+ sp = tree_search_offset(&block_group->free_space_offset, offset, 0, 1);
+ BUG_ON(!sp);
+ mutex_unlock(&block_group->alloc_mutex);
+
+ return ret;
+}
+
+int btrfs_add_free_space_lock(struct btrfs_block_group_cache *block_group,
+ u64 offset, u64 bytes)
+{
+ int ret;
+ struct btrfs_free_space *sp;
+
+ ret = __btrfs_add_free_space(block_group, offset, bytes);
+ sp = tree_search_offset(&block_group->free_space_offset, offset, 0, 1);
+ BUG_ON(!sp);
+
+ return ret;
+}
+
+int btrfs_remove_free_space(struct btrfs_block_group_cache *block_group,
+ u64 offset, u64 bytes)
+{
+ int ret = 0;
+
+ mutex_lock(&block_group->alloc_mutex);
+ ret = __btrfs_remove_free_space(block_group, offset, bytes);
+ mutex_unlock(&block_group->alloc_mutex);
+
+ return ret;
+}
+
+int btrfs_remove_free_space_lock(struct btrfs_block_group_cache *block_group,
+ u64 offset, u64 bytes)
+{
+ int ret;
+
+ ret = __btrfs_remove_free_space(block_group, offset, bytes);
+
+ return ret;
+}
+
+void btrfs_dump_free_space(struct btrfs_block_group_cache *block_group,
+ u64 bytes)
+{
+ struct btrfs_free_space *info;
+ struct rb_node *n;
+ int count = 0;
+
+ for (n = rb_first(&block_group->free_space_offset); n; n = rb_next(n)) {
+ info = rb_entry(n, struct btrfs_free_space, offset_index);
+ if (info->bytes >= bytes)
+ count++;
+ }
+ printk(KERN_INFO "%d blocks of free space at or bigger than bytes is"
+ "\n", count);
+}
+
+u64 btrfs_block_group_free_space(struct btrfs_block_group_cache *block_group)
+{
+ struct btrfs_free_space *info;
+ struct rb_node *n;
+ u64 ret = 0;
+
+ for (n = rb_first(&block_group->free_space_offset); n;
+ n = rb_next(n)) {
+ info = rb_entry(n, struct btrfs_free_space, offset_index);
+ ret += info->bytes;
+ }
+
+ return ret;
+}
+
+void btrfs_remove_free_space_cache(struct btrfs_block_group_cache *block_group)
+{
+ struct btrfs_free_space *info;
+ struct rb_node *node;
+
+ mutex_lock(&block_group->alloc_mutex);
+ while ((node = rb_last(&block_group->free_space_bytes)) != NULL) {
+ info = rb_entry(node, struct btrfs_free_space, bytes_index);
+ unlink_free_space(block_group, info);
+ kfree(info);
+ if (need_resched()) {
+ mutex_unlock(&block_group->alloc_mutex);
+ cond_resched();
+ mutex_lock(&block_group->alloc_mutex);
+ }
+ }
+ mutex_unlock(&block_group->alloc_mutex);
+}
+
+#if 0
+static struct btrfs_free_space *btrfs_find_free_space_offset(struct
+ btrfs_block_group_cache
+ *block_group, u64 offset,
+ u64 bytes)
+{
+ struct btrfs_free_space *ret;
+
+ mutex_lock(&block_group->alloc_mutex);
+ ret = tree_search_offset(&block_group->free_space_offset, offset,
+ bytes, 0);
+ mutex_unlock(&block_group->alloc_mutex);
+
+ return ret;
+}
+
+static struct btrfs_free_space *btrfs_find_free_space_bytes(struct
+ btrfs_block_group_cache
+ *block_group, u64 offset,
+ u64 bytes)
+{
+ struct btrfs_free_space *ret;
+
+ mutex_lock(&block_group->alloc_mutex);
+
+ ret = tree_search_bytes(&block_group->free_space_bytes, offset, bytes);
+ mutex_unlock(&block_group->alloc_mutex);
+
+ return ret;
+}
+#endif
+
+struct btrfs_free_space *btrfs_find_free_space(struct btrfs_block_group_cache
+ *block_group, u64 offset,
+ u64 bytes)
+{
+ struct btrfs_free_space *ret = NULL;
+
+ ret = tree_search_offset(&block_group->free_space_offset, offset,
+ bytes, 0);
+ if (!ret)
+ ret = tree_search_bytes(&block_group->free_space_bytes,
+ offset, bytes);
+
+ return ret;
+}
diff --git a/fs/btrfs/hash.h b/fs/btrfs/hash.h
new file mode 100644
index 0000000..2a020b2
--- /dev/null
+++ b/fs/btrfs/hash.h
@@ -0,0 +1,27 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __HASH__
+#define __HASH__
+
+#include "crc32c.h"
+static inline u64 btrfs_name_hash(const char *name, int len)
+{
+ return btrfs_crc32c((u32)~1, name, len);
+}
+#endif
diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
new file mode 100644
index 0000000..3d46fa1
--- /dev/null
+++ b/fs/btrfs/inode-item.c
@@ -0,0 +1,206 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include "ctree.h"
+#include "disk-io.h"
+#include "transaction.h"
+
+static int find_name_in_backref(struct btrfs_path *path, const char *name,
+ int name_len, struct btrfs_inode_ref **ref_ret)
+{
+ struct extent_buffer *leaf;
+ struct btrfs_inode_ref *ref;
+ unsigned long ptr;
+ unsigned long name_ptr;
+ u32 item_size;
+ u32 cur_offset = 0;
+ int len;
+
+ leaf = path->nodes[0];
+ item_size = btrfs_item_size_nr(leaf, path->slots[0]);
+ ptr = btrfs_item_ptr_offset(leaf, path->slots[0]);
+ while (cur_offset < item_size) {
+ ref = (struct btrfs_inode_ref *)(ptr + cur_offset);
+ len = btrfs_inode_ref_name_len(leaf, ref);
+ name_ptr = (unsigned long)(ref + 1);
+ cur_offset += len + sizeof(*ref);
+ if (len != name_len)
+ continue;
+ if (memcmp_extent_buffer(leaf, name, name_ptr, name_len) == 0) {
+ *ref_ret = ref;
+ return 1;
+ }
+ }
+ return 0;
+}
+
+int btrfs_del_inode_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ const char *name, int name_len,
+ u64 inode_objectid, u64 ref_objectid, u64 *index)
+{
+ struct btrfs_path *path;
+ struct btrfs_key key;
+ struct btrfs_inode_ref *ref;
+ struct extent_buffer *leaf;
+ unsigned long ptr;
+ unsigned long item_start;
+ u32 item_size;
+ u32 sub_item_len;
+ int ret;
+ int del_len = name_len + sizeof(*ref);
+
+ key.objectid = inode_objectid;
+ key.offset = ref_objectid;
+ btrfs_set_key_type(&key, BTRFS_INODE_REF_KEY);
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
+ if (ret > 0) {
+ ret = -ENOENT;
+ goto out;
+ } else if (ret < 0) {
+ goto out;
+ }
+ if (!find_name_in_backref(path, name, name_len, &ref)) {
+ ret = -ENOENT;
+ goto out;
+ }
+ leaf = path->nodes[0];
+ item_size = btrfs_item_size_nr(leaf, path->slots[0]);
+
+ if (index)
+ *index = btrfs_inode_ref_index(leaf, ref);
+
+ if (del_len == item_size) {
+ ret = btrfs_del_item(trans, root, path);
+ goto out;
+ }
+ ptr = (unsigned long)ref;
+ sub_item_len = name_len + sizeof(*ref);
+ item_start = btrfs_item_ptr_offset(leaf, path->slots[0]);
+ memmove_extent_buffer(leaf, ptr, ptr + sub_item_len,
+ item_size - (ptr + sub_item_len - item_start));
+ ret = btrfs_truncate_item(trans, root, path,
+ item_size - sub_item_len, 1);
+ BUG_ON(ret);
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
+int btrfs_insert_inode_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ const char *name, int name_len,
+ u64 inode_objectid, u64 ref_objectid, u64 index)
+{
+ struct btrfs_path *path;
+ struct btrfs_key key;
+ struct btrfs_inode_ref *ref;
+ unsigned long ptr;
+ int ret;
+ int ins_len = name_len + sizeof(*ref);
+
+ key.objectid = inode_objectid;
+ key.offset = ref_objectid;
+ btrfs_set_key_type(&key, BTRFS_INODE_REF_KEY);
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ ret = btrfs_insert_empty_item(trans, root, path, &key,
+ ins_len);
+ if (ret == -EEXIST) {
+ u32 old_size;
+
+ if (find_name_in_backref(path, name, name_len, &ref))
+ goto out;
+
+ old_size = btrfs_item_size_nr(path->nodes[0], path->slots[0]);
+ ret = btrfs_extend_item(trans, root, path, ins_len);
+ BUG_ON(ret);
+ ref = btrfs_item_ptr(path->nodes[0], path->slots[0],
+ struct btrfs_inode_ref);
+ ref = (struct btrfs_inode_ref *)((unsigned long)ref + old_size);
+ btrfs_set_inode_ref_name_len(path->nodes[0], ref, name_len);
+ btrfs_set_inode_ref_index(path->nodes[0], ref, index);
+ ptr = (unsigned long)(ref + 1);
+ ret = 0;
+ } else if (ret < 0) {
+ goto out;
+ } else {
+ ref = btrfs_item_ptr(path->nodes[0], path->slots[0],
+ struct btrfs_inode_ref);
+ btrfs_set_inode_ref_name_len(path->nodes[0], ref, name_len);
+ btrfs_set_inode_ref_index(path->nodes[0], ref, index);
+ ptr = (unsigned long)(ref + 1);
+ }
+ write_extent_buffer(path->nodes[0], name, ptr, name_len);
+ btrfs_mark_buffer_dirty(path->nodes[0]);
+
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
+int btrfs_insert_empty_inode(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, u64 objectid)
+{
+ struct btrfs_key key;
+ int ret;
+ key.objectid = objectid;
+ btrfs_set_key_type(&key, BTRFS_INODE_ITEM_KEY);
+ key.offset = 0;
+
+ ret = btrfs_insert_empty_item(trans, root, path, &key,
+ sizeof(struct btrfs_inode_item));
+ if (ret == 0 && objectid > root->highest_inode)
+ root->highest_inode = objectid;
+ return ret;
+}
+
+int btrfs_lookup_inode(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root, struct btrfs_path *path,
+ struct btrfs_key *location, int mod)
+{
+ int ins_len = mod < 0 ? -1 : 0;
+ int cow = mod != 0;
+ int ret;
+ int slot;
+ struct extent_buffer *leaf;
+ struct btrfs_key found_key;
+
+ ret = btrfs_search_slot(trans, root, location, path, ins_len, cow);
+ if (ret > 0 && btrfs_key_type(location) == BTRFS_ROOT_ITEM_KEY &&
+ location->offset == (u64)-1 && path->slots[0] != 0) {
+ slot = path->slots[0] - 1;
+ leaf = path->nodes[0];
+ btrfs_item_key_to_cpu(leaf, &found_key, slot);
+ if (found_key.objectid == location->objectid &&
+ btrfs_key_type(&found_key) == btrfs_key_type(location)) {
+ path->slots[0]--;
+ return 0;
+ }
+ }
+ return ret;
+}
diff --git a/fs/btrfs/inode-map.c b/fs/btrfs/inode-map.c
new file mode 100644
index 0000000..2aa7987
--- /dev/null
+++ b/fs/btrfs/inode-map.c
@@ -0,0 +1,144 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include "ctree.h"
+#include "disk-io.h"
+#include "transaction.h"
+
+int btrfs_find_highest_inode(struct btrfs_root *root, u64 *objectid)
+{
+ struct btrfs_path *path;
+ int ret;
+ struct extent_buffer *l;
+ struct btrfs_key search_key;
+ struct btrfs_key found_key;
+ int slot;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+
+ search_key.objectid = BTRFS_LAST_FREE_OBJECTID;
+ search_key.type = -1;
+ search_key.offset = (u64)-1;
+ ret = btrfs_search_slot(NULL, root, &search_key, path, 0, 0);
+ if (ret < 0)
+ goto error;
+ BUG_ON(ret == 0);
+ if (path->slots[0] > 0) {
+ slot = path->slots[0] - 1;
+ l = path->nodes[0];
+ btrfs_item_key_to_cpu(l, &found_key, slot);
+ *objectid = found_key.objectid;
+ } else {
+ *objectid = BTRFS_FIRST_FREE_OBJECTID;
+ }
+ ret = 0;
+error:
+ btrfs_free_path(path);
+ return ret;
+}
+
+/*
+ * walks the btree of allocated inodes and find a hole.
+ */
+int btrfs_find_free_objectid(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 dirid, u64 *objectid)
+{
+ struct btrfs_path *path;
+ struct btrfs_key key;
+ int ret;
+ int slot = 0;
+ u64 last_ino = 0;
+ int start_found;
+ struct extent_buffer *l;
+ struct btrfs_key search_key;
+ u64 search_start = dirid;
+
+ mutex_lock(&root->objectid_mutex);
+ if (root->last_inode_alloc >= BTRFS_FIRST_FREE_OBJECTID &&
+ root->last_inode_alloc < BTRFS_LAST_FREE_OBJECTID) {
+ *objectid = ++root->last_inode_alloc;
+ mutex_unlock(&root->objectid_mutex);
+ return 0;
+ }
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+ search_start = max(search_start, BTRFS_FIRST_FREE_OBJECTID);
+ search_key.objectid = search_start;
+ search_key.type = 0;
+ search_key.offset = 0;
+
+ btrfs_init_path(path);
+ start_found = 0;
+ ret = btrfs_search_slot(trans, root, &search_key, path, 0, 0);
+ if (ret < 0)
+ goto error;
+
+ while (1) {
+ l = path->nodes[0];
+ slot = path->slots[0];
+ if (slot >= btrfs_header_nritems(l)) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret == 0)
+ continue;
+ if (ret < 0)
+ goto error;
+ if (!start_found) {
+ *objectid = search_start;
+ start_found = 1;
+ goto found;
+ }
+ *objectid = last_ino > search_start ?
+ last_ino : search_start;
+ goto found;
+ }
+ btrfs_item_key_to_cpu(l, &key, slot);
+ if (key.objectid >= search_start) {
+ if (start_found) {
+ if (last_ino < search_start)
+ last_ino = search_start;
+ if (key.objectid > last_ino) {
+ *objectid = last_ino;
+ goto found;
+ }
+ } else if (key.objectid > search_start) {
+ *objectid = search_start;
+ goto found;
+ }
+ }
+ if (key.objectid >= BTRFS_LAST_FREE_OBJECTID)
+ break;
+
+ start_found = 1;
+ last_ino = key.objectid + 1;
+ path->slots[0]++;
+ }
+ BUG_ON(1);
+found:
+ btrfs_release_path(root, path);
+ btrfs_free_path(path);
+ BUG_ON(*objectid < search_start);
+ mutex_unlock(&root->objectid_mutex);
+ return 0;
+error:
+ btrfs_release_path(root, path);
+ btrfs_free_path(path);
+ mutex_unlock(&root->objectid_mutex);
+ return ret;
+}
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
new file mode 100644
index 0000000..8adfe05
--- /dev/null
+++ b/fs/btrfs/inode.c
@@ -0,0 +1,5035 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/kernel.h>
+#include <linux/bio.h>
+#include <linux/buffer_head.h>
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <linux/pagemap.h>
+#include <linux/highmem.h>
+#include <linux/time.h>
+#include <linux/init.h>
+#include <linux/string.h>
+#include <linux/smp_lock.h>
+#include <linux/backing-dev.h>
+#include <linux/mpage.h>
+#include <linux/swap.h>
+#include <linux/writeback.h>
+#include <linux/statfs.h>
+#include <linux/compat.h>
+#include <linux/bit_spinlock.h>
+#include <linux/version.h>
+#include <linux/xattr.h>
+#include <linux/posix_acl.h>
+#include <linux/falloc.h>
+#include "compat.h"
+#include "ctree.h"
+#include "disk-io.h"
+#include "transaction.h"
+#include "btrfs_inode.h"
+#include "ioctl.h"
+#include "print-tree.h"
+#include "volumes.h"
+#include "ordered-data.h"
+#include "xattr.h"
+#include "tree-log.h"
+#include "ref-cache.h"
+#include "compression.h"
+
+struct btrfs_iget_args {
+ u64 ino;
+ struct btrfs_root *root;
+};
+
+static struct inode_operations btrfs_dir_inode_operations;
+static struct inode_operations btrfs_symlink_inode_operations;
+static struct inode_operations btrfs_dir_ro_inode_operations;
+static struct inode_operations btrfs_special_inode_operations;
+static struct inode_operations btrfs_file_inode_operations;
+static struct address_space_operations btrfs_aops;
+static struct address_space_operations btrfs_symlink_aops;
+static struct file_operations btrfs_dir_file_operations;
+static struct extent_io_ops btrfs_extent_io_ops;
+
+static struct kmem_cache *btrfs_inode_cachep;
+struct kmem_cache *btrfs_trans_handle_cachep;
+struct kmem_cache *btrfs_transaction_cachep;
+struct kmem_cache *btrfs_bit_radix_cachep;
+struct kmem_cache *btrfs_path_cachep;
+
+#define S_SHIFT 12
+static unsigned char btrfs_type_by_mode[S_IFMT >> S_SHIFT] = {
+ [S_IFREG >> S_SHIFT] = BTRFS_FT_REG_FILE,
+ [S_IFDIR >> S_SHIFT] = BTRFS_FT_DIR,
+ [S_IFCHR >> S_SHIFT] = BTRFS_FT_CHRDEV,
+ [S_IFBLK >> S_SHIFT] = BTRFS_FT_BLKDEV,
+ [S_IFIFO >> S_SHIFT] = BTRFS_FT_FIFO,
+ [S_IFSOCK >> S_SHIFT] = BTRFS_FT_SOCK,
+ [S_IFLNK >> S_SHIFT] = BTRFS_FT_SYMLINK,
+};
+
+static void btrfs_truncate(struct inode *inode);
+static int btrfs_finish_ordered_io(struct inode *inode, u64 start, u64 end);
+static noinline int cow_file_range(struct inode *inode,
+ struct page *locked_page,
+ u64 start, u64 end, int *page_started,
+ unsigned long *nr_written, int unlock);
+
+/*
+ * a very lame attempt at stopping writes when the FS is 85% full. There
+ * are countless ways this is incorrect, but it is better than nothing.
+ */
+int btrfs_check_free_space(struct btrfs_root *root, u64 num_required,
+ int for_del)
+{
+ u64 total;
+ u64 used;
+ u64 thresh;
+ int ret = 0;
+
+ spin_lock(&root->fs_info->delalloc_lock);
+ total = btrfs_super_total_bytes(&root->fs_info->super_copy);
+ used = btrfs_super_bytes_used(&root->fs_info->super_copy);
+ if (for_del)
+ thresh = total * 90;
+ else
+ thresh = total * 85;
+
+ do_div(thresh, 100);
+
+ if (used + root->fs_info->delalloc_bytes + num_required > thresh)
+ ret = -ENOSPC;
+ spin_unlock(&root->fs_info->delalloc_lock);
+ return ret;
+}
+
+/*
+ * this does all the hard work for inserting an inline extent into
+ * the btree. The caller should have done a btrfs_drop_extents so that
+ * no overlapping inline items exist in the btree
+ */
+static noinline int insert_inline_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct inode *inode,
+ u64 start, size_t size, size_t compressed_size,
+ struct page **compressed_pages)
+{
+ struct btrfs_key key;
+ struct btrfs_path *path;
+ struct extent_buffer *leaf;
+ struct page *page = NULL;
+ char *kaddr;
+ unsigned long ptr;
+ struct btrfs_file_extent_item *ei;
+ int err = 0;
+ int ret;
+ size_t cur_size = size;
+ size_t datasize;
+ unsigned long offset;
+ int use_compress = 0;
+
+ if (compressed_size && compressed_pages) {
+ use_compress = 1;
+ cur_size = compressed_size;
+ }
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ btrfs_set_trans_block_group(trans, inode);
+
+ key.objectid = inode->i_ino;
+ key.offset = start;
+ btrfs_set_key_type(&key, BTRFS_EXTENT_DATA_KEY);
+ datasize = btrfs_file_extent_calc_inline_size(cur_size);
+
+ inode_add_bytes(inode, size);
+ ret = btrfs_insert_empty_item(trans, root, path, &key,
+ datasize);
+ BUG_ON(ret);
+ if (ret) {
+ err = ret;
+ goto fail;
+ }
+ leaf = path->nodes[0];
+ ei = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_file_extent_item);
+ btrfs_set_file_extent_generation(leaf, ei, trans->transid);
+ btrfs_set_file_extent_type(leaf, ei, BTRFS_FILE_EXTENT_INLINE);
+ btrfs_set_file_extent_encryption(leaf, ei, 0);
+ btrfs_set_file_extent_other_encoding(leaf, ei, 0);
+ btrfs_set_file_extent_ram_bytes(leaf, ei, size);
+ ptr = btrfs_file_extent_inline_start(ei);
+
+ if (use_compress) {
+ struct page *cpage;
+ int i = 0;
+ while (compressed_size > 0) {
+ cpage = compressed_pages[i];
+ cur_size = min_t(unsigned long, compressed_size,
+ PAGE_CACHE_SIZE);
+
+ kaddr = kmap(cpage);
+ write_extent_buffer(leaf, kaddr, ptr, cur_size);
+ kunmap(cpage);
+
+ i++;
+ ptr += cur_size;
+ compressed_size -= cur_size;
+ }
+ btrfs_set_file_extent_compression(leaf, ei,
+ BTRFS_COMPRESS_ZLIB);
+ } else {
+ page = find_get_page(inode->i_mapping,
+ start >> PAGE_CACHE_SHIFT);
+ btrfs_set_file_extent_compression(leaf, ei, 0);
+ kaddr = kmap_atomic(page, KM_USER0);
+ offset = start & (PAGE_CACHE_SIZE - 1);
+ write_extent_buffer(leaf, kaddr + offset, ptr, size);
+ kunmap_atomic(kaddr, KM_USER0);
+ page_cache_release(page);
+ }
+ btrfs_mark_buffer_dirty(leaf);
+ btrfs_free_path(path);
+
+ BTRFS_I(inode)->disk_i_size = inode->i_size;
+ btrfs_update_inode(trans, root, inode);
+ return 0;
+fail:
+ btrfs_free_path(path);
+ return err;
+}
+
+
+/*
+ * conditionally insert an inline extent into the file. This
+ * does the checks required to make sure the data is small enough
+ * to fit as an inline extent.
+ */
+static int cow_file_range_inline(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct inode *inode, u64 start, u64 end,
+ size_t compressed_size,
+ struct page **compressed_pages)
+{
+ u64 isize = i_size_read(inode);
+ u64 actual_end = min(end + 1, isize);
+ u64 inline_len = actual_end - start;
+ u64 aligned_end = (end + root->sectorsize - 1) &
+ ~((u64)root->sectorsize - 1);
+ u64 hint_byte;
+ u64 data_len = inline_len;
+ int ret;
+
+ if (compressed_size)
+ data_len = compressed_size;
+
+ if (start > 0 ||
+ actual_end >= PAGE_CACHE_SIZE ||
+ data_len >= BTRFS_MAX_INLINE_DATA_SIZE(root) ||
+ (!compressed_size &&
+ (actual_end & (root->sectorsize - 1)) == 0) ||
+ end + 1 < isize ||
+ data_len > root->fs_info->max_inline) {
+ return 1;
+ }
+
+ ret = btrfs_drop_extents(trans, root, inode, start,
+ aligned_end, start, &hint_byte);
+ BUG_ON(ret);
+
+ if (isize > actual_end)
+ inline_len = min_t(u64, isize, actual_end);
+ ret = insert_inline_extent(trans, root, inode, start,
+ inline_len, compressed_size,
+ compressed_pages);
+ BUG_ON(ret);
+ btrfs_drop_extent_cache(inode, start, aligned_end, 0);
+ return 0;
+}
+
+struct async_extent {
+ u64 start;
+ u64 ram_size;
+ u64 compressed_size;
+ struct page **pages;
+ unsigned long nr_pages;
+ struct list_head list;
+};
+
+struct async_cow {
+ struct inode *inode;
+ struct btrfs_root *root;
+ struct page *locked_page;
+ u64 start;
+ u64 end;
+ struct list_head extents;
+ struct btrfs_work work;
+};
+
+static noinline int add_async_extent(struct async_cow *cow,
+ u64 start, u64 ram_size,
+ u64 compressed_size,
+ struct page **pages,
+ unsigned long nr_pages)
+{
+ struct async_extent *async_extent;
+
+ async_extent = kmalloc(sizeof(*async_extent), GFP_NOFS);
+ async_extent->start = start;
+ async_extent->ram_size = ram_size;
+ async_extent->compressed_size = compressed_size;
+ async_extent->pages = pages;
+ async_extent->nr_pages = nr_pages;
+ list_add_tail(&async_extent->list, &cow->extents);
+ return 0;
+}
+
+/*
+ * we create compressed extents in two phases. The first
+ * phase compresses a range of pages that have already been
+ * locked (both pages and state bits are locked).
+ *
+ * This is done inside an ordered work queue, and the compression
+ * is spread across many cpus. The actual IO submission is step
+ * two, and the ordered work queue takes care of making sure that
+ * happens in the same order things were put onto the queue by
+ * writepages and friends.
+ *
+ * If this code finds it can't get good compression, it puts an
+ * entry onto the work queue to write the uncompressed bytes. This
+ * makes sure that both compressed inodes and uncompressed inodes
+ * are written in the same order that pdflush sent them down.
+ */
+static noinline int compress_file_range(struct inode *inode,
+ struct page *locked_page,
+ u64 start, u64 end,
+ struct async_cow *async_cow,
+ int *num_added)
+{
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_trans_handle *trans;
+ u64 num_bytes;
+ u64 orig_start;
+ u64 disk_num_bytes;
+ u64 blocksize = root->sectorsize;
+ u64 actual_end;
+ u64 isize = i_size_read(inode);
+ int ret = 0;
+ struct page **pages = NULL;
+ unsigned long nr_pages;
+ unsigned long nr_pages_ret = 0;
+ unsigned long total_compressed = 0;
+ unsigned long total_in = 0;
+ unsigned long max_compressed = 128 * 1024;
+ unsigned long max_uncompressed = 128 * 1024;
+ int i;
+ int will_compress;
+
+ orig_start = start;
+
+ actual_end = min_t(u64, isize, end + 1);
+again:
+ will_compress = 0;
+ nr_pages = (end >> PAGE_CACHE_SHIFT) - (start >> PAGE_CACHE_SHIFT) + 1;
+ nr_pages = min(nr_pages, (128 * 1024UL) / PAGE_CACHE_SIZE);
+
+ total_compressed = actual_end - start;
+
+ /* we want to make sure that amount of ram required to uncompress
+ * an extent is reasonable, so we limit the total size in ram
+ * of a compressed extent to 128k. This is a crucial number
+ * because it also controls how easily we can spread reads across
+ * cpus for decompression.
+ *
+ * We also want to make sure the amount of IO required to do
+ * a random read is reasonably small, so we limit the size of
+ * a compressed extent to 128k.
+ */
+ total_compressed = min(total_compressed, max_uncompressed);
+ num_bytes = (end - start + blocksize) & ~(blocksize - 1);
+ num_bytes = max(blocksize, num_bytes);
+ disk_num_bytes = num_bytes;
+ total_in = 0;
+ ret = 0;
+
+ /*
+ * we do compression for mount -o compress and when the
+ * inode has not been flagged as nocompress. This flag can
+ * change at any time if we discover bad compression ratios.
+ */
+ if (!btrfs_test_flag(inode, NOCOMPRESS) &&
+ btrfs_test_opt(root, COMPRESS)) {
+ WARN_ON(pages);
+ pages = kzalloc(sizeof(struct page *) * nr_pages, GFP_NOFS);
+
+ ret = btrfs_zlib_compress_pages(inode->i_mapping, start,
+ total_compressed, pages,
+ nr_pages, &nr_pages_ret,
+ &total_in,
+ &total_compressed,
+ max_compressed);
+
+ if (!ret) {
+ unsigned long offset = total_compressed &
+ (PAGE_CACHE_SIZE - 1);
+ struct page *page = pages[nr_pages_ret - 1];
+ char *kaddr;
+
+ /* zero the tail end of the last page, we might be
+ * sending it down to disk
+ */
+ if (offset) {
+ kaddr = kmap_atomic(page, KM_USER0);
+ memset(kaddr + offset, 0,
+ PAGE_CACHE_SIZE - offset);
+ kunmap_atomic(kaddr, KM_USER0);
+ }
+ will_compress = 1;
+ }
+ }
+ if (start == 0) {
+ trans = btrfs_join_transaction(root, 1);
+ BUG_ON(!trans);
+ btrfs_set_trans_block_group(trans, inode);
+
+ /* lets try to make an inline extent */
+ if (ret || total_in < (actual_end - start)) {
+ /* we didn't compress the entire range, try
+ * to make an uncompressed inline extent.
+ */
+ ret = cow_file_range_inline(trans, root, inode,
+ start, end, 0, NULL);
+ } else {
+ /* try making a compressed inline extent */
+ ret = cow_file_range_inline(trans, root, inode,
+ start, end,
+ total_compressed, pages);
+ }
+ btrfs_end_transaction(trans, root);
+ if (ret == 0) {
+ /*
+ * inline extent creation worked, we don't need
+ * to create any more async work items. Unlock
+ * and free up our temp pages.
+ */
+ extent_clear_unlock_delalloc(inode,
+ &BTRFS_I(inode)->io_tree,
+ start, end, NULL, 1, 0,
+ 0, 1, 1, 1);
+ ret = 0;
+ goto free_pages_out;
+ }
+ }
+
+ if (will_compress) {
+ /*
+ * we aren't doing an inline extent round the compressed size
+ * up to a block size boundary so the allocator does sane
+ * things
+ */
+ total_compressed = (total_compressed + blocksize - 1) &
+ ~(blocksize - 1);
+
+ /*
+ * one last check to make sure the compression is really a
+ * win, compare the page count read with the blocks on disk
+ */
+ total_in = (total_in + PAGE_CACHE_SIZE - 1) &
+ ~(PAGE_CACHE_SIZE - 1);
+ if (total_compressed >= total_in) {
+ will_compress = 0;
+ } else {
+ disk_num_bytes = total_compressed;
+ num_bytes = total_in;
+ }
+ }
+ if (!will_compress && pages) {
+ /*
+ * the compression code ran but failed to make things smaller,
+ * free any pages it allocated and our page pointer array
+ */
+ for (i = 0; i < nr_pages_ret; i++) {
+ WARN_ON(pages[i]->mapping);
+ page_cache_release(pages[i]);
+ }
+ kfree(pages);
+ pages = NULL;
+ total_compressed = 0;
+ nr_pages_ret = 0;
+
+ /* flag the file so we don't compress in the future */
+ btrfs_set_flag(inode, NOCOMPRESS);
+ }
+ if (will_compress) {
+ *num_added += 1;
+
+ /* the async work queues will take care of doing actual
+ * allocation on disk for these compressed pages,
+ * and will submit them to the elevator.
+ */
+ add_async_extent(async_cow, start, num_bytes,
+ total_compressed, pages, nr_pages_ret);
+
+ if (start + num_bytes < end && start + num_bytes < actual_end) {
+ start += num_bytes;
+ pages = NULL;
+ cond_resched();
+ goto again;
+ }
+ } else {
+ /*
+ * No compression, but we still need to write the pages in
+ * the file we've been given so far. redirty the locked
+ * page if it corresponds to our extent and set things up
+ * for the async work queue to run cow_file_range to do
+ * the normal delalloc dance
+ */
+ if (page_offset(locked_page) >= start &&
+ page_offset(locked_page) <= end) {
+ __set_page_dirty_nobuffers(locked_page);
+ /* unlocked later on in the async handlers */
+ }
+ add_async_extent(async_cow, start, end - start + 1, 0, NULL, 0);
+ *num_added += 1;
+ }
+
+out:
+ return 0;
+
+free_pages_out:
+ for (i = 0; i < nr_pages_ret; i++) {
+ WARN_ON(pages[i]->mapping);
+ page_cache_release(pages[i]);
+ }
+ kfree(pages);
+
+ goto out;
+}
+
+/*
+ * phase two of compressed writeback. This is the ordered portion
+ * of the code, which only gets called in the order the work was
+ * queued. We walk all the async extents created by compress_file_range
+ * and send them down to the disk.
+ */
+static noinline int submit_compressed_extents(struct inode *inode,
+ struct async_cow *async_cow)
+{
+ struct async_extent *async_extent;
+ u64 alloc_hint = 0;
+ struct btrfs_trans_handle *trans;
+ struct btrfs_key ins;
+ struct extent_map *em;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct extent_map_tree *em_tree = &BTRFS_I(inode)->extent_tree;
+ struct extent_io_tree *io_tree;
+ int ret;
+
+ if (list_empty(&async_cow->extents))
+ return 0;
+
+ trans = btrfs_join_transaction(root, 1);
+
+ while (!list_empty(&async_cow->extents)) {
+ async_extent = list_entry(async_cow->extents.next,
+ struct async_extent, list);
+ list_del(&async_extent->list);
+
+ io_tree = &BTRFS_I(inode)->io_tree;
+
+ /* did the compression code fall back to uncompressed IO? */
+ if (!async_extent->pages) {
+ int page_started = 0;
+ unsigned long nr_written = 0;
+
+ lock_extent(io_tree, async_extent->start,
+ async_extent->start +
+ async_extent->ram_size - 1, GFP_NOFS);
+
+ /* allocate blocks */
+ cow_file_range(inode, async_cow->locked_page,
+ async_extent->start,
+ async_extent->start +
+ async_extent->ram_size - 1,
+ &page_started, &nr_written, 0);
+
+ /*
+ * if page_started, cow_file_range inserted an
+ * inline extent and took care of all the unlocking
+ * and IO for us. Otherwise, we need to submit
+ * all those pages down to the drive.
+ */
+ if (!page_started)
+ extent_write_locked_range(io_tree,
+ inode, async_extent->start,
+ async_extent->start +
+ async_extent->ram_size - 1,
+ btrfs_get_extent,
+ WB_SYNC_ALL);
+ kfree(async_extent);
+ cond_resched();
+ continue;
+ }
+
+ lock_extent(io_tree, async_extent->start,
+ async_extent->start + async_extent->ram_size - 1,
+ GFP_NOFS);
+ /*
+ * here we're doing allocation and writeback of the
+ * compressed pages
+ */
+ btrfs_drop_extent_cache(inode, async_extent->start,
+ async_extent->start +
+ async_extent->ram_size - 1, 0);
+
+ ret = btrfs_reserve_extent(trans, root,
+ async_extent->compressed_size,
+ async_extent->compressed_size,
+ 0, alloc_hint,
+ (u64)-1, &ins, 1);
+ BUG_ON(ret);
+ em = alloc_extent_map(GFP_NOFS);
+ em->start = async_extent->start;
+ em->len = async_extent->ram_size;
+ em->orig_start = em->start;
+
+ em->block_start = ins.objectid;
+ em->block_len = ins.offset;
+ em->bdev = root->fs_info->fs_devices->latest_bdev;
+ set_bit(EXTENT_FLAG_PINNED, &em->flags);
+ set_bit(EXTENT_FLAG_COMPRESSED, &em->flags);
+
+ while (1) {
+ spin_lock(&em_tree->lock);
+ ret = add_extent_mapping(em_tree, em);
+ spin_unlock(&em_tree->lock);
+ if (ret != -EEXIST) {
+ free_extent_map(em);
+ break;
+ }
+ btrfs_drop_extent_cache(inode, async_extent->start,
+ async_extent->start +
+ async_extent->ram_size - 1, 0);
+ }
+
+ ret = btrfs_add_ordered_extent(inode, async_extent->start,
+ ins.objectid,
+ async_extent->ram_size,
+ ins.offset,
+ BTRFS_ORDERED_COMPRESSED);
+ BUG_ON(ret);
+
+ btrfs_end_transaction(trans, root);
+
+ /*
+ * clear dirty, set writeback and unlock the pages.
+ */
+ extent_clear_unlock_delalloc(inode,
+ &BTRFS_I(inode)->io_tree,
+ async_extent->start,
+ async_extent->start +
+ async_extent->ram_size - 1,
+ NULL, 1, 1, 0, 1, 1, 0);
+
+ ret = btrfs_submit_compressed_write(inode,
+ async_extent->start,
+ async_extent->ram_size,
+ ins.objectid,
+ ins.offset, async_extent->pages,
+ async_extent->nr_pages);
+
+ BUG_ON(ret);
+ trans = btrfs_join_transaction(root, 1);
+ alloc_hint = ins.objectid + ins.offset;
+ kfree(async_extent);
+ cond_resched();
+ }
+
+ btrfs_end_transaction(trans, root);
+ return 0;
+}
+
+/*
+ * when extent_io.c finds a delayed allocation range in the file,
+ * the call backs end up in this code. The basic idea is to
+ * allocate extents on disk for the range, and create ordered data structs
+ * in ram to track those extents.
+ *
+ * locked_page is the page that writepage had locked already. We use
+ * it to make sure we don't do extra locks or unlocks.
+ *
+ * *page_started is set to one if we unlock locked_page and do everything
+ * required to start IO on it. It may be clean and already done with
+ * IO when we return.
+ */
+static noinline int cow_file_range(struct inode *inode,
+ struct page *locked_page,
+ u64 start, u64 end, int *page_started,
+ unsigned long *nr_written,
+ int unlock)
+{
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_trans_handle *trans;
+ u64 alloc_hint = 0;
+ u64 num_bytes;
+ unsigned long ram_size;
+ u64 disk_num_bytes;
+ u64 cur_alloc_size;
+ u64 blocksize = root->sectorsize;
+ u64 actual_end;
+ u64 isize = i_size_read(inode);
+ struct btrfs_key ins;
+ struct extent_map *em;
+ struct extent_map_tree *em_tree = &BTRFS_I(inode)->extent_tree;
+ int ret = 0;
+
+ trans = btrfs_join_transaction(root, 1);
+ BUG_ON(!trans);
+ btrfs_set_trans_block_group(trans, inode);
+
+ actual_end = min_t(u64, isize, end + 1);
+
+ num_bytes = (end - start + blocksize) & ~(blocksize - 1);
+ num_bytes = max(blocksize, num_bytes);
+ disk_num_bytes = num_bytes;
+ ret = 0;
+
+ if (start == 0) {
+ /* lets try to make an inline extent */
+ ret = cow_file_range_inline(trans, root, inode,
+ start, end, 0, NULL);
+ if (ret == 0) {
+ extent_clear_unlock_delalloc(inode,
+ &BTRFS_I(inode)->io_tree,
+ start, end, NULL, 1, 1,
+ 1, 1, 1, 1);
+ *nr_written = *nr_written +
+ (end - start + PAGE_CACHE_SIZE) / PAGE_CACHE_SIZE;
+ *page_started = 1;
+ ret = 0;
+ goto out;
+ }
+ }
+
+ BUG_ON(disk_num_bytes >
+ btrfs_super_total_bytes(&root->fs_info->super_copy));
+
+ btrfs_drop_extent_cache(inode, start, start + num_bytes - 1, 0);
+
+ while (disk_num_bytes > 0) {
+ cur_alloc_size = min(disk_num_bytes, root->fs_info->max_extent);
+ ret = btrfs_reserve_extent(trans, root, cur_alloc_size,
+ root->sectorsize, 0, alloc_hint,
+ (u64)-1, &ins, 1);
+ BUG_ON(ret);
+
+ em = alloc_extent_map(GFP_NOFS);
+ em->start = start;
+ em->orig_start = em->start;
+
+ ram_size = ins.offset;
+ em->len = ins.offset;
+
+ em->block_start = ins.objectid;
+ em->block_len = ins.offset;
+ em->bdev = root->fs_info->fs_devices->latest_bdev;
+ set_bit(EXTENT_FLAG_PINNED, &em->flags);
+
+ while (1) {
+ spin_lock(&em_tree->lock);
+ ret = add_extent_mapping(em_tree, em);
+ spin_unlock(&em_tree->lock);
+ if (ret != -EEXIST) {
+ free_extent_map(em);
+ break;
+ }
+ btrfs_drop_extent_cache(inode, start,
+ start + ram_size - 1, 0);
+ }
+
+ cur_alloc_size = ins.offset;
+ ret = btrfs_add_ordered_extent(inode, start, ins.objectid,
+ ram_size, cur_alloc_size, 0);
+ BUG_ON(ret);
+
+ if (root->root_key.objectid ==
+ BTRFS_DATA_RELOC_TREE_OBJECTID) {
+ ret = btrfs_reloc_clone_csums(inode, start,
+ cur_alloc_size);
+ BUG_ON(ret);
+ }
+
+ if (disk_num_bytes < cur_alloc_size)
+ break;
+
+ /* we're not doing compressed IO, don't unlock the first
+ * page (which the caller expects to stay locked), don't
+ * clear any dirty bits and don't set any writeback bits
+ */
+ extent_clear_unlock_delalloc(inode, &BTRFS_I(inode)->io_tree,
+ start, start + ram_size - 1,
+ locked_page, unlock, 1,
+ 1, 0, 0, 0);
+ disk_num_bytes -= cur_alloc_size;
+ num_bytes -= cur_alloc_size;
+ alloc_hint = ins.objectid + ins.offset;
+ start += cur_alloc_size;
+ }
+out:
+ ret = 0;
+ btrfs_end_transaction(trans, root);
+
+ return ret;
+}
+
+/*
+ * work queue call back to started compression on a file and pages
+ */
+static noinline void async_cow_start(struct btrfs_work *work)
+{
+ struct async_cow *async_cow;
+ int num_added = 0;
+ async_cow = container_of(work, struct async_cow, work);
+
+ compress_file_range(async_cow->inode, async_cow->locked_page,
+ async_cow->start, async_cow->end, async_cow,
+ &num_added);
+ if (num_added == 0)
+ async_cow->inode = NULL;
+}
+
+/*
+ * work queue call back to submit previously compressed pages
+ */
+static noinline void async_cow_submit(struct btrfs_work *work)
+{
+ struct async_cow *async_cow;
+ struct btrfs_root *root;
+ unsigned long nr_pages;
+
+ async_cow = container_of(work, struct async_cow, work);
+
+ root = async_cow->root;
+ nr_pages = (async_cow->end - async_cow->start + PAGE_CACHE_SIZE) >>
+ PAGE_CACHE_SHIFT;
+
+ atomic_sub(nr_pages, &root->fs_info->async_delalloc_pages);
+
+ if (atomic_read(&root->fs_info->async_delalloc_pages) <
+ 5 * 1042 * 1024 &&
+ waitqueue_active(&root->fs_info->async_submit_wait))
+ wake_up(&root->fs_info->async_submit_wait);
+
+ if (async_cow->inode)
+ submit_compressed_extents(async_cow->inode, async_cow);
+}
+
+static noinline void async_cow_free(struct btrfs_work *work)
+{
+ struct async_cow *async_cow;
+ async_cow = container_of(work, struct async_cow, work);
+ kfree(async_cow);
+}
+
+static int cow_file_range_async(struct inode *inode, struct page *locked_page,
+ u64 start, u64 end, int *page_started,
+ unsigned long *nr_written)
+{
+ struct async_cow *async_cow;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ unsigned long nr_pages;
+ u64 cur_end;
+ int limit = 10 * 1024 * 1042;
+
+ if (!btrfs_test_opt(root, COMPRESS)) {
+ return cow_file_range(inode, locked_page, start, end,
+ page_started, nr_written, 1);
+ }
+
+ clear_extent_bit(&BTRFS_I(inode)->io_tree, start, end, EXTENT_LOCKED |
+ EXTENT_DELALLOC, 1, 0, GFP_NOFS);
+ while (start < end) {
+ async_cow = kmalloc(sizeof(*async_cow), GFP_NOFS);
+ async_cow->inode = inode;
+ async_cow->root = root;
+ async_cow->locked_page = locked_page;
+ async_cow->start = start;
+
+ if (btrfs_test_flag(inode, NOCOMPRESS))
+ cur_end = end;
+ else
+ cur_end = min(end, start + 512 * 1024 - 1);
+
+ async_cow->end = cur_end;
+ INIT_LIST_HEAD(&async_cow->extents);
+
+ async_cow->work.func = async_cow_start;
+ async_cow->work.ordered_func = async_cow_submit;
+ async_cow->work.ordered_free = async_cow_free;
+ async_cow->work.flags = 0;
+
+ nr_pages = (cur_end - start + PAGE_CACHE_SIZE) >>
+ PAGE_CACHE_SHIFT;
+ atomic_add(nr_pages, &root->fs_info->async_delalloc_pages);
+
+ btrfs_queue_worker(&root->fs_info->delalloc_workers,
+ &async_cow->work);
+
+ if (atomic_read(&root->fs_info->async_delalloc_pages) > limit) {
+ wait_event(root->fs_info->async_submit_wait,
+ (atomic_read(&root->fs_info->async_delalloc_pages) <
+ limit));
+ }
+
+ while (atomic_read(&root->fs_info->async_submit_draining) &&
+ atomic_read(&root->fs_info->async_delalloc_pages)) {
+ wait_event(root->fs_info->async_submit_wait,
+ (atomic_read(&root->fs_info->async_delalloc_pages) ==
+ 0));
+ }
+
+ *nr_written += nr_pages;
+ start = cur_end + 1;
+ }
+ *page_started = 1;
+ return 0;
+}
+
+static noinline int csum_exist_in_range(struct btrfs_root *root,
+ u64 bytenr, u64 num_bytes)
+{
+ int ret;
+ struct btrfs_ordered_sum *sums;
+ LIST_HEAD(list);
+
+ ret = btrfs_lookup_csums_range(root->fs_info->csum_root, bytenr,
+ bytenr + num_bytes - 1, &list);
+ if (ret == 0 && list_empty(&list))
+ return 0;
+
+ while (!list_empty(&list)) {
+ sums = list_entry(list.next, struct btrfs_ordered_sum, list);
+ list_del(&sums->list);
+ kfree(sums);
+ }
+ return 1;
+}
+
+/*
+ * when nowcow writeback call back. This checks for snapshots or COW copies
+ * of the extents that exist in the file, and COWs the file as required.
+ *
+ * If no cow copies or snapshots exist, we write directly to the existing
+ * blocks on disk
+ */
+static int run_delalloc_nocow(struct inode *inode, struct page *locked_page,
+ u64 start, u64 end, int *page_started, int force,
+ unsigned long *nr_written)
+{
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_trans_handle *trans;
+ struct extent_buffer *leaf;
+ struct btrfs_path *path;
+ struct btrfs_file_extent_item *fi;
+ struct btrfs_key found_key;
+ u64 cow_start;
+ u64 cur_offset;
+ u64 extent_end;
+ u64 disk_bytenr;
+ u64 num_bytes;
+ int extent_type;
+ int ret;
+ int type;
+ int nocow;
+ int check_prev = 1;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+ trans = btrfs_join_transaction(root, 1);
+ BUG_ON(!trans);
+
+ cow_start = (u64)-1;
+ cur_offset = start;
+ while (1) {
+ ret = btrfs_lookup_file_extent(trans, root, path, inode->i_ino,
+ cur_offset, 0);
+ BUG_ON(ret < 0);
+ if (ret > 0 && path->slots[0] > 0 && check_prev) {
+ leaf = path->nodes[0];
+ btrfs_item_key_to_cpu(leaf, &found_key,
+ path->slots[0] - 1);
+ if (found_key.objectid == inode->i_ino &&
+ found_key.type == BTRFS_EXTENT_DATA_KEY)
+ path->slots[0]--;
+ }
+ check_prev = 0;
+next_slot:
+ leaf = path->nodes[0];
+ if (path->slots[0] >= btrfs_header_nritems(leaf)) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret < 0)
+ BUG_ON(1);
+ if (ret > 0)
+ break;
+ leaf = path->nodes[0];
+ }
+
+ nocow = 0;
+ disk_bytenr = 0;
+ num_bytes = 0;
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+
+ if (found_key.objectid > inode->i_ino ||
+ found_key.type > BTRFS_EXTENT_DATA_KEY ||
+ found_key.offset > end)
+ break;
+
+ if (found_key.offset > cur_offset) {
+ extent_end = found_key.offset;
+ goto out_check;
+ }
+
+ fi = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_file_extent_item);
+ extent_type = btrfs_file_extent_type(leaf, fi);
+
+ if (extent_type == BTRFS_FILE_EXTENT_REG ||
+ extent_type == BTRFS_FILE_EXTENT_PREALLOC) {
+ disk_bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
+ extent_end = found_key.offset +
+ btrfs_file_extent_num_bytes(leaf, fi);
+ if (extent_end <= start) {
+ path->slots[0]++;
+ goto next_slot;
+ }
+ if (disk_bytenr == 0)
+ goto out_check;
+ if (btrfs_file_extent_compression(leaf, fi) ||
+ btrfs_file_extent_encryption(leaf, fi) ||
+ btrfs_file_extent_other_encoding(leaf, fi))
+ goto out_check;
+ if (extent_type == BTRFS_FILE_EXTENT_REG && !force)
+ goto out_check;
+ if (btrfs_extent_readonly(root, disk_bytenr))
+ goto out_check;
+ if (btrfs_cross_ref_exist(trans, root, inode->i_ino,
+ disk_bytenr))
+ goto out_check;
+ disk_bytenr += btrfs_file_extent_offset(leaf, fi);
+ disk_bytenr += cur_offset - found_key.offset;
+ num_bytes = min(end + 1, extent_end) - cur_offset;
+ /*
+ * force cow if csum exists in the range.
+ * this ensure that csum for a given extent are
+ * either valid or do not exist.
+ */
+ if (csum_exist_in_range(root, disk_bytenr, num_bytes))
+ goto out_check;
+ nocow = 1;
+ } else if (extent_type == BTRFS_FILE_EXTENT_INLINE) {
+ extent_end = found_key.offset +
+ btrfs_file_extent_inline_len(leaf, fi);
+ extent_end = ALIGN(extent_end, root->sectorsize);
+ } else {
+ BUG_ON(1);
+ }
+out_check:
+ if (extent_end <= start) {
+ path->slots[0]++;
+ goto next_slot;
+ }
+ if (!nocow) {
+ if (cow_start == (u64)-1)
+ cow_start = cur_offset;
+ cur_offset = extent_end;
+ if (cur_offset > end)
+ break;
+ path->slots[0]++;
+ goto next_slot;
+ }
+
+ btrfs_release_path(root, path);
+ if (cow_start != (u64)-1) {
+ ret = cow_file_range(inode, locked_page, cow_start,
+ found_key.offset - 1, page_started,
+ nr_written, 1);
+ BUG_ON(ret);
+ cow_start = (u64)-1;
+ }
+
+ if (extent_type == BTRFS_FILE_EXTENT_PREALLOC) {
+ struct extent_map *em;
+ struct extent_map_tree *em_tree;
+ em_tree = &BTRFS_I(inode)->extent_tree;
+ em = alloc_extent_map(GFP_NOFS);
+ em->start = cur_offset;
+ em->orig_start = em->start;
+ em->len = num_bytes;
+ em->block_len = num_bytes;
+ em->block_start = disk_bytenr;
+ em->bdev = root->fs_info->fs_devices->latest_bdev;
+ set_bit(EXTENT_FLAG_PINNED, &em->flags);
+ while (1) {
+ spin_lock(&em_tree->lock);
+ ret = add_extent_mapping(em_tree, em);
+ spin_unlock(&em_tree->lock);
+ if (ret != -EEXIST) {
+ free_extent_map(em);
+ break;
+ }
+ btrfs_drop_extent_cache(inode, em->start,
+ em->start + em->len - 1, 0);
+ }
+ type = BTRFS_ORDERED_PREALLOC;
+ } else {
+ type = BTRFS_ORDERED_NOCOW;
+ }
+
+ ret = btrfs_add_ordered_extent(inode, cur_offset, disk_bytenr,
+ num_bytes, num_bytes, type);
+ BUG_ON(ret);
+
+ extent_clear_unlock_delalloc(inode, &BTRFS_I(inode)->io_tree,
+ cur_offset, cur_offset + num_bytes - 1,
+ locked_page, 1, 1, 1, 0, 0, 0);
+ cur_offset = extent_end;
+ if (cur_offset > end)
+ break;
+ }
+ btrfs_release_path(root, path);
+
+ if (cur_offset <= end && cow_start == (u64)-1)
+ cow_start = cur_offset;
+ if (cow_start != (u64)-1) {
+ ret = cow_file_range(inode, locked_page, cow_start, end,
+ page_started, nr_written, 1);
+ BUG_ON(ret);
+ }
+
+ ret = btrfs_end_transaction(trans, root);
+ BUG_ON(ret);
+ btrfs_free_path(path);
+ return 0;
+}
+
+/*
+ * extent_io.c call back to do delayed allocation processing
+ */
+static int run_delalloc_range(struct inode *inode, struct page *locked_page,
+ u64 start, u64 end, int *page_started,
+ unsigned long *nr_written)
+{
+ int ret;
+
+ if (btrfs_test_flag(inode, NODATACOW))
+ ret = run_delalloc_nocow(inode, locked_page, start, end,
+ page_started, 1, nr_written);
+ else if (btrfs_test_flag(inode, PREALLOC))
+ ret = run_delalloc_nocow(inode, locked_page, start, end,
+ page_started, 0, nr_written);
+ else
+ ret = cow_file_range_async(inode, locked_page, start, end,
+ page_started, nr_written);
+
+ return ret;
+}
+
+/*
+ * extent_io.c set_bit_hook, used to track delayed allocation
+ * bytes in this file, and to maintain the list of inodes that
+ * have pending delalloc work to be done.
+ */
+static int btrfs_set_bit_hook(struct inode *inode, u64 start, u64 end,
+ unsigned long old, unsigned long bits)
+{
+ /*
+ * set_bit and clear bit hooks normally require _irqsave/restore
+ * but in this case, we are only testeing for the DELALLOC
+ * bit, which is only set or cleared with irqs on
+ */
+ if (!(old & EXTENT_DELALLOC) && (bits & EXTENT_DELALLOC)) {
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ spin_lock(&root->fs_info->delalloc_lock);
+ BTRFS_I(inode)->delalloc_bytes += end - start + 1;
+ root->fs_info->delalloc_bytes += end - start + 1;
+ if (list_empty(&BTRFS_I(inode)->delalloc_inodes)) {
+ list_add_tail(&BTRFS_I(inode)->delalloc_inodes,
+ &root->fs_info->delalloc_inodes);
+ }
+ spin_unlock(&root->fs_info->delalloc_lock);
+ }
+ return 0;
+}
+
+/*
+ * extent_io.c clear_bit_hook, see set_bit_hook for why
+ */
+static int btrfs_clear_bit_hook(struct inode *inode, u64 start, u64 end,
+ unsigned long old, unsigned long bits)
+{
+ /*
+ * set_bit and clear bit hooks normally require _irqsave/restore
+ * but in this case, we are only testeing for the DELALLOC
+ * bit, which is only set or cleared with irqs on
+ */
+ if ((old & EXTENT_DELALLOC) && (bits & EXTENT_DELALLOC)) {
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+
+ spin_lock(&root->fs_info->delalloc_lock);
+ if (end - start + 1 > root->fs_info->delalloc_bytes) {
+ printk(KERN_INFO "btrfs warning: delalloc account "
+ "%llu %llu\n",
+ (unsigned long long)end - start + 1,
+ (unsigned long long)
+ root->fs_info->delalloc_bytes);
+ root->fs_info->delalloc_bytes = 0;
+ BTRFS_I(inode)->delalloc_bytes = 0;
+ } else {
+ root->fs_info->delalloc_bytes -= end - start + 1;
+ BTRFS_I(inode)->delalloc_bytes -= end - start + 1;
+ }
+ if (BTRFS_I(inode)->delalloc_bytes == 0 &&
+ !list_empty(&BTRFS_I(inode)->delalloc_inodes)) {
+ list_del_init(&BTRFS_I(inode)->delalloc_inodes);
+ }
+ spin_unlock(&root->fs_info->delalloc_lock);
+ }
+ return 0;
+}
+
+/*
+ * extent_io.c merge_bio_hook, this must check the chunk tree to make sure
+ * we don't create bios that span stripes or chunks
+ */
+int btrfs_merge_bio_hook(struct page *page, unsigned long offset,
+ size_t size, struct bio *bio,
+ unsigned long bio_flags)
+{
+ struct btrfs_root *root = BTRFS_I(page->mapping->host)->root;
+ struct btrfs_mapping_tree *map_tree;
+ u64 logical = (u64)bio->bi_sector << 9;
+ u64 length = 0;
+ u64 map_length;
+ int ret;
+
+ if (bio_flags & EXTENT_BIO_COMPRESSED)
+ return 0;
+
+ length = bio->bi_size;
+ map_tree = &root->fs_info->mapping_tree;
+ map_length = length;
+ ret = btrfs_map_block(map_tree, READ, logical,
+ &map_length, NULL, 0);
+
+ if (map_length < length + size)
+ return 1;
+ return 0;
+}
+
+/*
+ * in order to insert checksums into the metadata in large chunks,
+ * we wait until bio submission time. All the pages in the bio are
+ * checksummed and sums are attached onto the ordered extent record.
+ *
+ * At IO completion time the cums attached on the ordered extent record
+ * are inserted into the btree
+ */
+static int __btrfs_submit_bio_start(struct inode *inode, int rw,
+ struct bio *bio, int mirror_num,
+ unsigned long bio_flags)
+{
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ int ret = 0;
+
+ ret = btrfs_csum_one_bio(root, inode, bio, 0, 0);
+ BUG_ON(ret);
+ return 0;
+}
+
+/*
+ * in order to insert checksums into the metadata in large chunks,
+ * we wait until bio submission time. All the pages in the bio are
+ * checksummed and sums are attached onto the ordered extent record.
+ *
+ * At IO completion time the cums attached on the ordered extent record
+ * are inserted into the btree
+ */
+static int __btrfs_submit_bio_done(struct inode *inode, int rw, struct bio *bio,
+ int mirror_num, unsigned long bio_flags)
+{
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ return btrfs_map_bio(root, rw, bio, mirror_num, 1);
+}
+
+/*
+ * extent_io.c submission hook. This does the right thing for csum calculation
+ * on write, or reading the csums from the tree before a read
+ */
+static int btrfs_submit_bio_hook(struct inode *inode, int rw, struct bio *bio,
+ int mirror_num, unsigned long bio_flags)
+{
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ int ret = 0;
+ int skip_sum;
+
+ skip_sum = btrfs_test_flag(inode, NODATASUM);
+
+ ret = btrfs_bio_wq_end_io(root->fs_info, bio, 0);
+ BUG_ON(ret);
+
+ if (!(rw & (1 << BIO_RW))) {
+ if (bio_flags & EXTENT_BIO_COMPRESSED) {
+ return btrfs_submit_compressed_read(inode, bio,
+ mirror_num, bio_flags);
+ } else if (!skip_sum)
+ btrfs_lookup_bio_sums(root, inode, bio, NULL);
+ goto mapit;
+ } else if (!skip_sum) {
+ /* csum items have already been cloned */
+ if (root->root_key.objectid == BTRFS_DATA_RELOC_TREE_OBJECTID)
+ goto mapit;
+ /* we're doing a write, do the async checksumming */
+ return btrfs_wq_submit_bio(BTRFS_I(inode)->root->fs_info,
+ inode, rw, bio, mirror_num,
+ bio_flags, __btrfs_submit_bio_start,
+ __btrfs_submit_bio_done);
+ }
+
+mapit:
+ return btrfs_map_bio(root, rw, bio, mirror_num, 0);
+}
+
+/*
+ * given a list of ordered sums record them in the inode. This happens
+ * at IO completion time based on sums calculated at bio submission time.
+ */
+static noinline int add_pending_csums(struct btrfs_trans_handle *trans,
+ struct inode *inode, u64 file_offset,
+ struct list_head *list)
+{
+ struct list_head *cur;
+ struct btrfs_ordered_sum *sum;
+
+ btrfs_set_trans_block_group(trans, inode);
+ list_for_each(cur, list) {
+ sum = list_entry(cur, struct btrfs_ordered_sum, list);
+ btrfs_csum_file_blocks(trans,
+ BTRFS_I(inode)->root->fs_info->csum_root, sum);
+ }
+ return 0;
+}
+
+int btrfs_set_extent_delalloc(struct inode *inode, u64 start, u64 end)
+{
+ if ((end & (PAGE_CACHE_SIZE - 1)) == 0)
+ WARN_ON(1);
+ return set_extent_delalloc(&BTRFS_I(inode)->io_tree, start, end,
+ GFP_NOFS);
+}
+
+/* see btrfs_writepage_start_hook for details on why this is required */
+struct btrfs_writepage_fixup {
+ struct page *page;
+ struct btrfs_work work;
+};
+
+static void btrfs_writepage_fixup_worker(struct btrfs_work *work)
+{
+ struct btrfs_writepage_fixup *fixup;
+ struct btrfs_ordered_extent *ordered;
+ struct page *page;
+ struct inode *inode;
+ u64 page_start;
+ u64 page_end;
+
+ fixup = container_of(work, struct btrfs_writepage_fixup, work);
+ page = fixup->page;
+again:
+ lock_page(page);
+ if (!page->mapping || !PageDirty(page) || !PageChecked(page)) {
+ ClearPageChecked(page);
+ goto out_page;
+ }
+
+ inode = page->mapping->host;
+ page_start = page_offset(page);
+ page_end = page_offset(page) + PAGE_CACHE_SIZE - 1;
+
+ lock_extent(&BTRFS_I(inode)->io_tree, page_start, page_end, GFP_NOFS);
+
+ /* already ordered? We're done */
+ if (test_range_bit(&BTRFS_I(inode)->io_tree, page_start, page_end,
+ EXTENT_ORDERED, 0)) {
+ goto out;
+ }
+
+ ordered = btrfs_lookup_ordered_extent(inode, page_start);
+ if (ordered) {
+ unlock_extent(&BTRFS_I(inode)->io_tree, page_start,
+ page_end, GFP_NOFS);
+ unlock_page(page);
+ btrfs_start_ordered_extent(inode, ordered, 1);
+ goto again;
+ }
+
+ btrfs_set_extent_delalloc(inode, page_start, page_end);
+ ClearPageChecked(page);
+out:
+ unlock_extent(&BTRFS_I(inode)->io_tree, page_start, page_end, GFP_NOFS);
+out_page:
+ unlock_page(page);
+ page_cache_release(page);
+}
+
+/*
+ * There are a few paths in the higher layers of the kernel that directly
+ * set the page dirty bit without asking the filesystem if it is a
+ * good idea. This causes problems because we want to make sure COW
+ * properly happens and the data=ordered rules are followed.
+ *
+ * In our case any range that doesn't have the ORDERED bit set
+ * hasn't been properly setup for IO. We kick off an async process
+ * to fix it up. The async helper will wait for ordered extents, set
+ * the delalloc bit and make it safe to write the page.
+ */
+static int btrfs_writepage_start_hook(struct page *page, u64 start, u64 end)
+{
+ struct inode *inode = page->mapping->host;
+ struct btrfs_writepage_fixup *fixup;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ int ret;
+
+ ret = test_range_bit(&BTRFS_I(inode)->io_tree, start, end,
+ EXTENT_ORDERED, 0);
+ if (ret)
+ return 0;
+
+ if (PageChecked(page))
+ return -EAGAIN;
+
+ fixup = kzalloc(sizeof(*fixup), GFP_NOFS);
+ if (!fixup)
+ return -EAGAIN;
+
+ SetPageChecked(page);
+ page_cache_get(page);
+ fixup->work.func = btrfs_writepage_fixup_worker;
+ fixup->page = page;
+ btrfs_queue_worker(&root->fs_info->fixup_workers, &fixup->work);
+ return -EAGAIN;
+}
+
+static int insert_reserved_file_extent(struct btrfs_trans_handle *trans,
+ struct inode *inode, u64 file_pos,
+ u64 disk_bytenr, u64 disk_num_bytes,
+ u64 num_bytes, u64 ram_bytes,
+ u8 compression, u8 encryption,
+ u16 other_encoding, int extent_type)
+{
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_file_extent_item *fi;
+ struct btrfs_path *path;
+ struct extent_buffer *leaf;
+ struct btrfs_key ins;
+ u64 hint;
+ int ret;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+
+ ret = btrfs_drop_extents(trans, root, inode, file_pos,
+ file_pos + num_bytes, file_pos, &hint);
+ BUG_ON(ret);
+
+ ins.objectid = inode->i_ino;
+ ins.offset = file_pos;
+ ins.type = BTRFS_EXTENT_DATA_KEY;
+ ret = btrfs_insert_empty_item(trans, root, path, &ins, sizeof(*fi));
+ BUG_ON(ret);
+ leaf = path->nodes[0];
+ fi = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_file_extent_item);
+ btrfs_set_file_extent_generation(leaf, fi, trans->transid);
+ btrfs_set_file_extent_type(leaf, fi, extent_type);
+ btrfs_set_file_extent_disk_bytenr(leaf, fi, disk_bytenr);
+ btrfs_set_file_extent_disk_num_bytes(leaf, fi, disk_num_bytes);
+ btrfs_set_file_extent_offset(leaf, fi, 0);
+ btrfs_set_file_extent_num_bytes(leaf, fi, num_bytes);
+ btrfs_set_file_extent_ram_bytes(leaf, fi, ram_bytes);
+ btrfs_set_file_extent_compression(leaf, fi, compression);
+ btrfs_set_file_extent_encryption(leaf, fi, encryption);
+ btrfs_set_file_extent_other_encoding(leaf, fi, other_encoding);
+ btrfs_mark_buffer_dirty(leaf);
+
+ inode_add_bytes(inode, num_bytes);
+ btrfs_drop_extent_cache(inode, file_pos, file_pos + num_bytes - 1, 0);
+
+ ins.objectid = disk_bytenr;
+ ins.offset = disk_num_bytes;
+ ins.type = BTRFS_EXTENT_ITEM_KEY;
+ ret = btrfs_alloc_reserved_extent(trans, root, leaf->start,
+ root->root_key.objectid,
+ trans->transid, inode->i_ino, &ins);
+ BUG_ON(ret);
+
+ btrfs_free_path(path);
+ return 0;
+}
+
+/* as ordered data IO finishes, this gets called so we can finish
+ * an ordered extent if the range of bytes in the file it covers are
+ * fully written.
+ */
+static int btrfs_finish_ordered_io(struct inode *inode, u64 start, u64 end)
+{
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_trans_handle *trans;
+ struct btrfs_ordered_extent *ordered_extent;
+ struct extent_io_tree *io_tree = &BTRFS_I(inode)->io_tree;
+ int compressed = 0;
+ int ret;
+
+ ret = btrfs_dec_test_ordered_pending(inode, start, end - start + 1);
+ if (!ret)
+ return 0;
+
+ trans = btrfs_join_transaction(root, 1);
+
+ ordered_extent = btrfs_lookup_ordered_extent(inode, start);
+ BUG_ON(!ordered_extent);
+ if (test_bit(BTRFS_ORDERED_NOCOW, &ordered_extent->flags))
+ goto nocow;
+
+ lock_extent(io_tree, ordered_extent->file_offset,
+ ordered_extent->file_offset + ordered_extent->len - 1,
+ GFP_NOFS);
+
+ if (test_bit(BTRFS_ORDERED_COMPRESSED, &ordered_extent->flags))
+ compressed = 1;
+ if (test_bit(BTRFS_ORDERED_PREALLOC, &ordered_extent->flags)) {
+ BUG_ON(compressed);
+ ret = btrfs_mark_extent_written(trans, root, inode,
+ ordered_extent->file_offset,
+ ordered_extent->file_offset +
+ ordered_extent->len);
+ BUG_ON(ret);
+ } else {
+ ret = insert_reserved_file_extent(trans, inode,
+ ordered_extent->file_offset,
+ ordered_extent->start,
+ ordered_extent->disk_len,
+ ordered_extent->len,
+ ordered_extent->len,
+ compressed, 0, 0,
+ BTRFS_FILE_EXTENT_REG);
+ BUG_ON(ret);
+ }
+ unlock_extent(io_tree, ordered_extent->file_offset,
+ ordered_extent->file_offset + ordered_extent->len - 1,
+ GFP_NOFS);
+nocow:
+ add_pending_csums(trans, inode, ordered_extent->file_offset,
+ &ordered_extent->list);
+
+ mutex_lock(&BTRFS_I(inode)->extent_mutex);
+ btrfs_ordered_update_i_size(inode, ordered_extent);
+ btrfs_update_inode(trans, root, inode);
+ btrfs_remove_ordered_extent(inode, ordered_extent);
+ mutex_unlock(&BTRFS_I(inode)->extent_mutex);
+
+ /* once for us */
+ btrfs_put_ordered_extent(ordered_extent);
+ /* once for the tree */
+ btrfs_put_ordered_extent(ordered_extent);
+
+ btrfs_end_transaction(trans, root);
+ return 0;
+}
+
+static int btrfs_writepage_end_io_hook(struct page *page, u64 start, u64 end,
+ struct extent_state *state, int uptodate)
+{
+ return btrfs_finish_ordered_io(page->mapping->host, start, end);
+}
+
+/*
+ * When IO fails, either with EIO or csum verification fails, we
+ * try other mirrors that might have a good copy of the data. This
+ * io_failure_record is used to record state as we go through all the
+ * mirrors. If another mirror has good data, the page is set up to date
+ * and things continue. If a good mirror can't be found, the original
+ * bio end_io callback is called to indicate things have failed.
+ */
+struct io_failure_record {
+ struct page *page;
+ u64 start;
+ u64 len;
+ u64 logical;
+ unsigned long bio_flags;
+ int last_mirror;
+};
+
+static int btrfs_io_failed_hook(struct bio *failed_bio,
+ struct page *page, u64 start, u64 end,
+ struct extent_state *state)
+{
+ struct io_failure_record *failrec = NULL;
+ u64 private;
+ struct extent_map *em;
+ struct inode *inode = page->mapping->host;
+ struct extent_io_tree *failure_tree = &BTRFS_I(inode)->io_failure_tree;
+ struct extent_map_tree *em_tree = &BTRFS_I(inode)->extent_tree;
+ struct bio *bio;
+ int num_copies;
+ int ret;
+ int rw;
+ u64 logical;
+
+ ret = get_state_private(failure_tree, start, &private);
+ if (ret) {
+ failrec = kmalloc(sizeof(*failrec), GFP_NOFS);
+ if (!failrec)
+ return -ENOMEM;
+ failrec->start = start;
+ failrec->len = end - start + 1;
+ failrec->last_mirror = 0;
+ failrec->bio_flags = 0;
+
+ spin_lock(&em_tree->lock);
+ em = lookup_extent_mapping(em_tree, start, failrec->len);
+ if (em->start > start || em->start + em->len < start) {
+ free_extent_map(em);
+ em = NULL;
+ }
+ spin_unlock(&em_tree->lock);
+
+ if (!em || IS_ERR(em)) {
+ kfree(failrec);
+ return -EIO;
+ }
+ logical = start - em->start;
+ logical = em->block_start + logical;
+ if (test_bit(EXTENT_FLAG_COMPRESSED, &em->flags)) {
+ logical = em->block_start;
+ failrec->bio_flags = EXTENT_BIO_COMPRESSED;
+ }
+ failrec->logical = logical;
+ free_extent_map(em);
+ set_extent_bits(failure_tree, start, end, EXTENT_LOCKED |
+ EXTENT_DIRTY, GFP_NOFS);
+ set_state_private(failure_tree, start,
+ (u64)(unsigned long)failrec);
+ } else {
+ failrec = (struct io_failure_record *)(unsigned long)private;
+ }
+ num_copies = btrfs_num_copies(
+ &BTRFS_I(inode)->root->fs_info->mapping_tree,
+ failrec->logical, failrec->len);
+ failrec->last_mirror++;
+ if (!state) {
+ spin_lock(&BTRFS_I(inode)->io_tree.lock);
+ state = find_first_extent_bit_state(&BTRFS_I(inode)->io_tree,
+ failrec->start,
+ EXTENT_LOCKED);
+ if (state && state->start != failrec->start)
+ state = NULL;
+ spin_unlock(&BTRFS_I(inode)->io_tree.lock);
+ }
+ if (!state || failrec->last_mirror > num_copies) {
+ set_state_private(failure_tree, failrec->start, 0);
+ clear_extent_bits(failure_tree, failrec->start,
+ failrec->start + failrec->len - 1,
+ EXTENT_LOCKED | EXTENT_DIRTY, GFP_NOFS);
+ kfree(failrec);
+ return -EIO;
+ }
+ bio = bio_alloc(GFP_NOFS, 1);
+ bio->bi_private = state;
+ bio->bi_end_io = failed_bio->bi_end_io;
+ bio->bi_sector = failrec->logical >> 9;
+ bio->bi_bdev = failed_bio->bi_bdev;
+ bio->bi_size = 0;
+
+ bio_add_page(bio, page, failrec->len, start - page_offset(page));
+ if (failed_bio->bi_rw & (1 << BIO_RW))
+ rw = WRITE;
+ else
+ rw = READ;
+
+ BTRFS_I(inode)->io_tree.ops->submit_bio_hook(inode, rw, bio,
+ failrec->last_mirror,
+ failrec->bio_flags);
+ return 0;
+}
+
+/*
+ * each time an IO finishes, we do a fast check in the IO failure tree
+ * to see if we need to process or clean up an io_failure_record
+ */
+static int btrfs_clean_io_failures(struct inode *inode, u64 start)
+{
+ u64 private;
+ u64 private_failure;
+ struct io_failure_record *failure;
+ int ret;
+
+ private = 0;
+ if (count_range_bits(&BTRFS_I(inode)->io_failure_tree, &private,
+ (u64)-1, 1, EXTENT_DIRTY)) {
+ ret = get_state_private(&BTRFS_I(inode)->io_failure_tree,
+ start, &private_failure);
+ if (ret == 0) {
+ failure = (struct io_failure_record *)(unsigned long)
+ private_failure;
+ set_state_private(&BTRFS_I(inode)->io_failure_tree,
+ failure->start, 0);
+ clear_extent_bits(&BTRFS_I(inode)->io_failure_tree,
+ failure->start,
+ failure->start + failure->len - 1,
+ EXTENT_DIRTY | EXTENT_LOCKED,
+ GFP_NOFS);
+ kfree(failure);
+ }
+ }
+ return 0;
+}
+
+/*
+ * when reads are done, we need to check csums to verify the data is correct
+ * if there's a match, we allow the bio to finish. If not, we go through
+ * the io_failure_record routines to find good copies
+ */
+static int btrfs_readpage_end_io_hook(struct page *page, u64 start, u64 end,
+ struct extent_state *state)
+{
+ size_t offset = start - ((u64)page->index << PAGE_CACHE_SHIFT);
+ struct inode *inode = page->mapping->host;
+ struct extent_io_tree *io_tree = &BTRFS_I(inode)->io_tree;
+ char *kaddr;
+ u64 private = ~(u32)0;
+ int ret;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ u32 csum = ~(u32)0;
+
+ if (PageChecked(page)) {
+ ClearPageChecked(page);
+ goto good;
+ }
+ if (btrfs_test_flag(inode, NODATASUM))
+ return 0;
+
+ if (root->root_key.objectid == BTRFS_DATA_RELOC_TREE_OBJECTID &&
+ test_range_bit(io_tree, start, end, EXTENT_NODATASUM, 1)) {
+ clear_extent_bits(io_tree, start, end, EXTENT_NODATASUM,
+ GFP_NOFS);
+ return 0;
+ }
+
+ if (state && state->start == start) {
+ private = state->private;
+ ret = 0;
+ } else {
+ ret = get_state_private(io_tree, start, &private);
+ }
+ kaddr = kmap_atomic(page, KM_USER0);
+ if (ret)
+ goto zeroit;
+
+ csum = btrfs_csum_data(root, kaddr + offset, csum, end - start + 1);
+ btrfs_csum_final(csum, (char *)&csum);
+ if (csum != private)
+ goto zeroit;
+
+ kunmap_atomic(kaddr, KM_USER0);
+good:
+ /* if the io failure tree for this inode is non-empty,
+ * check to see if we've recovered from a failed IO
+ */
+ btrfs_clean_io_failures(inode, start);
+ return 0;
+
+zeroit:
+ printk(KERN_INFO "btrfs csum failed ino %lu off %llu csum %u "
+ "private %llu\n", page->mapping->host->i_ino,
+ (unsigned long long)start, csum,
+ (unsigned long long)private);
+ memset(kaddr + offset, 1, end - start + 1);
+ flush_dcache_page(page);
+ kunmap_atomic(kaddr, KM_USER0);
+ if (private == 0)
+ return 0;
+ return -EIO;
+}
+
+/*
+ * This creates an orphan entry for the given inode in case something goes
+ * wrong in the middle of an unlink/truncate.
+ */
+int btrfs_orphan_add(struct btrfs_trans_handle *trans, struct inode *inode)
+{
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ int ret = 0;
+
+ spin_lock(&root->list_lock);
+
+ /* already on the orphan list, we're good */
+ if (!list_empty(&BTRFS_I(inode)->i_orphan)) {
+ spin_unlock(&root->list_lock);
+ return 0;
+ }
+
+ list_add(&BTRFS_I(inode)->i_orphan, &root->orphan_list);
+
+ spin_unlock(&root->list_lock);
+
+ /*
+ * insert an orphan item to track this unlinked/truncated file
+ */
+ ret = btrfs_insert_orphan_item(trans, root, inode->i_ino);
+
+ return ret;
+}
+
+/*
+ * We have done the truncate/delete so we can go ahead and remove the orphan
+ * item for this particular inode.
+ */
+int btrfs_orphan_del(struct btrfs_trans_handle *trans, struct inode *inode)
+{
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ int ret = 0;
+
+ spin_lock(&root->list_lock);
+
+ if (list_empty(&BTRFS_I(inode)->i_orphan)) {
+ spin_unlock(&root->list_lock);
+ return 0;
+ }
+
+ list_del_init(&BTRFS_I(inode)->i_orphan);
+ if (!trans) {
+ spin_unlock(&root->list_lock);
+ return 0;
+ }
+
+ spin_unlock(&root->list_lock);
+
+ ret = btrfs_del_orphan_item(trans, root, inode->i_ino);
+
+ return ret;
+}
+
+/*
+ * this cleans up any orphans that may be left on the list from the last use
+ * of this root.
+ */
+void btrfs_orphan_cleanup(struct btrfs_root *root)
+{
+ struct btrfs_path *path;
+ struct extent_buffer *leaf;
+ struct btrfs_item *item;
+ struct btrfs_key key, found_key;
+ struct btrfs_trans_handle *trans;
+ struct inode *inode;
+ int ret = 0, nr_unlink = 0, nr_truncate = 0;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return;
+ path->reada = -1;
+
+ key.objectid = BTRFS_ORPHAN_OBJECTID;
+ btrfs_set_key_type(&key, BTRFS_ORPHAN_ITEM_KEY);
+ key.offset = (u64)-1;
+
+
+ while (1) {
+ ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ if (ret < 0) {
+ printk(KERN_ERR "Error searching slot for orphan: %d"
+ "\n", ret);
+ break;
+ }
+
+ /*
+ * if ret == 0 means we found what we were searching for, which
+ * is weird, but possible, so only screw with path if we didnt
+ * find the key and see if we have stuff that matches
+ */
+ if (ret > 0) {
+ if (path->slots[0] == 0)
+ break;
+ path->slots[0]--;
+ }
+
+ /* pull out the item */
+ leaf = path->nodes[0];
+ item = btrfs_item_nr(leaf, path->slots[0]);
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+
+ /* make sure the item matches what we want */
+ if (found_key.objectid != BTRFS_ORPHAN_OBJECTID)
+ break;
+ if (btrfs_key_type(&found_key) != BTRFS_ORPHAN_ITEM_KEY)
+ break;
+
+ /* release the path since we're done with it */
+ btrfs_release_path(root, path);
+
+ /*
+ * this is where we are basically btrfs_lookup, without the
+ * crossing root thing. we store the inode number in the
+ * offset of the orphan item.
+ */
+ inode = btrfs_iget_locked(root->fs_info->sb,
+ found_key.offset, root);
+ if (!inode)
+ break;
+
+ if (inode->i_state & I_NEW) {
+ BTRFS_I(inode)->root = root;
+
+ /* have to set the location manually */
+ BTRFS_I(inode)->location.objectid = inode->i_ino;
+ BTRFS_I(inode)->location.type = BTRFS_INODE_ITEM_KEY;
+ BTRFS_I(inode)->location.offset = 0;
+
+ btrfs_read_locked_inode(inode);
+ unlock_new_inode(inode);
+ }
+
+ /*
+ * add this inode to the orphan list so btrfs_orphan_del does
+ * the proper thing when we hit it
+ */
+ spin_lock(&root->list_lock);
+ list_add(&BTRFS_I(inode)->i_orphan, &root->orphan_list);
+ spin_unlock(&root->list_lock);
+
+ /*
+ * if this is a bad inode, means we actually succeeded in
+ * removing the inode, but not the orphan record, which means
+ * we need to manually delete the orphan since iput will just
+ * do a destroy_inode
+ */
+ if (is_bad_inode(inode)) {
+ trans = btrfs_start_transaction(root, 1);
+ btrfs_orphan_del(trans, inode);
+ btrfs_end_transaction(trans, root);
+ iput(inode);
+ continue;
+ }
+
+ /* if we have links, this was a truncate, lets do that */
+ if (inode->i_nlink) {
+ nr_truncate++;
+ btrfs_truncate(inode);
+ } else {
+ nr_unlink++;
+ }
+
+ /* this will do delete_inode and everything for us */
+ iput(inode);
+ }
+
+ if (nr_unlink)
+ printk(KERN_INFO "btrfs: unlinked %d orphans\n", nr_unlink);
+ if (nr_truncate)
+ printk(KERN_INFO "btrfs: truncated %d orphans\n", nr_truncate);
+
+ btrfs_free_path(path);
+}
+
+/*
+ * read an inode from the btree into the in-memory inode
+ */
+void btrfs_read_locked_inode(struct inode *inode)
+{
+ struct btrfs_path *path;
+ struct extent_buffer *leaf;
+ struct btrfs_inode_item *inode_item;
+ struct btrfs_timespec *tspec;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_key location;
+ u64 alloc_group_block;
+ u32 rdev;
+ int ret;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+ memcpy(&location, &BTRFS_I(inode)->location, sizeof(location));
+
+ ret = btrfs_lookup_inode(NULL, root, path, &location, 0);
+ if (ret)
+ goto make_bad;
+
+ leaf = path->nodes[0];
+ inode_item = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_inode_item);
+
+ inode->i_mode = btrfs_inode_mode(leaf, inode_item);
+ inode->i_nlink = btrfs_inode_nlink(leaf, inode_item);
+ inode->i_uid = btrfs_inode_uid(leaf, inode_item);
+ inode->i_gid = btrfs_inode_gid(leaf, inode_item);
+ btrfs_i_size_write(inode, btrfs_inode_size(leaf, inode_item));
+
+ tspec = btrfs_inode_atime(inode_item);
+ inode->i_atime.tv_sec = btrfs_timespec_sec(leaf, tspec);
+ inode->i_atime.tv_nsec = btrfs_timespec_nsec(leaf, tspec);
+
+ tspec = btrfs_inode_mtime(inode_item);
+ inode->i_mtime.tv_sec = btrfs_timespec_sec(leaf, tspec);
+ inode->i_mtime.tv_nsec = btrfs_timespec_nsec(leaf, tspec);
+
+ tspec = btrfs_inode_ctime(inode_item);
+ inode->i_ctime.tv_sec = btrfs_timespec_sec(leaf, tspec);
+ inode->i_ctime.tv_nsec = btrfs_timespec_nsec(leaf, tspec);
+
+ inode_set_bytes(inode, btrfs_inode_nbytes(leaf, inode_item));
+ BTRFS_I(inode)->generation = btrfs_inode_generation(leaf, inode_item);
+ BTRFS_I(inode)->sequence = btrfs_inode_sequence(leaf, inode_item);
+ inode->i_generation = BTRFS_I(inode)->generation;
+ inode->i_rdev = 0;
+ rdev = btrfs_inode_rdev(leaf, inode_item);
+
+ BTRFS_I(inode)->index_cnt = (u64)-1;
+ BTRFS_I(inode)->flags = btrfs_inode_flags(leaf, inode_item);
+
+ alloc_group_block = btrfs_inode_block_group(leaf, inode_item);
+ BTRFS_I(inode)->block_group = btrfs_find_block_group(root, 0,
+ alloc_group_block, 0);
+ btrfs_free_path(path);
+ inode_item = NULL;
+
+ switch (inode->i_mode & S_IFMT) {
+ case S_IFREG:
+ inode->i_mapping->a_ops = &btrfs_aops;
+ inode->i_mapping->backing_dev_info = &root->fs_info->bdi;
+ BTRFS_I(inode)->io_tree.ops = &btrfs_extent_io_ops;
+ inode->i_fop = &btrfs_file_operations;
+ inode->i_op = &btrfs_file_inode_operations;
+ break;
+ case S_IFDIR:
+ inode->i_fop = &btrfs_dir_file_operations;
+ if (root == root->fs_info->tree_root)
+ inode->i_op = &btrfs_dir_ro_inode_operations;
+ else
+ inode->i_op = &btrfs_dir_inode_operations;
+ break;
+ case S_IFLNK:
+ inode->i_op = &btrfs_symlink_inode_operations;
+ inode->i_mapping->a_ops = &btrfs_symlink_aops;
+ inode->i_mapping->backing_dev_info = &root->fs_info->bdi;
+ break;
+ default:
+ init_special_inode(inode, inode->i_mode, rdev);
+ break;
+ }
+ return;
+
+make_bad:
+ btrfs_free_path(path);
+ make_bad_inode(inode);
+}
+
+/*
+ * given a leaf and an inode, copy the inode fields into the leaf
+ */
+static void fill_inode_item(struct btrfs_trans_handle *trans,
+ struct extent_buffer *leaf,
+ struct btrfs_inode_item *item,
+ struct inode *inode)
+{
+ btrfs_set_inode_uid(leaf, item, inode->i_uid);
+ btrfs_set_inode_gid(leaf, item, inode->i_gid);
+ btrfs_set_inode_size(leaf, item, BTRFS_I(inode)->disk_i_size);
+ btrfs_set_inode_mode(leaf, item, inode->i_mode);
+ btrfs_set_inode_nlink(leaf, item, inode->i_nlink);
+
+ btrfs_set_timespec_sec(leaf, btrfs_inode_atime(item),
+ inode->i_atime.tv_sec);
+ btrfs_set_timespec_nsec(leaf, btrfs_inode_atime(item),
+ inode->i_atime.tv_nsec);
+
+ btrfs_set_timespec_sec(leaf, btrfs_inode_mtime(item),
+ inode->i_mtime.tv_sec);
+ btrfs_set_timespec_nsec(leaf, btrfs_inode_mtime(item),
+ inode->i_mtime.tv_nsec);
+
+ btrfs_set_timespec_sec(leaf, btrfs_inode_ctime(item),
+ inode->i_ctime.tv_sec);
+ btrfs_set_timespec_nsec(leaf, btrfs_inode_ctime(item),
+ inode->i_ctime.tv_nsec);
+
+ btrfs_set_inode_nbytes(leaf, item, inode_get_bytes(inode));
+ btrfs_set_inode_generation(leaf, item, BTRFS_I(inode)->generation);
+ btrfs_set_inode_sequence(leaf, item, BTRFS_I(inode)->sequence);
+ btrfs_set_inode_transid(leaf, item, trans->transid);
+ btrfs_set_inode_rdev(leaf, item, inode->i_rdev);
+ btrfs_set_inode_flags(leaf, item, BTRFS_I(inode)->flags);
+ btrfs_set_inode_block_group(leaf, item, BTRFS_I(inode)->block_group);
+}
+
+/*
+ * copy everything in the in-memory inode into the btree.
+ */
+noinline int btrfs_update_inode(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct inode *inode)
+{
+ struct btrfs_inode_item *inode_item;
+ struct btrfs_path *path;
+ struct extent_buffer *leaf;
+ int ret;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+ ret = btrfs_lookup_inode(trans, root, path,
+ &BTRFS_I(inode)->location, 1);
+ if (ret) {
+ if (ret > 0)
+ ret = -ENOENT;
+ goto failed;
+ }
+
+ leaf = path->nodes[0];
+ inode_item = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_inode_item);
+
+ fill_inode_item(trans, leaf, inode_item, inode);
+ btrfs_mark_buffer_dirty(leaf);
+ btrfs_set_inode_last_trans(trans, inode);
+ ret = 0;
+failed:
+ btrfs_free_path(path);
+ return ret;
+}
+
+
+/*
+ * unlink helper that gets used here in inode.c and in the tree logging
+ * recovery code. It remove a link in a directory with a given name, and
+ * also drops the back refs in the inode to the directory
+ */
+int btrfs_unlink_inode(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct inode *dir, struct inode *inode,
+ const char *name, int name_len)
+{
+ struct btrfs_path *path;
+ int ret = 0;
+ struct extent_buffer *leaf;
+ struct btrfs_dir_item *di;
+ struct btrfs_key key;
+ u64 index;
+
+ path = btrfs_alloc_path();
+ if (!path) {
+ ret = -ENOMEM;
+ goto err;
+ }
+
+ di = btrfs_lookup_dir_item(trans, root, path, dir->i_ino,
+ name, name_len, -1);
+ if (IS_ERR(di)) {
+ ret = PTR_ERR(di);
+ goto err;
+ }
+ if (!di) {
+ ret = -ENOENT;
+ goto err;
+ }
+ leaf = path->nodes[0];
+ btrfs_dir_item_key_to_cpu(leaf, di, &key);
+ ret = btrfs_delete_one_dir_name(trans, root, path, di);
+ if (ret)
+ goto err;
+ btrfs_release_path(root, path);
+
+ ret = btrfs_del_inode_ref(trans, root, name, name_len,
+ inode->i_ino,
+ dir->i_ino, &index);
+ if (ret) {
+ printk(KERN_INFO "btrfs failed to delete reference to %.*s, "
+ "inode %lu parent %lu\n", name_len, name,
+ inode->i_ino, dir->i_ino);
+ goto err;
+ }
+
+ di = btrfs_lookup_dir_index_item(trans, root, path, dir->i_ino,
+ index, name, name_len, -1);
+ if (IS_ERR(di)) {
+ ret = PTR_ERR(di);
+ goto err;
+ }
+ if (!di) {
+ ret = -ENOENT;
+ goto err;
+ }
+ ret = btrfs_delete_one_dir_name(trans, root, path, di);
+ btrfs_release_path(root, path);
+
+ ret = btrfs_del_inode_ref_in_log(trans, root, name, name_len,
+ inode, dir->i_ino);
+ BUG_ON(ret != 0 && ret != -ENOENT);
+ if (ret != -ENOENT)
+ BTRFS_I(dir)->log_dirty_trans = trans->transid;
+
+ ret = btrfs_del_dir_entries_in_log(trans, root, name, name_len,
+ dir, index);
+ BUG_ON(ret);
+err:
+ btrfs_free_path(path);
+ if (ret)
+ goto out;
+
+ btrfs_i_size_write(dir, dir->i_size - name_len * 2);
+ inode->i_ctime = dir->i_mtime = dir->i_ctime = CURRENT_TIME;
+ btrfs_update_inode(trans, root, dir);
+ btrfs_drop_nlink(inode);
+ ret = btrfs_update_inode(trans, root, inode);
+ dir->i_sb->s_dirt = 1;
+out:
+ return ret;
+}
+
+static int btrfs_unlink(struct inode *dir, struct dentry *dentry)
+{
+ struct btrfs_root *root;
+ struct btrfs_trans_handle *trans;
+ struct inode *inode = dentry->d_inode;
+ int ret;
+ unsigned long nr = 0;
+
+ root = BTRFS_I(dir)->root;
+
+ ret = btrfs_check_free_space(root, 1, 1);
+ if (ret)
+ goto fail;
+
+ trans = btrfs_start_transaction(root, 1);
+
+ btrfs_set_trans_block_group(trans, dir);
+ ret = btrfs_unlink_inode(trans, root, dir, dentry->d_inode,
+ dentry->d_name.name, dentry->d_name.len);
+
+ if (inode->i_nlink == 0)
+ ret = btrfs_orphan_add(trans, inode);
+
+ nr = trans->blocks_used;
+
+ btrfs_end_transaction_throttle(trans, root);
+fail:
+ btrfs_btree_balance_dirty(root, nr);
+ return ret;
+}
+
+static int btrfs_rmdir(struct inode *dir, struct dentry *dentry)
+{
+ struct inode *inode = dentry->d_inode;
+ int err = 0;
+ int ret;
+ struct btrfs_root *root = BTRFS_I(dir)->root;
+ struct btrfs_trans_handle *trans;
+ unsigned long nr = 0;
+
+ /*
+ * the FIRST_FREE_OBJECTID check makes sure we don't try to rmdir
+ * the root of a subvolume or snapshot
+ */
+ if (inode->i_size > BTRFS_EMPTY_DIR_SIZE ||
+ inode->i_ino == BTRFS_FIRST_FREE_OBJECTID) {
+ return -ENOTEMPTY;
+ }
+
+ ret = btrfs_check_free_space(root, 1, 1);
+ if (ret)
+ goto fail;
+
+ trans = btrfs_start_transaction(root, 1);
+ btrfs_set_trans_block_group(trans, dir);
+
+ err = btrfs_orphan_add(trans, inode);
+ if (err)
+ goto fail_trans;
+
+ /* now the directory is empty */
+ err = btrfs_unlink_inode(trans, root, dir, dentry->d_inode,
+ dentry->d_name.name, dentry->d_name.len);
+ if (!err)
+ btrfs_i_size_write(inode, 0);
+
+fail_trans:
+ nr = trans->blocks_used;
+ ret = btrfs_end_transaction_throttle(trans, root);
+fail:
+ btrfs_btree_balance_dirty(root, nr);
+
+ if (ret && !err)
+ err = ret;
+ return err;
+}
+
+#if 0
+/*
+ * when truncating bytes in a file, it is possible to avoid reading
+ * the leaves that contain only checksum items. This can be the
+ * majority of the IO required to delete a large file, but it must
+ * be done carefully.
+ *
+ * The keys in the level just above the leaves are checked to make sure
+ * the lowest key in a given leaf is a csum key, and starts at an offset
+ * after the new size.
+ *
+ * Then the key for the next leaf is checked to make sure it also has
+ * a checksum item for the same file. If it does, we know our target leaf
+ * contains only checksum items, and it can be safely freed without reading
+ * it.
+ *
+ * This is just an optimization targeted at large files. It may do
+ * nothing. It will return 0 unless things went badly.
+ */
+static noinline int drop_csum_leaves(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct inode *inode, u64 new_size)
+{
+ struct btrfs_key key;
+ int ret;
+ int nritems;
+ struct btrfs_key found_key;
+ struct btrfs_key other_key;
+ struct btrfs_leaf_ref *ref;
+ u64 leaf_gen;
+ u64 leaf_start;
+
+ path->lowest_level = 1;
+ key.objectid = inode->i_ino;
+ key.type = BTRFS_CSUM_ITEM_KEY;
+ key.offset = new_size;
+again:
+ ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
+ if (ret < 0)
+ goto out;
+
+ if (path->nodes[1] == NULL) {
+ ret = 0;
+ goto out;
+ }
+ ret = 0;
+ btrfs_node_key_to_cpu(path->nodes[1], &found_key, path->slots[1]);
+ nritems = btrfs_header_nritems(path->nodes[1]);
+
+ if (!nritems)
+ goto out;
+
+ if (path->slots[1] >= nritems)
+ goto next_node;
+
+ /* did we find a key greater than anything we want to delete? */
+ if (found_key.objectid > inode->i_ino ||
+ (found_key.objectid == inode->i_ino && found_key.type > key.type))
+ goto out;
+
+ /* we check the next key in the node to make sure the leave contains
+ * only checksum items. This comparison doesn't work if our
+ * leaf is the last one in the node
+ */
+ if (path->slots[1] + 1 >= nritems) {
+next_node:
+ /* search forward from the last key in the node, this
+ * will bring us into the next node in the tree
+ */
+ btrfs_node_key_to_cpu(path->nodes[1], &found_key, nritems - 1);
+
+ /* unlikely, but we inc below, so check to be safe */
+ if (found_key.offset == (u64)-1)
+ goto out;
+
+ /* search_forward needs a path with locks held, do the
+ * search again for the original key. It is possible
+ * this will race with a balance and return a path that
+ * we could modify, but this drop is just an optimization
+ * and is allowed to miss some leaves.
+ */
+ btrfs_release_path(root, path);
+ found_key.offset++;
+
+ /* setup a max key for search_forward */
+ other_key.offset = (u64)-1;
+ other_key.type = key.type;
+ other_key.objectid = key.objectid;
+
+ path->keep_locks = 1;
+ ret = btrfs_search_forward(root, &found_key, &other_key,
+ path, 0, 0);
+ path->keep_locks = 0;
+ if (ret || found_key.objectid != key.objectid ||
+ found_key.type != key.type) {
+ ret = 0;
+ goto out;
+ }
+
+ key.offset = found_key.offset;
+ btrfs_release_path(root, path);
+ cond_resched();
+ goto again;
+ }
+
+ /* we know there's one more slot after us in the tree,
+ * read that key so we can verify it is also a checksum item
+ */
+ btrfs_node_key_to_cpu(path->nodes[1], &other_key, path->slots[1] + 1);
+
+ if (found_key.objectid < inode->i_ino)
+ goto next_key;
+
+ if (found_key.type != key.type || found_key.offset < new_size)
+ goto next_key;
+
+ /*
+ * if the key for the next leaf isn't a csum key from this objectid,
+ * we can't be sure there aren't good items inside this leaf.
+ * Bail out
+ */
+ if (other_key.objectid != inode->i_ino || other_key.type != key.type)
+ goto out;
+
+ leaf_start = btrfs_node_blockptr(path->nodes[1], path->slots[1]);
+ leaf_gen = btrfs_node_ptr_generation(path->nodes[1], path->slots[1]);
+ /*
+ * it is safe to delete this leaf, it contains only
+ * csum items from this inode at an offset >= new_size
+ */
+ ret = btrfs_del_leaf(trans, root, path, leaf_start);
+ BUG_ON(ret);
+
+ if (root->ref_cows && leaf_gen < trans->transid) {
+ ref = btrfs_alloc_leaf_ref(root, 0);
+ if (ref) {
+ ref->root_gen = root->root_key.offset;
+ ref->bytenr = leaf_start;
+ ref->owner = 0;
+ ref->generation = leaf_gen;
+ ref->nritems = 0;
+
+ ret = btrfs_add_leaf_ref(root, ref, 0);
+ WARN_ON(ret);
+ btrfs_free_leaf_ref(root, ref);
+ } else {
+ WARN_ON(1);
+ }
+ }
+next_key:
+ btrfs_release_path(root, path);
+
+ if (other_key.objectid == inode->i_ino &&
+ other_key.type == key.type && other_key.offset > key.offset) {
+ key.offset = other_key.offset;
+ cond_resched();
+ goto again;
+ }
+ ret = 0;
+out:
+ /* fixup any changes we've made to the path */
+ path->lowest_level = 0;
+ path->keep_locks = 0;
+ btrfs_release_path(root, path);
+ return ret;
+}
+
+#endif
+
+/*
+ * this can truncate away extent items, csum items and directory items.
+ * It starts at a high offset and removes keys until it can't find
+ * any higher than new_size
+ *
+ * csum items that cross the new i_size are truncated to the new size
+ * as well.
+ *
+ * min_type is the minimum key type to truncate down to. If set to 0, this
+ * will kill all the items on this inode, including the INODE_ITEM_KEY.
+ */
+noinline int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct inode *inode,
+ u64 new_size, u32 min_type)
+{
+ int ret;
+ struct btrfs_path *path;
+ struct btrfs_key key;
+ struct btrfs_key found_key;
+ u32 found_type;
+ struct extent_buffer *leaf;
+ struct btrfs_file_extent_item *fi;
+ u64 extent_start = 0;
+ u64 extent_num_bytes = 0;
+ u64 item_end = 0;
+ u64 root_gen = 0;
+ u64 root_owner = 0;
+ int found_extent;
+ int del_item;
+ int pending_del_nr = 0;
+ int pending_del_slot = 0;
+ int extent_type = -1;
+ int encoding;
+ u64 mask = root->sectorsize - 1;
+
+ if (root->ref_cows)
+ btrfs_drop_extent_cache(inode, new_size & (~mask), (u64)-1, 0);
+ path = btrfs_alloc_path();
+ path->reada = -1;
+ BUG_ON(!path);
+
+ /* FIXME, add redo link to tree so we don't leak on crash */
+ key.objectid = inode->i_ino;
+ key.offset = (u64)-1;
+ key.type = (u8)-1;
+
+ btrfs_init_path(path);
+
+search_again:
+ ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
+ if (ret < 0)
+ goto error;
+
+ if (ret > 0) {
+ /* there are no items in the tree for us to truncate, we're
+ * done
+ */
+ if (path->slots[0] == 0) {
+ ret = 0;
+ goto error;
+ }
+ path->slots[0]--;
+ }
+
+ while (1) {
+ fi = NULL;
+ leaf = path->nodes[0];
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+ found_type = btrfs_key_type(&found_key);
+ encoding = 0;
+
+ if (found_key.objectid != inode->i_ino)
+ break;
+
+ if (found_type < min_type)
+ break;
+
+ item_end = found_key.offset;
+ if (found_type == BTRFS_EXTENT_DATA_KEY) {
+ fi = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_file_extent_item);
+ extent_type = btrfs_file_extent_type(leaf, fi);
+ encoding = btrfs_file_extent_compression(leaf, fi);
+ encoding |= btrfs_file_extent_encryption(leaf, fi);
+ encoding |= btrfs_file_extent_other_encoding(leaf, fi);
+
+ if (extent_type != BTRFS_FILE_EXTENT_INLINE) {
+ item_end +=
+ btrfs_file_extent_num_bytes(leaf, fi);
+ } else if (extent_type == BTRFS_FILE_EXTENT_INLINE) {
+ item_end += btrfs_file_extent_inline_len(leaf,
+ fi);
+ }
+ item_end--;
+ }
+ if (item_end < new_size) {
+ if (found_type == BTRFS_DIR_ITEM_KEY)
+ found_type = BTRFS_INODE_ITEM_KEY;
+ else if (found_type == BTRFS_EXTENT_ITEM_KEY)
+ found_type = BTRFS_EXTENT_DATA_KEY;
+ else if (found_type == BTRFS_EXTENT_DATA_KEY)
+ found_type = BTRFS_XATTR_ITEM_KEY;
+ else if (found_type == BTRFS_XATTR_ITEM_KEY)
+ found_type = BTRFS_INODE_REF_KEY;
+ else if (found_type)
+ found_type--;
+ else
+ break;
+ btrfs_set_key_type(&key, found_type);
+ goto next;
+ }
+ if (found_key.offset >= new_size)
+ del_item = 1;
+ else
+ del_item = 0;
+ found_extent = 0;
+
+ /* FIXME, shrink the extent if the ref count is only 1 */
+ if (found_type != BTRFS_EXTENT_DATA_KEY)
+ goto delete;
+
+ if (extent_type != BTRFS_FILE_EXTENT_INLINE) {
+ u64 num_dec;
+ extent_start = btrfs_file_extent_disk_bytenr(leaf, fi);
+ if (!del_item && !encoding) {
+ u64 orig_num_bytes =
+ btrfs_file_extent_num_bytes(leaf, fi);
+ extent_num_bytes = new_size -
+ found_key.offset + root->sectorsize - 1;
+ extent_num_bytes = extent_num_bytes &
+ ~((u64)root->sectorsize - 1);
+ btrfs_set_file_extent_num_bytes(leaf, fi,
+ extent_num_bytes);
+ num_dec = (orig_num_bytes -
+ extent_num_bytes);
+ if (root->ref_cows && extent_start != 0)
+ inode_sub_bytes(inode, num_dec);
+ btrfs_mark_buffer_dirty(leaf);
+ } else {
+ extent_num_bytes =
+ btrfs_file_extent_disk_num_bytes(leaf,
+ fi);
+ /* FIXME blocksize != 4096 */
+ num_dec = btrfs_file_extent_num_bytes(leaf, fi);
+ if (extent_start != 0) {
+ found_extent = 1;
+ if (root->ref_cows)
+ inode_sub_bytes(inode, num_dec);
+ }
+ root_gen = btrfs_header_generation(leaf);
+ root_owner = btrfs_header_owner(leaf);
+ }
+ } else if (extent_type == BTRFS_FILE_EXTENT_INLINE) {
+ /*
+ * we can't truncate inline items that have had
+ * special encodings
+ */
+ if (!del_item &&
+ btrfs_file_extent_compression(leaf, fi) == 0 &&
+ btrfs_file_extent_encryption(leaf, fi) == 0 &&
+ btrfs_file_extent_other_encoding(leaf, fi) == 0) {
+ u32 size = new_size - found_key.offset;
+
+ if (root->ref_cows) {
+ inode_sub_bytes(inode, item_end + 1 -
+ new_size);
+ }
+ size =
+ btrfs_file_extent_calc_inline_size(size);
+ ret = btrfs_truncate_item(trans, root, path,
+ size, 1);
+ BUG_ON(ret);
+ } else if (root->ref_cows) {
+ inode_sub_bytes(inode, item_end + 1 -
+ found_key.offset);
+ }
+ }
+delete:
+ if (del_item) {
+ if (!pending_del_nr) {
+ /* no pending yet, add ourselves */
+ pending_del_slot = path->slots[0];
+ pending_del_nr = 1;
+ } else if (pending_del_nr &&
+ path->slots[0] + 1 == pending_del_slot) {
+ /* hop on the pending chunk */
+ pending_del_nr++;
+ pending_del_slot = path->slots[0];
+ } else {
+ BUG();
+ }
+ } else {
+ break;
+ }
+ if (found_extent) {
+ ret = btrfs_free_extent(trans, root, extent_start,
+ extent_num_bytes,
+ leaf->start, root_owner,
+ root_gen, inode->i_ino, 0);
+ BUG_ON(ret);
+ }
+next:
+ if (path->slots[0] == 0) {
+ if (pending_del_nr)
+ goto del_pending;
+ btrfs_release_path(root, path);
+ goto search_again;
+ }
+
+ path->slots[0]--;
+ if (pending_del_nr &&
+ path->slots[0] + 1 != pending_del_slot) {
+ struct btrfs_key debug;
+del_pending:
+ btrfs_item_key_to_cpu(path->nodes[0], &debug,
+ pending_del_slot);
+ ret = btrfs_del_items(trans, root, path,
+ pending_del_slot,
+ pending_del_nr);
+ BUG_ON(ret);
+ pending_del_nr = 0;
+ btrfs_release_path(root, path);
+ goto search_again;
+ }
+ }
+ ret = 0;
+error:
+ if (pending_del_nr) {
+ ret = btrfs_del_items(trans, root, path, pending_del_slot,
+ pending_del_nr);
+ }
+ btrfs_free_path(path);
+ inode->i_sb->s_dirt = 1;
+ return ret;
+}
+
+/*
+ * taken from block_truncate_page, but does cow as it zeros out
+ * any bytes left in the last page in the file.
+ */
+static int btrfs_truncate_page(struct address_space *mapping, loff_t from)
+{
+ struct inode *inode = mapping->host;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct extent_io_tree *io_tree = &BTRFS_I(inode)->io_tree;
+ struct btrfs_ordered_extent *ordered;
+ char *kaddr;
+ u32 blocksize = root->sectorsize;
+ pgoff_t index = from >> PAGE_CACHE_SHIFT;
+ unsigned offset = from & (PAGE_CACHE_SIZE-1);
+ struct page *page;
+ int ret = 0;
+ u64 page_start;
+ u64 page_end;
+
+ if ((offset & (blocksize - 1)) == 0)
+ goto out;
+
+ ret = -ENOMEM;
+again:
+ page = grab_cache_page(mapping, index);
+ if (!page)
+ goto out;
+
+ page_start = page_offset(page);
+ page_end = page_start + PAGE_CACHE_SIZE - 1;
+
+ if (!PageUptodate(page)) {
+ ret = btrfs_readpage(NULL, page);
+ lock_page(page);
+ if (page->mapping != mapping) {
+ unlock_page(page);
+ page_cache_release(page);
+ goto again;
+ }
+ if (!PageUptodate(page)) {
+ ret = -EIO;
+ goto out_unlock;
+ }
+ }
+ wait_on_page_writeback(page);
+
+ lock_extent(io_tree, page_start, page_end, GFP_NOFS);
+ set_page_extent_mapped(page);
+
+ ordered = btrfs_lookup_ordered_extent(inode, page_start);
+ if (ordered) {
+ unlock_extent(io_tree, page_start, page_end, GFP_NOFS);
+ unlock_page(page);
+ page_cache_release(page);
+ btrfs_start_ordered_extent(inode, ordered, 1);
+ btrfs_put_ordered_extent(ordered);
+ goto again;
+ }
+
+ btrfs_set_extent_delalloc(inode, page_start, page_end);
+ ret = 0;
+ if (offset != PAGE_CACHE_SIZE) {
+ kaddr = kmap(page);
+ memset(kaddr + offset, 0, PAGE_CACHE_SIZE - offset);
+ flush_dcache_page(page);
+ kunmap(page);
+ }
+ ClearPageChecked(page);
+ set_page_dirty(page);
+ unlock_extent(io_tree, page_start, page_end, GFP_NOFS);
+
+out_unlock:
+ unlock_page(page);
+ page_cache_release(page);
+out:
+ return ret;
+}
+
+int btrfs_cont_expand(struct inode *inode, loff_t size)
+{
+ struct btrfs_trans_handle *trans;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct extent_io_tree *io_tree = &BTRFS_I(inode)->io_tree;
+ struct extent_map *em;
+ u64 mask = root->sectorsize - 1;
+ u64 hole_start = (inode->i_size + mask) & ~mask;
+ u64 block_end = (size + mask) & ~mask;
+ u64 last_byte;
+ u64 cur_offset;
+ u64 hole_size;
+ int err;
+
+ if (size <= hole_start)
+ return 0;
+
+ err = btrfs_check_free_space(root, 1, 0);
+ if (err)
+ return err;
+
+ btrfs_truncate_page(inode->i_mapping, inode->i_size);
+
+ while (1) {
+ struct btrfs_ordered_extent *ordered;
+ btrfs_wait_ordered_range(inode, hole_start,
+ block_end - hole_start);
+ lock_extent(io_tree, hole_start, block_end - 1, GFP_NOFS);
+ ordered = btrfs_lookup_ordered_extent(inode, hole_start);
+ if (!ordered)
+ break;
+ unlock_extent(io_tree, hole_start, block_end - 1, GFP_NOFS);
+ btrfs_put_ordered_extent(ordered);
+ }
+
+ trans = btrfs_start_transaction(root, 1);
+ btrfs_set_trans_block_group(trans, inode);
+
+ cur_offset = hole_start;
+ while (1) {
+ em = btrfs_get_extent(inode, NULL, 0, cur_offset,
+ block_end - cur_offset, 0);
+ BUG_ON(IS_ERR(em) || !em);
+ last_byte = min(extent_map_end(em), block_end);
+ last_byte = (last_byte + mask) & ~mask;
+ if (test_bit(EXTENT_FLAG_VACANCY, &em->flags)) {
+ u64 hint_byte = 0;
+ hole_size = last_byte - cur_offset;
+ err = btrfs_drop_extents(trans, root, inode,
+ cur_offset,
+ cur_offset + hole_size,
+ cur_offset, &hint_byte);
+ if (err)
+ break;
+ err = btrfs_insert_file_extent(trans, root,
+ inode->i_ino, cur_offset, 0,
+ 0, hole_size, 0, hole_size,
+ 0, 0, 0);
+ btrfs_drop_extent_cache(inode, hole_start,
+ last_byte - 1, 0);
+ }
+ free_extent_map(em);
+ cur_offset = last_byte;
+ if (err || cur_offset >= block_end)
+ break;
+ }
+
+ btrfs_end_transaction(trans, root);
+ unlock_extent(io_tree, hole_start, block_end - 1, GFP_NOFS);
+ return err;
+}
+
+static int btrfs_setattr(struct dentry *dentry, struct iattr *attr)
+{
+ struct inode *inode = dentry->d_inode;
+ int err;
+
+ err = inode_change_ok(inode, attr);
+ if (err)
+ return err;
+
+ if (S_ISREG(inode->i_mode) &&
+ attr->ia_valid & ATTR_SIZE && attr->ia_size > inode->i_size) {
+ err = btrfs_cont_expand(inode, attr->ia_size);
+ if (err)
+ return err;
+ }
+
+ err = inode_setattr(inode, attr);
+
+ if (!err && ((attr->ia_valid & ATTR_MODE)))
+ err = btrfs_acl_chmod(inode);
+ return err;
+}
+
+void btrfs_delete_inode(struct inode *inode)
+{
+ struct btrfs_trans_handle *trans;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ unsigned long nr;
+ int ret;
+
+ truncate_inode_pages(&inode->i_data, 0);
+ if (is_bad_inode(inode)) {
+ btrfs_orphan_del(NULL, inode);
+ goto no_delete;
+ }
+ btrfs_wait_ordered_range(inode, 0, (u64)-1);
+
+ btrfs_i_size_write(inode, 0);
+ trans = btrfs_join_transaction(root, 1);
+
+ btrfs_set_trans_block_group(trans, inode);
+ ret = btrfs_truncate_inode_items(trans, root, inode, inode->i_size, 0);
+ if (ret) {
+ btrfs_orphan_del(NULL, inode);
+ goto no_delete_lock;
+ }
+
+ btrfs_orphan_del(trans, inode);
+
+ nr = trans->blocks_used;
+ clear_inode(inode);
+
+ btrfs_end_transaction(trans, root);
+ btrfs_btree_balance_dirty(root, nr);
+ return;
+
+no_delete_lock:
+ nr = trans->blocks_used;
+ btrfs_end_transaction(trans, root);
+ btrfs_btree_balance_dirty(root, nr);
+no_delete:
+ clear_inode(inode);
+}
+
+/*
+ * this returns the key found in the dir entry in the location pointer.
+ * If no dir entries were found, location->objectid is 0.
+ */
+static int btrfs_inode_by_name(struct inode *dir, struct dentry *dentry,
+ struct btrfs_key *location)
+{
+ const char *name = dentry->d_name.name;
+ int namelen = dentry->d_name.len;
+ struct btrfs_dir_item *di;
+ struct btrfs_path *path;
+ struct btrfs_root *root = BTRFS_I(dir)->root;
+ int ret = 0;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+
+ di = btrfs_lookup_dir_item(NULL, root, path, dir->i_ino, name,
+ namelen, 0);
+ if (IS_ERR(di))
+ ret = PTR_ERR(di);
+
+ if (!di || IS_ERR(di))
+ goto out_err;
+
+ btrfs_dir_item_key_to_cpu(path->nodes[0], di, location);
+out:
+ btrfs_free_path(path);
+ return ret;
+out_err:
+ location->objectid = 0;
+ goto out;
+}
+
+/*
+ * when we hit a tree root in a directory, the btrfs part of the inode
+ * needs to be changed to reflect the root directory of the tree root. This
+ * is kind of like crossing a mount point.
+ */
+static int fixup_tree_root_location(struct btrfs_root *root,
+ struct btrfs_key *location,
+ struct btrfs_root **sub_root,
+ struct dentry *dentry)
+{
+ struct btrfs_root_item *ri;
+
+ if (btrfs_key_type(location) != BTRFS_ROOT_ITEM_KEY)
+ return 0;
+ if (location->objectid == BTRFS_ROOT_TREE_OBJECTID)
+ return 0;
+
+ *sub_root = btrfs_read_fs_root(root->fs_info, location,
+ dentry->d_name.name,
+ dentry->d_name.len);
+ if (IS_ERR(*sub_root))
+ return PTR_ERR(*sub_root);
+
+ ri = &(*sub_root)->root_item;
+ location->objectid = btrfs_root_dirid(ri);
+ btrfs_set_key_type(location, BTRFS_INODE_ITEM_KEY);
+ location->offset = 0;
+
+ return 0;
+}
+
+static noinline void init_btrfs_i(struct inode *inode)
+{
+ struct btrfs_inode *bi = BTRFS_I(inode);
+
+ bi->i_acl = NULL;
+ bi->i_default_acl = NULL;
+
+ bi->generation = 0;
+ bi->sequence = 0;
+ bi->last_trans = 0;
+ bi->logged_trans = 0;
+ bi->delalloc_bytes = 0;
+ bi->disk_i_size = 0;
+ bi->flags = 0;
+ bi->index_cnt = (u64)-1;
+ bi->log_dirty_trans = 0;
+ extent_map_tree_init(&BTRFS_I(inode)->extent_tree, GFP_NOFS);
+ extent_io_tree_init(&BTRFS_I(inode)->io_tree,
+ inode->i_mapping, GFP_NOFS);
+ extent_io_tree_init(&BTRFS_I(inode)->io_failure_tree,
+ inode->i_mapping, GFP_NOFS);
+ INIT_LIST_HEAD(&BTRFS_I(inode)->delalloc_inodes);
+ btrfs_ordered_inode_tree_init(&BTRFS_I(inode)->ordered_tree);
+ mutex_init(&BTRFS_I(inode)->extent_mutex);
+ mutex_init(&BTRFS_I(inode)->log_mutex);
+}
+
+static int btrfs_init_locked_inode(struct inode *inode, void *p)
+{
+ struct btrfs_iget_args *args = p;
+ inode->i_ino = args->ino;
+ init_btrfs_i(inode);
+ BTRFS_I(inode)->root = args->root;
+ return 0;
+}
+
+static int btrfs_find_actor(struct inode *inode, void *opaque)
+{
+ struct btrfs_iget_args *args = opaque;
+ return args->ino == inode->i_ino &&
+ args->root == BTRFS_I(inode)->root;
+}
+
+struct inode *btrfs_ilookup(struct super_block *s, u64 objectid,
+ struct btrfs_root *root, int wait)
+{
+ struct inode *inode;
+ struct btrfs_iget_args args;
+ args.ino = objectid;
+ args.root = root;
+
+ if (wait) {
+ inode = ilookup5(s, objectid, btrfs_find_actor,
+ (void *)&args);
+ } else {
+ inode = ilookup5_nowait(s, objectid, btrfs_find_actor,
+ (void *)&args);
+ }
+ return inode;
+}
+
+struct inode *btrfs_iget_locked(struct super_block *s, u64 objectid,
+ struct btrfs_root *root)
+{
+ struct inode *inode;
+ struct btrfs_iget_args args;
+ args.ino = objectid;
+ args.root = root;
+
+ inode = iget5_locked(s, objectid, btrfs_find_actor,
+ btrfs_init_locked_inode,
+ (void *)&args);
+ return inode;
+}
+
+/* Get an inode object given its location and corresponding root.
+ * Returns in *is_new if the inode was read from disk
+ */
+struct inode *btrfs_iget(struct super_block *s, struct btrfs_key *location,
+ struct btrfs_root *root, int *is_new)
+{
+ struct inode *inode;
+
+ inode = btrfs_iget_locked(s, location->objectid, root);
+ if (!inode)
+ return ERR_PTR(-EACCES);
+
+ if (inode->i_state & I_NEW) {
+ BTRFS_I(inode)->root = root;
+ memcpy(&BTRFS_I(inode)->location, location, sizeof(*location));
+ btrfs_read_locked_inode(inode);
+ unlock_new_inode(inode);
+ if (is_new)
+ *is_new = 1;
+ } else {
+ if (is_new)
+ *is_new = 0;
+ }
+
+ return inode;
+}
+
+struct inode *btrfs_lookup_dentry(struct inode *dir, struct dentry *dentry)
+{
+ struct inode *inode;
+ struct btrfs_inode *bi = BTRFS_I(dir);
+ struct btrfs_root *root = bi->root;
+ struct btrfs_root *sub_root = root;
+ struct btrfs_key location;
+ int ret, new;
+
+ if (dentry->d_name.len > BTRFS_NAME_LEN)
+ return ERR_PTR(-ENAMETOOLONG);
+
+ ret = btrfs_inode_by_name(dir, dentry, &location);
+
+ if (ret < 0)
+ return ERR_PTR(ret);
+
+ inode = NULL;
+ if (location.objectid) {
+ ret = fixup_tree_root_location(root, &location, &sub_root,
+ dentry);
+ if (ret < 0)
+ return ERR_PTR(ret);
+ if (ret > 0)
+ return ERR_PTR(-ENOENT);
+ inode = btrfs_iget(dir->i_sb, &location, sub_root, &new);
+ if (IS_ERR(inode))
+ return ERR_CAST(inode);
+ }
+ return inode;
+}
+
+static struct dentry *btrfs_lookup(struct inode *dir, struct dentry *dentry,
+ struct nameidata *nd)
+{
+ struct inode *inode;
+
+ if (dentry->d_name.len > BTRFS_NAME_LEN)
+ return ERR_PTR(-ENAMETOOLONG);
+
+ inode = btrfs_lookup_dentry(dir, dentry);
+ if (IS_ERR(inode))
+ return ERR_CAST(inode);
+
+ return d_splice_alias(inode, dentry);
+}
+
+static unsigned char btrfs_filetype_table[] = {
+ DT_UNKNOWN, DT_REG, DT_DIR, DT_CHR, DT_BLK, DT_FIFO, DT_SOCK, DT_LNK
+};
+
+static int btrfs_real_readdir(struct file *filp, void *dirent,
+ filldir_t filldir)
+{
+ struct inode *inode = filp->f_dentry->d_inode;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_item *item;
+ struct btrfs_dir_item *di;
+ struct btrfs_key key;
+ struct btrfs_key found_key;
+ struct btrfs_path *path;
+ int ret;
+ u32 nritems;
+ struct extent_buffer *leaf;
+ int slot;
+ int advance;
+ unsigned char d_type;
+ int over = 0;
+ u32 di_cur;
+ u32 di_total;
+ u32 di_len;
+ int key_type = BTRFS_DIR_INDEX_KEY;
+ char tmp_name[32];
+ char *name_ptr;
+ int name_len;
+
+ /* FIXME, use a real flag for deciding about the key type */
+ if (root->fs_info->tree_root == root)
+ key_type = BTRFS_DIR_ITEM_KEY;
+
+ /* special case for "." */
+ if (filp->f_pos == 0) {
+ over = filldir(dirent, ".", 1,
+ 1, inode->i_ino,
+ DT_DIR);
+ if (over)
+ return 0;
+ filp->f_pos = 1;
+ }
+ /* special case for .., just use the back ref */
+ if (filp->f_pos == 1) {
+ u64 pino = parent_ino(filp->f_path.dentry);
+ over = filldir(dirent, "..", 2,
+ 2, pino, DT_DIR);
+ if (over)
+ return 0;
+ filp->f_pos = 2;
+ }
+ path = btrfs_alloc_path();
+ path->reada = 2;
+
+ btrfs_set_key_type(&key, key_type);
+ key.offset = filp->f_pos;
+ key.objectid = inode->i_ino;
+
+ ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ if (ret < 0)
+ goto err;
+ advance = 0;
+
+ while (1) {
+ leaf = path->nodes[0];
+ nritems = btrfs_header_nritems(leaf);
+ slot = path->slots[0];
+ if (advance || slot >= nritems) {
+ if (slot >= nritems - 1) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret)
+ break;
+ leaf = path->nodes[0];
+ nritems = btrfs_header_nritems(leaf);
+ slot = path->slots[0];
+ } else {
+ slot++;
+ path->slots[0]++;
+ }
+ }
+
+ advance = 1;
+ item = btrfs_item_nr(leaf, slot);
+ btrfs_item_key_to_cpu(leaf, &found_key, slot);
+
+ if (found_key.objectid != key.objectid)
+ break;
+ if (btrfs_key_type(&found_key) != key_type)
+ break;
+ if (found_key.offset < filp->f_pos)
+ continue;
+
+ filp->f_pos = found_key.offset;
+
+ di = btrfs_item_ptr(leaf, slot, struct btrfs_dir_item);
+ di_cur = 0;
+ di_total = btrfs_item_size(leaf, item);
+
+ while (di_cur < di_total) {
+ struct btrfs_key location;
+
+ name_len = btrfs_dir_name_len(leaf, di);
+ if (name_len <= sizeof(tmp_name)) {
+ name_ptr = tmp_name;
+ } else {
+ name_ptr = kmalloc(name_len, GFP_NOFS);
+ if (!name_ptr) {
+ ret = -ENOMEM;
+ goto err;
+ }
+ }
+ read_extent_buffer(leaf, name_ptr,
+ (unsigned long)(di + 1), name_len);
+
+ d_type = btrfs_filetype_table[btrfs_dir_type(leaf, di)];
+ btrfs_dir_item_key_to_cpu(leaf, di, &location);
+
+ /* is this a reference to our own snapshot? If so
+ * skip it
+ */
+ if (location.type == BTRFS_ROOT_ITEM_KEY &&
+ location.objectid == root->root_key.objectid) {
+ over = 0;
+ goto skip;
+ }
+ over = filldir(dirent, name_ptr, name_len,
+ found_key.offset, location.objectid,
+ d_type);
+
+skip:
+ if (name_ptr != tmp_name)
+ kfree(name_ptr);
+
+ if (over)
+ goto nopos;
+ di_len = btrfs_dir_name_len(leaf, di) +
+ btrfs_dir_data_len(leaf, di) + sizeof(*di);
+ di_cur += di_len;
+ di = (struct btrfs_dir_item *)((char *)di + di_len);
+ }
+ }
+
+ /* Reached end of directory/root. Bump pos past the last item. */
+ if (key_type == BTRFS_DIR_INDEX_KEY)
+ filp->f_pos = INT_LIMIT(typeof(filp->f_pos));
+ else
+ filp->f_pos++;
+nopos:
+ ret = 0;
+err:
+ btrfs_free_path(path);
+ return ret;
+}
+
+int btrfs_write_inode(struct inode *inode, int wait)
+{
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_trans_handle *trans;
+ int ret = 0;
+
+ if (root->fs_info->btree_inode == inode)
+ return 0;
+
+ if (wait) {
+ trans = btrfs_join_transaction(root, 1);
+ btrfs_set_trans_block_group(trans, inode);
+ ret = btrfs_commit_transaction(trans, root);
+ }
+ return ret;
+}
+
+/*
+ * This is somewhat expensive, updating the tree every time the
+ * inode changes. But, it is most likely to find the inode in cache.
+ * FIXME, needs more benchmarking...there are no reasons other than performance
+ * to keep or drop this code.
+ */
+void btrfs_dirty_inode(struct inode *inode)
+{
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_trans_handle *trans;
+
+ trans = btrfs_join_transaction(root, 1);
+ btrfs_set_trans_block_group(trans, inode);
+ btrfs_update_inode(trans, root, inode);
+ btrfs_end_transaction(trans, root);
+}
+
+/*
+ * find the highest existing sequence number in a directory
+ * and then set the in-memory index_cnt variable to reflect
+ * free sequence numbers
+ */
+static int btrfs_set_inode_index_count(struct inode *inode)
+{
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_key key, found_key;
+ struct btrfs_path *path;
+ struct extent_buffer *leaf;
+ int ret;
+
+ key.objectid = inode->i_ino;
+ btrfs_set_key_type(&key, BTRFS_DIR_INDEX_KEY);
+ key.offset = (u64)-1;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ if (ret < 0)
+ goto out;
+ /* FIXME: we should be able to handle this */
+ if (ret == 0)
+ goto out;
+ ret = 0;
+
+ /*
+ * MAGIC NUMBER EXPLANATION:
+ * since we search a directory based on f_pos we have to start at 2
+ * since '.' and '..' have f_pos of 0 and 1 respectively, so everybody
+ * else has to start at 2
+ */
+ if (path->slots[0] == 0) {
+ BTRFS_I(inode)->index_cnt = 2;
+ goto out;
+ }
+
+ path->slots[0]--;
+
+ leaf = path->nodes[0];
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+
+ if (found_key.objectid != inode->i_ino ||
+ btrfs_key_type(&found_key) != BTRFS_DIR_INDEX_KEY) {
+ BTRFS_I(inode)->index_cnt = 2;
+ goto out;
+ }
+
+ BTRFS_I(inode)->index_cnt = found_key.offset + 1;
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
+/*
+ * helper to find a free sequence number in a given directory. This current
+ * code is very simple, later versions will do smarter things in the btree
+ */
+int btrfs_set_inode_index(struct inode *dir, u64 *index)
+{
+ int ret = 0;
+
+ if (BTRFS_I(dir)->index_cnt == (u64)-1) {
+ ret = btrfs_set_inode_index_count(dir);
+ if (ret)
+ return ret;
+ }
+
+ *index = BTRFS_I(dir)->index_cnt;
+ BTRFS_I(dir)->index_cnt++;
+
+ return ret;
+}
+
+static struct inode *btrfs_new_inode(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct inode *dir,
+ const char *name, int name_len,
+ u64 ref_objectid, u64 objectid,
+ u64 alloc_hint, int mode, u64 *index)
+{
+ struct inode *inode;
+ struct btrfs_inode_item *inode_item;
+ struct btrfs_key *location;
+ struct btrfs_path *path;
+ struct btrfs_inode_ref *ref;
+ struct btrfs_key key[2];
+ u32 sizes[2];
+ unsigned long ptr;
+ int ret;
+ int owner;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+
+ inode = new_inode(root->fs_info->sb);
+ if (!inode)
+ return ERR_PTR(-ENOMEM);
+
+ if (dir) {
+ ret = btrfs_set_inode_index(dir, index);
+ if (ret)
+ return ERR_PTR(ret);
+ }
+ /*
+ * index_cnt is ignored for everything but a dir,
+ * btrfs_get_inode_index_count has an explanation for the magic
+ * number
+ */
+ init_btrfs_i(inode);
+ BTRFS_I(inode)->index_cnt = 2;
+ BTRFS_I(inode)->root = root;
+ BTRFS_I(inode)->generation = trans->transid;
+
+ if (mode & S_IFDIR)
+ owner = 0;
+ else
+ owner = 1;
+ BTRFS_I(inode)->block_group =
+ btrfs_find_block_group(root, 0, alloc_hint, owner);
+ if ((mode & S_IFREG)) {
+ if (btrfs_test_opt(root, NODATASUM))
+ btrfs_set_flag(inode, NODATASUM);
+ if (btrfs_test_opt(root, NODATACOW))
+ btrfs_set_flag(inode, NODATACOW);
+ }
+
+ key[0].objectid = objectid;
+ btrfs_set_key_type(&key[0], BTRFS_INODE_ITEM_KEY);
+ key[0].offset = 0;
+
+ key[1].objectid = objectid;
+ btrfs_set_key_type(&key[1], BTRFS_INODE_REF_KEY);
+ key[1].offset = ref_objectid;
+
+ sizes[0] = sizeof(struct btrfs_inode_item);
+ sizes[1] = name_len + sizeof(*ref);
+
+ ret = btrfs_insert_empty_items(trans, root, path, key, sizes, 2);
+ if (ret != 0)
+ goto fail;
+
+ if (objectid > root->highest_inode)
+ root->highest_inode = objectid;
+
+ inode->i_uid = current_fsuid();
+ inode->i_gid = current_fsgid();
+ inode->i_mode = mode;
+ inode->i_ino = objectid;
+ inode_set_bytes(inode, 0);
+ inode->i_mtime = inode->i_atime = inode->i_ctime = CURRENT_TIME;
+ inode_item = btrfs_item_ptr(path->nodes[0], path->slots[0],
+ struct btrfs_inode_item);
+ fill_inode_item(trans, path->nodes[0], inode_item, inode);
+
+ ref = btrfs_item_ptr(path->nodes[0], path->slots[0] + 1,
+ struct btrfs_inode_ref);
+ btrfs_set_inode_ref_name_len(path->nodes[0], ref, name_len);
+ btrfs_set_inode_ref_index(path->nodes[0], ref, *index);
+ ptr = (unsigned long)(ref + 1);
+ write_extent_buffer(path->nodes[0], name, ptr, name_len);
+
+ btrfs_mark_buffer_dirty(path->nodes[0]);
+ btrfs_free_path(path);
+
+ location = &BTRFS_I(inode)->location;
+ location->objectid = objectid;
+ location->offset = 0;
+ btrfs_set_key_type(location, BTRFS_INODE_ITEM_KEY);
+
+ insert_inode_hash(inode);
+ return inode;
+fail:
+ if (dir)
+ BTRFS_I(dir)->index_cnt--;
+ btrfs_free_path(path);
+ return ERR_PTR(ret);
+}
+
+static inline u8 btrfs_inode_type(struct inode *inode)
+{
+ return btrfs_type_by_mode[(inode->i_mode & S_IFMT) >> S_SHIFT];
+}
+
+/*
+ * utility function to add 'inode' into 'parent_inode' with
+ * a give name and a given sequence number.
+ * if 'add_backref' is true, also insert a backref from the
+ * inode to the parent directory.
+ */
+int btrfs_add_link(struct btrfs_trans_handle *trans,
+ struct inode *parent_inode, struct inode *inode,
+ const char *name, int name_len, int add_backref, u64 index)
+{
+ int ret;
+ struct btrfs_key key;
+ struct btrfs_root *root = BTRFS_I(parent_inode)->root;
+
+ key.objectid = inode->i_ino;
+ btrfs_set_key_type(&key, BTRFS_INODE_ITEM_KEY);
+ key.offset = 0;
+
+ ret = btrfs_insert_dir_item(trans, root, name, name_len,
+ parent_inode->i_ino,
+ &key, btrfs_inode_type(inode),
+ index);
+ if (ret == 0) {
+ if (add_backref) {
+ ret = btrfs_insert_inode_ref(trans, root,
+ name, name_len,
+ inode->i_ino,
+ parent_inode->i_ino,
+ index);
+ }
+ btrfs_i_size_write(parent_inode, parent_inode->i_size +
+ name_len * 2);
+ parent_inode->i_mtime = parent_inode->i_ctime = CURRENT_TIME;
+ ret = btrfs_update_inode(trans, root, parent_inode);
+ }
+ return ret;
+}
+
+static int btrfs_add_nondir(struct btrfs_trans_handle *trans,
+ struct dentry *dentry, struct inode *inode,
+ int backref, u64 index)
+{
+ int err = btrfs_add_link(trans, dentry->d_parent->d_inode,
+ inode, dentry->d_name.name,
+ dentry->d_name.len, backref, index);
+ if (!err) {
+ d_instantiate(dentry, inode);
+ return 0;
+ }
+ if (err > 0)
+ err = -EEXIST;
+ return err;
+}
+
+static int btrfs_mknod(struct inode *dir, struct dentry *dentry,
+ int mode, dev_t rdev)
+{
+ struct btrfs_trans_handle *trans;
+ struct btrfs_root *root = BTRFS_I(dir)->root;
+ struct inode *inode = NULL;
+ int err;
+ int drop_inode = 0;
+ u64 objectid;
+ unsigned long nr = 0;
+ u64 index = 0;
+
+ if (!new_valid_dev(rdev))
+ return -EINVAL;
+
+ err = btrfs_check_free_space(root, 1, 0);
+ if (err)
+ goto fail;
+
+ trans = btrfs_start_transaction(root, 1);
+ btrfs_set_trans_block_group(trans, dir);
+
+ err = btrfs_find_free_objectid(trans, root, dir->i_ino, &objectid);
+ if (err) {
+ err = -ENOSPC;
+ goto out_unlock;
+ }
+
+ inode = btrfs_new_inode(trans, root, dir, dentry->d_name.name,
+ dentry->d_name.len,
+ dentry->d_parent->d_inode->i_ino, objectid,
+ BTRFS_I(dir)->block_group, mode, &index);
+ err = PTR_ERR(inode);
+ if (IS_ERR(inode))
+ goto out_unlock;
+
+ err = btrfs_init_acl(inode, dir);
+ if (err) {
+ drop_inode = 1;
+ goto out_unlock;
+ }
+
+ btrfs_set_trans_block_group(trans, inode);
+ err = btrfs_add_nondir(trans, dentry, inode, 0, index);
+ if (err)
+ drop_inode = 1;
+ else {
+ inode->i_op = &btrfs_special_inode_operations;
+ init_special_inode(inode, inode->i_mode, rdev);
+ btrfs_update_inode(trans, root, inode);
+ }
+ dir->i_sb->s_dirt = 1;
+ btrfs_update_inode_block_group(trans, inode);
+ btrfs_update_inode_block_group(trans, dir);
+out_unlock:
+ nr = trans->blocks_used;
+ btrfs_end_transaction_throttle(trans, root);
+fail:
+ if (drop_inode) {
+ inode_dec_link_count(inode);
+ iput(inode);
+ }
+ btrfs_btree_balance_dirty(root, nr);
+ return err;
+}
+
+static int btrfs_create(struct inode *dir, struct dentry *dentry,
+ int mode, struct nameidata *nd)
+{
+ struct btrfs_trans_handle *trans;
+ struct btrfs_root *root = BTRFS_I(dir)->root;
+ struct inode *inode = NULL;
+ int err;
+ int drop_inode = 0;
+ unsigned long nr = 0;
+ u64 objectid;
+ u64 index = 0;
+
+ err = btrfs_check_free_space(root, 1, 0);
+ if (err)
+ goto fail;
+ trans = btrfs_start_transaction(root, 1);
+ btrfs_set_trans_block_group(trans, dir);
+
+ err = btrfs_find_free_objectid(trans, root, dir->i_ino, &objectid);
+ if (err) {
+ err = -ENOSPC;
+ goto out_unlock;
+ }
+
+ inode = btrfs_new_inode(trans, root, dir, dentry->d_name.name,
+ dentry->d_name.len,
+ dentry->d_parent->d_inode->i_ino,
+ objectid, BTRFS_I(dir)->block_group, mode,
+ &index);
+ err = PTR_ERR(inode);
+ if (IS_ERR(inode))
+ goto out_unlock;
+
+ err = btrfs_init_acl(inode, dir);
+ if (err) {
+ drop_inode = 1;
+ goto out_unlock;
+ }
+
+ btrfs_set_trans_block_group(trans, inode);
+ err = btrfs_add_nondir(trans, dentry, inode, 0, index);
+ if (err)
+ drop_inode = 1;
+ else {
+ inode->i_mapping->a_ops = &btrfs_aops;
+ inode->i_mapping->backing_dev_info = &root->fs_info->bdi;
+ inode->i_fop = &btrfs_file_operations;
+ inode->i_op = &btrfs_file_inode_operations;
+ BTRFS_I(inode)->io_tree.ops = &btrfs_extent_io_ops;
+ }
+ dir->i_sb->s_dirt = 1;
+ btrfs_update_inode_block_group(trans, inode);
+ btrfs_update_inode_block_group(trans, dir);
+out_unlock:
+ nr = trans->blocks_used;
+ btrfs_end_transaction_throttle(trans, root);
+fail:
+ if (drop_inode) {
+ inode_dec_link_count(inode);
+ iput(inode);
+ }
+ btrfs_btree_balance_dirty(root, nr);
+ return err;
+}
+
+static int btrfs_link(struct dentry *old_dentry, struct inode *dir,
+ struct dentry *dentry)
+{
+ struct btrfs_trans_handle *trans;
+ struct btrfs_root *root = BTRFS_I(dir)->root;
+ struct inode *inode = old_dentry->d_inode;
+ u64 index;
+ unsigned long nr = 0;
+ int err;
+ int drop_inode = 0;
+
+ if (inode->i_nlink == 0)
+ return -ENOENT;
+
+ btrfs_inc_nlink(inode);
+ err = btrfs_check_free_space(root, 1, 0);
+ if (err)
+ goto fail;
+ err = btrfs_set_inode_index(dir, &index);
+ if (err)
+ goto fail;
+
+ trans = btrfs_start_transaction(root, 1);
+
+ btrfs_set_trans_block_group(trans, dir);
+ atomic_inc(&inode->i_count);
+
+ err = btrfs_add_nondir(trans, dentry, inode, 1, index);
+
+ if (err)
+ drop_inode = 1;
+
+ dir->i_sb->s_dirt = 1;
+ btrfs_update_inode_block_group(trans, dir);
+ err = btrfs_update_inode(trans, root, inode);
+
+ if (err)
+ drop_inode = 1;
+
+ nr = trans->blocks_used;
+ btrfs_end_transaction_throttle(trans, root);
+fail:
+ if (drop_inode) {
+ inode_dec_link_count(inode);
+ iput(inode);
+ }
+ btrfs_btree_balance_dirty(root, nr);
+ return err;
+}
+
+static int btrfs_mkdir(struct inode *dir, struct dentry *dentry, int mode)
+{
+ struct inode *inode = NULL;
+ struct btrfs_trans_handle *trans;
+ struct btrfs_root *root = BTRFS_I(dir)->root;
+ int err = 0;
+ int drop_on_err = 0;
+ u64 objectid = 0;
+ u64 index = 0;
+ unsigned long nr = 1;
+
+ err = btrfs_check_free_space(root, 1, 0);
+ if (err)
+ goto out_unlock;
+
+ trans = btrfs_start_transaction(root, 1);
+ btrfs_set_trans_block_group(trans, dir);
+
+ if (IS_ERR(trans)) {
+ err = PTR_ERR(trans);
+ goto out_unlock;
+ }
+
+ err = btrfs_find_free_objectid(trans, root, dir->i_ino, &objectid);
+ if (err) {
+ err = -ENOSPC;
+ goto out_unlock;
+ }
+
+ inode = btrfs_new_inode(trans, root, dir, dentry->d_name.name,
+ dentry->d_name.len,
+ dentry->d_parent->d_inode->i_ino, objectid,
+ BTRFS_I(dir)->block_group, S_IFDIR | mode,
+ &index);
+ if (IS_ERR(inode)) {
+ err = PTR_ERR(inode);
+ goto out_fail;
+ }
+
+ drop_on_err = 1;
+
+ err = btrfs_init_acl(inode, dir);
+ if (err)
+ goto out_fail;
+
+ inode->i_op = &btrfs_dir_inode_operations;
+ inode->i_fop = &btrfs_dir_file_operations;
+ btrfs_set_trans_block_group(trans, inode);
+
+ btrfs_i_size_write(inode, 0);
+ err = btrfs_update_inode(trans, root, inode);
+ if (err)
+ goto out_fail;
+
+ err = btrfs_add_link(trans, dentry->d_parent->d_inode,
+ inode, dentry->d_name.name,
+ dentry->d_name.len, 0, index);
+ if (err)
+ goto out_fail;
+
+ d_instantiate(dentry, inode);
+ drop_on_err = 0;
+ dir->i_sb->s_dirt = 1;
+ btrfs_update_inode_block_group(trans, inode);
+ btrfs_update_inode_block_group(trans, dir);
+
+out_fail:
+ nr = trans->blocks_used;
+ btrfs_end_transaction_throttle(trans, root);
+
+out_unlock:
+ if (drop_on_err)
+ iput(inode);
+ btrfs_btree_balance_dirty(root, nr);
+ return err;
+}
+
+/* helper for btfs_get_extent. Given an existing extent in the tree,
+ * and an extent that you want to insert, deal with overlap and insert
+ * the new extent into the tree.
+ */
+static int merge_extent_mapping(struct extent_map_tree *em_tree,
+ struct extent_map *existing,
+ struct extent_map *em,
+ u64 map_start, u64 map_len)
+{
+ u64 start_diff;
+
+ BUG_ON(map_start < em->start || map_start >= extent_map_end(em));
+ start_diff = map_start - em->start;
+ em->start = map_start;
+ em->len = map_len;
+ if (em->block_start < EXTENT_MAP_LAST_BYTE &&
+ !test_bit(EXTENT_FLAG_COMPRESSED, &em->flags)) {
+ em->block_start += start_diff;
+ em->block_len -= start_diff;
+ }
+ return add_extent_mapping(em_tree, em);
+}
+
+static noinline int uncompress_inline(struct btrfs_path *path,
+ struct inode *inode, struct page *page,
+ size_t pg_offset, u64 extent_offset,
+ struct btrfs_file_extent_item *item)
+{
+ int ret;
+ struct extent_buffer *leaf = path->nodes[0];
+ char *tmp;
+ size_t max_size;
+ unsigned long inline_size;
+ unsigned long ptr;
+
+ WARN_ON(pg_offset != 0);
+ max_size = btrfs_file_extent_ram_bytes(leaf, item);
+ inline_size = btrfs_file_extent_inline_item_len(leaf,
+ btrfs_item_nr(leaf, path->slots[0]));
+ tmp = kmalloc(inline_size, GFP_NOFS);
+ ptr = btrfs_file_extent_inline_start(item);
+
+ read_extent_buffer(leaf, tmp, ptr, inline_size);
+
+ max_size = min_t(unsigned long, PAGE_CACHE_SIZE, max_size);
+ ret = btrfs_zlib_decompress(tmp, page, extent_offset,
+ inline_size, max_size);
+ if (ret) {
+ char *kaddr = kmap_atomic(page, KM_USER0);
+ unsigned long copy_size = min_t(u64,
+ PAGE_CACHE_SIZE - pg_offset,
+ max_size - extent_offset);
+ memset(kaddr + pg_offset, 0, copy_size);
+ kunmap_atomic(kaddr, KM_USER0);
+ }
+ kfree(tmp);
+ return 0;
+}
+
+/*
+ * a bit scary, this does extent mapping from logical file offset to the disk.
+ * the ugly parts come from merging extents from the disk with the in-ram
+ * representation. This gets more complex because of the data=ordered code,
+ * where the in-ram extents might be locked pending data=ordered completion.
+ *
+ * This also copies inline extents directly into the page.
+ */
+
+struct extent_map *btrfs_get_extent(struct inode *inode, struct page *page,
+ size_t pg_offset, u64 start, u64 len,
+ int create)
+{
+ int ret;
+ int err = 0;
+ u64 bytenr;
+ u64 extent_start = 0;
+ u64 extent_end = 0;
+ u64 objectid = inode->i_ino;
+ u32 found_type;
+ struct btrfs_path *path = NULL;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_file_extent_item *item;
+ struct extent_buffer *leaf;
+ struct btrfs_key found_key;
+ struct extent_map *em = NULL;
+ struct extent_map_tree *em_tree = &BTRFS_I(inode)->extent_tree;
+ struct extent_io_tree *io_tree = &BTRFS_I(inode)->io_tree;
+ struct btrfs_trans_handle *trans = NULL;
+ int compressed;
+
+again:
+ spin_lock(&em_tree->lock);
+ em = lookup_extent_mapping(em_tree, start, len);
+ if (em)
+ em->bdev = root->fs_info->fs_devices->latest_bdev;
+ spin_unlock(&em_tree->lock);
+
+ if (em) {
+ if (em->start > start || em->start + em->len <= start)
+ free_extent_map(em);
+ else if (em->block_start == EXTENT_MAP_INLINE && page)
+ free_extent_map(em);
+ else
+ goto out;
+ }
+ em = alloc_extent_map(GFP_NOFS);
+ if (!em) {
+ err = -ENOMEM;
+ goto out;
+ }
+ em->bdev = root->fs_info->fs_devices->latest_bdev;
+ em->start = EXTENT_MAP_HOLE;
+ em->orig_start = EXTENT_MAP_HOLE;
+ em->len = (u64)-1;
+ em->block_len = (u64)-1;
+
+ if (!path) {
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+ }
+
+ ret = btrfs_lookup_file_extent(trans, root, path,
+ objectid, start, trans != NULL);
+ if (ret < 0) {
+ err = ret;
+ goto out;
+ }
+
+ if (ret != 0) {
+ if (path->slots[0] == 0)
+ goto not_found;
+ path->slots[0]--;
+ }
+
+ leaf = path->nodes[0];
+ item = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_file_extent_item);
+ /* are we inside the extent that was found? */
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+ found_type = btrfs_key_type(&found_key);
+ if (found_key.objectid != objectid ||
+ found_type != BTRFS_EXTENT_DATA_KEY) {
+ goto not_found;
+ }
+
+ found_type = btrfs_file_extent_type(leaf, item);
+ extent_start = found_key.offset;
+ compressed = btrfs_file_extent_compression(leaf, item);
+ if (found_type == BTRFS_FILE_EXTENT_REG ||
+ found_type == BTRFS_FILE_EXTENT_PREALLOC) {
+ extent_end = extent_start +
+ btrfs_file_extent_num_bytes(leaf, item);
+ } else if (found_type == BTRFS_FILE_EXTENT_INLINE) {
+ size_t size;
+ size = btrfs_file_extent_inline_len(leaf, item);
+ extent_end = (extent_start + size + root->sectorsize - 1) &
+ ~((u64)root->sectorsize - 1);
+ }
+
+ if (start >= extent_end) {
+ path->slots[0]++;
+ if (path->slots[0] >= btrfs_header_nritems(leaf)) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret < 0) {
+ err = ret;
+ goto out;
+ }
+ if (ret > 0)
+ goto not_found;
+ leaf = path->nodes[0];
+ }
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+ if (found_key.objectid != objectid ||
+ found_key.type != BTRFS_EXTENT_DATA_KEY)
+ goto not_found;
+ if (start + len <= found_key.offset)
+ goto not_found;
+ em->start = start;
+ em->len = found_key.offset - start;
+ goto not_found_em;
+ }
+
+ if (found_type == BTRFS_FILE_EXTENT_REG ||
+ found_type == BTRFS_FILE_EXTENT_PREALLOC) {
+ em->start = extent_start;
+ em->len = extent_end - extent_start;
+ em->orig_start = extent_start -
+ btrfs_file_extent_offset(leaf, item);
+ bytenr = btrfs_file_extent_disk_bytenr(leaf, item);
+ if (bytenr == 0) {
+ em->block_start = EXTENT_MAP_HOLE;
+ goto insert;
+ }
+ if (compressed) {
+ set_bit(EXTENT_FLAG_COMPRESSED, &em->flags);
+ em->block_start = bytenr;
+ em->block_len = btrfs_file_extent_disk_num_bytes(leaf,
+ item);
+ } else {
+ bytenr += btrfs_file_extent_offset(leaf, item);
+ em->block_start = bytenr;
+ em->block_len = em->len;
+ if (found_type == BTRFS_FILE_EXTENT_PREALLOC)
+ set_bit(EXTENT_FLAG_PREALLOC, &em->flags);
+ }
+ goto insert;
+ } else if (found_type == BTRFS_FILE_EXTENT_INLINE) {
+ unsigned long ptr;
+ char *map;
+ size_t size;
+ size_t extent_offset;
+ size_t copy_size;
+
+ em->block_start = EXTENT_MAP_INLINE;
+ if (!page || create) {
+ em->start = extent_start;
+ em->len = extent_end - extent_start;
+ goto out;
+ }
+
+ size = btrfs_file_extent_inline_len(leaf, item);
+ extent_offset = page_offset(page) + pg_offset - extent_start;
+ copy_size = min_t(u64, PAGE_CACHE_SIZE - pg_offset,
+ size - extent_offset);
+ em->start = extent_start + extent_offset;
+ em->len = (copy_size + root->sectorsize - 1) &
+ ~((u64)root->sectorsize - 1);
+ em->orig_start = EXTENT_MAP_INLINE;
+ if (compressed)
+ set_bit(EXTENT_FLAG_COMPRESSED, &em->flags);
+ ptr = btrfs_file_extent_inline_start(item) + extent_offset;
+ if (create == 0 && !PageUptodate(page)) {
+ if (btrfs_file_extent_compression(leaf, item) ==
+ BTRFS_COMPRESS_ZLIB) {
+ ret = uncompress_inline(path, inode, page,
+ pg_offset,
+ extent_offset, item);
+ BUG_ON(ret);
+ } else {
+ map = kmap(page);
+ read_extent_buffer(leaf, map + pg_offset, ptr,
+ copy_size);
+ kunmap(page);
+ }
+ flush_dcache_page(page);
+ } else if (create && PageUptodate(page)) {
+ if (!trans) {
+ kunmap(page);
+ free_extent_map(em);
+ em = NULL;
+ btrfs_release_path(root, path);
+ trans = btrfs_join_transaction(root, 1);
+ goto again;
+ }
+ map = kmap(page);
+ write_extent_buffer(leaf, map + pg_offset, ptr,
+ copy_size);
+ kunmap(page);
+ btrfs_mark_buffer_dirty(leaf);
+ }
+ set_extent_uptodate(io_tree, em->start,
+ extent_map_end(em) - 1, GFP_NOFS);
+ goto insert;
+ } else {
+ printk(KERN_ERR "btrfs unknown found_type %d\n", found_type);
+ WARN_ON(1);
+ }
+not_found:
+ em->start = start;
+ em->len = len;
+not_found_em:
+ em->block_start = EXTENT_MAP_HOLE;
+ set_bit(EXTENT_FLAG_VACANCY, &em->flags);
+insert:
+ btrfs_release_path(root, path);
+ if (em->start > start || extent_map_end(em) <= start) {
+ printk(KERN_ERR "Btrfs: bad extent! em: [%llu %llu] passed "
+ "[%llu %llu]\n", (unsigned long long)em->start,
+ (unsigned long long)em->len,
+ (unsigned long long)start,
+ (unsigned long long)len);
+ err = -EIO;
+ goto out;
+ }
+
+ err = 0;
+ spin_lock(&em_tree->lock);
+ ret = add_extent_mapping(em_tree, em);
+ /* it is possible that someone inserted the extent into the tree
+ * while we had the lock dropped. It is also possible that
+ * an overlapping map exists in the tree
+ */
+ if (ret == -EEXIST) {
+ struct extent_map *existing;
+
+ ret = 0;
+
+ existing = lookup_extent_mapping(em_tree, start, len);
+ if (existing && (existing->start > start ||
+ existing->start + existing->len <= start)) {
+ free_extent_map(existing);
+ existing = NULL;
+ }
+ if (!existing) {
+ existing = lookup_extent_mapping(em_tree, em->start,
+ em->len);
+ if (existing) {
+ err = merge_extent_mapping(em_tree, existing,
+ em, start,
+ root->sectorsize);
+ free_extent_map(existing);
+ if (err) {
+ free_extent_map(em);
+ em = NULL;
+ }
+ } else {
+ err = -EIO;
+ free_extent_map(em);
+ em = NULL;
+ }
+ } else {
+ free_extent_map(em);
+ em = existing;
+ err = 0;
+ }
+ }
+ spin_unlock(&em_tree->lock);
+out:
+ if (path)
+ btrfs_free_path(path);
+ if (trans) {
+ ret = btrfs_end_transaction(trans, root);
+ if (!err)
+ err = ret;
+ }
+ if (err) {
+ free_extent_map(em);
+ WARN_ON(1);
+ return ERR_PTR(err);
+ }
+ return em;
+}
+
+static ssize_t btrfs_direct_IO(int rw, struct kiocb *iocb,
+ const struct iovec *iov, loff_t offset,
+ unsigned long nr_segs)
+{
+ return -EINVAL;
+}
+
+static sector_t btrfs_bmap(struct address_space *mapping, sector_t iblock)
+{
+ return extent_bmap(mapping, iblock, btrfs_get_extent);
+}
+
+int btrfs_readpage(struct file *file, struct page *page)
+{
+ struct extent_io_tree *tree;
+ tree = &BTRFS_I(page->mapping->host)->io_tree;
+ return extent_read_full_page(tree, page, btrfs_get_extent);
+}
+
+static int btrfs_writepage(struct page *page, struct writeback_control *wbc)
+{
+ struct extent_io_tree *tree;
+
+
+ if (current->flags & PF_MEMALLOC) {
+ redirty_page_for_writepage(wbc, page);
+ unlock_page(page);
+ return 0;
+ }
+ tree = &BTRFS_I(page->mapping->host)->io_tree;
+ return extent_write_full_page(tree, page, btrfs_get_extent, wbc);
+}
+
+int btrfs_writepages(struct address_space *mapping,
+ struct writeback_control *wbc)
+{
+ struct extent_io_tree *tree;
+
+ tree = &BTRFS_I(mapping->host)->io_tree;
+ return extent_writepages(tree, mapping, btrfs_get_extent, wbc);
+}
+
+static int
+btrfs_readpages(struct file *file, struct address_space *mapping,
+ struct list_head *pages, unsigned nr_pages)
+{
+ struct extent_io_tree *tree;
+ tree = &BTRFS_I(mapping->host)->io_tree;
+ return extent_readpages(tree, mapping, pages, nr_pages,
+ btrfs_get_extent);
+}
+static int __btrfs_releasepage(struct page *page, gfp_t gfp_flags)
+{
+ struct extent_io_tree *tree;
+ struct extent_map_tree *map;
+ int ret;
+
+ tree = &BTRFS_I(page->mapping->host)->io_tree;
+ map = &BTRFS_I(page->mapping->host)->extent_tree;
+ ret = try_release_extent_mapping(map, tree, page, gfp_flags);
+ if (ret == 1) {
+ ClearPagePrivate(page);
+ set_page_private(page, 0);
+ page_cache_release(page);
+ }
+ return ret;
+}
+
+static int btrfs_releasepage(struct page *page, gfp_t gfp_flags)
+{
+ if (PageWriteback(page) || PageDirty(page))
+ return 0;
+ return __btrfs_releasepage(page, gfp_flags);
+}
+
+static void btrfs_invalidatepage(struct page *page, unsigned long offset)
+{
+ struct extent_io_tree *tree;
+ struct btrfs_ordered_extent *ordered;
+ u64 page_start = page_offset(page);
+ u64 page_end = page_start + PAGE_CACHE_SIZE - 1;
+
+ wait_on_page_writeback(page);
+ tree = &BTRFS_I(page->mapping->host)->io_tree;
+ if (offset) {
+ btrfs_releasepage(page, GFP_NOFS);
+ return;
+ }
+
+ lock_extent(tree, page_start, page_end, GFP_NOFS);
+ ordered = btrfs_lookup_ordered_extent(page->mapping->host,
+ page_offset(page));
+ if (ordered) {
+ /*
+ * IO on this page will never be started, so we need
+ * to account for any ordered extents now
+ */
+ clear_extent_bit(tree, page_start, page_end,
+ EXTENT_DIRTY | EXTENT_DELALLOC |
+ EXTENT_LOCKED, 1, 0, GFP_NOFS);
+ btrfs_finish_ordered_io(page->mapping->host,
+ page_start, page_end);
+ btrfs_put_ordered_extent(ordered);
+ lock_extent(tree, page_start, page_end, GFP_NOFS);
+ }
+ clear_extent_bit(tree, page_start, page_end,
+ EXTENT_LOCKED | EXTENT_DIRTY | EXTENT_DELALLOC |
+ EXTENT_ORDERED,
+ 1, 1, GFP_NOFS);
+ __btrfs_releasepage(page, GFP_NOFS);
+
+ ClearPageChecked(page);
+ if (PagePrivate(page)) {
+ ClearPagePrivate(page);
+ set_page_private(page, 0);
+ page_cache_release(page);
+ }
+}
+
+/*
+ * btrfs_page_mkwrite() is not allowed to change the file size as it gets
+ * called from a page fault handler when a page is first dirtied. Hence we must
+ * be careful to check for EOF conditions here. We set the page up correctly
+ * for a written page which means we get ENOSPC checking when writing into
+ * holes and correct delalloc and unwritten extent mapping on filesystems that
+ * support these features.
+ *
+ * We are not allowed to take the i_mutex here so we have to play games to
+ * protect against truncate races as the page could now be beyond EOF. Because
+ * vmtruncate() writes the inode size before removing pages, once we have the
+ * page lock we can determine safely if the page is beyond EOF. If it is not
+ * beyond EOF, then the page is guaranteed safe against truncation until we
+ * unlock the page.
+ */
+int btrfs_page_mkwrite(struct vm_area_struct *vma, struct page *page)
+{
+ struct inode *inode = fdentry(vma->vm_file)->d_inode;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct extent_io_tree *io_tree = &BTRFS_I(inode)->io_tree;
+ struct btrfs_ordered_extent *ordered;
+ char *kaddr;
+ unsigned long zero_start;
+ loff_t size;
+ int ret;
+ u64 page_start;
+ u64 page_end;
+
+ ret = btrfs_check_free_space(root, PAGE_CACHE_SIZE, 0);
+ if (ret)
+ goto out;
+
+ ret = -EINVAL;
+again:
+ lock_page(page);
+ size = i_size_read(inode);
+ page_start = page_offset(page);
+ page_end = page_start + PAGE_CACHE_SIZE - 1;
+
+ if ((page->mapping != inode->i_mapping) ||
+ (page_start >= size)) {
+ /* page got truncated out from underneath us */
+ goto out_unlock;
+ }
+ wait_on_page_writeback(page);
+
+ lock_extent(io_tree, page_start, page_end, GFP_NOFS);
+ set_page_extent_mapped(page);
+
+ /*
+ * we can't set the delalloc bits if there are pending ordered
+ * extents. Drop our locks and wait for them to finish
+ */
+ ordered = btrfs_lookup_ordered_extent(inode, page_start);
+ if (ordered) {
+ unlock_extent(io_tree, page_start, page_end, GFP_NOFS);
+ unlock_page(page);
+ btrfs_start_ordered_extent(inode, ordered, 1);
+ btrfs_put_ordered_extent(ordered);
+ goto again;
+ }
+
+ btrfs_set_extent_delalloc(inode, page_start, page_end);
+ ret = 0;
+
+ /* page is wholly or partially inside EOF */
+ if (page_start + PAGE_CACHE_SIZE > size)
+ zero_start = size & ~PAGE_CACHE_MASK;
+ else
+ zero_start = PAGE_CACHE_SIZE;
+
+ if (zero_start != PAGE_CACHE_SIZE) {
+ kaddr = kmap(page);
+ memset(kaddr + zero_start, 0, PAGE_CACHE_SIZE - zero_start);
+ flush_dcache_page(page);
+ kunmap(page);
+ }
+ ClearPageChecked(page);
+ set_page_dirty(page);
+ unlock_extent(io_tree, page_start, page_end, GFP_NOFS);
+
+out_unlock:
+ unlock_page(page);
+out:
+ return ret;
+}
+
+static void btrfs_truncate(struct inode *inode)
+{
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ int ret;
+ struct btrfs_trans_handle *trans;
+ unsigned long nr;
+ u64 mask = root->sectorsize - 1;
+
+ if (!S_ISREG(inode->i_mode))
+ return;
+ if (IS_APPEND(inode) || IS_IMMUTABLE(inode))
+ return;
+
+ btrfs_truncate_page(inode->i_mapping, inode->i_size);
+ btrfs_wait_ordered_range(inode, inode->i_size & (~mask), (u64)-1);
+
+ trans = btrfs_start_transaction(root, 1);
+ btrfs_set_trans_block_group(trans, inode);
+ btrfs_i_size_write(inode, inode->i_size);
+
+ ret = btrfs_orphan_add(trans, inode);
+ if (ret)
+ goto out;
+ /* FIXME, add redo link to tree so we don't leak on crash */
+ ret = btrfs_truncate_inode_items(trans, root, inode, inode->i_size,
+ BTRFS_EXTENT_DATA_KEY);
+ btrfs_update_inode(trans, root, inode);
+
+ ret = btrfs_orphan_del(trans, inode);
+ BUG_ON(ret);
+
+out:
+ nr = trans->blocks_used;
+ ret = btrfs_end_transaction_throttle(trans, root);
+ BUG_ON(ret);
+ btrfs_btree_balance_dirty(root, nr);
+}
+
+/*
+ * create a new subvolume directory/inode (helper for the ioctl).
+ */
+int btrfs_create_subvol_root(struct btrfs_trans_handle *trans,
+ struct btrfs_root *new_root, struct dentry *dentry,
+ u64 new_dirid, u64 alloc_hint)
+{
+ struct inode *inode;
+ int error;
+ u64 index = 0;
+
+ inode = btrfs_new_inode(trans, new_root, NULL, "..", 2, new_dirid,
+ new_dirid, alloc_hint, S_IFDIR | 0700, &index);
+ if (IS_ERR(inode))
+ return PTR_ERR(inode);
+ inode->i_op = &btrfs_dir_inode_operations;
+ inode->i_fop = &btrfs_dir_file_operations;
+
+ inode->i_nlink = 1;
+ btrfs_i_size_write(inode, 0);
+
+ error = btrfs_update_inode(trans, new_root, inode);
+ if (error)
+ return error;
+
+ d_instantiate(dentry, inode);
+ return 0;
+}
+
+/* helper function for file defrag and space balancing. This
+ * forces readahead on a given range of bytes in an inode
+ */
+unsigned long btrfs_force_ra(struct address_space *mapping,
+ struct file_ra_state *ra, struct file *file,
+ pgoff_t offset, pgoff_t last_index)
+{
+ pgoff_t req_size = last_index - offset + 1;
+
+ page_cache_sync_readahead(mapping, ra, file, offset, req_size);
+ return offset + req_size;
+}
+
+struct inode *btrfs_alloc_inode(struct super_block *sb)
+{
+ struct btrfs_inode *ei;
+
+ ei = kmem_cache_alloc(btrfs_inode_cachep, GFP_NOFS);
+ if (!ei)
+ return NULL;
+ ei->last_trans = 0;
+ ei->logged_trans = 0;
+ btrfs_ordered_inode_tree_init(&ei->ordered_tree);
+ ei->i_acl = BTRFS_ACL_NOT_CACHED;
+ ei->i_default_acl = BTRFS_ACL_NOT_CACHED;
+ INIT_LIST_HEAD(&ei->i_orphan);
+ return &ei->vfs_inode;
+}
+
+void btrfs_destroy_inode(struct inode *inode)
+{
+ struct btrfs_ordered_extent *ordered;
+ WARN_ON(!list_empty(&inode->i_dentry));
+ WARN_ON(inode->i_data.nrpages);
+
+ if (BTRFS_I(inode)->i_acl &&
+ BTRFS_I(inode)->i_acl != BTRFS_ACL_NOT_CACHED)
+ posix_acl_release(BTRFS_I(inode)->i_acl);
+ if (BTRFS_I(inode)->i_default_acl &&
+ BTRFS_I(inode)->i_default_acl != BTRFS_ACL_NOT_CACHED)
+ posix_acl_release(BTRFS_I(inode)->i_default_acl);
+
+ spin_lock(&BTRFS_I(inode)->root->list_lock);
+ if (!list_empty(&BTRFS_I(inode)->i_orphan)) {
+ printk(KERN_ERR "BTRFS: inode %lu: inode still on the orphan"
+ " list\n", inode->i_ino);
+ dump_stack();
+ }
+ spin_unlock(&BTRFS_I(inode)->root->list_lock);
+
+ while (1) {
+ ordered = btrfs_lookup_first_ordered_extent(inode, (u64)-1);
+ if (!ordered)
+ break;
+ else {
+ printk(KERN_ERR "btrfs found ordered "
+ "extent %llu %llu on inode cleanup\n",
+ (unsigned long long)ordered->file_offset,
+ (unsigned long long)ordered->len);
+ btrfs_remove_ordered_extent(inode, ordered);
+ btrfs_put_ordered_extent(ordered);
+ btrfs_put_ordered_extent(ordered);
+ }
+ }
+ btrfs_drop_extent_cache(inode, 0, (u64)-1, 0);
+ kmem_cache_free(btrfs_inode_cachep, BTRFS_I(inode));
+}
+
+static void init_once(void *foo)
+{
+ struct btrfs_inode *ei = (struct btrfs_inode *) foo;
+
+ inode_init_once(&ei->vfs_inode);
+}
+
+void btrfs_destroy_cachep(void)
+{
+ if (btrfs_inode_cachep)
+ kmem_cache_destroy(btrfs_inode_cachep);
+ if (btrfs_trans_handle_cachep)
+ kmem_cache_destroy(btrfs_trans_handle_cachep);
+ if (btrfs_transaction_cachep)
+ kmem_cache_destroy(btrfs_transaction_cachep);
+ if (btrfs_bit_radix_cachep)
+ kmem_cache_destroy(btrfs_bit_radix_cachep);
+ if (btrfs_path_cachep)
+ kmem_cache_destroy(btrfs_path_cachep);
+}
+
+struct kmem_cache *btrfs_cache_create(const char *name, size_t size,
+ unsigned long extra_flags,
+ void (*ctor)(void *))
+{
+ return kmem_cache_create(name, size, 0, (SLAB_RECLAIM_ACCOUNT |
+ SLAB_MEM_SPREAD | extra_flags), ctor);
+}
+
+int btrfs_init_cachep(void)
+{
+ btrfs_inode_cachep = btrfs_cache_create("btrfs_inode_cache",
+ sizeof(struct btrfs_inode),
+ 0, init_once);
+ if (!btrfs_inode_cachep)
+ goto fail;
+ btrfs_trans_handle_cachep =
+ btrfs_cache_create("btrfs_trans_handle_cache",
+ sizeof(struct btrfs_trans_handle),
+ 0, NULL);
+ if (!btrfs_trans_handle_cachep)
+ goto fail;
+ btrfs_transaction_cachep = btrfs_cache_create("btrfs_transaction_cache",
+ sizeof(struct btrfs_transaction),
+ 0, NULL);
+ if (!btrfs_transaction_cachep)
+ goto fail;
+ btrfs_path_cachep = btrfs_cache_create("btrfs_path_cache",
+ sizeof(struct btrfs_path),
+ 0, NULL);
+ if (!btrfs_path_cachep)
+ goto fail;
+ btrfs_bit_radix_cachep = btrfs_cache_create("btrfs_radix", 256,
+ SLAB_DESTROY_BY_RCU, NULL);
+ if (!btrfs_bit_radix_cachep)
+ goto fail;
+ return 0;
+fail:
+ btrfs_destroy_cachep();
+ return -ENOMEM;
+}
+
+static int btrfs_getattr(struct vfsmount *mnt,
+ struct dentry *dentry, struct kstat *stat)
+{
+ struct inode *inode = dentry->d_inode;
+ generic_fillattr(inode, stat);
+ stat->dev = BTRFS_I(inode)->root->anon_super.s_dev;
+ stat->blksize = PAGE_CACHE_SIZE;
+ stat->blocks = (inode_get_bytes(inode) +
+ BTRFS_I(inode)->delalloc_bytes) >> 9;
+ return 0;
+}
+
+static int btrfs_rename(struct inode *old_dir, struct dentry *old_dentry,
+ struct inode *new_dir, struct dentry *new_dentry)
+{
+ struct btrfs_trans_handle *trans;
+ struct btrfs_root *root = BTRFS_I(old_dir)->root;
+ struct inode *new_inode = new_dentry->d_inode;
+ struct inode *old_inode = old_dentry->d_inode;
+ struct timespec ctime = CURRENT_TIME;
+ u64 index = 0;
+ int ret;
+
+ /* we're not allowed to rename between subvolumes */
+ if (BTRFS_I(old_inode)->root->root_key.objectid !=
+ BTRFS_I(new_dir)->root->root_key.objectid)
+ return -EXDEV;
+
+ if (S_ISDIR(old_inode->i_mode) && new_inode &&
+ new_inode->i_size > BTRFS_EMPTY_DIR_SIZE) {
+ return -ENOTEMPTY;
+ }
+
+ /* to rename a snapshot or subvolume, we need to juggle the
+ * backrefs. This isn't coded yet
+ */
+ if (old_inode->i_ino == BTRFS_FIRST_FREE_OBJECTID)
+ return -EXDEV;
+
+ ret = btrfs_check_free_space(root, 1, 0);
+ if (ret)
+ goto out_unlock;
+
+ trans = btrfs_start_transaction(root, 1);
+
+ btrfs_set_trans_block_group(trans, new_dir);
+
+ btrfs_inc_nlink(old_dentry->d_inode);
+ old_dir->i_ctime = old_dir->i_mtime = ctime;
+ new_dir->i_ctime = new_dir->i_mtime = ctime;
+ old_inode->i_ctime = ctime;
+
+ ret = btrfs_unlink_inode(trans, root, old_dir, old_dentry->d_inode,
+ old_dentry->d_name.name,
+ old_dentry->d_name.len);
+ if (ret)
+ goto out_fail;
+
+ if (new_inode) {
+ new_inode->i_ctime = CURRENT_TIME;
+ ret = btrfs_unlink_inode(trans, root, new_dir,
+ new_dentry->d_inode,
+ new_dentry->d_name.name,
+ new_dentry->d_name.len);
+ if (ret)
+ goto out_fail;
+ if (new_inode->i_nlink == 0) {
+ ret = btrfs_orphan_add(trans, new_dentry->d_inode);
+ if (ret)
+ goto out_fail;
+ }
+
+ }
+ ret = btrfs_set_inode_index(new_dir, &index);
+ if (ret)
+ goto out_fail;
+
+ ret = btrfs_add_link(trans, new_dentry->d_parent->d_inode,
+ old_inode, new_dentry->d_name.name,
+ new_dentry->d_name.len, 1, index);
+ if (ret)
+ goto out_fail;
+
+out_fail:
+ btrfs_end_transaction_throttle(trans, root);
+out_unlock:
+ return ret;
+}
+
+/*
+ * some fairly slow code that needs optimization. This walks the list
+ * of all the inodes with pending delalloc and forces them to disk.
+ */
+int btrfs_start_delalloc_inodes(struct btrfs_root *root)
+{
+ struct list_head *head = &root->fs_info->delalloc_inodes;
+ struct btrfs_inode *binode;
+ struct inode *inode;
+
+ if (root->fs_info->sb->s_flags & MS_RDONLY)
+ return -EROFS;
+
+ spin_lock(&root->fs_info->delalloc_lock);
+ while (!list_empty(head)) {
+ binode = list_entry(head->next, struct btrfs_inode,
+ delalloc_inodes);
+ inode = igrab(&binode->vfs_inode);
+ if (!inode)
+ list_del_init(&binode->delalloc_inodes);
+ spin_unlock(&root->fs_info->delalloc_lock);
+ if (inode) {
+ filemap_flush(inode->i_mapping);
+ iput(inode);
+ }
+ cond_resched();
+ spin_lock(&root->fs_info->delalloc_lock);
+ }
+ spin_unlock(&root->fs_info->delalloc_lock);
+
+ /* the filemap_flush will queue IO into the worker threads, but
+ * we have to make sure the IO is actually started and that
+ * ordered extents get created before we return
+ */
+ atomic_inc(&root->fs_info->async_submit_draining);
+ while (atomic_read(&root->fs_info->nr_async_submits) ||
+ atomic_read(&root->fs_info->async_delalloc_pages)) {
+ wait_event(root->fs_info->async_submit_wait,
+ (atomic_read(&root->fs_info->nr_async_submits) == 0 &&
+ atomic_read(&root->fs_info->async_delalloc_pages) == 0));
+ }
+ atomic_dec(&root->fs_info->async_submit_draining);
+ return 0;
+}
+
+static int btrfs_symlink(struct inode *dir, struct dentry *dentry,
+ const char *symname)
+{
+ struct btrfs_trans_handle *trans;
+ struct btrfs_root *root = BTRFS_I(dir)->root;
+ struct btrfs_path *path;
+ struct btrfs_key key;
+ struct inode *inode = NULL;
+ int err;
+ int drop_inode = 0;
+ u64 objectid;
+ u64 index = 0 ;
+ int name_len;
+ int datasize;
+ unsigned long ptr;
+ struct btrfs_file_extent_item *ei;
+ struct extent_buffer *leaf;
+ unsigned long nr = 0;
+
+ name_len = strlen(symname) + 1;
+ if (name_len > BTRFS_MAX_INLINE_DATA_SIZE(root))
+ return -ENAMETOOLONG;
+
+ err = btrfs_check_free_space(root, 1, 0);
+ if (err)
+ goto out_fail;
+
+ trans = btrfs_start_transaction(root, 1);
+ btrfs_set_trans_block_group(trans, dir);
+
+ err = btrfs_find_free_objectid(trans, root, dir->i_ino, &objectid);
+ if (err) {
+ err = -ENOSPC;
+ goto out_unlock;
+ }
+
+ inode = btrfs_new_inode(trans, root, dir, dentry->d_name.name,
+ dentry->d_name.len,
+ dentry->d_parent->d_inode->i_ino, objectid,
+ BTRFS_I(dir)->block_group, S_IFLNK|S_IRWXUGO,
+ &index);
+ err = PTR_ERR(inode);
+ if (IS_ERR(inode))
+ goto out_unlock;
+
+ err = btrfs_init_acl(inode, dir);
+ if (err) {
+ drop_inode = 1;
+ goto out_unlock;
+ }
+
+ btrfs_set_trans_block_group(trans, inode);
+ err = btrfs_add_nondir(trans, dentry, inode, 0, index);
+ if (err)
+ drop_inode = 1;
+ else {
+ inode->i_mapping->a_ops = &btrfs_aops;
+ inode->i_mapping->backing_dev_info = &root->fs_info->bdi;
+ inode->i_fop = &btrfs_file_operations;
+ inode->i_op = &btrfs_file_inode_operations;
+ BTRFS_I(inode)->io_tree.ops = &btrfs_extent_io_ops;
+ }
+ dir->i_sb->s_dirt = 1;
+ btrfs_update_inode_block_group(trans, inode);
+ btrfs_update_inode_block_group(trans, dir);
+ if (drop_inode)
+ goto out_unlock;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+ key.objectid = inode->i_ino;
+ key.offset = 0;
+ btrfs_set_key_type(&key, BTRFS_EXTENT_DATA_KEY);
+ datasize = btrfs_file_extent_calc_inline_size(name_len);
+ err = btrfs_insert_empty_item(trans, root, path, &key,
+ datasize);
+ if (err) {
+ drop_inode = 1;
+ goto out_unlock;
+ }
+ leaf = path->nodes[0];
+ ei = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_file_extent_item);
+ btrfs_set_file_extent_generation(leaf, ei, trans->transid);
+ btrfs_set_file_extent_type(leaf, ei,
+ BTRFS_FILE_EXTENT_INLINE);
+ btrfs_set_file_extent_encryption(leaf, ei, 0);
+ btrfs_set_file_extent_compression(leaf, ei, 0);
+ btrfs_set_file_extent_other_encoding(leaf, ei, 0);
+ btrfs_set_file_extent_ram_bytes(leaf, ei, name_len);
+
+ ptr = btrfs_file_extent_inline_start(ei);
+ write_extent_buffer(leaf, symname, ptr, name_len);
+ btrfs_mark_buffer_dirty(leaf);
+ btrfs_free_path(path);
+
+ inode->i_op = &btrfs_symlink_inode_operations;
+ inode->i_mapping->a_ops = &btrfs_symlink_aops;
+ inode->i_mapping->backing_dev_info = &root->fs_info->bdi;
+ inode_set_bytes(inode, name_len);
+ btrfs_i_size_write(inode, name_len - 1);
+ err = btrfs_update_inode(trans, root, inode);
+ if (err)
+ drop_inode = 1;
+
+out_unlock:
+ nr = trans->blocks_used;
+ btrfs_end_transaction_throttle(trans, root);
+out_fail:
+ if (drop_inode) {
+ inode_dec_link_count(inode);
+ iput(inode);
+ }
+ btrfs_btree_balance_dirty(root, nr);
+ return err;
+}
+
+static int prealloc_file_range(struct inode *inode, u64 start, u64 end,
+ u64 alloc_hint, int mode)
+{
+ struct btrfs_trans_handle *trans;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_key ins;
+ u64 alloc_size;
+ u64 cur_offset = start;
+ u64 num_bytes = end - start;
+ int ret = 0;
+
+ trans = btrfs_join_transaction(root, 1);
+ BUG_ON(!trans);
+ btrfs_set_trans_block_group(trans, inode);
+
+ while (num_bytes > 0) {
+ alloc_size = min(num_bytes, root->fs_info->max_extent);
+ ret = btrfs_reserve_extent(trans, root, alloc_size,
+ root->sectorsize, 0, alloc_hint,
+ (u64)-1, &ins, 1);
+ if (ret) {
+ WARN_ON(1);
+ goto out;
+ }
+ ret = insert_reserved_file_extent(trans, inode,
+ cur_offset, ins.objectid,
+ ins.offset, ins.offset,
+ ins.offset, 0, 0, 0,
+ BTRFS_FILE_EXTENT_PREALLOC);
+ BUG_ON(ret);
+ num_bytes -= ins.offset;
+ cur_offset += ins.offset;
+ alloc_hint = ins.objectid + ins.offset;
+ }
+out:
+ if (cur_offset > start) {
+ inode->i_ctime = CURRENT_TIME;
+ btrfs_set_flag(inode, PREALLOC);
+ if (!(mode & FALLOC_FL_KEEP_SIZE) &&
+ cur_offset > i_size_read(inode))
+ btrfs_i_size_write(inode, cur_offset);
+ ret = btrfs_update_inode(trans, root, inode);
+ BUG_ON(ret);
+ }
+
+ btrfs_end_transaction(trans, root);
+ return ret;
+}
+
+static long btrfs_fallocate(struct inode *inode, int mode,
+ loff_t offset, loff_t len)
+{
+ u64 cur_offset;
+ u64 last_byte;
+ u64 alloc_start;
+ u64 alloc_end;
+ u64 alloc_hint = 0;
+ u64 mask = BTRFS_I(inode)->root->sectorsize - 1;
+ struct extent_map *em;
+ int ret;
+
+ alloc_start = offset & ~mask;
+ alloc_end = (offset + len + mask) & ~mask;
+
+ mutex_lock(&inode->i_mutex);
+ if (alloc_start > inode->i_size) {
+ ret = btrfs_cont_expand(inode, alloc_start);
+ if (ret)
+ goto out;
+ }
+
+ while (1) {
+ struct btrfs_ordered_extent *ordered;
+ lock_extent(&BTRFS_I(inode)->io_tree, alloc_start,
+ alloc_end - 1, GFP_NOFS);
+ ordered = btrfs_lookup_first_ordered_extent(inode,
+ alloc_end - 1);
+ if (ordered &&
+ ordered->file_offset + ordered->len > alloc_start &&
+ ordered->file_offset < alloc_end) {
+ btrfs_put_ordered_extent(ordered);
+ unlock_extent(&BTRFS_I(inode)->io_tree,
+ alloc_start, alloc_end - 1, GFP_NOFS);
+ btrfs_wait_ordered_range(inode, alloc_start,
+ alloc_end - alloc_start);
+ } else {
+ if (ordered)
+ btrfs_put_ordered_extent(ordered);
+ break;
+ }
+ }
+
+ cur_offset = alloc_start;
+ while (1) {
+ em = btrfs_get_extent(inode, NULL, 0, cur_offset,
+ alloc_end - cur_offset, 0);
+ BUG_ON(IS_ERR(em) || !em);
+ last_byte = min(extent_map_end(em), alloc_end);
+ last_byte = (last_byte + mask) & ~mask;
+ if (em->block_start == EXTENT_MAP_HOLE) {
+ ret = prealloc_file_range(inode, cur_offset,
+ last_byte, alloc_hint, mode);
+ if (ret < 0) {
+ free_extent_map(em);
+ break;
+ }
+ }
+ if (em->block_start <= EXTENT_MAP_LAST_BYTE)
+ alloc_hint = em->block_start;
+ free_extent_map(em);
+
+ cur_offset = last_byte;
+ if (cur_offset >= alloc_end) {
+ ret = 0;
+ break;
+ }
+ }
+ unlock_extent(&BTRFS_I(inode)->io_tree, alloc_start, alloc_end - 1,
+ GFP_NOFS);
+out:
+ mutex_unlock(&inode->i_mutex);
+ return ret;
+}
+
+static int btrfs_set_page_dirty(struct page *page)
+{
+ return __set_page_dirty_nobuffers(page);
+}
+
+static int btrfs_permission(struct inode *inode, int mask)
+{
+ if (btrfs_test_flag(inode, READONLY) && (mask & MAY_WRITE))
+ return -EACCES;
+ return generic_permission(inode, mask, btrfs_check_acl);
+}
+
+static struct inode_operations btrfs_dir_inode_operations = {
+ .getattr = btrfs_getattr,
+ .lookup = btrfs_lookup,
+ .create = btrfs_create,
+ .unlink = btrfs_unlink,
+ .link = btrfs_link,
+ .mkdir = btrfs_mkdir,
+ .rmdir = btrfs_rmdir,
+ .rename = btrfs_rename,
+ .symlink = btrfs_symlink,
+ .setattr = btrfs_setattr,
+ .mknod = btrfs_mknod,
+ .setxattr = btrfs_setxattr,
+ .getxattr = btrfs_getxattr,
+ .listxattr = btrfs_listxattr,
+ .removexattr = btrfs_removexattr,
+ .permission = btrfs_permission,
+};
+static struct inode_operations btrfs_dir_ro_inode_operations = {
+ .lookup = btrfs_lookup,
+ .permission = btrfs_permission,
+};
+static struct file_operations btrfs_dir_file_operations = {
+ .llseek = generic_file_llseek,
+ .read = generic_read_dir,
+ .readdir = btrfs_real_readdir,
+ .unlocked_ioctl = btrfs_ioctl,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = btrfs_ioctl,
+#endif
+ .release = btrfs_release_file,
+ .fsync = btrfs_sync_file,
+};
+
+static struct extent_io_ops btrfs_extent_io_ops = {
+ .fill_delalloc = run_delalloc_range,
+ .submit_bio_hook = btrfs_submit_bio_hook,
+ .merge_bio_hook = btrfs_merge_bio_hook,
+ .readpage_end_io_hook = btrfs_readpage_end_io_hook,
+ .writepage_end_io_hook = btrfs_writepage_end_io_hook,
+ .writepage_start_hook = btrfs_writepage_start_hook,
+ .readpage_io_failed_hook = btrfs_io_failed_hook,
+ .set_bit_hook = btrfs_set_bit_hook,
+ .clear_bit_hook = btrfs_clear_bit_hook,
+};
+
+static struct address_space_operations btrfs_aops = {
+ .readpage = btrfs_readpage,
+ .writepage = btrfs_writepage,
+ .writepages = btrfs_writepages,
+ .readpages = btrfs_readpages,
+ .sync_page = block_sync_page,
+ .bmap = btrfs_bmap,
+ .direct_IO = btrfs_direct_IO,
+ .invalidatepage = btrfs_invalidatepage,
+ .releasepage = btrfs_releasepage,
+ .set_page_dirty = btrfs_set_page_dirty,
+};
+
+static struct address_space_operations btrfs_symlink_aops = {
+ .readpage = btrfs_readpage,
+ .writepage = btrfs_writepage,
+ .invalidatepage = btrfs_invalidatepage,
+ .releasepage = btrfs_releasepage,
+};
+
+static struct inode_operations btrfs_file_inode_operations = {
+ .truncate = btrfs_truncate,
+ .getattr = btrfs_getattr,
+ .setattr = btrfs_setattr,
+ .setxattr = btrfs_setxattr,
+ .getxattr = btrfs_getxattr,
+ .listxattr = btrfs_listxattr,
+ .removexattr = btrfs_removexattr,
+ .permission = btrfs_permission,
+ .fallocate = btrfs_fallocate,
+};
+static struct inode_operations btrfs_special_inode_operations = {
+ .getattr = btrfs_getattr,
+ .setattr = btrfs_setattr,
+ .permission = btrfs_permission,
+ .setxattr = btrfs_setxattr,
+ .getxattr = btrfs_getxattr,
+ .listxattr = btrfs_listxattr,
+ .removexattr = btrfs_removexattr,
+};
+static struct inode_operations btrfs_symlink_inode_operations = {
+ .readlink = generic_readlink,
+ .follow_link = page_follow_link_light,
+ .put_link = page_put_link,
+ .permission = btrfs_permission,
+};
diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
new file mode 100644
index 0000000..c2aa33e
--- /dev/null
+++ b/fs/btrfs/ioctl.c
@@ -0,0 +1,1132 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/kernel.h>
+#include <linux/bio.h>
+#include <linux/buffer_head.h>
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <linux/fsnotify.h>
+#include <linux/pagemap.h>
+#include <linux/highmem.h>
+#include <linux/time.h>
+#include <linux/init.h>
+#include <linux/string.h>
+#include <linux/smp_lock.h>
+#include <linux/backing-dev.h>
+#include <linux/mount.h>
+#include <linux/mpage.h>
+#include <linux/namei.h>
+#include <linux/swap.h>
+#include <linux/writeback.h>
+#include <linux/statfs.h>
+#include <linux/compat.h>
+#include <linux/bit_spinlock.h>
+#include <linux/security.h>
+#include <linux/version.h>
+#include <linux/xattr.h>
+#include <linux/vmalloc.h>
+#include "compat.h"
+#include "ctree.h"
+#include "disk-io.h"
+#include "transaction.h"
+#include "btrfs_inode.h"
+#include "ioctl.h"
+#include "print-tree.h"
+#include "volumes.h"
+#include "locking.h"
+
+
+
+static noinline int create_subvol(struct btrfs_root *root,
+ struct dentry *dentry,
+ char *name, int namelen)
+{
+ struct btrfs_trans_handle *trans;
+ struct btrfs_key key;
+ struct btrfs_root_item root_item;
+ struct btrfs_inode_item *inode_item;
+ struct extent_buffer *leaf;
+ struct btrfs_root *new_root = root;
+ struct inode *dir;
+ int ret;
+ int err;
+ u64 objectid;
+ u64 new_dirid = BTRFS_FIRST_FREE_OBJECTID;
+ u64 index = 0;
+ unsigned long nr = 1;
+
+ ret = btrfs_check_free_space(root, 1, 0);
+ if (ret)
+ goto fail_commit;
+
+ trans = btrfs_start_transaction(root, 1);
+ BUG_ON(!trans);
+
+ ret = btrfs_find_free_objectid(trans, root->fs_info->tree_root,
+ 0, &objectid);
+ if (ret)
+ goto fail;
+
+ leaf = btrfs_alloc_free_block(trans, root, root->leafsize, 0,
+ objectid, trans->transid, 0, 0, 0);
+ if (IS_ERR(leaf)) {
+ ret = PTR_ERR(leaf);
+ goto fail;
+ }
+
+ btrfs_set_header_nritems(leaf, 0);
+ btrfs_set_header_level(leaf, 0);
+ btrfs_set_header_bytenr(leaf, leaf->start);
+ btrfs_set_header_generation(leaf, trans->transid);
+ btrfs_set_header_owner(leaf, objectid);
+
+ write_extent_buffer(leaf, root->fs_info->fsid,
+ (unsigned long)btrfs_header_fsid(leaf),
+ BTRFS_FSID_SIZE);
+ btrfs_mark_buffer_dirty(leaf);
+
+ inode_item = &root_item.inode;
+ memset(inode_item, 0, sizeof(*inode_item));
+ inode_item->generation = cpu_to_le64(1);
+ inode_item->size = cpu_to_le64(3);
+ inode_item->nlink = cpu_to_le32(1);
+ inode_item->nbytes = cpu_to_le64(root->leafsize);
+ inode_item->mode = cpu_to_le32(S_IFDIR | 0755);
+
+ btrfs_set_root_bytenr(&root_item, leaf->start);
+ btrfs_set_root_generation(&root_item, trans->transid);
+ btrfs_set_root_level(&root_item, 0);
+ btrfs_set_root_refs(&root_item, 1);
+ btrfs_set_root_used(&root_item, 0);
+ btrfs_set_root_last_snapshot(&root_item, 0);
+
+ memset(&root_item.drop_progress, 0, sizeof(root_item.drop_progress));
+ root_item.drop_level = 0;
+
+ btrfs_tree_unlock(leaf);
+ free_extent_buffer(leaf);
+ leaf = NULL;
+
+ btrfs_set_root_dirid(&root_item, new_dirid);
+
+ key.objectid = objectid;
+ key.offset = 1;
+ btrfs_set_key_type(&key, BTRFS_ROOT_ITEM_KEY);
+ ret = btrfs_insert_root(trans, root->fs_info->tree_root, &key,
+ &root_item);
+ if (ret)
+ goto fail;
+
+ /*
+ * insert the directory item
+ */
+ key.offset = (u64)-1;
+ dir = dentry->d_parent->d_inode;
+ ret = btrfs_set_inode_index(dir, &index);
+ BUG_ON(ret);
+
+ ret = btrfs_insert_dir_item(trans, root,
+ name, namelen, dir->i_ino, &key,
+ BTRFS_FT_DIR, index);
+ if (ret)
+ goto fail;
+
+ btrfs_i_size_write(dir, dir->i_size + namelen * 2);
+ ret = btrfs_update_inode(trans, root, dir);
+ BUG_ON(ret);
+
+ /* add the backref first */
+ ret = btrfs_add_root_ref(trans, root->fs_info->tree_root,
+ objectid, BTRFS_ROOT_BACKREF_KEY,
+ root->root_key.objectid,
+ dir->i_ino, index, name, namelen);
+
+ BUG_ON(ret);
+
+ /* now add the forward ref */
+ ret = btrfs_add_root_ref(trans, root->fs_info->tree_root,
+ root->root_key.objectid, BTRFS_ROOT_REF_KEY,
+ objectid,
+ dir->i_ino, index, name, namelen);
+
+ BUG_ON(ret);
+
+ ret = btrfs_commit_transaction(trans, root);
+ if (ret)
+ goto fail_commit;
+
+ new_root = btrfs_read_fs_root_no_name(root->fs_info, &key);
+ BUG_ON(!new_root);
+
+ trans = btrfs_start_transaction(new_root, 1);
+ BUG_ON(!trans);
+
+ ret = btrfs_create_subvol_root(trans, new_root, dentry, new_dirid,
+ BTRFS_I(dir)->block_group);
+ if (ret)
+ goto fail;
+
+fail:
+ nr = trans->blocks_used;
+ err = btrfs_commit_transaction(trans, new_root);
+ if (err && !ret)
+ ret = err;
+fail_commit:
+ btrfs_btree_balance_dirty(root, nr);
+ return ret;
+}
+
+static int create_snapshot(struct btrfs_root *root, struct dentry *dentry,
+ char *name, int namelen)
+{
+ struct btrfs_pending_snapshot *pending_snapshot;
+ struct btrfs_trans_handle *trans;
+ int ret = 0;
+ int err;
+ unsigned long nr = 0;
+
+ if (!root->ref_cows)
+ return -EINVAL;
+
+ ret = btrfs_check_free_space(root, 1, 0);
+ if (ret)
+ goto fail_unlock;
+
+ pending_snapshot = kzalloc(sizeof(*pending_snapshot), GFP_NOFS);
+ if (!pending_snapshot) {
+ ret = -ENOMEM;
+ goto fail_unlock;
+ }
+ pending_snapshot->name = kmalloc(namelen + 1, GFP_NOFS);
+ if (!pending_snapshot->name) {
+ ret = -ENOMEM;
+ kfree(pending_snapshot);
+ goto fail_unlock;
+ }
+ memcpy(pending_snapshot->name, name, namelen);
+ pending_snapshot->name[namelen] = '\0';
+ pending_snapshot->dentry = dentry;
+ trans = btrfs_start_transaction(root, 1);
+ BUG_ON(!trans);
+ pending_snapshot->root = root;
+ list_add(&pending_snapshot->list,
+ &trans->transaction->pending_snapshots);
+ err = btrfs_commit_transaction(trans, root);
+
+fail_unlock:
+ btrfs_btree_balance_dirty(root, nr);
+ return ret;
+}
+
+/* copy of may_create in fs/namei.c() */
+static inline int btrfs_may_create(struct inode *dir, struct dentry *child)
+{
+ if (child->d_inode)
+ return -EEXIST;
+ if (IS_DEADDIR(dir))
+ return -ENOENT;
+ return inode_permission(dir, MAY_WRITE | MAY_EXEC);
+}
+
+/*
+ * Create a new subvolume below @parent. This is largely modeled after
+ * sys_mkdirat and vfs_mkdir, but we only do a single component lookup
+ * inside this filesystem so it's quite a bit simpler.
+ */
+static noinline int btrfs_mksubvol(struct path *parent, char *name,
+ int mode, int namelen,
+ struct btrfs_root *snap_src)
+{
+ struct dentry *dentry;
+ int error;
+
+ mutex_lock_nested(&parent->dentry->d_inode->i_mutex, I_MUTEX_PARENT);
+
+ dentry = lookup_one_len(name, parent->dentry, namelen);
+ error = PTR_ERR(dentry);
+ if (IS_ERR(dentry))
+ goto out_unlock;
+
+ error = -EEXIST;
+ if (dentry->d_inode)
+ goto out_dput;
+
+ if (!IS_POSIXACL(parent->dentry->d_inode))
+ mode &= ~current->fs->umask;
+
+ error = mnt_want_write(parent->mnt);
+ if (error)
+ goto out_dput;
+
+ error = btrfs_may_create(parent->dentry->d_inode, dentry);
+ if (error)
+ goto out_drop_write;
+
+ /*
+ * Actually perform the low-level subvolume creation after all
+ * this VFS fuzz.
+ *
+ * Eventually we want to pass in an inode under which we create this
+ * subvolume, but for now all are under the filesystem root.
+ *
+ * Also we should pass on the mode eventually to allow creating new
+ * subvolume with specific mode bits.
+ */
+ if (snap_src) {
+ struct dentry *dir = dentry->d_parent;
+ struct dentry *test = dir->d_parent;
+ struct btrfs_path *path = btrfs_alloc_path();
+ int ret;
+ u64 test_oid;
+ u64 parent_oid = BTRFS_I(dir->d_inode)->root->root_key.objectid;
+
+ test_oid = snap_src->root_key.objectid;
+
+ ret = btrfs_find_root_ref(snap_src->fs_info->tree_root,
+ path, parent_oid, test_oid);
+ if (ret == 0)
+ goto create;
+ btrfs_release_path(snap_src->fs_info->tree_root, path);
+
+ /* we need to make sure we aren't creating a directory loop
+ * by taking a snapshot of something that has our current
+ * subvol in its directory tree. So, this loops through
+ * the dentries and checks the forward refs for each subvolume
+ * to see if is references the subvolume where we are
+ * placing this new snapshot.
+ */
+ while (1) {
+ if (!test ||
+ dir == snap_src->fs_info->sb->s_root ||
+ test == snap_src->fs_info->sb->s_root ||
+ test->d_inode->i_sb != snap_src->fs_info->sb) {
+ break;
+ }
+ if (S_ISLNK(test->d_inode->i_mode)) {
+ printk(KERN_INFO "Btrfs symlink in snapshot "
+ "path, failed\n");
+ error = -EMLINK;
+ btrfs_free_path(path);
+ goto out_drop_write;
+ }
+ test_oid =
+ BTRFS_I(test->d_inode)->root->root_key.objectid;
+ ret = btrfs_find_root_ref(snap_src->fs_info->tree_root,
+ path, test_oid, parent_oid);
+ if (ret == 0) {
+ printk(KERN_INFO "Btrfs snapshot creation "
+ "failed, looping\n");
+ error = -EMLINK;
+ btrfs_free_path(path);
+ goto out_drop_write;
+ }
+ btrfs_release_path(snap_src->fs_info->tree_root, path);
+ test = test->d_parent;
+ }
+create:
+ btrfs_free_path(path);
+ error = create_snapshot(snap_src, dentry, name, namelen);
+ } else {
+ error = create_subvol(BTRFS_I(parent->dentry->d_inode)->root,
+ dentry, name, namelen);
+ }
+ if (error)
+ goto out_drop_write;
+
+ fsnotify_mkdir(parent->dentry->d_inode, dentry);
+out_drop_write:
+ mnt_drop_write(parent->mnt);
+out_dput:
+ dput(dentry);
+out_unlock:
+ mutex_unlock(&parent->dentry->d_inode->i_mutex);
+ return error;
+}
+
+
+static int btrfs_defrag_file(struct file *file)
+{
+ struct inode *inode = fdentry(file)->d_inode;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct extent_io_tree *io_tree = &BTRFS_I(inode)->io_tree;
+ struct btrfs_ordered_extent *ordered;
+ struct page *page;
+ unsigned long last_index;
+ unsigned long ra_pages = root->fs_info->bdi.ra_pages;
+ unsigned long total_read = 0;
+ u64 page_start;
+ u64 page_end;
+ unsigned long i;
+ int ret;
+
+ ret = btrfs_check_free_space(root, inode->i_size, 0);
+ if (ret)
+ return -ENOSPC;
+
+ mutex_lock(&inode->i_mutex);
+ last_index = inode->i_size >> PAGE_CACHE_SHIFT;
+ for (i = 0; i <= last_index; i++) {
+ if (total_read % ra_pages == 0) {
+ btrfs_force_ra(inode->i_mapping, &file->f_ra, file, i,
+ min(last_index, i + ra_pages - 1));
+ }
+ total_read++;
+again:
+ page = grab_cache_page(inode->i_mapping, i);
+ if (!page)
+ goto out_unlock;
+ if (!PageUptodate(page)) {
+ btrfs_readpage(NULL, page);
+ lock_page(page);
+ if (!PageUptodate(page)) {
+ unlock_page(page);
+ page_cache_release(page);
+ goto out_unlock;
+ }
+ }
+
+ wait_on_page_writeback(page);
+
+ page_start = (u64)page->index << PAGE_CACHE_SHIFT;
+ page_end = page_start + PAGE_CACHE_SIZE - 1;
+ lock_extent(io_tree, page_start, page_end, GFP_NOFS);
+
+ ordered = btrfs_lookup_ordered_extent(inode, page_start);
+ if (ordered) {
+ unlock_extent(io_tree, page_start, page_end, GFP_NOFS);
+ unlock_page(page);
+ page_cache_release(page);
+ btrfs_start_ordered_extent(inode, ordered, 1);
+ btrfs_put_ordered_extent(ordered);
+ goto again;
+ }
+ set_page_extent_mapped(page);
+
+ /*
+ * this makes sure page_mkwrite is called on the
+ * page if it is dirtied again later
+ */
+ clear_page_dirty_for_io(page);
+
+ btrfs_set_extent_delalloc(inode, page_start, page_end);
+
+ unlock_extent(io_tree, page_start, page_end, GFP_NOFS);
+ set_page_dirty(page);
+ unlock_page(page);
+ page_cache_release(page);
+ balance_dirty_pages_ratelimited_nr(inode->i_mapping, 1);
+ }
+
+out_unlock:
+ mutex_unlock(&inode->i_mutex);
+ return 0;
+}
+
+/*
+ * Called inside transaction, so use GFP_NOFS
+ */
+
+static int btrfs_ioctl_resize(struct btrfs_root *root, void __user *arg)
+{
+ u64 new_size;
+ u64 old_size;
+ u64 devid = 1;
+ struct btrfs_ioctl_vol_args *vol_args;
+ struct btrfs_trans_handle *trans;
+ struct btrfs_device *device = NULL;
+ char *sizestr;
+ char *devstr = NULL;
+ int ret = 0;
+ int namelen;
+ int mod = 0;
+
+ if (root->fs_info->sb->s_flags & MS_RDONLY)
+ return -EROFS;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ vol_args = kmalloc(sizeof(*vol_args), GFP_NOFS);
+
+ if (!vol_args)
+ return -ENOMEM;
+
+ if (copy_from_user(vol_args, arg, sizeof(*vol_args))) {
+ ret = -EFAULT;
+ goto out;
+ }
+
+ vol_args->name[BTRFS_PATH_NAME_MAX] = '\0';
+ namelen = strlen(vol_args->name);
+
+ mutex_lock(&root->fs_info->volume_mutex);
+ sizestr = vol_args->name;
+ devstr = strchr(sizestr, ':');
+ if (devstr) {
+ char *end;
+ sizestr = devstr + 1;
+ *devstr = '\0';
+ devstr = vol_args->name;
+ devid = simple_strtoull(devstr, &end, 10);
+ printk(KERN_INFO "resizing devid %llu\n", devid);
+ }
+ device = btrfs_find_device(root, devid, NULL, NULL);
+ if (!device) {
+ printk(KERN_INFO "resizer unable to find device %llu\n", devid);
+ ret = -EINVAL;
+ goto out_unlock;
+ }
+ if (!strcmp(sizestr, "max"))
+ new_size = device->bdev->bd_inode->i_size;
+ else {
+ if (sizestr[0] == '-') {
+ mod = -1;
+ sizestr++;
+ } else if (sizestr[0] == '+') {
+ mod = 1;
+ sizestr++;
+ }
+ new_size = btrfs_parse_size(sizestr);
+ if (new_size == 0) {
+ ret = -EINVAL;
+ goto out_unlock;
+ }
+ }
+
+ old_size = device->total_bytes;
+
+ if (mod < 0) {
+ if (new_size > old_size) {
+ ret = -EINVAL;
+ goto out_unlock;
+ }
+ new_size = old_size - new_size;
+ } else if (mod > 0) {
+ new_size = old_size + new_size;
+ }
+
+ if (new_size < 256 * 1024 * 1024) {
+ ret = -EINVAL;
+ goto out_unlock;
+ }
+ if (new_size > device->bdev->bd_inode->i_size) {
+ ret = -EFBIG;
+ goto out_unlock;
+ }
+
+ do_div(new_size, root->sectorsize);
+ new_size *= root->sectorsize;
+
+ printk(KERN_INFO "new size for %s is %llu\n",
+ device->name, (unsigned long long)new_size);
+
+ if (new_size > old_size) {
+ trans = btrfs_start_transaction(root, 1);
+ ret = btrfs_grow_device(trans, device, new_size);
+ btrfs_commit_transaction(trans, root);
+ } else {
+ ret = btrfs_shrink_device(device, new_size);
+ }
+
+out_unlock:
+ mutex_unlock(&root->fs_info->volume_mutex);
+out:
+ kfree(vol_args);
+ return ret;
+}
+
+static noinline int btrfs_ioctl_snap_create(struct file *file,
+ void __user *arg, int subvol)
+{
+ struct btrfs_root *root = BTRFS_I(fdentry(file)->d_inode)->root;
+ struct btrfs_ioctl_vol_args *vol_args;
+ struct btrfs_dir_item *di;
+ struct btrfs_path *path;
+ struct file *src_file;
+ u64 root_dirid;
+ int namelen;
+ int ret = 0;
+
+ if (root->fs_info->sb->s_flags & MS_RDONLY)
+ return -EROFS;
+
+ vol_args = kmalloc(sizeof(*vol_args), GFP_NOFS);
+
+ if (!vol_args)
+ return -ENOMEM;
+
+ if (copy_from_user(vol_args, arg, sizeof(*vol_args))) {
+ ret = -EFAULT;
+ goto out;
+ }
+
+ vol_args->name[BTRFS_PATH_NAME_MAX] = '\0';
+ namelen = strlen(vol_args->name);
+ if (strchr(vol_args->name, '/')) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ path = btrfs_alloc_path();
+ if (!path) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ root_dirid = root->fs_info->sb->s_root->d_inode->i_ino,
+ di = btrfs_lookup_dir_item(NULL, root->fs_info->tree_root,
+ path, root_dirid,
+ vol_args->name, namelen, 0);
+ btrfs_free_path(path);
+
+ if (di && !IS_ERR(di)) {
+ ret = -EEXIST;
+ goto out;
+ }
+
+ if (IS_ERR(di)) {
+ ret = PTR_ERR(di);
+ goto out;
+ }
+
+ if (subvol) {
+ ret = btrfs_mksubvol(&file->f_path, vol_args->name,
+ file->f_path.dentry->d_inode->i_mode,
+ namelen, NULL);
+ } else {
+ struct inode *src_inode;
+ src_file = fget(vol_args->fd);
+ if (!src_file) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ src_inode = src_file->f_path.dentry->d_inode;
+ if (src_inode->i_sb != file->f_path.dentry->d_inode->i_sb) {
+ printk(KERN_INFO "btrfs: Snapshot src from "
+ "another FS\n");
+ ret = -EINVAL;
+ fput(src_file);
+ goto out;
+ }
+ ret = btrfs_mksubvol(&file->f_path, vol_args->name,
+ file->f_path.dentry->d_inode->i_mode,
+ namelen, BTRFS_I(src_inode)->root);
+ fput(src_file);
+ }
+
+out:
+ kfree(vol_args);
+ return ret;
+}
+
+static int btrfs_ioctl_defrag(struct file *file)
+{
+ struct inode *inode = fdentry(file)->d_inode;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ int ret;
+
+ ret = mnt_want_write(file->f_path.mnt);
+ if (ret)
+ return ret;
+
+ switch (inode->i_mode & S_IFMT) {
+ case S_IFDIR:
+ if (!capable(CAP_SYS_ADMIN)) {
+ ret = -EPERM;
+ goto out;
+ }
+ btrfs_defrag_root(root, 0);
+ btrfs_defrag_root(root->fs_info->extent_root, 0);
+ break;
+ case S_IFREG:
+ if (!(file->f_mode & FMODE_WRITE)) {
+ ret = -EINVAL;
+ goto out;
+ }
+ btrfs_defrag_file(file);
+ break;
+ }
+out:
+ mnt_drop_write(file->f_path.mnt);
+ return ret;
+}
+
+static long btrfs_ioctl_add_dev(struct btrfs_root *root, void __user *arg)
+{
+ struct btrfs_ioctl_vol_args *vol_args;
+ int ret;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ vol_args = kmalloc(sizeof(*vol_args), GFP_NOFS);
+
+ if (!vol_args)
+ return -ENOMEM;
+
+ if (copy_from_user(vol_args, arg, sizeof(*vol_args))) {
+ ret = -EFAULT;
+ goto out;
+ }
+ vol_args->name[BTRFS_PATH_NAME_MAX] = '\0';
+ ret = btrfs_init_new_device(root, vol_args->name);
+
+out:
+ kfree(vol_args);
+ return ret;
+}
+
+static long btrfs_ioctl_rm_dev(struct btrfs_root *root, void __user *arg)
+{
+ struct btrfs_ioctl_vol_args *vol_args;
+ int ret;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ if (root->fs_info->sb->s_flags & MS_RDONLY)
+ return -EROFS;
+
+ vol_args = kmalloc(sizeof(*vol_args), GFP_NOFS);
+
+ if (!vol_args)
+ return -ENOMEM;
+
+ if (copy_from_user(vol_args, arg, sizeof(*vol_args))) {
+ ret = -EFAULT;
+ goto out;
+ }
+ vol_args->name[BTRFS_PATH_NAME_MAX] = '\0';
+ ret = btrfs_rm_device(root, vol_args->name);
+
+out:
+ kfree(vol_args);
+ return ret;
+}
+
+static long btrfs_ioctl_clone(struct file *file, unsigned long srcfd,
+ u64 off, u64 olen, u64 destoff)
+{
+ struct inode *inode = fdentry(file)->d_inode;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct file *src_file;
+ struct inode *src;
+ struct btrfs_trans_handle *trans;
+ struct btrfs_path *path;
+ struct extent_buffer *leaf;
+ char *buf;
+ struct btrfs_key key;
+ u32 nritems;
+ int slot;
+ int ret;
+ u64 len = olen;
+ u64 bs = root->fs_info->sb->s_blocksize;
+ u64 hint_byte;
+
+ /*
+ * TODO:
+ * - split compressed inline extents. annoying: we need to
+ * decompress into destination's address_space (the file offset
+ * may change, so source mapping won't do), then recompress (or
+ * otherwise reinsert) a subrange.
+ * - allow ranges within the same file to be cloned (provided
+ * they don't overlap)?
+ */
+
+ /* the destination must be opened for writing */
+ if (!(file->f_mode & FMODE_WRITE))
+ return -EINVAL;
+
+ ret = mnt_want_write(file->f_path.mnt);
+ if (ret)
+ return ret;
+
+ src_file = fget(srcfd);
+ if (!src_file) {
+ ret = -EBADF;
+ goto out_drop_write;
+ }
+ src = src_file->f_dentry->d_inode;
+
+ ret = -EINVAL;
+ if (src == inode)
+ goto out_fput;
+
+ ret = -EISDIR;
+ if (S_ISDIR(src->i_mode) || S_ISDIR(inode->i_mode))
+ goto out_fput;
+
+ ret = -EXDEV;
+ if (src->i_sb != inode->i_sb || BTRFS_I(src)->root != root)
+ goto out_fput;
+
+ ret = -ENOMEM;
+ buf = vmalloc(btrfs_level_size(root, 0));
+ if (!buf)
+ goto out_fput;
+
+ path = btrfs_alloc_path();
+ if (!path) {
+ vfree(buf);
+ goto out_fput;
+ }
+ path->reada = 2;
+
+ if (inode < src) {
+ mutex_lock(&inode->i_mutex);
+ mutex_lock(&src->i_mutex);
+ } else {
+ mutex_lock(&src->i_mutex);
+ mutex_lock(&inode->i_mutex);
+ }
+
+ /* determine range to clone */
+ ret = -EINVAL;
+ if (off >= src->i_size || off + len > src->i_size)
+ goto out_unlock;
+ if (len == 0)
+ olen = len = src->i_size - off;
+ /* if we extend to eof, continue to block boundary */
+ if (off + len == src->i_size)
+ len = ((src->i_size + bs-1) & ~(bs-1))
+ - off;
+
+ /* verify the end result is block aligned */
+ if ((off & (bs-1)) ||
+ ((off + len) & (bs-1)))
+ goto out_unlock;
+
+ /* do any pending delalloc/csum calc on src, one way or
+ another, and lock file content */
+ while (1) {
+ struct btrfs_ordered_extent *ordered;
+ lock_extent(&BTRFS_I(src)->io_tree, off, off+len, GFP_NOFS);
+ ordered = btrfs_lookup_first_ordered_extent(inode, off+len);
+ if (BTRFS_I(src)->delalloc_bytes == 0 && !ordered)
+ break;
+ unlock_extent(&BTRFS_I(src)->io_tree, off, off+len, GFP_NOFS);
+ if (ordered)
+ btrfs_put_ordered_extent(ordered);
+ btrfs_wait_ordered_range(src, off, off+len);
+ }
+
+ trans = btrfs_start_transaction(root, 1);
+ BUG_ON(!trans);
+
+ /* punch hole in destination first */
+ btrfs_drop_extents(trans, root, inode, off, off+len, 0, &hint_byte);
+
+ /* clone data */
+ key.objectid = src->i_ino;
+ key.type = BTRFS_EXTENT_DATA_KEY;
+ key.offset = 0;
+
+ while (1) {
+ /*
+ * note the key will change type as we walk through the
+ * tree.
+ */
+ ret = btrfs_search_slot(trans, root, &key, path, 0, 0);
+ if (ret < 0)
+ goto out;
+
+ nritems = btrfs_header_nritems(path->nodes[0]);
+ if (path->slots[0] >= nritems) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret < 0)
+ goto out;
+ if (ret > 0)
+ break;
+ nritems = btrfs_header_nritems(path->nodes[0]);
+ }
+ leaf = path->nodes[0];
+ slot = path->slots[0];
+
+ btrfs_item_key_to_cpu(leaf, &key, slot);
+ if (btrfs_key_type(&key) > BTRFS_EXTENT_DATA_KEY ||
+ key.objectid != src->i_ino)
+ break;
+
+ if (btrfs_key_type(&key) == BTRFS_EXTENT_DATA_KEY) {
+ struct btrfs_file_extent_item *extent;
+ int type;
+ u32 size;
+ struct btrfs_key new_key;
+ u64 disko = 0, diskl = 0;
+ u64 datao = 0, datal = 0;
+ u8 comp;
+
+ size = btrfs_item_size_nr(leaf, slot);
+ read_extent_buffer(leaf, buf,
+ btrfs_item_ptr_offset(leaf, slot),
+ size);
+
+ extent = btrfs_item_ptr(leaf, slot,
+ struct btrfs_file_extent_item);
+ comp = btrfs_file_extent_compression(leaf, extent);
+ type = btrfs_file_extent_type(leaf, extent);
+ if (type == BTRFS_FILE_EXTENT_REG) {
+ disko = btrfs_file_extent_disk_bytenr(leaf,
+ extent);
+ diskl = btrfs_file_extent_disk_num_bytes(leaf,
+ extent);
+ datao = btrfs_file_extent_offset(leaf, extent);
+ datal = btrfs_file_extent_num_bytes(leaf,
+ extent);
+ } else if (type == BTRFS_FILE_EXTENT_INLINE) {
+ /* take upper bound, may be compressed */
+ datal = btrfs_file_extent_ram_bytes(leaf,
+ extent);
+ }
+ btrfs_release_path(root, path);
+
+ if (key.offset + datal < off ||
+ key.offset >= off+len)
+ goto next;
+
+ memcpy(&new_key, &key, sizeof(new_key));
+ new_key.objectid = inode->i_ino;
+ new_key.offset = key.offset + destoff - off;
+
+ if (type == BTRFS_FILE_EXTENT_REG) {
+ ret = btrfs_insert_empty_item(trans, root, path,
+ &new_key, size);
+ if (ret)
+ goto out;
+
+ leaf = path->nodes[0];
+ slot = path->slots[0];
+ write_extent_buffer(leaf, buf,
+ btrfs_item_ptr_offset(leaf, slot),
+ size);
+
+ extent = btrfs_item_ptr(leaf, slot,
+ struct btrfs_file_extent_item);
+
+ if (off > key.offset) {
+ datao += off - key.offset;
+ datal -= off - key.offset;
+ }
+ if (key.offset + datao + datal + key.offset >
+ off + len)
+ datal = off + len - key.offset - datao;
+ /* disko == 0 means it's a hole */
+ if (!disko)
+ datao = 0;
+
+ btrfs_set_file_extent_offset(leaf, extent,
+ datao);
+ btrfs_set_file_extent_num_bytes(leaf, extent,
+ datal);
+ if (disko) {
+ inode_add_bytes(inode, datal);
+ ret = btrfs_inc_extent_ref(trans, root,
+ disko, diskl, leaf->start,
+ root->root_key.objectid,
+ trans->transid,
+ inode->i_ino);
+ BUG_ON(ret);
+ }
+ } else if (type == BTRFS_FILE_EXTENT_INLINE) {
+ u64 skip = 0;
+ u64 trim = 0;
+ if (off > key.offset) {
+ skip = off - key.offset;
+ new_key.offset += skip;
+ }
+
+ if (key.offset + datal > off+len)
+ trim = key.offset + datal - (off+len);
+
+ if (comp && (skip || trim)) {
+ ret = -EINVAL;
+ goto out;
+ }
+ size -= skip + trim;
+ datal -= skip + trim;
+ ret = btrfs_insert_empty_item(trans, root, path,
+ &new_key, size);
+ if (ret)
+ goto out;
+
+ if (skip) {
+ u32 start =
+ btrfs_file_extent_calc_inline_size(0);
+ memmove(buf+start, buf+start+skip,
+ datal);
+ }
+
+ leaf = path->nodes[0];
+ slot = path->slots[0];
+ write_extent_buffer(leaf, buf,
+ btrfs_item_ptr_offset(leaf, slot),
+ size);
+ inode_add_bytes(inode, datal);
+ }
+
+ btrfs_mark_buffer_dirty(leaf);
+ }
+
+next:
+ btrfs_release_path(root, path);
+ key.offset++;
+ }
+ ret = 0;
+out:
+ btrfs_release_path(root, path);
+ if (ret == 0) {
+ inode->i_mtime = inode->i_ctime = CURRENT_TIME;
+ if (destoff + olen > inode->i_size)
+ btrfs_i_size_write(inode, destoff + olen);
+ BTRFS_I(inode)->flags = BTRFS_I(src)->flags;
+ ret = btrfs_update_inode(trans, root, inode);
+ }
+ btrfs_end_transaction(trans, root);
+ unlock_extent(&BTRFS_I(src)->io_tree, off, off+len, GFP_NOFS);
+ if (ret)
+ vmtruncate(inode, 0);
+out_unlock:
+ mutex_unlock(&src->i_mutex);
+ mutex_unlock(&inode->i_mutex);
+ vfree(buf);
+ btrfs_free_path(path);
+out_fput:
+ fput(src_file);
+out_drop_write:
+ mnt_drop_write(file->f_path.mnt);
+ return ret;
+}
+
+static long btrfs_ioctl_clone_range(struct file *file, void __user *argp)
+{
+ struct btrfs_ioctl_clone_range_args args;
+
+ if (copy_from_user(&args, argp, sizeof(args)))
+ return -EFAULT;
+ return btrfs_ioctl_clone(file, args.src_fd, args.src_offset,
+ args.src_length, args.dest_offset);
+}
+
+/*
+ * there are many ways the trans_start and trans_end ioctls can lead
+ * to deadlocks. They should only be used by applications that
+ * basically own the machine, and have a very in depth understanding
+ * of all the possible deadlocks and enospc problems.
+ */
+static long btrfs_ioctl_trans_start(struct file *file)
+{
+ struct inode *inode = fdentry(file)->d_inode;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_trans_handle *trans;
+ int ret = 0;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ if (file->private_data) {
+ ret = -EINPROGRESS;
+ goto out;
+ }
+
+ ret = mnt_want_write(file->f_path.mnt);
+ if (ret)
+ goto out;
+
+ mutex_lock(&root->fs_info->trans_mutex);
+ root->fs_info->open_ioctl_trans++;
+ mutex_unlock(&root->fs_info->trans_mutex);
+
+ trans = btrfs_start_ioctl_transaction(root, 0);
+ if (trans)
+ file->private_data = trans;
+ else
+ ret = -ENOMEM;
+ /*printk(KERN_INFO "btrfs_ioctl_trans_start on %p\n", file);*/
+out:
+ return ret;
+}
+
+/*
+ * there are many ways the trans_start and trans_end ioctls can lead
+ * to deadlocks. They should only be used by applications that
+ * basically own the machine, and have a very in depth understanding
+ * of all the possible deadlocks and enospc problems.
+ */
+long btrfs_ioctl_trans_end(struct file *file)
+{
+ struct inode *inode = fdentry(file)->d_inode;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_trans_handle *trans;
+ int ret = 0;
+
+ trans = file->private_data;
+ if (!trans) {
+ ret = -EINVAL;
+ goto out;
+ }
+ btrfs_end_transaction(trans, root);
+ file->private_data = NULL;
+
+ mutex_lock(&root->fs_info->trans_mutex);
+ root->fs_info->open_ioctl_trans--;
+ mutex_unlock(&root->fs_info->trans_mutex);
+
+ mnt_drop_write(file->f_path.mnt);
+
+out:
+ return ret;
+}
+
+long btrfs_ioctl(struct file *file, unsigned int
+ cmd, unsigned long arg)
+{
+ struct btrfs_root *root = BTRFS_I(fdentry(file)->d_inode)->root;
+ void __user *argp = (void __user *)arg;
+
+ switch (cmd) {
+ case BTRFS_IOC_SNAP_CREATE:
+ return btrfs_ioctl_snap_create(file, argp, 0);
+ case BTRFS_IOC_SUBVOL_CREATE:
+ return btrfs_ioctl_snap_create(file, argp, 1);
+ case BTRFS_IOC_DEFRAG:
+ return btrfs_ioctl_defrag(file);
+ case BTRFS_IOC_RESIZE:
+ return btrfs_ioctl_resize(root, argp);
+ case BTRFS_IOC_ADD_DEV:
+ return btrfs_ioctl_add_dev(root, argp);
+ case BTRFS_IOC_RM_DEV:
+ return btrfs_ioctl_rm_dev(root, argp);
+ case BTRFS_IOC_BALANCE:
+ return btrfs_balance(root->fs_info->dev_root);
+ case BTRFS_IOC_CLONE:
+ return btrfs_ioctl_clone(file, arg, 0, 0, 0);
+ case BTRFS_IOC_CLONE_RANGE:
+ return btrfs_ioctl_clone_range(file, argp);
+ case BTRFS_IOC_TRANS_START:
+ return btrfs_ioctl_trans_start(file);
+ case BTRFS_IOC_TRANS_END:
+ return btrfs_ioctl_trans_end(file);
+ case BTRFS_IOC_SYNC:
+ btrfs_sync_fs(file->f_dentry->d_sb, 1);
+ return 0;
+ }
+
+ return -ENOTTY;
+}
diff --git a/fs/btrfs/ioctl.h b/fs/btrfs/ioctl.h
new file mode 100644
index 0000000..78049ea
--- /dev/null
+++ b/fs/btrfs/ioctl.h
@@ -0,0 +1,67 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __IOCTL_
+#define __IOCTL_
+#include <linux/ioctl.h>
+
+#define BTRFS_IOCTL_MAGIC 0x94
+#define BTRFS_VOL_NAME_MAX 255
+#define BTRFS_PATH_NAME_MAX 3072
+
+struct btrfs_ioctl_vol_args {
+ __s64 fd;
+ char name[BTRFS_PATH_NAME_MAX + 1];
+};
+
+#define BTRFS_IOC_SNAP_CREATE _IOW(BTRFS_IOCTL_MAGIC, 1, \
+ struct btrfs_ioctl_vol_args)
+#define BTRFS_IOC_DEFRAG _IOW(BTRFS_IOCTL_MAGIC, 2, \
+ struct btrfs_ioctl_vol_args)
+#define BTRFS_IOC_RESIZE _IOW(BTRFS_IOCTL_MAGIC, 3, \
+ struct btrfs_ioctl_vol_args)
+#define BTRFS_IOC_SCAN_DEV _IOW(BTRFS_IOCTL_MAGIC, 4, \
+ struct btrfs_ioctl_vol_args)
+/* trans start and trans end are dangerous, and only for
+ * use by applications that know how to avoid the
+ * resulting deadlocks
+ */
+#define BTRFS_IOC_TRANS_START _IO(BTRFS_IOCTL_MAGIC, 6)
+#define BTRFS_IOC_TRANS_END _IO(BTRFS_IOCTL_MAGIC, 7)
+#define BTRFS_IOC_SYNC _IO(BTRFS_IOCTL_MAGIC, 8)
+
+#define BTRFS_IOC_CLONE _IOW(BTRFS_IOCTL_MAGIC, 9, int)
+#define BTRFS_IOC_ADD_DEV _IOW(BTRFS_IOCTL_MAGIC, 10, \
+ struct btrfs_ioctl_vol_args)
+#define BTRFS_IOC_RM_DEV _IOW(BTRFS_IOCTL_MAGIC, 11, \
+ struct btrfs_ioctl_vol_args)
+#define BTRFS_IOC_BALANCE _IOW(BTRFS_IOCTL_MAGIC, 12, \
+ struct btrfs_ioctl_vol_args)
+struct btrfs_ioctl_clone_range_args {
+ __s64 src_fd;
+ __u64 src_offset, src_length;
+ __u64 dest_offset;
+};
+
+#define BTRFS_IOC_CLONE_RANGE _IOW(BTRFS_IOCTL_MAGIC, 13, \
+ struct btrfs_ioctl_clone_range_args)
+
+#define BTRFS_IOC_SUBVOL_CREATE _IOW(BTRFS_IOCTL_MAGIC, 14, \
+ struct btrfs_ioctl_vol_args)
+
+#endif
diff --git a/fs/btrfs/locking.c b/fs/btrfs/locking.c
new file mode 100644
index 0000000..39bae77
--- /dev/null
+++ b/fs/btrfs/locking.c
@@ -0,0 +1,88 @@
+/*
+ * Copyright (C) 2008 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+#include <linux/sched.h>
+#include <linux/gfp.h>
+#include <linux/pagemap.h>
+#include <linux/spinlock.h>
+#include <linux/page-flags.h>
+#include <asm/bug.h>
+#include "ctree.h"
+#include "extent_io.h"
+#include "locking.h"
+
+/*
+ * locks the per buffer mutex in an extent buffer. This uses adaptive locks
+ * and the spin is not tuned very extensively. The spinning does make a big
+ * difference in almost every workload, but spinning for the right amount of
+ * time needs some help.
+ *
+ * In general, we want to spin as long as the lock holder is doing btree
+ * searches, and we should give up if they are in more expensive code.
+ */
+
+int btrfs_tree_lock(struct extent_buffer *eb)
+{
+ int i;
+
+ if (mutex_trylock(&eb->mutex))
+ return 0;
+ for (i = 0; i < 512; i++) {
+ cpu_relax();
+ if (mutex_trylock(&eb->mutex))
+ return 0;
+ }
+ cpu_relax();
+ mutex_lock_nested(&eb->mutex, BTRFS_MAX_LEVEL - btrfs_header_level(eb));
+ return 0;
+}
+
+int btrfs_try_tree_lock(struct extent_buffer *eb)
+{
+ return mutex_trylock(&eb->mutex);
+}
+
+int btrfs_tree_unlock(struct extent_buffer *eb)
+{
+ mutex_unlock(&eb->mutex);
+ return 0;
+}
+
+int btrfs_tree_locked(struct extent_buffer *eb)
+{
+ return mutex_is_locked(&eb->mutex);
+}
+
+/*
+ * btrfs_search_slot uses this to decide if it should drop its locks
+ * before doing something expensive like allocating free blocks for cow.
+ */
+int btrfs_path_lock_waiting(struct btrfs_path *path, int level)
+{
+ int i;
+ struct extent_buffer *eb;
+ for (i = level; i <= level + 1 && i < BTRFS_MAX_LEVEL; i++) {
+ eb = path->nodes[i];
+ if (!eb)
+ break;
+ smp_mb();
+ if (!list_empty(&eb->mutex.wait_list))
+ return 1;
+ }
+ return 0;
+}
+
diff --git a/fs/btrfs/locking.h b/fs/btrfs/locking.h
new file mode 100644
index 0000000..bc1faef
--- /dev/null
+++ b/fs/btrfs/locking.h
@@ -0,0 +1,27 @@
+/*
+ * Copyright (C) 2008 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __BTRFS_LOCKING_
+#define __BTRFS_LOCKING_
+
+int btrfs_tree_lock(struct extent_buffer *eb);
+int btrfs_tree_unlock(struct extent_buffer *eb);
+int btrfs_tree_locked(struct extent_buffer *eb);
+int btrfs_try_tree_lock(struct extent_buffer *eb);
+int btrfs_path_lock_waiting(struct btrfs_path *path, int level);
+#endif
diff --git a/fs/btrfs/ordered-data.c b/fs/btrfs/ordered-data.c
new file mode 100644
index 0000000..a209401
--- /dev/null
+++ b/fs/btrfs/ordered-data.c
@@ -0,0 +1,730 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/gfp.h>
+#include <linux/slab.h>
+#include <linux/blkdev.h>
+#include <linux/writeback.h>
+#include <linux/pagevec.h>
+#include "ctree.h"
+#include "transaction.h"
+#include "btrfs_inode.h"
+#include "extent_io.h"
+
+static u64 entry_end(struct btrfs_ordered_extent *entry)
+{
+ if (entry->file_offset + entry->len < entry->file_offset)
+ return (u64)-1;
+ return entry->file_offset + entry->len;
+}
+
+/* returns NULL if the insertion worked, or it returns the node it did find
+ * in the tree
+ */
+static struct rb_node *tree_insert(struct rb_root *root, u64 file_offset,
+ struct rb_node *node)
+{
+ struct rb_node **p = &root->rb_node;
+ struct rb_node *parent = NULL;
+ struct btrfs_ordered_extent *entry;
+
+ while (*p) {
+ parent = *p;
+ entry = rb_entry(parent, struct btrfs_ordered_extent, rb_node);
+
+ if (file_offset < entry->file_offset)
+ p = &(*p)->rb_left;
+ else if (file_offset >= entry_end(entry))
+ p = &(*p)->rb_right;
+ else
+ return parent;
+ }
+
+ rb_link_node(node, parent, p);
+ rb_insert_color(node, root);
+ return NULL;
+}
+
+/*
+ * look for a given offset in the tree, and if it can't be found return the
+ * first lesser offset
+ */
+static struct rb_node *__tree_search(struct rb_root *root, u64 file_offset,
+ struct rb_node **prev_ret)
+{
+ struct rb_node *n = root->rb_node;
+ struct rb_node *prev = NULL;
+ struct rb_node *test;
+ struct btrfs_ordered_extent *entry;
+ struct btrfs_ordered_extent *prev_entry = NULL;
+
+ while (n) {
+ entry = rb_entry(n, struct btrfs_ordered_extent, rb_node);
+ prev = n;
+ prev_entry = entry;
+
+ if (file_offset < entry->file_offset)
+ n = n->rb_left;
+ else if (file_offset >= entry_end(entry))
+ n = n->rb_right;
+ else
+ return n;
+ }
+ if (!prev_ret)
+ return NULL;
+
+ while (prev && file_offset >= entry_end(prev_entry)) {
+ test = rb_next(prev);
+ if (!test)
+ break;
+ prev_entry = rb_entry(test, struct btrfs_ordered_extent,
+ rb_node);
+ if (file_offset < entry_end(prev_entry))
+ break;
+
+ prev = test;
+ }
+ if (prev)
+ prev_entry = rb_entry(prev, struct btrfs_ordered_extent,
+ rb_node);
+ while (prev && file_offset < entry_end(prev_entry)) {
+ test = rb_prev(prev);
+ if (!test)
+ break;
+ prev_entry = rb_entry(test, struct btrfs_ordered_extent,
+ rb_node);
+ prev = test;
+ }
+ *prev_ret = prev;
+ return NULL;
+}
+
+/*
+ * helper to check if a given offset is inside a given entry
+ */
+static int offset_in_entry(struct btrfs_ordered_extent *entry, u64 file_offset)
+{
+ if (file_offset < entry->file_offset ||
+ entry->file_offset + entry->len <= file_offset)
+ return 0;
+ return 1;
+}
+
+/*
+ * look find the first ordered struct that has this offset, otherwise
+ * the first one less than this offset
+ */
+static inline struct rb_node *tree_search(struct btrfs_ordered_inode_tree *tree,
+ u64 file_offset)
+{
+ struct rb_root *root = &tree->tree;
+ struct rb_node *prev;
+ struct rb_node *ret;
+ struct btrfs_ordered_extent *entry;
+
+ if (tree->last) {
+ entry = rb_entry(tree->last, struct btrfs_ordered_extent,
+ rb_node);
+ if (offset_in_entry(entry, file_offset))
+ return tree->last;
+ }
+ ret = __tree_search(root, file_offset, &prev);
+ if (!ret)
+ ret = prev;
+ if (ret)
+ tree->last = ret;
+ return ret;
+}
+
+/* allocate and add a new ordered_extent into the per-inode tree.
+ * file_offset is the logical offset in the file
+ *
+ * start is the disk block number of an extent already reserved in the
+ * extent allocation tree
+ *
+ * len is the length of the extent
+ *
+ * This also sets the EXTENT_ORDERED bit on the range in the inode.
+ *
+ * The tree is given a single reference on the ordered extent that was
+ * inserted.
+ */
+int btrfs_add_ordered_extent(struct inode *inode, u64 file_offset,
+ u64 start, u64 len, u64 disk_len, int type)
+{
+ struct btrfs_ordered_inode_tree *tree;
+ struct rb_node *node;
+ struct btrfs_ordered_extent *entry;
+
+ tree = &BTRFS_I(inode)->ordered_tree;
+ entry = kzalloc(sizeof(*entry), GFP_NOFS);
+ if (!entry)
+ return -ENOMEM;
+
+ mutex_lock(&tree->mutex);
+ entry->file_offset = file_offset;
+ entry->start = start;
+ entry->len = len;
+ entry->disk_len = disk_len;
+ entry->inode = inode;
+ if (type != BTRFS_ORDERED_IO_DONE && type != BTRFS_ORDERED_COMPLETE)
+ set_bit(type, &entry->flags);
+
+ /* one ref for the tree */
+ atomic_set(&entry->refs, 1);
+ init_waitqueue_head(&entry->wait);
+ INIT_LIST_HEAD(&entry->list);
+ INIT_LIST_HEAD(&entry->root_extent_list);
+
+ node = tree_insert(&tree->tree, file_offset,
+ &entry->rb_node);
+ BUG_ON(node);
+
+ set_extent_ordered(&BTRFS_I(inode)->io_tree, file_offset,
+ entry_end(entry) - 1, GFP_NOFS);
+
+ spin_lock(&BTRFS_I(inode)->root->fs_info->ordered_extent_lock);
+ list_add_tail(&entry->root_extent_list,
+ &BTRFS_I(inode)->root->fs_info->ordered_extents);
+ spin_unlock(&BTRFS_I(inode)->root->fs_info->ordered_extent_lock);
+
+ mutex_unlock(&tree->mutex);
+ BUG_ON(node);
+ return 0;
+}
+
+/*
+ * Add a struct btrfs_ordered_sum into the list of checksums to be inserted
+ * when an ordered extent is finished. If the list covers more than one
+ * ordered extent, it is split across multiples.
+ */
+int btrfs_add_ordered_sum(struct inode *inode,
+ struct btrfs_ordered_extent *entry,
+ struct btrfs_ordered_sum *sum)
+{
+ struct btrfs_ordered_inode_tree *tree;
+
+ tree = &BTRFS_I(inode)->ordered_tree;
+ mutex_lock(&tree->mutex);
+ list_add_tail(&sum->list, &entry->list);
+ mutex_unlock(&tree->mutex);
+ return 0;
+}
+
+/*
+ * this is used to account for finished IO across a given range
+ * of the file. The IO should not span ordered extents. If
+ * a given ordered_extent is completely done, 1 is returned, otherwise
+ * 0.
+ *
+ * test_and_set_bit on a flag in the struct btrfs_ordered_extent is used
+ * to make sure this function only returns 1 once for a given ordered extent.
+ */
+int btrfs_dec_test_ordered_pending(struct inode *inode,
+ u64 file_offset, u64 io_size)
+{
+ struct btrfs_ordered_inode_tree *tree;
+ struct rb_node *node;
+ struct btrfs_ordered_extent *entry;
+ struct extent_io_tree *io_tree = &BTRFS_I(inode)->io_tree;
+ int ret;
+
+ tree = &BTRFS_I(inode)->ordered_tree;
+ mutex_lock(&tree->mutex);
+ clear_extent_ordered(io_tree, file_offset, file_offset + io_size - 1,
+ GFP_NOFS);
+ node = tree_search(tree, file_offset);
+ if (!node) {
+ ret = 1;
+ goto out;
+ }
+
+ entry = rb_entry(node, struct btrfs_ordered_extent, rb_node);
+ if (!offset_in_entry(entry, file_offset)) {
+ ret = 1;
+ goto out;
+ }
+
+ ret = test_range_bit(io_tree, entry->file_offset,
+ entry->file_offset + entry->len - 1,
+ EXTENT_ORDERED, 0);
+ if (ret == 0)
+ ret = test_and_set_bit(BTRFS_ORDERED_IO_DONE, &entry->flags);
+out:
+ mutex_unlock(&tree->mutex);
+ return ret == 0;
+}
+
+/*
+ * used to drop a reference on an ordered extent. This will free
+ * the extent if the last reference is dropped
+ */
+int btrfs_put_ordered_extent(struct btrfs_ordered_extent *entry)
+{
+ struct list_head *cur;
+ struct btrfs_ordered_sum *sum;
+
+ if (atomic_dec_and_test(&entry->refs)) {
+ while (!list_empty(&entry->list)) {
+ cur = entry->list.next;
+ sum = list_entry(cur, struct btrfs_ordered_sum, list);
+ list_del(&sum->list);
+ kfree(sum);
+ }
+ kfree(entry);
+ }
+ return 0;
+}
+
+/*
+ * remove an ordered extent from the tree. No references are dropped
+ * but, anyone waiting on this extent is woken up.
+ */
+int btrfs_remove_ordered_extent(struct inode *inode,
+ struct btrfs_ordered_extent *entry)
+{
+ struct btrfs_ordered_inode_tree *tree;
+ struct rb_node *node;
+
+ tree = &BTRFS_I(inode)->ordered_tree;
+ mutex_lock(&tree->mutex);
+ node = &entry->rb_node;
+ rb_erase(node, &tree->tree);
+ tree->last = NULL;
+ set_bit(BTRFS_ORDERED_COMPLETE, &entry->flags);
+
+ spin_lock(&BTRFS_I(inode)->root->fs_info->ordered_extent_lock);
+ list_del_init(&entry->root_extent_list);
+ spin_unlock(&BTRFS_I(inode)->root->fs_info->ordered_extent_lock);
+
+ mutex_unlock(&tree->mutex);
+ wake_up(&entry->wait);
+ return 0;
+}
+
+/*
+ * wait for all the ordered extents in a root. This is done when balancing
+ * space between drives.
+ */
+int btrfs_wait_ordered_extents(struct btrfs_root *root, int nocow_only)
+{
+ struct list_head splice;
+ struct list_head *cur;
+ struct btrfs_ordered_extent *ordered;
+ struct inode *inode;
+
+ INIT_LIST_HEAD(&splice);
+
+ spin_lock(&root->fs_info->ordered_extent_lock);
+ list_splice_init(&root->fs_info->ordered_extents, &splice);
+ while (!list_empty(&splice)) {
+ cur = splice.next;
+ ordered = list_entry(cur, struct btrfs_ordered_extent,
+ root_extent_list);
+ if (nocow_only &&
+ !test_bit(BTRFS_ORDERED_NOCOW, &ordered->flags) &&
+ !test_bit(BTRFS_ORDERED_PREALLOC, &ordered->flags)) {
+ list_move(&ordered->root_extent_list,
+ &root->fs_info->ordered_extents);
+ cond_resched_lock(&root->fs_info->ordered_extent_lock);
+ continue;
+ }
+
+ list_del_init(&ordered->root_extent_list);
+ atomic_inc(&ordered->refs);
+
+ /*
+ * the inode may be getting freed (in sys_unlink path).
+ */
+ inode = igrab(ordered->inode);
+
+ spin_unlock(&root->fs_info->ordered_extent_lock);
+
+ if (inode) {
+ btrfs_start_ordered_extent(inode, ordered, 1);
+ btrfs_put_ordered_extent(ordered);
+ iput(inode);
+ } else {
+ btrfs_put_ordered_extent(ordered);
+ }
+
+ spin_lock(&root->fs_info->ordered_extent_lock);
+ }
+ spin_unlock(&root->fs_info->ordered_extent_lock);
+ return 0;
+}
+
+/*
+ * Used to start IO or wait for a given ordered extent to finish.
+ *
+ * If wait is one, this effectively waits on page writeback for all the pages
+ * in the extent, and it waits on the io completion code to insert
+ * metadata into the btree corresponding to the extent
+ */
+void btrfs_start_ordered_extent(struct inode *inode,
+ struct btrfs_ordered_extent *entry,
+ int wait)
+{
+ u64 start = entry->file_offset;
+ u64 end = start + entry->len - 1;
+
+ /*
+ * pages in the range can be dirty, clean or writeback. We
+ * start IO on any dirty ones so the wait doesn't stall waiting
+ * for pdflush to find them
+ */
+ btrfs_fdatawrite_range(inode->i_mapping, start, end, WB_SYNC_ALL);
+ if (wait) {
+ wait_event(entry->wait, test_bit(BTRFS_ORDERED_COMPLETE,
+ &entry->flags));
+ }
+}
+
+/*
+ * Used to wait on ordered extents across a large range of bytes.
+ */
+int btrfs_wait_ordered_range(struct inode *inode, u64 start, u64 len)
+{
+ u64 end;
+ u64 orig_end;
+ u64 wait_end;
+ struct btrfs_ordered_extent *ordered;
+
+ if (start + len < start) {
+ orig_end = INT_LIMIT(loff_t);
+ } else {
+ orig_end = start + len - 1;
+ if (orig_end > INT_LIMIT(loff_t))
+ orig_end = INT_LIMIT(loff_t);
+ }
+ wait_end = orig_end;
+again:
+ /* start IO across the range first to instantiate any delalloc
+ * extents
+ */
+ btrfs_fdatawrite_range(inode->i_mapping, start, orig_end, WB_SYNC_NONE);
+
+ /* The compression code will leave pages locked but return from
+ * writepage without setting the page writeback. Starting again
+ * with WB_SYNC_ALL will end up waiting for the IO to actually start.
+ */
+ btrfs_fdatawrite_range(inode->i_mapping, start, orig_end, WB_SYNC_ALL);
+
+ btrfs_wait_on_page_writeback_range(inode->i_mapping,
+ start >> PAGE_CACHE_SHIFT,
+ orig_end >> PAGE_CACHE_SHIFT);
+
+ end = orig_end;
+ while (1) {
+ ordered = btrfs_lookup_first_ordered_extent(inode, end);
+ if (!ordered)
+ break;
+ if (ordered->file_offset > orig_end) {
+ btrfs_put_ordered_extent(ordered);
+ break;
+ }
+ if (ordered->file_offset + ordered->len < start) {
+ btrfs_put_ordered_extent(ordered);
+ break;
+ }
+ btrfs_start_ordered_extent(inode, ordered, 1);
+ end = ordered->file_offset;
+ btrfs_put_ordered_extent(ordered);
+ if (end == 0 || end == start)
+ break;
+ end--;
+ }
+ if (test_range_bit(&BTRFS_I(inode)->io_tree, start, orig_end,
+ EXTENT_ORDERED | EXTENT_DELALLOC, 0)) {
+ schedule_timeout(1);
+ goto again;
+ }
+ return 0;
+}
+
+/*
+ * find an ordered extent corresponding to file_offset. return NULL if
+ * nothing is found, otherwise take a reference on the extent and return it
+ */
+struct btrfs_ordered_extent *btrfs_lookup_ordered_extent(struct inode *inode,
+ u64 file_offset)
+{
+ struct btrfs_ordered_inode_tree *tree;
+ struct rb_node *node;
+ struct btrfs_ordered_extent *entry = NULL;
+
+ tree = &BTRFS_I(inode)->ordered_tree;
+ mutex_lock(&tree->mutex);
+ node = tree_search(tree, file_offset);
+ if (!node)
+ goto out;
+
+ entry = rb_entry(node, struct btrfs_ordered_extent, rb_node);
+ if (!offset_in_entry(entry, file_offset))
+ entry = NULL;
+ if (entry)
+ atomic_inc(&entry->refs);
+out:
+ mutex_unlock(&tree->mutex);
+ return entry;
+}
+
+/*
+ * lookup and return any extent before 'file_offset'. NULL is returned
+ * if none is found
+ */
+struct btrfs_ordered_extent *
+btrfs_lookup_first_ordered_extent(struct inode *inode, u64 file_offset)
+{
+ struct btrfs_ordered_inode_tree *tree;
+ struct rb_node *node;
+ struct btrfs_ordered_extent *entry = NULL;
+
+ tree = &BTRFS_I(inode)->ordered_tree;
+ mutex_lock(&tree->mutex);
+ node = tree_search(tree, file_offset);
+ if (!node)
+ goto out;
+
+ entry = rb_entry(node, struct btrfs_ordered_extent, rb_node);
+ atomic_inc(&entry->refs);
+out:
+ mutex_unlock(&tree->mutex);
+ return entry;
+}
+
+/*
+ * After an extent is done, call this to conditionally update the on disk
+ * i_size. i_size is updated to cover any fully written part of the file.
+ */
+int btrfs_ordered_update_i_size(struct inode *inode,
+ struct btrfs_ordered_extent *ordered)
+{
+ struct btrfs_ordered_inode_tree *tree = &BTRFS_I(inode)->ordered_tree;
+ struct extent_io_tree *io_tree = &BTRFS_I(inode)->io_tree;
+ u64 disk_i_size;
+ u64 new_i_size;
+ u64 i_size_test;
+ struct rb_node *node;
+ struct btrfs_ordered_extent *test;
+
+ mutex_lock(&tree->mutex);
+ disk_i_size = BTRFS_I(inode)->disk_i_size;
+
+ /*
+ * if the disk i_size is already at the inode->i_size, or
+ * this ordered extent is inside the disk i_size, we're done
+ */
+ if (disk_i_size >= inode->i_size ||
+ ordered->file_offset + ordered->len <= disk_i_size) {
+ goto out;
+ }
+
+ /*
+ * we can't update the disk_isize if there are delalloc bytes
+ * between disk_i_size and this ordered extent
+ */
+ if (test_range_bit(io_tree, disk_i_size,
+ ordered->file_offset + ordered->len - 1,
+ EXTENT_DELALLOC, 0)) {
+ goto out;
+ }
+ /*
+ * walk backward from this ordered extent to disk_i_size.
+ * if we find an ordered extent then we can't update disk i_size
+ * yet
+ */
+ node = &ordered->rb_node;
+ while (1) {
+ node = rb_prev(node);
+ if (!node)
+ break;
+ test = rb_entry(node, struct btrfs_ordered_extent, rb_node);
+ if (test->file_offset + test->len <= disk_i_size)
+ break;
+ if (test->file_offset >= inode->i_size)
+ break;
+ if (test->file_offset >= disk_i_size)
+ goto out;
+ }
+ new_i_size = min_t(u64, entry_end(ordered), i_size_read(inode));
+
+ /*
+ * at this point, we know we can safely update i_size to at least
+ * the offset from this ordered extent. But, we need to
+ * walk forward and see if ios from higher up in the file have
+ * finished.
+ */
+ node = rb_next(&ordered->rb_node);
+ i_size_test = 0;
+ if (node) {
+ /*
+ * do we have an area where IO might have finished
+ * between our ordered extent and the next one.
+ */
+ test = rb_entry(node, struct btrfs_ordered_extent, rb_node);
+ if (test->file_offset > entry_end(ordered))
+ i_size_test = test->file_offset;
+ } else {
+ i_size_test = i_size_read(inode);
+ }
+
+ /*
+ * i_size_test is the end of a region after this ordered
+ * extent where there are no ordered extents. As long as there
+ * are no delalloc bytes in this area, it is safe to update
+ * disk_i_size to the end of the region.
+ */
+ if (i_size_test > entry_end(ordered) &&
+ !test_range_bit(io_tree, entry_end(ordered), i_size_test - 1,
+ EXTENT_DELALLOC, 0)) {
+ new_i_size = min_t(u64, i_size_test, i_size_read(inode));
+ }
+ BTRFS_I(inode)->disk_i_size = new_i_size;
+out:
+ mutex_unlock(&tree->mutex);
+ return 0;
+}
+
+/*
+ * search the ordered extents for one corresponding to 'offset' and
+ * try to find a checksum. This is used because we allow pages to
+ * be reclaimed before their checksum is actually put into the btree
+ */
+int btrfs_find_ordered_sum(struct inode *inode, u64 offset, u64 disk_bytenr,
+ u32 *sum)
+{
+ struct btrfs_ordered_sum *ordered_sum;
+ struct btrfs_sector_sum *sector_sums;
+ struct btrfs_ordered_extent *ordered;
+ struct btrfs_ordered_inode_tree *tree = &BTRFS_I(inode)->ordered_tree;
+ struct list_head *cur;
+ unsigned long num_sectors;
+ unsigned long i;
+ u32 sectorsize = BTRFS_I(inode)->root->sectorsize;
+ int ret = 1;
+
+ ordered = btrfs_lookup_ordered_extent(inode, offset);
+ if (!ordered)
+ return 1;
+
+ mutex_lock(&tree->mutex);
+ list_for_each_prev(cur, &ordered->list) {
+ ordered_sum = list_entry(cur, struct btrfs_ordered_sum, list);
+ if (disk_bytenr >= ordered_sum->bytenr) {
+ num_sectors = ordered_sum->len / sectorsize;
+ sector_sums = ordered_sum->sums;
+ for (i = 0; i < num_sectors; i++) {
+ if (sector_sums[i].bytenr == disk_bytenr) {
+ *sum = sector_sums[i].sum;
+ ret = 0;
+ goto out;
+ }
+ }
+ }
+ }
+out:
+ mutex_unlock(&tree->mutex);
+ btrfs_put_ordered_extent(ordered);
+ return ret;
+}
+
+
+/**
+ * taken from mm/filemap.c because it isn't exported
+ *
+ * __filemap_fdatawrite_range - start writeback on mapping dirty pages in range
+ * @mapping: address space structure to write
+ * @start: offset in bytes where the range starts
+ * @end: offset in bytes where the range ends (inclusive)
+ * @sync_mode: enable synchronous operation
+ *
+ * Start writeback against all of a mapping's dirty pages that lie
+ * within the byte offsets <start, end> inclusive.
+ *
+ * If sync_mode is WB_SYNC_ALL then this is a "data integrity" operation, as
+ * opposed to a regular memory cleansing writeback. The difference between
+ * these two operations is that if a dirty page/buffer is encountered, it must
+ * be waited upon, and not just skipped over.
+ */
+int btrfs_fdatawrite_range(struct address_space *mapping, loff_t start,
+ loff_t end, int sync_mode)
+{
+ struct writeback_control wbc = {
+ .sync_mode = sync_mode,
+ .nr_to_write = mapping->nrpages * 2,
+ .range_start = start,
+ .range_end = end,
+ .for_writepages = 1,
+ };
+ return btrfs_writepages(mapping, &wbc);
+}
+
+/**
+ * taken from mm/filemap.c because it isn't exported
+ *
+ * wait_on_page_writeback_range - wait for writeback to complete
+ * @mapping: target address_space
+ * @start: beginning page index
+ * @end: ending page index
+ *
+ * Wait for writeback to complete against pages indexed by start->end
+ * inclusive
+ */
+int btrfs_wait_on_page_writeback_range(struct address_space *mapping,
+ pgoff_t start, pgoff_t end)
+{
+ struct pagevec pvec;
+ int nr_pages;
+ int ret = 0;
+ pgoff_t index;
+
+ if (end < start)
+ return 0;
+
+ pagevec_init(&pvec, 0);
+ index = start;
+ while ((index <= end) &&
+ (nr_pages = pagevec_lookup_tag(&pvec, mapping, &index,
+ PAGECACHE_TAG_WRITEBACK,
+ min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1)) != 0) {
+ unsigned i;
+
+ for (i = 0; i < nr_pages; i++) {
+ struct page *page = pvec.pages[i];
+
+ /* until radix tree lookup accepts end_index */
+ if (page->index > end)
+ continue;
+
+ wait_on_page_writeback(page);
+ if (PageError(page))
+ ret = -EIO;
+ }
+ pagevec_release(&pvec);
+ cond_resched();
+ }
+
+ /* Check for outstanding write errors */
+ if (test_and_clear_bit(AS_ENOSPC, &mapping->flags))
+ ret = -ENOSPC;
+ if (test_and_clear_bit(AS_EIO, &mapping->flags))
+ ret = -EIO;
+
+ return ret;
+}
diff --git a/fs/btrfs/ordered-data.h b/fs/btrfs/ordered-data.h
new file mode 100644
index 0000000..ab66d5e
--- /dev/null
+++ b/fs/btrfs/ordered-data.h
@@ -0,0 +1,158 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __BTRFS_ORDERED_DATA__
+#define __BTRFS_ORDERED_DATA__
+
+/* one of these per inode */
+struct btrfs_ordered_inode_tree {
+ struct mutex mutex;
+ struct rb_root tree;
+ struct rb_node *last;
+};
+
+/*
+ * these are used to collect checksums done just before bios submission.
+ * They are attached via a list into the ordered extent, and
+ * checksum items are inserted into the tree after all the blocks in
+ * the ordered extent are on disk
+ */
+struct btrfs_sector_sum {
+ /* bytenr on disk */
+ u64 bytenr;
+ u32 sum;
+};
+
+struct btrfs_ordered_sum {
+ /* bytenr is the start of this extent on disk */
+ u64 bytenr;
+
+ /*
+ * this is the length in bytes covered by the sums array below.
+ */
+ unsigned long len;
+ struct list_head list;
+ /* last field is a variable length array of btrfs_sector_sums */
+ struct btrfs_sector_sum sums[];
+};
+
+/*
+ * bits for the flags field:
+ *
+ * BTRFS_ORDERED_IO_DONE is set when all of the blocks are written.
+ * It is used to make sure metadata is inserted into the tree only once
+ * per extent.
+ *
+ * BTRFS_ORDERED_COMPLETE is set when the extent is removed from the
+ * rbtree, just before waking any waiters. It is used to indicate the
+ * IO is done and any metadata is inserted into the tree.
+ */
+#define BTRFS_ORDERED_IO_DONE 0 /* set when all the pages are written */
+
+#define BTRFS_ORDERED_COMPLETE 1 /* set when removed from the tree */
+
+#define BTRFS_ORDERED_NOCOW 2 /* set when we want to write in place */
+
+#define BTRFS_ORDERED_COMPRESSED 3 /* writing a compressed extent */
+
+#define BTRFS_ORDERED_PREALLOC 4 /* set when writing to prealloced extent */
+
+struct btrfs_ordered_extent {
+ /* logical offset in the file */
+ u64 file_offset;
+
+ /* disk byte number */
+ u64 start;
+
+ /* ram length of the extent in bytes */
+ u64 len;
+
+ /* extent length on disk */
+ u64 disk_len;
+
+ /* flags (described above) */
+ unsigned long flags;
+
+ /* reference count */
+ atomic_t refs;
+
+ /* the inode we belong to */
+ struct inode *inode;
+
+ /* list of checksums for insertion when the extent io is done */
+ struct list_head list;
+
+ /* used to wait for the BTRFS_ORDERED_COMPLETE bit */
+ wait_queue_head_t wait;
+
+ /* our friendly rbtree entry */
+ struct rb_node rb_node;
+
+ /* a per root list of all the pending ordered extents */
+ struct list_head root_extent_list;
+};
+
+
+/*
+ * calculates the total size you need to allocate for an ordered sum
+ * structure spanning 'bytes' in the file
+ */
+static inline int btrfs_ordered_sum_size(struct btrfs_root *root,
+ unsigned long bytes)
+{
+ unsigned long num_sectors = (bytes + root->sectorsize - 1) /
+ root->sectorsize;
+ num_sectors++;
+ return sizeof(struct btrfs_ordered_sum) +
+ num_sectors * sizeof(struct btrfs_sector_sum);
+}
+
+static inline void
+btrfs_ordered_inode_tree_init(struct btrfs_ordered_inode_tree *t)
+{
+ mutex_init(&t->mutex);
+ t->tree.rb_node = NULL;
+ t->last = NULL;
+}
+
+int btrfs_put_ordered_extent(struct btrfs_ordered_extent *entry);
+int btrfs_remove_ordered_extent(struct inode *inode,
+ struct btrfs_ordered_extent *entry);
+int btrfs_dec_test_ordered_pending(struct inode *inode,
+ u64 file_offset, u64 io_size);
+int btrfs_add_ordered_extent(struct inode *inode, u64 file_offset,
+ u64 start, u64 len, u64 disk_len, int tyep);
+int btrfs_add_ordered_sum(struct inode *inode,
+ struct btrfs_ordered_extent *entry,
+ struct btrfs_ordered_sum *sum);
+struct btrfs_ordered_extent *btrfs_lookup_ordered_extent(struct inode *inode,
+ u64 file_offset);
+void btrfs_start_ordered_extent(struct inode *inode,
+ struct btrfs_ordered_extent *entry, int wait);
+int btrfs_wait_ordered_range(struct inode *inode, u64 start, u64 len);
+struct btrfs_ordered_extent *
+btrfs_lookup_first_ordered_extent(struct inode * inode, u64 file_offset);
+int btrfs_ordered_update_i_size(struct inode *inode,
+ struct btrfs_ordered_extent *ordered);
+int btrfs_find_ordered_sum(struct inode *inode, u64 offset, u64 disk_bytenr, u32 *sum);
+int btrfs_wait_on_page_writeback_range(struct address_space *mapping,
+ pgoff_t start, pgoff_t end);
+int btrfs_fdatawrite_range(struct address_space *mapping, loff_t start,
+ loff_t end, int sync_mode);
+int btrfs_wait_ordered_extents(struct btrfs_root *root, int nocow_only);
+#endif
diff --git a/fs/btrfs/orphan.c b/fs/btrfs/orphan.c
new file mode 100644
index 0000000..3c0d52a
--- /dev/null
+++ b/fs/btrfs/orphan.c
@@ -0,0 +1,67 @@
+/*
+ * Copyright (C) 2008 Red Hat. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include "ctree.h"
+#include "disk-io.h"
+
+int btrfs_insert_orphan_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 offset)
+{
+ struct btrfs_path *path;
+ struct btrfs_key key;
+ int ret = 0;
+
+ key.objectid = BTRFS_ORPHAN_OBJECTID;
+ btrfs_set_key_type(&key, BTRFS_ORPHAN_ITEM_KEY);
+ key.offset = offset;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ ret = btrfs_insert_empty_item(trans, root, path, &key, 0);
+
+ btrfs_free_path(path);
+ return ret;
+}
+
+int btrfs_del_orphan_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 offset)
+{
+ struct btrfs_path *path;
+ struct btrfs_key key;
+ int ret = 0;
+
+ key.objectid = BTRFS_ORPHAN_OBJECTID;
+ btrfs_set_key_type(&key, BTRFS_ORPHAN_ITEM_KEY);
+ key.offset = offset;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
+ if (ret)
+ goto out;
+
+ ret = btrfs_del_item(trans, root, path);
+
+out:
+ btrfs_free_path(path);
+ return ret;
+}
diff --git a/fs/btrfs/print-tree.c b/fs/btrfs/print-tree.c
new file mode 100644
index 0000000..5f8f218
--- /dev/null
+++ b/fs/btrfs/print-tree.c
@@ -0,0 +1,216 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include "ctree.h"
+#include "disk-io.h"
+#include "print-tree.h"
+
+static void print_chunk(struct extent_buffer *eb, struct btrfs_chunk *chunk)
+{
+ int num_stripes = btrfs_chunk_num_stripes(eb, chunk);
+ int i;
+ printk(KERN_INFO "\t\tchunk length %llu owner %llu type %llu "
+ "num_stripes %d\n",
+ (unsigned long long)btrfs_chunk_length(eb, chunk),
+ (unsigned long long)btrfs_chunk_owner(eb, chunk),
+ (unsigned long long)btrfs_chunk_type(eb, chunk),
+ num_stripes);
+ for (i = 0 ; i < num_stripes ; i++) {
+ printk(KERN_INFO "\t\t\tstripe %d devid %llu offset %llu\n", i,
+ (unsigned long long)btrfs_stripe_devid_nr(eb, chunk, i),
+ (unsigned long long)btrfs_stripe_offset_nr(eb, chunk, i));
+ }
+}
+static void print_dev_item(struct extent_buffer *eb,
+ struct btrfs_dev_item *dev_item)
+{
+ printk(KERN_INFO "\t\tdev item devid %llu "
+ "total_bytes %llu bytes used %llu\n",
+ (unsigned long long)btrfs_device_id(eb, dev_item),
+ (unsigned long long)btrfs_device_total_bytes(eb, dev_item),
+ (unsigned long long)btrfs_device_bytes_used(eb, dev_item));
+}
+void btrfs_print_leaf(struct btrfs_root *root, struct extent_buffer *l)
+{
+ int i;
+ u32 nr = btrfs_header_nritems(l);
+ struct btrfs_item *item;
+ struct btrfs_extent_item *ei;
+ struct btrfs_root_item *ri;
+ struct btrfs_dir_item *di;
+ struct btrfs_inode_item *ii;
+ struct btrfs_block_group_item *bi;
+ struct btrfs_file_extent_item *fi;
+ struct btrfs_key key;
+ struct btrfs_key found_key;
+ struct btrfs_extent_ref *ref;
+ struct btrfs_dev_extent *dev_extent;
+ u32 type;
+
+ printk(KERN_INFO "leaf %llu total ptrs %d free space %d\n",
+ (unsigned long long)btrfs_header_bytenr(l), nr,
+ btrfs_leaf_free_space(root, l));
+ for (i = 0 ; i < nr ; i++) {
+ item = btrfs_item_nr(l, i);
+ btrfs_item_key_to_cpu(l, &key, i);
+ type = btrfs_key_type(&key);
+ printk(KERN_INFO "\titem %d key (%llu %x %llu) itemoff %d "
+ "itemsize %d\n",
+ i,
+ (unsigned long long)key.objectid, type,
+ (unsigned long long)key.offset,
+ btrfs_item_offset(l, item), btrfs_item_size(l, item));
+ switch (type) {
+ case BTRFS_INODE_ITEM_KEY:
+ ii = btrfs_item_ptr(l, i, struct btrfs_inode_item);
+ printk(KERN_INFO "\t\tinode generation %llu size %llu "
+ "mode %o\n",
+ (unsigned long long)
+ btrfs_inode_generation(l, ii),
+ (unsigned long long)btrfs_inode_size(l, ii),
+ btrfs_inode_mode(l, ii));
+ break;
+ case BTRFS_DIR_ITEM_KEY:
+ di = btrfs_item_ptr(l, i, struct btrfs_dir_item);
+ btrfs_dir_item_key_to_cpu(l, di, &found_key);
+ printk(KERN_INFO "\t\tdir oid %llu type %u\n",
+ (unsigned long long)found_key.objectid,
+ btrfs_dir_type(l, di));
+ break;
+ case BTRFS_ROOT_ITEM_KEY:
+ ri = btrfs_item_ptr(l, i, struct btrfs_root_item);
+ printk(KERN_INFO "\t\troot data bytenr %llu refs %u\n",
+ (unsigned long long)
+ btrfs_disk_root_bytenr(l, ri),
+ btrfs_disk_root_refs(l, ri));
+ break;
+ case BTRFS_EXTENT_ITEM_KEY:
+ ei = btrfs_item_ptr(l, i, struct btrfs_extent_item);
+ printk(KERN_INFO "\t\textent data refs %u\n",
+ btrfs_extent_refs(l, ei));
+ break;
+ case BTRFS_EXTENT_REF_KEY:
+ ref = btrfs_item_ptr(l, i, struct btrfs_extent_ref);
+ printk(KERN_INFO "\t\textent back ref root %llu "
+ "gen %llu owner %llu num_refs %lu\n",
+ (unsigned long long)btrfs_ref_root(l, ref),
+ (unsigned long long)btrfs_ref_generation(l, ref),
+ (unsigned long long)btrfs_ref_objectid(l, ref),
+ (unsigned long)btrfs_ref_num_refs(l, ref));
+ break;
+
+ case BTRFS_EXTENT_DATA_KEY:
+ fi = btrfs_item_ptr(l, i,
+ struct btrfs_file_extent_item);
+ if (btrfs_file_extent_type(l, fi) ==
+ BTRFS_FILE_EXTENT_INLINE) {
+ printk(KERN_INFO "\t\tinline extent data "
+ "size %u\n",
+ btrfs_file_extent_inline_len(l, fi));
+ break;
+ }
+ printk(KERN_INFO "\t\textent data disk bytenr %llu "
+ "nr %llu\n",
+ (unsigned long long)
+ btrfs_file_extent_disk_bytenr(l, fi),
+ (unsigned long long)
+ btrfs_file_extent_disk_num_bytes(l, fi));
+ printk(KERN_INFO "\t\textent data offset %llu "
+ "nr %llu ram %llu\n",
+ (unsigned long long)
+ btrfs_file_extent_offset(l, fi),
+ (unsigned long long)
+ btrfs_file_extent_num_bytes(l, fi),
+ (unsigned long long)
+ btrfs_file_extent_ram_bytes(l, fi));
+ break;
+ case BTRFS_BLOCK_GROUP_ITEM_KEY:
+ bi = btrfs_item_ptr(l, i,
+ struct btrfs_block_group_item);
+ printk(KERN_INFO "\t\tblock group used %llu\n",
+ (unsigned long long)
+ btrfs_disk_block_group_used(l, bi));
+ break;
+ case BTRFS_CHUNK_ITEM_KEY:
+ print_chunk(l, btrfs_item_ptr(l, i,
+ struct btrfs_chunk));
+ break;
+ case BTRFS_DEV_ITEM_KEY:
+ print_dev_item(l, btrfs_item_ptr(l, i,
+ struct btrfs_dev_item));
+ break;
+ case BTRFS_DEV_EXTENT_KEY:
+ dev_extent = btrfs_item_ptr(l, i,
+ struct btrfs_dev_extent);
+ printk(KERN_INFO "\t\tdev extent chunk_tree %llu\n"
+ "\t\tchunk objectid %llu chunk offset %llu "
+ "length %llu\n",
+ (unsigned long long)
+ btrfs_dev_extent_chunk_tree(l, dev_extent),
+ (unsigned long long)
+ btrfs_dev_extent_chunk_objectid(l, dev_extent),
+ (unsigned long long)
+ btrfs_dev_extent_chunk_offset(l, dev_extent),
+ (unsigned long long)
+ btrfs_dev_extent_length(l, dev_extent));
+ };
+ }
+}
+
+void btrfs_print_tree(struct btrfs_root *root, struct extent_buffer *c)
+{
+ int i; u32 nr;
+ struct btrfs_key key;
+ int level;
+
+ if (!c)
+ return;
+ nr = btrfs_header_nritems(c);
+ level = btrfs_header_level(c);
+ if (level == 0) {
+ btrfs_print_leaf(root, c);
+ return;
+ }
+ printk(KERN_INFO "node %llu level %d total ptrs %d free spc %u\n",
+ (unsigned long long)btrfs_header_bytenr(c),
+ btrfs_header_level(c), nr,
+ (u32)BTRFS_NODEPTRS_PER_BLOCK(root) - nr);
+ for (i = 0; i < nr; i++) {
+ btrfs_node_key_to_cpu(c, &key, i);
+ printk(KERN_INFO "\tkey %d (%llu %u %llu) block %llu\n",
+ i,
+ (unsigned long long)key.objectid,
+ key.type,
+ (unsigned long long)key.offset,
+ (unsigned long long)btrfs_node_blockptr(c, i));
+ }
+ for (i = 0; i < nr; i++) {
+ struct extent_buffer *next = read_tree_block(root,
+ btrfs_node_blockptr(c, i),
+ btrfs_level_size(root, level - 1),
+ btrfs_node_ptr_generation(c, i));
+ if (btrfs_is_leaf(next) &&
+ btrfs_header_level(c) != 1)
+ BUG();
+ if (btrfs_header_level(next) !=
+ btrfs_header_level(c) - 1)
+ BUG();
+ btrfs_print_tree(root, next);
+ free_extent_buffer(next);
+ }
+}
diff --git a/fs/btrfs/print-tree.h b/fs/btrfs/print-tree.h
new file mode 100644
index 0000000..da75efe
--- /dev/null
+++ b/fs/btrfs/print-tree.h
@@ -0,0 +1,23 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __PRINT_TREE_
+#define __PRINT_TREE_
+void btrfs_print_leaf(struct btrfs_root *root, struct extent_buffer *l);
+void btrfs_print_tree(struct btrfs_root *root, struct extent_buffer *t);
+#endif
diff --git a/fs/btrfs/ref-cache.c b/fs/btrfs/ref-cache.c
new file mode 100644
index 0000000..6f0acc4
--- /dev/null
+++ b/fs/btrfs/ref-cache.c
@@ -0,0 +1,230 @@
+/*
+ * Copyright (C) 2008 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/sched.h>
+#include "ctree.h"
+#include "ref-cache.h"
+#include "transaction.h"
+
+/*
+ * leaf refs are used to cache the information about which extents
+ * a given leaf has references on. This allows us to process that leaf
+ * in btrfs_drop_snapshot without needing to read it back from disk.
+ */
+
+/*
+ * kmalloc a leaf reference struct and update the counters for the
+ * total ref cache size
+ */
+struct btrfs_leaf_ref *btrfs_alloc_leaf_ref(struct btrfs_root *root,
+ int nr_extents)
+{
+ struct btrfs_leaf_ref *ref;
+ size_t size = btrfs_leaf_ref_size(nr_extents);
+
+ ref = kmalloc(size, GFP_NOFS);
+ if (ref) {
+ spin_lock(&root->fs_info->ref_cache_lock);
+ root->fs_info->total_ref_cache_size += size;
+ spin_unlock(&root->fs_info->ref_cache_lock);
+
+ memset(ref, 0, sizeof(*ref));
+ atomic_set(&ref->usage, 1);
+ INIT_LIST_HEAD(&ref->list);
+ }
+ return ref;
+}
+
+/*
+ * free a leaf reference struct and update the counters for the
+ * total ref cache size
+ */
+void btrfs_free_leaf_ref(struct btrfs_root *root, struct btrfs_leaf_ref *ref)
+{
+ if (!ref)
+ return;
+ WARN_ON(atomic_read(&ref->usage) == 0);
+ if (atomic_dec_and_test(&ref->usage)) {
+ size_t size = btrfs_leaf_ref_size(ref->nritems);
+
+ BUG_ON(ref->in_tree);
+ kfree(ref);
+
+ spin_lock(&root->fs_info->ref_cache_lock);
+ root->fs_info->total_ref_cache_size -= size;
+ spin_unlock(&root->fs_info->ref_cache_lock);
+ }
+}
+
+static struct rb_node *tree_insert(struct rb_root *root, u64 bytenr,
+ struct rb_node *node)
+{
+ struct rb_node **p = &root->rb_node;
+ struct rb_node *parent = NULL;
+ struct btrfs_leaf_ref *entry;
+
+ while (*p) {
+ parent = *p;
+ entry = rb_entry(parent, struct btrfs_leaf_ref, rb_node);
+
+ if (bytenr < entry->bytenr)
+ p = &(*p)->rb_left;
+ else if (bytenr > entry->bytenr)
+ p = &(*p)->rb_right;
+ else
+ return parent;
+ }
+
+ entry = rb_entry(node, struct btrfs_leaf_ref, rb_node);
+ rb_link_node(node, parent, p);
+ rb_insert_color(node, root);
+ return NULL;
+}
+
+static struct rb_node *tree_search(struct rb_root *root, u64 bytenr)
+{
+ struct rb_node *n = root->rb_node;
+ struct btrfs_leaf_ref *entry;
+
+ while (n) {
+ entry = rb_entry(n, struct btrfs_leaf_ref, rb_node);
+ WARN_ON(!entry->in_tree);
+
+ if (bytenr < entry->bytenr)
+ n = n->rb_left;
+ else if (bytenr > entry->bytenr)
+ n = n->rb_right;
+ else
+ return n;
+ }
+ return NULL;
+}
+
+int btrfs_remove_leaf_refs(struct btrfs_root *root, u64 max_root_gen,
+ int shared)
+{
+ struct btrfs_leaf_ref *ref = NULL;
+ struct btrfs_leaf_ref_tree *tree = root->ref_tree;
+
+ if (shared)
+ tree = &root->fs_info->shared_ref_tree;
+ if (!tree)
+ return 0;
+
+ spin_lock(&tree->lock);
+ while (!list_empty(&tree->list)) {
+ ref = list_entry(tree->list.next, struct btrfs_leaf_ref, list);
+ BUG_ON(ref->tree != tree);
+ if (ref->root_gen > max_root_gen)
+ break;
+ if (!xchg(&ref->in_tree, 0)) {
+ cond_resched_lock(&tree->lock);
+ continue;
+ }
+
+ rb_erase(&ref->rb_node, &tree->root);
+ list_del_init(&ref->list);
+
+ spin_unlock(&tree->lock);
+ btrfs_free_leaf_ref(root, ref);
+ cond_resched();
+ spin_lock(&tree->lock);
+ }
+ spin_unlock(&tree->lock);
+ return 0;
+}
+
+/*
+ * find the leaf ref for a given extent. This returns the ref struct with
+ * a usage reference incremented
+ */
+struct btrfs_leaf_ref *btrfs_lookup_leaf_ref(struct btrfs_root *root,
+ u64 bytenr)
+{
+ struct rb_node *rb;
+ struct btrfs_leaf_ref *ref = NULL;
+ struct btrfs_leaf_ref_tree *tree = root->ref_tree;
+again:
+ if (tree) {
+ spin_lock(&tree->lock);
+ rb = tree_search(&tree->root, bytenr);
+ if (rb)
+ ref = rb_entry(rb, struct btrfs_leaf_ref, rb_node);
+ if (ref)
+ atomic_inc(&ref->usage);
+ spin_unlock(&tree->lock);
+ if (ref)
+ return ref;
+ }
+ if (tree != &root->fs_info->shared_ref_tree) {
+ tree = &root->fs_info->shared_ref_tree;
+ goto again;
+ }
+ return NULL;
+}
+
+/*
+ * add a fully filled in leaf ref struct
+ * remove all the refs older than a given root generation
+ */
+int btrfs_add_leaf_ref(struct btrfs_root *root, struct btrfs_leaf_ref *ref,
+ int shared)
+{
+ int ret = 0;
+ struct rb_node *rb;
+ struct btrfs_leaf_ref_tree *tree = root->ref_tree;
+
+ if (shared)
+ tree = &root->fs_info->shared_ref_tree;
+
+ spin_lock(&tree->lock);
+ rb = tree_insert(&tree->root, ref->bytenr, &ref->rb_node);
+ if (rb) {
+ ret = -EEXIST;
+ } else {
+ atomic_inc(&ref->usage);
+ ref->tree = tree;
+ ref->in_tree = 1;
+ list_add_tail(&ref->list, &tree->list);
+ }
+ spin_unlock(&tree->lock);
+ return ret;
+}
+
+/*
+ * remove a single leaf ref from the tree. This drops the ref held by the tree
+ * only
+ */
+int btrfs_remove_leaf_ref(struct btrfs_root *root, struct btrfs_leaf_ref *ref)
+{
+ struct btrfs_leaf_ref_tree *tree;
+
+ if (!xchg(&ref->in_tree, 0))
+ return 0;
+
+ tree = ref->tree;
+ spin_lock(&tree->lock);
+
+ rb_erase(&ref->rb_node, &tree->root);
+ list_del_init(&ref->list);
+
+ spin_unlock(&tree->lock);
+
+ btrfs_free_leaf_ref(root, ref);
+ return 0;
+}
diff --git a/fs/btrfs/ref-cache.h b/fs/btrfs/ref-cache.h
new file mode 100644
index 0000000..16f3183
--- /dev/null
+++ b/fs/btrfs/ref-cache.h
@@ -0,0 +1,77 @@
+/*
+ * Copyright (C) 2008 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+#ifndef __REFCACHE__
+#define __REFCACHE__
+
+struct btrfs_extent_info {
+ /* bytenr and num_bytes find the extent in the extent allocation tree */
+ u64 bytenr;
+ u64 num_bytes;
+
+ /* objectid and offset find the back reference for the file */
+ u64 objectid;
+ u64 offset;
+};
+
+struct btrfs_leaf_ref {
+ struct rb_node rb_node;
+ struct btrfs_leaf_ref_tree *tree;
+ int in_tree;
+ atomic_t usage;
+
+ u64 root_gen;
+ u64 bytenr;
+ u64 owner;
+ u64 generation;
+ int nritems;
+
+ struct list_head list;
+ struct btrfs_extent_info extents[];
+};
+
+static inline size_t btrfs_leaf_ref_size(int nr_extents)
+{
+ return sizeof(struct btrfs_leaf_ref) +
+ sizeof(struct btrfs_extent_info) * nr_extents;
+}
+
+static inline void btrfs_leaf_ref_tree_init(struct btrfs_leaf_ref_tree *tree)
+{
+ tree->root.rb_node = NULL;
+ INIT_LIST_HEAD(&tree->list);
+ spin_lock_init(&tree->lock);
+}
+
+static inline int btrfs_leaf_ref_tree_empty(struct btrfs_leaf_ref_tree *tree)
+{
+ return RB_EMPTY_ROOT(&tree->root);
+}
+
+void btrfs_leaf_ref_tree_init(struct btrfs_leaf_ref_tree *tree);
+struct btrfs_leaf_ref *btrfs_alloc_leaf_ref(struct btrfs_root *root,
+ int nr_extents);
+void btrfs_free_leaf_ref(struct btrfs_root *root, struct btrfs_leaf_ref *ref);
+struct btrfs_leaf_ref *btrfs_lookup_leaf_ref(struct btrfs_root *root,
+ u64 bytenr);
+int btrfs_add_leaf_ref(struct btrfs_root *root, struct btrfs_leaf_ref *ref,
+ int shared);
+int btrfs_remove_leaf_refs(struct btrfs_root *root, u64 max_root_gen,
+ int shared);
+int btrfs_remove_leaf_ref(struct btrfs_root *root, struct btrfs_leaf_ref *ref);
+
+#endif
diff --git a/fs/btrfs/root-tree.c b/fs/btrfs/root-tree.c
new file mode 100644
index 0000000..b48650d
--- /dev/null
+++ b/fs/btrfs/root-tree.c
@@ -0,0 +1,366 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include "ctree.h"
+#include "transaction.h"
+#include "disk-io.h"
+#include "print-tree.h"
+
+/*
+ * search forward for a root, starting with objectid 'search_start'
+ * if a root key is found, the objectid we find is filled into 'found_objectid'
+ * and 0 is returned. < 0 is returned on error, 1 if there is nothing
+ * left in the tree.
+ */
+int btrfs_search_root(struct btrfs_root *root, u64 search_start,
+ u64 *found_objectid)
+{
+ struct btrfs_path *path;
+ struct btrfs_key search_key;
+ int ret;
+
+ root = root->fs_info->tree_root;
+ search_key.objectid = search_start;
+ search_key.type = (u8)-1;
+ search_key.offset = (u64)-1;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+again:
+ ret = btrfs_search_slot(NULL, root, &search_key, path, 0, 0);
+ if (ret < 0)
+ goto out;
+ if (ret == 0) {
+ ret = 1;
+ goto out;
+ }
+ if (path->slots[0] >= btrfs_header_nritems(path->nodes[0])) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret)
+ goto out;
+ }
+ btrfs_item_key_to_cpu(path->nodes[0], &search_key, path->slots[0]);
+ if (search_key.type != BTRFS_ROOT_ITEM_KEY) {
+ search_key.offset++;
+ btrfs_release_path(root, path);
+ goto again;
+ }
+ ret = 0;
+ *found_objectid = search_key.objectid;
+
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
+/*
+ * lookup the root with the highest offset for a given objectid. The key we do
+ * find is copied into 'key'. If we find something return 0, otherwise 1, < 0
+ * on error.
+ */
+int btrfs_find_last_root(struct btrfs_root *root, u64 objectid,
+ struct btrfs_root_item *item, struct btrfs_key *key)
+{
+ struct btrfs_path *path;
+ struct btrfs_key search_key;
+ struct btrfs_key found_key;
+ struct extent_buffer *l;
+ int ret;
+ int slot;
+
+ search_key.objectid = objectid;
+ search_key.type = BTRFS_ROOT_ITEM_KEY;
+ search_key.offset = (u64)-1;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+ ret = btrfs_search_slot(NULL, root, &search_key, path, 0, 0);
+ if (ret < 0)
+ goto out;
+
+ BUG_ON(ret == 0);
+ l = path->nodes[0];
+ BUG_ON(path->slots[0] == 0);
+ slot = path->slots[0] - 1;
+ btrfs_item_key_to_cpu(l, &found_key, slot);
+ if (found_key.objectid != objectid) {
+ ret = 1;
+ goto out;
+ }
+ read_extent_buffer(l, item, btrfs_item_ptr_offset(l, slot),
+ sizeof(*item));
+ memcpy(key, &found_key, sizeof(found_key));
+ ret = 0;
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
+/*
+ * copy the data in 'item' into the btree
+ */
+int btrfs_update_root(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root, struct btrfs_key *key, struct btrfs_root_item
+ *item)
+{
+ struct btrfs_path *path;
+ struct extent_buffer *l;
+ int ret;
+ int slot;
+ unsigned long ptr;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+ ret = btrfs_search_slot(trans, root, key, path, 0, 1);
+ if (ret < 0)
+ goto out;
+
+ if (ret != 0) {
+ btrfs_print_leaf(root, path->nodes[0]);
+ printk(KERN_CRIT "unable to update root key %llu %u %llu\n",
+ (unsigned long long)key->objectid, key->type,
+ (unsigned long long)key->offset);
+ BUG_ON(1);
+ }
+
+ l = path->nodes[0];
+ slot = path->slots[0];
+ ptr = btrfs_item_ptr_offset(l, slot);
+ write_extent_buffer(l, item, ptr, sizeof(*item));
+ btrfs_mark_buffer_dirty(path->nodes[0]);
+out:
+ btrfs_release_path(root, path);
+ btrfs_free_path(path);
+ return ret;
+}
+
+int btrfs_insert_root(struct btrfs_trans_handle *trans, struct btrfs_root
+ *root, struct btrfs_key *key, struct btrfs_root_item
+ *item)
+{
+ int ret;
+ ret = btrfs_insert_item(trans, root, key, item, sizeof(*item));
+ return ret;
+}
+
+/*
+ * at mount time we want to find all the old transaction snapshots that were in
+ * the process of being deleted if we crashed. This is any root item with an
+ * offset lower than the latest root. They need to be queued for deletion to
+ * finish what was happening when we crashed.
+ */
+int btrfs_find_dead_roots(struct btrfs_root *root, u64 objectid,
+ struct btrfs_root *latest)
+{
+ struct btrfs_root *dead_root;
+ struct btrfs_item *item;
+ struct btrfs_root_item *ri;
+ struct btrfs_key key;
+ struct btrfs_key found_key;
+ struct btrfs_path *path;
+ int ret;
+ u32 nritems;
+ struct extent_buffer *leaf;
+ int slot;
+
+ key.objectid = objectid;
+ btrfs_set_key_type(&key, BTRFS_ROOT_ITEM_KEY);
+ key.offset = 0;
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+again:
+ ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ if (ret < 0)
+ goto err;
+ while (1) {
+ leaf = path->nodes[0];
+ nritems = btrfs_header_nritems(leaf);
+ slot = path->slots[0];
+ if (slot >= nritems) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret)
+ break;
+ leaf = path->nodes[0];
+ nritems = btrfs_header_nritems(leaf);
+ slot = path->slots[0];
+ }
+ item = btrfs_item_nr(leaf, slot);
+ btrfs_item_key_to_cpu(leaf, &key, slot);
+ if (btrfs_key_type(&key) != BTRFS_ROOT_ITEM_KEY)
+ goto next;
+
+ if (key.objectid < objectid)
+ goto next;
+
+ if (key.objectid > objectid)
+ break;
+
+ ri = btrfs_item_ptr(leaf, slot, struct btrfs_root_item);
+ if (btrfs_disk_root_refs(leaf, ri) != 0)
+ goto next;
+
+ memcpy(&found_key, &key, sizeof(key));
+ key.offset++;
+ btrfs_release_path(root, path);
+ dead_root =
+ btrfs_read_fs_root_no_radix(root->fs_info->tree_root,
+ &found_key);
+ if (IS_ERR(dead_root)) {
+ ret = PTR_ERR(dead_root);
+ goto err;
+ }
+
+ if (objectid == BTRFS_TREE_RELOC_OBJECTID)
+ ret = btrfs_add_dead_reloc_root(dead_root);
+ else
+ ret = btrfs_add_dead_root(dead_root, latest);
+ if (ret)
+ goto err;
+ goto again;
+next:
+ slot++;
+ path->slots[0]++;
+ }
+ ret = 0;
+err:
+ btrfs_free_path(path);
+ return ret;
+}
+
+/* drop the root item for 'key' from 'root' */
+int btrfs_del_root(struct btrfs_trans_handle *trans, struct btrfs_root *root,
+ struct btrfs_key *key)
+{
+ struct btrfs_path *path;
+ int ret;
+ u32 refs;
+ struct btrfs_root_item *ri;
+ struct extent_buffer *leaf;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+ ret = btrfs_search_slot(trans, root, key, path, -1, 1);
+ if (ret < 0)
+ goto out;
+
+ BUG_ON(ret != 0);
+ leaf = path->nodes[0];
+ ri = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_root_item);
+
+ refs = btrfs_disk_root_refs(leaf, ri);
+ BUG_ON(refs != 0);
+ ret = btrfs_del_item(trans, root, path);
+out:
+ btrfs_release_path(root, path);
+ btrfs_free_path(path);
+ return ret;
+}
+
+#if 0 /* this will get used when snapshot deletion is implemented */
+int btrfs_del_root_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *tree_root,
+ u64 root_id, u8 type, u64 ref_id)
+{
+ struct btrfs_key key;
+ int ret;
+ struct btrfs_path *path;
+
+ path = btrfs_alloc_path();
+
+ key.objectid = root_id;
+ key.type = type;
+ key.offset = ref_id;
+
+ ret = btrfs_search_slot(trans, tree_root, &key, path, -1, 1);
+ BUG_ON(ret);
+
+ ret = btrfs_del_item(trans, tree_root, path);
+ BUG_ON(ret);
+
+ btrfs_free_path(path);
+ return ret;
+}
+#endif
+
+int btrfs_find_root_ref(struct btrfs_root *tree_root,
+ struct btrfs_path *path,
+ u64 root_id, u64 ref_id)
+{
+ struct btrfs_key key;
+ int ret;
+
+ key.objectid = root_id;
+ key.type = BTRFS_ROOT_REF_KEY;
+ key.offset = ref_id;
+
+ ret = btrfs_search_slot(NULL, tree_root, &key, path, 0, 0);
+ return ret;
+}
+
+
+/*
+ * add a btrfs_root_ref item. type is either BTRFS_ROOT_REF_KEY
+ * or BTRFS_ROOT_BACKREF_KEY.
+ *
+ * The dirid, sequence, name and name_len refer to the directory entry
+ * that is referencing the root.
+ *
+ * For a forward ref, the root_id is the id of the tree referencing
+ * the root and ref_id is the id of the subvol or snapshot.
+ *
+ * For a back ref the root_id is the id of the subvol or snapshot and
+ * ref_id is the id of the tree referencing it.
+ */
+int btrfs_add_root_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *tree_root,
+ u64 root_id, u8 type, u64 ref_id,
+ u64 dirid, u64 sequence,
+ const char *name, int name_len)
+{
+ struct btrfs_key key;
+ int ret;
+ struct btrfs_path *path;
+ struct btrfs_root_ref *ref;
+ struct extent_buffer *leaf;
+ unsigned long ptr;
+
+
+ path = btrfs_alloc_path();
+
+ key.objectid = root_id;
+ key.type = type;
+ key.offset = ref_id;
+
+ ret = btrfs_insert_empty_item(trans, tree_root, path, &key,
+ sizeof(*ref) + name_len);
+ BUG_ON(ret);
+
+ leaf = path->nodes[0];
+ ref = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_root_ref);
+ btrfs_set_root_ref_dirid(leaf, ref, dirid);
+ btrfs_set_root_ref_sequence(leaf, ref, sequence);
+ btrfs_set_root_ref_name_len(leaf, ref, name_len);
+ ptr = (unsigned long)(ref + 1);
+ write_extent_buffer(leaf, name, ptr, name_len);
+ btrfs_mark_buffer_dirty(leaf);
+
+ btrfs_free_path(path);
+ return ret;
+}
diff --git a/fs/btrfs/struct-funcs.c b/fs/btrfs/struct-funcs.c
new file mode 100644
index 0000000..c0f7eca
--- /dev/null
+++ b/fs/btrfs/struct-funcs.c
@@ -0,0 +1,139 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/highmem.h>
+
+/* this is some deeply nasty code. ctree.h has a different
+ * definition for this BTRFS_SETGET_FUNCS macro, behind a #ifndef
+ *
+ * The end result is that anyone who #includes ctree.h gets a
+ * declaration for the btrfs_set_foo functions and btrfs_foo functions
+ *
+ * This file declares the macros and then #includes ctree.h, which results
+ * in cpp creating the function here based on the template below.
+ *
+ * These setget functions do all the extent_buffer related mapping
+ * required to efficiently read and write specific fields in the extent
+ * buffers. Every pointer to metadata items in btrfs is really just
+ * an unsigned long offset into the extent buffer which has been
+ * cast to a specific type. This gives us all the gcc type checking.
+ *
+ * The extent buffer api is used to do all the kmapping and page
+ * spanning work required to get extent buffers in highmem and have
+ * a metadata blocksize different from the page size.
+ *
+ * The macro starts with a simple function prototype declaration so that
+ * sparse won't complain about it being static.
+ */
+
+#define BTRFS_SETGET_FUNCS(name, type, member, bits) \
+u##bits btrfs_##name(struct extent_buffer *eb, type *s); \
+void btrfs_set_##name(struct extent_buffer *eb, type *s, u##bits val); \
+u##bits btrfs_##name(struct extent_buffer *eb, \
+ type *s) \
+{ \
+ unsigned long part_offset = (unsigned long)s; \
+ unsigned long offset = part_offset + offsetof(type, member); \
+ type *p; \
+ /* ugly, but we want the fast path here */ \
+ if (eb->map_token && offset >= eb->map_start && \
+ offset + sizeof(((type *)0)->member) <= eb->map_start + \
+ eb->map_len) { \
+ p = (type *)(eb->kaddr + part_offset - eb->map_start); \
+ return le##bits##_to_cpu(p->member); \
+ } \
+ { \
+ int err; \
+ char *map_token; \
+ char *kaddr; \
+ int unmap_on_exit = (eb->map_token == NULL); \
+ unsigned long map_start; \
+ unsigned long map_len; \
+ u##bits res; \
+ err = map_extent_buffer(eb, offset, \
+ sizeof(((type *)0)->member), \
+ &map_token, &kaddr, \
+ &map_start, &map_len, KM_USER1); \
+ if (err) { \
+ __le##bits leres; \
+ read_eb_member(eb, s, type, member, &leres); \
+ return le##bits##_to_cpu(leres); \
+ } \
+ p = (type *)(kaddr + part_offset - map_start); \
+ res = le##bits##_to_cpu(p->member); \
+ if (unmap_on_exit) \
+ unmap_extent_buffer(eb, map_token, KM_USER1); \
+ return res; \
+ } \
+} \
+void btrfs_set_##name(struct extent_buffer *eb, \
+ type *s, u##bits val) \
+{ \
+ unsigned long part_offset = (unsigned long)s; \
+ unsigned long offset = part_offset + offsetof(type, member); \
+ type *p; \
+ /* ugly, but we want the fast path here */ \
+ if (eb->map_token && offset >= eb->map_start && \
+ offset + sizeof(((type *)0)->member) <= eb->map_start + \
+ eb->map_len) { \
+ p = (type *)(eb->kaddr + part_offset - eb->map_start); \
+ p->member = cpu_to_le##bits(val); \
+ return; \
+ } \
+ { \
+ int err; \
+ char *map_token; \
+ char *kaddr; \
+ int unmap_on_exit = (eb->map_token == NULL); \
+ unsigned long map_start; \
+ unsigned long map_len; \
+ err = map_extent_buffer(eb, offset, \
+ sizeof(((type *)0)->member), \
+ &map_token, &kaddr, \
+ &map_start, &map_len, KM_USER1); \
+ if (err) { \
+ __le##bits val2; \
+ val2 = cpu_to_le##bits(val); \
+ write_eb_member(eb, s, type, member, &val2); \
+ return; \
+ } \
+ p = (type *)(kaddr + part_offset - map_start); \
+ p->member = cpu_to_le##bits(val); \
+ if (unmap_on_exit) \
+ unmap_extent_buffer(eb, map_token, KM_USER1); \
+ } \
+}
+
+#include "ctree.h"
+
+void btrfs_node_key(struct extent_buffer *eb,
+ struct btrfs_disk_key *disk_key, int nr)
+{
+ unsigned long ptr = btrfs_node_key_ptr_offset(nr);
+ if (eb->map_token && ptr >= eb->map_start &&
+ ptr + sizeof(*disk_key) <= eb->map_start + eb->map_len) {
+ memcpy(disk_key, eb->kaddr + ptr - eb->map_start,
+ sizeof(*disk_key));
+ return;
+ } else if (eb->map_token) {
+ unmap_extent_buffer(eb, eb->map_token, KM_USER1);
+ eb->map_token = NULL;
+ }
+ read_eb_member(eb, (struct btrfs_key_ptr *)ptr,
+ struct btrfs_key_ptr, key, disk_key);
+}
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
new file mode 100644
index 0000000..b4c101d
--- /dev/null
+++ b/fs/btrfs/super.c
@@ -0,0 +1,720 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/blkdev.h>
+#include <linux/module.h>
+#include <linux/buffer_head.h>
+#include <linux/fs.h>
+#include <linux/pagemap.h>
+#include <linux/highmem.h>
+#include <linux/time.h>
+#include <linux/init.h>
+#include <linux/string.h>
+#include <linux/smp_lock.h>
+#include <linux/backing-dev.h>
+#include <linux/mount.h>
+#include <linux/mpage.h>
+#include <linux/swap.h>
+#include <linux/writeback.h>
+#include <linux/statfs.h>
+#include <linux/compat.h>
+#include <linux/parser.h>
+#include <linux/ctype.h>
+#include <linux/namei.h>
+#include <linux/miscdevice.h>
+#include <linux/version.h>
+#include "compat.h"
+#include "ctree.h"
+#include "disk-io.h"
+#include "transaction.h"
+#include "btrfs_inode.h"
+#include "ioctl.h"
+#include "print-tree.h"
+#include "xattr.h"
+#include "volumes.h"
+#include "version.h"
+#include "export.h"
+#include "compression.h"
+
+#define BTRFS_SUPER_MAGIC 0x9123683E
+
+static struct super_operations btrfs_super_ops;
+
+static void btrfs_put_super(struct super_block *sb)
+{
+ struct btrfs_root *root = btrfs_sb(sb);
+ int ret;
+
+ ret = close_ctree(root);
+ sb->s_fs_info = NULL;
+}
+
+enum {
+ Opt_degraded, Opt_subvol, Opt_device, Opt_nodatasum, Opt_nodatacow,
+ Opt_max_extent, Opt_max_inline, Opt_alloc_start, Opt_nobarrier,
+ Opt_ssd, Opt_thread_pool, Opt_noacl, Opt_compress, Opt_err,
+};
+
+static match_table_t tokens = {
+ {Opt_degraded, "degraded"},
+ {Opt_subvol, "subvol=%s"},
+ {Opt_device, "device=%s"},
+ {Opt_nodatasum, "nodatasum"},
+ {Opt_nodatacow, "nodatacow"},
+ {Opt_nobarrier, "nobarrier"},
+ {Opt_max_extent, "max_extent=%s"},
+ {Opt_max_inline, "max_inline=%s"},
+ {Opt_alloc_start, "alloc_start=%s"},
+ {Opt_thread_pool, "thread_pool=%d"},
+ {Opt_compress, "compress"},
+ {Opt_ssd, "ssd"},
+ {Opt_noacl, "noacl"},
+ {Opt_err, NULL},
+};
+
+u64 btrfs_parse_size(char *str)
+{
+ u64 res;
+ int mult = 1;
+ char *end;
+ char last;
+
+ res = simple_strtoul(str, &end, 10);
+
+ last = end[0];
+ if (isalpha(last)) {
+ last = tolower(last);
+ switch (last) {
+ case 'g':
+ mult *= 1024;
+ case 'm':
+ mult *= 1024;
+ case 'k':
+ mult *= 1024;
+ }
+ res = res * mult;
+ }
+ return res;
+}
+
+/*
+ * Regular mount options parser. Everything that is needed only when
+ * reading in a new superblock is parsed here.
+ */
+int btrfs_parse_options(struct btrfs_root *root, char *options)
+{
+ struct btrfs_fs_info *info = root->fs_info;
+ substring_t args[MAX_OPT_ARGS];
+ char *p, *num;
+ int intarg;
+
+ if (!options)
+ return 0;
+
+ /*
+ * strsep changes the string, duplicate it because parse_options
+ * gets called twice
+ */
+ options = kstrdup(options, GFP_NOFS);
+ if (!options)
+ return -ENOMEM;
+
+
+ while ((p = strsep(&options, ",")) != NULL) {
+ int token;
+ if (!*p)
+ continue;
+
+ token = match_token(p, tokens, args);
+ switch (token) {
+ case Opt_degraded:
+ printk(KERN_INFO "btrfs: allowing degraded mounts\n");
+ btrfs_set_opt(info->mount_opt, DEGRADED);
+ break;
+ case Opt_subvol:
+ case Opt_device:
+ /*
+ * These are parsed by btrfs_parse_early_options
+ * and can be happily ignored here.
+ */
+ break;
+ case Opt_nodatasum:
+ printk(KERN_INFO "btrfs: setting nodatacsum\n");
+ btrfs_set_opt(info->mount_opt, NODATASUM);
+ break;
+ case Opt_nodatacow:
+ printk(KERN_INFO "btrfs: setting nodatacow\n");
+ btrfs_set_opt(info->mount_opt, NODATACOW);
+ btrfs_set_opt(info->mount_opt, NODATASUM);
+ break;
+ case Opt_compress:
+ printk(KERN_INFO "btrfs: use compression\n");
+ btrfs_set_opt(info->mount_opt, COMPRESS);
+ break;
+ case Opt_ssd:
+ printk(KERN_INFO "btrfs: use ssd allocation scheme\n");
+ btrfs_set_opt(info->mount_opt, SSD);
+ break;
+ case Opt_nobarrier:
+ printk(KERN_INFO "btrfs: turning off barriers\n");
+ btrfs_set_opt(info->mount_opt, NOBARRIER);
+ break;
+ case Opt_thread_pool:
+ intarg = 0;
+ match_int(&args[0], &intarg);
+ if (intarg) {
+ info->thread_pool_size = intarg;
+ printk(KERN_INFO "btrfs: thread pool %d\n",
+ info->thread_pool_size);
+ }
+ break;
+ case Opt_max_extent:
+ num = match_strdup(&args[0]);
+ if (num) {
+ info->max_extent = btrfs_parse_size(num);
+ kfree(num);
+
+ info->max_extent = max_t(u64,
+ info->max_extent, root->sectorsize);
+ printk(KERN_INFO "btrfs: max_extent at %llu\n",
+ info->max_extent);
+ }
+ break;
+ case Opt_max_inline:
+ num = match_strdup(&args[0]);
+ if (num) {
+ info->max_inline = btrfs_parse_size(num);
+ kfree(num);
+
+ if (info->max_inline) {
+ info->max_inline = max_t(u64,
+ info->max_inline,
+ root->sectorsize);
+ }
+ printk(KERN_INFO "btrfs: max_inline at %llu\n",
+ info->max_inline);
+ }
+ break;
+ case Opt_alloc_start:
+ num = match_strdup(&args[0]);
+ if (num) {
+ info->alloc_start = btrfs_parse_size(num);
+ kfree(num);
+ printk(KERN_INFO
+ "btrfs: allocations start at %llu\n",
+ info->alloc_start);
+ }
+ break;
+ case Opt_noacl:
+ root->fs_info->sb->s_flags &= ~MS_POSIXACL;
+ break;
+ default:
+ break;
+ }
+ }
+ kfree(options);
+ return 0;
+}
+
+/*
+ * Parse mount options that are required early in the mount process.
+ *
+ * All other options will be parsed on much later in the mount process and
+ * only when we need to allocate a new super block.
+ */
+static int btrfs_parse_early_options(const char *options, fmode_t flags,
+ void *holder, char **subvol_name,
+ struct btrfs_fs_devices **fs_devices)
+{
+ substring_t args[MAX_OPT_ARGS];
+ char *opts, *p;
+ int error = 0;
+
+ if (!options)
+ goto out;
+
+ /*
+ * strsep changes the string, duplicate it because parse_options
+ * gets called twice
+ */
+ opts = kstrdup(options, GFP_KERNEL);
+ if (!opts)
+ return -ENOMEM;
+
+ while ((p = strsep(&opts, ",")) != NULL) {
+ int token;
+ if (!*p)
+ continue;
+
+ token = match_token(p, tokens, args);
+ switch (token) {
+ case Opt_subvol:
+ *subvol_name = match_strdup(&args[0]);
+ break;
+ case Opt_device:
+ error = btrfs_scan_one_device(match_strdup(&args[0]),
+ flags, holder, fs_devices);
+ if (error)
+ goto out_free_opts;
+ break;
+ default:
+ break;
+ }
+ }
+
+ out_free_opts:
+ kfree(opts);
+ out:
+ /*
+ * If no subvolume name is specified we use the default one. Allocate
+ * a copy of the string "." here so that code later in the
+ * mount path doesn't care if it's the default volume or another one.
+ */
+ if (!*subvol_name) {
+ *subvol_name = kstrdup(".", GFP_KERNEL);
+ if (!*subvol_name)
+ return -ENOMEM;
+ }
+ return error;
+}
+
+static int btrfs_fill_super(struct super_block *sb,
+ struct btrfs_fs_devices *fs_devices,
+ void *data, int silent)
+{
+ struct inode *inode;
+ struct dentry *root_dentry;
+ struct btrfs_super_block *disk_super;
+ struct btrfs_root *tree_root;
+ struct btrfs_inode *bi;
+ int err;
+
+ sb->s_maxbytes = MAX_LFS_FILESIZE;
+ sb->s_magic = BTRFS_SUPER_MAGIC;
+ sb->s_op = &btrfs_super_ops;
+ sb->s_export_op = &btrfs_export_ops;
+ sb->s_xattr = btrfs_xattr_handlers;
+ sb->s_time_gran = 1;
+ sb->s_flags |= MS_POSIXACL;
+
+ tree_root = open_ctree(sb, fs_devices, (char *)data);
+
+ if (IS_ERR(tree_root)) {
+ printk("btrfs: open_ctree failed\n");
+ return PTR_ERR(tree_root);
+ }
+ sb->s_fs_info = tree_root;
+ disk_super = &tree_root->fs_info->super_copy;
+ inode = btrfs_iget_locked(sb, BTRFS_FIRST_FREE_OBJECTID,
+ tree_root->fs_info->fs_root);
+ bi = BTRFS_I(inode);
+ bi->location.objectid = inode->i_ino;
+ bi->location.offset = 0;
+ bi->root = tree_root->fs_info->fs_root;
+
+ btrfs_set_key_type(&bi->location, BTRFS_INODE_ITEM_KEY);
+
+ if (!inode) {
+ err = -ENOMEM;
+ goto fail_close;
+ }
+ if (inode->i_state & I_NEW) {
+ btrfs_read_locked_inode(inode);
+ unlock_new_inode(inode);
+ }
+
+ root_dentry = d_alloc_root(inode);
+ if (!root_dentry) {
+ iput(inode);
+ err = -ENOMEM;
+ goto fail_close;
+ }
+#if 0
+ /* this does the super kobj at the same time */
+ err = btrfs_sysfs_add_super(tree_root->fs_info);
+ if (err)
+ goto fail_close;
+#endif
+
+ sb->s_root = root_dentry;
+
+ save_mount_options(sb, data);
+ return 0;
+
+fail_close:
+ close_ctree(tree_root);
+ return err;
+}
+
+int btrfs_sync_fs(struct super_block *sb, int wait)
+{
+ struct btrfs_trans_handle *trans;
+ struct btrfs_root *root;
+ int ret;
+ root = btrfs_sb(sb);
+
+ if (sb->s_flags & MS_RDONLY)
+ return 0;
+
+ sb->s_dirt = 0;
+ if (!wait) {
+ filemap_flush(root->fs_info->btree_inode->i_mapping);
+ return 0;
+ }
+
+ btrfs_start_delalloc_inodes(root);
+ btrfs_wait_ordered_extents(root, 0);
+
+ btrfs_clean_old_snapshots(root);
+ trans = btrfs_start_transaction(root, 1);
+ ret = btrfs_commit_transaction(trans, root);
+ sb->s_dirt = 0;
+ return ret;
+}
+
+static void btrfs_write_super(struct super_block *sb)
+{
+ sb->s_dirt = 0;
+}
+
+static int btrfs_test_super(struct super_block *s, void *data)
+{
+ struct btrfs_fs_devices *test_fs_devices = data;
+ struct btrfs_root *root = btrfs_sb(s);
+
+ return root->fs_info->fs_devices == test_fs_devices;
+}
+
+/*
+ * Find a superblock for the given device / mount point.
+ *
+ * Note: This is based on get_sb_bdev from fs/super.c with a few additions
+ * for multiple device setup. Make sure to keep it in sync.
+ */
+static int btrfs_get_sb(struct file_system_type *fs_type, int flags,
+ const char *dev_name, void *data, struct vfsmount *mnt)
+{
+ char *subvol_name = NULL;
+ struct block_device *bdev = NULL;
+ struct super_block *s;
+ struct dentry *root;
+ struct btrfs_fs_devices *fs_devices = NULL;
+ fmode_t mode = FMODE_READ;
+ int error = 0;
+
+ if (!(flags & MS_RDONLY))
+ mode |= FMODE_WRITE;
+
+ error = btrfs_parse_early_options(data, mode, fs_type,
+ &subvol_name, &fs_devices);
+ if (error)
+ return error;
+
+ error = btrfs_scan_one_device(dev_name, mode, fs_type, &fs_devices);
+ if (error)
+ goto error_free_subvol_name;
+
+ error = btrfs_open_devices(fs_devices, mode, fs_type);
+ if (error)
+ goto error_free_subvol_name;
+
+ if (!(flags & MS_RDONLY) && fs_devices->rw_devices == 0) {
+ error = -EACCES;
+ goto error_close_devices;
+ }
+
+ bdev = fs_devices->latest_bdev;
+ s = sget(fs_type, btrfs_test_super, set_anon_super, fs_devices);
+ if (IS_ERR(s))
+ goto error_s;
+
+ if (s->s_root) {
+ if ((flags ^ s->s_flags) & MS_RDONLY) {
+ up_write(&s->s_umount);
+ deactivate_super(s);
+ error = -EBUSY;
+ goto error_close_devices;
+ }
+
+ btrfs_close_devices(fs_devices);
+ } else {
+ char b[BDEVNAME_SIZE];
+
+ s->s_flags = flags;
+ strlcpy(s->s_id, bdevname(bdev, b), sizeof(s->s_id));
+ error = btrfs_fill_super(s, fs_devices, data,
+ flags & MS_SILENT ? 1 : 0);
+ if (error) {
+ up_write(&s->s_umount);
+ deactivate_super(s);
+ goto error_free_subvol_name;
+ }
+
+ btrfs_sb(s)->fs_info->bdev_holder = fs_type;
+ s->s_flags |= MS_ACTIVE;
+ }
+
+ if (!strcmp(subvol_name, "."))
+ root = dget(s->s_root);
+ else {
+ mutex_lock(&s->s_root->d_inode->i_mutex);
+ root = lookup_one_len(subvol_name, s->s_root,
+ strlen(subvol_name));
+ mutex_unlock(&s->s_root->d_inode->i_mutex);
+
+ if (IS_ERR(root)) {
+ up_write(&s->s_umount);
+ deactivate_super(s);
+ error = PTR_ERR(root);
+ goto error_free_subvol_name;
+ }
+ if (!root->d_inode) {
+ dput(root);
+ up_write(&s->s_umount);
+ deactivate_super(s);
+ error = -ENXIO;
+ goto error_free_subvol_name;
+ }
+ }
+
+ mnt->mnt_sb = s;
+ mnt->mnt_root = root;
+
+ kfree(subvol_name);
+ return 0;
+
+error_s:
+ error = PTR_ERR(s);
+error_close_devices:
+ btrfs_close_devices(fs_devices);
+error_free_subvol_name:
+ kfree(subvol_name);
+ return error;
+}
+
+static int btrfs_remount(struct super_block *sb, int *flags, char *data)
+{
+ struct btrfs_root *root = btrfs_sb(sb);
+ int ret;
+
+ if ((*flags & MS_RDONLY) == (sb->s_flags & MS_RDONLY))
+ return 0;
+
+ if (*flags & MS_RDONLY) {
+ sb->s_flags |= MS_RDONLY;
+
+ ret = btrfs_commit_super(root);
+ WARN_ON(ret);
+ } else {
+ if (root->fs_info->fs_devices->rw_devices == 0)
+ return -EACCES;
+
+ if (btrfs_super_log_root(&root->fs_info->super_copy) != 0)
+ return -EINVAL;
+
+ ret = btrfs_cleanup_reloc_trees(root);
+ WARN_ON(ret);
+
+ ret = btrfs_cleanup_fs_roots(root->fs_info);
+ WARN_ON(ret);
+
+ sb->s_flags &= ~MS_RDONLY;
+ }
+
+ return 0;
+}
+
+static int btrfs_statfs(struct dentry *dentry, struct kstatfs *buf)
+{
+ struct btrfs_root *root = btrfs_sb(dentry->d_sb);
+ struct btrfs_super_block *disk_super = &root->fs_info->super_copy;
+ int bits = dentry->d_sb->s_blocksize_bits;
+ __be32 *fsid = (__be32 *)root->fs_info->fsid;
+
+ buf->f_namelen = BTRFS_NAME_LEN;
+ buf->f_blocks = btrfs_super_total_bytes(disk_super) >> bits;
+ buf->f_bfree = buf->f_blocks -
+ (btrfs_super_bytes_used(disk_super) >> bits);
+ buf->f_bavail = buf->f_bfree;
+ buf->f_bsize = dentry->d_sb->s_blocksize;
+ buf->f_type = BTRFS_SUPER_MAGIC;
+
+ /* We treat it as constant endianness (it doesn't matter _which_)
+ because we want the fsid to come out the same whether mounted
+ on a big-endian or little-endian host */
+ buf->f_fsid.val[0] = be32_to_cpu(fsid[0]) ^ be32_to_cpu(fsid[2]);
+ buf->f_fsid.val[1] = be32_to_cpu(fsid[1]) ^ be32_to_cpu(fsid[3]);
+ /* Mask in the root object ID too, to disambiguate subvols */
+ buf->f_fsid.val[0] ^= BTRFS_I(dentry->d_inode)->root->objectid >> 32;
+ buf->f_fsid.val[1] ^= BTRFS_I(dentry->d_inode)->root->objectid;
+
+ return 0;
+}
+
+static struct file_system_type btrfs_fs_type = {
+ .owner = THIS_MODULE,
+ .name = "btrfs",
+ .get_sb = btrfs_get_sb,
+ .kill_sb = kill_anon_super,
+ .fs_flags = FS_REQUIRES_DEV,
+};
+
+/*
+ * used by btrfsctl to scan devices when no FS is mounted
+ */
+static long btrfs_control_ioctl(struct file *file, unsigned int cmd,
+ unsigned long arg)
+{
+ struct btrfs_ioctl_vol_args *vol;
+ struct btrfs_fs_devices *fs_devices;
+ int ret = 0;
+ int len;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ vol = kmalloc(sizeof(*vol), GFP_KERNEL);
+ if (copy_from_user(vol, (void __user *)arg, sizeof(*vol))) {
+ ret = -EFAULT;
+ goto out;
+ }
+ len = strnlen(vol->name, BTRFS_PATH_NAME_MAX);
+ switch (cmd) {
+ case BTRFS_IOC_SCAN_DEV:
+ ret = btrfs_scan_one_device(vol->name, FMODE_READ,
+ &btrfs_fs_type, &fs_devices);
+ break;
+ }
+out:
+ kfree(vol);
+ return ret;
+}
+
+static void btrfs_write_super_lockfs(struct super_block *sb)
+{
+ struct btrfs_root *root = btrfs_sb(sb);
+ mutex_lock(&root->fs_info->transaction_kthread_mutex);
+ mutex_lock(&root->fs_info->cleaner_mutex);
+}
+
+static void btrfs_unlockfs(struct super_block *sb)
+{
+ struct btrfs_root *root = btrfs_sb(sb);
+ mutex_unlock(&root->fs_info->cleaner_mutex);
+ mutex_unlock(&root->fs_info->transaction_kthread_mutex);
+}
+
+static struct super_operations btrfs_super_ops = {
+ .delete_inode = btrfs_delete_inode,
+ .put_super = btrfs_put_super,
+ .write_super = btrfs_write_super,
+ .sync_fs = btrfs_sync_fs,
+ .show_options = generic_show_options,
+ .write_inode = btrfs_write_inode,
+ .dirty_inode = btrfs_dirty_inode,
+ .alloc_inode = btrfs_alloc_inode,
+ .destroy_inode = btrfs_destroy_inode,
+ .statfs = btrfs_statfs,
+ .remount_fs = btrfs_remount,
+ .write_super_lockfs = btrfs_write_super_lockfs,
+ .unlockfs = btrfs_unlockfs,
+};
+
+static const struct file_operations btrfs_ctl_fops = {
+ .unlocked_ioctl = btrfs_control_ioctl,
+ .compat_ioctl = btrfs_control_ioctl,
+ .owner = THIS_MODULE,
+};
+
+static struct miscdevice btrfs_misc = {
+ .minor = MISC_DYNAMIC_MINOR,
+ .name = "btrfs-control",
+ .fops = &btrfs_ctl_fops
+};
+
+static int btrfs_interface_init(void)
+{
+ return misc_register(&btrfs_misc);
+}
+
+static void btrfs_interface_exit(void)
+{
+ if (misc_deregister(&btrfs_misc) < 0)
+ printk(KERN_INFO "misc_deregister failed for control device");
+}
+
+static int __init init_btrfs_fs(void)
+{
+ int err;
+
+ err = btrfs_init_sysfs();
+ if (err)
+ return err;
+
+ err = btrfs_init_cachep();
+ if (err)
+ goto free_sysfs;
+
+ err = extent_io_init();
+ if (err)
+ goto free_cachep;
+
+ err = extent_map_init();
+ if (err)
+ goto free_extent_io;
+
+ err = btrfs_interface_init();
+ if (err)
+ goto free_extent_map;
+
+ err = register_filesystem(&btrfs_fs_type);
+ if (err)
+ goto unregister_ioctl;
+
+ printk(KERN_INFO "%s loaded\n", BTRFS_BUILD_VERSION);
+ return 0;
+
+unregister_ioctl:
+ btrfs_interface_exit();
+free_extent_map:
+ extent_map_exit();
+free_extent_io:
+ extent_io_exit();
+free_cachep:
+ btrfs_destroy_cachep();
+free_sysfs:
+ btrfs_exit_sysfs();
+ return err;
+}
+
+static void __exit exit_btrfs_fs(void)
+{
+ btrfs_destroy_cachep();
+ extent_map_exit();
+ extent_io_exit();
+ btrfs_interface_exit();
+ unregister_filesystem(&btrfs_fs_type);
+ btrfs_exit_sysfs();
+ btrfs_cleanup_fs_uuids();
+ btrfs_zlib_exit();
+}
+
+module_init(init_btrfs_fs)
+module_exit(exit_btrfs_fs)
+
+MODULE_LICENSE("GPL");
diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
new file mode 100644
index 0000000..a240b6f
--- /dev/null
+++ b/fs/btrfs/sysfs.c
@@ -0,0 +1,269 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/completion.h>
+#include <linux/buffer_head.h>
+#include <linux/module.h>
+#include <linux/kobject.h>
+
+#include "ctree.h"
+#include "disk-io.h"
+#include "transaction.h"
+
+static ssize_t root_blocks_used_show(struct btrfs_root *root, char *buf)
+{
+ return snprintf(buf, PAGE_SIZE, "%llu\n",
+ (unsigned long long)btrfs_root_used(&root->root_item));
+}
+
+static ssize_t root_block_limit_show(struct btrfs_root *root, char *buf)
+{
+ return snprintf(buf, PAGE_SIZE, "%llu\n",
+ (unsigned long long)btrfs_root_limit(&root->root_item));
+}
+
+static ssize_t super_blocks_used_show(struct btrfs_fs_info *fs, char *buf)
+{
+
+ return snprintf(buf, PAGE_SIZE, "%llu\n",
+ (unsigned long long)btrfs_super_bytes_used(&fs->super_copy));
+}
+
+static ssize_t super_total_blocks_show(struct btrfs_fs_info *fs, char *buf)
+{
+ return snprintf(buf, PAGE_SIZE, "%llu\n",
+ (unsigned long long)btrfs_super_total_bytes(&fs->super_copy));
+}
+
+static ssize_t super_blocksize_show(struct btrfs_fs_info *fs, char *buf)
+{
+ return snprintf(buf, PAGE_SIZE, "%llu\n",
+ (unsigned long long)btrfs_super_sectorsize(&fs->super_copy));
+}
+
+/* this is for root attrs (subvols/snapshots) */
+struct btrfs_root_attr {
+ struct attribute attr;
+ ssize_t (*show)(struct btrfs_root *, char *);
+ ssize_t (*store)(struct btrfs_root *, const char *, size_t);
+};
+
+#define ROOT_ATTR(name, mode, show, store) \
+static struct btrfs_root_attr btrfs_root_attr_##name = __ATTR(name, mode, \
+ show, store)
+
+ROOT_ATTR(blocks_used, 0444, root_blocks_used_show, NULL);
+ROOT_ATTR(block_limit, 0644, root_block_limit_show, NULL);
+
+static struct attribute *btrfs_root_attrs[] = {
+ &btrfs_root_attr_blocks_used.attr,
+ &btrfs_root_attr_block_limit.attr,
+ NULL,
+};
+
+/* this is for super attrs (actual full fs) */
+struct btrfs_super_attr {
+ struct attribute attr;
+ ssize_t (*show)(struct btrfs_fs_info *, char *);
+ ssize_t (*store)(struct btrfs_fs_info *, const char *, size_t);
+};
+
+#define SUPER_ATTR(name, mode, show, store) \
+static struct btrfs_super_attr btrfs_super_attr_##name = __ATTR(name, mode, \
+ show, store)
+
+SUPER_ATTR(blocks_used, 0444, super_blocks_used_show, NULL);
+SUPER_ATTR(total_blocks, 0444, super_total_blocks_show, NULL);
+SUPER_ATTR(blocksize, 0444, super_blocksize_show, NULL);
+
+static struct attribute *btrfs_super_attrs[] = {
+ &btrfs_super_attr_blocks_used.attr,
+ &btrfs_super_attr_total_blocks.attr,
+ &btrfs_super_attr_blocksize.attr,
+ NULL,
+};
+
+static ssize_t btrfs_super_attr_show(struct kobject *kobj,
+ struct attribute *attr, char *buf)
+{
+ struct btrfs_fs_info *fs = container_of(kobj, struct btrfs_fs_info,
+ super_kobj);
+ struct btrfs_super_attr *a = container_of(attr,
+ struct btrfs_super_attr,
+ attr);
+
+ return a->show ? a->show(fs, buf) : 0;
+}
+
+static ssize_t btrfs_super_attr_store(struct kobject *kobj,
+ struct attribute *attr,
+ const char *buf, size_t len)
+{
+ struct btrfs_fs_info *fs = container_of(kobj, struct btrfs_fs_info,
+ super_kobj);
+ struct btrfs_super_attr *a = container_of(attr,
+ struct btrfs_super_attr,
+ attr);
+
+ return a->store ? a->store(fs, buf, len) : 0;
+}
+
+static ssize_t btrfs_root_attr_show(struct kobject *kobj,
+ struct attribute *attr, char *buf)
+{
+ struct btrfs_root *root = container_of(kobj, struct btrfs_root,
+ root_kobj);
+ struct btrfs_root_attr *a = container_of(attr,
+ struct btrfs_root_attr,
+ attr);
+
+ return a->show ? a->show(root, buf) : 0;
+}
+
+static ssize_t btrfs_root_attr_store(struct kobject *kobj,
+ struct attribute *attr,
+ const char *buf, size_t len)
+{
+ struct btrfs_root *root = container_of(kobj, struct btrfs_root,
+ root_kobj);
+ struct btrfs_root_attr *a = container_of(attr,
+ struct btrfs_root_attr,
+ attr);
+ return a->store ? a->store(root, buf, len) : 0;
+}
+
+static void btrfs_super_release(struct kobject *kobj)
+{
+ struct btrfs_fs_info *fs = container_of(kobj, struct btrfs_fs_info,
+ super_kobj);
+ complete(&fs->kobj_unregister);
+}
+
+static void btrfs_root_release(struct kobject *kobj)
+{
+ struct btrfs_root *root = container_of(kobj, struct btrfs_root,
+ root_kobj);
+ complete(&root->kobj_unregister);
+}
+
+static struct sysfs_ops btrfs_super_attr_ops = {
+ .show = btrfs_super_attr_show,
+ .store = btrfs_super_attr_store,
+};
+
+static struct sysfs_ops btrfs_root_attr_ops = {
+ .show = btrfs_root_attr_show,
+ .store = btrfs_root_attr_store,
+};
+
+static struct kobj_type btrfs_root_ktype = {
+ .default_attrs = btrfs_root_attrs,
+ .sysfs_ops = &btrfs_root_attr_ops,
+ .release = btrfs_root_release,
+};
+
+static struct kobj_type btrfs_super_ktype = {
+ .default_attrs = btrfs_super_attrs,
+ .sysfs_ops = &btrfs_super_attr_ops,
+ .release = btrfs_super_release,
+};
+
+/* /sys/fs/btrfs/ entry */
+static struct kset *btrfs_kset;
+
+int btrfs_sysfs_add_super(struct btrfs_fs_info *fs)
+{
+ int error;
+ char *name;
+ char c;
+ int len = strlen(fs->sb->s_id) + 1;
+ int i;
+
+ name = kmalloc(len, GFP_NOFS);
+ if (!name) {
+ error = -ENOMEM;
+ goto fail;
+ }
+
+ for (i = 0; i < len; i++) {
+ c = fs->sb->s_id[i];
+ if (c == '/' || c == '\\')
+ c = '!';
+ name[i] = c;
+ }
+ name[len] = '\0';
+
+ fs->super_kobj.kset = btrfs_kset;
+ error = kobject_init_and_add(&fs->super_kobj, &btrfs_super_ktype,
+ NULL, "%s", name);
+ kfree(name);
+ if (error)
+ goto fail;
+
+ return 0;
+
+fail:
+ printk(KERN_ERR "btrfs: sysfs creation for super failed\n");
+ return error;
+}
+
+int btrfs_sysfs_add_root(struct btrfs_root *root)
+{
+ int error;
+
+ error = kobject_init_and_add(&root->root_kobj, &btrfs_root_ktype,
+ &root->fs_info->super_kobj,
+ "%s", root->name);
+ if (error)
+ goto fail;
+
+ return 0;
+
+fail:
+ printk(KERN_ERR "btrfs: sysfs creation for root failed\n");
+ return error;
+}
+
+void btrfs_sysfs_del_root(struct btrfs_root *root)
+{
+ kobject_put(&root->root_kobj);
+ wait_for_completion(&root->kobj_unregister);
+}
+
+void btrfs_sysfs_del_super(struct btrfs_fs_info *fs)
+{
+ kobject_put(&fs->super_kobj);
+ wait_for_completion(&fs->kobj_unregister);
+}
+
+int btrfs_init_sysfs(void)
+{
+ btrfs_kset = kset_create_and_add("btrfs", NULL, fs_kobj);
+ if (!btrfs_kset)
+ return -ENOMEM;
+ return 0;
+}
+
+void btrfs_exit_sysfs(void)
+{
+ kset_unregister(btrfs_kset);
+}
+
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
new file mode 100644
index 0000000..8a08f94
--- /dev/null
+++ b/fs/btrfs/transaction.c
@@ -0,0 +1,1097 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/fs.h>
+#include <linux/sched.h>
+#include <linux/writeback.h>
+#include <linux/pagemap.h>
+#include <linux/blkdev.h>
+#include "ctree.h"
+#include "disk-io.h"
+#include "transaction.h"
+#include "locking.h"
+#include "ref-cache.h"
+#include "tree-log.h"
+
+#define BTRFS_ROOT_TRANS_TAG 0
+
+static noinline void put_transaction(struct btrfs_transaction *transaction)
+{
+ WARN_ON(transaction->use_count == 0);
+ transaction->use_count--;
+ if (transaction->use_count == 0) {
+ list_del_init(&transaction->list);
+ memset(transaction, 0, sizeof(*transaction));
+ kmem_cache_free(btrfs_transaction_cachep, transaction);
+ }
+}
+
+/*
+ * either allocate a new transaction or hop into the existing one
+ */
+static noinline int join_transaction(struct btrfs_root *root)
+{
+ struct btrfs_transaction *cur_trans;
+ cur_trans = root->fs_info->running_transaction;
+ if (!cur_trans) {
+ cur_trans = kmem_cache_alloc(btrfs_transaction_cachep,
+ GFP_NOFS);
+ BUG_ON(!cur_trans);
+ root->fs_info->generation++;
+ root->fs_info->last_alloc = 0;
+ root->fs_info->last_data_alloc = 0;
+ cur_trans->num_writers = 1;
+ cur_trans->num_joined = 0;
+ cur_trans->transid = root->fs_info->generation;
+ init_waitqueue_head(&cur_trans->writer_wait);
+ init_waitqueue_head(&cur_trans->commit_wait);
+ cur_trans->in_commit = 0;
+ cur_trans->blocked = 0;
+ cur_trans->use_count = 1;
+ cur_trans->commit_done = 0;
+ cur_trans->start_time = get_seconds();
+ INIT_LIST_HEAD(&cur_trans->pending_snapshots);
+ list_add_tail(&cur_trans->list, &root->fs_info->trans_list);
+ extent_io_tree_init(&cur_trans->dirty_pages,
+ root->fs_info->btree_inode->i_mapping,
+ GFP_NOFS);
+ spin_lock(&root->fs_info->new_trans_lock);
+ root->fs_info->running_transaction = cur_trans;
+ spin_unlock(&root->fs_info->new_trans_lock);
+ } else {
+ cur_trans->num_writers++;
+ cur_trans->num_joined++;
+ }
+
+ return 0;
+}
+
+/*
+ * this does all the record keeping required to make sure that a reference
+ * counted root is properly recorded in a given transaction. This is required
+ * to make sure the old root from before we joined the transaction is deleted
+ * when the transaction commits
+ */
+noinline int btrfs_record_root_in_trans(struct btrfs_root *root)
+{
+ struct btrfs_dirty_root *dirty;
+ u64 running_trans_id = root->fs_info->running_transaction->transid;
+ if (root->ref_cows && root->last_trans < running_trans_id) {
+ WARN_ON(root == root->fs_info->extent_root);
+ if (root->root_item.refs != 0) {
+ radix_tree_tag_set(&root->fs_info->fs_roots_radix,
+ (unsigned long)root->root_key.objectid,
+ BTRFS_ROOT_TRANS_TAG);
+
+ dirty = kmalloc(sizeof(*dirty), GFP_NOFS);
+ BUG_ON(!dirty);
+ dirty->root = kmalloc(sizeof(*dirty->root), GFP_NOFS);
+ BUG_ON(!dirty->root);
+ dirty->latest_root = root;
+ INIT_LIST_HEAD(&dirty->list);
+
+ root->commit_root = btrfs_root_node(root);
+
+ memcpy(dirty->root, root, sizeof(*root));
+ spin_lock_init(&dirty->root->node_lock);
+ spin_lock_init(&dirty->root->list_lock);
+ mutex_init(&dirty->root->objectid_mutex);
+ mutex_init(&dirty->root->log_mutex);
+ INIT_LIST_HEAD(&dirty->root->dead_list);
+ dirty->root->node = root->commit_root;
+ dirty->root->commit_root = NULL;
+
+ spin_lock(&root->list_lock);
+ list_add(&dirty->root->dead_list, &root->dead_list);
+ spin_unlock(&root->list_lock);
+
+ root->dirty_root = dirty;
+ } else {
+ WARN_ON(1);
+ }
+ root->last_trans = running_trans_id;
+ }
+ return 0;
+}
+
+/* wait for commit against the current transaction to become unblocked
+ * when this is done, it is safe to start a new transaction, but the current
+ * transaction might not be fully on disk.
+ */
+static void wait_current_trans(struct btrfs_root *root)
+{
+ struct btrfs_transaction *cur_trans;
+
+ cur_trans = root->fs_info->running_transaction;
+ if (cur_trans && cur_trans->blocked) {
+ DEFINE_WAIT(wait);
+ cur_trans->use_count++;
+ while (1) {
+ prepare_to_wait(&root->fs_info->transaction_wait, &wait,
+ TASK_UNINTERRUPTIBLE);
+ if (cur_trans->blocked) {
+ mutex_unlock(&root->fs_info->trans_mutex);
+ schedule();
+ mutex_lock(&root->fs_info->trans_mutex);
+ finish_wait(&root->fs_info->transaction_wait,
+ &wait);
+ } else {
+ finish_wait(&root->fs_info->transaction_wait,
+ &wait);
+ break;
+ }
+ }
+ put_transaction(cur_trans);
+ }
+}
+
+static struct btrfs_trans_handle *start_transaction(struct btrfs_root *root,
+ int num_blocks, int wait)
+{
+ struct btrfs_trans_handle *h =
+ kmem_cache_alloc(btrfs_trans_handle_cachep, GFP_NOFS);
+ int ret;
+
+ mutex_lock(&root->fs_info->trans_mutex);
+ if (!root->fs_info->log_root_recovering &&
+ ((wait == 1 && !root->fs_info->open_ioctl_trans) || wait == 2))
+ wait_current_trans(root);
+ ret = join_transaction(root);
+ BUG_ON(ret);
+
+ btrfs_record_root_in_trans(root);
+ h->transid = root->fs_info->running_transaction->transid;
+ h->transaction = root->fs_info->running_transaction;
+ h->blocks_reserved = num_blocks;
+ h->blocks_used = 0;
+ h->block_group = 0;
+ h->alloc_exclude_nr = 0;
+ h->alloc_exclude_start = 0;
+ root->fs_info->running_transaction->use_count++;
+ mutex_unlock(&root->fs_info->trans_mutex);
+ return h;
+}
+
+struct btrfs_trans_handle *btrfs_start_transaction(struct btrfs_root *root,
+ int num_blocks)
+{
+ return start_transaction(root, num_blocks, 1);
+}
+struct btrfs_trans_handle *btrfs_join_transaction(struct btrfs_root *root,
+ int num_blocks)
+{
+ return start_transaction(root, num_blocks, 0);
+}
+
+struct btrfs_trans_handle *btrfs_start_ioctl_transaction(struct btrfs_root *r,
+ int num_blocks)
+{
+ return start_transaction(r, num_blocks, 2);
+}
+
+/* wait for a transaction commit to be fully complete */
+static noinline int wait_for_commit(struct btrfs_root *root,
+ struct btrfs_transaction *commit)
+{
+ DEFINE_WAIT(wait);
+ mutex_lock(&root->fs_info->trans_mutex);
+ while (!commit->commit_done) {
+ prepare_to_wait(&commit->commit_wait, &wait,
+ TASK_UNINTERRUPTIBLE);
+ if (commit->commit_done)
+ break;
+ mutex_unlock(&root->fs_info->trans_mutex);
+ schedule();
+ mutex_lock(&root->fs_info->trans_mutex);
+ }
+ mutex_unlock(&root->fs_info->trans_mutex);
+ finish_wait(&commit->commit_wait, &wait);
+ return 0;
+}
+
+/*
+ * rate limit against the drop_snapshot code. This helps to slow down new
+ * operations if the drop_snapshot code isn't able to keep up.
+ */
+static void throttle_on_drops(struct btrfs_root *root)
+{
+ struct btrfs_fs_info *info = root->fs_info;
+ int harder_count = 0;
+
+harder:
+ if (atomic_read(&info->throttles)) {
+ DEFINE_WAIT(wait);
+ int thr;
+ thr = atomic_read(&info->throttle_gen);
+
+ do {
+ prepare_to_wait(&info->transaction_throttle,
+ &wait, TASK_UNINTERRUPTIBLE);
+ if (!atomic_read(&info->throttles)) {
+ finish_wait(&info->transaction_throttle, &wait);
+ break;
+ }
+ schedule();
+ finish_wait(&info->transaction_throttle, &wait);
+ } while (thr == atomic_read(&info->throttle_gen));
+ harder_count++;
+
+ if (root->fs_info->total_ref_cache_size > 1 * 1024 * 1024 &&
+ harder_count < 2)
+ goto harder;
+
+ if (root->fs_info->total_ref_cache_size > 5 * 1024 * 1024 &&
+ harder_count < 10)
+ goto harder;
+
+ if (root->fs_info->total_ref_cache_size > 10 * 1024 * 1024 &&
+ harder_count < 20)
+ goto harder;
+ }
+}
+
+void btrfs_throttle(struct btrfs_root *root)
+{
+ mutex_lock(&root->fs_info->trans_mutex);
+ if (!root->fs_info->open_ioctl_trans)
+ wait_current_trans(root);
+ mutex_unlock(&root->fs_info->trans_mutex);
+
+ throttle_on_drops(root);
+}
+
+static int __btrfs_end_transaction(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, int throttle)
+{
+ struct btrfs_transaction *cur_trans;
+ struct btrfs_fs_info *info = root->fs_info;
+
+ mutex_lock(&info->trans_mutex);
+ cur_trans = info->running_transaction;
+ WARN_ON(cur_trans != trans->transaction);
+ WARN_ON(cur_trans->num_writers < 1);
+ cur_trans->num_writers--;
+
+ if (waitqueue_active(&cur_trans->writer_wait))
+ wake_up(&cur_trans->writer_wait);
+ put_transaction(cur_trans);
+ mutex_unlock(&info->trans_mutex);
+ memset(trans, 0, sizeof(*trans));
+ kmem_cache_free(btrfs_trans_handle_cachep, trans);
+
+ if (throttle)
+ throttle_on_drops(root);
+
+ return 0;
+}
+
+int btrfs_end_transaction(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root)
+{
+ return __btrfs_end_transaction(trans, root, 0);
+}
+
+int btrfs_end_transaction_throttle(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root)
+{
+ return __btrfs_end_transaction(trans, root, 1);
+}
+
+/*
+ * when btree blocks are allocated, they have some corresponding bits set for
+ * them in one of two extent_io trees. This is used to make sure all of
+ * those extents are on disk for transaction or log commit
+ */
+int btrfs_write_and_wait_marked_extents(struct btrfs_root *root,
+ struct extent_io_tree *dirty_pages)
+{
+ int ret;
+ int err = 0;
+ int werr = 0;
+ struct page *page;
+ struct inode *btree_inode = root->fs_info->btree_inode;
+ u64 start = 0;
+ u64 end;
+ unsigned long index;
+
+ while (1) {
+ ret = find_first_extent_bit(dirty_pages, start, &start, &end,
+ EXTENT_DIRTY);
+ if (ret)
+ break;
+ while (start <= end) {
+ cond_resched();
+
+ index = start >> PAGE_CACHE_SHIFT;
+ start = (u64)(index + 1) << PAGE_CACHE_SHIFT;
+ page = find_get_page(btree_inode->i_mapping, index);
+ if (!page)
+ continue;
+
+ btree_lock_page_hook(page);
+ if (!page->mapping) {
+ unlock_page(page);
+ page_cache_release(page);
+ continue;
+ }
+
+ if (PageWriteback(page)) {
+ if (PageDirty(page))
+ wait_on_page_writeback(page);
+ else {
+ unlock_page(page);
+ page_cache_release(page);
+ continue;
+ }
+ }
+ err = write_one_page(page, 0);
+ if (err)
+ werr = err;
+ page_cache_release(page);
+ }
+ }
+ while (1) {
+ ret = find_first_extent_bit(dirty_pages, 0, &start, &end,
+ EXTENT_DIRTY);
+ if (ret)
+ break;
+
+ clear_extent_dirty(dirty_pages, start, end, GFP_NOFS);
+ while (start <= end) {
+ index = start >> PAGE_CACHE_SHIFT;
+ start = (u64)(index + 1) << PAGE_CACHE_SHIFT;
+ page = find_get_page(btree_inode->i_mapping, index);
+ if (!page)
+ continue;
+ if (PageDirty(page)) {
+ btree_lock_page_hook(page);
+ wait_on_page_writeback(page);
+ err = write_one_page(page, 0);
+ if (err)
+ werr = err;
+ }
+ wait_on_page_writeback(page);
+ page_cache_release(page);
+ cond_resched();
+ }
+ }
+ if (err)
+ werr = err;
+ return werr;
+}
+
+int btrfs_write_and_wait_transaction(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root)
+{
+ if (!trans || !trans->transaction) {
+ struct inode *btree_inode;
+ btree_inode = root->fs_info->btree_inode;
+ return filemap_write_and_wait(btree_inode->i_mapping);
+ }
+ return btrfs_write_and_wait_marked_extents(root,
+ &trans->transaction->dirty_pages);
+}
+
+/*
+ * this is used to update the root pointer in the tree of tree roots.
+ *
+ * But, in the case of the extent allocation tree, updating the root
+ * pointer may allocate blocks which may change the root of the extent
+ * allocation tree.
+ *
+ * So, this loops and repeats and makes sure the cowonly root didn't
+ * change while the root pointer was being updated in the metadata.
+ */
+static int update_cowonly_root(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root)
+{
+ int ret;
+ u64 old_root_bytenr;
+ struct btrfs_root *tree_root = root->fs_info->tree_root;
+
+ btrfs_extent_post_op(trans, root);
+ btrfs_write_dirty_block_groups(trans, root);
+ btrfs_extent_post_op(trans, root);
+
+ while (1) {
+ old_root_bytenr = btrfs_root_bytenr(&root->root_item);
+ if (old_root_bytenr == root->node->start)
+ break;
+ btrfs_set_root_bytenr(&root->root_item,
+ root->node->start);
+ btrfs_set_root_level(&root->root_item,
+ btrfs_header_level(root->node));
+ btrfs_set_root_generation(&root->root_item, trans->transid);
+
+ btrfs_extent_post_op(trans, root);
+
+ ret = btrfs_update_root(trans, tree_root,
+ &root->root_key,
+ &root->root_item);
+ BUG_ON(ret);
+ btrfs_write_dirty_block_groups(trans, root);
+ btrfs_extent_post_op(trans, root);
+ }
+ return 0;
+}
+
+/*
+ * update all the cowonly tree roots on disk
+ */
+int btrfs_commit_tree_roots(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root)
+{
+ struct btrfs_fs_info *fs_info = root->fs_info;
+ struct list_head *next;
+ struct extent_buffer *eb;
+
+ btrfs_extent_post_op(trans, fs_info->tree_root);
+
+ eb = btrfs_lock_root_node(fs_info->tree_root);
+ btrfs_cow_block(trans, fs_info->tree_root, eb, NULL, 0, &eb, 0);
+ btrfs_tree_unlock(eb);
+ free_extent_buffer(eb);
+
+ btrfs_extent_post_op(trans, fs_info->tree_root);
+
+ while (!list_empty(&fs_info->dirty_cowonly_roots)) {
+ next = fs_info->dirty_cowonly_roots.next;
+ list_del_init(next);
+ root = list_entry(next, struct btrfs_root, dirty_list);
+
+ update_cowonly_root(trans, root);
+ }
+ return 0;
+}
+
+/*
+ * dead roots are old snapshots that need to be deleted. This allocates
+ * a dirty root struct and adds it into the list of dead roots that need to
+ * be deleted
+ */
+int btrfs_add_dead_root(struct btrfs_root *root, struct btrfs_root *latest)
+{
+ struct btrfs_dirty_root *dirty;
+
+ dirty = kmalloc(sizeof(*dirty), GFP_NOFS);
+ if (!dirty)
+ return -ENOMEM;
+ dirty->root = root;
+ dirty->latest_root = latest;
+
+ mutex_lock(&root->fs_info->trans_mutex);
+ list_add(&dirty->list, &latest->fs_info->dead_roots);
+ mutex_unlock(&root->fs_info->trans_mutex);
+ return 0;
+}
+
+/*
+ * at transaction commit time we need to schedule the old roots for
+ * deletion via btrfs_drop_snapshot. This runs through all the
+ * reference counted roots that were modified in the current
+ * transaction and puts them into the drop list
+ */
+static noinline int add_dirty_roots(struct btrfs_trans_handle *trans,
+ struct radix_tree_root *radix,
+ struct list_head *list)
+{
+ struct btrfs_dirty_root *dirty;
+ struct btrfs_root *gang[8];
+ struct btrfs_root *root;
+ int i;
+ int ret;
+ int err = 0;
+ u32 refs;
+
+ while (1) {
+ ret = radix_tree_gang_lookup_tag(radix, (void **)gang, 0,
+ ARRAY_SIZE(gang),
+ BTRFS_ROOT_TRANS_TAG);
+ if (ret == 0)
+ break;
+ for (i = 0; i < ret; i++) {
+ root = gang[i];
+ radix_tree_tag_clear(radix,
+ (unsigned long)root->root_key.objectid,
+ BTRFS_ROOT_TRANS_TAG);
+
+ BUG_ON(!root->ref_tree);
+ dirty = root->dirty_root;
+
+ btrfs_free_log(trans, root);
+ btrfs_free_reloc_root(trans, root);
+
+ if (root->commit_root == root->node) {
+ WARN_ON(root->node->start !=
+ btrfs_root_bytenr(&root->root_item));
+
+ free_extent_buffer(root->commit_root);
+ root->commit_root = NULL;
+ root->dirty_root = NULL;
+
+ spin_lock(&root->list_lock);
+ list_del_init(&dirty->root->dead_list);
+ spin_unlock(&root->list_lock);
+
+ kfree(dirty->root);
+ kfree(dirty);
+
+ /* make sure to update the root on disk
+ * so we get any updates to the block used
+ * counts
+ */
+ err = btrfs_update_root(trans,
+ root->fs_info->tree_root,
+ &root->root_key,
+ &root->root_item);
+ continue;
+ }
+
+ memset(&root->root_item.drop_progress, 0,
+ sizeof(struct btrfs_disk_key));
+ root->root_item.drop_level = 0;
+ root->commit_root = NULL;
+ root->dirty_root = NULL;
+ root->root_key.offset = root->fs_info->generation;
+ btrfs_set_root_bytenr(&root->root_item,
+ root->node->start);
+ btrfs_set_root_level(&root->root_item,
+ btrfs_header_level(root->node));
+ btrfs_set_root_generation(&root->root_item,
+ root->root_key.offset);
+
+ err = btrfs_insert_root(trans, root->fs_info->tree_root,
+ &root->root_key,
+ &root->root_item);
+ if (err)
+ break;
+
+ refs = btrfs_root_refs(&dirty->root->root_item);
+ btrfs_set_root_refs(&dirty->root->root_item, refs - 1);
+ err = btrfs_update_root(trans, root->fs_info->tree_root,
+ &dirty->root->root_key,
+ &dirty->root->root_item);
+
+ BUG_ON(err);
+ if (refs == 1) {
+ list_add(&dirty->list, list);
+ } else {
+ WARN_ON(1);
+ free_extent_buffer(dirty->root->node);
+ kfree(dirty->root);
+ kfree(dirty);
+ }
+ }
+ }
+ return err;
+}
+
+/*
+ * defrag a given btree. If cacheonly == 1, this won't read from the disk,
+ * otherwise every leaf in the btree is read and defragged.
+ */
+int btrfs_defrag_root(struct btrfs_root *root, int cacheonly)
+{
+ struct btrfs_fs_info *info = root->fs_info;
+ int ret;
+ struct btrfs_trans_handle *trans;
+ unsigned long nr;
+
+ smp_mb();
+ if (root->defrag_running)
+ return 0;
+ trans = btrfs_start_transaction(root, 1);
+ while (1) {
+ root->defrag_running = 1;
+ ret = btrfs_defrag_leaves(trans, root, cacheonly);
+ nr = trans->blocks_used;
+ btrfs_end_transaction(trans, root);
+ btrfs_btree_balance_dirty(info->tree_root, nr);
+ cond_resched();
+
+ trans = btrfs_start_transaction(root, 1);
+ if (root->fs_info->closing || ret != -EAGAIN)
+ break;
+ }
+ root->defrag_running = 0;
+ smp_mb();
+ btrfs_end_transaction(trans, root);
+ return 0;
+}
+
+/*
+ * Given a list of roots that need to be deleted, call btrfs_drop_snapshot on
+ * all of them
+ */
+static noinline int drop_dirty_roots(struct btrfs_root *tree_root,
+ struct list_head *list)
+{
+ struct btrfs_dirty_root *dirty;
+ struct btrfs_trans_handle *trans;
+ unsigned long nr;
+ u64 num_bytes;
+ u64 bytes_used;
+ u64 max_useless;
+ int ret = 0;
+ int err;
+
+ while (!list_empty(list)) {
+ struct btrfs_root *root;
+
+ dirty = list_entry(list->prev, struct btrfs_dirty_root, list);
+ list_del_init(&dirty->list);
+
+ num_bytes = btrfs_root_used(&dirty->root->root_item);
+ root = dirty->latest_root;
+ atomic_inc(&root->fs_info->throttles);
+
+ while (1) {
+ trans = btrfs_start_transaction(tree_root, 1);
+ mutex_lock(&root->fs_info->drop_mutex);
+ ret = btrfs_drop_snapshot(trans, dirty->root);
+ if (ret != -EAGAIN)
+ break;
+ mutex_unlock(&root->fs_info->drop_mutex);
+
+ err = btrfs_update_root(trans,
+ tree_root,
+ &dirty->root->root_key,
+ &dirty->root->root_item);
+ if (err)
+ ret = err;
+ nr = trans->blocks_used;
+ ret = btrfs_end_transaction(trans, tree_root);
+ BUG_ON(ret);
+
+ btrfs_btree_balance_dirty(tree_root, nr);
+ cond_resched();
+ }
+ BUG_ON(ret);
+ atomic_dec(&root->fs_info->throttles);
+ wake_up(&root->fs_info->transaction_throttle);
+
+ num_bytes -= btrfs_root_used(&dirty->root->root_item);
+ bytes_used = btrfs_root_used(&root->root_item);
+ if (num_bytes) {
+ btrfs_record_root_in_trans(root);
+ btrfs_set_root_used(&root->root_item,
+ bytes_used - num_bytes);
+ }
+
+ ret = btrfs_del_root(trans, tree_root, &dirty->root->root_key);
+ if (ret) {
+ BUG();
+ break;
+ }
+ mutex_unlock(&root->fs_info->drop_mutex);
+
+ spin_lock(&root->list_lock);
+ list_del_init(&dirty->root->dead_list);
+ if (!list_empty(&root->dead_list)) {
+ struct btrfs_root *oldest;
+ oldest = list_entry(root->dead_list.prev,
+ struct btrfs_root, dead_list);
+ max_useless = oldest->root_key.offset - 1;
+ } else {
+ max_useless = root->root_key.offset - 1;
+ }
+ spin_unlock(&root->list_lock);
+
+ nr = trans->blocks_used;
+ ret = btrfs_end_transaction(trans, tree_root);
+ BUG_ON(ret);
+
+ ret = btrfs_remove_leaf_refs(root, max_useless, 0);
+ BUG_ON(ret);
+
+ free_extent_buffer(dirty->root->node);
+ kfree(dirty->root);
+ kfree(dirty);
+
+ btrfs_btree_balance_dirty(tree_root, nr);
+ cond_resched();
+ }
+ return ret;
+}
+
+/*
+ * new snapshots need to be created at a very specific time in the
+ * transaction commit. This does the actual creation
+ */
+static noinline int create_pending_snapshot(struct btrfs_trans_handle *trans,
+ struct btrfs_fs_info *fs_info,
+ struct btrfs_pending_snapshot *pending)
+{
+ struct btrfs_key key;
+ struct btrfs_root_item *new_root_item;
+ struct btrfs_root *tree_root = fs_info->tree_root;
+ struct btrfs_root *root = pending->root;
+ struct extent_buffer *tmp;
+ struct extent_buffer *old;
+ int ret;
+ u64 objectid;
+
+ new_root_item = kmalloc(sizeof(*new_root_item), GFP_NOFS);
+ if (!new_root_item) {
+ ret = -ENOMEM;
+ goto fail;
+ }
+ ret = btrfs_find_free_objectid(trans, tree_root, 0, &objectid);
+ if (ret)
+ goto fail;
+
+ btrfs_record_root_in_trans(root);
+ btrfs_set_root_last_snapshot(&root->root_item, trans->transid);
+ memcpy(new_root_item, &root->root_item, sizeof(*new_root_item));
+
+ key.objectid = objectid;
+ key.offset = trans->transid;
+ btrfs_set_key_type(&key, BTRFS_ROOT_ITEM_KEY);
+
+ old = btrfs_lock_root_node(root);
+ btrfs_cow_block(trans, root, old, NULL, 0, &old, 0);
+
+ btrfs_copy_root(trans, root, old, &tmp, objectid);
+ btrfs_tree_unlock(old);
+ free_extent_buffer(old);
+
+ btrfs_set_root_bytenr(new_root_item, tmp->start);
+ btrfs_set_root_level(new_root_item, btrfs_header_level(tmp));
+ btrfs_set_root_generation(new_root_item, trans->transid);
+ ret = btrfs_insert_root(trans, root->fs_info->tree_root, &key,
+ new_root_item);
+ btrfs_tree_unlock(tmp);
+ free_extent_buffer(tmp);
+ if (ret)
+ goto fail;
+
+ key.offset = (u64)-1;
+ memcpy(&pending->root_key, &key, sizeof(key));
+fail:
+ kfree(new_root_item);
+ return ret;
+}
+
+static noinline int finish_pending_snapshot(struct btrfs_fs_info *fs_info,
+ struct btrfs_pending_snapshot *pending)
+{
+ int ret;
+ int namelen;
+ u64 index = 0;
+ struct btrfs_trans_handle *trans;
+ struct inode *parent_inode;
+ struct inode *inode;
+ struct btrfs_root *parent_root;
+
+ parent_inode = pending->dentry->d_parent->d_inode;
+ parent_root = BTRFS_I(parent_inode)->root;
+ trans = btrfs_join_transaction(parent_root, 1);
+
+ /*
+ * insert the directory item
+ */
+ namelen = strlen(pending->name);
+ ret = btrfs_set_inode_index(parent_inode, &index);
+ ret = btrfs_insert_dir_item(trans, parent_root,
+ pending->name, namelen,
+ parent_inode->i_ino,
+ &pending->root_key, BTRFS_FT_DIR, index);
+
+ if (ret)
+ goto fail;
+
+ btrfs_i_size_write(parent_inode, parent_inode->i_size + namelen * 2);
+ ret = btrfs_update_inode(trans, parent_root, parent_inode);
+ BUG_ON(ret);
+
+ /* add the backref first */
+ ret = btrfs_add_root_ref(trans, parent_root->fs_info->tree_root,
+ pending->root_key.objectid,
+ BTRFS_ROOT_BACKREF_KEY,
+ parent_root->root_key.objectid,
+ parent_inode->i_ino, index, pending->name,
+ namelen);
+
+ BUG_ON(ret);
+
+ /* now add the forward ref */
+ ret = btrfs_add_root_ref(trans, parent_root->fs_info->tree_root,
+ parent_root->root_key.objectid,
+ BTRFS_ROOT_REF_KEY,
+ pending->root_key.objectid,
+ parent_inode->i_ino, index, pending->name,
+ namelen);
+
+ inode = btrfs_lookup_dentry(parent_inode, pending->dentry);
+ d_instantiate(pending->dentry, inode);
+fail:
+ btrfs_end_transaction(trans, fs_info->fs_root);
+ return ret;
+}
+
+/*
+ * create all the snapshots we've scheduled for creation
+ */
+static noinline int create_pending_snapshots(struct btrfs_trans_handle *trans,
+ struct btrfs_fs_info *fs_info)
+{
+ struct btrfs_pending_snapshot *pending;
+ struct list_head *head = &trans->transaction->pending_snapshots;
+ struct list_head *cur;
+ int ret;
+
+ list_for_each(cur, head) {
+ pending = list_entry(cur, struct btrfs_pending_snapshot, list);
+ ret = create_pending_snapshot(trans, fs_info, pending);
+ BUG_ON(ret);
+ }
+ return 0;
+}
+
+static noinline int finish_pending_snapshots(struct btrfs_trans_handle *trans,
+ struct btrfs_fs_info *fs_info)
+{
+ struct btrfs_pending_snapshot *pending;
+ struct list_head *head = &trans->transaction->pending_snapshots;
+ int ret;
+
+ while (!list_empty(head)) {
+ pending = list_entry(head->next,
+ struct btrfs_pending_snapshot, list);
+ ret = finish_pending_snapshot(fs_info, pending);
+ BUG_ON(ret);
+ list_del(&pending->list);
+ kfree(pending->name);
+ kfree(pending);
+ }
+ return 0;
+}
+
+int btrfs_commit_transaction(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root)
+{
+ unsigned long joined = 0;
+ unsigned long timeout = 1;
+ struct btrfs_transaction *cur_trans;
+ struct btrfs_transaction *prev_trans = NULL;
+ struct btrfs_root *chunk_root = root->fs_info->chunk_root;
+ struct list_head dirty_fs_roots;
+ struct extent_io_tree *pinned_copy;
+ DEFINE_WAIT(wait);
+ int ret;
+
+ INIT_LIST_HEAD(&dirty_fs_roots);
+ mutex_lock(&root->fs_info->trans_mutex);
+ if (trans->transaction->in_commit) {
+ cur_trans = trans->transaction;
+ trans->transaction->use_count++;
+ mutex_unlock(&root->fs_info->trans_mutex);
+ btrfs_end_transaction(trans, root);
+
+ ret = wait_for_commit(root, cur_trans);
+ BUG_ON(ret);
+
+ mutex_lock(&root->fs_info->trans_mutex);
+ put_transaction(cur_trans);
+ mutex_unlock(&root->fs_info->trans_mutex);
+
+ return 0;
+ }
+
+ pinned_copy = kmalloc(sizeof(*pinned_copy), GFP_NOFS);
+ if (!pinned_copy)
+ return -ENOMEM;
+
+ extent_io_tree_init(pinned_copy,
+ root->fs_info->btree_inode->i_mapping, GFP_NOFS);
+
+ trans->transaction->in_commit = 1;
+ trans->transaction->blocked = 1;
+ cur_trans = trans->transaction;
+ if (cur_trans->list.prev != &root->fs_info->trans_list) {
+ prev_trans = list_entry(cur_trans->list.prev,
+ struct btrfs_transaction, list);
+ if (!prev_trans->commit_done) {
+ prev_trans->use_count++;
+ mutex_unlock(&root->fs_info->trans_mutex);
+
+ wait_for_commit(root, prev_trans);
+
+ mutex_lock(&root->fs_info->trans_mutex);
+ put_transaction(prev_trans);
+ }
+ }
+
+ do {
+ int snap_pending = 0;
+ joined = cur_trans->num_joined;
+ if (!list_empty(&trans->transaction->pending_snapshots))
+ snap_pending = 1;
+
+ WARN_ON(cur_trans != trans->transaction);
+ prepare_to_wait(&cur_trans->writer_wait, &wait,
+ TASK_UNINTERRUPTIBLE);
+
+ if (cur_trans->num_writers > 1)
+ timeout = MAX_SCHEDULE_TIMEOUT;
+ else
+ timeout = 1;
+
+ mutex_unlock(&root->fs_info->trans_mutex);
+
+ if (snap_pending) {
+ ret = btrfs_wait_ordered_extents(root, 1);
+ BUG_ON(ret);
+ }
+
+ schedule_timeout(timeout);
+
+ mutex_lock(&root->fs_info->trans_mutex);
+ finish_wait(&cur_trans->writer_wait, &wait);
+ } while (cur_trans->num_writers > 1 ||
+ (cur_trans->num_joined != joined));
+
+ ret = create_pending_snapshots(trans, root->fs_info);
+ BUG_ON(ret);
+
+ WARN_ON(cur_trans != trans->transaction);
+
+ /* btrfs_commit_tree_roots is responsible for getting the
+ * various roots consistent with each other. Every pointer
+ * in the tree of tree roots has to point to the most up to date
+ * root for every subvolume and other tree. So, we have to keep
+ * the tree logging code from jumping in and changing any
+ * of the trees.
+ *
+ * At this point in the commit, there can't be any tree-log
+ * writers, but a little lower down we drop the trans mutex
+ * and let new people in. By holding the tree_log_mutex
+ * from now until after the super is written, we avoid races
+ * with the tree-log code.
+ */
+ mutex_lock(&root->fs_info->tree_log_mutex);
+ /*
+ * keep tree reloc code from adding new reloc trees
+ */
+ mutex_lock(&root->fs_info->tree_reloc_mutex);
+
+
+ ret = add_dirty_roots(trans, &root->fs_info->fs_roots_radix,
+ &dirty_fs_roots);
+ BUG_ON(ret);
+
+ /* add_dirty_roots gets rid of all the tree log roots, it is now
+ * safe to free the root of tree log roots
+ */
+ btrfs_free_log_root_tree(trans, root->fs_info);
+
+ ret = btrfs_commit_tree_roots(trans, root);
+ BUG_ON(ret);
+
+ cur_trans = root->fs_info->running_transaction;
+ spin_lock(&root->fs_info->new_trans_lock);
+ root->fs_info->running_transaction = NULL;
+ spin_unlock(&root->fs_info->new_trans_lock);
+ btrfs_set_super_generation(&root->fs_info->super_copy,
+ cur_trans->transid);
+ btrfs_set_super_root(&root->fs_info->super_copy,
+ root->fs_info->tree_root->node->start);
+ btrfs_set_super_root_level(&root->fs_info->super_copy,
+ btrfs_header_level(root->fs_info->tree_root->node));
+
+ btrfs_set_super_chunk_root(&root->fs_info->super_copy,
+ chunk_root->node->start);
+ btrfs_set_super_chunk_root_level(&root->fs_info->super_copy,
+ btrfs_header_level(chunk_root->node));
+ btrfs_set_super_chunk_root_generation(&root->fs_info->super_copy,
+ btrfs_header_generation(chunk_root->node));
+
+ if (!root->fs_info->log_root_recovering) {
+ btrfs_set_super_log_root(&root->fs_info->super_copy, 0);
+ btrfs_set_super_log_root_level(&root->fs_info->super_copy, 0);
+ }
+
+ memcpy(&root->fs_info->super_for_commit, &root->fs_info->super_copy,
+ sizeof(root->fs_info->super_copy));
+
+ btrfs_copy_pinned(root, pinned_copy);
+
+ trans->transaction->blocked = 0;
+ wake_up(&root->fs_info->transaction_throttle);
+ wake_up(&root->fs_info->transaction_wait);
+
+ mutex_unlock(&root->fs_info->trans_mutex);
+ ret = btrfs_write_and_wait_transaction(trans, root);
+ BUG_ON(ret);
+ write_ctree_super(trans, root, 0);
+
+ /*
+ * the super is written, we can safely allow the tree-loggers
+ * to go about their business
+ */
+ mutex_unlock(&root->fs_info->tree_log_mutex);
+
+ btrfs_finish_extent_commit(trans, root, pinned_copy);
+ kfree(pinned_copy);
+
+ btrfs_drop_dead_reloc_roots(root);
+ mutex_unlock(&root->fs_info->tree_reloc_mutex);
+
+ /* do the directory inserts of any pending snapshot creations */
+ finish_pending_snapshots(trans, root->fs_info);
+
+ mutex_lock(&root->fs_info->trans_mutex);
+
+ cur_trans->commit_done = 1;
+ root->fs_info->last_trans_committed = cur_trans->transid;
+ wake_up(&cur_trans->commit_wait);
+
+ put_transaction(cur_trans);
+ put_transaction(cur_trans);
+
+ list_splice_init(&dirty_fs_roots, &root->fs_info->dead_roots);
+ if (root->fs_info->closing)
+ list_splice_init(&root->fs_info->dead_roots, &dirty_fs_roots);
+
+ mutex_unlock(&root->fs_info->trans_mutex);
+
+ kmem_cache_free(btrfs_trans_handle_cachep, trans);
+
+ if (root->fs_info->closing)
+ drop_dirty_roots(root->fs_info->tree_root, &dirty_fs_roots);
+ return ret;
+}
+
+/*
+ * interface function to delete all the snapshots we have scheduled for deletion
+ */
+int btrfs_clean_old_snapshots(struct btrfs_root *root)
+{
+ struct list_head dirty_roots;
+ INIT_LIST_HEAD(&dirty_roots);
+again:
+ mutex_lock(&root->fs_info->trans_mutex);
+ list_splice_init(&root->fs_info->dead_roots, &dirty_roots);
+ mutex_unlock(&root->fs_info->trans_mutex);
+
+ if (!list_empty(&dirty_roots)) {
+ drop_dirty_roots(root, &dirty_roots);
+ goto again;
+ }
+ return 0;
+}
diff --git a/fs/btrfs/transaction.h b/fs/btrfs/transaction.h
new file mode 100644
index 0000000..ea29211
--- /dev/null
+++ b/fs/btrfs/transaction.h
@@ -0,0 +1,106 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __BTRFS_TRANSACTION__
+#define __BTRFS_TRANSACTION__
+#include "btrfs_inode.h"
+
+struct btrfs_transaction {
+ u64 transid;
+ unsigned long num_writers;
+ unsigned long num_joined;
+ int in_commit;
+ int use_count;
+ int commit_done;
+ int blocked;
+ struct list_head list;
+ struct extent_io_tree dirty_pages;
+ unsigned long start_time;
+ wait_queue_head_t writer_wait;
+ wait_queue_head_t commit_wait;
+ struct list_head pending_snapshots;
+};
+
+struct btrfs_trans_handle {
+ u64 transid;
+ unsigned long blocks_reserved;
+ unsigned long blocks_used;
+ struct btrfs_transaction *transaction;
+ u64 block_group;
+ u64 alloc_exclude_start;
+ u64 alloc_exclude_nr;
+};
+
+struct btrfs_pending_snapshot {
+ struct dentry *dentry;
+ struct btrfs_root *root;
+ char *name;
+ struct btrfs_key root_key;
+ struct list_head list;
+};
+
+struct btrfs_dirty_root {
+ struct list_head list;
+ struct btrfs_root *root;
+ struct btrfs_root *latest_root;
+};
+
+static inline void btrfs_set_trans_block_group(struct btrfs_trans_handle *trans,
+ struct inode *inode)
+{
+ trans->block_group = BTRFS_I(inode)->block_group;
+}
+
+static inline void btrfs_update_inode_block_group(
+ struct btrfs_trans_handle *trans,
+ struct inode *inode)
+{
+ BTRFS_I(inode)->block_group = trans->block_group;
+}
+
+static inline void btrfs_set_inode_last_trans(struct btrfs_trans_handle *trans,
+ struct inode *inode)
+{
+ BTRFS_I(inode)->last_trans = trans->transaction->transid;
+}
+
+int btrfs_end_transaction(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root);
+struct btrfs_trans_handle *btrfs_start_transaction(struct btrfs_root *root,
+ int num_blocks);
+struct btrfs_trans_handle *btrfs_join_transaction(struct btrfs_root *root,
+ int num_blocks);
+struct btrfs_trans_handle *btrfs_start_ioctl_transaction(struct btrfs_root *r,
+ int num_blocks);
+int btrfs_write_and_wait_transaction(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root);
+int btrfs_commit_tree_roots(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root);
+
+int btrfs_add_dead_root(struct btrfs_root *root, struct btrfs_root *latest);
+int btrfs_defrag_root(struct btrfs_root *root, int cacheonly);
+int btrfs_clean_old_snapshots(struct btrfs_root *root);
+int btrfs_commit_transaction(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root);
+int btrfs_end_transaction_throttle(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root);
+void btrfs_throttle(struct btrfs_root *root);
+int btrfs_record_root_in_trans(struct btrfs_root *root);
+int btrfs_write_and_wait_marked_extents(struct btrfs_root *root,
+ struct extent_io_tree *dirty_pages);
+#endif
diff --git a/fs/btrfs/tree-defrag.c b/fs/btrfs/tree-defrag.c
new file mode 100644
index 0000000..3e8358c
--- /dev/null
+++ b/fs/btrfs/tree-defrag.c
@@ -0,0 +1,147 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/sched.h>
+#include "ctree.h"
+#include "disk-io.h"
+#include "print-tree.h"
+#include "transaction.h"
+#include "locking.h"
+
+/* defrag all the leaves in a given btree. If cache_only == 1, don't read
+ * things from disk, otherwise read all the leaves and try to get key order to
+ * better reflect disk order
+ */
+
+int btrfs_defrag_leaves(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, int cache_only)
+{
+ struct btrfs_path *path = NULL;
+ struct btrfs_key key;
+ int ret = 0;
+ int wret;
+ int level;
+ int orig_level;
+ int is_extent = 0;
+ int next_key_ret = 0;
+ u64 last_ret = 0;
+ u64 min_trans = 0;
+
+ if (cache_only)
+ goto out;
+
+ if (root->fs_info->extent_root == root) {
+ /*
+ * there's recursion here right now in the tree locking,
+ * we can't defrag the extent root without deadlock
+ */
+ goto out;
+ }
+
+ if (root->ref_cows == 0 && !is_extent)
+ goto out;
+
+ if (btrfs_test_opt(root, SSD))
+ goto out;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ level = btrfs_header_level(root->node);
+ orig_level = level;
+
+ if (level == 0)
+ goto out;
+
+ if (root->defrag_progress.objectid == 0) {
+ struct extent_buffer *root_node;
+ u32 nritems;
+
+ root_node = btrfs_lock_root_node(root);
+ nritems = btrfs_header_nritems(root_node);
+ root->defrag_max.objectid = 0;
+ /* from above we know this is not a leaf */
+ btrfs_node_key_to_cpu(root_node, &root->defrag_max,
+ nritems - 1);
+ btrfs_tree_unlock(root_node);
+ free_extent_buffer(root_node);
+ memset(&key, 0, sizeof(key));
+ } else {
+ memcpy(&key, &root->defrag_progress, sizeof(key));
+ }
+
+ path->keep_locks = 1;
+ if (cache_only)
+ min_trans = root->defrag_trans_start;
+
+ ret = btrfs_search_forward(root, &key, NULL, path,
+ cache_only, min_trans);
+ if (ret < 0)
+ goto out;
+ if (ret > 0) {
+ ret = 0;
+ goto out;
+ }
+ btrfs_release_path(root, path);
+ wret = btrfs_search_slot(trans, root, &key, path, 0, 1);
+
+ if (wret < 0) {
+ ret = wret;
+ goto out;
+ }
+ if (!path->nodes[1]) {
+ ret = 0;
+ goto out;
+ }
+ path->slots[1] = btrfs_header_nritems(path->nodes[1]);
+ next_key_ret = btrfs_find_next_key(root, path, &key, 1, cache_only,
+ min_trans);
+ ret = btrfs_realloc_node(trans, root,
+ path->nodes[1], 0,
+ cache_only, &last_ret,
+ &root->defrag_progress);
+ WARN_ON(ret && ret != -EAGAIN);
+ if (next_key_ret == 0) {
+ memcpy(&root->defrag_progress, &key, sizeof(key));
+ ret = -EAGAIN;
+ }
+
+ btrfs_release_path(root, path);
+ if (is_extent)
+ btrfs_extent_post_op(trans, root);
+out:
+ if (path)
+ btrfs_free_path(path);
+ if (ret == -EAGAIN) {
+ if (root->defrag_max.objectid > root->defrag_progress.objectid)
+ goto done;
+ if (root->defrag_max.type > root->defrag_progress.type)
+ goto done;
+ if (root->defrag_max.offset > root->defrag_progress.offset)
+ goto done;
+ ret = 0;
+ }
+done:
+ if (ret != -EAGAIN) {
+ memset(&root->defrag_progress, 0,
+ sizeof(root->defrag_progress));
+ root->defrag_trans_start = trans->transid;
+ }
+ return ret;
+}
diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c
new file mode 100644
index 0000000..d81cda2
--- /dev/null
+++ b/fs/btrfs/tree-log.c
@@ -0,0 +1,2898 @@
+/*
+ * Copyright (C) 2008 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/sched.h>
+#include "ctree.h"
+#include "transaction.h"
+#include "disk-io.h"
+#include "locking.h"
+#include "print-tree.h"
+#include "compat.h"
+#include "tree-log.h"
+
+/* magic values for the inode_only field in btrfs_log_inode:
+ *
+ * LOG_INODE_ALL means to log everything
+ * LOG_INODE_EXISTS means to log just enough to recreate the inode
+ * during log replay
+ */
+#define LOG_INODE_ALL 0
+#define LOG_INODE_EXISTS 1
+
+/*
+ * stages for the tree walking. The first
+ * stage (0) is to only pin down the blocks we find
+ * the second stage (1) is to make sure that all the inodes
+ * we find in the log are created in the subvolume.
+ *
+ * The last stage is to deal with directories and links and extents
+ * and all the other fun semantics
+ */
+#define LOG_WALK_PIN_ONLY 0
+#define LOG_WALK_REPLAY_INODES 1
+#define LOG_WALK_REPLAY_ALL 2
+
+static int __btrfs_log_inode(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct inode *inode,
+ int inode_only);
+static int link_to_fixup_dir(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, u64 objectid);
+
+/*
+ * tree logging is a special write ahead log used to make sure that
+ * fsyncs and O_SYNCs can happen without doing full tree commits.
+ *
+ * Full tree commits are expensive because they require commonly
+ * modified blocks to be recowed, creating many dirty pages in the
+ * extent tree an 4x-6x higher write load than ext3.
+ *
+ * Instead of doing a tree commit on every fsync, we use the
+ * key ranges and transaction ids to find items for a given file or directory
+ * that have changed in this transaction. Those items are copied into
+ * a special tree (one per subvolume root), that tree is written to disk
+ * and then the fsync is considered complete.
+ *
+ * After a crash, items are copied out of the log-tree back into the
+ * subvolume tree. Any file data extents found are recorded in the extent
+ * allocation tree, and the log-tree freed.
+ *
+ * The log tree is read three times, once to pin down all the extents it is
+ * using in ram and once, once to create all the inodes logged in the tree
+ * and once to do all the other items.
+ */
+
+/*
+ * btrfs_add_log_tree adds a new per-subvolume log tree into the
+ * tree of log tree roots. This must be called with a tree log transaction
+ * running (see start_log_trans).
+ */
+static int btrfs_add_log_tree(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root)
+{
+ struct btrfs_key key;
+ struct btrfs_root_item root_item;
+ struct btrfs_inode_item *inode_item;
+ struct extent_buffer *leaf;
+ struct btrfs_root *new_root = root;
+ int ret;
+ u64 objectid = root->root_key.objectid;
+
+ leaf = btrfs_alloc_free_block(trans, root, root->leafsize, 0,
+ BTRFS_TREE_LOG_OBJECTID,
+ trans->transid, 0, 0, 0);
+ if (IS_ERR(leaf)) {
+ ret = PTR_ERR(leaf);
+ return ret;
+ }
+
+ btrfs_set_header_nritems(leaf, 0);
+ btrfs_set_header_level(leaf, 0);
+ btrfs_set_header_bytenr(leaf, leaf->start);
+ btrfs_set_header_generation(leaf, trans->transid);
+ btrfs_set_header_owner(leaf, BTRFS_TREE_LOG_OBJECTID);
+
+ write_extent_buffer(leaf, root->fs_info->fsid,
+ (unsigned long)btrfs_header_fsid(leaf),
+ BTRFS_FSID_SIZE);
+ btrfs_mark_buffer_dirty(leaf);
+
+ inode_item = &root_item.inode;
+ memset(inode_item, 0, sizeof(*inode_item));
+ inode_item->generation = cpu_to_le64(1);
+ inode_item->size = cpu_to_le64(3);
+ inode_item->nlink = cpu_to_le32(1);
+ inode_item->nbytes = cpu_to_le64(root->leafsize);
+ inode_item->mode = cpu_to_le32(S_IFDIR | 0755);
+
+ btrfs_set_root_bytenr(&root_item, leaf->start);
+ btrfs_set_root_generation(&root_item, trans->transid);
+ btrfs_set_root_level(&root_item, 0);
+ btrfs_set_root_refs(&root_item, 0);
+ btrfs_set_root_used(&root_item, 0);
+
+ memset(&root_item.drop_progress, 0, sizeof(root_item.drop_progress));
+ root_item.drop_level = 0;
+
+ btrfs_tree_unlock(leaf);
+ free_extent_buffer(leaf);
+ leaf = NULL;
+
+ btrfs_set_root_dirid(&root_item, 0);
+
+ key.objectid = BTRFS_TREE_LOG_OBJECTID;
+ key.offset = objectid;
+ btrfs_set_key_type(&key, BTRFS_ROOT_ITEM_KEY);
+ ret = btrfs_insert_root(trans, root->fs_info->log_root_tree, &key,
+ &root_item);
+ if (ret)
+ goto fail;
+
+ new_root = btrfs_read_fs_root_no_radix(root->fs_info->log_root_tree,
+ &key);
+ BUG_ON(!new_root);
+
+ WARN_ON(root->log_root);
+ root->log_root = new_root;
+
+ /*
+ * log trees do not get reference counted because they go away
+ * before a real commit is actually done. They do store pointers
+ * to file data extents, and those reference counts still get
+ * updated (along with back refs to the log tree).
+ */
+ new_root->ref_cows = 0;
+ new_root->last_trans = trans->transid;
+
+ /*
+ * we need to make sure the root block for this new tree
+ * is marked as dirty in the dirty_log_pages tree. This
+ * is how it gets flushed down to disk at tree log commit time.
+ *
+ * the tree logging mutex keeps others from coming in and changing
+ * the new_root->node, so we can safely access it here
+ */
+ set_extent_dirty(&new_root->dirty_log_pages, new_root->node->start,
+ new_root->node->start + new_root->node->len - 1,
+ GFP_NOFS);
+
+fail:
+ return ret;
+}
+
+/*
+ * start a sub transaction and setup the log tree
+ * this increments the log tree writer count to make the people
+ * syncing the tree wait for us to finish
+ */
+static int start_log_trans(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root)
+{
+ int ret;
+ mutex_lock(&root->fs_info->tree_log_mutex);
+ if (!root->fs_info->log_root_tree) {
+ ret = btrfs_init_log_root_tree(trans, root->fs_info);
+ BUG_ON(ret);
+ }
+ if (!root->log_root) {
+ ret = btrfs_add_log_tree(trans, root);
+ BUG_ON(ret);
+ }
+ atomic_inc(&root->fs_info->tree_log_writers);
+ root->fs_info->tree_log_batch++;
+ mutex_unlock(&root->fs_info->tree_log_mutex);
+ return 0;
+}
+
+/*
+ * returns 0 if there was a log transaction running and we were able
+ * to join, or returns -ENOENT if there were not transactions
+ * in progress
+ */
+static int join_running_log_trans(struct btrfs_root *root)
+{
+ int ret = -ENOENT;
+
+ smp_mb();
+ if (!root->log_root)
+ return -ENOENT;
+
+ mutex_lock(&root->fs_info->tree_log_mutex);
+ if (root->log_root) {
+ ret = 0;
+ atomic_inc(&root->fs_info->tree_log_writers);
+ root->fs_info->tree_log_batch++;
+ }
+ mutex_unlock(&root->fs_info->tree_log_mutex);
+ return ret;
+}
+
+/*
+ * indicate we're done making changes to the log tree
+ * and wake up anyone waiting to do a sync
+ */
+static int end_log_trans(struct btrfs_root *root)
+{
+ atomic_dec(&root->fs_info->tree_log_writers);
+ smp_mb();
+ if (waitqueue_active(&root->fs_info->tree_log_wait))
+ wake_up(&root->fs_info->tree_log_wait);
+ return 0;
+}
+
+
+/*
+ * the walk control struct is used to pass state down the chain when
+ * processing the log tree. The stage field tells us which part
+ * of the log tree processing we are currently doing. The others
+ * are state fields used for that specific part
+ */
+struct walk_control {
+ /* should we free the extent on disk when done? This is used
+ * at transaction commit time while freeing a log tree
+ */
+ int free;
+
+ /* should we write out the extent buffer? This is used
+ * while flushing the log tree to disk during a sync
+ */
+ int write;
+
+ /* should we wait for the extent buffer io to finish? Also used
+ * while flushing the log tree to disk for a sync
+ */
+ int wait;
+
+ /* pin only walk, we record which extents on disk belong to the
+ * log trees
+ */
+ int pin;
+
+ /* what stage of the replay code we're currently in */
+ int stage;
+
+ /* the root we are currently replaying */
+ struct btrfs_root *replay_dest;
+
+ /* the trans handle for the current replay */
+ struct btrfs_trans_handle *trans;
+
+ /* the function that gets used to process blocks we find in the
+ * tree. Note the extent_buffer might not be up to date when it is
+ * passed in, and it must be checked or read if you need the data
+ * inside it
+ */
+ int (*process_func)(struct btrfs_root *log, struct extent_buffer *eb,
+ struct walk_control *wc, u64 gen);
+};
+
+/*
+ * process_func used to pin down extents, write them or wait on them
+ */
+static int process_one_buffer(struct btrfs_root *log,
+ struct extent_buffer *eb,
+ struct walk_control *wc, u64 gen)
+{
+ if (wc->pin) {
+ mutex_lock(&log->fs_info->pinned_mutex);
+ btrfs_update_pinned_extents(log->fs_info->extent_root,
+ eb->start, eb->len, 1);
+ mutex_unlock(&log->fs_info->pinned_mutex);
+ }
+
+ if (btrfs_buffer_uptodate(eb, gen)) {
+ if (wc->write)
+ btrfs_write_tree_block(eb);
+ if (wc->wait)
+ btrfs_wait_tree_block_writeback(eb);
+ }
+ return 0;
+}
+
+/*
+ * Item overwrite used by replay and tree logging. eb, slot and key all refer
+ * to the src data we are copying out.
+ *
+ * root is the tree we are copying into, and path is a scratch
+ * path for use in this function (it should be released on entry and
+ * will be released on exit).
+ *
+ * If the key is already in the destination tree the existing item is
+ * overwritten. If the existing item isn't big enough, it is extended.
+ * If it is too large, it is truncated.
+ *
+ * If the key isn't in the destination yet, a new item is inserted.
+ */
+static noinline int overwrite_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct extent_buffer *eb, int slot,
+ struct btrfs_key *key)
+{
+ int ret;
+ u32 item_size;
+ u64 saved_i_size = 0;
+ int save_old_i_size = 0;
+ unsigned long src_ptr;
+ unsigned long dst_ptr;
+ int overwrite_root = 0;
+
+ if (root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID)
+ overwrite_root = 1;
+
+ item_size = btrfs_item_size_nr(eb, slot);
+ src_ptr = btrfs_item_ptr_offset(eb, slot);
+
+ /* look for the key in the destination tree */
+ ret = btrfs_search_slot(NULL, root, key, path, 0, 0);
+ if (ret == 0) {
+ char *src_copy;
+ char *dst_copy;
+ u32 dst_size = btrfs_item_size_nr(path->nodes[0],
+ path->slots[0]);
+ if (dst_size != item_size)
+ goto insert;
+
+ if (item_size == 0) {
+ btrfs_release_path(root, path);
+ return 0;
+ }
+ dst_copy = kmalloc(item_size, GFP_NOFS);
+ src_copy = kmalloc(item_size, GFP_NOFS);
+
+ read_extent_buffer(eb, src_copy, src_ptr, item_size);
+
+ dst_ptr = btrfs_item_ptr_offset(path->nodes[0], path->slots[0]);
+ read_extent_buffer(path->nodes[0], dst_copy, dst_ptr,
+ item_size);
+ ret = memcmp(dst_copy, src_copy, item_size);
+
+ kfree(dst_copy);
+ kfree(src_copy);
+ /*
+ * they have the same contents, just return, this saves
+ * us from cowing blocks in the destination tree and doing
+ * extra writes that may not have been done by a previous
+ * sync
+ */
+ if (ret == 0) {
+ btrfs_release_path(root, path);
+ return 0;
+ }
+
+ }
+insert:
+ btrfs_release_path(root, path);
+ /* try to insert the key into the destination tree */
+ ret = btrfs_insert_empty_item(trans, root, path,
+ key, item_size);
+
+ /* make sure any existing item is the correct size */
+ if (ret == -EEXIST) {
+ u32 found_size;
+ found_size = btrfs_item_size_nr(path->nodes[0],
+ path->slots[0]);
+ if (found_size > item_size) {
+ btrfs_truncate_item(trans, root, path, item_size, 1);
+ } else if (found_size < item_size) {
+ ret = btrfs_extend_item(trans, root, path,
+ item_size - found_size);
+ BUG_ON(ret);
+ }
+ } else if (ret) {
+ BUG();
+ }
+ dst_ptr = btrfs_item_ptr_offset(path->nodes[0],
+ path->slots[0]);
+
+ /* don't overwrite an existing inode if the generation number
+ * was logged as zero. This is done when the tree logging code
+ * is just logging an inode to make sure it exists after recovery.
+ *
+ * Also, don't overwrite i_size on directories during replay.
+ * log replay inserts and removes directory items based on the
+ * state of the tree found in the subvolume, and i_size is modified
+ * as it goes
+ */
+ if (key->type == BTRFS_INODE_ITEM_KEY && ret == -EEXIST) {
+ struct btrfs_inode_item *src_item;
+ struct btrfs_inode_item *dst_item;
+
+ src_item = (struct btrfs_inode_item *)src_ptr;
+ dst_item = (struct btrfs_inode_item *)dst_ptr;
+
+ if (btrfs_inode_generation(eb, src_item) == 0)
+ goto no_copy;
+
+ if (overwrite_root &&
+ S_ISDIR(btrfs_inode_mode(eb, src_item)) &&
+ S_ISDIR(btrfs_inode_mode(path->nodes[0], dst_item))) {
+ save_old_i_size = 1;
+ saved_i_size = btrfs_inode_size(path->nodes[0],
+ dst_item);
+ }
+ }
+
+ copy_extent_buffer(path->nodes[0], eb, dst_ptr,
+ src_ptr, item_size);
+
+ if (save_old_i_size) {
+ struct btrfs_inode_item *dst_item;
+ dst_item = (struct btrfs_inode_item *)dst_ptr;
+ btrfs_set_inode_size(path->nodes[0], dst_item, saved_i_size);
+ }
+
+ /* make sure the generation is filled in */
+ if (key->type == BTRFS_INODE_ITEM_KEY) {
+ struct btrfs_inode_item *dst_item;
+ dst_item = (struct btrfs_inode_item *)dst_ptr;
+ if (btrfs_inode_generation(path->nodes[0], dst_item) == 0) {
+ btrfs_set_inode_generation(path->nodes[0], dst_item,
+ trans->transid);
+ }
+ }
+no_copy:
+ btrfs_mark_buffer_dirty(path->nodes[0]);
+ btrfs_release_path(root, path);
+ return 0;
+}
+
+/*
+ * simple helper to read an inode off the disk from a given root
+ * This can only be called for subvolume roots and not for the log
+ */
+static noinline struct inode *read_one_inode(struct btrfs_root *root,
+ u64 objectid)
+{
+ struct inode *inode;
+ inode = btrfs_iget_locked(root->fs_info->sb, objectid, root);
+ if (inode->i_state & I_NEW) {
+ BTRFS_I(inode)->root = root;
+ BTRFS_I(inode)->location.objectid = objectid;
+ BTRFS_I(inode)->location.type = BTRFS_INODE_ITEM_KEY;
+ BTRFS_I(inode)->location.offset = 0;
+ btrfs_read_locked_inode(inode);
+ unlock_new_inode(inode);
+
+ }
+ if (is_bad_inode(inode)) {
+ iput(inode);
+ inode = NULL;
+ }
+ return inode;
+}
+
+/* replays a single extent in 'eb' at 'slot' with 'key' into the
+ * subvolume 'root'. path is released on entry and should be released
+ * on exit.
+ *
+ * extents in the log tree have not been allocated out of the extent
+ * tree yet. So, this completes the allocation, taking a reference
+ * as required if the extent already exists or creating a new extent
+ * if it isn't in the extent allocation tree yet.
+ *
+ * The extent is inserted into the file, dropping any existing extents
+ * from the file that overlap the new one.
+ */
+static noinline int replay_one_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct extent_buffer *eb, int slot,
+ struct btrfs_key *key)
+{
+ int found_type;
+ u64 mask = root->sectorsize - 1;
+ u64 extent_end;
+ u64 alloc_hint;
+ u64 start = key->offset;
+ u64 saved_nbytes;
+ struct btrfs_file_extent_item *item;
+ struct inode *inode = NULL;
+ unsigned long size;
+ int ret = 0;
+
+ item = btrfs_item_ptr(eb, slot, struct btrfs_file_extent_item);
+ found_type = btrfs_file_extent_type(eb, item);
+
+ if (found_type == BTRFS_FILE_EXTENT_REG ||
+ found_type == BTRFS_FILE_EXTENT_PREALLOC)
+ extent_end = start + btrfs_file_extent_num_bytes(eb, item);
+ else if (found_type == BTRFS_FILE_EXTENT_INLINE) {
+ size = btrfs_file_extent_inline_len(eb, item);
+ extent_end = (start + size + mask) & ~mask;
+ } else {
+ ret = 0;
+ goto out;
+ }
+
+ inode = read_one_inode(root, key->objectid);
+ if (!inode) {
+ ret = -EIO;
+ goto out;
+ }
+
+ /*
+ * first check to see if we already have this extent in the
+ * file. This must be done before the btrfs_drop_extents run
+ * so we don't try to drop this extent.
+ */
+ ret = btrfs_lookup_file_extent(trans, root, path, inode->i_ino,
+ start, 0);
+
+ if (ret == 0 &&
+ (found_type == BTRFS_FILE_EXTENT_REG ||
+ found_type == BTRFS_FILE_EXTENT_PREALLOC)) {
+ struct btrfs_file_extent_item cmp1;
+ struct btrfs_file_extent_item cmp2;
+ struct btrfs_file_extent_item *existing;
+ struct extent_buffer *leaf;
+
+ leaf = path->nodes[0];
+ existing = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_file_extent_item);
+
+ read_extent_buffer(eb, &cmp1, (unsigned long)item,
+ sizeof(cmp1));
+ read_extent_buffer(leaf, &cmp2, (unsigned long)existing,
+ sizeof(cmp2));
+
+ /*
+ * we already have a pointer to this exact extent,
+ * we don't have to do anything
+ */
+ if (memcmp(&cmp1, &cmp2, sizeof(cmp1)) == 0) {
+ btrfs_release_path(root, path);
+ goto out;
+ }
+ }
+ btrfs_release_path(root, path);
+
+ saved_nbytes = inode_get_bytes(inode);
+ /* drop any overlapping extents */
+ ret = btrfs_drop_extents(trans, root, inode,
+ start, extent_end, start, &alloc_hint);
+ BUG_ON(ret);
+
+ if (found_type == BTRFS_FILE_EXTENT_REG ||
+ found_type == BTRFS_FILE_EXTENT_PREALLOC) {
+ unsigned long dest_offset;
+ struct btrfs_key ins;
+
+ ret = btrfs_insert_empty_item(trans, root, path, key,
+ sizeof(*item));
+ BUG_ON(ret);
+ dest_offset = btrfs_item_ptr_offset(path->nodes[0],
+ path->slots[0]);
+ copy_extent_buffer(path->nodes[0], eb, dest_offset,
+ (unsigned long)item, sizeof(*item));
+
+ ins.objectid = btrfs_file_extent_disk_bytenr(eb, item);
+ ins.offset = btrfs_file_extent_disk_num_bytes(eb, item);
+ ins.type = BTRFS_EXTENT_ITEM_KEY;
+
+ if (ins.objectid > 0) {
+ u64 csum_start;
+ u64 csum_end;
+ LIST_HEAD(ordered_sums);
+ /*
+ * is this extent already allocated in the extent
+ * allocation tree? If so, just add a reference
+ */
+ ret = btrfs_lookup_extent(root, ins.objectid,
+ ins.offset);
+ if (ret == 0) {
+ ret = btrfs_inc_extent_ref(trans, root,
+ ins.objectid, ins.offset,
+ path->nodes[0]->start,
+ root->root_key.objectid,
+ trans->transid, key->objectid);
+ } else {
+ /*
+ * insert the extent pointer in the extent
+ * allocation tree
+ */
+ ret = btrfs_alloc_logged_extent(trans, root,
+ path->nodes[0]->start,
+ root->root_key.objectid,
+ trans->transid, key->objectid,
+ &ins);
+ BUG_ON(ret);
+ }
+ btrfs_release_path(root, path);
+
+ if (btrfs_file_extent_compression(eb, item)) {
+ csum_start = ins.objectid;
+ csum_end = csum_start + ins.offset;
+ } else {
+ csum_start = ins.objectid +
+ btrfs_file_extent_offset(eb, item);
+ csum_end = csum_start +
+ btrfs_file_extent_num_bytes(eb, item);
+ }
+
+ ret = btrfs_lookup_csums_range(root->log_root,
+ csum_start, csum_end - 1,
+ &ordered_sums);
+ BUG_ON(ret);
+ while (!list_empty(&ordered_sums)) {
+ struct btrfs_ordered_sum *sums;
+ sums = list_entry(ordered_sums.next,
+ struct btrfs_ordered_sum,
+ list);
+ ret = btrfs_csum_file_blocks(trans,
+ root->fs_info->csum_root,
+ sums);
+ BUG_ON(ret);
+ list_del(&sums->list);
+ kfree(sums);
+ }
+ } else {
+ btrfs_release_path(root, path);
+ }
+ } else if (found_type == BTRFS_FILE_EXTENT_INLINE) {
+ /* inline extents are easy, we just overwrite them */
+ ret = overwrite_item(trans, root, path, eb, slot, key);
+ BUG_ON(ret);
+ }
+
+ inode_set_bytes(inode, saved_nbytes);
+ btrfs_update_inode(trans, root, inode);
+out:
+ if (inode)
+ iput(inode);
+ return ret;
+}
+
+/*
+ * when cleaning up conflicts between the directory names in the
+ * subvolume, directory names in the log and directory names in the
+ * inode back references, we may have to unlink inodes from directories.
+ *
+ * This is a helper function to do the unlink of a specific directory
+ * item
+ */
+static noinline int drop_one_dir_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct inode *dir,
+ struct btrfs_dir_item *di)
+{
+ struct inode *inode;
+ char *name;
+ int name_len;
+ struct extent_buffer *leaf;
+ struct btrfs_key location;
+ int ret;
+
+ leaf = path->nodes[0];
+
+ btrfs_dir_item_key_to_cpu(leaf, di, &location);
+ name_len = btrfs_dir_name_len(leaf, di);
+ name = kmalloc(name_len, GFP_NOFS);
+ read_extent_buffer(leaf, name, (unsigned long)(di + 1), name_len);
+ btrfs_release_path(root, path);
+
+ inode = read_one_inode(root, location.objectid);
+ BUG_ON(!inode);
+
+ ret = link_to_fixup_dir(trans, root, path, location.objectid);
+ BUG_ON(ret);
+ ret = btrfs_unlink_inode(trans, root, dir, inode, name, name_len);
+ BUG_ON(ret);
+ kfree(name);
+
+ iput(inode);
+ return ret;
+}
+
+/*
+ * helper function to see if a given name and sequence number found
+ * in an inode back reference are already in a directory and correctly
+ * point to this inode
+ */
+static noinline int inode_in_dir(struct btrfs_root *root,
+ struct btrfs_path *path,
+ u64 dirid, u64 objectid, u64 index,
+ const char *name, int name_len)
+{
+ struct btrfs_dir_item *di;
+ struct btrfs_key location;
+ int match = 0;
+
+ di = btrfs_lookup_dir_index_item(NULL, root, path, dirid,
+ index, name, name_len, 0);
+ if (di && !IS_ERR(di)) {
+ btrfs_dir_item_key_to_cpu(path->nodes[0], di, &location);
+ if (location.objectid != objectid)
+ goto out;
+ } else
+ goto out;
+ btrfs_release_path(root, path);
+
+ di = btrfs_lookup_dir_item(NULL, root, path, dirid, name, name_len, 0);
+ if (di && !IS_ERR(di)) {
+ btrfs_dir_item_key_to_cpu(path->nodes[0], di, &location);
+ if (location.objectid != objectid)
+ goto out;
+ } else
+ goto out;
+ match = 1;
+out:
+ btrfs_release_path(root, path);
+ return match;
+}
+
+/*
+ * helper function to check a log tree for a named back reference in
+ * an inode. This is used to decide if a back reference that is
+ * found in the subvolume conflicts with what we find in the log.
+ *
+ * inode backreferences may have multiple refs in a single item,
+ * during replay we process one reference at a time, and we don't
+ * want to delete valid links to a file from the subvolume if that
+ * link is also in the log.
+ */
+static noinline int backref_in_log(struct btrfs_root *log,
+ struct btrfs_key *key,
+ char *name, int namelen)
+{
+ struct btrfs_path *path;
+ struct btrfs_inode_ref *ref;
+ unsigned long ptr;
+ unsigned long ptr_end;
+ unsigned long name_ptr;
+ int found_name_len;
+ int item_size;
+ int ret;
+ int match = 0;
+
+ path = btrfs_alloc_path();
+ ret = btrfs_search_slot(NULL, log, key, path, 0, 0);
+ if (ret != 0)
+ goto out;
+
+ item_size = btrfs_item_size_nr(path->nodes[0], path->slots[0]);
+ ptr = btrfs_item_ptr_offset(path->nodes[0], path->slots[0]);
+ ptr_end = ptr + item_size;
+ while (ptr < ptr_end) {
+ ref = (struct btrfs_inode_ref *)ptr;
+ found_name_len = btrfs_inode_ref_name_len(path->nodes[0], ref);
+ if (found_name_len == namelen) {
+ name_ptr = (unsigned long)(ref + 1);
+ ret = memcmp_extent_buffer(path->nodes[0], name,
+ name_ptr, namelen);
+ if (ret == 0) {
+ match = 1;
+ goto out;
+ }
+ }
+ ptr = (unsigned long)(ref + 1) + found_name_len;
+ }
+out:
+ btrfs_free_path(path);
+ return match;
+}
+
+
+/*
+ * replay one inode back reference item found in the log tree.
+ * eb, slot and key refer to the buffer and key found in the log tree.
+ * root is the destination we are replaying into, and path is for temp
+ * use by this function. (it should be released on return).
+ */
+static noinline int add_inode_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_root *log,
+ struct btrfs_path *path,
+ struct extent_buffer *eb, int slot,
+ struct btrfs_key *key)
+{
+ struct inode *dir;
+ int ret;
+ struct btrfs_key location;
+ struct btrfs_inode_ref *ref;
+ struct btrfs_dir_item *di;
+ struct inode *inode;
+ char *name;
+ int namelen;
+ unsigned long ref_ptr;
+ unsigned long ref_end;
+
+ location.objectid = key->objectid;
+ location.type = BTRFS_INODE_ITEM_KEY;
+ location.offset = 0;
+
+ /*
+ * it is possible that we didn't log all the parent directories
+ * for a given inode. If we don't find the dir, just don't
+ * copy the back ref in. The link count fixup code will take
+ * care of the rest
+ */
+ dir = read_one_inode(root, key->offset);
+ if (!dir)
+ return -ENOENT;
+
+ inode = read_one_inode(root, key->objectid);
+ BUG_ON(!dir);
+
+ ref_ptr = btrfs_item_ptr_offset(eb, slot);
+ ref_end = ref_ptr + btrfs_item_size_nr(eb, slot);
+
+again:
+ ref = (struct btrfs_inode_ref *)ref_ptr;
+
+ namelen = btrfs_inode_ref_name_len(eb, ref);
+ name = kmalloc(namelen, GFP_NOFS);
+ BUG_ON(!name);
+
+ read_extent_buffer(eb, name, (unsigned long)(ref + 1), namelen);
+
+ /* if we already have a perfect match, we're done */
+ if (inode_in_dir(root, path, dir->i_ino, inode->i_ino,
+ btrfs_inode_ref_index(eb, ref),
+ name, namelen)) {
+ goto out;
+ }
+
+ /*
+ * look for a conflicting back reference in the metadata.
+ * if we find one we have to unlink that name of the file
+ * before we add our new link. Later on, we overwrite any
+ * existing back reference, and we don't want to create
+ * dangling pointers in the directory.
+ */
+conflict_again:
+ ret = btrfs_search_slot(NULL, root, key, path, 0, 0);
+ if (ret == 0) {
+ char *victim_name;
+ int victim_name_len;
+ struct btrfs_inode_ref *victim_ref;
+ unsigned long ptr;
+ unsigned long ptr_end;
+ struct extent_buffer *leaf = path->nodes[0];
+
+ /* are we trying to overwrite a back ref for the root directory
+ * if so, just jump out, we're done
+ */
+ if (key->objectid == key->offset)
+ goto out_nowrite;
+
+ /* check all the names in this back reference to see
+ * if they are in the log. if so, we allow them to stay
+ * otherwise they must be unlinked as a conflict
+ */
+ ptr = btrfs_item_ptr_offset(leaf, path->slots[0]);
+ ptr_end = ptr + btrfs_item_size_nr(leaf, path->slots[0]);
+ while (ptr < ptr_end) {
+ victim_ref = (struct btrfs_inode_ref *)ptr;
+ victim_name_len = btrfs_inode_ref_name_len(leaf,
+ victim_ref);
+ victim_name = kmalloc(victim_name_len, GFP_NOFS);
+ BUG_ON(!victim_name);
+
+ read_extent_buffer(leaf, victim_name,
+ (unsigned long)(victim_ref + 1),
+ victim_name_len);
+
+ if (!backref_in_log(log, key, victim_name,
+ victim_name_len)) {
+ btrfs_inc_nlink(inode);
+ btrfs_release_path(root, path);
+ ret = btrfs_unlink_inode(trans, root, dir,
+ inode, victim_name,
+ victim_name_len);
+ kfree(victim_name);
+ btrfs_release_path(root, path);
+ goto conflict_again;
+ }
+ kfree(victim_name);
+ ptr = (unsigned long)(victim_ref + 1) + victim_name_len;
+ }
+ BUG_ON(ret);
+ }
+ btrfs_release_path(root, path);
+
+ /* look for a conflicting sequence number */
+ di = btrfs_lookup_dir_index_item(trans, root, path, dir->i_ino,
+ btrfs_inode_ref_index(eb, ref),
+ name, namelen, 0);
+ if (di && !IS_ERR(di)) {
+ ret = drop_one_dir_item(trans, root, path, dir, di);
+ BUG_ON(ret);
+ }
+ btrfs_release_path(root, path);
+
+
+ /* look for a conflicting name */
+ di = btrfs_lookup_dir_item(trans, root, path, dir->i_ino,
+ name, namelen, 0);
+ if (di && !IS_ERR(di)) {
+ ret = drop_one_dir_item(trans, root, path, dir, di);
+ BUG_ON(ret);
+ }
+ btrfs_release_path(root, path);
+
+ /* insert our name */
+ ret = btrfs_add_link(trans, dir, inode, name, namelen, 0,
+ btrfs_inode_ref_index(eb, ref));
+ BUG_ON(ret);
+
+ btrfs_update_inode(trans, root, inode);
+
+out:
+ ref_ptr = (unsigned long)(ref + 1) + namelen;
+ kfree(name);
+ if (ref_ptr < ref_end)
+ goto again;
+
+ /* finally write the back reference in the inode */
+ ret = overwrite_item(trans, root, path, eb, slot, key);
+ BUG_ON(ret);
+
+out_nowrite:
+ btrfs_release_path(root, path);
+ iput(dir);
+ iput(inode);
+ return 0;
+}
+
+/*
+ * There are a few corners where the link count of the file can't
+ * be properly maintained during replay. So, instead of adding
+ * lots of complexity to the log code, we just scan the backrefs
+ * for any file that has been through replay.
+ *
+ * The scan will update the link count on the inode to reflect the
+ * number of back refs found. If it goes down to zero, the iput
+ * will free the inode.
+ */
+static noinline int fixup_inode_link_count(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct inode *inode)
+{
+ struct btrfs_path *path;
+ int ret;
+ struct btrfs_key key;
+ u64 nlink = 0;
+ unsigned long ptr;
+ unsigned long ptr_end;
+ int name_len;
+
+ key.objectid = inode->i_ino;
+ key.type = BTRFS_INODE_REF_KEY;
+ key.offset = (u64)-1;
+
+ path = btrfs_alloc_path();
+
+ while (1) {
+ ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ if (ret < 0)
+ break;
+ if (ret > 0) {
+ if (path->slots[0] == 0)
+ break;
+ path->slots[0]--;
+ }
+ btrfs_item_key_to_cpu(path->nodes[0], &key,
+ path->slots[0]);
+ if (key.objectid != inode->i_ino ||
+ key.type != BTRFS_INODE_REF_KEY)
+ break;
+ ptr = btrfs_item_ptr_offset(path->nodes[0], path->slots[0]);
+ ptr_end = ptr + btrfs_item_size_nr(path->nodes[0],
+ path->slots[0]);
+ while (ptr < ptr_end) {
+ struct btrfs_inode_ref *ref;
+
+ ref = (struct btrfs_inode_ref *)ptr;
+ name_len = btrfs_inode_ref_name_len(path->nodes[0],
+ ref);
+ ptr = (unsigned long)(ref + 1) + name_len;
+ nlink++;
+ }
+
+ if (key.offset == 0)
+ break;
+ key.offset--;
+ btrfs_release_path(root, path);
+ }
+ btrfs_free_path(path);
+ if (nlink != inode->i_nlink) {
+ inode->i_nlink = nlink;
+ btrfs_update_inode(trans, root, inode);
+ }
+ BTRFS_I(inode)->index_cnt = (u64)-1;
+
+ return 0;
+}
+
+static noinline int fixup_inode_link_counts(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path)
+{
+ int ret;
+ struct btrfs_key key;
+ struct inode *inode;
+
+ key.objectid = BTRFS_TREE_LOG_FIXUP_OBJECTID;
+ key.type = BTRFS_ORPHAN_ITEM_KEY;
+ key.offset = (u64)-1;
+ while (1) {
+ ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
+ if (ret < 0)
+ break;
+
+ if (ret == 1) {
+ if (path->slots[0] == 0)
+ break;
+ path->slots[0]--;
+ }
+
+ btrfs_item_key_to_cpu(path->nodes[0], &key, path->slots[0]);
+ if (key.objectid != BTRFS_TREE_LOG_FIXUP_OBJECTID ||
+ key.type != BTRFS_ORPHAN_ITEM_KEY)
+ break;
+
+ ret = btrfs_del_item(trans, root, path);
+ BUG_ON(ret);
+
+ btrfs_release_path(root, path);
+ inode = read_one_inode(root, key.offset);
+ BUG_ON(!inode);
+
+ ret = fixup_inode_link_count(trans, root, inode);
+ BUG_ON(ret);
+
+ iput(inode);
+
+ if (key.offset == 0)
+ break;
+ key.offset--;
+ }
+ btrfs_release_path(root, path);
+ return 0;
+}
+
+
+/*
+ * record a given inode in the fixup dir so we can check its link
+ * count when replay is done. The link count is incremented here
+ * so the inode won't go away until we check it
+ */
+static noinline int link_to_fixup_dir(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ u64 objectid)
+{
+ struct btrfs_key key;
+ int ret = 0;
+ struct inode *inode;
+
+ inode = read_one_inode(root, objectid);
+ BUG_ON(!inode);
+
+ key.objectid = BTRFS_TREE_LOG_FIXUP_OBJECTID;
+ btrfs_set_key_type(&key, BTRFS_ORPHAN_ITEM_KEY);
+ key.offset = objectid;
+
+ ret = btrfs_insert_empty_item(trans, root, path, &key, 0);
+
+ btrfs_release_path(root, path);
+ if (ret == 0) {
+ btrfs_inc_nlink(inode);
+ btrfs_update_inode(trans, root, inode);
+ } else if (ret == -EEXIST) {
+ ret = 0;
+ } else {
+ BUG();
+ }
+ iput(inode);
+
+ return ret;
+}
+
+/*
+ * when replaying the log for a directory, we only insert names
+ * for inodes that actually exist. This means an fsync on a directory
+ * does not implicitly fsync all the new files in it
+ */
+static noinline int insert_one_name(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ u64 dirid, u64 index,
+ char *name, int name_len, u8 type,
+ struct btrfs_key *location)
+{
+ struct inode *inode;
+ struct inode *dir;
+ int ret;
+
+ inode = read_one_inode(root, location->objectid);
+ if (!inode)
+ return -ENOENT;
+
+ dir = read_one_inode(root, dirid);
+ if (!dir) {
+ iput(inode);
+ return -EIO;
+ }
+ ret = btrfs_add_link(trans, dir, inode, name, name_len, 1, index);
+
+ /* FIXME, put inode into FIXUP list */
+
+ iput(inode);
+ iput(dir);
+ return ret;
+}
+
+/*
+ * take a single entry in a log directory item and replay it into
+ * the subvolume.
+ *
+ * if a conflicting item exists in the subdirectory already,
+ * the inode it points to is unlinked and put into the link count
+ * fix up tree.
+ *
+ * If a name from the log points to a file or directory that does
+ * not exist in the FS, it is skipped. fsyncs on directories
+ * do not force down inodes inside that directory, just changes to the
+ * names or unlinks in a directory.
+ */
+static noinline int replay_one_name(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct extent_buffer *eb,
+ struct btrfs_dir_item *di,
+ struct btrfs_key *key)
+{
+ char *name;
+ int name_len;
+ struct btrfs_dir_item *dst_di;
+ struct btrfs_key found_key;
+ struct btrfs_key log_key;
+ struct inode *dir;
+ u8 log_type;
+ int exists;
+ int ret;
+
+ dir = read_one_inode(root, key->objectid);
+ BUG_ON(!dir);
+
+ name_len = btrfs_dir_name_len(eb, di);
+ name = kmalloc(name_len, GFP_NOFS);
+ log_type = btrfs_dir_type(eb, di);
+ read_extent_buffer(eb, name, (unsigned long)(di + 1),
+ name_len);
+
+ btrfs_dir_item_key_to_cpu(eb, di, &log_key);
+ exists = btrfs_lookup_inode(trans, root, path, &log_key, 0);
+ if (exists == 0)
+ exists = 1;
+ else
+ exists = 0;
+ btrfs_release_path(root, path);
+
+ if (key->type == BTRFS_DIR_ITEM_KEY) {
+ dst_di = btrfs_lookup_dir_item(trans, root, path, key->objectid,
+ name, name_len, 1);
+ } else if (key->type == BTRFS_DIR_INDEX_KEY) {
+ dst_di = btrfs_lookup_dir_index_item(trans, root, path,
+ key->objectid,
+ key->offset, name,
+ name_len, 1);
+ } else {
+ BUG();
+ }
+ if (!dst_di || IS_ERR(dst_di)) {
+ /* we need a sequence number to insert, so we only
+ * do inserts for the BTRFS_DIR_INDEX_KEY types
+ */
+ if (key->type != BTRFS_DIR_INDEX_KEY)
+ goto out;
+ goto insert;
+ }
+
+ btrfs_dir_item_key_to_cpu(path->nodes[0], dst_di, &found_key);
+ /* the existing item matches the logged item */
+ if (found_key.objectid == log_key.objectid &&
+ found_key.type == log_key.type &&
+ found_key.offset == log_key.offset &&
+ btrfs_dir_type(path->nodes[0], dst_di) == log_type) {
+ goto out;
+ }
+
+ /*
+ * don't drop the conflicting directory entry if the inode
+ * for the new entry doesn't exist
+ */
+ if (!exists)
+ goto out;
+
+ ret = drop_one_dir_item(trans, root, path, dir, dst_di);
+ BUG_ON(ret);
+
+ if (key->type == BTRFS_DIR_INDEX_KEY)
+ goto insert;
+out:
+ btrfs_release_path(root, path);
+ kfree(name);
+ iput(dir);
+ return 0;
+
+insert:
+ btrfs_release_path(root, path);
+ ret = insert_one_name(trans, root, path, key->objectid, key->offset,
+ name, name_len, log_type, &log_key);
+
+ if (ret && ret != -ENOENT)
+ BUG();
+ goto out;
+}
+
+/*
+ * find all the names in a directory item and reconcile them into
+ * the subvolume. Only BTRFS_DIR_ITEM_KEY types will have more than
+ * one name in a directory item, but the same code gets used for
+ * both directory index types
+ */
+static noinline int replay_one_dir_item(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path,
+ struct extent_buffer *eb, int slot,
+ struct btrfs_key *key)
+{
+ int ret;
+ u32 item_size = btrfs_item_size_nr(eb, slot);
+ struct btrfs_dir_item *di;
+ int name_len;
+ unsigned long ptr;
+ unsigned long ptr_end;
+
+ ptr = btrfs_item_ptr_offset(eb, slot);
+ ptr_end = ptr + item_size;
+ while (ptr < ptr_end) {
+ di = (struct btrfs_dir_item *)ptr;
+ name_len = btrfs_dir_name_len(eb, di);
+ ret = replay_one_name(trans, root, path, eb, di, key);
+ BUG_ON(ret);
+ ptr = (unsigned long)(di + 1);
+ ptr += name_len;
+ }
+ return 0;
+}
+
+/*
+ * directory replay has two parts. There are the standard directory
+ * items in the log copied from the subvolume, and range items
+ * created in the log while the subvolume was logged.
+ *
+ * The range items tell us which parts of the key space the log
+ * is authoritative for. During replay, if a key in the subvolume
+ * directory is in a logged range item, but not actually in the log
+ * that means it was deleted from the directory before the fsync
+ * and should be removed.
+ */
+static noinline int find_dir_range(struct btrfs_root *root,
+ struct btrfs_path *path,
+ u64 dirid, int key_type,
+ u64 *start_ret, u64 *end_ret)
+{
+ struct btrfs_key key;
+ u64 found_end;
+ struct btrfs_dir_log_item *item;
+ int ret;
+ int nritems;
+
+ if (*start_ret == (u64)-1)
+ return 1;
+
+ key.objectid = dirid;
+ key.type = key_type;
+ key.offset = *start_ret;
+
+ ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ if (ret < 0)
+ goto out;
+ if (ret > 0) {
+ if (path->slots[0] == 0)
+ goto out;
+ path->slots[0]--;
+ }
+ if (ret != 0)
+ btrfs_item_key_to_cpu(path->nodes[0], &key, path->slots[0]);
+
+ if (key.type != key_type || key.objectid != dirid) {
+ ret = 1;
+ goto next;
+ }
+ item = btrfs_item_ptr(path->nodes[0], path->slots[0],
+ struct btrfs_dir_log_item);
+ found_end = btrfs_dir_log_end(path->nodes[0], item);
+
+ if (*start_ret >= key.offset && *start_ret <= found_end) {
+ ret = 0;
+ *start_ret = key.offset;
+ *end_ret = found_end;
+ goto out;
+ }
+ ret = 1;
+next:
+ /* check the next slot in the tree to see if it is a valid item */
+ nritems = btrfs_header_nritems(path->nodes[0]);
+ if (path->slots[0] >= nritems) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret)
+ goto out;
+ } else {
+ path->slots[0]++;
+ }
+
+ btrfs_item_key_to_cpu(path->nodes[0], &key, path->slots[0]);
+
+ if (key.type != key_type || key.objectid != dirid) {
+ ret = 1;
+ goto out;
+ }
+ item = btrfs_item_ptr(path->nodes[0], path->slots[0],
+ struct btrfs_dir_log_item);
+ found_end = btrfs_dir_log_end(path->nodes[0], item);
+ *start_ret = key.offset;
+ *end_ret = found_end;
+ ret = 0;
+out:
+ btrfs_release_path(root, path);
+ return ret;
+}
+
+/*
+ * this looks for a given directory item in the log. If the directory
+ * item is not in the log, the item is removed and the inode it points
+ * to is unlinked
+ */
+static noinline int check_item_in_log(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_root *log,
+ struct btrfs_path *path,
+ struct btrfs_path *log_path,
+ struct inode *dir,
+ struct btrfs_key *dir_key)
+{
+ int ret;
+ struct extent_buffer *eb;
+ int slot;
+ u32 item_size;
+ struct btrfs_dir_item *di;
+ struct btrfs_dir_item *log_di;
+ int name_len;
+ unsigned long ptr;
+ unsigned long ptr_end;
+ char *name;
+ struct inode *inode;
+ struct btrfs_key location;
+
+again:
+ eb = path->nodes[0];
+ slot = path->slots[0];
+ item_size = btrfs_item_size_nr(eb, slot);
+ ptr = btrfs_item_ptr_offset(eb, slot);
+ ptr_end = ptr + item_size;
+ while (ptr < ptr_end) {
+ di = (struct btrfs_dir_item *)ptr;
+ name_len = btrfs_dir_name_len(eb, di);
+ name = kmalloc(name_len, GFP_NOFS);
+ if (!name) {
+ ret = -ENOMEM;
+ goto out;
+ }
+ read_extent_buffer(eb, name, (unsigned long)(di + 1),
+ name_len);
+ log_di = NULL;
+ if (dir_key->type == BTRFS_DIR_ITEM_KEY) {
+ log_di = btrfs_lookup_dir_item(trans, log, log_path,
+ dir_key->objectid,
+ name, name_len, 0);
+ } else if (dir_key->type == BTRFS_DIR_INDEX_KEY) {
+ log_di = btrfs_lookup_dir_index_item(trans, log,
+ log_path,
+ dir_key->objectid,
+ dir_key->offset,
+ name, name_len, 0);
+ }
+ if (!log_di || IS_ERR(log_di)) {
+ btrfs_dir_item_key_to_cpu(eb, di, &location);
+ btrfs_release_path(root, path);
+ btrfs_release_path(log, log_path);
+ inode = read_one_inode(root, location.objectid);
+ BUG_ON(!inode);
+
+ ret = link_to_fixup_dir(trans, root,
+ path, location.objectid);
+ BUG_ON(ret);
+ btrfs_inc_nlink(inode);
+ ret = btrfs_unlink_inode(trans, root, dir, inode,
+ name, name_len);
+ BUG_ON(ret);
+ kfree(name);
+ iput(inode);
+
+ /* there might still be more names under this key
+ * check and repeat if required
+ */
+ ret = btrfs_search_slot(NULL, root, dir_key, path,
+ 0, 0);
+ if (ret == 0)
+ goto again;
+ ret = 0;
+ goto out;
+ }
+ btrfs_release_path(log, log_path);
+ kfree(name);
+
+ ptr = (unsigned long)(di + 1);
+ ptr += name_len;
+ }
+ ret = 0;
+out:
+ btrfs_release_path(root, path);
+ btrfs_release_path(log, log_path);
+ return ret;
+}
+
+/*
+ * deletion replay happens before we copy any new directory items
+ * out of the log or out of backreferences from inodes. It
+ * scans the log to find ranges of keys that log is authoritative for,
+ * and then scans the directory to find items in those ranges that are
+ * not present in the log.
+ *
+ * Anything we don't find in the log is unlinked and removed from the
+ * directory.
+ */
+static noinline int replay_dir_deletes(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_root *log,
+ struct btrfs_path *path,
+ u64 dirid)
+{
+ u64 range_start;
+ u64 range_end;
+ int key_type = BTRFS_DIR_LOG_ITEM_KEY;
+ int ret = 0;
+ struct btrfs_key dir_key;
+ struct btrfs_key found_key;
+ struct btrfs_path *log_path;
+ struct inode *dir;
+
+ dir_key.objectid = dirid;
+ dir_key.type = BTRFS_DIR_ITEM_KEY;
+ log_path = btrfs_alloc_path();
+ if (!log_path)
+ return -ENOMEM;
+
+ dir = read_one_inode(root, dirid);
+ /* it isn't an error if the inode isn't there, that can happen
+ * because we replay the deletes before we copy in the inode item
+ * from the log
+ */
+ if (!dir) {
+ btrfs_free_path(log_path);
+ return 0;
+ }
+again:
+ range_start = 0;
+ range_end = 0;
+ while (1) {
+ ret = find_dir_range(log, path, dirid, key_type,
+ &range_start, &range_end);
+ if (ret != 0)
+ break;
+
+ dir_key.offset = range_start;
+ while (1) {
+ int nritems;
+ ret = btrfs_search_slot(NULL, root, &dir_key, path,
+ 0, 0);
+ if (ret < 0)
+ goto out;
+
+ nritems = btrfs_header_nritems(path->nodes[0]);
+ if (path->slots[0] >= nritems) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret)
+ break;
+ }
+ btrfs_item_key_to_cpu(path->nodes[0], &found_key,
+ path->slots[0]);
+ if (found_key.objectid != dirid ||
+ found_key.type != dir_key.type)
+ goto next_type;
+
+ if (found_key.offset > range_end)
+ break;
+
+ ret = check_item_in_log(trans, root, log, path,
+ log_path, dir, &found_key);
+ BUG_ON(ret);
+ if (found_key.offset == (u64)-1)
+ break;
+ dir_key.offset = found_key.offset + 1;
+ }
+ btrfs_release_path(root, path);
+ if (range_end == (u64)-1)
+ break;
+ range_start = range_end + 1;
+ }
+
+next_type:
+ ret = 0;
+ if (key_type == BTRFS_DIR_LOG_ITEM_KEY) {
+ key_type = BTRFS_DIR_LOG_INDEX_KEY;
+ dir_key.type = BTRFS_DIR_INDEX_KEY;
+ btrfs_release_path(root, path);
+ goto again;
+ }
+out:
+ btrfs_release_path(root, path);
+ btrfs_free_path(log_path);
+ iput(dir);
+ return ret;
+}
+
+/*
+ * the process_func used to replay items from the log tree. This
+ * gets called in two different stages. The first stage just looks
+ * for inodes and makes sure they are all copied into the subvolume.
+ *
+ * The second stage copies all the other item types from the log into
+ * the subvolume. The two stage approach is slower, but gets rid of
+ * lots of complexity around inodes referencing other inodes that exist
+ * only in the log (references come from either directory items or inode
+ * back refs).
+ */
+static int replay_one_buffer(struct btrfs_root *log, struct extent_buffer *eb,
+ struct walk_control *wc, u64 gen)
+{
+ int nritems;
+ struct btrfs_path *path;
+ struct btrfs_root *root = wc->replay_dest;
+ struct btrfs_key key;
+ u32 item_size;
+ int level;
+ int i;
+ int ret;
+
+ btrfs_read_buffer(eb, gen);
+
+ level = btrfs_header_level(eb);
+
+ if (level != 0)
+ return 0;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+
+ nritems = btrfs_header_nritems(eb);
+ for (i = 0; i < nritems; i++) {
+ btrfs_item_key_to_cpu(eb, &key, i);
+ item_size = btrfs_item_size_nr(eb, i);
+
+ /* inode keys are done during the first stage */
+ if (key.type == BTRFS_INODE_ITEM_KEY &&
+ wc->stage == LOG_WALK_REPLAY_INODES) {
+ struct inode *inode;
+ struct btrfs_inode_item *inode_item;
+ u32 mode;
+
+ inode_item = btrfs_item_ptr(eb, i,
+ struct btrfs_inode_item);
+ mode = btrfs_inode_mode(eb, inode_item);
+ if (S_ISDIR(mode)) {
+ ret = replay_dir_deletes(wc->trans,
+ root, log, path, key.objectid);
+ BUG_ON(ret);
+ }
+ ret = overwrite_item(wc->trans, root, path,
+ eb, i, &key);
+ BUG_ON(ret);
+
+ /* for regular files, truncate away
+ * extents past the new EOF
+ */
+ if (S_ISREG(mode)) {
+ inode = read_one_inode(root,
+ key.objectid);
+ BUG_ON(!inode);
+
+ ret = btrfs_truncate_inode_items(wc->trans,
+ root, inode, inode->i_size,
+ BTRFS_EXTENT_DATA_KEY);
+ BUG_ON(ret);
+ iput(inode);
+ }
+ ret = link_to_fixup_dir(wc->trans, root,
+ path, key.objectid);
+ BUG_ON(ret);
+ }
+ if (wc->stage < LOG_WALK_REPLAY_ALL)
+ continue;
+
+ /* these keys are simply copied */
+ if (key.type == BTRFS_XATTR_ITEM_KEY) {
+ ret = overwrite_item(wc->trans, root, path,
+ eb, i, &key);
+ BUG_ON(ret);
+ } else if (key.type == BTRFS_INODE_REF_KEY) {
+ ret = add_inode_ref(wc->trans, root, log, path,
+ eb, i, &key);
+ BUG_ON(ret && ret != -ENOENT);
+ } else if (key.type == BTRFS_EXTENT_DATA_KEY) {
+ ret = replay_one_extent(wc->trans, root, path,
+ eb, i, &key);
+ BUG_ON(ret);
+ } else if (key.type == BTRFS_DIR_ITEM_KEY ||
+ key.type == BTRFS_DIR_INDEX_KEY) {
+ ret = replay_one_dir_item(wc->trans, root, path,
+ eb, i, &key);
+ BUG_ON(ret);
+ }
+ }
+ btrfs_free_path(path);
+ return 0;
+}
+
+static noinline int walk_down_log_tree(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, int *level,
+ struct walk_control *wc)
+{
+ u64 root_owner;
+ u64 root_gen;
+ u64 bytenr;
+ u64 ptr_gen;
+ struct extent_buffer *next;
+ struct extent_buffer *cur;
+ struct extent_buffer *parent;
+ u32 blocksize;
+ int ret = 0;
+
+ WARN_ON(*level < 0);
+ WARN_ON(*level >= BTRFS_MAX_LEVEL);
+
+ while (*level > 0) {
+ WARN_ON(*level < 0);
+ WARN_ON(*level >= BTRFS_MAX_LEVEL);
+ cur = path->nodes[*level];
+
+ if (btrfs_header_level(cur) != *level)
+ WARN_ON(1);
+
+ if (path->slots[*level] >=
+ btrfs_header_nritems(cur))
+ break;
+
+ bytenr = btrfs_node_blockptr(cur, path->slots[*level]);
+ ptr_gen = btrfs_node_ptr_generation(cur, path->slots[*level]);
+ blocksize = btrfs_level_size(root, *level - 1);
+
+ parent = path->nodes[*level];
+ root_owner = btrfs_header_owner(parent);
+ root_gen = btrfs_header_generation(parent);
+
+ next = btrfs_find_create_tree_block(root, bytenr, blocksize);
+
+ wc->process_func(root, next, wc, ptr_gen);
+
+ if (*level == 1) {
+ path->slots[*level]++;
+ if (wc->free) {
+ btrfs_read_buffer(next, ptr_gen);
+
+ btrfs_tree_lock(next);
+ clean_tree_block(trans, root, next);
+ btrfs_wait_tree_block_writeback(next);
+ btrfs_tree_unlock(next);
+
+ ret = btrfs_drop_leaf_ref(trans, root, next);
+ BUG_ON(ret);
+
+ WARN_ON(root_owner !=
+ BTRFS_TREE_LOG_OBJECTID);
+ ret = btrfs_free_reserved_extent(root,
+ bytenr, blocksize);
+ BUG_ON(ret);
+ }
+ free_extent_buffer(next);
+ continue;
+ }
+ btrfs_read_buffer(next, ptr_gen);
+
+ WARN_ON(*level <= 0);
+ if (path->nodes[*level-1])
+ free_extent_buffer(path->nodes[*level-1]);
+ path->nodes[*level-1] = next;
+ *level = btrfs_header_level(next);
+ path->slots[*level] = 0;
+ cond_resched();
+ }
+ WARN_ON(*level < 0);
+ WARN_ON(*level >= BTRFS_MAX_LEVEL);
+
+ if (path->nodes[*level] == root->node)
+ parent = path->nodes[*level];
+ else
+ parent = path->nodes[*level + 1];
+
+ bytenr = path->nodes[*level]->start;
+
+ blocksize = btrfs_level_size(root, *level);
+ root_owner = btrfs_header_owner(parent);
+ root_gen = btrfs_header_generation(parent);
+
+ wc->process_func(root, path->nodes[*level], wc,
+ btrfs_header_generation(path->nodes[*level]));
+
+ if (wc->free) {
+ next = path->nodes[*level];
+ btrfs_tree_lock(next);
+ clean_tree_block(trans, root, next);
+ btrfs_wait_tree_block_writeback(next);
+ btrfs_tree_unlock(next);
+
+ if (*level == 0) {
+ ret = btrfs_drop_leaf_ref(trans, root, next);
+ BUG_ON(ret);
+ }
+ WARN_ON(root_owner != BTRFS_TREE_LOG_OBJECTID);
+ ret = btrfs_free_reserved_extent(root, bytenr, blocksize);
+ BUG_ON(ret);
+ }
+ free_extent_buffer(path->nodes[*level]);
+ path->nodes[*level] = NULL;
+ *level += 1;
+
+ cond_resched();
+ return 0;
+}
+
+static noinline int walk_up_log_tree(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_path *path, int *level,
+ struct walk_control *wc)
+{
+ u64 root_owner;
+ u64 root_gen;
+ int i;
+ int slot;
+ int ret;
+
+ for (i = *level; i < BTRFS_MAX_LEVEL - 1 && path->nodes[i]; i++) {
+ slot = path->slots[i];
+ if (slot < btrfs_header_nritems(path->nodes[i]) - 1) {
+ struct extent_buffer *node;
+ node = path->nodes[i];
+ path->slots[i]++;
+ *level = i;
+ WARN_ON(*level == 0);
+ return 0;
+ } else {
+ struct extent_buffer *parent;
+ if (path->nodes[*level] == root->node)
+ parent = path->nodes[*level];
+ else
+ parent = path->nodes[*level + 1];
+
+ root_owner = btrfs_header_owner(parent);
+ root_gen = btrfs_header_generation(parent);
+ wc->process_func(root, path->nodes[*level], wc,
+ btrfs_header_generation(path->nodes[*level]));
+ if (wc->free) {
+ struct extent_buffer *next;
+
+ next = path->nodes[*level];
+
+ btrfs_tree_lock(next);
+ clean_tree_block(trans, root, next);
+ btrfs_wait_tree_block_writeback(next);
+ btrfs_tree_unlock(next);
+
+ if (*level == 0) {
+ ret = btrfs_drop_leaf_ref(trans, root,
+ next);
+ BUG_ON(ret);
+ }
+
+ WARN_ON(root_owner != BTRFS_TREE_LOG_OBJECTID);
+ ret = btrfs_free_reserved_extent(root,
+ path->nodes[*level]->start,
+ path->nodes[*level]->len);
+ BUG_ON(ret);
+ }
+ free_extent_buffer(path->nodes[*level]);
+ path->nodes[*level] = NULL;
+ *level = i + 1;
+ }
+ }
+ return 1;
+}
+
+/*
+ * drop the reference count on the tree rooted at 'snap'. This traverses
+ * the tree freeing any blocks that have a ref count of zero after being
+ * decremented.
+ */
+static int walk_log_tree(struct btrfs_trans_handle *trans,
+ struct btrfs_root *log, struct walk_control *wc)
+{
+ int ret = 0;
+ int wret;
+ int level;
+ struct btrfs_path *path;
+ int i;
+ int orig_level;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+
+ level = btrfs_header_level(log->node);
+ orig_level = level;
+ path->nodes[level] = log->node;
+ extent_buffer_get(log->node);
+ path->slots[level] = 0;
+
+ while (1) {
+ wret = walk_down_log_tree(trans, log, path, &level, wc);
+ if (wret > 0)
+ break;
+ if (wret < 0)
+ ret = wret;
+
+ wret = walk_up_log_tree(trans, log, path, &level, wc);
+ if (wret > 0)
+ break;
+ if (wret < 0)
+ ret = wret;
+ }
+
+ /* was the root node processed? if not, catch it here */
+ if (path->nodes[orig_level]) {
+ wc->process_func(log, path->nodes[orig_level], wc,
+ btrfs_header_generation(path->nodes[orig_level]));
+ if (wc->free) {
+ struct extent_buffer *next;
+
+ next = path->nodes[orig_level];
+
+ btrfs_tree_lock(next);
+ clean_tree_block(trans, log, next);
+ btrfs_wait_tree_block_writeback(next);
+ btrfs_tree_unlock(next);
+
+ if (orig_level == 0) {
+ ret = btrfs_drop_leaf_ref(trans, log,
+ next);
+ BUG_ON(ret);
+ }
+ WARN_ON(log->root_key.objectid !=
+ BTRFS_TREE_LOG_OBJECTID);
+ ret = btrfs_free_reserved_extent(log, next->start,
+ next->len);
+ BUG_ON(ret);
+ }
+ }
+
+ for (i = 0; i <= orig_level; i++) {
+ if (path->nodes[i]) {
+ free_extent_buffer(path->nodes[i]);
+ path->nodes[i] = NULL;
+ }
+ }
+ btrfs_free_path(path);
+ if (wc->free)
+ free_extent_buffer(log->node);
+ return ret;
+}
+
+static int wait_log_commit(struct btrfs_root *log)
+{
+ DEFINE_WAIT(wait);
+ u64 transid = log->fs_info->tree_log_transid;
+
+ do {
+ prepare_to_wait(&log->fs_info->tree_log_wait, &wait,
+ TASK_UNINTERRUPTIBLE);
+ mutex_unlock(&log->fs_info->tree_log_mutex);
+ if (atomic_read(&log->fs_info->tree_log_commit))
+ schedule();
+ finish_wait(&log->fs_info->tree_log_wait, &wait);
+ mutex_lock(&log->fs_info->tree_log_mutex);
+ } while (transid == log->fs_info->tree_log_transid &&
+ atomic_read(&log->fs_info->tree_log_commit));
+ return 0;
+}
+
+/*
+ * btrfs_sync_log does sends a given tree log down to the disk and
+ * updates the super blocks to record it. When this call is done,
+ * you know that any inodes previously logged are safely on disk
+ */
+int btrfs_sync_log(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root)
+{
+ int ret;
+ unsigned long batch;
+ struct btrfs_root *log = root->log_root;
+
+ mutex_lock(&log->fs_info->tree_log_mutex);
+ if (atomic_read(&log->fs_info->tree_log_commit)) {
+ wait_log_commit(log);
+ goto out;
+ }
+ atomic_set(&log->fs_info->tree_log_commit, 1);
+
+ while (1) {
+ batch = log->fs_info->tree_log_batch;
+ mutex_unlock(&log->fs_info->tree_log_mutex);
+ schedule_timeout_uninterruptible(1);
+ mutex_lock(&log->fs_info->tree_log_mutex);
+
+ while (atomic_read(&log->fs_info->tree_log_writers)) {
+ DEFINE_WAIT(wait);
+ prepare_to_wait(&log->fs_info->tree_log_wait, &wait,
+ TASK_UNINTERRUPTIBLE);
+ mutex_unlock(&log->fs_info->tree_log_mutex);
+ if (atomic_read(&log->fs_info->tree_log_writers))
+ schedule();
+ mutex_lock(&log->fs_info->tree_log_mutex);
+ finish_wait(&log->fs_info->tree_log_wait, &wait);
+ }
+ if (batch == log->fs_info->tree_log_batch)
+ break;
+ }
+
+ ret = btrfs_write_and_wait_marked_extents(log, &log->dirty_log_pages);
+ BUG_ON(ret);
+ ret = btrfs_write_and_wait_marked_extents(root->fs_info->log_root_tree,
+ &root->fs_info->log_root_tree->dirty_log_pages);
+ BUG_ON(ret);
+
+ btrfs_set_super_log_root(&root->fs_info->super_for_commit,
+ log->fs_info->log_root_tree->node->start);
+ btrfs_set_super_log_root_level(&root->fs_info->super_for_commit,
+ btrfs_header_level(log->fs_info->log_root_tree->node));
+
+ write_ctree_super(trans, log->fs_info->tree_root, 2);
+ log->fs_info->tree_log_transid++;
+ log->fs_info->tree_log_batch = 0;
+ atomic_set(&log->fs_info->tree_log_commit, 0);
+ smp_mb();
+ if (waitqueue_active(&log->fs_info->tree_log_wait))
+ wake_up(&log->fs_info->tree_log_wait);
+out:
+ mutex_unlock(&log->fs_info->tree_log_mutex);
+ return 0;
+}
+
+/* * free all the extents used by the tree log. This should be called
+ * at commit time of the full transaction
+ */
+int btrfs_free_log(struct btrfs_trans_handle *trans, struct btrfs_root *root)
+{
+ int ret;
+ struct btrfs_root *log;
+ struct key;
+ u64 start;
+ u64 end;
+ struct walk_control wc = {
+ .free = 1,
+ .process_func = process_one_buffer
+ };
+
+ if (!root->log_root || root->fs_info->log_root_recovering)
+ return 0;
+
+ log = root->log_root;
+ ret = walk_log_tree(trans, log, &wc);
+ BUG_ON(ret);
+
+ while (1) {
+ ret = find_first_extent_bit(&log->dirty_log_pages,
+ 0, &start, &end, EXTENT_DIRTY);
+ if (ret)
+ break;
+
+ clear_extent_dirty(&log->dirty_log_pages,
+ start, end, GFP_NOFS);
+ }
+
+ log = root->log_root;
+ ret = btrfs_del_root(trans, root->fs_info->log_root_tree,
+ &log->root_key);
+ BUG_ON(ret);
+ root->log_root = NULL;
+ kfree(root->log_root);
+ return 0;
+}
+
+/*
+ * helper function to update the item for a given subvolumes log root
+ * in the tree of log roots
+ */
+static int update_log_root(struct btrfs_trans_handle *trans,
+ struct btrfs_root *log)
+{
+ u64 bytenr = btrfs_root_bytenr(&log->root_item);
+ int ret;
+
+ if (log->node->start == bytenr)
+ return 0;
+
+ btrfs_set_root_bytenr(&log->root_item, log->node->start);
+ btrfs_set_root_generation(&log->root_item, trans->transid);
+ btrfs_set_root_level(&log->root_item, btrfs_header_level(log->node));
+ ret = btrfs_update_root(trans, log->fs_info->log_root_tree,
+ &log->root_key, &log->root_item);
+ BUG_ON(ret);
+ return ret;
+}
+
+/*
+ * If both a file and directory are logged, and unlinks or renames are
+ * mixed in, we have a few interesting corners:
+ *
+ * create file X in dir Y
+ * link file X to X.link in dir Y
+ * fsync file X
+ * unlink file X but leave X.link
+ * fsync dir Y
+ *
+ * After a crash we would expect only X.link to exist. But file X
+ * didn't get fsync'd again so the log has back refs for X and X.link.
+ *
+ * We solve this by removing directory entries and inode backrefs from the
+ * log when a file that was logged in the current transaction is
+ * unlinked. Any later fsync will include the updated log entries, and
+ * we'll be able to reconstruct the proper directory items from backrefs.
+ *
+ * This optimizations allows us to avoid relogging the entire inode
+ * or the entire directory.
+ */
+int btrfs_del_dir_entries_in_log(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ const char *name, int name_len,
+ struct inode *dir, u64 index)
+{
+ struct btrfs_root *log;
+ struct btrfs_dir_item *di;
+ struct btrfs_path *path;
+ int ret;
+ int bytes_del = 0;
+
+ if (BTRFS_I(dir)->logged_trans < trans->transid)
+ return 0;
+
+ ret = join_running_log_trans(root);
+ if (ret)
+ return 0;
+
+ mutex_lock(&BTRFS_I(dir)->log_mutex);
+
+ log = root->log_root;
+ path = btrfs_alloc_path();
+ di = btrfs_lookup_dir_item(trans, log, path, dir->i_ino,
+ name, name_len, -1);
+ if (di && !IS_ERR(di)) {
+ ret = btrfs_delete_one_dir_name(trans, log, path, di);
+ bytes_del += name_len;
+ BUG_ON(ret);
+ }
+ btrfs_release_path(log, path);
+ di = btrfs_lookup_dir_index_item(trans, log, path, dir->i_ino,
+ index, name, name_len, -1);
+ if (di && !IS_ERR(di)) {
+ ret = btrfs_delete_one_dir_name(trans, log, path, di);
+ bytes_del += name_len;
+ BUG_ON(ret);
+ }
+
+ /* update the directory size in the log to reflect the names
+ * we have removed
+ */
+ if (bytes_del) {
+ struct btrfs_key key;
+
+ key.objectid = dir->i_ino;
+ key.offset = 0;
+ key.type = BTRFS_INODE_ITEM_KEY;
+ btrfs_release_path(log, path);
+
+ ret = btrfs_search_slot(trans, log, &key, path, 0, 1);
+ if (ret == 0) {
+ struct btrfs_inode_item *item;
+ u64 i_size;
+
+ item = btrfs_item_ptr(path->nodes[0], path->slots[0],
+ struct btrfs_inode_item);
+ i_size = btrfs_inode_size(path->nodes[0], item);
+ if (i_size > bytes_del)
+ i_size -= bytes_del;
+ else
+ i_size = 0;
+ btrfs_set_inode_size(path->nodes[0], item, i_size);
+ btrfs_mark_buffer_dirty(path->nodes[0]);
+ } else
+ ret = 0;
+ btrfs_release_path(log, path);
+ }
+
+ btrfs_free_path(path);
+ mutex_unlock(&BTRFS_I(dir)->log_mutex);
+ end_log_trans(root);
+
+ return 0;
+}
+
+/* see comments for btrfs_del_dir_entries_in_log */
+int btrfs_del_inode_ref_in_log(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ const char *name, int name_len,
+ struct inode *inode, u64 dirid)
+{
+ struct btrfs_root *log;
+ u64 index;
+ int ret;
+
+ if (BTRFS_I(inode)->logged_trans < trans->transid)
+ return 0;
+
+ ret = join_running_log_trans(root);
+ if (ret)
+ return 0;
+ log = root->log_root;
+ mutex_lock(&BTRFS_I(inode)->log_mutex);
+
+ ret = btrfs_del_inode_ref(trans, log, name, name_len, inode->i_ino,
+ dirid, &index);
+ mutex_unlock(&BTRFS_I(inode)->log_mutex);
+ end_log_trans(root);
+
+ return ret;
+}
+
+/*
+ * creates a range item in the log for 'dirid'. first_offset and
+ * last_offset tell us which parts of the key space the log should
+ * be considered authoritative for.
+ */
+static noinline int insert_dir_log_key(struct btrfs_trans_handle *trans,
+ struct btrfs_root *log,
+ struct btrfs_path *path,
+ int key_type, u64 dirid,
+ u64 first_offset, u64 last_offset)
+{
+ int ret;
+ struct btrfs_key key;
+ struct btrfs_dir_log_item *item;
+
+ key.objectid = dirid;
+ key.offset = first_offset;
+ if (key_type == BTRFS_DIR_ITEM_KEY)
+ key.type = BTRFS_DIR_LOG_ITEM_KEY;
+ else
+ key.type = BTRFS_DIR_LOG_INDEX_KEY;
+ ret = btrfs_insert_empty_item(trans, log, path, &key, sizeof(*item));
+ BUG_ON(ret);
+
+ item = btrfs_item_ptr(path->nodes[0], path->slots[0],
+ struct btrfs_dir_log_item);
+ btrfs_set_dir_log_end(path->nodes[0], item, last_offset);
+ btrfs_mark_buffer_dirty(path->nodes[0]);
+ btrfs_release_path(log, path);
+ return 0;
+}
+
+/*
+ * log all the items included in the current transaction for a given
+ * directory. This also creates the range items in the log tree required
+ * to replay anything deleted before the fsync
+ */
+static noinline int log_dir_items(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct inode *inode,
+ struct btrfs_path *path,
+ struct btrfs_path *dst_path, int key_type,
+ u64 min_offset, u64 *last_offset_ret)
+{
+ struct btrfs_key min_key;
+ struct btrfs_key max_key;
+ struct btrfs_root *log = root->log_root;
+ struct extent_buffer *src;
+ int ret;
+ int i;
+ int nritems;
+ u64 first_offset = min_offset;
+ u64 last_offset = (u64)-1;
+
+ log = root->log_root;
+ max_key.objectid = inode->i_ino;
+ max_key.offset = (u64)-1;
+ max_key.type = key_type;
+
+ min_key.objectid = inode->i_ino;
+ min_key.type = key_type;
+ min_key.offset = min_offset;
+
+ path->keep_locks = 1;
+
+ ret = btrfs_search_forward(root, &min_key, &max_key,
+ path, 0, trans->transid);
+
+ /*
+ * we didn't find anything from this transaction, see if there
+ * is anything at all
+ */
+ if (ret != 0 || min_key.objectid != inode->i_ino ||
+ min_key.type != key_type) {
+ min_key.objectid = inode->i_ino;
+ min_key.type = key_type;
+ min_key.offset = (u64)-1;
+ btrfs_release_path(root, path);
+ ret = btrfs_search_slot(NULL, root, &min_key, path, 0, 0);
+ if (ret < 0) {
+ btrfs_release_path(root, path);
+ return ret;
+ }
+ ret = btrfs_previous_item(root, path, inode->i_ino, key_type);
+
+ /* if ret == 0 there are items for this type,
+ * create a range to tell us the last key of this type.
+ * otherwise, there are no items in this directory after
+ * *min_offset, and we create a range to indicate that.
+ */
+ if (ret == 0) {
+ struct btrfs_key tmp;
+ btrfs_item_key_to_cpu(path->nodes[0], &tmp,
+ path->slots[0]);
+ if (key_type == tmp.type)
+ first_offset = max(min_offset, tmp.offset) + 1;
+ }
+ goto done;
+ }
+
+ /* go backward to find any previous key */
+ ret = btrfs_previous_item(root, path, inode->i_ino, key_type);
+ if (ret == 0) {
+ struct btrfs_key tmp;
+ btrfs_item_key_to_cpu(path->nodes[0], &tmp, path->slots[0]);
+ if (key_type == tmp.type) {
+ first_offset = tmp.offset;
+ ret = overwrite_item(trans, log, dst_path,
+ path->nodes[0], path->slots[0],
+ &tmp);
+ }
+ }
+ btrfs_release_path(root, path);
+
+ /* find the first key from this transaction again */
+ ret = btrfs_search_slot(NULL, root, &min_key, path, 0, 0);
+ if (ret != 0) {
+ WARN_ON(1);
+ goto done;
+ }
+
+ /*
+ * we have a block from this transaction, log every item in it
+ * from our directory
+ */
+ while (1) {
+ struct btrfs_key tmp;
+ src = path->nodes[0];
+ nritems = btrfs_header_nritems(src);
+ for (i = path->slots[0]; i < nritems; i++) {
+ btrfs_item_key_to_cpu(src, &min_key, i);
+
+ if (min_key.objectid != inode->i_ino ||
+ min_key.type != key_type)
+ goto done;
+ ret = overwrite_item(trans, log, dst_path, src, i,
+ &min_key);
+ BUG_ON(ret);
+ }
+ path->slots[0] = nritems;
+
+ /*
+ * look ahead to the next item and see if it is also
+ * from this directory and from this transaction
+ */
+ ret = btrfs_next_leaf(root, path);
+ if (ret == 1) {
+ last_offset = (u64)-1;
+ goto done;
+ }
+ btrfs_item_key_to_cpu(path->nodes[0], &tmp, path->slots[0]);
+ if (tmp.objectid != inode->i_ino || tmp.type != key_type) {
+ last_offset = (u64)-1;
+ goto done;
+ }
+ if (btrfs_header_generation(path->nodes[0]) != trans->transid) {
+ ret = overwrite_item(trans, log, dst_path,
+ path->nodes[0], path->slots[0],
+ &tmp);
+
+ BUG_ON(ret);
+ last_offset = tmp.offset;
+ goto done;
+ }
+ }
+done:
+ *last_offset_ret = last_offset;
+ btrfs_release_path(root, path);
+ btrfs_release_path(log, dst_path);
+
+ /* insert the log range keys to indicate where the log is valid */
+ ret = insert_dir_log_key(trans, log, path, key_type, inode->i_ino,
+ first_offset, last_offset);
+ BUG_ON(ret);
+ return 0;
+}
+
+/*
+ * logging directories is very similar to logging inodes, We find all the items
+ * from the current transaction and write them to the log.
+ *
+ * The recovery code scans the directory in the subvolume, and if it finds a
+ * key in the range logged that is not present in the log tree, then it means
+ * that dir entry was unlinked during the transaction.
+ *
+ * In order for that scan to work, we must include one key smaller than
+ * the smallest logged by this transaction and one key larger than the largest
+ * key logged by this transaction.
+ */
+static noinline int log_directory_changes(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct inode *inode,
+ struct btrfs_path *path,
+ struct btrfs_path *dst_path)
+{
+ u64 min_key;
+ u64 max_key;
+ int ret;
+ int key_type = BTRFS_DIR_ITEM_KEY;
+
+again:
+ min_key = 0;
+ max_key = 0;
+ while (1) {
+ ret = log_dir_items(trans, root, inode, path,
+ dst_path, key_type, min_key,
+ &max_key);
+ BUG_ON(ret);
+ if (max_key == (u64)-1)
+ break;
+ min_key = max_key + 1;
+ }
+
+ if (key_type == BTRFS_DIR_ITEM_KEY) {
+ key_type = BTRFS_DIR_INDEX_KEY;
+ goto again;
+ }
+ return 0;
+}
+
+/*
+ * a helper function to drop items from the log before we relog an
+ * inode. max_key_type indicates the highest item type to remove.
+ * This cannot be run for file data extents because it does not
+ * free the extents they point to.
+ */
+static int drop_objectid_items(struct btrfs_trans_handle *trans,
+ struct btrfs_root *log,
+ struct btrfs_path *path,
+ u64 objectid, int max_key_type)
+{
+ int ret;
+ struct btrfs_key key;
+ struct btrfs_key found_key;
+
+ key.objectid = objectid;
+ key.type = max_key_type;
+ key.offset = (u64)-1;
+
+ while (1) {
+ ret = btrfs_search_slot(trans, log, &key, path, -1, 1);
+
+ if (ret != 1)
+ break;
+
+ if (path->slots[0] == 0)
+ break;
+
+ path->slots[0]--;
+ btrfs_item_key_to_cpu(path->nodes[0], &found_key,
+ path->slots[0]);
+
+ if (found_key.objectid != objectid)
+ break;
+
+ ret = btrfs_del_item(trans, log, path);
+ BUG_ON(ret);
+ btrfs_release_path(log, path);
+ }
+ btrfs_release_path(log, path);
+ return 0;
+}
+
+static noinline int copy_items(struct btrfs_trans_handle *trans,
+ struct btrfs_root *log,
+ struct btrfs_path *dst_path,
+ struct extent_buffer *src,
+ int start_slot, int nr, int inode_only)
+{
+ unsigned long src_offset;
+ unsigned long dst_offset;
+ struct btrfs_file_extent_item *extent;
+ struct btrfs_inode_item *inode_item;
+ int ret;
+ struct btrfs_key *ins_keys;
+ u32 *ins_sizes;
+ char *ins_data;
+ int i;
+ struct list_head ordered_sums;
+
+ INIT_LIST_HEAD(&ordered_sums);
+
+ ins_data = kmalloc(nr * sizeof(struct btrfs_key) +
+ nr * sizeof(u32), GFP_NOFS);
+ ins_sizes = (u32 *)ins_data;
+ ins_keys = (struct btrfs_key *)(ins_data + nr * sizeof(u32));
+
+ for (i = 0; i < nr; i++) {
+ ins_sizes[i] = btrfs_item_size_nr(src, i + start_slot);
+ btrfs_item_key_to_cpu(src, ins_keys + i, i + start_slot);
+ }
+ ret = btrfs_insert_empty_items(trans, log, dst_path,
+ ins_keys, ins_sizes, nr);
+ BUG_ON(ret);
+
+ for (i = 0; i < nr; i++) {
+ dst_offset = btrfs_item_ptr_offset(dst_path->nodes[0],
+ dst_path->slots[0]);
+
+ src_offset = btrfs_item_ptr_offset(src, start_slot + i);
+
+ copy_extent_buffer(dst_path->nodes[0], src, dst_offset,
+ src_offset, ins_sizes[i]);
+
+ if (inode_only == LOG_INODE_EXISTS &&
+ ins_keys[i].type == BTRFS_INODE_ITEM_KEY) {
+ inode_item = btrfs_item_ptr(dst_path->nodes[0],
+ dst_path->slots[0],
+ struct btrfs_inode_item);
+ btrfs_set_inode_size(dst_path->nodes[0], inode_item, 0);
+
+ /* set the generation to zero so the recover code
+ * can tell the difference between an logging
+ * just to say 'this inode exists' and a logging
+ * to say 'update this inode with these values'
+ */
+ btrfs_set_inode_generation(dst_path->nodes[0],
+ inode_item, 0);
+ }
+ /* take a reference on file data extents so that truncates
+ * or deletes of this inode don't have to relog the inode
+ * again
+ */
+ if (btrfs_key_type(ins_keys + i) == BTRFS_EXTENT_DATA_KEY) {
+ int found_type;
+ extent = btrfs_item_ptr(src, start_slot + i,
+ struct btrfs_file_extent_item);
+
+ found_type = btrfs_file_extent_type(src, extent);
+ if (found_type == BTRFS_FILE_EXTENT_REG ||
+ found_type == BTRFS_FILE_EXTENT_PREALLOC) {
+ u64 ds = btrfs_file_extent_disk_bytenr(src,
+ extent);
+ u64 dl = btrfs_file_extent_disk_num_bytes(src,
+ extent);
+ u64 cs = btrfs_file_extent_offset(src, extent);
+ u64 cl = btrfs_file_extent_num_bytes(src,
+ extent);;
+ if (btrfs_file_extent_compression(src,
+ extent)) {
+ cs = 0;
+ cl = dl;
+ }
+ /* ds == 0 is a hole */
+ if (ds != 0) {
+ ret = btrfs_inc_extent_ref(trans, log,
+ ds, dl,
+ dst_path->nodes[0]->start,
+ BTRFS_TREE_LOG_OBJECTID,
+ trans->transid,
+ ins_keys[i].objectid);
+ BUG_ON(ret);
+ ret = btrfs_lookup_csums_range(
+ log->fs_info->csum_root,
+ ds + cs, ds + cs + cl - 1,
+ &ordered_sums);
+ BUG_ON(ret);
+ }
+ }
+ }
+ dst_path->slots[0]++;
+ }
+
+ btrfs_mark_buffer_dirty(dst_path->nodes[0]);
+ btrfs_release_path(log, dst_path);
+ kfree(ins_data);
+
+ /*
+ * we have to do this after the loop above to avoid changing the
+ * log tree while trying to change the log tree.
+ */
+ while (!list_empty(&ordered_sums)) {
+ struct btrfs_ordered_sum *sums = list_entry(ordered_sums.next,
+ struct btrfs_ordered_sum,
+ list);
+ ret = btrfs_csum_file_blocks(trans, log, sums);
+ BUG_ON(ret);
+ list_del(&sums->list);
+ kfree(sums);
+ }
+ return 0;
+}
+
+/* log a single inode in the tree log.
+ * At least one parent directory for this inode must exist in the tree
+ * or be logged already.
+ *
+ * Any items from this inode changed by the current transaction are copied
+ * to the log tree. An extra reference is taken on any extents in this
+ * file, allowing us to avoid a whole pile of corner cases around logging
+ * blocks that have been removed from the tree.
+ *
+ * See LOG_INODE_ALL and related defines for a description of what inode_only
+ * does.
+ *
+ * This handles both files and directories.
+ */
+static int __btrfs_log_inode(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct inode *inode,
+ int inode_only)
+{
+ struct btrfs_path *path;
+ struct btrfs_path *dst_path;
+ struct btrfs_key min_key;
+ struct btrfs_key max_key;
+ struct btrfs_root *log = root->log_root;
+ struct extent_buffer *src = NULL;
+ u32 size;
+ int ret;
+ int nritems;
+ int ins_start_slot = 0;
+ int ins_nr;
+
+ log = root->log_root;
+
+ path = btrfs_alloc_path();
+ dst_path = btrfs_alloc_path();
+
+ min_key.objectid = inode->i_ino;
+ min_key.type = BTRFS_INODE_ITEM_KEY;
+ min_key.offset = 0;
+
+ max_key.objectid = inode->i_ino;
+ if (inode_only == LOG_INODE_EXISTS || S_ISDIR(inode->i_mode))
+ max_key.type = BTRFS_XATTR_ITEM_KEY;
+ else
+ max_key.type = (u8)-1;
+ max_key.offset = (u64)-1;
+
+ /*
+ * if this inode has already been logged and we're in inode_only
+ * mode, we don't want to delete the things that have already
+ * been written to the log.
+ *
+ * But, if the inode has been through an inode_only log,
+ * the logged_trans field is not set. This allows us to catch
+ * any new names for this inode in the backrefs by logging it
+ * again
+ */
+ if (inode_only == LOG_INODE_EXISTS &&
+ BTRFS_I(inode)->logged_trans == trans->transid) {
+ btrfs_free_path(path);
+ btrfs_free_path(dst_path);
+ goto out;
+ }
+ mutex_lock(&BTRFS_I(inode)->log_mutex);
+
+ /*
+ * a brute force approach to making sure we get the most uptodate
+ * copies of everything.
+ */
+ if (S_ISDIR(inode->i_mode)) {
+ int max_key_type = BTRFS_DIR_LOG_INDEX_KEY;
+
+ if (inode_only == LOG_INODE_EXISTS)
+ max_key_type = BTRFS_XATTR_ITEM_KEY;
+ ret = drop_objectid_items(trans, log, path,
+ inode->i_ino, max_key_type);
+ } else {
+ ret = btrfs_truncate_inode_items(trans, log, inode, 0, 0);
+ }
+ BUG_ON(ret);
+ path->keep_locks = 1;
+
+ while (1) {
+ ins_nr = 0;
+ ret = btrfs_search_forward(root, &min_key, &max_key,
+ path, 0, trans->transid);
+ if (ret != 0)
+ break;
+again:
+ /* note, ins_nr might be > 0 here, cleanup outside the loop */
+ if (min_key.objectid != inode->i_ino)
+ break;
+ if (min_key.type > max_key.type)
+ break;
+
+ src = path->nodes[0];
+ size = btrfs_item_size_nr(src, path->slots[0]);
+ if (ins_nr && ins_start_slot + ins_nr == path->slots[0]) {
+ ins_nr++;
+ goto next_slot;
+ } else if (!ins_nr) {
+ ins_start_slot = path->slots[0];
+ ins_nr = 1;
+ goto next_slot;
+ }
+
+ ret = copy_items(trans, log, dst_path, src, ins_start_slot,
+ ins_nr, inode_only);
+ BUG_ON(ret);
+ ins_nr = 1;
+ ins_start_slot = path->slots[0];
+next_slot:
+
+ nritems = btrfs_header_nritems(path->nodes[0]);
+ path->slots[0]++;
+ if (path->slots[0] < nritems) {
+ btrfs_item_key_to_cpu(path->nodes[0], &min_key,
+ path->slots[0]);
+ goto again;
+ }
+ if (ins_nr) {
+ ret = copy_items(trans, log, dst_path, src,
+ ins_start_slot,
+ ins_nr, inode_only);
+ BUG_ON(ret);
+ ins_nr = 0;
+ }
+ btrfs_release_path(root, path);
+
+ if (min_key.offset < (u64)-1)
+ min_key.offset++;
+ else if (min_key.type < (u8)-1)
+ min_key.type++;
+ else if (min_key.objectid < (u64)-1)
+ min_key.objectid++;
+ else
+ break;
+ }
+ if (ins_nr) {
+ ret = copy_items(trans, log, dst_path, src,
+ ins_start_slot,
+ ins_nr, inode_only);
+ BUG_ON(ret);
+ ins_nr = 0;
+ }
+ WARN_ON(ins_nr);
+ if (inode_only == LOG_INODE_ALL && S_ISDIR(inode->i_mode)) {
+ btrfs_release_path(root, path);
+ btrfs_release_path(log, dst_path);
+ BTRFS_I(inode)->log_dirty_trans = 0;
+ ret = log_directory_changes(trans, root, inode, path, dst_path);
+ BUG_ON(ret);
+ }
+ BTRFS_I(inode)->logged_trans = trans->transid;
+ mutex_unlock(&BTRFS_I(inode)->log_mutex);
+
+ btrfs_free_path(path);
+ btrfs_free_path(dst_path);
+
+ mutex_lock(&root->fs_info->tree_log_mutex);
+ ret = update_log_root(trans, log);
+ BUG_ON(ret);
+ mutex_unlock(&root->fs_info->tree_log_mutex);
+out:
+ return 0;
+}
+
+int btrfs_log_inode(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct inode *inode,
+ int inode_only)
+{
+ int ret;
+
+ start_log_trans(trans, root);
+ ret = __btrfs_log_inode(trans, root, inode, inode_only);
+ end_log_trans(root);
+ return ret;
+}
+
+/*
+ * helper function around btrfs_log_inode to make sure newly created
+ * parent directories also end up in the log. A minimal inode and backref
+ * only logging is done of any parent directories that are older than
+ * the last committed transaction
+ */
+int btrfs_log_dentry(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct dentry *dentry)
+{
+ int inode_only = LOG_INODE_ALL;
+ struct super_block *sb;
+ int ret;
+
+ start_log_trans(trans, root);
+ sb = dentry->d_inode->i_sb;
+ while (1) {
+ ret = __btrfs_log_inode(trans, root, dentry->d_inode,
+ inode_only);
+ BUG_ON(ret);
+ inode_only = LOG_INODE_EXISTS;
+
+ dentry = dentry->d_parent;
+ if (!dentry || !dentry->d_inode || sb != dentry->d_inode->i_sb)
+ break;
+
+ if (BTRFS_I(dentry->d_inode)->generation <=
+ root->fs_info->last_trans_committed)
+ break;
+ }
+ end_log_trans(root);
+ return 0;
+}
+
+/*
+ * it is not safe to log dentry if the chunk root has added new
+ * chunks. This returns 0 if the dentry was logged, and 1 otherwise.
+ * If this returns 1, you must commit the transaction to safely get your
+ * data on disk.
+ */
+int btrfs_log_dentry_safe(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct dentry *dentry)
+{
+ u64 gen;
+ gen = root->fs_info->last_trans_new_blockgroup;
+ if (gen > root->fs_info->last_trans_committed)
+ return 1;
+ else
+ return btrfs_log_dentry(trans, root, dentry);
+}
+
+/*
+ * should be called during mount to recover any replay any log trees
+ * from the FS
+ */
+int btrfs_recover_log_trees(struct btrfs_root *log_root_tree)
+{
+ int ret;
+ struct btrfs_path *path;
+ struct btrfs_trans_handle *trans;
+ struct btrfs_key key;
+ struct btrfs_key found_key;
+ struct btrfs_key tmp_key;
+ struct btrfs_root *log;
+ struct btrfs_fs_info *fs_info = log_root_tree->fs_info;
+ u64 highest_inode;
+ struct walk_control wc = {
+ .process_func = process_one_buffer,
+ .stage = 0,
+ };
+
+ fs_info->log_root_recovering = 1;
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+
+ trans = btrfs_start_transaction(fs_info->tree_root, 1);
+
+ wc.trans = trans;
+ wc.pin = 1;
+
+ walk_log_tree(trans, log_root_tree, &wc);
+
+again:
+ key.objectid = BTRFS_TREE_LOG_OBJECTID;
+ key.offset = (u64)-1;
+ btrfs_set_key_type(&key, BTRFS_ROOT_ITEM_KEY);
+
+ while (1) {
+ ret = btrfs_search_slot(NULL, log_root_tree, &key, path, 0, 0);
+ if (ret < 0)
+ break;
+ if (ret > 0) {
+ if (path->slots[0] == 0)
+ break;
+ path->slots[0]--;
+ }
+ btrfs_item_key_to_cpu(path->nodes[0], &found_key,
+ path->slots[0]);
+ btrfs_release_path(log_root_tree, path);
+ if (found_key.objectid != BTRFS_TREE_LOG_OBJECTID)
+ break;
+
+ log = btrfs_read_fs_root_no_radix(log_root_tree,
+ &found_key);
+ BUG_ON(!log);
+
+
+ tmp_key.objectid = found_key.offset;
+ tmp_key.type = BTRFS_ROOT_ITEM_KEY;
+ tmp_key.offset = (u64)-1;
+
+ wc.replay_dest = btrfs_read_fs_root_no_name(fs_info, &tmp_key);
+ BUG_ON(!wc.replay_dest);
+
+ wc.replay_dest->log_root = log;
+ btrfs_record_root_in_trans(wc.replay_dest);
+ ret = walk_log_tree(trans, log, &wc);
+ BUG_ON(ret);
+
+ if (wc.stage == LOG_WALK_REPLAY_ALL) {
+ ret = fixup_inode_link_counts(trans, wc.replay_dest,
+ path);
+ BUG_ON(ret);
+ }
+ ret = btrfs_find_highest_inode(wc.replay_dest, &highest_inode);
+ if (ret == 0) {
+ wc.replay_dest->highest_inode = highest_inode;
+ wc.replay_dest->last_inode_alloc = highest_inode;
+ }
+
+ key.offset = found_key.offset - 1;
+ wc.replay_dest->log_root = NULL;
+ free_extent_buffer(log->node);
+ kfree(log);
+
+ if (found_key.offset == 0)
+ break;
+ }
+ btrfs_release_path(log_root_tree, path);
+
+ /* step one is to pin it all, step two is to replay just inodes */
+ if (wc.pin) {
+ wc.pin = 0;
+ wc.process_func = replay_one_buffer;
+ wc.stage = LOG_WALK_REPLAY_INODES;
+ goto again;
+ }
+ /* step three is to replay everything */
+ if (wc.stage < LOG_WALK_REPLAY_ALL) {
+ wc.stage++;
+ goto again;
+ }
+
+ btrfs_free_path(path);
+
+ free_extent_buffer(log_root_tree->node);
+ log_root_tree->log_root = NULL;
+ fs_info->log_root_recovering = 0;
+
+ /* step 4: commit the transaction, which also unpins the blocks */
+ btrfs_commit_transaction(trans, fs_info->tree_root);
+
+ kfree(log_root_tree);
+ return 0;
+}
diff --git a/fs/btrfs/tree-log.h b/fs/btrfs/tree-log.h
new file mode 100644
index 0000000..b9409b3
--- /dev/null
+++ b/fs/btrfs/tree-log.h
@@ -0,0 +1,41 @@
+/*
+ * Copyright (C) 2008 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __TREE_LOG_
+#define __TREE_LOG_
+
+int btrfs_sync_log(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root);
+int btrfs_free_log(struct btrfs_trans_handle *trans, struct btrfs_root *root);
+int btrfs_log_dentry(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct dentry *dentry);
+int btrfs_recover_log_trees(struct btrfs_root *tree_root);
+int btrfs_log_dentry_safe(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct dentry *dentry);
+int btrfs_log_inode(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, struct inode *inode,
+ int inode_only);
+int btrfs_del_dir_entries_in_log(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ const char *name, int name_len,
+ struct inode *dir, u64 index);
+int btrfs_del_inode_ref_in_log(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ const char *name, int name_len,
+ struct inode *inode, u64 dirid);
+#endif
diff --git a/fs/btrfs/version.h b/fs/btrfs/version.h
new file mode 100644
index 0000000..9bf3946
--- /dev/null
+++ b/fs/btrfs/version.h
@@ -0,0 +1,4 @@
+#ifndef __BTRFS_VERSION_H
+#define __BTRFS_VERSION_H
+#define BTRFS_BUILD_VERSION "Btrfs"
+#endif
diff --git a/fs/btrfs/version.sh b/fs/btrfs/version.sh
new file mode 100644
index 0000000..1ca1952
--- /dev/null
+++ b/fs/btrfs/version.sh
@@ -0,0 +1,43 @@
+#!/bin/bash
+#
+# determine-version -- report a useful version for releases
+#
+# Copyright 2008, Aron Griffis <agriffis@n01se.net>
+# Copyright 2008, Oracle
+# Released under the GNU GPLv2
+
+v="v0.16"
+
+which git &> /dev/null
+if [ $? == 0 ]; then
+ git branch >& /dev/null
+ if [ $? == 0 ]; then
+ if head=`git rev-parse --verify HEAD 2>/dev/null`; then
+ if tag=`git describe --tags 2>/dev/null`; then
+ v="$tag"
+ fi
+
+ # Are there uncommitted changes?
+ git update-index --refresh --unmerged > /dev/null
+ if git diff-index --name-only HEAD | \
+ grep -v "^scripts/package" \
+ | read dummy; then
+ v="$v"-dirty
+ fi
+ fi
+ fi
+fi
+
+echo "#ifndef __BUILD_VERSION" > .build-version.h
+echo "#define __BUILD_VERSION" >> .build-version.h
+echo "#define BTRFS_BUILD_VERSION \"Btrfs $v\"" >> .build-version.h
+echo "#endif" >> .build-version.h
+
+diff -q version.h .build-version.h >& /dev/null
+
+if [ $? == 0 ]; then
+ rm .build-version.h
+ exit 0
+fi
+
+mv .build-version.h version.h
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
new file mode 100644
index 0000000..b187b53
--- /dev/null
+++ b/fs/btrfs/volumes.c
@@ -0,0 +1,3218 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+#include <linux/sched.h>
+#include <linux/bio.h>
+#include <linux/buffer_head.h>
+#include <linux/blkdev.h>
+#include <linux/random.h>
+#include <linux/version.h>
+#include <asm/div64.h>
+#include "compat.h"
+#include "ctree.h"
+#include "extent_map.h"
+#include "disk-io.h"
+#include "transaction.h"
+#include "print-tree.h"
+#include "volumes.h"
+#include "async-thread.h"
+
+struct map_lookup {
+ u64 type;
+ int io_align;
+ int io_width;
+ int stripe_len;
+ int sector_size;
+ int num_stripes;
+ int sub_stripes;
+ struct btrfs_bio_stripe stripes[];
+};
+
+static int init_first_rw_device(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_device *device);
+static int btrfs_relocate_sys_chunks(struct btrfs_root *root);
+
+#define map_lookup_size(n) (sizeof(struct map_lookup) + \
+ (sizeof(struct btrfs_bio_stripe) * (n)))
+
+static DEFINE_MUTEX(uuid_mutex);
+static LIST_HEAD(fs_uuids);
+
+void btrfs_lock_volumes(void)
+{
+ mutex_lock(&uuid_mutex);
+}
+
+void btrfs_unlock_volumes(void)
+{
+ mutex_unlock(&uuid_mutex);
+}
+
+static void lock_chunks(struct btrfs_root *root)
+{
+ mutex_lock(&root->fs_info->chunk_mutex);
+}
+
+static void unlock_chunks(struct btrfs_root *root)
+{
+ mutex_unlock(&root->fs_info->chunk_mutex);
+}
+
+static void free_fs_devices(struct btrfs_fs_devices *fs_devices)
+{
+ struct btrfs_device *device;
+ WARN_ON(fs_devices->opened);
+ while (!list_empty(&fs_devices->devices)) {
+ device = list_entry(fs_devices->devices.next,
+ struct btrfs_device, dev_list);
+ list_del(&device->dev_list);
+ kfree(device->name);
+ kfree(device);
+ }
+ kfree(fs_devices);
+}
+
+int btrfs_cleanup_fs_uuids(void)
+{
+ struct btrfs_fs_devices *fs_devices;
+
+ while (!list_empty(&fs_uuids)) {
+ fs_devices = list_entry(fs_uuids.next,
+ struct btrfs_fs_devices, list);
+ list_del(&fs_devices->list);
+ free_fs_devices(fs_devices);
+ }
+ return 0;
+}
+
+static noinline struct btrfs_device *__find_device(struct list_head *head,
+ u64 devid, u8 *uuid)
+{
+ struct btrfs_device *dev;
+ struct list_head *cur;
+
+ list_for_each(cur, head) {
+ dev = list_entry(cur, struct btrfs_device, dev_list);
+ if (dev->devid == devid &&
+ (!uuid || !memcmp(dev->uuid, uuid, BTRFS_UUID_SIZE))) {
+ return dev;
+ }
+ }
+ return NULL;
+}
+
+static noinline struct btrfs_fs_devices *find_fsid(u8 *fsid)
+{
+ struct list_head *cur;
+ struct btrfs_fs_devices *fs_devices;
+
+ list_for_each(cur, &fs_uuids) {
+ fs_devices = list_entry(cur, struct btrfs_fs_devices, list);
+ if (memcmp(fsid, fs_devices->fsid, BTRFS_FSID_SIZE) == 0)
+ return fs_devices;
+ }
+ return NULL;
+}
+
+/*
+ * we try to collect pending bios for a device so we don't get a large
+ * number of procs sending bios down to the same device. This greatly
+ * improves the schedulers ability to collect and merge the bios.
+ *
+ * But, it also turns into a long list of bios to process and that is sure
+ * to eventually make the worker thread block. The solution here is to
+ * make some progress and then put this work struct back at the end of
+ * the list if the block device is congested. This way, multiple devices
+ * can make progress from a single worker thread.
+ */
+static noinline int run_scheduled_bios(struct btrfs_device *device)
+{
+ struct bio *pending;
+ struct backing_dev_info *bdi;
+ struct btrfs_fs_info *fs_info;
+ struct bio *tail;
+ struct bio *cur;
+ int again = 0;
+ unsigned long num_run = 0;
+ unsigned long limit;
+
+ bdi = device->bdev->bd_inode->i_mapping->backing_dev_info;
+ fs_info = device->dev_root->fs_info;
+ limit = btrfs_async_submit_limit(fs_info);
+ limit = limit * 2 / 3;
+
+loop:
+ spin_lock(&device->io_lock);
+
+ /* take all the bios off the list at once and process them
+ * later on (without the lock held). But, remember the
+ * tail and other pointers so the bios can be properly reinserted
+ * into the list if we hit congestion
+ */
+ pending = device->pending_bios;
+ tail = device->pending_bio_tail;
+ WARN_ON(pending && !tail);
+ device->pending_bios = NULL;
+ device->pending_bio_tail = NULL;
+
+ /*
+ * if pending was null this time around, no bios need processing
+ * at all and we can stop. Otherwise it'll loop back up again
+ * and do an additional check so no bios are missed.
+ *
+ * device->running_pending is used to synchronize with the
+ * schedule_bio code.
+ */
+ if (pending) {
+ again = 1;
+ device->running_pending = 1;
+ } else {
+ again = 0;
+ device->running_pending = 0;
+ }
+ spin_unlock(&device->io_lock);
+
+ while (pending) {
+ cur = pending;
+ pending = pending->bi_next;
+ cur->bi_next = NULL;
+ atomic_dec(&fs_info->nr_async_bios);
+
+ if (atomic_read(&fs_info->nr_async_bios) < limit &&
+ waitqueue_active(&fs_info->async_submit_wait))
+ wake_up(&fs_info->async_submit_wait);
+
+ BUG_ON(atomic_read(&cur->bi_cnt) == 0);
+ bio_get(cur);
+ submit_bio(cur->bi_rw, cur);
+ bio_put(cur);
+ num_run++;
+
+ /*
+ * we made progress, there is more work to do and the bdi
+ * is now congested. Back off and let other work structs
+ * run instead
+ */
+ if (pending && bdi_write_congested(bdi) &&
+ fs_info->fs_devices->open_devices > 1) {
+ struct bio *old_head;
+
+ spin_lock(&device->io_lock);
+
+ old_head = device->pending_bios;
+ device->pending_bios = pending;
+ if (device->pending_bio_tail)
+ tail->bi_next = old_head;
+ else
+ device->pending_bio_tail = tail;
+
+ spin_unlock(&device->io_lock);
+ btrfs_requeue_work(&device->work);
+ goto done;
+ }
+ }
+ if (again)
+ goto loop;
+done:
+ return 0;
+}
+
+static void pending_bios_fn(struct btrfs_work *work)
+{
+ struct btrfs_device *device;
+
+ device = container_of(work, struct btrfs_device, work);
+ run_scheduled_bios(device);
+}
+
+static noinline int device_list_add(const char *path,
+ struct btrfs_super_block *disk_super,
+ u64 devid, struct btrfs_fs_devices **fs_devices_ret)
+{
+ struct btrfs_device *device;
+ struct btrfs_fs_devices *fs_devices;
+ u64 found_transid = btrfs_super_generation(disk_super);
+
+ fs_devices = find_fsid(disk_super->fsid);
+ if (!fs_devices) {
+ fs_devices = kzalloc(sizeof(*fs_devices), GFP_NOFS);
+ if (!fs_devices)
+ return -ENOMEM;
+ INIT_LIST_HEAD(&fs_devices->devices);
+ INIT_LIST_HEAD(&fs_devices->alloc_list);
+ list_add(&fs_devices->list, &fs_uuids);
+ memcpy(fs_devices->fsid, disk_super->fsid, BTRFS_FSID_SIZE);
+ fs_devices->latest_devid = devid;
+ fs_devices->latest_trans = found_transid;
+ device = NULL;
+ } else {
+ device = __find_device(&fs_devices->devices, devid,
+ disk_super->dev_item.uuid);
+ }
+ if (!device) {
+ if (fs_devices->opened)
+ return -EBUSY;
+
+ device = kzalloc(sizeof(*device), GFP_NOFS);
+ if (!device) {
+ /* we can safely leave the fs_devices entry around */
+ return -ENOMEM;
+ }
+ device->devid = devid;
+ device->work.func = pending_bios_fn;
+ memcpy(device->uuid, disk_super->dev_item.uuid,
+ BTRFS_UUID_SIZE);
+ device->barriers = 1;
+ spin_lock_init(&device->io_lock);
+ device->name = kstrdup(path, GFP_NOFS);
+ if (!device->name) {
+ kfree(device);
+ return -ENOMEM;
+ }
+ INIT_LIST_HEAD(&device->dev_alloc_list);
+ list_add(&device->dev_list, &fs_devices->devices);
+ device->fs_devices = fs_devices;
+ fs_devices->num_devices++;
+ }
+
+ if (found_transid > fs_devices->latest_trans) {
+ fs_devices->latest_devid = devid;
+ fs_devices->latest_trans = found_transid;
+ }
+ *fs_devices_ret = fs_devices;
+ return 0;
+}
+
+static struct btrfs_fs_devices *clone_fs_devices(struct btrfs_fs_devices *orig)
+{
+ struct btrfs_fs_devices *fs_devices;
+ struct btrfs_device *device;
+ struct btrfs_device *orig_dev;
+
+ fs_devices = kzalloc(sizeof(*fs_devices), GFP_NOFS);
+ if (!fs_devices)
+ return ERR_PTR(-ENOMEM);
+
+ INIT_LIST_HEAD(&fs_devices->devices);
+ INIT_LIST_HEAD(&fs_devices->alloc_list);
+ INIT_LIST_HEAD(&fs_devices->list);
+ fs_devices->latest_devid = orig->latest_devid;
+ fs_devices->latest_trans = orig->latest_trans;
+ memcpy(fs_devices->fsid, orig->fsid, sizeof(fs_devices->fsid));
+
+ list_for_each_entry(orig_dev, &orig->devices, dev_list) {
+ device = kzalloc(sizeof(*device), GFP_NOFS);
+ if (!device)
+ goto error;
+
+ device->name = kstrdup(orig_dev->name, GFP_NOFS);
+ if (!device->name)
+ goto error;
+
+ device->devid = orig_dev->devid;
+ device->work.func = pending_bios_fn;
+ memcpy(device->uuid, orig_dev->uuid, sizeof(device->uuid));
+ device->barriers = 1;
+ spin_lock_init(&device->io_lock);
+ INIT_LIST_HEAD(&device->dev_list);
+ INIT_LIST_HEAD(&device->dev_alloc_list);
+
+ list_add(&device->dev_list, &fs_devices->devices);
+ device->fs_devices = fs_devices;
+ fs_devices->num_devices++;
+ }
+ return fs_devices;
+error:
+ free_fs_devices(fs_devices);
+ return ERR_PTR(-ENOMEM);
+}
+
+int btrfs_close_extra_devices(struct btrfs_fs_devices *fs_devices)
+{
+ struct list_head *tmp;
+ struct list_head *cur;
+ struct btrfs_device *device;
+
+ mutex_lock(&uuid_mutex);
+again:
+ list_for_each_safe(cur, tmp, &fs_devices->devices) {
+ device = list_entry(cur, struct btrfs_device, dev_list);
+ if (device->in_fs_metadata)
+ continue;
+
+ if (device->bdev) {
+ close_bdev_exclusive(device->bdev, device->mode);
+ device->bdev = NULL;
+ fs_devices->open_devices--;
+ }
+ if (device->writeable) {
+ list_del_init(&device->dev_alloc_list);
+ device->writeable = 0;
+ fs_devices->rw_devices--;
+ }
+ list_del_init(&device->dev_list);
+ fs_devices->num_devices--;
+ kfree(device->name);
+ kfree(device);
+ }
+
+ if (fs_devices->seed) {
+ fs_devices = fs_devices->seed;
+ goto again;
+ }
+
+ mutex_unlock(&uuid_mutex);
+ return 0;
+}
+
+static int __btrfs_close_devices(struct btrfs_fs_devices *fs_devices)
+{
+ struct list_head *cur;
+ struct btrfs_device *device;
+
+ if (--fs_devices->opened > 0)
+ return 0;
+
+ list_for_each(cur, &fs_devices->devices) {
+ device = list_entry(cur, struct btrfs_device, dev_list);
+ if (device->bdev) {
+ close_bdev_exclusive(device->bdev, device->mode);
+ fs_devices->open_devices--;
+ }
+ if (device->writeable) {
+ list_del_init(&device->dev_alloc_list);
+ fs_devices->rw_devices--;
+ }
+
+ device->bdev = NULL;
+ device->writeable = 0;
+ device->in_fs_metadata = 0;
+ }
+ WARN_ON(fs_devices->open_devices);
+ WARN_ON(fs_devices->rw_devices);
+ fs_devices->opened = 0;
+ fs_devices->seeding = 0;
+
+ return 0;
+}
+
+int btrfs_close_devices(struct btrfs_fs_devices *fs_devices)
+{
+ struct btrfs_fs_devices *seed_devices = NULL;
+ int ret;
+
+ mutex_lock(&uuid_mutex);
+ ret = __btrfs_close_devices(fs_devices);
+ if (!fs_devices->opened) {
+ seed_devices = fs_devices->seed;
+ fs_devices->seed = NULL;
+ }
+ mutex_unlock(&uuid_mutex);
+
+ while (seed_devices) {
+ fs_devices = seed_devices;
+ seed_devices = fs_devices->seed;
+ __btrfs_close_devices(fs_devices);
+ free_fs_devices(fs_devices);
+ }
+ return ret;
+}
+
+static int __btrfs_open_devices(struct btrfs_fs_devices *fs_devices,
+ fmode_t flags, void *holder)
+{
+ struct block_device *bdev;
+ struct list_head *head = &fs_devices->devices;
+ struct list_head *cur;
+ struct btrfs_device *device;
+ struct block_device *latest_bdev = NULL;
+ struct buffer_head *bh;
+ struct btrfs_super_block *disk_super;
+ u64 latest_devid = 0;
+ u64 latest_transid = 0;
+ u64 devid;
+ int seeding = 1;
+ int ret = 0;
+
+ list_for_each(cur, head) {
+ device = list_entry(cur, struct btrfs_device, dev_list);
+ if (device->bdev)
+ continue;
+ if (!device->name)
+ continue;
+
+ bdev = open_bdev_exclusive(device->name, flags, holder);
+ if (IS_ERR(bdev)) {
+ printk(KERN_INFO "open %s failed\n", device->name);
+ goto error;
+ }
+ set_blocksize(bdev, 4096);
+
+ bh = btrfs_read_dev_super(bdev);
+ if (!bh)
+ goto error_close;
+
+ disk_super = (struct btrfs_super_block *)bh->b_data;
+ devid = le64_to_cpu(disk_super->dev_item.devid);
+ if (devid != device->devid)
+ goto error_brelse;
+
+ if (memcmp(device->uuid, disk_super->dev_item.uuid,
+ BTRFS_UUID_SIZE))
+ goto error_brelse;
+
+ device->generation = btrfs_super_generation(disk_super);
+ if (!latest_transid || device->generation > latest_transid) {
+ latest_devid = devid;
+ latest_transid = device->generation;
+ latest_bdev = bdev;
+ }
+
+ if (btrfs_super_flags(disk_super) & BTRFS_SUPER_FLAG_SEEDING) {
+ device->writeable = 0;
+ } else {
+ device->writeable = !bdev_read_only(bdev);
+ seeding = 0;
+ }
+
+ device->bdev = bdev;
+ device->in_fs_metadata = 0;
+ device->mode = flags;
+
+ fs_devices->open_devices++;
+ if (device->writeable) {
+ fs_devices->rw_devices++;
+ list_add(&device->dev_alloc_list,
+ &fs_devices->alloc_list);
+ }
+ continue;
+
+error_brelse:
+ brelse(bh);
+error_close:
+ close_bdev_exclusive(bdev, FMODE_READ);
+error:
+ continue;
+ }
+ if (fs_devices->open_devices == 0) {
+ ret = -EIO;
+ goto out;
+ }
+ fs_devices->seeding = seeding;
+ fs_devices->opened = 1;
+ fs_devices->latest_bdev = latest_bdev;
+ fs_devices->latest_devid = latest_devid;
+ fs_devices->latest_trans = latest_transid;
+ fs_devices->total_rw_bytes = 0;
+out:
+ return ret;
+}
+
+int btrfs_open_devices(struct btrfs_fs_devices *fs_devices,
+ fmode_t flags, void *holder)
+{
+ int ret;
+
+ mutex_lock(&uuid_mutex);
+ if (fs_devices->opened) {
+ fs_devices->opened++;
+ ret = 0;
+ } else {
+ ret = __btrfs_open_devices(fs_devices, flags, holder);
+ }
+ mutex_unlock(&uuid_mutex);
+ return ret;
+}
+
+int btrfs_scan_one_device(const char *path, fmode_t flags, void *holder,
+ struct btrfs_fs_devices **fs_devices_ret)
+{
+ struct btrfs_super_block *disk_super;
+ struct block_device *bdev;
+ struct buffer_head *bh;
+ int ret;
+ u64 devid;
+ u64 transid;
+
+ mutex_lock(&uuid_mutex);
+
+ bdev = open_bdev_exclusive(path, flags, holder);
+
+ if (IS_ERR(bdev)) {
+ ret = PTR_ERR(bdev);
+ goto error;
+ }
+
+ ret = set_blocksize(bdev, 4096);
+ if (ret)
+ goto error_close;
+ bh = btrfs_read_dev_super(bdev);
+ if (!bh) {
+ ret = -EIO;
+ goto error_close;
+ }
+ disk_super = (struct btrfs_super_block *)bh->b_data;
+ devid = le64_to_cpu(disk_super->dev_item.devid);
+ transid = btrfs_super_generation(disk_super);
+ if (disk_super->label[0])
+ printk(KERN_INFO "device label %s ", disk_super->label);
+ else {
+ /* FIXME, make a readl uuid parser */
+ printk(KERN_INFO "device fsid %llx-%llx ",
+ *(unsigned long long *)disk_super->fsid,
+ *(unsigned long long *)(disk_super->fsid + 8));
+ }
+ printk(KERN_INFO "devid %llu transid %llu %s\n",
+ (unsigned long long)devid, (unsigned long long)transid, path);
+ ret = device_list_add(path, disk_super, devid, fs_devices_ret);
+
+ brelse(bh);
+error_close:
+ close_bdev_exclusive(bdev, flags);
+error:
+ mutex_unlock(&uuid_mutex);
+ return ret;
+}
+
+/*
+ * this uses a pretty simple search, the expectation is that it is
+ * called very infrequently and that a given device has a small number
+ * of extents
+ */
+static noinline int find_free_dev_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_device *device,
+ u64 num_bytes, u64 *start)
+{
+ struct btrfs_key key;
+ struct btrfs_root *root = device->dev_root;
+ struct btrfs_dev_extent *dev_extent = NULL;
+ struct btrfs_path *path;
+ u64 hole_size = 0;
+ u64 last_byte = 0;
+ u64 search_start = 0;
+ u64 search_end = device->total_bytes;
+ int ret;
+ int slot = 0;
+ int start_found;
+ struct extent_buffer *l;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+ path->reada = 2;
+ start_found = 0;
+
+ /* FIXME use last free of some kind */
+
+ /* we don't want to overwrite the superblock on the drive,
+ * so we make sure to start at an offset of at least 1MB
+ */
+ search_start = max((u64)1024 * 1024, search_start);
+
+ if (root->fs_info->alloc_start + num_bytes <= device->total_bytes)
+ search_start = max(root->fs_info->alloc_start, search_start);
+
+ key.objectid = device->devid;
+ key.offset = search_start;
+ key.type = BTRFS_DEV_EXTENT_KEY;
+ ret = btrfs_search_slot(trans, root, &key, path, 0, 0);
+ if (ret < 0)
+ goto error;
+ ret = btrfs_previous_item(root, path, 0, key.type);
+ if (ret < 0)
+ goto error;
+ l = path->nodes[0];
+ btrfs_item_key_to_cpu(l, &key, path->slots[0]);
+ while (1) {
+ l = path->nodes[0];
+ slot = path->slots[0];
+ if (slot >= btrfs_header_nritems(l)) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret == 0)
+ continue;
+ if (ret < 0)
+ goto error;
+no_more_items:
+ if (!start_found) {
+ if (search_start >= search_end) {
+ ret = -ENOSPC;
+ goto error;
+ }
+ *start = search_start;
+ start_found = 1;
+ goto check_pending;
+ }
+ *start = last_byte > search_start ?
+ last_byte : search_start;
+ if (search_end <= *start) {
+ ret = -ENOSPC;
+ goto error;
+ }
+ goto check_pending;
+ }
+ btrfs_item_key_to_cpu(l, &key, slot);
+
+ if (key.objectid < device->devid)
+ goto next;
+
+ if (key.objectid > device->devid)
+ goto no_more_items;
+
+ if (key.offset >= search_start && key.offset > last_byte &&
+ start_found) {
+ if (last_byte < search_start)
+ last_byte = search_start;
+ hole_size = key.offset - last_byte;
+ if (key.offset > last_byte &&
+ hole_size >= num_bytes) {
+ *start = last_byte;
+ goto check_pending;
+ }
+ }
+ if (btrfs_key_type(&key) != BTRFS_DEV_EXTENT_KEY)
+ goto next;
+
+ start_found = 1;
+ dev_extent = btrfs_item_ptr(l, slot, struct btrfs_dev_extent);
+ last_byte = key.offset + btrfs_dev_extent_length(l, dev_extent);
+next:
+ path->slots[0]++;
+ cond_resched();
+ }
+check_pending:
+ /* we have to make sure we didn't find an extent that has already
+ * been allocated by the map tree or the original allocation
+ */
+ BUG_ON(*start < search_start);
+
+ if (*start + num_bytes > search_end) {
+ ret = -ENOSPC;
+ goto error;
+ }
+ /* check for pending inserts here */
+ ret = 0;
+
+error:
+ btrfs_free_path(path);
+ return ret;
+}
+
+static int btrfs_free_dev_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_device *device,
+ u64 start)
+{
+ int ret;
+ struct btrfs_path *path;
+ struct btrfs_root *root = device->dev_root;
+ struct btrfs_key key;
+ struct btrfs_key found_key;
+ struct extent_buffer *leaf = NULL;
+ struct btrfs_dev_extent *extent = NULL;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ key.objectid = device->devid;
+ key.offset = start;
+ key.type = BTRFS_DEV_EXTENT_KEY;
+
+ ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
+ if (ret > 0) {
+ ret = btrfs_previous_item(root, path, key.objectid,
+ BTRFS_DEV_EXTENT_KEY);
+ BUG_ON(ret);
+ leaf = path->nodes[0];
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+ extent = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_dev_extent);
+ BUG_ON(found_key.offset > start || found_key.offset +
+ btrfs_dev_extent_length(leaf, extent) < start);
+ ret = 0;
+ } else if (ret == 0) {
+ leaf = path->nodes[0];
+ extent = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_dev_extent);
+ }
+ BUG_ON(ret);
+
+ if (device->bytes_used > 0)
+ device->bytes_used -= btrfs_dev_extent_length(leaf, extent);
+ ret = btrfs_del_item(trans, root, path);
+ BUG_ON(ret);
+
+ btrfs_free_path(path);
+ return ret;
+}
+
+int btrfs_alloc_dev_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_device *device,
+ u64 chunk_tree, u64 chunk_objectid,
+ u64 chunk_offset, u64 start, u64 num_bytes)
+{
+ int ret;
+ struct btrfs_path *path;
+ struct btrfs_root *root = device->dev_root;
+ struct btrfs_dev_extent *extent;
+ struct extent_buffer *leaf;
+ struct btrfs_key key;
+
+ WARN_ON(!device->in_fs_metadata);
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ key.objectid = device->devid;
+ key.offset = start;
+ key.type = BTRFS_DEV_EXTENT_KEY;
+ ret = btrfs_insert_empty_item(trans, root, path, &key,
+ sizeof(*extent));
+ BUG_ON(ret);
+
+ leaf = path->nodes[0];
+ extent = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_dev_extent);
+ btrfs_set_dev_extent_chunk_tree(leaf, extent, chunk_tree);
+ btrfs_set_dev_extent_chunk_objectid(leaf, extent, chunk_objectid);
+ btrfs_set_dev_extent_chunk_offset(leaf, extent, chunk_offset);
+
+ write_extent_buffer(leaf, root->fs_info->chunk_tree_uuid,
+ (unsigned long)btrfs_dev_extent_chunk_tree_uuid(extent),
+ BTRFS_UUID_SIZE);
+
+ btrfs_set_dev_extent_length(leaf, extent, num_bytes);
+ btrfs_mark_buffer_dirty(leaf);
+ btrfs_free_path(path);
+ return ret;
+}
+
+static noinline int find_next_chunk(struct btrfs_root *root,
+ u64 objectid, u64 *offset)
+{
+ struct btrfs_path *path;
+ int ret;
+ struct btrfs_key key;
+ struct btrfs_chunk *chunk;
+ struct btrfs_key found_key;
+
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+
+ key.objectid = objectid;
+ key.offset = (u64)-1;
+ key.type = BTRFS_CHUNK_ITEM_KEY;
+
+ ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ if (ret < 0)
+ goto error;
+
+ BUG_ON(ret == 0);
+
+ ret = btrfs_previous_item(root, path, 0, BTRFS_CHUNK_ITEM_KEY);
+ if (ret) {
+ *offset = 0;
+ } else {
+ btrfs_item_key_to_cpu(path->nodes[0], &found_key,
+ path->slots[0]);
+ if (found_key.objectid != objectid)
+ *offset = 0;
+ else {
+ chunk = btrfs_item_ptr(path->nodes[0], path->slots[0],
+ struct btrfs_chunk);
+ *offset = found_key.offset +
+ btrfs_chunk_length(path->nodes[0], chunk);
+ }
+ }
+ ret = 0;
+error:
+ btrfs_free_path(path);
+ return ret;
+}
+
+static noinline int find_next_devid(struct btrfs_root *root, u64 *objectid)
+{
+ int ret;
+ struct btrfs_key key;
+ struct btrfs_key found_key;
+ struct btrfs_path *path;
+
+ root = root->fs_info->chunk_root;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ key.objectid = BTRFS_DEV_ITEMS_OBJECTID;
+ key.type = BTRFS_DEV_ITEM_KEY;
+ key.offset = (u64)-1;
+
+ ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ if (ret < 0)
+ goto error;
+
+ BUG_ON(ret == 0);
+
+ ret = btrfs_previous_item(root, path, BTRFS_DEV_ITEMS_OBJECTID,
+ BTRFS_DEV_ITEM_KEY);
+ if (ret) {
+ *objectid = 1;
+ } else {
+ btrfs_item_key_to_cpu(path->nodes[0], &found_key,
+ path->slots[0]);
+ *objectid = found_key.offset + 1;
+ }
+ ret = 0;
+error:
+ btrfs_free_path(path);
+ return ret;
+}
+
+/*
+ * the device information is stored in the chunk root
+ * the btrfs_device struct should be fully filled in
+ */
+int btrfs_add_device(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_device *device)
+{
+ int ret;
+ struct btrfs_path *path;
+ struct btrfs_dev_item *dev_item;
+ struct extent_buffer *leaf;
+ struct btrfs_key key;
+ unsigned long ptr;
+
+ root = root->fs_info->chunk_root;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ key.objectid = BTRFS_DEV_ITEMS_OBJECTID;
+ key.type = BTRFS_DEV_ITEM_KEY;
+ key.offset = device->devid;
+
+ ret = btrfs_insert_empty_item(trans, root, path, &key,
+ sizeof(*dev_item));
+ if (ret)
+ goto out;
+
+ leaf = path->nodes[0];
+ dev_item = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_dev_item);
+
+ btrfs_set_device_id(leaf, dev_item, device->devid);
+ btrfs_set_device_generation(leaf, dev_item, 0);
+ btrfs_set_device_type(leaf, dev_item, device->type);
+ btrfs_set_device_io_align(leaf, dev_item, device->io_align);
+ btrfs_set_device_io_width(leaf, dev_item, device->io_width);
+ btrfs_set_device_sector_size(leaf, dev_item, device->sector_size);
+ btrfs_set_device_total_bytes(leaf, dev_item, device->total_bytes);
+ btrfs_set_device_bytes_used(leaf, dev_item, device->bytes_used);
+ btrfs_set_device_group(leaf, dev_item, 0);
+ btrfs_set_device_seek_speed(leaf, dev_item, 0);
+ btrfs_set_device_bandwidth(leaf, dev_item, 0);
+ btrfs_set_device_start_offset(leaf, dev_item, 0);
+
+ ptr = (unsigned long)btrfs_device_uuid(dev_item);
+ write_extent_buffer(leaf, device->uuid, ptr, BTRFS_UUID_SIZE);
+ ptr = (unsigned long)btrfs_device_fsid(dev_item);
+ write_extent_buffer(leaf, root->fs_info->fsid, ptr, BTRFS_UUID_SIZE);
+ btrfs_mark_buffer_dirty(leaf);
+
+ ret = 0;
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
+static int btrfs_rm_dev_item(struct btrfs_root *root,
+ struct btrfs_device *device)
+{
+ int ret;
+ struct btrfs_path *path;
+ struct btrfs_key key;
+ struct btrfs_trans_handle *trans;
+
+ root = root->fs_info->chunk_root;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ trans = btrfs_start_transaction(root, 1);
+ key.objectid = BTRFS_DEV_ITEMS_OBJECTID;
+ key.type = BTRFS_DEV_ITEM_KEY;
+ key.offset = device->devid;
+ lock_chunks(root);
+
+ ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
+ if (ret < 0)
+ goto out;
+
+ if (ret > 0) {
+ ret = -ENOENT;
+ goto out;
+ }
+
+ ret = btrfs_del_item(trans, root, path);
+ if (ret)
+ goto out;
+out:
+ btrfs_free_path(path);
+ unlock_chunks(root);
+ btrfs_commit_transaction(trans, root);
+ return ret;
+}
+
+int btrfs_rm_device(struct btrfs_root *root, char *device_path)
+{
+ struct btrfs_device *device;
+ struct btrfs_device *next_device;
+ struct block_device *bdev;
+ struct buffer_head *bh = NULL;
+ struct btrfs_super_block *disk_super;
+ u64 all_avail;
+ u64 devid;
+ u64 num_devices;
+ u8 *dev_uuid;
+ int ret = 0;
+
+ mutex_lock(&uuid_mutex);
+ mutex_lock(&root->fs_info->volume_mutex);
+
+ all_avail = root->fs_info->avail_data_alloc_bits |
+ root->fs_info->avail_system_alloc_bits |
+ root->fs_info->avail_metadata_alloc_bits;
+
+ if ((all_avail & BTRFS_BLOCK_GROUP_RAID10) &&
+ root->fs_info->fs_devices->rw_devices <= 4) {
+ printk(KERN_ERR "btrfs: unable to go below four devices "
+ "on raid10\n");
+ ret = -EINVAL;
+ goto out;
+ }
+
+ if ((all_avail & BTRFS_BLOCK_GROUP_RAID1) &&
+ root->fs_info->fs_devices->rw_devices <= 2) {
+ printk(KERN_ERR "btrfs: unable to go below two "
+ "devices on raid1\n");
+ ret = -EINVAL;
+ goto out;
+ }
+
+ if (strcmp(device_path, "missing") == 0) {
+ struct list_head *cur;
+ struct list_head *devices;
+ struct btrfs_device *tmp;
+
+ device = NULL;
+ devices = &root->fs_info->fs_devices->devices;
+ list_for_each(cur, devices) {
+ tmp = list_entry(cur, struct btrfs_device, dev_list);
+ if (tmp->in_fs_metadata && !tmp->bdev) {
+ device = tmp;
+ break;
+ }
+ }
+ bdev = NULL;
+ bh = NULL;
+ disk_super = NULL;
+ if (!device) {
+ printk(KERN_ERR "btrfs: no missing devices found to "
+ "remove\n");
+ goto out;
+ }
+ } else {
+ bdev = open_bdev_exclusive(device_path, FMODE_READ,
+ root->fs_info->bdev_holder);
+ if (IS_ERR(bdev)) {
+ ret = PTR_ERR(bdev);
+ goto out;
+ }
+
+ set_blocksize(bdev, 4096);
+ bh = btrfs_read_dev_super(bdev);
+ if (!bh) {
+ ret = -EIO;
+ goto error_close;
+ }
+ disk_super = (struct btrfs_super_block *)bh->b_data;
+ devid = le64_to_cpu(disk_super->dev_item.devid);
+ dev_uuid = disk_super->dev_item.uuid;
+ device = btrfs_find_device(root, devid, dev_uuid,
+ disk_super->fsid);
+ if (!device) {
+ ret = -ENOENT;
+ goto error_brelse;
+ }
+ }
+
+ if (device->writeable && root->fs_info->fs_devices->rw_devices == 1) {
+ printk(KERN_ERR "btrfs: unable to remove the only writeable "
+ "device\n");
+ ret = -EINVAL;
+ goto error_brelse;
+ }
+
+ if (device->writeable) {
+ list_del_init(&device->dev_alloc_list);
+ root->fs_info->fs_devices->rw_devices--;
+ }
+
+ ret = btrfs_shrink_device(device, 0);
+ if (ret)
+ goto error_brelse;
+
+ ret = btrfs_rm_dev_item(root->fs_info->chunk_root, device);
+ if (ret)
+ goto error_brelse;
+
+ device->in_fs_metadata = 0;
+ list_del_init(&device->dev_list);
+ device->fs_devices->num_devices--;
+
+ next_device = list_entry(root->fs_info->fs_devices->devices.next,
+ struct btrfs_device, dev_list);
+ if (device->bdev == root->fs_info->sb->s_bdev)
+ root->fs_info->sb->s_bdev = next_device->bdev;
+ if (device->bdev == root->fs_info->fs_devices->latest_bdev)
+ root->fs_info->fs_devices->latest_bdev = next_device->bdev;
+
+ if (device->bdev) {
+ close_bdev_exclusive(device->bdev, device->mode);
+ device->bdev = NULL;
+ device->fs_devices->open_devices--;
+ }
+
+ num_devices = btrfs_super_num_devices(&root->fs_info->super_copy) - 1;
+ btrfs_set_super_num_devices(&root->fs_info->super_copy, num_devices);
+
+ if (device->fs_devices->open_devices == 0) {
+ struct btrfs_fs_devices *fs_devices;
+ fs_devices = root->fs_info->fs_devices;
+ while (fs_devices) {
+ if (fs_devices->seed == device->fs_devices)
+ break;
+ fs_devices = fs_devices->seed;
+ }
+ fs_devices->seed = device->fs_devices->seed;
+ device->fs_devices->seed = NULL;
+ __btrfs_close_devices(device->fs_devices);
+ free_fs_devices(device->fs_devices);
+ }
+
+ /*
+ * at this point, the device is zero sized. We want to
+ * remove it from the devices list and zero out the old super
+ */
+ if (device->writeable) {
+ /* make sure this device isn't detected as part of
+ * the FS anymore
+ */
+ memset(&disk_super->magic, 0, sizeof(disk_super->magic));
+ set_buffer_dirty(bh);
+ sync_dirty_buffer(bh);
+ }
+
+ kfree(device->name);
+ kfree(device);
+ ret = 0;
+
+error_brelse:
+ brelse(bh);
+error_close:
+ if (bdev)
+ close_bdev_exclusive(bdev, FMODE_READ);
+out:
+ mutex_unlock(&root->fs_info->volume_mutex);
+ mutex_unlock(&uuid_mutex);
+ return ret;
+}
+
+/*
+ * does all the dirty work required for changing file system's UUID.
+ */
+static int btrfs_prepare_sprout(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root)
+{
+ struct btrfs_fs_devices *fs_devices = root->fs_info->fs_devices;
+ struct btrfs_fs_devices *old_devices;
+ struct btrfs_fs_devices *seed_devices;
+ struct btrfs_super_block *disk_super = &root->fs_info->super_copy;
+ struct btrfs_device *device;
+ u64 super_flags;
+
+ BUG_ON(!mutex_is_locked(&uuid_mutex));
+ if (!fs_devices->seeding)
+ return -EINVAL;
+
+ seed_devices = kzalloc(sizeof(*fs_devices), GFP_NOFS);
+ if (!seed_devices)
+ return -ENOMEM;
+
+ old_devices = clone_fs_devices(fs_devices);
+ if (IS_ERR(old_devices)) {
+ kfree(seed_devices);
+ return PTR_ERR(old_devices);
+ }
+
+ list_add(&old_devices->list, &fs_uuids);
+
+ memcpy(seed_devices, fs_devices, sizeof(*seed_devices));
+ seed_devices->opened = 1;
+ INIT_LIST_HEAD(&seed_devices->devices);
+ INIT_LIST_HEAD(&seed_devices->alloc_list);
+ list_splice_init(&fs_devices->devices, &seed_devices->devices);
+ list_splice_init(&fs_devices->alloc_list, &seed_devices->alloc_list);
+ list_for_each_entry(device, &seed_devices->devices, dev_list) {
+ device->fs_devices = seed_devices;
+ }
+
+ fs_devices->seeding = 0;
+ fs_devices->num_devices = 0;
+ fs_devices->open_devices = 0;
+ fs_devices->seed = seed_devices;
+
+ generate_random_uuid(fs_devices->fsid);
+ memcpy(root->fs_info->fsid, fs_devices->fsid, BTRFS_FSID_SIZE);
+ memcpy(disk_super->fsid, fs_devices->fsid, BTRFS_FSID_SIZE);
+ super_flags = btrfs_super_flags(disk_super) &
+ ~BTRFS_SUPER_FLAG_SEEDING;
+ btrfs_set_super_flags(disk_super, super_flags);
+
+ return 0;
+}
+
+/*
+ * strore the expected generation for seed devices in device items.
+ */
+static int btrfs_finish_sprout(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root)
+{
+ struct btrfs_path *path;
+ struct extent_buffer *leaf;
+ struct btrfs_dev_item *dev_item;
+ struct btrfs_device *device;
+ struct btrfs_key key;
+ u8 fs_uuid[BTRFS_UUID_SIZE];
+ u8 dev_uuid[BTRFS_UUID_SIZE];
+ u64 devid;
+ int ret;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ root = root->fs_info->chunk_root;
+ key.objectid = BTRFS_DEV_ITEMS_OBJECTID;
+ key.offset = 0;
+ key.type = BTRFS_DEV_ITEM_KEY;
+
+ while (1) {
+ ret = btrfs_search_slot(trans, root, &key, path, 0, 1);
+ if (ret < 0)
+ goto error;
+
+ leaf = path->nodes[0];
+next_slot:
+ if (path->slots[0] >= btrfs_header_nritems(leaf)) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret > 0)
+ break;
+ if (ret < 0)
+ goto error;
+ leaf = path->nodes[0];
+ btrfs_item_key_to_cpu(leaf, &key, path->slots[0]);
+ btrfs_release_path(root, path);
+ continue;
+ }
+
+ btrfs_item_key_to_cpu(leaf, &key, path->slots[0]);
+ if (key.objectid != BTRFS_DEV_ITEMS_OBJECTID ||
+ key.type != BTRFS_DEV_ITEM_KEY)
+ break;
+
+ dev_item = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_dev_item);
+ devid = btrfs_device_id(leaf, dev_item);
+ read_extent_buffer(leaf, dev_uuid,
+ (unsigned long)btrfs_device_uuid(dev_item),
+ BTRFS_UUID_SIZE);
+ read_extent_buffer(leaf, fs_uuid,
+ (unsigned long)btrfs_device_fsid(dev_item),
+ BTRFS_UUID_SIZE);
+ device = btrfs_find_device(root, devid, dev_uuid, fs_uuid);
+ BUG_ON(!device);
+
+ if (device->fs_devices->seeding) {
+ btrfs_set_device_generation(leaf, dev_item,
+ device->generation);
+ btrfs_mark_buffer_dirty(leaf);
+ }
+
+ path->slots[0]++;
+ goto next_slot;
+ }
+ ret = 0;
+error:
+ btrfs_free_path(path);
+ return ret;
+}
+
+int btrfs_init_new_device(struct btrfs_root *root, char *device_path)
+{
+ struct btrfs_trans_handle *trans;
+ struct btrfs_device *device;
+ struct block_device *bdev;
+ struct list_head *cur;
+ struct list_head *devices;
+ struct super_block *sb = root->fs_info->sb;
+ u64 total_bytes;
+ int seeding_dev = 0;
+ int ret = 0;
+
+ if ((sb->s_flags & MS_RDONLY) && !root->fs_info->fs_devices->seeding)
+ return -EINVAL;
+
+ bdev = open_bdev_exclusive(device_path, 0, root->fs_info->bdev_holder);
+ if (!bdev)
+ return -EIO;
+
+ if (root->fs_info->fs_devices->seeding) {
+ seeding_dev = 1;
+ down_write(&sb->s_umount);
+ mutex_lock(&uuid_mutex);
+ }
+
+ filemap_write_and_wait(bdev->bd_inode->i_mapping);
+ mutex_lock(&root->fs_info->volume_mutex);
+
+ devices = &root->fs_info->fs_devices->devices;
+ list_for_each(cur, devices) {
+ device = list_entry(cur, struct btrfs_device, dev_list);
+ if (device->bdev == bdev) {
+ ret = -EEXIST;
+ goto error;
+ }
+ }
+
+ device = kzalloc(sizeof(*device), GFP_NOFS);
+ if (!device) {
+ /* we can safely leave the fs_devices entry around */
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ device->name = kstrdup(device_path, GFP_NOFS);
+ if (!device->name) {
+ kfree(device);
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ ret = find_next_devid(root, &device->devid);
+ if (ret) {
+ kfree(device);
+ goto error;
+ }
+
+ trans = btrfs_start_transaction(root, 1);
+ lock_chunks(root);
+
+ device->barriers = 1;
+ device->writeable = 1;
+ device->work.func = pending_bios_fn;
+ generate_random_uuid(device->uuid);
+ spin_lock_init(&device->io_lock);
+ device->generation = trans->transid;
+ device->io_width = root->sectorsize;
+ device->io_align = root->sectorsize;
+ device->sector_size = root->sectorsize;
+ device->total_bytes = i_size_read(bdev->bd_inode);
+ device->dev_root = root->fs_info->dev_root;
+ device->bdev = bdev;
+ device->in_fs_metadata = 1;
+ device->mode = 0;
+ set_blocksize(device->bdev, 4096);
+
+ if (seeding_dev) {
+ sb->s_flags &= ~MS_RDONLY;
+ ret = btrfs_prepare_sprout(trans, root);
+ BUG_ON(ret);
+ }
+
+ device->fs_devices = root->fs_info->fs_devices;
+ list_add(&device->dev_list, &root->fs_info->fs_devices->devices);
+ list_add(&device->dev_alloc_list,
+ &root->fs_info->fs_devices->alloc_list);
+ root->fs_info->fs_devices->num_devices++;
+ root->fs_info->fs_devices->open_devices++;
+ root->fs_info->fs_devices->rw_devices++;
+ root->fs_info->fs_devices->total_rw_bytes += device->total_bytes;
+
+ total_bytes = btrfs_super_total_bytes(&root->fs_info->super_copy);
+ btrfs_set_super_total_bytes(&root->fs_info->super_copy,
+ total_bytes + device->total_bytes);
+
+ total_bytes = btrfs_super_num_devices(&root->fs_info->super_copy);
+ btrfs_set_super_num_devices(&root->fs_info->super_copy,
+ total_bytes + 1);
+
+ if (seeding_dev) {
+ ret = init_first_rw_device(trans, root, device);
+ BUG_ON(ret);
+ ret = btrfs_finish_sprout(trans, root);
+ BUG_ON(ret);
+ } else {
+ ret = btrfs_add_device(trans, root, device);
+ }
+
+ unlock_chunks(root);
+ btrfs_commit_transaction(trans, root);
+
+ if (seeding_dev) {
+ mutex_unlock(&uuid_mutex);
+ up_write(&sb->s_umount);
+
+ ret = btrfs_relocate_sys_chunks(root);
+ BUG_ON(ret);
+ }
+out:
+ mutex_unlock(&root->fs_info->volume_mutex);
+ return ret;
+error:
+ close_bdev_exclusive(bdev, 0);
+ if (seeding_dev) {
+ mutex_unlock(&uuid_mutex);
+ up_write(&sb->s_umount);
+ }
+ goto out;
+}
+
+static noinline int btrfs_update_device(struct btrfs_trans_handle *trans,
+ struct btrfs_device *device)
+{
+ int ret;
+ struct btrfs_path *path;
+ struct btrfs_root *root;
+ struct btrfs_dev_item *dev_item;
+ struct extent_buffer *leaf;
+ struct btrfs_key key;
+
+ root = device->dev_root->fs_info->chunk_root;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ key.objectid = BTRFS_DEV_ITEMS_OBJECTID;
+ key.type = BTRFS_DEV_ITEM_KEY;
+ key.offset = device->devid;
+
+ ret = btrfs_search_slot(trans, root, &key, path, 0, 1);
+ if (ret < 0)
+ goto out;
+
+ if (ret > 0) {
+ ret = -ENOENT;
+ goto out;
+ }
+
+ leaf = path->nodes[0];
+ dev_item = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_dev_item);
+
+ btrfs_set_device_id(leaf, dev_item, device->devid);
+ btrfs_set_device_type(leaf, dev_item, device->type);
+ btrfs_set_device_io_align(leaf, dev_item, device->io_align);
+ btrfs_set_device_io_width(leaf, dev_item, device->io_width);
+ btrfs_set_device_sector_size(leaf, dev_item, device->sector_size);
+ btrfs_set_device_total_bytes(leaf, dev_item, device->total_bytes);
+ btrfs_set_device_bytes_used(leaf, dev_item, device->bytes_used);
+ btrfs_mark_buffer_dirty(leaf);
+
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
+static int __btrfs_grow_device(struct btrfs_trans_handle *trans,
+ struct btrfs_device *device, u64 new_size)
+{
+ struct btrfs_super_block *super_copy =
+ &device->dev_root->fs_info->super_copy;
+ u64 old_total = btrfs_super_total_bytes(super_copy);
+ u64 diff = new_size - device->total_bytes;
+
+ if (!device->writeable)
+ return -EACCES;
+ if (new_size <= device->total_bytes)
+ return -EINVAL;
+
+ btrfs_set_super_total_bytes(super_copy, old_total + diff);
+ device->fs_devices->total_rw_bytes += diff;
+
+ device->total_bytes = new_size;
+ return btrfs_update_device(trans, device);
+}
+
+int btrfs_grow_device(struct btrfs_trans_handle *trans,
+ struct btrfs_device *device, u64 new_size)
+{
+ int ret;
+ lock_chunks(device->dev_root);
+ ret = __btrfs_grow_device(trans, device, new_size);
+ unlock_chunks(device->dev_root);
+ return ret;
+}
+
+static int btrfs_free_chunk(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 chunk_tree, u64 chunk_objectid,
+ u64 chunk_offset)
+{
+ int ret;
+ struct btrfs_path *path;
+ struct btrfs_key key;
+
+ root = root->fs_info->chunk_root;
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ key.objectid = chunk_objectid;
+ key.offset = chunk_offset;
+ key.type = BTRFS_CHUNK_ITEM_KEY;
+
+ ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
+ BUG_ON(ret);
+
+ ret = btrfs_del_item(trans, root, path);
+ BUG_ON(ret);
+
+ btrfs_free_path(path);
+ return 0;
+}
+
+static int btrfs_del_sys_chunk(struct btrfs_root *root, u64 chunk_objectid, u64
+ chunk_offset)
+{
+ struct btrfs_super_block *super_copy = &root->fs_info->super_copy;
+ struct btrfs_disk_key *disk_key;
+ struct btrfs_chunk *chunk;
+ u8 *ptr;
+ int ret = 0;
+ u32 num_stripes;
+ u32 array_size;
+ u32 len = 0;
+ u32 cur;
+ struct btrfs_key key;
+
+ array_size = btrfs_super_sys_array_size(super_copy);
+
+ ptr = super_copy->sys_chunk_array;
+ cur = 0;
+
+ while (cur < array_size) {
+ disk_key = (struct btrfs_disk_key *)ptr;
+ btrfs_disk_key_to_cpu(&key, disk_key);
+
+ len = sizeof(*disk_key);
+
+ if (key.type == BTRFS_CHUNK_ITEM_KEY) {
+ chunk = (struct btrfs_chunk *)(ptr + len);
+ num_stripes = btrfs_stack_chunk_num_stripes(chunk);
+ len += btrfs_chunk_item_size(num_stripes);
+ } else {
+ ret = -EIO;
+ break;
+ }
+ if (key.objectid == chunk_objectid &&
+ key.offset == chunk_offset) {
+ memmove(ptr, ptr + len, array_size - (cur + len));
+ array_size -= len;
+ btrfs_set_super_sys_array_size(super_copy, array_size);
+ } else {
+ ptr += len;
+ cur += len;
+ }
+ }
+ return ret;
+}
+
+static int btrfs_relocate_chunk(struct btrfs_root *root,
+ u64 chunk_tree, u64 chunk_objectid,
+ u64 chunk_offset)
+{
+ struct extent_map_tree *em_tree;
+ struct btrfs_root *extent_root;
+ struct btrfs_trans_handle *trans;
+ struct extent_map *em;
+ struct map_lookup *map;
+ int ret;
+ int i;
+
+ printk(KERN_INFO "btrfs relocating chunk %llu\n",
+ (unsigned long long)chunk_offset);
+ root = root->fs_info->chunk_root;
+ extent_root = root->fs_info->extent_root;
+ em_tree = &root->fs_info->mapping_tree.map_tree;
+
+ /* step one, relocate all the extents inside this chunk */
+ ret = btrfs_relocate_block_group(extent_root, chunk_offset);
+ BUG_ON(ret);
+
+ trans = btrfs_start_transaction(root, 1);
+ BUG_ON(!trans);
+
+ lock_chunks(root);
+
+ /*
+ * step two, delete the device extents and the
+ * chunk tree entries
+ */
+ spin_lock(&em_tree->lock);
+ em = lookup_extent_mapping(em_tree, chunk_offset, 1);
+ spin_unlock(&em_tree->lock);
+
+ BUG_ON(em->start > chunk_offset ||
+ em->start + em->len < chunk_offset);
+ map = (struct map_lookup *)em->bdev;
+
+ for (i = 0; i < map->num_stripes; i++) {
+ ret = btrfs_free_dev_extent(trans, map->stripes[i].dev,
+ map->stripes[i].physical);
+ BUG_ON(ret);
+
+ if (map->stripes[i].dev) {
+ ret = btrfs_update_device(trans, map->stripes[i].dev);
+ BUG_ON(ret);
+ }
+ }
+ ret = btrfs_free_chunk(trans, root, chunk_tree, chunk_objectid,
+ chunk_offset);
+
+ BUG_ON(ret);
+
+ if (map->type & BTRFS_BLOCK_GROUP_SYSTEM) {
+ ret = btrfs_del_sys_chunk(root, chunk_objectid, chunk_offset);
+ BUG_ON(ret);
+ }
+
+ ret = btrfs_remove_block_group(trans, extent_root, chunk_offset);
+ BUG_ON(ret);
+
+ spin_lock(&em_tree->lock);
+ remove_extent_mapping(em_tree, em);
+ spin_unlock(&em_tree->lock);
+
+ kfree(map);
+ em->bdev = NULL;
+
+ /* once for the tree */
+ free_extent_map(em);
+ /* once for us */
+ free_extent_map(em);
+
+ unlock_chunks(root);
+ btrfs_end_transaction(trans, root);
+ return 0;
+}
+
+static int btrfs_relocate_sys_chunks(struct btrfs_root *root)
+{
+ struct btrfs_root *chunk_root = root->fs_info->chunk_root;
+ struct btrfs_path *path;
+ struct extent_buffer *leaf;
+ struct btrfs_chunk *chunk;
+ struct btrfs_key key;
+ struct btrfs_key found_key;
+ u64 chunk_tree = chunk_root->root_key.objectid;
+ u64 chunk_type;
+ int ret;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ key.objectid = BTRFS_FIRST_CHUNK_TREE_OBJECTID;
+ key.offset = (u64)-1;
+ key.type = BTRFS_CHUNK_ITEM_KEY;
+
+ while (1) {
+ ret = btrfs_search_slot(NULL, chunk_root, &key, path, 0, 0);
+ if (ret < 0)
+ goto error;
+ BUG_ON(ret == 0);
+
+ ret = btrfs_previous_item(chunk_root, path, key.objectid,
+ key.type);
+ if (ret < 0)
+ goto error;
+ if (ret > 0)
+ break;
+
+ leaf = path->nodes[0];
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+
+ chunk = btrfs_item_ptr(leaf, path->slots[0],
+ struct btrfs_chunk);
+ chunk_type = btrfs_chunk_type(leaf, chunk);
+ btrfs_release_path(chunk_root, path);
+
+ if (chunk_type & BTRFS_BLOCK_GROUP_SYSTEM) {
+ ret = btrfs_relocate_chunk(chunk_root, chunk_tree,
+ found_key.objectid,
+ found_key.offset);
+ BUG_ON(ret);
+ }
+
+ if (found_key.offset == 0)
+ break;
+ key.offset = found_key.offset - 1;
+ }
+ ret = 0;
+error:
+ btrfs_free_path(path);
+ return ret;
+}
+
+static u64 div_factor(u64 num, int factor)
+{
+ if (factor == 10)
+ return num;
+ num *= factor;
+ do_div(num, 10);
+ return num;
+}
+
+int btrfs_balance(struct btrfs_root *dev_root)
+{
+ int ret;
+ struct list_head *cur;
+ struct list_head *devices = &dev_root->fs_info->fs_devices->devices;
+ struct btrfs_device *device;
+ u64 old_size;
+ u64 size_to_free;
+ struct btrfs_path *path;
+ struct btrfs_key key;
+ struct btrfs_chunk *chunk;
+ struct btrfs_root *chunk_root = dev_root->fs_info->chunk_root;
+ struct btrfs_trans_handle *trans;
+ struct btrfs_key found_key;
+
+ if (dev_root->fs_info->sb->s_flags & MS_RDONLY)
+ return -EROFS;
+
+ mutex_lock(&dev_root->fs_info->volume_mutex);
+ dev_root = dev_root->fs_info->dev_root;
+
+ /* step one make some room on all the devices */
+ list_for_each(cur, devices) {
+ device = list_entry(cur, struct btrfs_device, dev_list);
+ old_size = device->total_bytes;
+ size_to_free = div_factor(old_size, 1);
+ size_to_free = min(size_to_free, (u64)1 * 1024 * 1024);
+ if (!device->writeable ||
+ device->total_bytes - device->bytes_used > size_to_free)
+ continue;
+
+ ret = btrfs_shrink_device(device, old_size - size_to_free);
+ BUG_ON(ret);
+
+ trans = btrfs_start_transaction(dev_root, 1);
+ BUG_ON(!trans);
+
+ ret = btrfs_grow_device(trans, device, old_size);
+ BUG_ON(ret);
+
+ btrfs_end_transaction(trans, dev_root);
+ }
+
+ /* step two, relocate all the chunks */
+ path = btrfs_alloc_path();
+ BUG_ON(!path);
+
+ key.objectid = BTRFS_FIRST_CHUNK_TREE_OBJECTID;
+ key.offset = (u64)-1;
+ key.type = BTRFS_CHUNK_ITEM_KEY;
+
+ while (1) {
+ ret = btrfs_search_slot(NULL, chunk_root, &key, path, 0, 0);
+ if (ret < 0)
+ goto error;
+
+ /*
+ * this shouldn't happen, it means the last relocate
+ * failed
+ */
+ if (ret == 0)
+ break;
+
+ ret = btrfs_previous_item(chunk_root, path, 0,
+ BTRFS_CHUNK_ITEM_KEY);
+ if (ret)
+ break;
+
+ btrfs_item_key_to_cpu(path->nodes[0], &found_key,
+ path->slots[0]);
+ if (found_key.objectid != key.objectid)
+ break;
+
+ chunk = btrfs_item_ptr(path->nodes[0],
+ path->slots[0],
+ struct btrfs_chunk);
+ key.offset = found_key.offset;
+ /* chunk zero is special */
+ if (key.offset == 0)
+ break;
+
+ btrfs_release_path(chunk_root, path);
+ ret = btrfs_relocate_chunk(chunk_root,
+ chunk_root->root_key.objectid,
+ found_key.objectid,
+ found_key.offset);
+ BUG_ON(ret);
+ }
+ ret = 0;
+error:
+ btrfs_free_path(path);
+ mutex_unlock(&dev_root->fs_info->volume_mutex);
+ return ret;
+}
+
+/*
+ * shrinking a device means finding all of the device extents past
+ * the new size, and then following the back refs to the chunks.
+ * The chunk relocation code actually frees the device extent
+ */
+int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
+{
+ struct btrfs_trans_handle *trans;
+ struct btrfs_root *root = device->dev_root;
+ struct btrfs_dev_extent *dev_extent = NULL;
+ struct btrfs_path *path;
+ u64 length;
+ u64 chunk_tree;
+ u64 chunk_objectid;
+ u64 chunk_offset;
+ int ret;
+ int slot;
+ struct extent_buffer *l;
+ struct btrfs_key key;
+ struct btrfs_super_block *super_copy = &root->fs_info->super_copy;
+ u64 old_total = btrfs_super_total_bytes(super_copy);
+ u64 diff = device->total_bytes - new_size;
+
+ if (new_size >= device->total_bytes)
+ return -EINVAL;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ trans = btrfs_start_transaction(root, 1);
+ if (!trans) {
+ ret = -ENOMEM;
+ goto done;
+ }
+
+ path->reada = 2;
+
+ lock_chunks(root);
+
+ device->total_bytes = new_size;
+ if (device->writeable)
+ device->fs_devices->total_rw_bytes -= diff;
+ ret = btrfs_update_device(trans, device);
+ if (ret) {
+ unlock_chunks(root);
+ btrfs_end_transaction(trans, root);
+ goto done;
+ }
+ WARN_ON(diff > old_total);
+ btrfs_set_super_total_bytes(super_copy, old_total - diff);
+ unlock_chunks(root);
+ btrfs_end_transaction(trans, root);
+
+ key.objectid = device->devid;
+ key.offset = (u64)-1;
+ key.type = BTRFS_DEV_EXTENT_KEY;
+
+ while (1) {
+ ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ if (ret < 0)
+ goto done;
+
+ ret = btrfs_previous_item(root, path, 0, key.type);
+ if (ret < 0)
+ goto done;
+ if (ret) {
+ ret = 0;
+ goto done;
+ }
+
+ l = path->nodes[0];
+ slot = path->slots[0];
+ btrfs_item_key_to_cpu(l, &key, path->slots[0]);
+
+ if (key.objectid != device->devid)
+ goto done;
+
+ dev_extent = btrfs_item_ptr(l, slot, struct btrfs_dev_extent);
+ length = btrfs_dev_extent_length(l, dev_extent);
+
+ if (key.offset + length <= new_size)
+ goto done;
+
+ chunk_tree = btrfs_dev_extent_chunk_tree(l, dev_extent);
+ chunk_objectid = btrfs_dev_extent_chunk_objectid(l, dev_extent);
+ chunk_offset = btrfs_dev_extent_chunk_offset(l, dev_extent);
+ btrfs_release_path(root, path);
+
+ ret = btrfs_relocate_chunk(root, chunk_tree, chunk_objectid,
+ chunk_offset);
+ if (ret)
+ goto done;
+ }
+
+done:
+ btrfs_free_path(path);
+ return ret;
+}
+
+static int btrfs_add_system_chunk(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_key *key,
+ struct btrfs_chunk *chunk, int item_size)
+{
+ struct btrfs_super_block *super_copy = &root->fs_info->super_copy;
+ struct btrfs_disk_key disk_key;
+ u32 array_size;
+ u8 *ptr;
+
+ array_size = btrfs_super_sys_array_size(super_copy);
+ if (array_size + item_size > BTRFS_SYSTEM_CHUNK_ARRAY_SIZE)
+ return -EFBIG;
+
+ ptr = super_copy->sys_chunk_array + array_size;
+ btrfs_cpu_key_to_disk(&disk_key, key);
+ memcpy(ptr, &disk_key, sizeof(disk_key));
+ ptr += sizeof(disk_key);
+ memcpy(ptr, chunk, item_size);
+ item_size += sizeof(disk_key);
+ btrfs_set_super_sys_array_size(super_copy, array_size + item_size);
+ return 0;
+}
+
+static noinline u64 chunk_bytes_by_type(u64 type, u64 calc_size,
+ int num_stripes, int sub_stripes)
+{
+ if (type & (BTRFS_BLOCK_GROUP_RAID1 | BTRFS_BLOCK_GROUP_DUP))
+ return calc_size;
+ else if (type & BTRFS_BLOCK_GROUP_RAID10)
+ return calc_size * (num_stripes / sub_stripes);
+ else
+ return calc_size * num_stripes;
+}
+
+static int __btrfs_alloc_chunk(struct btrfs_trans_handle *trans,
+ struct btrfs_root *extent_root,
+ struct map_lookup **map_ret,
+ u64 *num_bytes, u64 *stripe_size,
+ u64 start, u64 type)
+{
+ struct btrfs_fs_info *info = extent_root->fs_info;
+ struct btrfs_device *device = NULL;
+ struct btrfs_fs_devices *fs_devices = info->fs_devices;
+ struct list_head *cur;
+ struct map_lookup *map = NULL;
+ struct extent_map_tree *em_tree;
+ struct extent_map *em;
+ struct list_head private_devs;
+ int min_stripe_size = 1 * 1024 * 1024;
+ u64 calc_size = 1024 * 1024 * 1024;
+ u64 max_chunk_size = calc_size;
+ u64 min_free;
+ u64 avail;
+ u64 max_avail = 0;
+ u64 dev_offset;
+ int num_stripes = 1;
+ int min_stripes = 1;
+ int sub_stripes = 0;
+ int looped = 0;
+ int ret;
+ int index;
+ int stripe_len = 64 * 1024;
+
+ if ((type & BTRFS_BLOCK_GROUP_RAID1) &&
+ (type & BTRFS_BLOCK_GROUP_DUP)) {
+ WARN_ON(1);
+ type &= ~BTRFS_BLOCK_GROUP_DUP;
+ }
+ if (list_empty(&fs_devices->alloc_list))
+ return -ENOSPC;
+
+ if (type & (BTRFS_BLOCK_GROUP_RAID0)) {
+ num_stripes = fs_devices->rw_devices;
+ min_stripes = 2;
+ }
+ if (type & (BTRFS_BLOCK_GROUP_DUP)) {
+ num_stripes = 2;
+ min_stripes = 2;
+ }
+ if (type & (BTRFS_BLOCK_GROUP_RAID1)) {
+ num_stripes = min_t(u64, 2, fs_devices->rw_devices);
+ if (num_stripes < 2)
+ return -ENOSPC;
+ min_stripes = 2;
+ }
+ if (type & (BTRFS_BLOCK_GROUP_RAID10)) {
+ num_stripes = fs_devices->rw_devices;
+ if (num_stripes < 4)
+ return -ENOSPC;
+ num_stripes &= ~(u32)1;
+ sub_stripes = 2;
+ min_stripes = 4;
+ }
+
+ if (type & BTRFS_BLOCK_GROUP_DATA) {
+ max_chunk_size = 10 * calc_size;
+ min_stripe_size = 64 * 1024 * 1024;
+ } else if (type & BTRFS_BLOCK_GROUP_METADATA) {
+ max_chunk_size = 4 * calc_size;
+ min_stripe_size = 32 * 1024 * 1024;
+ } else if (type & BTRFS_BLOCK_GROUP_SYSTEM) {
+ calc_size = 8 * 1024 * 1024;
+ max_chunk_size = calc_size * 2;
+ min_stripe_size = 1 * 1024 * 1024;
+ }
+
+ /* we don't want a chunk larger than 10% of writeable space */
+ max_chunk_size = min(div_factor(fs_devices->total_rw_bytes, 1),
+ max_chunk_size);
+
+again:
+ if (!map || map->num_stripes != num_stripes) {
+ kfree(map);
+ map = kmalloc(map_lookup_size(num_stripes), GFP_NOFS);
+ if (!map)
+ return -ENOMEM;
+ map->num_stripes = num_stripes;
+ }
+
+ if (calc_size * num_stripes > max_chunk_size) {
+ calc_size = max_chunk_size;
+ do_div(calc_size, num_stripes);
+ do_div(calc_size, stripe_len);
+ calc_size *= stripe_len;
+ }
+ /* we don't want tiny stripes */
+ calc_size = max_t(u64, min_stripe_size, calc_size);
+
+ do_div(calc_size, stripe_len);
+ calc_size *= stripe_len;
+
+ cur = fs_devices->alloc_list.next;
+ index = 0;
+
+ if (type & BTRFS_BLOCK_GROUP_DUP)
+ min_free = calc_size * 2;
+ else
+ min_free = calc_size;
+
+ /*
+ * we add 1MB because we never use the first 1MB of the device, unless
+ * we've looped, then we are likely allocating the maximum amount of
+ * space left already
+ */
+ if (!looped)
+ min_free += 1024 * 1024;
+
+ INIT_LIST_HEAD(&private_devs);
+ while (index < num_stripes) {
+ device = list_entry(cur, struct btrfs_device, dev_alloc_list);
+ BUG_ON(!device->writeable);
+ if (device->total_bytes > device->bytes_used)
+ avail = device->total_bytes - device->bytes_used;
+ else
+ avail = 0;
+ cur = cur->next;
+
+ if (device->in_fs_metadata && avail >= min_free) {
+ ret = find_free_dev_extent(trans, device,
+ min_free, &dev_offset);
+ if (ret == 0) {
+ list_move_tail(&device->dev_alloc_list,
+ &private_devs);
+ map->stripes[index].dev = device;
+ map->stripes[index].physical = dev_offset;
+ index++;
+ if (type & BTRFS_BLOCK_GROUP_DUP) {
+ map->stripes[index].dev = device;
+ map->stripes[index].physical =
+ dev_offset + calc_size;
+ index++;
+ }
+ }
+ } else if (device->in_fs_metadata && avail > max_avail)
+ max_avail = avail;
+ if (cur == &fs_devices->alloc_list)
+ break;
+ }
+ list_splice(&private_devs, &fs_devices->alloc_list);
+ if (index < num_stripes) {
+ if (index >= min_stripes) {
+ num_stripes = index;
+ if (type & (BTRFS_BLOCK_GROUP_RAID10)) {
+ num_stripes /= sub_stripes;
+ num_stripes *= sub_stripes;
+ }
+ looped = 1;
+ goto again;
+ }
+ if (!looped && max_avail > 0) {
+ looped = 1;
+ calc_size = max_avail;
+ goto again;
+ }
+ kfree(map);
+ return -ENOSPC;
+ }
+ map->sector_size = extent_root->sectorsize;
+ map->stripe_len = stripe_len;
+ map->io_align = stripe_len;
+ map->io_width = stripe_len;
+ map->type = type;
+ map->num_stripes = num_stripes;
+ map->sub_stripes = sub_stripes;
+
+ *map_ret = map;
+ *stripe_size = calc_size;
+ *num_bytes = chunk_bytes_by_type(type, calc_size,
+ num_stripes, sub_stripes);
+
+ em = alloc_extent_map(GFP_NOFS);
+ if (!em) {
+ kfree(map);
+ return -ENOMEM;
+ }
+ em->bdev = (struct block_device *)map;
+ em->start = start;
+ em->len = *num_bytes;
+ em->block_start = 0;
+ em->block_len = em->len;
+
+ em_tree = &extent_root->fs_info->mapping_tree.map_tree;
+ spin_lock(&em_tree->lock);
+ ret = add_extent_mapping(em_tree, em);
+ spin_unlock(&em_tree->lock);
+ BUG_ON(ret);
+ free_extent_map(em);
+
+ ret = btrfs_make_block_group(trans, extent_root, 0, type,
+ BTRFS_FIRST_CHUNK_TREE_OBJECTID,
+ start, *num_bytes);
+ BUG_ON(ret);
+
+ index = 0;
+ while (index < map->num_stripes) {
+ device = map->stripes[index].dev;
+ dev_offset = map->stripes[index].physical;
+
+ ret = btrfs_alloc_dev_extent(trans, device,
+ info->chunk_root->root_key.objectid,
+ BTRFS_FIRST_CHUNK_TREE_OBJECTID,
+ start, dev_offset, calc_size);
+ BUG_ON(ret);
+ index++;
+ }
+
+ return 0;
+}
+
+static int __finish_chunk_alloc(struct btrfs_trans_handle *trans,
+ struct btrfs_root *extent_root,
+ struct map_lookup *map, u64 chunk_offset,
+ u64 chunk_size, u64 stripe_size)
+{
+ u64 dev_offset;
+ struct btrfs_key key;
+ struct btrfs_root *chunk_root = extent_root->fs_info->chunk_root;
+ struct btrfs_device *device;
+ struct btrfs_chunk *chunk;
+ struct btrfs_stripe *stripe;
+ size_t item_size = btrfs_chunk_item_size(map->num_stripes);
+ int index = 0;
+ int ret;
+
+ chunk = kzalloc(item_size, GFP_NOFS);
+ if (!chunk)
+ return -ENOMEM;
+
+ index = 0;
+ while (index < map->num_stripes) {
+ device = map->stripes[index].dev;
+ device->bytes_used += stripe_size;
+ ret = btrfs_update_device(trans, device);
+ BUG_ON(ret);
+ index++;
+ }
+
+ index = 0;
+ stripe = &chunk->stripe;
+ while (index < map->num_stripes) {
+ device = map->stripes[index].dev;
+ dev_offset = map->stripes[index].physical;
+
+ btrfs_set_stack_stripe_devid(stripe, device->devid);
+ btrfs_set_stack_stripe_offset(stripe, dev_offset);
+ memcpy(stripe->dev_uuid, device->uuid, BTRFS_UUID_SIZE);
+ stripe++;
+ index++;
+ }
+
+ btrfs_set_stack_chunk_length(chunk, chunk_size);
+ btrfs_set_stack_chunk_owner(chunk, extent_root->root_key.objectid);
+ btrfs_set_stack_chunk_stripe_len(chunk, map->stripe_len);
+ btrfs_set_stack_chunk_type(chunk, map->type);
+ btrfs_set_stack_chunk_num_stripes(chunk, map->num_stripes);
+ btrfs_set_stack_chunk_io_align(chunk, map->stripe_len);
+ btrfs_set_stack_chunk_io_width(chunk, map->stripe_len);
+ btrfs_set_stack_chunk_sector_size(chunk, extent_root->sectorsize);
+ btrfs_set_stack_chunk_sub_stripes(chunk, map->sub_stripes);
+
+ key.objectid = BTRFS_FIRST_CHUNK_TREE_OBJECTID;
+ key.type = BTRFS_CHUNK_ITEM_KEY;
+ key.offset = chunk_offset;
+
+ ret = btrfs_insert_item(trans, chunk_root, &key, chunk, item_size);
+ BUG_ON(ret);
+
+ if (map->type & BTRFS_BLOCK_GROUP_SYSTEM) {
+ ret = btrfs_add_system_chunk(trans, chunk_root, &key, chunk,
+ item_size);
+ BUG_ON(ret);
+ }
+ kfree(chunk);
+ return 0;
+}
+
+/*
+ * Chunk allocation falls into two parts. The first part does works
+ * that make the new allocated chunk useable, but not do any operation
+ * that modifies the chunk tree. The second part does the works that
+ * require modifying the chunk tree. This division is important for the
+ * bootstrap process of adding storage to a seed btrfs.
+ */
+int btrfs_alloc_chunk(struct btrfs_trans_handle *trans,
+ struct btrfs_root *extent_root, u64 type)
+{
+ u64 chunk_offset;
+ u64 chunk_size;
+ u64 stripe_size;
+ struct map_lookup *map;
+ struct btrfs_root *chunk_root = extent_root->fs_info->chunk_root;
+ int ret;
+
+ ret = find_next_chunk(chunk_root, BTRFS_FIRST_CHUNK_TREE_OBJECTID,
+ &chunk_offset);
+ if (ret)
+ return ret;
+
+ ret = __btrfs_alloc_chunk(trans, extent_root, &map, &chunk_size,
+ &stripe_size, chunk_offset, type);
+ if (ret)
+ return ret;
+
+ ret = __finish_chunk_alloc(trans, extent_root, map, chunk_offset,
+ chunk_size, stripe_size);
+ BUG_ON(ret);
+ return 0;
+}
+
+static noinline int init_first_rw_device(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_device *device)
+{
+ u64 chunk_offset;
+ u64 sys_chunk_offset;
+ u64 chunk_size;
+ u64 sys_chunk_size;
+ u64 stripe_size;
+ u64 sys_stripe_size;
+ u64 alloc_profile;
+ struct map_lookup *map;
+ struct map_lookup *sys_map;
+ struct btrfs_fs_info *fs_info = root->fs_info;
+ struct btrfs_root *extent_root = fs_info->extent_root;
+ int ret;
+
+ ret = find_next_chunk(fs_info->chunk_root,
+ BTRFS_FIRST_CHUNK_TREE_OBJECTID, &chunk_offset);
+ BUG_ON(ret);
+
+ alloc_profile = BTRFS_BLOCK_GROUP_METADATA |
+ (fs_info->metadata_alloc_profile &
+ fs_info->avail_metadata_alloc_bits);
+ alloc_profile = btrfs_reduce_alloc_profile(root, alloc_profile);
+
+ ret = __btrfs_alloc_chunk(trans, extent_root, &map, &chunk_size,
+ &stripe_size, chunk_offset, alloc_profile);
+ BUG_ON(ret);
+
+ sys_chunk_offset = chunk_offset + chunk_size;
+
+ alloc_profile = BTRFS_BLOCK_GROUP_SYSTEM |
+ (fs_info->system_alloc_profile &
+ fs_info->avail_system_alloc_bits);
+ alloc_profile = btrfs_reduce_alloc_profile(root, alloc_profile);
+
+ ret = __btrfs_alloc_chunk(trans, extent_root, &sys_map,
+ &sys_chunk_size, &sys_stripe_size,
+ sys_chunk_offset, alloc_profile);
+ BUG_ON(ret);
+
+ ret = btrfs_add_device(trans, fs_info->chunk_root, device);
+ BUG_ON(ret);
+
+ /*
+ * Modifying chunk tree needs allocating new blocks from both
+ * system block group and metadata block group. So we only can
+ * do operations require modifying the chunk tree after both
+ * block groups were created.
+ */
+ ret = __finish_chunk_alloc(trans, extent_root, map, chunk_offset,
+ chunk_size, stripe_size);
+ BUG_ON(ret);
+
+ ret = __finish_chunk_alloc(trans, extent_root, sys_map,
+ sys_chunk_offset, sys_chunk_size,
+ sys_stripe_size);
+ BUG_ON(ret);
+ return 0;
+}
+
+int btrfs_chunk_readonly(struct btrfs_root *root, u64 chunk_offset)
+{
+ struct extent_map *em;
+ struct map_lookup *map;
+ struct btrfs_mapping_tree *map_tree = &root->fs_info->mapping_tree;
+ int readonly = 0;
+ int i;
+
+ spin_lock(&map_tree->map_tree.lock);
+ em = lookup_extent_mapping(&map_tree->map_tree, chunk_offset, 1);
+ spin_unlock(&map_tree->map_tree.lock);
+ if (!em)
+ return 1;
+
+ map = (struct map_lookup *)em->bdev;
+ for (i = 0; i < map->num_stripes; i++) {
+ if (!map->stripes[i].dev->writeable) {
+ readonly = 1;
+ break;
+ }
+ }
+ free_extent_map(em);
+ return readonly;
+}
+
+void btrfs_mapping_init(struct btrfs_mapping_tree *tree)
+{
+ extent_map_tree_init(&tree->map_tree, GFP_NOFS);
+}
+
+void btrfs_mapping_tree_free(struct btrfs_mapping_tree *tree)
+{
+ struct extent_map *em;
+
+ while (1) {
+ spin_lock(&tree->map_tree.lock);
+ em = lookup_extent_mapping(&tree->map_tree, 0, (u64)-1);
+ if (em)
+ remove_extent_mapping(&tree->map_tree, em);
+ spin_unlock(&tree->map_tree.lock);
+ if (!em)
+ break;
+ kfree(em->bdev);
+ /* once for us */
+ free_extent_map(em);
+ /* once for the tree */
+ free_extent_map(em);
+ }
+}
+
+int btrfs_num_copies(struct btrfs_mapping_tree *map_tree, u64 logical, u64 len)
+{
+ struct extent_map *em;
+ struct map_lookup *map;
+ struct extent_map_tree *em_tree = &map_tree->map_tree;
+ int ret;
+
+ spin_lock(&em_tree->lock);
+ em = lookup_extent_mapping(em_tree, logical, len);
+ spin_unlock(&em_tree->lock);
+ BUG_ON(!em);
+
+ BUG_ON(em->start > logical || em->start + em->len < logical);
+ map = (struct map_lookup *)em->bdev;
+ if (map->type & (BTRFS_BLOCK_GROUP_DUP | BTRFS_BLOCK_GROUP_RAID1))
+ ret = map->num_stripes;
+ else if (map->type & BTRFS_BLOCK_GROUP_RAID10)
+ ret = map->sub_stripes;
+ else
+ ret = 1;
+ free_extent_map(em);
+ return ret;
+}
+
+static int find_live_mirror(struct map_lookup *map, int first, int num,
+ int optimal)
+{
+ int i;
+ if (map->stripes[optimal].dev->bdev)
+ return optimal;
+ for (i = first; i < first + num; i++) {
+ if (map->stripes[i].dev->bdev)
+ return i;
+ }
+ /* we couldn't find one that doesn't fail. Just return something
+ * and the io error handling code will clean up eventually
+ */
+ return optimal;
+}
+
+static int __btrfs_map_block(struct btrfs_mapping_tree *map_tree, int rw,
+ u64 logical, u64 *length,
+ struct btrfs_multi_bio **multi_ret,
+ int mirror_num, struct page *unplug_page)
+{
+ struct extent_map *em;
+ struct map_lookup *map;
+ struct extent_map_tree *em_tree = &map_tree->map_tree;
+ u64 offset;
+ u64 stripe_offset;
+ u64 stripe_nr;
+ int stripes_allocated = 8;
+ int stripes_required = 1;
+ int stripe_index;
+ int i;
+ int num_stripes;
+ int max_errors = 0;
+ struct btrfs_multi_bio *multi = NULL;
+
+ if (multi_ret && !(rw & (1 << BIO_RW)))
+ stripes_allocated = 1;
+again:
+ if (multi_ret) {
+ multi = kzalloc(btrfs_multi_bio_size(stripes_allocated),
+ GFP_NOFS);
+ if (!multi)
+ return -ENOMEM;
+
+ atomic_set(&multi->error, 0);
+ }
+
+ spin_lock(&em_tree->lock);
+ em = lookup_extent_mapping(em_tree, logical, *length);
+ spin_unlock(&em_tree->lock);
+
+ if (!em && unplug_page)
+ return 0;
+
+ if (!em) {
+ printk(KERN_CRIT "unable to find logical %llu len %llu\n",
+ (unsigned long long)logical,
+ (unsigned long long)*length);
+ BUG();
+ }
+
+ BUG_ON(em->start > logical || em->start + em->len < logical);
+ map = (struct map_lookup *)em->bdev;
+ offset = logical - em->start;
+
+ if (mirror_num > map->num_stripes)
+ mirror_num = 0;
+
+ /* if our multi bio struct is too small, back off and try again */
+ if (rw & (1 << BIO_RW)) {
+ if (map->type & (BTRFS_BLOCK_GROUP_RAID1 |
+ BTRFS_BLOCK_GROUP_DUP)) {
+ stripes_required = map->num_stripes;
+ max_errors = 1;
+ } else if (map->type & BTRFS_BLOCK_GROUP_RAID10) {
+ stripes_required = map->sub_stripes;
+ max_errors = 1;
+ }
+ }
+ if (multi_ret && rw == WRITE &&
+ stripes_allocated < stripes_required) {
+ stripes_allocated = map->num_stripes;
+ free_extent_map(em);
+ kfree(multi);
+ goto again;
+ }
+ stripe_nr = offset;
+ /*
+ * stripe_nr counts the total number of stripes we have to stride
+ * to get to this block
+ */
+ do_div(stripe_nr, map->stripe_len);
+
+ stripe_offset = stripe_nr * map->stripe_len;
+ BUG_ON(offset < stripe_offset);
+
+ /* stripe_offset is the offset of this block in its stripe*/
+ stripe_offset = offset - stripe_offset;
+
+ if (map->type & (BTRFS_BLOCK_GROUP_RAID0 | BTRFS_BLOCK_GROUP_RAID1 |
+ BTRFS_BLOCK_GROUP_RAID10 |
+ BTRFS_BLOCK_GROUP_DUP)) {
+ /* we limit the length of each bio to what fits in a stripe */
+ *length = min_t(u64, em->len - offset,
+ map->stripe_len - stripe_offset);
+ } else {
+ *length = em->len - offset;
+ }
+
+ if (!multi_ret && !unplug_page)
+ goto out;
+
+ num_stripes = 1;
+ stripe_index = 0;
+ if (map->type & BTRFS_BLOCK_GROUP_RAID1) {
+ if (unplug_page || (rw & (1 << BIO_RW)))
+ num_stripes = map->num_stripes;
+ else if (mirror_num)
+ stripe_index = mirror_num - 1;
+ else {
+ stripe_index = find_live_mirror(map, 0,
+ map->num_stripes,
+ current->pid % map->num_stripes);
+ }
+
+ } else if (map->type & BTRFS_BLOCK_GROUP_DUP) {
+ if (rw & (1 << BIO_RW))
+ num_stripes = map->num_stripes;
+ else if (mirror_num)
+ stripe_index = mirror_num - 1;
+
+ } else if (map->type & BTRFS_BLOCK_GROUP_RAID10) {
+ int factor = map->num_stripes / map->sub_stripes;
+
+ stripe_index = do_div(stripe_nr, factor);
+ stripe_index *= map->sub_stripes;
+
+ if (unplug_page || (rw & (1 << BIO_RW)))
+ num_stripes = map->sub_stripes;
+ else if (mirror_num)
+ stripe_index += mirror_num - 1;
+ else {
+ stripe_index = find_live_mirror(map, stripe_index,
+ map->sub_stripes, stripe_index +
+ current->pid % map->sub_stripes);
+ }
+ } else {
+ /*
+ * after this do_div call, stripe_nr is the number of stripes
+ * on this device we have to walk to find the data, and
+ * stripe_index is the number of our device in the stripe array
+ */
+ stripe_index = do_div(stripe_nr, map->num_stripes);
+ }
+ BUG_ON(stripe_index >= map->num_stripes);
+
+ for (i = 0; i < num_stripes; i++) {
+ if (unplug_page) {
+ struct btrfs_device *device;
+ struct backing_dev_info *bdi;
+
+ device = map->stripes[stripe_index].dev;
+ if (device->bdev) {
+ bdi = blk_get_backing_dev_info(device->bdev);
+ if (bdi->unplug_io_fn)
+ bdi->unplug_io_fn(bdi, unplug_page);
+ }
+ } else {
+ multi->stripes[i].physical =
+ map->stripes[stripe_index].physical +
+ stripe_offset + stripe_nr * map->stripe_len;
+ multi->stripes[i].dev = map->stripes[stripe_index].dev;
+ }
+ stripe_index++;
+ }
+ if (multi_ret) {
+ *multi_ret = multi;
+ multi->num_stripes = num_stripes;
+ multi->max_errors = max_errors;
+ }
+out:
+ free_extent_map(em);
+ return 0;
+}
+
+int btrfs_map_block(struct btrfs_mapping_tree *map_tree, int rw,
+ u64 logical, u64 *length,
+ struct btrfs_multi_bio **multi_ret, int mirror_num)
+{
+ return __btrfs_map_block(map_tree, rw, logical, length, multi_ret,
+ mirror_num, NULL);
+}
+
+int btrfs_rmap_block(struct btrfs_mapping_tree *map_tree,
+ u64 chunk_start, u64 physical, u64 devid,
+ u64 **logical, int *naddrs, int *stripe_len)
+{
+ struct extent_map_tree *em_tree = &map_tree->map_tree;
+ struct extent_map *em;
+ struct map_lookup *map;
+ u64 *buf;
+ u64 bytenr;
+ u64 length;
+ u64 stripe_nr;
+ int i, j, nr = 0;
+
+ spin_lock(&em_tree->lock);
+ em = lookup_extent_mapping(em_tree, chunk_start, 1);
+ spin_unlock(&em_tree->lock);
+
+ BUG_ON(!em || em->start != chunk_start);
+ map = (struct map_lookup *)em->bdev;
+
+ length = em->len;
+ if (map->type & BTRFS_BLOCK_GROUP_RAID10)
+ do_div(length, map->num_stripes / map->sub_stripes);
+ else if (map->type & BTRFS_BLOCK_GROUP_RAID0)
+ do_div(length, map->num_stripes);
+
+ buf = kzalloc(sizeof(u64) * map->num_stripes, GFP_NOFS);
+ BUG_ON(!buf);
+
+ for (i = 0; i < map->num_stripes; i++) {
+ if (devid && map->stripes[i].dev->devid != devid)
+ continue;
+ if (map->stripes[i].physical > physical ||
+ map->stripes[i].physical + length <= physical)
+ continue;
+
+ stripe_nr = physical - map->stripes[i].physical;
+ do_div(stripe_nr, map->stripe_len);
+
+ if (map->type & BTRFS_BLOCK_GROUP_RAID10) {
+ stripe_nr = stripe_nr * map->num_stripes + i;
+ do_div(stripe_nr, map->sub_stripes);
+ } else if (map->type & BTRFS_BLOCK_GROUP_RAID0) {
+ stripe_nr = stripe_nr * map->num_stripes + i;
+ }
+ bytenr = chunk_start + stripe_nr * map->stripe_len;
+ WARN_ON(nr >= map->num_stripes);
+ for (j = 0; j < nr; j++) {
+ if (buf[j] == bytenr)
+ break;
+ }
+ if (j == nr) {
+ WARN_ON(nr >= map->num_stripes);
+ buf[nr++] = bytenr;
+ }
+ }
+
+ for (i = 0; i > nr; i++) {
+ struct btrfs_multi_bio *multi;
+ struct btrfs_bio_stripe *stripe;
+ int ret;
+
+ length = 1;
+ ret = btrfs_map_block(map_tree, WRITE, buf[i],
+ &length, &multi, 0);
+ BUG_ON(ret);
+
+ stripe = multi->stripes;
+ for (j = 0; j < multi->num_stripes; j++) {
+ if (stripe->physical >= physical &&
+ physical < stripe->physical + length)
+ break;
+ }
+ BUG_ON(j >= multi->num_stripes);
+ kfree(multi);
+ }
+
+ *logical = buf;
+ *naddrs = nr;
+ *stripe_len = map->stripe_len;
+
+ free_extent_map(em);
+ return 0;
+}
+
+int btrfs_unplug_page(struct btrfs_mapping_tree *map_tree,
+ u64 logical, struct page *page)
+{
+ u64 length = PAGE_CACHE_SIZE;
+ return __btrfs_map_block(map_tree, READ, logical, &length,
+ NULL, 0, page);
+}
+
+static void end_bio_multi_stripe(struct bio *bio, int err)
+{
+ struct btrfs_multi_bio *multi = bio->bi_private;
+ int is_orig_bio = 0;
+
+ if (err)
+ atomic_inc(&multi->error);
+
+ if (bio == multi->orig_bio)
+ is_orig_bio = 1;
+
+ if (atomic_dec_and_test(&multi->stripes_pending)) {
+ if (!is_orig_bio) {
+ bio_put(bio);
+ bio = multi->orig_bio;
+ }
+ bio->bi_private = multi->private;
+ bio->bi_end_io = multi->end_io;
+ /* only send an error to the higher layers if it is
+ * beyond the tolerance of the multi-bio
+ */
+ if (atomic_read(&multi->error) > multi->max_errors) {
+ err = -EIO;
+ } else if (err) {
+ /*
+ * this bio is actually up to date, we didn't
+ * go over the max number of errors
+ */
+ set_bit(BIO_UPTODATE, &bio->bi_flags);
+ err = 0;
+ }
+ kfree(multi);
+
+ bio_endio(bio, err);
+ } else if (!is_orig_bio) {
+ bio_put(bio);
+ }
+}
+
+struct async_sched {
+ struct bio *bio;
+ int rw;
+ struct btrfs_fs_info *info;
+ struct btrfs_work work;
+};
+
+/*
+ * see run_scheduled_bios for a description of why bios are collected for
+ * async submit.
+ *
+ * This will add one bio to the pending list for a device and make sure
+ * the work struct is scheduled.
+ */
+static noinline int schedule_bio(struct btrfs_root *root,
+ struct btrfs_device *device,
+ int rw, struct bio *bio)
+{
+ int should_queue = 1;
+
+ /* don't bother with additional async steps for reads, right now */
+ if (!(rw & (1 << BIO_RW))) {
+ bio_get(bio);
+ submit_bio(rw, bio);
+ bio_put(bio);
+ return 0;
+ }
+
+ /*
+ * nr_async_bios allows us to reliably return congestion to the
+ * higher layers. Otherwise, the async bio makes it appear we have
+ * made progress against dirty pages when we've really just put it
+ * on a queue for later
+ */
+ atomic_inc(&root->fs_info->nr_async_bios);
+ WARN_ON(bio->bi_next);
+ bio->bi_next = NULL;
+ bio->bi_rw |= rw;
+
+ spin_lock(&device->io_lock);
+
+ if (device->pending_bio_tail)
+ device->pending_bio_tail->bi_next = bio;
+
+ device->pending_bio_tail = bio;
+ if (!device->pending_bios)
+ device->pending_bios = bio;
+ if (device->running_pending)
+ should_queue = 0;
+
+ spin_unlock(&device->io_lock);
+
+ if (should_queue)
+ btrfs_queue_worker(&root->fs_info->submit_workers,
+ &device->work);
+ return 0;
+}
+
+int btrfs_map_bio(struct btrfs_root *root, int rw, struct bio *bio,
+ int mirror_num, int async_submit)
+{
+ struct btrfs_mapping_tree *map_tree;
+ struct btrfs_device *dev;
+ struct bio *first_bio = bio;
+ u64 logical = (u64)bio->bi_sector << 9;
+ u64 length = 0;
+ u64 map_length;
+ struct btrfs_multi_bio *multi = NULL;
+ int ret;
+ int dev_nr = 0;
+ int total_devs = 1;
+
+ length = bio->bi_size;
+ map_tree = &root->fs_info->mapping_tree;
+ map_length = length;
+
+ ret = btrfs_map_block(map_tree, rw, logical, &map_length, &multi,
+ mirror_num);
+ BUG_ON(ret);
+
+ total_devs = multi->num_stripes;
+ if (map_length < length) {
+ printk(KERN_CRIT "mapping failed logical %llu bio len %llu "
+ "len %llu\n", (unsigned long long)logical,
+ (unsigned long long)length,
+ (unsigned long long)map_length);
+ BUG();
+ }
+ multi->end_io = first_bio->bi_end_io;
+ multi->private = first_bio->bi_private;
+ multi->orig_bio = first_bio;
+ atomic_set(&multi->stripes_pending, multi->num_stripes);
+
+ while (dev_nr < total_devs) {
+ if (total_devs > 1) {
+ if (dev_nr < total_devs - 1) {
+ bio = bio_clone(first_bio, GFP_NOFS);
+ BUG_ON(!bio);
+ } else {
+ bio = first_bio;
+ }
+ bio->bi_private = multi;
+ bio->bi_end_io = end_bio_multi_stripe;
+ }
+ bio->bi_sector = multi->stripes[dev_nr].physical >> 9;
+ dev = multi->stripes[dev_nr].dev;
+ BUG_ON(rw == WRITE && !dev->writeable);
+ if (dev && dev->bdev) {
+ bio->bi_bdev = dev->bdev;
+ if (async_submit)
+ schedule_bio(root, dev, rw, bio);
+ else
+ submit_bio(rw, bio);
+ } else {
+ bio->bi_bdev = root->fs_info->fs_devices->latest_bdev;
+ bio->bi_sector = logical >> 9;
+ bio_endio(bio, -EIO);
+ }
+ dev_nr++;
+ }
+ if (total_devs == 1)
+ kfree(multi);
+ return 0;
+}
+
+struct btrfs_device *btrfs_find_device(struct btrfs_root *root, u64 devid,
+ u8 *uuid, u8 *fsid)
+{
+ struct btrfs_device *device;
+ struct btrfs_fs_devices *cur_devices;
+
+ cur_devices = root->fs_info->fs_devices;
+ while (cur_devices) {
+ if (!fsid ||
+ !memcmp(cur_devices->fsid, fsid, BTRFS_UUID_SIZE)) {
+ device = __find_device(&cur_devices->devices,
+ devid, uuid);
+ if (device)
+ return device;
+ }
+ cur_devices = cur_devices->seed;
+ }
+ return NULL;
+}
+
+static struct btrfs_device *add_missing_dev(struct btrfs_root *root,
+ u64 devid, u8 *dev_uuid)
+{
+ struct btrfs_device *device;
+ struct btrfs_fs_devices *fs_devices = root->fs_info->fs_devices;
+
+ device = kzalloc(sizeof(*device), GFP_NOFS);
+ if (!device)
+ return NULL;
+ list_add(&device->dev_list,
+ &fs_devices->devices);
+ device->barriers = 1;
+ device->dev_root = root->fs_info->dev_root;
+ device->devid = devid;
+ device->work.func = pending_bios_fn;
+ device->fs_devices = fs_devices;
+ fs_devices->num_devices++;
+ spin_lock_init(&device->io_lock);
+ INIT_LIST_HEAD(&device->dev_alloc_list);
+ memcpy(device->uuid, dev_uuid, BTRFS_UUID_SIZE);
+ return device;
+}
+
+static int read_one_chunk(struct btrfs_root *root, struct btrfs_key *key,
+ struct extent_buffer *leaf,
+ struct btrfs_chunk *chunk)
+{
+ struct btrfs_mapping_tree *map_tree = &root->fs_info->mapping_tree;
+ struct map_lookup *map;
+ struct extent_map *em;
+ u64 logical;
+ u64 length;
+ u64 devid;
+ u8 uuid[BTRFS_UUID_SIZE];
+ int num_stripes;
+ int ret;
+ int i;
+
+ logical = key->offset;
+ length = btrfs_chunk_length(leaf, chunk);
+
+ spin_lock(&map_tree->map_tree.lock);
+ em = lookup_extent_mapping(&map_tree->map_tree, logical, 1);
+ spin_unlock(&map_tree->map_tree.lock);
+
+ /* already mapped? */
+ if (em && em->start <= logical && em->start + em->len > logical) {
+ free_extent_map(em);
+ return 0;
+ } else if (em) {
+ free_extent_map(em);
+ }
+
+ map = kzalloc(sizeof(*map), GFP_NOFS);
+ if (!map)
+ return -ENOMEM;
+
+ em = alloc_extent_map(GFP_NOFS);
+ if (!em)
+ return -ENOMEM;
+ num_stripes = btrfs_chunk_num_stripes(leaf, chunk);
+ map = kmalloc(map_lookup_size(num_stripes), GFP_NOFS);
+ if (!map) {
+ free_extent_map(em);
+ return -ENOMEM;
+ }
+
+ em->bdev = (struct block_device *)map;
+ em->start = logical;
+ em->len = length;
+ em->block_start = 0;
+ em->block_len = em->len;
+
+ map->num_stripes = num_stripes;
+ map->io_width = btrfs_chunk_io_width(leaf, chunk);
+ map->io_align = btrfs_chunk_io_align(leaf, chunk);
+ map->sector_size = btrfs_chunk_sector_size(leaf, chunk);
+ map->stripe_len = btrfs_chunk_stripe_len(leaf, chunk);
+ map->type = btrfs_chunk_type(leaf, chunk);
+ map->sub_stripes = btrfs_chunk_sub_stripes(leaf, chunk);
+ for (i = 0; i < num_stripes; i++) {
+ map->stripes[i].physical =
+ btrfs_stripe_offset_nr(leaf, chunk, i);
+ devid = btrfs_stripe_devid_nr(leaf, chunk, i);
+ read_extent_buffer(leaf, uuid, (unsigned long)
+ btrfs_stripe_dev_uuid_nr(chunk, i),
+ BTRFS_UUID_SIZE);
+ map->stripes[i].dev = btrfs_find_device(root, devid, uuid,
+ NULL);
+ if (!map->stripes[i].dev && !btrfs_test_opt(root, DEGRADED)) {
+ kfree(map);
+ free_extent_map(em);
+ return -EIO;
+ }
+ if (!map->stripes[i].dev) {
+ map->stripes[i].dev =
+ add_missing_dev(root, devid, uuid);
+ if (!map->stripes[i].dev) {
+ kfree(map);
+ free_extent_map(em);
+ return -EIO;
+ }
+ }
+ map->stripes[i].dev->in_fs_metadata = 1;
+ }
+
+ spin_lock(&map_tree->map_tree.lock);
+ ret = add_extent_mapping(&map_tree->map_tree, em);
+ spin_unlock(&map_tree->map_tree.lock);
+ BUG_ON(ret);
+ free_extent_map(em);
+
+ return 0;
+}
+
+static int fill_device_from_item(struct extent_buffer *leaf,
+ struct btrfs_dev_item *dev_item,
+ struct btrfs_device *device)
+{
+ unsigned long ptr;
+
+ device->devid = btrfs_device_id(leaf, dev_item);
+ device->total_bytes = btrfs_device_total_bytes(leaf, dev_item);
+ device->bytes_used = btrfs_device_bytes_used(leaf, dev_item);
+ device->type = btrfs_device_type(leaf, dev_item);
+ device->io_align = btrfs_device_io_align(leaf, dev_item);
+ device->io_width = btrfs_device_io_width(leaf, dev_item);
+ device->sector_size = btrfs_device_sector_size(leaf, dev_item);
+
+ ptr = (unsigned long)btrfs_device_uuid(dev_item);
+ read_extent_buffer(leaf, device->uuid, ptr, BTRFS_UUID_SIZE);
+
+ return 0;
+}
+
+static int open_seed_devices(struct btrfs_root *root, u8 *fsid)
+{
+ struct btrfs_fs_devices *fs_devices;
+ int ret;
+
+ mutex_lock(&uuid_mutex);
+
+ fs_devices = root->fs_info->fs_devices->seed;
+ while (fs_devices) {
+ if (!memcmp(fs_devices->fsid, fsid, BTRFS_UUID_SIZE)) {
+ ret = 0;
+ goto out;
+ }
+ fs_devices = fs_devices->seed;
+ }
+
+ fs_devices = find_fsid(fsid);
+ if (!fs_devices) {
+ ret = -ENOENT;
+ goto out;
+ }
+
+ fs_devices = clone_fs_devices(fs_devices);
+ if (IS_ERR(fs_devices)) {
+ ret = PTR_ERR(fs_devices);
+ goto out;
+ }
+
+ ret = __btrfs_open_devices(fs_devices, FMODE_READ,
+ root->fs_info->bdev_holder);
+ if (ret)
+ goto out;
+
+ if (!fs_devices->seeding) {
+ __btrfs_close_devices(fs_devices);
+ free_fs_devices(fs_devices);
+ ret = -EINVAL;
+ goto out;
+ }
+
+ fs_devices->seed = root->fs_info->fs_devices->seed;
+ root->fs_info->fs_devices->seed = fs_devices;
+out:
+ mutex_unlock(&uuid_mutex);
+ return ret;
+}
+
+static int read_one_dev(struct btrfs_root *root,
+ struct extent_buffer *leaf,
+ struct btrfs_dev_item *dev_item)
+{
+ struct btrfs_device *device;
+ u64 devid;
+ int ret;
+ u8 fs_uuid[BTRFS_UUID_SIZE];
+ u8 dev_uuid[BTRFS_UUID_SIZE];
+
+ devid = btrfs_device_id(leaf, dev_item);
+ read_extent_buffer(leaf, dev_uuid,
+ (unsigned long)btrfs_device_uuid(dev_item),
+ BTRFS_UUID_SIZE);
+ read_extent_buffer(leaf, fs_uuid,
+ (unsigned long)btrfs_device_fsid(dev_item),
+ BTRFS_UUID_SIZE);
+
+ if (memcmp(fs_uuid, root->fs_info->fsid, BTRFS_UUID_SIZE)) {
+ ret = open_seed_devices(root, fs_uuid);
+ if (ret && !btrfs_test_opt(root, DEGRADED))
+ return ret;
+ }
+
+ device = btrfs_find_device(root, devid, dev_uuid, fs_uuid);
+ if (!device || !device->bdev) {
+ if (!btrfs_test_opt(root, DEGRADED))
+ return -EIO;
+
+ if (!device) {
+ printk(KERN_WARNING "warning devid %llu missing\n",
+ (unsigned long long)devid);
+ device = add_missing_dev(root, devid, dev_uuid);
+ if (!device)
+ return -ENOMEM;
+ }
+ }
+
+ if (device->fs_devices != root->fs_info->fs_devices) {
+ BUG_ON(device->writeable);
+ if (device->generation !=
+ btrfs_device_generation(leaf, dev_item))
+ return -EINVAL;
+ }
+
+ fill_device_from_item(leaf, dev_item, device);
+ device->dev_root = root->fs_info->dev_root;
+ device->in_fs_metadata = 1;
+ if (device->writeable)
+ device->fs_devices->total_rw_bytes += device->total_bytes;
+ ret = 0;
+ return ret;
+}
+
+int btrfs_read_super_device(struct btrfs_root *root, struct extent_buffer *buf)
+{
+ struct btrfs_dev_item *dev_item;
+
+ dev_item = (struct btrfs_dev_item *)offsetof(struct btrfs_super_block,
+ dev_item);
+ return read_one_dev(root, buf, dev_item);
+}
+
+int btrfs_read_sys_array(struct btrfs_root *root)
+{
+ struct btrfs_super_block *super_copy = &root->fs_info->super_copy;
+ struct extent_buffer *sb;
+ struct btrfs_disk_key *disk_key;
+ struct btrfs_chunk *chunk;
+ u8 *ptr;
+ unsigned long sb_ptr;
+ int ret = 0;
+ u32 num_stripes;
+ u32 array_size;
+ u32 len = 0;
+ u32 cur;
+ struct btrfs_key key;
+
+ sb = btrfs_find_create_tree_block(root, BTRFS_SUPER_INFO_OFFSET,
+ BTRFS_SUPER_INFO_SIZE);
+ if (!sb)
+ return -ENOMEM;
+ btrfs_set_buffer_uptodate(sb);
+ write_extent_buffer(sb, super_copy, 0, BTRFS_SUPER_INFO_SIZE);
+ array_size = btrfs_super_sys_array_size(super_copy);
+
+ ptr = super_copy->sys_chunk_array;
+ sb_ptr = offsetof(struct btrfs_super_block, sys_chunk_array);
+ cur = 0;
+
+ while (cur < array_size) {
+ disk_key = (struct btrfs_disk_key *)ptr;
+ btrfs_disk_key_to_cpu(&key, disk_key);
+
+ len = sizeof(*disk_key); ptr += len;
+ sb_ptr += len;
+ cur += len;
+
+ if (key.type == BTRFS_CHUNK_ITEM_KEY) {
+ chunk = (struct btrfs_chunk *)sb_ptr;
+ ret = read_one_chunk(root, &key, sb, chunk);
+ if (ret)
+ break;
+ num_stripes = btrfs_chunk_num_stripes(sb, chunk);
+ len = btrfs_chunk_item_size(num_stripes);
+ } else {
+ ret = -EIO;
+ break;
+ }
+ ptr += len;
+ sb_ptr += len;
+ cur += len;
+ }
+ free_extent_buffer(sb);
+ return ret;
+}
+
+int btrfs_read_chunk_tree(struct btrfs_root *root)
+{
+ struct btrfs_path *path;
+ struct extent_buffer *leaf;
+ struct btrfs_key key;
+ struct btrfs_key found_key;
+ int ret;
+ int slot;
+
+ root = root->fs_info->chunk_root;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ /* first we search for all of the device items, and then we
+ * read in all of the chunk items. This way we can create chunk
+ * mappings that reference all of the devices that are afound
+ */
+ key.objectid = BTRFS_DEV_ITEMS_OBJECTID;
+ key.offset = 0;
+ key.type = 0;
+again:
+ ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ while (1) {
+ leaf = path->nodes[0];
+ slot = path->slots[0];
+ if (slot >= btrfs_header_nritems(leaf)) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret == 0)
+ continue;
+ if (ret < 0)
+ goto error;
+ break;
+ }
+ btrfs_item_key_to_cpu(leaf, &found_key, slot);
+ if (key.objectid == BTRFS_DEV_ITEMS_OBJECTID) {
+ if (found_key.objectid != BTRFS_DEV_ITEMS_OBJECTID)
+ break;
+ if (found_key.type == BTRFS_DEV_ITEM_KEY) {
+ struct btrfs_dev_item *dev_item;
+ dev_item = btrfs_item_ptr(leaf, slot,
+ struct btrfs_dev_item);
+ ret = read_one_dev(root, leaf, dev_item);
+ if (ret)
+ goto error;
+ }
+ } else if (found_key.type == BTRFS_CHUNK_ITEM_KEY) {
+ struct btrfs_chunk *chunk;
+ chunk = btrfs_item_ptr(leaf, slot, struct btrfs_chunk);
+ ret = read_one_chunk(root, &found_key, leaf, chunk);
+ if (ret)
+ goto error;
+ }
+ path->slots[0]++;
+ }
+ if (key.objectid == BTRFS_DEV_ITEMS_OBJECTID) {
+ key.objectid = 0;
+ btrfs_release_path(root, path);
+ goto again;
+ }
+ ret = 0;
+error:
+ btrfs_free_path(path);
+ return ret;
+}
diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h
new file mode 100644
index 0000000..86c44e9
--- /dev/null
+++ b/fs/btrfs/volumes.h
@@ -0,0 +1,162 @@
+/*
+ * Copyright (C) 2007 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __BTRFS_VOLUMES_
+#define __BTRFS_VOLUMES_
+
+#include <linux/bio.h>
+#include "async-thread.h"
+
+struct buffer_head;
+struct btrfs_device {
+ struct list_head dev_list;
+ struct list_head dev_alloc_list;
+ struct btrfs_fs_devices *fs_devices;
+ struct btrfs_root *dev_root;
+ struct bio *pending_bios;
+ struct bio *pending_bio_tail;
+ int running_pending;
+ u64 generation;
+
+ int barriers;
+ int writeable;
+ int in_fs_metadata;
+
+ spinlock_t io_lock;
+
+ struct block_device *bdev;
+
+ /* the mode sent to open_bdev_exclusive */
+ fmode_t mode;
+
+ char *name;
+
+ /* the internal btrfs device id */
+ u64 devid;
+
+ /* size of the device */
+ u64 total_bytes;
+
+ /* bytes used */
+ u64 bytes_used;
+
+ /* optimal io alignment for this device */
+ u32 io_align;
+
+ /* optimal io width for this device */
+ u32 io_width;
+
+ /* minimal io size for this device */
+ u32 sector_size;
+
+ /* type and info about this device */
+ u64 type;
+
+ /* physical drive uuid (or lvm uuid) */
+ u8 uuid[BTRFS_UUID_SIZE];
+
+ struct btrfs_work work;
+};
+
+struct btrfs_fs_devices {
+ u8 fsid[BTRFS_FSID_SIZE]; /* FS specific uuid */
+
+ /* the device with this id has the most recent coyp of the super */
+ u64 latest_devid;
+ u64 latest_trans;
+ u64 num_devices;
+ u64 open_devices;
+ u64 rw_devices;
+ u64 total_rw_bytes;
+ struct block_device *latest_bdev;
+ /* all of the devices in the FS */
+ struct list_head devices;
+
+ /* devices not currently being allocated */
+ struct list_head alloc_list;
+ struct list_head list;
+
+ struct btrfs_fs_devices *seed;
+ int seeding;
+
+ int opened;
+};
+
+struct btrfs_bio_stripe {
+ struct btrfs_device *dev;
+ u64 physical;
+};
+
+struct btrfs_multi_bio {
+ atomic_t stripes_pending;
+ bio_end_io_t *end_io;
+ struct bio *orig_bio;
+ void *private;
+ atomic_t error;
+ int max_errors;
+ int num_stripes;
+ struct btrfs_bio_stripe stripes[];
+};
+
+#define btrfs_multi_bio_size(n) (sizeof(struct btrfs_multi_bio) + \
+ (sizeof(struct btrfs_bio_stripe) * (n)))
+
+int btrfs_alloc_dev_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_device *device,
+ u64 chunk_tree, u64 chunk_objectid,
+ u64 chunk_offset, u64 start, u64 num_bytes);
+int btrfs_map_block(struct btrfs_mapping_tree *map_tree, int rw,
+ u64 logical, u64 *length,
+ struct btrfs_multi_bio **multi_ret, int mirror_num);
+int btrfs_rmap_block(struct btrfs_mapping_tree *map_tree,
+ u64 chunk_start, u64 physical, u64 devid,
+ u64 **logical, int *naddrs, int *stripe_len);
+int btrfs_read_sys_array(struct btrfs_root *root);
+int btrfs_read_chunk_tree(struct btrfs_root *root);
+int btrfs_alloc_chunk(struct btrfs_trans_handle *trans,
+ struct btrfs_root *extent_root, u64 type);
+void btrfs_mapping_init(struct btrfs_mapping_tree *tree);
+void btrfs_mapping_tree_free(struct btrfs_mapping_tree *tree);
+int btrfs_map_bio(struct btrfs_root *root, int rw, struct bio *bio,
+ int mirror_num, int async_submit);
+int btrfs_read_super_device(struct btrfs_root *root, struct extent_buffer *buf);
+int btrfs_open_devices(struct btrfs_fs_devices *fs_devices,
+ fmode_t flags, void *holder);
+int btrfs_scan_one_device(const char *path, fmode_t flags, void *holder,
+ struct btrfs_fs_devices **fs_devices_ret);
+int btrfs_close_devices(struct btrfs_fs_devices *fs_devices);
+int btrfs_close_extra_devices(struct btrfs_fs_devices *fs_devices);
+int btrfs_add_device(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct btrfs_device *device);
+int btrfs_rm_device(struct btrfs_root *root, char *device_path);
+int btrfs_cleanup_fs_uuids(void);
+int btrfs_num_copies(struct btrfs_mapping_tree *map_tree, u64 logical, u64 len);
+int btrfs_unplug_page(struct btrfs_mapping_tree *map_tree,
+ u64 logical, struct page *page);
+int btrfs_grow_device(struct btrfs_trans_handle *trans,
+ struct btrfs_device *device, u64 new_size);
+struct btrfs_device *btrfs_find_device(struct btrfs_root *root, u64 devid,
+ u8 *uuid, u8 *fsid);
+int btrfs_shrink_device(struct btrfs_device *device, u64 new_size);
+int btrfs_init_new_device(struct btrfs_root *root, char *path);
+int btrfs_balance(struct btrfs_root *dev_root);
+void btrfs_unlock_volumes(void);
+void btrfs_lock_volumes(void);
+int btrfs_chunk_readonly(struct btrfs_root *root, u64 chunk_offset);
+#endif
diff --git a/fs/btrfs/xattr.c b/fs/btrfs/xattr.c
new file mode 100644
index 0000000..7f332e2
--- /dev/null
+++ b/fs/btrfs/xattr.c
@@ -0,0 +1,322 @@
+/*
+ * Copyright (C) 2007 Red Hat. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <linux/init.h>
+#include <linux/fs.h>
+#include <linux/slab.h>
+#include <linux/rwsem.h>
+#include <linux/xattr.h>
+#include "ctree.h"
+#include "btrfs_inode.h"
+#include "transaction.h"
+#include "xattr.h"
+#include "disk-io.h"
+
+
+ssize_t __btrfs_getxattr(struct inode *inode, const char *name,
+ void *buffer, size_t size)
+{
+ struct btrfs_dir_item *di;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_path *path;
+ struct extent_buffer *leaf;
+ int ret = 0;
+ unsigned long data_ptr;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ /* lookup the xattr by name */
+ di = btrfs_lookup_xattr(NULL, root, path, inode->i_ino, name,
+ strlen(name), 0);
+ if (!di || IS_ERR(di)) {
+ ret = -ENODATA;
+ goto out;
+ }
+
+ leaf = path->nodes[0];
+ /* if size is 0, that means we want the size of the attr */
+ if (!size) {
+ ret = btrfs_dir_data_len(leaf, di);
+ goto out;
+ }
+
+ /* now get the data out of our dir_item */
+ if (btrfs_dir_data_len(leaf, di) > size) {
+ ret = -ERANGE;
+ goto out;
+ }
+ data_ptr = (unsigned long)((char *)(di + 1) +
+ btrfs_dir_name_len(leaf, di));
+ read_extent_buffer(leaf, buffer, data_ptr,
+ btrfs_dir_data_len(leaf, di));
+ ret = btrfs_dir_data_len(leaf, di);
+
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
+int __btrfs_setxattr(struct inode *inode, const char *name,
+ const void *value, size_t size, int flags)
+{
+ struct btrfs_dir_item *di;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_trans_handle *trans;
+ struct btrfs_path *path;
+ int ret = 0, mod = 0;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ trans = btrfs_start_transaction(root, 1);
+ btrfs_set_trans_block_group(trans, inode);
+
+ /* first lets see if we already have this xattr */
+ di = btrfs_lookup_xattr(trans, root, path, inode->i_ino, name,
+ strlen(name), -1);
+ if (IS_ERR(di)) {
+ ret = PTR_ERR(di);
+ goto out;
+ }
+
+ /* ok we already have this xattr, lets remove it */
+ if (di) {
+ /* if we want create only exit */
+ if (flags & XATTR_CREATE) {
+ ret = -EEXIST;
+ goto out;
+ }
+
+ ret = btrfs_delete_one_dir_name(trans, root, path, di);
+ if (ret)
+ goto out;
+ btrfs_release_path(root, path);
+
+ /* if we don't have a value then we are removing the xattr */
+ if (!value) {
+ mod = 1;
+ goto out;
+ }
+ } else {
+ btrfs_release_path(root, path);
+
+ if (flags & XATTR_REPLACE) {
+ /* we couldn't find the attr to replace */
+ ret = -ENODATA;
+ goto out;
+ }
+ }
+
+ /* ok we have to create a completely new xattr */
+ ret = btrfs_insert_xattr_item(trans, root, name, strlen(name),
+ value, size, inode->i_ino);
+ if (ret)
+ goto out;
+ mod = 1;
+
+out:
+ if (mod) {
+ inode->i_ctime = CURRENT_TIME;
+ ret = btrfs_update_inode(trans, root, inode);
+ }
+
+ btrfs_end_transaction(trans, root);
+ btrfs_free_path(path);
+ return ret;
+}
+
+ssize_t btrfs_listxattr(struct dentry *dentry, char *buffer, size_t size)
+{
+ struct btrfs_key key, found_key;
+ struct inode *inode = dentry->d_inode;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_path *path;
+ struct btrfs_item *item;
+ struct extent_buffer *leaf;
+ struct btrfs_dir_item *di;
+ int ret = 0, slot, advance;
+ size_t total_size = 0, size_left = size;
+ unsigned long name_ptr;
+ size_t name_len;
+ u32 nritems;
+
+ /*
+ * ok we want all objects associated with this id.
+ * NOTE: we set key.offset = 0; because we want to start with the
+ * first xattr that we find and walk forward
+ */
+ key.objectid = inode->i_ino;
+ btrfs_set_key_type(&key, BTRFS_XATTR_ITEM_KEY);
+ key.offset = 0;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+ path->reada = 2;
+
+ /* search for our xattrs */
+ ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ if (ret < 0)
+ goto err;
+ ret = 0;
+ advance = 0;
+ while (1) {
+ leaf = path->nodes[0];
+ nritems = btrfs_header_nritems(leaf);
+ slot = path->slots[0];
+
+ /* this is where we start walking through the path */
+ if (advance || slot >= nritems) {
+ /*
+ * if we've reached the last slot in this leaf we need
+ * to go to the next leaf and reset everything
+ */
+ if (slot >= nritems-1) {
+ ret = btrfs_next_leaf(root, path);
+ if (ret)
+ break;
+ leaf = path->nodes[0];
+ nritems = btrfs_header_nritems(leaf);
+ slot = path->slots[0];
+ } else {
+ /*
+ * just walking through the slots on this leaf
+ */
+ slot++;
+ path->slots[0]++;
+ }
+ }
+ advance = 1;
+
+ item = btrfs_item_nr(leaf, slot);
+ btrfs_item_key_to_cpu(leaf, &found_key, slot);
+
+ /* check to make sure this item is what we want */
+ if (found_key.objectid != key.objectid)
+ break;
+ if (btrfs_key_type(&found_key) != BTRFS_XATTR_ITEM_KEY)
+ break;
+
+ di = btrfs_item_ptr(leaf, slot, struct btrfs_dir_item);
+
+ name_len = btrfs_dir_name_len(leaf, di);
+ total_size += name_len + 1;
+
+ /* we are just looking for how big our buffer needs to be */
+ if (!size)
+ continue;
+
+ if (!buffer || (name_len + 1) > size_left) {
+ ret = -ERANGE;
+ goto err;
+ }
+
+ name_ptr = (unsigned long)(di + 1);
+ read_extent_buffer(leaf, buffer, name_ptr, name_len);
+ buffer[name_len] = '\0';
+
+ size_left -= name_len + 1;
+ buffer += name_len + 1;
+ }
+ ret = total_size;
+
+err:
+ btrfs_free_path(path);
+
+ return ret;
+}
+
+/*
+ * List of handlers for synthetic system.* attributes. All real ondisk
+ * attributes are handled directly.
+ */
+struct xattr_handler *btrfs_xattr_handlers[] = {
+#ifdef CONFIG_FS_POSIX_ACL
+ &btrfs_xattr_acl_access_handler,
+ &btrfs_xattr_acl_default_handler,
+#endif
+ NULL,
+};
+
+/*
+ * Check if the attribute is in a supported namespace.
+ *
+ * This applied after the check for the synthetic attributes in the system
+ * namespace.
+ */
+static bool btrfs_is_valid_xattr(const char *name)
+{
+ return !strncmp(name, XATTR_SECURITY_PREFIX,
+ XATTR_SECURITY_PREFIX_LEN) ||
+ !strncmp(name, XATTR_SYSTEM_PREFIX, XATTR_SYSTEM_PREFIX_LEN) ||
+ !strncmp(name, XATTR_TRUSTED_PREFIX, XATTR_TRUSTED_PREFIX_LEN) ||
+ !strncmp(name, XATTR_USER_PREFIX, XATTR_USER_PREFIX_LEN);
+}
+
+ssize_t btrfs_getxattr(struct dentry *dentry, const char *name,
+ void *buffer, size_t size)
+{
+ /*
+ * If this is a request for a synthetic attribute in the system.*
+ * namespace use the generic infrastructure to resolve a handler
+ * for it via sb->s_xattr.
+ */
+ if (!strncmp(name, XATTR_SYSTEM_PREFIX, XATTR_SYSTEM_PREFIX_LEN))
+ return generic_getxattr(dentry, name, buffer, size);
+
+ if (!btrfs_is_valid_xattr(name))
+ return -EOPNOTSUPP;
+ return __btrfs_getxattr(dentry->d_inode, name, buffer, size);
+}
+
+int btrfs_setxattr(struct dentry *dentry, const char *name, const void *value,
+ size_t size, int flags)
+{
+ /*
+ * If this is a request for a synthetic attribute in the system.*
+ * namespace use the generic infrastructure to resolve a handler
+ * for it via sb->s_xattr.
+ */
+ if (!strncmp(name, XATTR_SYSTEM_PREFIX, XATTR_SYSTEM_PREFIX_LEN))
+ return generic_setxattr(dentry, name, value, size, flags);
+
+ if (!btrfs_is_valid_xattr(name))
+ return -EOPNOTSUPP;
+
+ if (size == 0)
+ value = ""; /* empty EA, do not remove */
+ return __btrfs_setxattr(dentry->d_inode, name, value, size, flags);
+}
+
+int btrfs_removexattr(struct dentry *dentry, const char *name)
+{
+ /*
+ * If this is a request for a synthetic attribute in the system.*
+ * namespace use the generic infrastructure to resolve a handler
+ * for it via sb->s_xattr.
+ */
+ if (!strncmp(name, XATTR_SYSTEM_PREFIX, XATTR_SYSTEM_PREFIX_LEN))
+ return generic_removexattr(dentry, name);
+
+ if (!btrfs_is_valid_xattr(name))
+ return -EOPNOTSUPP;
+ return __btrfs_setxattr(dentry->d_inode, name, NULL, 0, XATTR_REPLACE);
+}
diff --git a/fs/btrfs/xattr.h b/fs/btrfs/xattr.h
new file mode 100644
index 0000000..5b1d08f
--- /dev/null
+++ b/fs/btrfs/xattr.h
@@ -0,0 +1,39 @@
+/*
+ * Copyright (C) 2007 Red Hat. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __XATTR__
+#define __XATTR__
+
+#include <linux/xattr.h>
+
+extern struct xattr_handler btrfs_xattr_acl_access_handler;
+extern struct xattr_handler btrfs_xattr_acl_default_handler;
+extern struct xattr_handler *btrfs_xattr_handlers[];
+
+extern ssize_t __btrfs_getxattr(struct inode *inode, const char *name,
+ void *buffer, size_t size);
+extern int __btrfs_setxattr(struct inode *inode, const char *name,
+ const void *value, size_t size, int flags);
+
+extern ssize_t btrfs_getxattr(struct dentry *dentry, const char *name,
+ void *buffer, size_t size);
+extern int btrfs_setxattr(struct dentry *dentry, const char *name,
+ const void *value, size_t size, int flags);
+extern int btrfs_removexattr(struct dentry *dentry, const char *name);
+
+#endif /* __XATTR__ */
diff --git a/fs/btrfs/zlib.c b/fs/btrfs/zlib.c
new file mode 100644
index 0000000..ecfbce8
--- /dev/null
+++ b/fs/btrfs/zlib.c
@@ -0,0 +1,632 @@
+/*
+ * Copyright (C) 2008 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ *
+ * Based on jffs2 zlib code:
+ * Copyright © 2001-2007 Red Hat, Inc.
+ * Created by David Woodhouse <dwmw2@infradead.org>
+ */
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/zlib.h>
+#include <linux/zutil.h>
+#include <linux/vmalloc.h>
+#include <linux/init.h>
+#include <linux/err.h>
+#include <linux/sched.h>
+#include <linux/pagemap.h>
+#include <linux/bio.h>
+#include "compression.h"
+
+/* Plan: call deflate() with avail_in == *sourcelen,
+ avail_out = *dstlen - 12 and flush == Z_FINISH.
+ If it doesn't manage to finish, call it again with
+ avail_in == 0 and avail_out set to the remaining 12
+ bytes for it to clean up.
+ Q: Is 12 bytes sufficient?
+*/
+#define STREAM_END_SPACE 12
+
+struct workspace {
+ z_stream inf_strm;
+ z_stream def_strm;
+ char *buf;
+ struct list_head list;
+};
+
+static LIST_HEAD(idle_workspace);
+static DEFINE_SPINLOCK(workspace_lock);
+static unsigned long num_workspace;
+static atomic_t alloc_workspace = ATOMIC_INIT(0);
+static DECLARE_WAIT_QUEUE_HEAD(workspace_wait);
+
+/*
+ * this finds an available zlib workspace or allocates a new one
+ * NULL or an ERR_PTR is returned if things go bad.
+ */
+static struct workspace *find_zlib_workspace(void)
+{
+ struct workspace *workspace;
+ int ret;
+ int cpus = num_online_cpus();
+
+again:
+ spin_lock(&workspace_lock);
+ if (!list_empty(&idle_workspace)) {
+ workspace = list_entry(idle_workspace.next, struct workspace,
+ list);
+ list_del(&workspace->list);
+ num_workspace--;
+ spin_unlock(&workspace_lock);
+ return workspace;
+
+ }
+ spin_unlock(&workspace_lock);
+ if (atomic_read(&alloc_workspace) > cpus) {
+ DEFINE_WAIT(wait);
+ prepare_to_wait(&workspace_wait, &wait, TASK_UNINTERRUPTIBLE);
+ if (atomic_read(&alloc_workspace) > cpus)
+ schedule();
+ finish_wait(&workspace_wait, &wait);
+ goto again;
+ }
+ atomic_inc(&alloc_workspace);
+ workspace = kzalloc(sizeof(*workspace), GFP_NOFS);
+ if (!workspace) {
+ ret = -ENOMEM;
+ goto fail;
+ }
+
+ workspace->def_strm.workspace = vmalloc(zlib_deflate_workspacesize());
+ if (!workspace->def_strm.workspace) {
+ ret = -ENOMEM;
+ goto fail;
+ }
+ workspace->inf_strm.workspace = vmalloc(zlib_inflate_workspacesize());
+ if (!workspace->inf_strm.workspace) {
+ ret = -ENOMEM;
+ goto fail_inflate;
+ }
+ workspace->buf = kmalloc(PAGE_CACHE_SIZE, GFP_NOFS);
+ if (!workspace->buf) {
+ ret = -ENOMEM;
+ goto fail_kmalloc;
+ }
+ return workspace;
+
+fail_kmalloc:
+ vfree(workspace->inf_strm.workspace);
+fail_inflate:
+ vfree(workspace->def_strm.workspace);
+fail:
+ kfree(workspace);
+ atomic_dec(&alloc_workspace);
+ wake_up(&workspace_wait);
+ return ERR_PTR(ret);
+}
+
+/*
+ * put a workspace struct back on the list or free it if we have enough
+ * idle ones sitting around
+ */
+static int free_workspace(struct workspace *workspace)
+{
+ spin_lock(&workspace_lock);
+ if (num_workspace < num_online_cpus()) {
+ list_add_tail(&workspace->list, &idle_workspace);
+ num_workspace++;
+ spin_unlock(&workspace_lock);
+ if (waitqueue_active(&workspace_wait))
+ wake_up(&workspace_wait);
+ return 0;
+ }
+ spin_unlock(&workspace_lock);
+ vfree(workspace->def_strm.workspace);
+ vfree(workspace->inf_strm.workspace);
+ kfree(workspace->buf);
+ kfree(workspace);
+
+ atomic_dec(&alloc_workspace);
+ if (waitqueue_active(&workspace_wait))
+ wake_up(&workspace_wait);
+ return 0;
+}
+
+/*
+ * cleanup function for module exit
+ */
+static void free_workspaces(void)
+{
+ struct workspace *workspace;
+ while (!list_empty(&idle_workspace)) {
+ workspace = list_entry(idle_workspace.next, struct workspace,
+ list);
+ list_del(&workspace->list);
+ vfree(workspace->def_strm.workspace);
+ vfree(workspace->inf_strm.workspace);
+ kfree(workspace->buf);
+ kfree(workspace);
+ atomic_dec(&alloc_workspace);
+ }
+}
+
+/*
+ * given an address space and start/len, compress the bytes.
+ *
+ * pages are allocated to hold the compressed result and stored
+ * in 'pages'
+ *
+ * out_pages is used to return the number of pages allocated. There
+ * may be pages allocated even if we return an error
+ *
+ * total_in is used to return the number of bytes actually read. It
+ * may be smaller then len if we had to exit early because we
+ * ran out of room in the pages array or because we cross the
+ * max_out threshold.
+ *
+ * total_out is used to return the total number of compressed bytes
+ *
+ * max_out tells us the max number of bytes that we're allowed to
+ * stuff into pages
+ */
+int btrfs_zlib_compress_pages(struct address_space *mapping,
+ u64 start, unsigned long len,
+ struct page **pages,
+ unsigned long nr_dest_pages,
+ unsigned long *out_pages,
+ unsigned long *total_in,
+ unsigned long *total_out,
+ unsigned long max_out)
+{
+ int ret;
+ struct workspace *workspace;
+ char *data_in;
+ char *cpage_out;
+ int nr_pages = 0;
+ struct page *in_page = NULL;
+ struct page *out_page = NULL;
+ int out_written = 0;
+ int in_read = 0;
+ unsigned long bytes_left;
+
+ *out_pages = 0;
+ *total_out = 0;
+ *total_in = 0;
+
+ workspace = find_zlib_workspace();
+ if (!workspace)
+ return -1;
+
+ if (Z_OK != zlib_deflateInit(&workspace->def_strm, 3)) {
+ printk(KERN_WARNING "deflateInit failed\n");
+ ret = -1;
+ goto out;
+ }
+
+ workspace->def_strm.total_in = 0;
+ workspace->def_strm.total_out = 0;
+
+ in_page = find_get_page(mapping, start >> PAGE_CACHE_SHIFT);
+ data_in = kmap(in_page);
+
+ out_page = alloc_page(GFP_NOFS | __GFP_HIGHMEM);
+ cpage_out = kmap(out_page);
+ pages[0] = out_page;
+ nr_pages = 1;
+
+ workspace->def_strm.next_in = data_in;
+ workspace->def_strm.next_out = cpage_out;
+ workspace->def_strm.avail_out = PAGE_CACHE_SIZE;
+ workspace->def_strm.avail_in = min(len, PAGE_CACHE_SIZE);
+
+ out_written = 0;
+ in_read = 0;
+
+ while (workspace->def_strm.total_in < len) {
+ ret = zlib_deflate(&workspace->def_strm, Z_SYNC_FLUSH);
+ if (ret != Z_OK) {
+ printk(KERN_DEBUG "btrfs deflate in loop returned %d\n",
+ ret);
+ zlib_deflateEnd(&workspace->def_strm);
+ ret = -1;
+ goto out;
+ }
+
+ /* we're making it bigger, give up */
+ if (workspace->def_strm.total_in > 8192 &&
+ workspace->def_strm.total_in <
+ workspace->def_strm.total_out) {
+ ret = -1;
+ goto out;
+ }
+ /* we need another page for writing out. Test this
+ * before the total_in so we will pull in a new page for
+ * the stream end if required
+ */
+ if (workspace->def_strm.avail_out == 0) {
+ kunmap(out_page);
+ if (nr_pages == nr_dest_pages) {
+ out_page = NULL;
+ ret = -1;
+ goto out;
+ }
+ out_page = alloc_page(GFP_NOFS | __GFP_HIGHMEM);
+ cpage_out = kmap(out_page);
+ pages[nr_pages] = out_page;
+ nr_pages++;
+ workspace->def_strm.avail_out = PAGE_CACHE_SIZE;
+ workspace->def_strm.next_out = cpage_out;
+ }
+ /* we're all done */
+ if (workspace->def_strm.total_in >= len)
+ break;
+
+ /* we've read in a full page, get a new one */
+ if (workspace->def_strm.avail_in == 0) {
+ if (workspace->def_strm.total_out > max_out)
+ break;
+
+ bytes_left = len - workspace->def_strm.total_in;
+ kunmap(in_page);
+ page_cache_release(in_page);
+
+ start += PAGE_CACHE_SIZE;
+ in_page = find_get_page(mapping,
+ start >> PAGE_CACHE_SHIFT);
+ data_in = kmap(in_page);
+ workspace->def_strm.avail_in = min(bytes_left,
+ PAGE_CACHE_SIZE);
+ workspace->def_strm.next_in = data_in;
+ }
+ }
+ workspace->def_strm.avail_in = 0;
+ ret = zlib_deflate(&workspace->def_strm, Z_FINISH);
+ zlib_deflateEnd(&workspace->def_strm);
+
+ if (ret != Z_STREAM_END) {
+ ret = -1;
+ goto out;
+ }
+
+ if (workspace->def_strm.total_out >= workspace->def_strm.total_in) {
+ ret = -1;
+ goto out;
+ }
+
+ ret = 0;
+ *total_out = workspace->def_strm.total_out;
+ *total_in = workspace->def_strm.total_in;
+out:
+ *out_pages = nr_pages;
+ if (out_page)
+ kunmap(out_page);
+
+ if (in_page) {
+ kunmap(in_page);
+ page_cache_release(in_page);
+ }
+ free_workspace(workspace);
+ return ret;
+}
+
+/*
+ * pages_in is an array of pages with compressed data.
+ *
+ * disk_start is the starting logical offset of this array in the file
+ *
+ * bvec is a bio_vec of pages from the file that we want to decompress into
+ *
+ * vcnt is the count of pages in the biovec
+ *
+ * srclen is the number of bytes in pages_in
+ *
+ * The basic idea is that we have a bio that was created by readpages.
+ * The pages in the bio are for the uncompressed data, and they may not
+ * be contiguous. They all correspond to the range of bytes covered by
+ * the compressed extent.
+ */
+int btrfs_zlib_decompress_biovec(struct page **pages_in,
+ u64 disk_start,
+ struct bio_vec *bvec,
+ int vcnt,
+ size_t srclen)
+{
+ int ret = 0;
+ int wbits = MAX_WBITS;
+ struct workspace *workspace;
+ char *data_in;
+ size_t total_out = 0;
+ unsigned long page_bytes_left;
+ unsigned long page_in_index = 0;
+ unsigned long page_out_index = 0;
+ struct page *page_out;
+ unsigned long total_pages_in = (srclen + PAGE_CACHE_SIZE - 1) /
+ PAGE_CACHE_SIZE;
+ unsigned long buf_start;
+ unsigned long buf_offset;
+ unsigned long bytes;
+ unsigned long working_bytes;
+ unsigned long pg_offset;
+ unsigned long start_byte;
+ unsigned long current_buf_start;
+ char *kaddr;
+
+ workspace = find_zlib_workspace();
+ if (!workspace)
+ return -ENOMEM;
+
+ data_in = kmap(pages_in[page_in_index]);
+ workspace->inf_strm.next_in = data_in;
+ workspace->inf_strm.avail_in = min_t(size_t, srclen, PAGE_CACHE_SIZE);
+ workspace->inf_strm.total_in = 0;
+
+ workspace->inf_strm.total_out = 0;
+ workspace->inf_strm.next_out = workspace->buf;
+ workspace->inf_strm.avail_out = PAGE_CACHE_SIZE;
+ page_out = bvec[page_out_index].bv_page;
+ page_bytes_left = PAGE_CACHE_SIZE;
+ pg_offset = 0;
+
+ /* If it's deflate, and it's got no preset dictionary, then
+ we can tell zlib to skip the adler32 check. */
+ if (srclen > 2 && !(data_in[1] & PRESET_DICT) &&
+ ((data_in[0] & 0x0f) == Z_DEFLATED) &&
+ !(((data_in[0]<<8) + data_in[1]) % 31)) {
+
+ wbits = -((data_in[0] >> 4) + 8);
+ workspace->inf_strm.next_in += 2;
+ workspace->inf_strm.avail_in -= 2;
+ }
+
+ if (Z_OK != zlib_inflateInit2(&workspace->inf_strm, wbits)) {
+ printk(KERN_WARNING "inflateInit failed\n");
+ ret = -1;
+ goto out;
+ }
+ while (workspace->inf_strm.total_in < srclen) {
+ ret = zlib_inflate(&workspace->inf_strm, Z_NO_FLUSH);
+ if (ret != Z_OK && ret != Z_STREAM_END)
+ break;
+ /*
+ * buf start is the byte offset we're of the start of
+ * our workspace buffer
+ */
+ buf_start = total_out;
+
+ /* total_out is the last byte of the workspace buffer */
+ total_out = workspace->inf_strm.total_out;
+
+ working_bytes = total_out - buf_start;
+
+ /*
+ * start byte is the first byte of the page we're currently
+ * copying into relative to the start of the compressed data.
+ */
+ start_byte = page_offset(page_out) - disk_start;
+
+ if (working_bytes == 0) {
+ /* we didn't make progress in this inflate
+ * call, we're done
+ */
+ if (ret != Z_STREAM_END)
+ ret = -1;
+ break;
+ }
+
+ /* we haven't yet hit data corresponding to this page */
+ if (total_out <= start_byte)
+ goto next;
+
+ /*
+ * the start of the data we care about is offset into
+ * the middle of our working buffer
+ */
+ if (total_out > start_byte && buf_start < start_byte) {
+ buf_offset = start_byte - buf_start;
+ working_bytes -= buf_offset;
+ } else {
+ buf_offset = 0;
+ }
+ current_buf_start = buf_start;
+
+ /* copy bytes from the working buffer into the pages */
+ while (working_bytes > 0) {
+ bytes = min(PAGE_CACHE_SIZE - pg_offset,
+ PAGE_CACHE_SIZE - buf_offset);
+ bytes = min(bytes, working_bytes);
+ kaddr = kmap_atomic(page_out, KM_USER0);
+ memcpy(kaddr + pg_offset, workspace->buf + buf_offset,
+ bytes);
+ kunmap_atomic(kaddr, KM_USER0);
+ flush_dcache_page(page_out);
+
+ pg_offset += bytes;
+ page_bytes_left -= bytes;
+ buf_offset += bytes;
+ working_bytes -= bytes;
+ current_buf_start += bytes;
+
+ /* check if we need to pick another page */
+ if (page_bytes_left == 0) {
+ page_out_index++;
+ if (page_out_index >= vcnt) {
+ ret = 0;
+ goto done;
+ }
+
+ page_out = bvec[page_out_index].bv_page;
+ pg_offset = 0;
+ page_bytes_left = PAGE_CACHE_SIZE;
+ start_byte = page_offset(page_out) - disk_start;
+
+ /*
+ * make sure our new page is covered by this
+ * working buffer
+ */
+ if (total_out <= start_byte)
+ goto next;
+
+ /* the next page in the biovec might not
+ * be adjacent to the last page, but it
+ * might still be found inside this working
+ * buffer. bump our offset pointer
+ */
+ if (total_out > start_byte &&
+ current_buf_start < start_byte) {
+ buf_offset = start_byte - buf_start;
+ working_bytes = total_out - start_byte;
+ current_buf_start = buf_start +
+ buf_offset;
+ }
+ }
+ }
+next:
+ workspace->inf_strm.next_out = workspace->buf;
+ workspace->inf_strm.avail_out = PAGE_CACHE_SIZE;
+
+ if (workspace->inf_strm.avail_in == 0) {
+ unsigned long tmp;
+ kunmap(pages_in[page_in_index]);
+ page_in_index++;
+ if (page_in_index >= total_pages_in) {
+ data_in = NULL;
+ break;
+ }
+ data_in = kmap(pages_in[page_in_index]);
+ workspace->inf_strm.next_in = data_in;
+ tmp = srclen - workspace->inf_strm.total_in;
+ workspace->inf_strm.avail_in = min(tmp,
+ PAGE_CACHE_SIZE);
+ }
+ }
+ if (ret != Z_STREAM_END)
+ ret = -1;
+ else
+ ret = 0;
+done:
+ zlib_inflateEnd(&workspace->inf_strm);
+ if (data_in)
+ kunmap(pages_in[page_in_index]);
+out:
+ free_workspace(workspace);
+ return ret;
+}
+
+/*
+ * a less complex decompression routine. Our compressed data fits in a
+ * single page, and we want to read a single page out of it.
+ * start_byte tells us the offset into the compressed data we're interested in
+ */
+int btrfs_zlib_decompress(unsigned char *data_in,
+ struct page *dest_page,
+ unsigned long start_byte,
+ size_t srclen, size_t destlen)
+{
+ int ret = 0;
+ int wbits = MAX_WBITS;
+ struct workspace *workspace;
+ unsigned long bytes_left = destlen;
+ unsigned long total_out = 0;
+ char *kaddr;
+
+ if (destlen > PAGE_CACHE_SIZE)
+ return -ENOMEM;
+
+ workspace = find_zlib_workspace();
+ if (!workspace)
+ return -ENOMEM;
+
+ workspace->inf_strm.next_in = data_in;
+ workspace->inf_strm.avail_in = srclen;
+ workspace->inf_strm.total_in = 0;
+
+ workspace->inf_strm.next_out = workspace->buf;
+ workspace->inf_strm.avail_out = PAGE_CACHE_SIZE;
+ workspace->inf_strm.total_out = 0;
+ /* If it's deflate, and it's got no preset dictionary, then
+ we can tell zlib to skip the adler32 check. */
+ if (srclen > 2 && !(data_in[1] & PRESET_DICT) &&
+ ((data_in[0] & 0x0f) == Z_DEFLATED) &&
+ !(((data_in[0]<<8) + data_in[1]) % 31)) {
+
+ wbits = -((data_in[0] >> 4) + 8);
+ workspace->inf_strm.next_in += 2;
+ workspace->inf_strm.avail_in -= 2;
+ }
+
+ if (Z_OK != zlib_inflateInit2(&workspace->inf_strm, wbits)) {
+ printk(KERN_WARNING "inflateInit failed\n");
+ ret = -1;
+ goto out;
+ }
+
+ while (bytes_left > 0) {
+ unsigned long buf_start;
+ unsigned long buf_offset;
+ unsigned long bytes;
+ unsigned long pg_offset = 0;
+
+ ret = zlib_inflate(&workspace->inf_strm, Z_NO_FLUSH);
+ if (ret != Z_OK && ret != Z_STREAM_END)
+ break;
+
+ buf_start = total_out;
+ total_out = workspace->inf_strm.total_out;
+
+ if (total_out == buf_start) {
+ ret = -1;
+ break;
+ }
+
+ if (total_out <= start_byte)
+ goto next;
+
+ if (total_out > start_byte && buf_start < start_byte)
+ buf_offset = start_byte - buf_start;
+ else
+ buf_offset = 0;
+
+ bytes = min(PAGE_CACHE_SIZE - pg_offset,
+ PAGE_CACHE_SIZE - buf_offset);
+ bytes = min(bytes, bytes_left);
+
+ kaddr = kmap_atomic(dest_page, KM_USER0);
+ memcpy(kaddr + pg_offset, workspace->buf + buf_offset, bytes);
+ kunmap_atomic(kaddr, KM_USER0);
+
+ pg_offset += bytes;
+ bytes_left -= bytes;
+next:
+ workspace->inf_strm.next_out = workspace->buf;
+ workspace->inf_strm.avail_out = PAGE_CACHE_SIZE;
+ }
+
+ if (ret != Z_STREAM_END && bytes_left != 0)
+ ret = -1;
+ else
+ ret = 0;
+
+ zlib_inflateEnd(&workspace->inf_strm);
+out:
+ free_workspace(workspace);
+ return ret;
+}
+
+void btrfs_zlib_exit(void)
+{
+ free_workspaces();
+}
diff --git a/fs/coda/sysctl.c b/fs/coda/sysctl.c
index 81b7771..43c96ce 100644
--- a/fs/coda/sysctl.c
+++ b/fs/coda/sysctl.c
@@ -11,7 +11,9 @@
#include "coda_int.h"
+#ifdef CONFIG_SYSCTL
static struct ctl_table_header *fs_table_header;
+#endif
static ctl_table coda_table[] = {
{
@@ -41,6 +43,7 @@
{}
};
+#ifdef CONFIG_SYSCTL
static ctl_table fs_table[] = {
{
.ctl_name = CTL_UNNUMBERED,
@@ -50,7 +53,7 @@
},
{}
};
-
+#endif
void coda_sysctl_init(void)
{
diff --git a/fs/dcache.c b/fs/dcache.c
index e88c23b..4547f66 100644
--- a/fs/dcache.c
+++ b/fs/dcache.c
@@ -1567,10 +1567,6 @@
spin_unlock(&dcache_lock);
}
-#define do_switch(x,y) do { \
- __typeof__ (x) __tmp = x; \
- x = y; y = __tmp; } while (0)
-
/*
* When switching names, the actual string doesn't strictly have to
* be preserved in the target - because we're dropping the target
@@ -1589,7 +1585,7 @@
/*
* Both external: swap the pointers
*/
- do_switch(target->d_name.name, dentry->d_name.name);
+ swap(target->d_name.name, dentry->d_name.name);
} else {
/*
* dentry:internal, target:external. Steal target's
@@ -1620,7 +1616,7 @@
return;
}
}
- do_switch(dentry->d_name.len, target->d_name.len);
+ swap(dentry->d_name.len, target->d_name.len);
}
/*
@@ -1680,7 +1676,7 @@
/* Switch the names.. */
switch_names(dentry, target);
- do_switch(dentry->d_name.hash, target->d_name.hash);
+ swap(dentry->d_name.hash, target->d_name.hash);
/* ... and switch the parents */
if (IS_ROOT(dentry)) {
@@ -1688,7 +1684,7 @@
target->d_parent = target;
INIT_LIST_HEAD(&target->d_u.d_child);
} else {
- do_switch(dentry->d_parent, target->d_parent);
+ swap(dentry->d_parent, target->d_parent);
/* And add them back to the (new) parent lists */
list_add(&target->d_u.d_child, &target->d_parent->d_subdirs);
@@ -1789,7 +1785,7 @@
struct dentry *dparent, *aparent;
switch_names(dentry, anon);
- do_switch(dentry->d_name.hash, anon->d_name.hash);
+ swap(dentry->d_name.hash, anon->d_name.hash);
dparent = dentry->d_parent;
aparent = anon->d_parent;
diff --git a/fs/dquot.c b/fs/dquot.c
index 61bfff6..48c0571 100644
--- a/fs/dquot.c
+++ b/fs/dquot.c
@@ -2090,10 +2090,12 @@
}
if (di->dqb_valid & QIF_BTIME) {
dm->dqb_btime = di->dqb_btime;
+ check_blim = 1;
__set_bit(DQ_LASTSET_B + QIF_BTIME_B, &dquot->dq_flags);
}
if (di->dqb_valid & QIF_ITIME) {
dm->dqb_itime = di->dqb_itime;
+ check_ilim = 1;
__set_bit(DQ_LASTSET_B + QIF_ITIME_B, &dquot->dq_flags);
}
diff --git a/fs/ext2/ialloc.c b/fs/ext2/ialloc.c
index c454d5d..66321a8 100644
--- a/fs/ext2/ialloc.c
+++ b/fs/ext2/ialloc.c
@@ -565,12 +565,8 @@
inode->i_blocks = 0;
inode->i_mtime = inode->i_atime = inode->i_ctime = CURRENT_TIME_SEC;
memset(ei->i_data, 0, sizeof(ei->i_data));
- ei->i_flags = EXT2_I(dir)->i_flags & ~EXT2_BTREE_FL;
- if (S_ISLNK(mode))
- ei->i_flags &= ~(EXT2_IMMUTABLE_FL|EXT2_APPEND_FL);
- /* dirsync is only applied to directories */
- if (!S_ISDIR(mode))
- ei->i_flags &= ~EXT2_DIRSYNC_FL;
+ ei->i_flags =
+ ext2_mask_flags(mode, EXT2_I(dir)->i_flags & EXT2_FL_INHERITED);
ei->i_faddr = 0;
ei->i_frag_no = 0;
ei->i_frag_size = 0;
diff --git a/fs/ext2/inode.c b/fs/ext2/inode.c
index 02b39a5..23fff2f 100644
--- a/fs/ext2/inode.c
+++ b/fs/ext2/inode.c
@@ -498,8 +498,6 @@
* ext2_splice_branch - splice the allocated branch onto inode.
* @inode: owner
* @block: (logical) number of block we are adding
- * @chain: chain of indirect blocks (with a missing link - see
- * ext2_alloc_branch)
* @where: location of missing link
* @num: number of indirect blocks we are adding
* @blks: number of direct blocks we are adding
diff --git a/fs/ext2/ioctl.c b/fs/ext2/ioctl.c
index de876fa..7cb4bad 100644
--- a/fs/ext2/ioctl.c
+++ b/fs/ext2/ioctl.c
@@ -50,8 +50,7 @@
goto setflags_out;
}
- if (!S_ISDIR(inode->i_mode))
- flags &= ~EXT2_DIRSYNC_FL;
+ flags = ext2_mask_flags(inode->i_mode, flags);
mutex_lock(&inode->i_mutex);
/* Is it quota file? Do not allow user to mess with it */
diff --git a/fs/ext2/super.c b/fs/ext2/super.c
index 647cd88..da8bdea 100644
--- a/fs/ext2/super.c
+++ b/fs/ext2/super.c
@@ -132,6 +132,7 @@
percpu_counter_destroy(&sbi->s_dirs_counter);
brelse (sbi->s_sbh);
sb->s_fs_info = NULL;
+ kfree(sbi->s_blockgroup_lock);
kfree(sbi);
return;
@@ -756,6 +757,13 @@
sbi = kzalloc(sizeof(*sbi), GFP_KERNEL);
if (!sbi)
return -ENOMEM;
+
+ sbi->s_blockgroup_lock =
+ kzalloc(sizeof(struct blockgroup_lock), GFP_KERNEL);
+ if (!sbi->s_blockgroup_lock) {
+ kfree(sbi);
+ return -ENOMEM;
+ }
sb->s_fs_info = sbi;
sbi->s_sb_block = sb_block;
@@ -983,7 +991,7 @@
printk ("EXT2-fs: not enough memory\n");
goto failed_mount;
}
- bgl_lock_init(&sbi->s_blockgroup_lock);
+ bgl_lock_init(sbi->s_blockgroup_lock);
sbi->s_debts = kcalloc(sbi->s_groups_count, sizeof(*sbi->s_debts), GFP_KERNEL);
if (!sbi->s_debts) {
printk ("EXT2-fs: not enough memory\n");
diff --git a/fs/ext3/hash.c b/fs/ext3/hash.c
index c30e149..7d215b4 100644
--- a/fs/ext3/hash.c
+++ b/fs/ext3/hash.c
@@ -35,23 +35,71 @@
/* The old legacy hash */
-static __u32 dx_hack_hash (const char *name, int len)
+static __u32 dx_hack_hash_unsigned(const char *name, int len)
{
- __u32 hash0 = 0x12a3fe2d, hash1 = 0x37abe8f9;
- while (len--) {
- __u32 hash = hash1 + (hash0 ^ (*name++ * 7152373));
+ __u32 hash, hash0 = 0x12a3fe2d, hash1 = 0x37abe8f9;
+ const unsigned char *ucp = (const unsigned char *) name;
- if (hash & 0x80000000) hash -= 0x7fffffff;
+ while (len--) {
+ hash = hash1 + (hash0 ^ (((int) *ucp++) * 7152373));
+
+ if (hash & 0x80000000)
+ hash -= 0x7fffffff;
hash1 = hash0;
hash0 = hash;
}
- return (hash0 << 1);
+ return hash0 << 1;
}
-static void str2hashbuf(const char *msg, int len, __u32 *buf, int num)
+static __u32 dx_hack_hash_signed(const char *name, int len)
+{
+ __u32 hash, hash0 = 0x12a3fe2d, hash1 = 0x37abe8f9;
+ const signed char *scp = (const signed char *) name;
+
+ while (len--) {
+ hash = hash1 + (hash0 ^ (((int) *scp++) * 7152373));
+
+ if (hash & 0x80000000)
+ hash -= 0x7fffffff;
+ hash1 = hash0;
+ hash0 = hash;
+ }
+ return hash0 << 1;
+}
+
+static void str2hashbuf_signed(const char *msg, int len, __u32 *buf, int num)
{
__u32 pad, val;
int i;
+ const signed char *scp = (const signed char *) msg;
+
+ pad = (__u32)len | ((__u32)len << 8);
+ pad |= pad << 16;
+
+ val = pad;
+ if (len > num*4)
+ len = num * 4;
+ for (i = 0; i < len; i++) {
+ if ((i % 4) == 0)
+ val = pad;
+ val = ((int) scp[i]) + (val << 8);
+ if ((i % 4) == 3) {
+ *buf++ = val;
+ val = pad;
+ num--;
+ }
+ }
+ if (--num >= 0)
+ *buf++ = val;
+ while (--num >= 0)
+ *buf++ = pad;
+}
+
+static void str2hashbuf_unsigned(const char *msg, int len, __u32 *buf, int num)
+{
+ __u32 pad, val;
+ int i;
+ const unsigned char *ucp = (const unsigned char *) msg;
pad = (__u32)len | ((__u32)len << 8);
pad |= pad << 16;
@@ -62,7 +110,7 @@
for (i=0; i < len; i++) {
if ((i % 4) == 0)
val = pad;
- val = msg[i] + (val << 8);
+ val = ((int) ucp[i]) + (val << 8);
if ((i % 4) == 3) {
*buf++ = val;
val = pad;
@@ -95,6 +143,8 @@
const char *p;
int i;
__u32 in[8], buf[4];
+ void (*str2hashbuf)(const char *, int, __u32 *, int) =
+ str2hashbuf_signed;
/* Initialize the default seed for the hash checksum functions */
buf[0] = 0x67452301;
@@ -113,13 +163,18 @@
}
switch (hinfo->hash_version) {
- case DX_HASH_LEGACY:
- hash = dx_hack_hash(name, len);
+ case DX_HASH_LEGACY_UNSIGNED:
+ hash = dx_hack_hash_unsigned(name, len);
break;
+ case DX_HASH_LEGACY:
+ hash = dx_hack_hash_signed(name, len);
+ break;
+ case DX_HASH_HALF_MD4_UNSIGNED:
+ str2hashbuf = str2hashbuf_unsigned;
case DX_HASH_HALF_MD4:
p = name;
while (len > 0) {
- str2hashbuf(p, len, in, 8);
+ (*str2hashbuf)(p, len, in, 8);
half_md4_transform(buf, in);
len -= 32;
p += 32;
@@ -127,10 +182,12 @@
minor_hash = buf[2];
hash = buf[1];
break;
+ case DX_HASH_TEA_UNSIGNED:
+ str2hashbuf = str2hashbuf_unsigned;
case DX_HASH_TEA:
p = name;
while (len > 0) {
- str2hashbuf(p, len, in, 4);
+ (*str2hashbuf)(p, len, in, 4);
TEA_transform(buf, in);
len -= 16;
p += 16;
diff --git a/fs/ext3/ialloc.c b/fs/ext3/ialloc.c
index 5655fbc..8de6c72 100644
--- a/fs/ext3/ialloc.c
+++ b/fs/ext3/ialloc.c
@@ -559,12 +559,8 @@
ei->i_dir_start_lookup = 0;
ei->i_disksize = 0;
- ei->i_flags = EXT3_I(dir)->i_flags & ~EXT3_INDEX_FL;
- if (S_ISLNK(mode))
- ei->i_flags &= ~(EXT3_IMMUTABLE_FL|EXT3_APPEND_FL);
- /* dirsync only applies to directories */
- if (!S_ISDIR(mode))
- ei->i_flags &= ~EXT3_DIRSYNC_FL;
+ ei->i_flags =
+ ext3_mask_flags(mode, EXT3_I(dir)->i_flags & EXT3_FL_INHERITED);
#ifdef EXT3_FRAGMENTS
ei->i_faddr = 0;
ei->i_frag_no = 0;
diff --git a/fs/ext3/ioctl.c b/fs/ext3/ioctl.c
index b7394d0..5e86ce9 100644
--- a/fs/ext3/ioctl.c
+++ b/fs/ext3/ioctl.c
@@ -53,8 +53,7 @@
goto flags_out;
}
- if (!S_ISDIR(inode->i_mode))
- flags &= ~EXT3_DIRSYNC_FL;
+ flags = ext3_mask_flags(inode->i_mode, flags);
mutex_lock(&inode->i_mutex);
/* Is it quota file? Do not allow user to mess with it */
diff --git a/fs/ext3/namei.c b/fs/ext3/namei.c
index 1dd2abe..69a3d19 100644
--- a/fs/ext3/namei.c
+++ b/fs/ext3/namei.c
@@ -74,10 +74,6 @@
#define assert(test) J_ASSERT(test)
#endif
-#ifndef swap
-#define swap(x, y) do { typeof(x) z = x; x = y; y = z; } while (0)
-#endif
-
#ifdef DX_DEBUG
#define dxtrace(command) command
#else
@@ -368,6 +364,8 @@
goto fail;
}
hinfo->hash_version = root->info.hash_version;
+ if (hinfo->hash_version <= DX_HASH_TEA)
+ hinfo->hash_version += EXT3_SB(dir->i_sb)->s_hash_unsigned;
hinfo->seed = EXT3_SB(dir->i_sb)->s_hash_seed;
if (entry)
ext3fs_dirhash(entry->name, entry->len, hinfo);
@@ -636,6 +634,9 @@
dir = dir_file->f_path.dentry->d_inode;
if (!(EXT3_I(dir)->i_flags & EXT3_INDEX_FL)) {
hinfo.hash_version = EXT3_SB(dir->i_sb)->s_def_hash_version;
+ if (hinfo.hash_version <= DX_HASH_TEA)
+ hinfo.hash_version +=
+ EXT3_SB(dir->i_sb)->s_hash_unsigned;
hinfo.seed = EXT3_SB(dir->i_sb)->s_hash_seed;
count = htree_dirblock_to_tree(dir_file, dir, 0, &hinfo,
start_hash, start_minor_hash);
@@ -1156,9 +1157,9 @@
u32 hash2;
struct dx_map_entry *map;
char *data1 = (*bh)->b_data, *data2;
- unsigned split, move, size, i;
+ unsigned split, move, size;
struct ext3_dir_entry_2 *de = NULL, *de2;
- int err = 0;
+ int err = 0, i;
bh2 = ext3_append (handle, dir, &newblock, &err);
if (!(bh2)) {
@@ -1398,6 +1399,8 @@
/* Initialize as for dx_probe */
hinfo.hash_version = root->info.hash_version;
+ if (hinfo.hash_version <= DX_HASH_TEA)
+ hinfo.hash_version += EXT3_SB(dir->i_sb)->s_hash_unsigned;
hinfo.seed = EXT3_SB(dir->i_sb)->s_hash_seed;
ext3fs_dirhash(name, namelen, &hinfo);
frame = frames;
diff --git a/fs/ext3/super.c b/fs/ext3/super.c
index c22d014..5d047a0 100644
--- a/fs/ext3/super.c
+++ b/fs/ext3/super.c
@@ -439,6 +439,7 @@
ext3_blkdev_remove(sbi);
}
sb->s_fs_info = NULL;
+ kfree(sbi->s_blockgroup_lock);
kfree(sbi);
return;
}
@@ -682,6 +683,26 @@
ext3_nfs_get_inode);
}
+/*
+ * Try to release metadata pages (indirect blocks, directories) which are
+ * mapped via the block device. Since these pages could have journal heads
+ * which would prevent try_to_free_buffers() from freeing them, we must use
+ * jbd layer's try_to_free_buffers() function to release them.
+ */
+static int bdev_try_to_free_page(struct super_block *sb, struct page *page,
+ gfp_t wait)
+{
+ journal_t *journal = EXT3_SB(sb)->s_journal;
+
+ WARN_ON(PageChecked(page));
+ if (!page_has_buffers(page))
+ return 0;
+ if (journal)
+ return journal_try_to_free_buffers(journal, page,
+ wait & ~__GFP_WAIT);
+ return try_to_free_buffers(page);
+}
+
#ifdef CONFIG_QUOTA
#define QTYPE2NAME(t) ((t)==USRQUOTA?"user":"group")
#define QTYPE2MOPT(on, t) ((t)==USRQUOTA?((on)##USRJQUOTA):((on)##GRPJQUOTA))
@@ -748,6 +769,7 @@
.quota_read = ext3_quota_read,
.quota_write = ext3_quota_write,
#endif
+ .bdev_try_to_free_page = bdev_try_to_free_page,
};
static const struct export_operations ext3_export_ops = {
@@ -1546,6 +1568,13 @@
sbi = kzalloc(sizeof(*sbi), GFP_KERNEL);
if (!sbi)
return -ENOMEM;
+
+ sbi->s_blockgroup_lock =
+ kzalloc(sizeof(struct blockgroup_lock), GFP_KERNEL);
+ if (!sbi->s_blockgroup_lock) {
+ kfree(sbi);
+ return -ENOMEM;
+ }
sb->s_fs_info = sbi;
sbi->s_mount_opt = 0;
sbi->s_resuid = EXT3_DEF_RESUID;
@@ -1742,6 +1771,18 @@
for (i=0; i < 4; i++)
sbi->s_hash_seed[i] = le32_to_cpu(es->s_hash_seed[i]);
sbi->s_def_hash_version = es->s_def_hash_version;
+ i = le32_to_cpu(es->s_flags);
+ if (i & EXT2_FLAGS_UNSIGNED_HASH)
+ sbi->s_hash_unsigned = 3;
+ else if ((i & EXT2_FLAGS_SIGNED_HASH) == 0) {
+#ifdef __CHAR_UNSIGNED__
+ es->s_flags |= cpu_to_le32(EXT2_FLAGS_UNSIGNED_HASH);
+ sbi->s_hash_unsigned = 3;
+#else
+ es->s_flags |= cpu_to_le32(EXT2_FLAGS_SIGNED_HASH);
+#endif
+ sb->s_dirt = 1;
+ }
if (sbi->s_blocks_per_group > blocksize * 8) {
printk (KERN_ERR
@@ -1786,7 +1827,7 @@
goto failed_mount;
}
- bgl_lock_init(&sbi->s_blockgroup_lock);
+ bgl_lock_init(sbi->s_blockgroup_lock);
for (i = 0; i < db_count; i++) {
block = descriptor_loc(sb, logic_sb_block, i);
diff --git a/fs/ext4/balloc.c b/fs/ext4/balloc.c
index 38b3acf..6bba06b 100644
--- a/fs/ext4/balloc.c
+++ b/fs/ext4/balloc.c
@@ -20,6 +20,7 @@
#include "ext4.h"
#include "ext4_jbd2.h"
#include "group.h"
+#include "mballoc.h"
/*
* balloc.c contains the blocks allocation and deallocation routines
@@ -100,10 +101,10 @@
* essentially implementing a per-group read-only flag. */
if (!ext4_group_desc_csum_verify(sbi, block_group, gdp)) {
ext4_error(sb, __func__,
- "Checksum bad for group %lu\n", block_group);
- gdp->bg_free_blocks_count = 0;
- gdp->bg_free_inodes_count = 0;
- gdp->bg_itable_unused = 0;
+ "Checksum bad for group %u", block_group);
+ ext4_free_blks_set(sb, gdp, 0);
+ ext4_free_inodes_set(sb, gdp, 0);
+ ext4_itable_unused_set(sb, gdp, 0);
memset(bh->b_data, 0xff, sb->s_blocksize);
return 0;
}
@@ -205,15 +206,15 @@
ext4_group_t block_group,
struct buffer_head **bh)
{
- unsigned long group_desc;
- unsigned long offset;
+ unsigned int group_desc;
+ unsigned int offset;
struct ext4_group_desc *desc;
struct ext4_sb_info *sbi = EXT4_SB(sb);
if (block_group >= sbi->s_groups_count) {
ext4_error(sb, "ext4_get_group_desc",
"block_group >= groups_count - "
- "block_group = %lu, groups_count = %lu",
+ "block_group = %u, groups_count = %u",
block_group, sbi->s_groups_count);
return NULL;
@@ -225,7 +226,7 @@
if (!sbi->s_group_desc[group_desc]) {
ext4_error(sb, "ext4_get_group_desc",
"Group descriptor not loaded - "
- "block_group = %lu, group_desc = %lu, desc = %lu",
+ "block_group = %u, group_desc = %u, desc = %u",
block_group, group_desc, offset);
return NULL;
}
@@ -315,29 +316,50 @@
if (unlikely(!bh)) {
ext4_error(sb, __func__,
"Cannot read block bitmap - "
- "block_group = %lu, block_bitmap = %llu",
+ "block_group = %u, block_bitmap = %llu",
block_group, bitmap_blk);
return NULL;
}
- if (buffer_uptodate(bh) &&
- !(desc->bg_flags & cpu_to_le16(EXT4_BG_BLOCK_UNINIT)))
+
+ if (bitmap_uptodate(bh))
return bh;
lock_buffer(bh);
+ if (bitmap_uptodate(bh)) {
+ unlock_buffer(bh);
+ return bh;
+ }
spin_lock(sb_bgl_lock(EXT4_SB(sb), block_group));
if (desc->bg_flags & cpu_to_le16(EXT4_BG_BLOCK_UNINIT)) {
ext4_init_block_bitmap(sb, bh, block_group, desc);
+ set_bitmap_uptodate(bh);
set_buffer_uptodate(bh);
- unlock_buffer(bh);
spin_unlock(sb_bgl_lock(EXT4_SB(sb), block_group));
+ unlock_buffer(bh);
return bh;
}
spin_unlock(sb_bgl_lock(EXT4_SB(sb), block_group));
+ if (buffer_uptodate(bh)) {
+ /*
+ * if not uninit if bh is uptodate,
+ * bitmap is also uptodate
+ */
+ set_bitmap_uptodate(bh);
+ unlock_buffer(bh);
+ return bh;
+ }
+ /*
+ * submit the buffer_head for read. We can
+ * safely mark the bitmap as uptodate now.
+ * We do it here so the bitmap uptodate bit
+ * get set with buffer lock held.
+ */
+ set_bitmap_uptodate(bh);
if (bh_submit_read(bh) < 0) {
put_bh(bh);
ext4_error(sb, __func__,
"Cannot read block bitmap - "
- "block_group = %lu, block_bitmap = %llu",
+ "block_group = %u, block_bitmap = %llu",
block_group, bitmap_blk);
return NULL;
}
@@ -350,62 +372,44 @@
}
/**
- * ext4_free_blocks_sb() -- Free given blocks and update quota
+ * ext4_add_groupblocks() -- Add given blocks to an existing group
* @handle: handle to this transaction
* @sb: super block
- * @block: start physcial block to free
+ * @block: start physcial block to add to the block group
* @count: number of blocks to free
- * @pdquot_freed_blocks: pointer to quota
*
- * XXX This function is only used by the on-line resizing code, which
- * should probably be fixed up to call the mballoc variant. There
- * this needs to be cleaned up later; in fact, I'm not convinced this
- * is 100% correct in the face of the mballoc code. The online resizing
- * code needs to be fixed up to more tightly (and correctly) interlock
- * with the mballoc code.
+ * This marks the blocks as free in the bitmap. We ask the
+ * mballoc to reload the buddy after this by setting group
+ * EXT4_GROUP_INFO_NEED_INIT_BIT flag
*/
-void ext4_free_blocks_sb(handle_t *handle, struct super_block *sb,
- ext4_fsblk_t block, unsigned long count,
- unsigned long *pdquot_freed_blocks)
+void ext4_add_groupblocks(handle_t *handle, struct super_block *sb,
+ ext4_fsblk_t block, unsigned long count)
{
struct buffer_head *bitmap_bh = NULL;
struct buffer_head *gd_bh;
ext4_group_t block_group;
ext4_grpblk_t bit;
- unsigned long i;
- unsigned long overflow;
+ unsigned int i;
struct ext4_group_desc *desc;
struct ext4_super_block *es;
struct ext4_sb_info *sbi;
- int err = 0, ret;
- ext4_grpblk_t group_freed;
+ int err = 0, ret, blk_free_count;
+ ext4_grpblk_t blocks_freed;
+ struct ext4_group_info *grp;
- *pdquot_freed_blocks = 0;
sbi = EXT4_SB(sb);
es = sbi->s_es;
- if (block < le32_to_cpu(es->s_first_data_block) ||
- block + count < block ||
- block + count > ext4_blocks_count(es)) {
- ext4_error(sb, "ext4_free_blocks",
- "Freeing blocks not in datazone - "
- "block = %llu, count = %lu", block, count);
- goto error_return;
- }
+ ext4_debug("Adding block(s) %llu-%llu\n", block, block + count - 1);
- ext4_debug("freeing block(s) %llu-%llu\n", block, block + count - 1);
-
-do_more:
- overflow = 0;
ext4_get_group_no_and_offset(sb, block, &block_group, &bit);
+ grp = ext4_get_group_info(sb, block_group);
/*
* Check to see if we are freeing blocks across a group
* boundary.
*/
if (bit + count > EXT4_BLOCKS_PER_GROUP(sb)) {
- overflow = bit + count - EXT4_BLOCKS_PER_GROUP(sb);
- count -= overflow;
+ goto error_return;
}
- brelse(bitmap_bh);
bitmap_bh = ext4_read_block_bitmap(sb, block_group);
if (!bitmap_bh)
goto error_return;
@@ -418,18 +422,17 @@
in_range(block, ext4_inode_table(sb, desc), sbi->s_itb_per_group) ||
in_range(block + count - 1, ext4_inode_table(sb, desc),
sbi->s_itb_per_group)) {
- ext4_error(sb, "ext4_free_blocks",
- "Freeing blocks in system zones - "
+ ext4_error(sb, __func__,
+ "Adding blocks in system zones - "
"Block = %llu, count = %lu",
block, count);
goto error_return;
}
/*
- * We are about to start releasing blocks in the bitmap,
+ * We are about to add blocks to the bitmap,
* so we need undo access.
*/
- /* @@@ check errors */
BUFFER_TRACE(bitmap_bh, "getting undo access");
err = ext4_journal_get_undo_access(handle, bitmap_bh);
if (err)
@@ -444,107 +447,55 @@
err = ext4_journal_get_write_access(handle, gd_bh);
if (err)
goto error_return;
-
- jbd_lock_bh_state(bitmap_bh);
-
- for (i = 0, group_freed = 0; i < count; i++) {
- /*
- * An HJ special. This is expensive...
- */
-#ifdef CONFIG_JBD2_DEBUG
- jbd_unlock_bh_state(bitmap_bh);
- {
- struct buffer_head *debug_bh;
- debug_bh = sb_find_get_block(sb, block + i);
- if (debug_bh) {
- BUFFER_TRACE(debug_bh, "Deleted!");
- if (!bh2jh(bitmap_bh)->b_committed_data)
- BUFFER_TRACE(debug_bh,
- "No commited data in bitmap");
- BUFFER_TRACE2(debug_bh, bitmap_bh, "bitmap");
- __brelse(debug_bh);
- }
- }
- jbd_lock_bh_state(bitmap_bh);
-#endif
- if (need_resched()) {
- jbd_unlock_bh_state(bitmap_bh);
- cond_resched();
- jbd_lock_bh_state(bitmap_bh);
- }
- /* @@@ This prevents newly-allocated data from being
- * freed and then reallocated within the same
- * transaction.
- *
- * Ideally we would want to allow that to happen, but to
- * do so requires making jbd2_journal_forget() capable of
- * revoking the queued write of a data block, which
- * implies blocking on the journal lock. *forget()
- * cannot block due to truncate races.
- *
- * Eventually we can fix this by making jbd2_journal_forget()
- * return a status indicating whether or not it was able
- * to revoke the buffer. On successful revoke, it is
- * safe not to set the allocation bit in the committed
- * bitmap, because we know that there is no outstanding
- * activity on the buffer any more and so it is safe to
- * reallocate it.
- */
- BUFFER_TRACE(bitmap_bh, "set in b_committed_data");
- J_ASSERT_BH(bitmap_bh,
- bh2jh(bitmap_bh)->b_committed_data != NULL);
- ext4_set_bit_atomic(sb_bgl_lock(sbi, block_group), bit + i,
- bh2jh(bitmap_bh)->b_committed_data);
-
- /*
- * We clear the bit in the bitmap after setting the committed
- * data bit, because this is the reverse order to that which
- * the allocator uses.
- */
+ /*
+ * make sure we don't allow a parallel init on other groups in the
+ * same buddy cache
+ */
+ down_write(&grp->alloc_sem);
+ for (i = 0, blocks_freed = 0; i < count; i++) {
BUFFER_TRACE(bitmap_bh, "clear bit");
if (!ext4_clear_bit_atomic(sb_bgl_lock(sbi, block_group),
bit + i, bitmap_bh->b_data)) {
- jbd_unlock_bh_state(bitmap_bh);
ext4_error(sb, __func__,
"bit already cleared for block %llu",
(ext4_fsblk_t)(block + i));
- jbd_lock_bh_state(bitmap_bh);
BUFFER_TRACE(bitmap_bh, "bit already cleared");
} else {
- group_freed++;
+ blocks_freed++;
}
}
- jbd_unlock_bh_state(bitmap_bh);
-
spin_lock(sb_bgl_lock(sbi, block_group));
- le16_add_cpu(&desc->bg_free_blocks_count, group_freed);
+ blk_free_count = blocks_freed + ext4_free_blks_count(sb, desc);
+ ext4_free_blks_set(sb, desc, blk_free_count);
desc->bg_checksum = ext4_group_desc_csum(sbi, block_group, desc);
spin_unlock(sb_bgl_lock(sbi, block_group));
- percpu_counter_add(&sbi->s_freeblocks_counter, count);
+ percpu_counter_add(&sbi->s_freeblocks_counter, blocks_freed);
if (sbi->s_log_groups_per_flex) {
ext4_group_t flex_group = ext4_flex_group(sbi, block_group);
spin_lock(sb_bgl_lock(sbi, flex_group));
- sbi->s_flex_groups[flex_group].free_blocks += count;
+ sbi->s_flex_groups[flex_group].free_blocks += blocks_freed;
spin_unlock(sb_bgl_lock(sbi, flex_group));
}
+ /*
+ * request to reload the buddy with the
+ * new bitmap information
+ */
+ set_bit(EXT4_GROUP_INFO_NEED_INIT_BIT, &(grp->bb_state));
+ ext4_mb_update_group_info(grp, blocks_freed);
+ up_write(&grp->alloc_sem);
/* We dirtied the bitmap block */
BUFFER_TRACE(bitmap_bh, "dirtied bitmap block");
- err = ext4_journal_dirty_metadata(handle, bitmap_bh);
+ err = ext4_handle_dirty_metadata(handle, NULL, bitmap_bh);
/* And the group descriptor block */
BUFFER_TRACE(gd_bh, "dirtied group descriptor block");
- ret = ext4_journal_dirty_metadata(handle, gd_bh);
- if (!err) err = ret;
- *pdquot_freed_blocks += group_freed;
-
- if (overflow && !err) {
- block += count;
- count = overflow;
- goto do_more;
- }
+ ret = ext4_handle_dirty_metadata(handle, NULL, gd_bh);
+ if (!err)
+ err = ret;
sb->s_dirt = 1;
+
error_return:
brelse(bitmap_bh);
ext4_std_error(sb, err);
@@ -614,7 +565,7 @@
if (dirty_blocks < 0) {
printk(KERN_CRIT "Dirty block accounting "
"went wrong %lld\n",
- dirty_blocks);
+ (long long)dirty_blocks);
}
}
/* Check whether we have space after
@@ -666,101 +617,45 @@
return jbd2_journal_force_commit_nested(EXT4_SB(sb)->s_journal);
}
-#define EXT4_META_BLOCK 0x1
-
-static ext4_fsblk_t do_blk_alloc(handle_t *handle, struct inode *inode,
- ext4_lblk_t iblock, ext4_fsblk_t goal,
- unsigned long *count, int *errp, int flags)
-{
- struct ext4_allocation_request ar;
- ext4_fsblk_t ret;
-
- memset(&ar, 0, sizeof(ar));
- /* Fill with neighbour allocated blocks */
-
- ar.inode = inode;
- ar.goal = goal;
- ar.len = *count;
- ar.logical = iblock;
-
- if (S_ISREG(inode->i_mode) && !(flags & EXT4_META_BLOCK))
- /* enable in-core preallocation for data block allocation */
- ar.flags = EXT4_MB_HINT_DATA;
- else
- /* disable in-core preallocation for non-regular files */
- ar.flags = 0;
-
- ret = ext4_mb_new_blocks(handle, &ar, errp);
- *count = ar.len;
- return ret;
-}
-
/*
* ext4_new_meta_blocks() -- allocate block for meta data (indexing) blocks
*
* @handle: handle to this transaction
* @inode: file inode
* @goal: given target block(filesystem wide)
- * @count: total number of blocks need
+ * @count: pointer to total number of blocks needed
* @errp: error code
*
- * Return 1st allocated block numberon success, *count stores total account
+ * Return 1st allocated block number on success, *count stores total account
* error stores in errp pointer
*/
ext4_fsblk_t ext4_new_meta_blocks(handle_t *handle, struct inode *inode,
ext4_fsblk_t goal, unsigned long *count, int *errp)
{
+ struct ext4_allocation_request ar;
ext4_fsblk_t ret;
- ret = do_blk_alloc(handle, inode, 0, goal,
- count, errp, EXT4_META_BLOCK);
+
+ memset(&ar, 0, sizeof(ar));
+ /* Fill with neighbour allocated blocks */
+ ar.inode = inode;
+ ar.goal = goal;
+ ar.len = count ? *count : 1;
+
+ ret = ext4_mb_new_blocks(handle, &ar, errp);
+ if (count)
+ *count = ar.len;
+
/*
* Account for the allocated meta blocks
*/
if (!(*errp) && EXT4_I(inode)->i_delalloc_reserved_flag) {
spin_lock(&EXT4_I(inode)->i_block_reservation_lock);
- EXT4_I(inode)->i_allocated_meta_blocks += *count;
+ EXT4_I(inode)->i_allocated_meta_blocks += ar.len;
spin_unlock(&EXT4_I(inode)->i_block_reservation_lock);
}
return ret;
}
-/*
- * ext4_new_meta_block() -- allocate block for meta data (indexing) blocks
- *
- * @handle: handle to this transaction
- * @inode: file inode
- * @goal: given target block(filesystem wide)
- * @errp: error code
- *
- * Return allocated block number on success
- */
-ext4_fsblk_t ext4_new_meta_block(handle_t *handle, struct inode *inode,
- ext4_fsblk_t goal, int *errp)
-{
- unsigned long count = 1;
- return ext4_new_meta_blocks(handle, inode, goal, &count, errp);
-}
-
-/*
- * ext4_new_blocks() -- allocate data blocks
- *
- * @handle: handle to this transaction
- * @inode: file inode
- * @goal: given target block(filesystem wide)
- * @count: total number of blocks need
- * @errp: error code
- *
- * Return 1st allocated block numberon success, *count stores total account
- * error stores in errp pointer
- */
-
-ext4_fsblk_t ext4_new_blocks(handle_t *handle, struct inode *inode,
- ext4_lblk_t iblock, ext4_fsblk_t goal,
- unsigned long *count, int *errp)
-{
- return do_blk_alloc(handle, inode, iblock, goal, count, errp, 0);
-}
-
/**
* ext4_count_free_blocks() -- count filesystem free blocks
* @sb: superblock
@@ -776,7 +671,7 @@
#ifdef EXT4FS_DEBUG
struct ext4_super_block *es;
ext4_fsblk_t bitmap_count;
- unsigned long x;
+ unsigned int x;
struct buffer_head *bitmap_bh = NULL;
es = EXT4_SB(sb)->s_es;
@@ -796,7 +691,7 @@
continue;
x = ext4_count_free(bitmap_bh, sb->s_blocksize);
- printk(KERN_DEBUG "group %lu: stored = %d, counted = %lu\n",
+ printk(KERN_DEBUG "group %lu: stored = %d, counted = %u\n",
i, le16_to_cpu(gdp->bg_free_blocks_count), x);
bitmap_count += x;
}
@@ -812,7 +707,7 @@
gdp = ext4_get_group_desc(sb, i, NULL);
if (!gdp)
continue;
- desc_count += le16_to_cpu(gdp->bg_free_blocks_count);
+ desc_count += ext4_free_blks_count(sb, gdp);
}
return desc_count;
diff --git a/fs/ext4/bitmap.c b/fs/ext4/bitmap.c
index 0a7a666..fa3af81 100644
--- a/fs/ext4/bitmap.c
+++ b/fs/ext4/bitmap.c
@@ -15,10 +15,9 @@
static const int nibblemap[] = {4, 3, 3, 2, 3, 2, 2, 1, 3, 2, 2, 1, 2, 1, 1, 0};
-unsigned long ext4_count_free(struct buffer_head *map, unsigned int numchars)
+unsigned int ext4_count_free(struct buffer_head *map, unsigned int numchars)
{
- unsigned int i;
- unsigned long sum = 0;
+ unsigned int i, sum = 0;
if (!map)
return 0;
diff --git a/fs/ext4/dir.c b/fs/ext4/dir.c
index fed5b61..2df2e40 100644
--- a/fs/ext4/dir.c
+++ b/fs/ext4/dir.c
@@ -64,7 +64,7 @@
int ext4_check_dir_entry(const char *function, struct inode *dir,
struct ext4_dir_entry_2 *de,
struct buffer_head *bh,
- unsigned long offset)
+ unsigned int offset)
{
const char *error_msg = NULL;
const int rlen = ext4_rec_len_from_disk(de->rec_len);
@@ -84,9 +84,9 @@
if (error_msg != NULL)
ext4_error(dir->i_sb, function,
"bad entry in directory #%lu: %s - "
- "offset=%lu, inode=%lu, rec_len=%d, name_len=%d",
+ "offset=%u, inode=%u, rec_len=%d, name_len=%d",
dir->i_ino, error_msg, offset,
- (unsigned long) le32_to_cpu(de->inode),
+ le32_to_cpu(de->inode),
rlen, de->name_len);
return error_msg == NULL ? 1 : 0;
}
@@ -95,7 +95,7 @@
void *dirent, filldir_t filldir)
{
int error = 0;
- unsigned long offset;
+ unsigned int offset;
int i, stored;
struct ext4_dir_entry_2 *de;
struct super_block *sb;
@@ -405,7 +405,7 @@
sb = inode->i_sb;
if (!fname) {
- printk(KERN_ERR "ext4: call_filldir: called with "
+ printk(KERN_ERR "EXT4-fs: call_filldir: called with "
"null fname?!?\n");
return 0;
}
diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index 6c46c64..c668e43 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -19,6 +19,7 @@
#include <linux/types.h>
#include <linux/blkdev.h>
#include <linux/magic.h>
+#include <linux/jbd2.h>
#include "ext4_i.h"
/*
@@ -94,9 +95,9 @@
/* phys. block for ^^^ */
ext4_fsblk_t pright;
/* how many blocks we want to allocate */
- unsigned long len;
+ unsigned int len;
/* flags. see above EXT4_MB_HINT_* */
- unsigned long flags;
+ unsigned int flags;
};
/*
@@ -156,12 +157,12 @@
__le32 bg_block_bitmap_lo; /* Blocks bitmap block */
__le32 bg_inode_bitmap_lo; /* Inodes bitmap block */
__le32 bg_inode_table_lo; /* Inodes table block */
- __le16 bg_free_blocks_count; /* Free blocks count */
- __le16 bg_free_inodes_count; /* Free inodes count */
- __le16 bg_used_dirs_count; /* Directories count */
+ __le16 bg_free_blocks_count_lo;/* Free blocks count */
+ __le16 bg_free_inodes_count_lo;/* Free inodes count */
+ __le16 bg_used_dirs_count_lo; /* Directories count */
__le16 bg_flags; /* EXT4_BG_flags (INODE_UNINIT, etc) */
__u32 bg_reserved[2]; /* Likely block/inode bitmap checksum */
- __le16 bg_itable_unused; /* Unused inodes count */
+ __le16 bg_itable_unused_lo; /* Unused inodes count */
__le16 bg_checksum; /* crc16(sb_uuid+group+desc) */
__le32 bg_block_bitmap_hi; /* Blocks bitmap block MSB */
__le32 bg_inode_bitmap_hi; /* Inodes bitmap block MSB */
@@ -169,7 +170,7 @@
__le16 bg_free_blocks_count_hi;/* Free blocks count MSB */
__le16 bg_free_inodes_count_hi;/* Free inodes count MSB */
__le16 bg_used_dirs_count_hi; /* Directories count MSB */
- __le16 bg_itable_unused_hi; /* Unused inodes count MSB */
+ __le16 bg_itable_unused_hi; /* Unused inodes count MSB */
__u32 bg_reserved2[3];
};
@@ -328,6 +329,7 @@
uid_t s_resuid;
gid_t s_resgid;
unsigned long s_commit_interval;
+ u32 s_min_batch_time, s_max_batch_time;
#ifdef CONFIG_QUOTA
int s_jquota_fmt;
char *s_qf_names[MAXQUOTAS];
@@ -534,7 +536,6 @@
#define EXT4_MOUNT_QUOTA 0x80000 /* Some quota option set */
#define EXT4_MOUNT_USRQUOTA 0x100000 /* "old" user quota */
#define EXT4_MOUNT_GRPQUOTA 0x200000 /* "old" group quota */
-#define EXT4_MOUNT_EXTENTS 0x400000 /* Extents support */
#define EXT4_MOUNT_JOURNAL_CHECKSUM 0x800000 /* Journal checksums */
#define EXT4_MOUNT_JOURNAL_ASYNC_COMMIT 0x1000000 /* Journal Async Commit */
#define EXT4_MOUNT_I_VERSION 0x2000000 /* i_version support */
@@ -726,11 +727,11 @@
*/
#define EXT4_HAS_COMPAT_FEATURE(sb,mask) \
- (EXT4_SB(sb)->s_es->s_feature_compat & cpu_to_le32(mask))
+ ((EXT4_SB(sb)->s_es->s_feature_compat & cpu_to_le32(mask)) != 0)
#define EXT4_HAS_RO_COMPAT_FEATURE(sb,mask) \
- (EXT4_SB(sb)->s_es->s_feature_ro_compat & cpu_to_le32(mask))
+ ((EXT4_SB(sb)->s_es->s_feature_ro_compat & cpu_to_le32(mask)) != 0)
#define EXT4_HAS_INCOMPAT_FEATURE(sb,mask) \
- (EXT4_SB(sb)->s_es->s_feature_incompat & cpu_to_le32(mask))
+ ((EXT4_SB(sb)->s_es->s_feature_incompat & cpu_to_le32(mask)) != 0)
#define EXT4_SET_COMPAT_FEATURE(sb,mask) \
EXT4_SB(sb)->s_es->s_feature_compat |= cpu_to_le32(mask)
#define EXT4_SET_RO_COMPAT_FEATURE(sb,mask) \
@@ -806,6 +807,12 @@
#define EXT4_DEFM_JMODE_WBACK 0x0060
/*
+ * Default journal batch times
+ */
+#define EXT4_DEF_MIN_BATCH_TIME 0
+#define EXT4_DEF_MAX_BATCH_TIME 15000 /* 15ms */
+
+/*
* Structure of a directory entry
*/
#define EXT4_NAME_LEN 255
@@ -891,6 +898,9 @@
#define DX_HASH_LEGACY 0
#define DX_HASH_HALF_MD4 1
#define DX_HASH_TEA 2
+#define DX_HASH_LEGACY_UNSIGNED 3
+#define DX_HASH_HALF_MD4_UNSIGNED 4
+#define DX_HASH_TEA_UNSIGNED 5
#ifdef __KERNEL__
@@ -955,7 +965,7 @@
#define ERR_BAD_DX_DIR -75000
void ext4_get_group_no_and_offset(struct super_block *sb, ext4_fsblk_t blocknr,
- unsigned long *blockgrpp, ext4_grpblk_t *offsetp);
+ ext4_group_t *blockgrpp, ext4_grpblk_t *offsetp);
extern struct proc_dir_entry *ext4_proc_root;
@@ -987,6 +997,9 @@
# define ATTRIB_NORET __attribute__((noreturn))
# define NORET_AND noreturn,
+/* bitmap.c */
+extern unsigned int ext4_count_free(struct buffer_head *, unsigned);
+
/* balloc.c */
extern unsigned int ext4_block_group(struct super_block *sb,
ext4_fsblk_t blocknr);
@@ -995,20 +1008,14 @@
extern int ext4_bg_has_super(struct super_block *sb, ext4_group_t group);
extern unsigned long ext4_bg_num_gdb(struct super_block *sb,
ext4_group_t group);
-extern ext4_fsblk_t ext4_new_meta_block(handle_t *handle, struct inode *inode,
- ext4_fsblk_t goal, int *errp);
extern ext4_fsblk_t ext4_new_meta_blocks(handle_t *handle, struct inode *inode,
ext4_fsblk_t goal, unsigned long *count, int *errp);
-extern ext4_fsblk_t ext4_new_blocks(handle_t *handle, struct inode *inode,
- ext4_lblk_t iblock, ext4_fsblk_t goal,
- unsigned long *count, int *errp);
extern int ext4_claim_free_blocks(struct ext4_sb_info *sbi, s64 nblocks);
extern int ext4_has_free_blocks(struct ext4_sb_info *sbi, s64 nblocks);
extern void ext4_free_blocks(handle_t *handle, struct inode *inode,
ext4_fsblk_t block, unsigned long count, int metadata);
-extern void ext4_free_blocks_sb(handle_t *handle, struct super_block *sb,
- ext4_fsblk_t block, unsigned long count,
- unsigned long *pdquot_freed_blocks);
+extern void ext4_add_groupblocks(handle_t *handle, struct super_block *sb,
+ ext4_fsblk_t block, unsigned long count);
extern ext4_fsblk_t ext4_count_free_blocks(struct super_block *);
extern void ext4_check_blocks_bitmap(struct super_block *);
extern struct ext4_group_desc * ext4_get_group_desc(struct super_block * sb,
@@ -1019,7 +1026,7 @@
/* dir.c */
extern int ext4_check_dir_entry(const char *, struct inode *,
struct ext4_dir_entry_2 *,
- struct buffer_head *, unsigned long);
+ struct buffer_head *, unsigned int);
extern int ext4_htree_store_dirent(struct file *dir_file, __u32 hash,
__u32 minor_hash,
struct ext4_dir_entry_2 *dirent);
@@ -1039,7 +1046,6 @@
extern unsigned long ext4_count_free_inodes(struct super_block *);
extern unsigned long ext4_count_dirs(struct super_block *);
extern void ext4_check_inodes_bitmap(struct super_block *);
-extern unsigned long ext4_count_free(struct buffer_head *, unsigned);
/* mballoc.c */
extern long ext4_mb_stats;
@@ -1054,12 +1060,13 @@
extern void exit_ext4_mballoc(void);
extern void ext4_mb_free_blocks(handle_t *, struct inode *,
unsigned long, unsigned long, int, unsigned long *);
-extern int ext4_mb_add_more_groupinfo(struct super_block *sb,
+extern int ext4_mb_add_groupinfo(struct super_block *sb,
ext4_group_t i, struct ext4_group_desc *desc);
extern void ext4_mb_update_group_info(struct ext4_group_info *grp,
ext4_grpblk_t add);
-
-
+extern int ext4_mb_get_buddy_cache_lock(struct super_block *, ext4_group_t);
+extern void ext4_mb_put_buddy_cache_lock(struct super_block *,
+ ext4_group_t, int);
/* inode.c */
int ext4_forget(handle_t *handle, int is_metadata, struct inode *inode,
struct buffer_head *bh, ext4_fsblk_t blocknr);
@@ -1069,10 +1076,6 @@
ext4_lblk_t, int, int *);
int ext4_get_block(struct inode *inode, sector_t iblock,
struct buffer_head *bh_result, int create);
-int ext4_get_blocks_handle(handle_t *handle, struct inode *inode,
- ext4_lblk_t iblock, unsigned long maxblocks,
- struct buffer_head *bh_result,
- int create, int extend_disksize);
extern struct inode *ext4_iget(struct super_block *, unsigned long);
extern int ext4_write_inode(struct inode *, int);
@@ -1123,6 +1126,9 @@
__attribute__ ((format (printf, 3, 4)));
extern void ext4_warning(struct super_block *, const char *, const char *, ...)
__attribute__ ((format (printf, 3, 4)));
+extern void ext4_grp_locked_error(struct super_block *, ext4_group_t,
+ const char *, const char *, ...)
+ __attribute__ ((format (printf, 4, 5)));
extern void ext4_update_dynamic_rev(struct super_block *sb);
extern int ext4_update_compat_feature(handle_t *handle, struct super_block *sb,
__u32 compat);
@@ -1136,12 +1142,28 @@
struct ext4_group_desc *bg);
extern ext4_fsblk_t ext4_inode_table(struct super_block *sb,
struct ext4_group_desc *bg);
+extern __u32 ext4_free_blks_count(struct super_block *sb,
+ struct ext4_group_desc *bg);
+extern __u32 ext4_free_inodes_count(struct super_block *sb,
+ struct ext4_group_desc *bg);
+extern __u32 ext4_used_dirs_count(struct super_block *sb,
+ struct ext4_group_desc *bg);
+extern __u32 ext4_itable_unused_count(struct super_block *sb,
+ struct ext4_group_desc *bg);
extern void ext4_block_bitmap_set(struct super_block *sb,
struct ext4_group_desc *bg, ext4_fsblk_t blk);
extern void ext4_inode_bitmap_set(struct super_block *sb,
struct ext4_group_desc *bg, ext4_fsblk_t blk);
extern void ext4_inode_table_set(struct super_block *sb,
struct ext4_group_desc *bg, ext4_fsblk_t blk);
+extern void ext4_free_blks_set(struct super_block *sb,
+ struct ext4_group_desc *bg, __u32 count);
+extern void ext4_free_inodes_set(struct super_block *sb,
+ struct ext4_group_desc *bg, __u32 count);
+extern void ext4_used_dirs_set(struct super_block *sb,
+ struct ext4_group_desc *bg, __u32 count);
+extern void ext4_itable_unused_set(struct super_block *sb,
+ struct ext4_group_desc *bg, __u32 count);
static inline ext4_fsblk_t ext4_blocks_count(struct ext4_super_block *es)
{
@@ -1246,6 +1268,50 @@
return ;
}
+struct ext4_group_info {
+ unsigned long bb_state;
+ struct rb_root bb_free_root;
+ unsigned short bb_first_free;
+ unsigned short bb_free;
+ unsigned short bb_fragments;
+ struct list_head bb_prealloc_list;
+#ifdef DOUBLE_CHECK
+ void *bb_bitmap;
+#endif
+ struct rw_semaphore alloc_sem;
+ unsigned short bb_counters[];
+};
+
+#define EXT4_GROUP_INFO_NEED_INIT_BIT 0
+#define EXT4_GROUP_INFO_LOCKED_BIT 1
+
+#define EXT4_MB_GRP_NEED_INIT(grp) \
+ (test_bit(EXT4_GROUP_INFO_NEED_INIT_BIT, &((grp)->bb_state)))
+
+static inline void ext4_lock_group(struct super_block *sb, ext4_group_t group)
+{
+ struct ext4_group_info *grinfo = ext4_get_group_info(sb, group);
+
+ bit_spin_lock(EXT4_GROUP_INFO_LOCKED_BIT, &(grinfo->bb_state));
+}
+
+static inline void ext4_unlock_group(struct super_block *sb,
+ ext4_group_t group)
+{
+ struct ext4_group_info *grinfo = ext4_get_group_info(sb, group);
+
+ bit_spin_unlock(EXT4_GROUP_INFO_LOCKED_BIT, &(grinfo->bb_state));
+}
+
+static inline int ext4_is_group_locked(struct super_block *sb,
+ ext4_group_t group)
+{
+ struct ext4_group_info *grinfo = ext4_get_group_info(sb, group);
+
+ return bit_spin_is_locked(EXT4_GROUP_INFO_LOCKED_BIT,
+ &(grinfo->bb_state));
+}
+
/*
* Inodes and files operations
*/
@@ -1271,18 +1337,38 @@
extern int ext4_ext_index_trans_blocks(struct inode *inode, int nrblocks,
int chunk);
extern int ext4_ext_get_blocks(handle_t *handle, struct inode *inode,
- ext4_lblk_t iblock,
- unsigned long max_blocks, struct buffer_head *bh_result,
- int create, int extend_disksize);
+ ext4_lblk_t iblock, unsigned int max_blocks,
+ struct buffer_head *bh_result,
+ int create, int extend_disksize);
extern void ext4_ext_truncate(struct inode *);
extern void ext4_ext_init(struct super_block *);
extern void ext4_ext_release(struct super_block *);
extern long ext4_fallocate(struct inode *inode, int mode, loff_t offset,
loff_t len);
extern int ext4_get_blocks_wrap(handle_t *handle, struct inode *inode,
- sector_t block, unsigned long max_blocks,
+ sector_t block, unsigned int max_blocks,
struct buffer_head *bh, int create,
int extend_disksize, int flag);
+extern int ext4_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
+ __u64 start, __u64 len);
+
+/*
+ * Add new method to test wether block and inode bitmaps are properly
+ * initialized. With uninit_bg reading the block from disk is not enough
+ * to mark the bitmap uptodate. We need to also zero-out the bitmap
+ */
+#define BH_BITMAP_UPTODATE BH_JBDPrivateStart
+
+static inline int bitmap_uptodate(struct buffer_head *bh)
+{
+ return (buffer_uptodate(bh) &&
+ test_bit(BH_BITMAP_UPTODATE, &(bh)->b_state));
+}
+static inline void set_bitmap_uptodate(struct buffer_head *bh)
+{
+ set_bit(BH_BITMAP_UPTODATE, &(bh)->b_state);
+}
+
#endif /* __KERNEL__ */
#endif /* _EXT4_H */
diff --git a/fs/ext4/ext4_extents.h b/fs/ext4/ext4_extents.h
index bec7ce5..18cb67b 100644
--- a/fs/ext4/ext4_extents.h
+++ b/fs/ext4/ext4_extents.h
@@ -194,11 +194,6 @@
return le16_to_cpu(ext_inode_hdr(inode)->eh_depth);
}
-static inline void ext4_ext_tree_changed(struct inode *inode)
-{
- EXT4_I(inode)->i_ext_generation++;
-}
-
static inline void
ext4_ext_invalidate_cache(struct inode *inode)
{
diff --git a/fs/ext4/ext4_i.h b/fs/ext4/ext4_i.h
index 5c124c0..e69acc1 100644
--- a/fs/ext4/ext4_i.h
+++ b/fs/ext4/ext4_i.h
@@ -31,7 +31,7 @@
typedef __u32 ext4_lblk_t;
/* data type for block group number */
-typedef unsigned long ext4_group_t;
+typedef unsigned int ext4_group_t;
#define rsv_start rsv_window._rsv_start
#define rsv_end rsv_window._rsv_end
@@ -100,9 +100,6 @@
*/
loff_t i_disksize;
- /* on-disk additional length */
- __u16 i_extra_isize;
-
/*
* i_data_sem is for serialising ext4_truncate() against
* ext4_getblock(). In the 2.4 ext2 design, great chunks of inode's
@@ -117,7 +114,6 @@
struct inode vfs_inode;
struct jbd2_inode jinode;
- unsigned long i_ext_generation;
struct ext4_ext_cache i_cached_extent;
/*
* File creation time. Its function is same as that of
@@ -130,10 +126,14 @@
spinlock_t i_prealloc_lock;
/* allocation reservation info for delalloc */
- unsigned long i_reserved_data_blocks;
- unsigned long i_reserved_meta_blocks;
- unsigned long i_allocated_meta_blocks;
+ unsigned int i_reserved_data_blocks;
+ unsigned int i_reserved_meta_blocks;
+ unsigned int i_allocated_meta_blocks;
unsigned short i_delalloc_reserved_flag;
+
+ /* on-disk additional length */
+ __u16 i_extra_isize;
+
spinlock_t i_block_reservation_lock;
};
diff --git a/fs/ext4/ext4_jbd2.c b/fs/ext4/ext4_jbd2.c
index c75384b..ad13a84 100644
--- a/fs/ext4/ext4_jbd2.c
+++ b/fs/ext4/ext4_jbd2.c
@@ -7,53 +7,96 @@
int __ext4_journal_get_undo_access(const char *where, handle_t *handle,
struct buffer_head *bh)
{
- int err = jbd2_journal_get_undo_access(handle, bh);
- if (err)
- ext4_journal_abort_handle(where, __func__, bh, handle, err);
+ int err = 0;
+
+ if (ext4_handle_valid(handle)) {
+ err = jbd2_journal_get_undo_access(handle, bh);
+ if (err)
+ ext4_journal_abort_handle(where, __func__, bh,
+ handle, err);
+ }
return err;
}
int __ext4_journal_get_write_access(const char *where, handle_t *handle,
struct buffer_head *bh)
{
- int err = jbd2_journal_get_write_access(handle, bh);
- if (err)
- ext4_journal_abort_handle(where, __func__, bh, handle, err);
+ int err = 0;
+
+ if (ext4_handle_valid(handle)) {
+ err = jbd2_journal_get_write_access(handle, bh);
+ if (err)
+ ext4_journal_abort_handle(where, __func__, bh,
+ handle, err);
+ }
return err;
}
int __ext4_journal_forget(const char *where, handle_t *handle,
struct buffer_head *bh)
{
- int err = jbd2_journal_forget(handle, bh);
- if (err)
- ext4_journal_abort_handle(where, __func__, bh, handle, err);
+ int err = 0;
+
+ if (ext4_handle_valid(handle)) {
+ err = jbd2_journal_forget(handle, bh);
+ if (err)
+ ext4_journal_abort_handle(where, __func__, bh,
+ handle, err);
+ }
return err;
}
int __ext4_journal_revoke(const char *where, handle_t *handle,
ext4_fsblk_t blocknr, struct buffer_head *bh)
{
- int err = jbd2_journal_revoke(handle, blocknr, bh);
- if (err)
- ext4_journal_abort_handle(where, __func__, bh, handle, err);
+ int err = 0;
+
+ if (ext4_handle_valid(handle)) {
+ err = jbd2_journal_revoke(handle, blocknr, bh);
+ if (err)
+ ext4_journal_abort_handle(where, __func__, bh,
+ handle, err);
+ }
return err;
}
int __ext4_journal_get_create_access(const char *where,
handle_t *handle, struct buffer_head *bh)
{
- int err = jbd2_journal_get_create_access(handle, bh);
- if (err)
- ext4_journal_abort_handle(where, __func__, bh, handle, err);
+ int err = 0;
+
+ if (ext4_handle_valid(handle)) {
+ err = jbd2_journal_get_create_access(handle, bh);
+ if (err)
+ ext4_journal_abort_handle(where, __func__, bh,
+ handle, err);
+ }
return err;
}
-int __ext4_journal_dirty_metadata(const char *where,
- handle_t *handle, struct buffer_head *bh)
+int __ext4_handle_dirty_metadata(const char *where, handle_t *handle,
+ struct inode *inode, struct buffer_head *bh)
{
- int err = jbd2_journal_dirty_metadata(handle, bh);
- if (err)
- ext4_journal_abort_handle(where, __func__, bh, handle, err);
+ int err = 0;
+
+ if (ext4_handle_valid(handle)) {
+ err = jbd2_journal_dirty_metadata(handle, bh);
+ if (err)
+ ext4_journal_abort_handle(where, __func__, bh,
+ handle, err);
+ } else {
+ mark_buffer_dirty(bh);
+ if (inode && inode_needs_sync(inode)) {
+ sync_dirty_buffer(bh);
+ if (buffer_req(bh) && !buffer_uptodate(bh)) {
+ ext4_error(inode->i_sb, __func__,
+ "IO error syncing inode, "
+ "inode=%lu, block=%llu",
+ inode->i_ino,
+ (unsigned long long) bh->b_blocknr);
+ err = -EIO;
+ }
+ }
+ }
return err;
}
diff --git a/fs/ext4/ext4_jbd2.h b/fs/ext4/ext4_jbd2.h
index b455c68..be2f426 100644
--- a/fs/ext4/ext4_jbd2.h
+++ b/fs/ext4/ext4_jbd2.h
@@ -32,8 +32,8 @@
* 5 levels of tree + root which are stored in the inode. */
#define EXT4_SINGLEDATA_TRANS_BLOCKS(sb) \
- (EXT4_HAS_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_EXTENTS) \
- || test_opt(sb, EXTENTS) ? 27U : 8U)
+ (EXT4_HAS_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_EXTENTS) \
+ ? 27U : 8U)
/* Extended attribute operations touch at most two data buffers,
* two bitmap buffers, and two group summaries, in addition to the inode
@@ -122,12 +122,6 @@
* been done yet.
*/
-static inline void ext4_journal_release_buffer(handle_t *handle,
- struct buffer_head *bh)
-{
- jbd2_journal_release_buffer(handle, bh);
-}
-
void ext4_journal_abort_handle(const char *caller, const char *err_fn,
struct buffer_head *bh, handle_t *handle, int err);
@@ -146,8 +140,8 @@
int __ext4_journal_get_create_access(const char *where,
handle_t *handle, struct buffer_head *bh);
-int __ext4_journal_dirty_metadata(const char *where,
- handle_t *handle, struct buffer_head *bh);
+int __ext4_handle_dirty_metadata(const char *where, handle_t *handle,
+ struct inode *inode, struct buffer_head *bh);
#define ext4_journal_get_undo_access(handle, bh) \
__ext4_journal_get_undo_access(__func__, (handle), (bh))
@@ -157,14 +151,57 @@
__ext4_journal_revoke(__func__, (handle), (blocknr), (bh))
#define ext4_journal_get_create_access(handle, bh) \
__ext4_journal_get_create_access(__func__, (handle), (bh))
-#define ext4_journal_dirty_metadata(handle, bh) \
- __ext4_journal_dirty_metadata(__func__, (handle), (bh))
#define ext4_journal_forget(handle, bh) \
__ext4_journal_forget(__func__, (handle), (bh))
+#define ext4_handle_dirty_metadata(handle, inode, bh) \
+ __ext4_handle_dirty_metadata(__func__, (handle), (inode), (bh))
handle_t *ext4_journal_start_sb(struct super_block *sb, int nblocks);
int __ext4_journal_stop(const char *where, handle_t *handle);
+#define EXT4_NOJOURNAL_HANDLE ((handle_t *) 0x1)
+
+static inline int ext4_handle_valid(handle_t *handle)
+{
+ if (handle == EXT4_NOJOURNAL_HANDLE)
+ return 0;
+ return 1;
+}
+
+static inline void ext4_handle_sync(handle_t *handle)
+{
+ if (ext4_handle_valid(handle))
+ handle->h_sync = 1;
+}
+
+static inline void ext4_handle_release_buffer(handle_t *handle,
+ struct buffer_head *bh)
+{
+ if (ext4_handle_valid(handle))
+ jbd2_journal_release_buffer(handle, bh);
+}
+
+static inline int ext4_handle_is_aborted(handle_t *handle)
+{
+ if (ext4_handle_valid(handle))
+ return is_handle_aborted(handle);
+ return 0;
+}
+
+static inline int ext4_handle_has_enough_credits(handle_t *handle, int needed)
+{
+ if (ext4_handle_valid(handle) && handle->h_buffer_credits < needed)
+ return 0;
+ return 1;
+}
+
+static inline void ext4_journal_release_buffer(handle_t *handle,
+ struct buffer_head *bh)
+{
+ if (ext4_handle_valid(handle))
+ jbd2_journal_release_buffer(handle, bh);
+}
+
static inline handle_t *ext4_journal_start(struct inode *inode, int nblocks)
{
return ext4_journal_start_sb(inode->i_sb, nblocks);
@@ -180,27 +217,37 @@
static inline int ext4_journal_extend(handle_t *handle, int nblocks)
{
- return jbd2_journal_extend(handle, nblocks);
+ if (ext4_handle_valid(handle))
+ return jbd2_journal_extend(handle, nblocks);
+ return 0;
}
static inline int ext4_journal_restart(handle_t *handle, int nblocks)
{
- return jbd2_journal_restart(handle, nblocks);
+ if (ext4_handle_valid(handle))
+ return jbd2_journal_restart(handle, nblocks);
+ return 0;
}
static inline int ext4_journal_blocks_per_page(struct inode *inode)
{
- return jbd2_journal_blocks_per_page(inode);
+ if (EXT4_JOURNAL(inode) != NULL)
+ return jbd2_journal_blocks_per_page(inode);
+ return 0;
}
static inline int ext4_journal_force_commit(journal_t *journal)
{
- return jbd2_journal_force_commit(journal);
+ if (journal)
+ return jbd2_journal_force_commit(journal);
+ return 0;
}
static inline int ext4_jbd2_file_inode(handle_t *handle, struct inode *inode)
{
- return jbd2_journal_file_inode(handle, &EXT4_I(inode)->jinode);
+ if (ext4_handle_valid(handle))
+ return jbd2_journal_file_inode(handle, &EXT4_I(inode)->jinode);
+ return 0;
}
/* super.c */
@@ -208,6 +255,8 @@
static inline int ext4_should_journal_data(struct inode *inode)
{
+ if (EXT4_JOURNAL(inode) == NULL)
+ return 0;
if (!S_ISREG(inode->i_mode))
return 1;
if (test_opt(inode->i_sb, DATA_FLAGS) == EXT4_MOUNT_JOURNAL_DATA)
@@ -219,6 +268,8 @@
static inline int ext4_should_order_data(struct inode *inode)
{
+ if (EXT4_JOURNAL(inode) == NULL)
+ return 0;
if (!S_ISREG(inode->i_mode))
return 0;
if (EXT4_I(inode)->i_flags & EXT4_JOURNAL_DATA_FL)
@@ -230,6 +281,8 @@
static inline int ext4_should_writeback_data(struct inode *inode)
{
+ if (EXT4_JOURNAL(inode) == NULL)
+ return 0;
if (!S_ISREG(inode->i_mode))
return 0;
if (EXT4_I(inode)->i_flags & EXT4_JOURNAL_DATA_FL)
diff --git a/fs/ext4/ext4_sb.h b/fs/ext4/ext4_sb.h
index b21f167..039b6ea 100644
--- a/fs/ext4/ext4_sb.h
+++ b/fs/ext4/ext4_sb.h
@@ -57,6 +57,7 @@
u32 s_next_generation;
u32 s_hash_seed[4];
int s_def_hash_version;
+ int s_hash_unsigned; /* 3 if hash should be signed, 0 if not */
struct percpu_counter s_freeblocks_counter;
struct percpu_counter s_freeinodes_counter;
struct percpu_counter s_dirs_counter;
@@ -73,6 +74,8 @@
struct journal_s *s_journal;
struct list_head s_orphan;
unsigned long s_commit_interval;
+ u32 s_max_batch_time;
+ u32 s_min_batch_time;
struct block_device *journal_bdev;
#ifdef CONFIG_JBD2_DEBUG
struct timer_list turn_ro_timer; /* For turning read-only (crash simulation) */
@@ -101,7 +104,8 @@
spinlock_t s_reserve_lock;
spinlock_t s_md_lock;
tid_t s_last_transaction;
- unsigned short *s_mb_offsets, *s_mb_maxs;
+ unsigned short *s_mb_offsets;
+ unsigned int *s_mb_maxs;
/* tunables */
unsigned long s_stripe;
diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index ea2ce3c..54bf062 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -97,6 +97,8 @@
{
int err;
+ if (!ext4_handle_valid(handle))
+ return 0;
if (handle->h_buffer_credits > needed)
return 0;
err = ext4_journal_extend(handle, needed);
@@ -134,7 +136,7 @@
int err;
if (path->p_bh) {
/* path points to block */
- err = ext4_journal_dirty_metadata(handle, path->p_bh);
+ err = ext4_handle_dirty_metadata(handle, inode, path->p_bh);
} else {
/* path points to leaf/index in inode body */
err = ext4_mark_inode_dirty(handle, inode);
@@ -191,7 +193,7 @@
ext4_fsblk_t goal, newblock;
goal = ext4_ext_find_goal(inode, path, le32_to_cpu(ex->ee_block));
- newblock = ext4_new_meta_block(handle, inode, goal, err);
+ newblock = ext4_new_meta_blocks(handle, inode, goal, NULL, err);
return newblock;
}
@@ -780,7 +782,7 @@
set_buffer_uptodate(bh);
unlock_buffer(bh);
- err = ext4_journal_dirty_metadata(handle, bh);
+ err = ext4_handle_dirty_metadata(handle, inode, bh);
if (err)
goto cleanup;
brelse(bh);
@@ -859,7 +861,7 @@
set_buffer_uptodate(bh);
unlock_buffer(bh);
- err = ext4_journal_dirty_metadata(handle, bh);
+ err = ext4_handle_dirty_metadata(handle, inode, bh);
if (err)
goto cleanup;
brelse(bh);
@@ -955,7 +957,7 @@
set_buffer_uptodate(bh);
unlock_buffer(bh);
- err = ext4_journal_dirty_metadata(handle, bh);
+ err = ext4_handle_dirty_metadata(handle, inode, bh);
if (err)
goto out;
@@ -1160,15 +1162,13 @@
while (--depth >= 0) {
ix = path[depth].p_idx;
if (ix != EXT_LAST_INDEX(path[depth].p_hdr))
- break;
+ goto got_index;
}
- if (depth < 0) {
- /* we've gone up to the root and
- * found no index to the right */
- return 0;
- }
+ /* we've gone up to the root and found no index to the right */
+ return 0;
+got_index:
/* we've found index to the right, let's
* follow it and find the closest allocated
* block to the right */
@@ -1201,7 +1201,6 @@
*phys = ext_pblock(ex);
put_bh(bh);
return 0;
-
}
/*
@@ -1622,7 +1621,6 @@
ext4_ext_drop_refs(npath);
kfree(npath);
}
- ext4_ext_tree_changed(inode);
ext4_ext_invalidate_cache(inode);
return err;
}
@@ -2233,7 +2231,6 @@
}
}
out:
- ext4_ext_tree_changed(inode);
ext4_ext_drop_refs(path);
kfree(path);
ext4_journal_stop(handle);
@@ -2250,7 +2247,7 @@
* possible initialization would be here
*/
- if (test_opt(sb, EXTENTS)) {
+ if (EXT4_HAS_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_EXTENTS)) {
printk(KERN_INFO "EXT4-fs: file extents enabled");
#ifdef AGGRESSIVE_TEST
printk(", aggressive tests");
@@ -2275,7 +2272,7 @@
*/
void ext4_ext_release(struct super_block *sb)
{
- if (!test_opt(sb, EXTENTS))
+ if (!EXT4_HAS_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_EXTENTS))
return;
#ifdef EXTENTS_STATS
@@ -2380,7 +2377,7 @@
struct inode *inode,
struct ext4_ext_path *path,
ext4_lblk_t iblock,
- unsigned long max_blocks)
+ unsigned int max_blocks)
{
struct ext4_extent *ex, newex, orig_ex;
struct ext4_extent *ex1 = NULL;
@@ -2536,7 +2533,7 @@
*/
newdepth = ext_depth(inode);
/*
- * update the extent length after successfull insert of the
+ * update the extent length after successful insert of the
* split extent
*/
orig_ex.ee_len = cpu_to_le16(ee_len -
@@ -2678,26 +2675,26 @@
*/
int ext4_ext_get_blocks(handle_t *handle, struct inode *inode,
ext4_lblk_t iblock,
- unsigned long max_blocks, struct buffer_head *bh_result,
+ unsigned int max_blocks, struct buffer_head *bh_result,
int create, int extend_disksize)
{
struct ext4_ext_path *path = NULL;
struct ext4_extent_header *eh;
struct ext4_extent newex, *ex;
- ext4_fsblk_t goal, newblock;
- int err = 0, depth, ret;
- unsigned long allocated = 0;
+ ext4_fsblk_t newblock;
+ int err = 0, depth, ret, cache_type;
+ unsigned int allocated = 0;
struct ext4_allocation_request ar;
loff_t disksize;
__clear_bit(BH_New, &bh_result->b_state);
- ext_debug("blocks %u/%lu requested for inode %u\n",
+ ext_debug("blocks %u/%u requested for inode %u\n",
iblock, max_blocks, inode->i_ino);
/* check in cache */
- goal = ext4_ext_in_cache(inode, iblock, &newex);
- if (goal) {
- if (goal == EXT4_EXT_CACHE_GAP) {
+ cache_type = ext4_ext_in_cache(inode, iblock, &newex);
+ if (cache_type) {
+ if (cache_type == EXT4_EXT_CACHE_GAP) {
if (!create) {
/*
* block isn't allocated yet and
@@ -2706,7 +2703,7 @@
goto out2;
}
/* we should allocate requested block */
- } else if (goal == EXT4_EXT_CACHE_EXTENT) {
+ } else if (cache_type == EXT4_EXT_CACHE_EXTENT) {
/* block is already allocated */
newblock = iblock
- le32_to_cpu(newex.ee_block)
@@ -2854,7 +2851,7 @@
if (!newblock)
goto out2;
ext_debug("allocate new block: goal %llu, found %llu/%lu\n",
- goal, newblock, allocated);
+ ar.goal, newblock, allocated);
/* try to insert new extent into found leaf and return */
ext4_ext_store_pblock(&newex, newblock);
@@ -2950,7 +2947,7 @@
* transaction synchronous.
*/
if (IS_SYNC(inode))
- handle->h_sync = 1;
+ ext4_handle_sync(handle);
out_stop:
up_write(&EXT4_I(inode)->i_data_sem);
@@ -3004,7 +3001,7 @@
handle_t *handle;
ext4_lblk_t block;
loff_t new_size;
- unsigned long max_blocks;
+ unsigned int max_blocks;
int ret = 0;
int ret2 = 0;
int retries = 0;
@@ -3083,7 +3080,7 @@
/*
* Callback function called for each extent to gather FIEMAP information.
*/
-int ext4_ext_fiemap_cb(struct inode *inode, struct ext4_ext_path *path,
+static int ext4_ext_fiemap_cb(struct inode *inode, struct ext4_ext_path *path,
struct ext4_ext_cache *newex, struct ext4_extent *ex,
void *data)
{
@@ -3152,7 +3149,8 @@
/* fiemap flags we can handle specified here */
#define EXT4_FIEMAP_FLAGS (FIEMAP_FLAG_SYNC|FIEMAP_FLAG_XATTR)
-int ext4_xattr_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo)
+static int ext4_xattr_fiemap(struct inode *inode,
+ struct fiemap_extent_info *fieinfo)
{
__u64 physical = 0;
__u64 length;
diff --git a/fs/ext4/file.c b/fs/ext4/file.c
index 6bd11fb..f731cb5 100644
--- a/fs/ext4/file.c
+++ b/fs/ext4/file.c
@@ -140,9 +140,6 @@
return 0;
}
-extern int ext4_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
- __u64 start, __u64 len);
-
const struct file_operations ext4_file_operations = {
.llseek = generic_file_llseek,
.read = do_sync_read,
diff --git a/fs/ext4/hash.c b/fs/ext4/hash.c
index 556ca8e..ac8f168 100644
--- a/fs/ext4/hash.c
+++ b/fs/ext4/hash.c
@@ -35,23 +35,43 @@
/* The old legacy hash */
-static __u32 dx_hack_hash(const char *name, int len)
+static __u32 dx_hack_hash_unsigned(const char *name, int len)
{
- __u32 hash0 = 0x12a3fe2d, hash1 = 0x37abe8f9;
- while (len--) {
- __u32 hash = hash1 + (hash0 ^ (*name++ * 7152373));
+ __u32 hash, hash0 = 0x12a3fe2d, hash1 = 0x37abe8f9;
+ const unsigned char *ucp = (const unsigned char *) name;
- if (hash & 0x80000000) hash -= 0x7fffffff;
+ while (len--) {
+ hash = hash1 + (hash0 ^ (((int) *ucp++) * 7152373));
+
+ if (hash & 0x80000000)
+ hash -= 0x7fffffff;
hash1 = hash0;
hash0 = hash;
}
- return (hash0 << 1);
+ return hash0 << 1;
}
-static void str2hashbuf(const char *msg, int len, __u32 *buf, int num)
+static __u32 dx_hack_hash_signed(const char *name, int len)
+{
+ __u32 hash, hash0 = 0x12a3fe2d, hash1 = 0x37abe8f9;
+ const signed char *scp = (const signed char *) name;
+
+ while (len--) {
+ hash = hash1 + (hash0 ^ (((int) *scp++) * 7152373));
+
+ if (hash & 0x80000000)
+ hash -= 0x7fffffff;
+ hash1 = hash0;
+ hash0 = hash;
+ }
+ return hash0 << 1;
+}
+
+static void str2hashbuf_signed(const char *msg, int len, __u32 *buf, int num)
{
__u32 pad, val;
int i;
+ const signed char *scp = (const signed char *) msg;
pad = (__u32)len | ((__u32)len << 8);
pad |= pad << 16;
@@ -62,7 +82,35 @@
for (i = 0; i < len; i++) {
if ((i % 4) == 0)
val = pad;
- val = msg[i] + (val << 8);
+ val = ((int) scp[i]) + (val << 8);
+ if ((i % 4) == 3) {
+ *buf++ = val;
+ val = pad;
+ num--;
+ }
+ }
+ if (--num >= 0)
+ *buf++ = val;
+ while (--num >= 0)
+ *buf++ = pad;
+}
+
+static void str2hashbuf_unsigned(const char *msg, int len, __u32 *buf, int num)
+{
+ __u32 pad, val;
+ int i;
+ const unsigned char *ucp = (const unsigned char *) msg;
+
+ pad = (__u32)len | ((__u32)len << 8);
+ pad |= pad << 16;
+
+ val = pad;
+ if (len > num*4)
+ len = num * 4;
+ for (i = 0; i < len; i++) {
+ if ((i % 4) == 0)
+ val = pad;
+ val = ((int) ucp[i]) + (val << 8);
if ((i % 4) == 3) {
*buf++ = val;
val = pad;
@@ -95,6 +143,8 @@
const char *p;
int i;
__u32 in[8], buf[4];
+ void (*str2hashbuf)(const char *, int, __u32 *, int) =
+ str2hashbuf_signed;
/* Initialize the default seed for the hash checksum functions */
buf[0] = 0x67452301;
@@ -113,13 +163,18 @@
}
switch (hinfo->hash_version) {
- case DX_HASH_LEGACY:
- hash = dx_hack_hash(name, len);
+ case DX_HASH_LEGACY_UNSIGNED:
+ hash = dx_hack_hash_unsigned(name, len);
break;
+ case DX_HASH_LEGACY:
+ hash = dx_hack_hash_signed(name, len);
+ break;
+ case DX_HASH_HALF_MD4_UNSIGNED:
+ str2hashbuf = str2hashbuf_unsigned;
case DX_HASH_HALF_MD4:
p = name;
while (len > 0) {
- str2hashbuf(p, len, in, 8);
+ (*str2hashbuf)(p, len, in, 8);
half_md4_transform(buf, in);
len -= 32;
p += 32;
@@ -127,10 +182,12 @@
minor_hash = buf[2];
hash = buf[1];
break;
+ case DX_HASH_TEA_UNSIGNED:
+ str2hashbuf = str2hashbuf_unsigned;
case DX_HASH_TEA:
p = name;
while (len > 0) {
- str2hashbuf(p, len, in, 4);
+ (*str2hashbuf)(p, len, in, 4);
TEA_transform(buf, in);
len -= 16;
p += 16;
diff --git a/fs/ext4/ialloc.c b/fs/ext4/ialloc.c
index 6e60528..4fb86a0 100644
--- a/fs/ext4/ialloc.c
+++ b/fs/ext4/ialloc.c
@@ -74,17 +74,17 @@
/* If checksum is bad mark all blocks and inodes use to prevent
* allocation, essentially implementing a per-group read-only flag. */
if (!ext4_group_desc_csum_verify(sbi, block_group, gdp)) {
- ext4_error(sb, __func__, "Checksum bad for group %lu\n",
+ ext4_error(sb, __func__, "Checksum bad for group %u",
block_group);
- gdp->bg_free_blocks_count = 0;
- gdp->bg_free_inodes_count = 0;
- gdp->bg_itable_unused = 0;
+ ext4_free_blks_set(sb, gdp, 0);
+ ext4_free_inodes_set(sb, gdp, 0);
+ ext4_itable_unused_set(sb, gdp, 0);
memset(bh->b_data, 0xff, sb->s_blocksize);
return 0;
}
memset(bh->b_data, 0, (EXT4_INODES_PER_GROUP(sb) + 7) / 8);
- mark_bitmap_end(EXT4_INODES_PER_GROUP(sb), EXT4_BLOCKS_PER_GROUP(sb),
+ mark_bitmap_end(EXT4_INODES_PER_GROUP(sb), sb->s_blocksize * 8,
bh->b_data);
return EXT4_INODES_PER_GROUP(sb);
@@ -111,29 +111,49 @@
if (unlikely(!bh)) {
ext4_error(sb, __func__,
"Cannot read inode bitmap - "
- "block_group = %lu, inode_bitmap = %llu",
+ "block_group = %u, inode_bitmap = %llu",
block_group, bitmap_blk);
return NULL;
}
- if (buffer_uptodate(bh) &&
- !(desc->bg_flags & cpu_to_le16(EXT4_BG_INODE_UNINIT)))
+ if (bitmap_uptodate(bh))
return bh;
lock_buffer(bh);
+ if (bitmap_uptodate(bh)) {
+ unlock_buffer(bh);
+ return bh;
+ }
spin_lock(sb_bgl_lock(EXT4_SB(sb), block_group));
if (desc->bg_flags & cpu_to_le16(EXT4_BG_INODE_UNINIT)) {
ext4_init_inode_bitmap(sb, bh, block_group, desc);
+ set_bitmap_uptodate(bh);
set_buffer_uptodate(bh);
- unlock_buffer(bh);
spin_unlock(sb_bgl_lock(EXT4_SB(sb), block_group));
+ unlock_buffer(bh);
return bh;
}
spin_unlock(sb_bgl_lock(EXT4_SB(sb), block_group));
+ if (buffer_uptodate(bh)) {
+ /*
+ * if not uninit if bh is uptodate,
+ * bitmap is also uptodate
+ */
+ set_bitmap_uptodate(bh);
+ unlock_buffer(bh);
+ return bh;
+ }
+ /*
+ * submit the buffer_head for read. We can
+ * safely mark the bitmap as uptodate now.
+ * We do it here so the bitmap uptodate bit
+ * get set with buffer lock held.
+ */
+ set_bitmap_uptodate(bh);
if (bh_submit_read(bh) < 0) {
put_bh(bh);
ext4_error(sb, __func__,
"Cannot read inode bitmap - "
- "block_group = %lu, inode_bitmap = %llu",
+ "block_group = %u, inode_bitmap = %llu",
block_group, bitmap_blk);
return NULL;
}
@@ -168,7 +188,7 @@
struct ext4_group_desc *gdp;
struct ext4_super_block *es;
struct ext4_sb_info *sbi;
- int fatal = 0, err;
+ int fatal = 0, err, count;
ext4_group_t flex_group;
if (atomic_read(&inode->i_count) > 1) {
@@ -190,6 +210,11 @@
ino = inode->i_ino;
ext4_debug("freeing inode %lu\n", ino);
+ trace_mark(ext4_free_inode,
+ "dev %s ino %lu mode %d uid %lu gid %lu bocks %llu",
+ sb->s_id, inode->i_ino, inode->i_mode,
+ (unsigned long) inode->i_uid, (unsigned long) inode->i_gid,
+ (unsigned long long) inode->i_blocks);
/*
* Note: we must free any quota before locking the superblock,
@@ -236,9 +261,12 @@
if (gdp) {
spin_lock(sb_bgl_lock(sbi, block_group));
- le16_add_cpu(&gdp->bg_free_inodes_count, 1);
- if (is_directory)
- le16_add_cpu(&gdp->bg_used_dirs_count, -1);
+ count = ext4_free_inodes_count(sb, gdp) + 1;
+ ext4_free_inodes_set(sb, gdp, count);
+ if (is_directory) {
+ count = ext4_used_dirs_count(sb, gdp) - 1;
+ ext4_used_dirs_set(sb, gdp, count);
+ }
gdp->bg_checksum = ext4_group_desc_csum(sbi,
block_group, gdp);
spin_unlock(sb_bgl_lock(sbi, block_group));
@@ -253,12 +281,12 @@
spin_unlock(sb_bgl_lock(sbi, flex_group));
}
}
- BUFFER_TRACE(bh2, "call ext4_journal_dirty_metadata");
- err = ext4_journal_dirty_metadata(handle, bh2);
+ BUFFER_TRACE(bh2, "call ext4_handle_dirty_metadata");
+ err = ext4_handle_dirty_metadata(handle, NULL, bh2);
if (!fatal) fatal = err;
}
- BUFFER_TRACE(bitmap_bh, "call ext4_journal_dirty_metadata");
- err = ext4_journal_dirty_metadata(handle, bitmap_bh);
+ BUFFER_TRACE(bitmap_bh, "call ext4_handle_dirty_metadata");
+ err = ext4_handle_dirty_metadata(handle, NULL, bitmap_bh);
if (!fatal)
fatal = err;
sb->s_dirt = 1;
@@ -291,13 +319,13 @@
for (group = 0; group < ngroups; group++) {
desc = ext4_get_group_desc(sb, group, NULL);
- if (!desc || !desc->bg_free_inodes_count)
+ if (!desc || !ext4_free_inodes_count(sb, desc))
continue;
- if (le16_to_cpu(desc->bg_free_inodes_count) < avefreei)
+ if (ext4_free_inodes_count(sb, desc) < avefreei)
continue;
if (!best_desc ||
- (le16_to_cpu(desc->bg_free_blocks_count) >
- le16_to_cpu(best_desc->bg_free_blocks_count))) {
+ (ext4_free_blks_count(sb, desc) >
+ ext4_free_blks_count(sb, best_desc))) {
*best_group = group;
best_desc = desc;
ret = 0;
@@ -369,7 +397,7 @@
for (i = best_flex * flex_size; i < ngroups &&
i < (best_flex + 1) * flex_size; i++) {
desc = ext4_get_group_desc(sb, i, &bh);
- if (le16_to_cpu(desc->bg_free_inodes_count)) {
+ if (ext4_free_inodes_count(sb, desc)) {
*best_group = i;
goto out;
}
@@ -443,17 +471,17 @@
for (i = 0; i < ngroups; i++) {
grp = (parent_group + i) % ngroups;
desc = ext4_get_group_desc(sb, grp, NULL);
- if (!desc || !desc->bg_free_inodes_count)
+ if (!desc || !ext4_free_inodes_count(sb, desc))
continue;
- if (le16_to_cpu(desc->bg_used_dirs_count) >= best_ndir)
+ if (ext4_used_dirs_count(sb, desc) >= best_ndir)
continue;
- if (le16_to_cpu(desc->bg_free_inodes_count) < avefreei)
+ if (ext4_free_inodes_count(sb, desc) < avefreei)
continue;
- if (le16_to_cpu(desc->bg_free_blocks_count) < avefreeb)
+ if (ext4_free_blks_count(sb, desc) < avefreeb)
continue;
*group = grp;
ret = 0;
- best_ndir = le16_to_cpu(desc->bg_used_dirs_count);
+ best_ndir = ext4_used_dirs_count(sb, desc);
}
if (ret == 0)
return ret;
@@ -479,13 +507,13 @@
for (i = 0; i < ngroups; i++) {
*group = (parent_group + i) % ngroups;
desc = ext4_get_group_desc(sb, *group, NULL);
- if (!desc || !desc->bg_free_inodes_count)
+ if (!desc || !ext4_free_inodes_count(sb, desc))
continue;
- if (le16_to_cpu(desc->bg_used_dirs_count) >= max_dirs)
+ if (ext4_used_dirs_count(sb, desc) >= max_dirs)
continue;
- if (le16_to_cpu(desc->bg_free_inodes_count) < min_inodes)
+ if (ext4_free_inodes_count(sb, desc) < min_inodes)
continue;
- if (le16_to_cpu(desc->bg_free_blocks_count) < min_blocks)
+ if (ext4_free_blks_count(sb, desc) < min_blocks)
continue;
return 0;
}
@@ -494,8 +522,8 @@
for (i = 0; i < ngroups; i++) {
*group = (parent_group + i) % ngroups;
desc = ext4_get_group_desc(sb, *group, NULL);
- if (desc && desc->bg_free_inodes_count &&
- le16_to_cpu(desc->bg_free_inodes_count) >= avefreei)
+ if (desc && ext4_free_inodes_count(sb, desc) &&
+ ext4_free_inodes_count(sb, desc) >= avefreei)
return 0;
}
@@ -524,8 +552,8 @@
*/
*group = parent_group;
desc = ext4_get_group_desc(sb, *group, NULL);
- if (desc && le16_to_cpu(desc->bg_free_inodes_count) &&
- le16_to_cpu(desc->bg_free_blocks_count))
+ if (desc && ext4_free_inodes_count(sb, desc) &&
+ ext4_free_blks_count(sb, desc))
return 0;
/*
@@ -548,8 +576,8 @@
if (*group >= ngroups)
*group -= ngroups;
desc = ext4_get_group_desc(sb, *group, NULL);
- if (desc && le16_to_cpu(desc->bg_free_inodes_count) &&
- le16_to_cpu(desc->bg_free_blocks_count))
+ if (desc && ext4_free_inodes_count(sb, desc) &&
+ ext4_free_blks_count(sb, desc))
return 0;
}
@@ -562,7 +590,7 @@
if (++*group >= ngroups)
*group = 0;
desc = ext4_get_group_desc(sb, *group, NULL);
- if (desc && le16_to_cpu(desc->bg_free_inodes_count))
+ if (desc && ext4_free_inodes_count(sb, desc))
return 0;
}
@@ -570,6 +598,79 @@
}
/*
+ * claim the inode from the inode bitmap. If the group
+ * is uninit we need to take the groups's sb_bgl_lock
+ * and clear the uninit flag. The inode bitmap update
+ * and group desc uninit flag clear should be done
+ * after holding sb_bgl_lock so that ext4_read_inode_bitmap
+ * doesn't race with the ext4_claim_inode
+ */
+static int ext4_claim_inode(struct super_block *sb,
+ struct buffer_head *inode_bitmap_bh,
+ unsigned long ino, ext4_group_t group, int mode)
+{
+ int free = 0, retval = 0, count;
+ struct ext4_sb_info *sbi = EXT4_SB(sb);
+ struct ext4_group_desc *gdp = ext4_get_group_desc(sb, group, NULL);
+
+ spin_lock(sb_bgl_lock(sbi, group));
+ if (ext4_set_bit(ino, inode_bitmap_bh->b_data)) {
+ /* not a free inode */
+ retval = 1;
+ goto err_ret;
+ }
+ ino++;
+ if ((group == 0 && ino < EXT4_FIRST_INO(sb)) ||
+ ino > EXT4_INODES_PER_GROUP(sb)) {
+ spin_unlock(sb_bgl_lock(sbi, group));
+ ext4_error(sb, __func__,
+ "reserved inode or inode > inodes count - "
+ "block_group = %u, inode=%lu", group,
+ ino + group * EXT4_INODES_PER_GROUP(sb));
+ return 1;
+ }
+ /* If we didn't allocate from within the initialized part of the inode
+ * table then we need to initialize up to this inode. */
+ if (EXT4_HAS_RO_COMPAT_FEATURE(sb, EXT4_FEATURE_RO_COMPAT_GDT_CSUM)) {
+
+ if (gdp->bg_flags & cpu_to_le16(EXT4_BG_INODE_UNINIT)) {
+ gdp->bg_flags &= cpu_to_le16(~EXT4_BG_INODE_UNINIT);
+ /* When marking the block group with
+ * ~EXT4_BG_INODE_UNINIT we don't want to depend
+ * on the value of bg_itable_unused even though
+ * mke2fs could have initialized the same for us.
+ * Instead we calculated the value below
+ */
+
+ free = 0;
+ } else {
+ free = EXT4_INODES_PER_GROUP(sb) -
+ ext4_itable_unused_count(sb, gdp);
+ }
+
+ /*
+ * Check the relative inode number against the last used
+ * relative inode number in this group. if it is greater
+ * we need to update the bg_itable_unused count
+ *
+ */
+ if (ino > free)
+ ext4_itable_unused_set(sb, gdp,
+ (EXT4_INODES_PER_GROUP(sb) - ino));
+ }
+ count = ext4_free_inodes_count(sb, gdp) - 1;
+ ext4_free_inodes_set(sb, gdp, count);
+ if (S_ISDIR(mode)) {
+ count = ext4_used_dirs_count(sb, gdp) + 1;
+ ext4_used_dirs_set(sb, gdp, count);
+ }
+ gdp->bg_checksum = ext4_group_desc_csum(sbi, group, gdp);
+err_ret:
+ spin_unlock(sb_bgl_lock(sbi, group));
+ return retval;
+}
+
+/*
* There are two policies for allocating an inode. If the new inode is
* a directory, then a forward search is made for a block group with both
* free space and a low directory-to-inode ratio; if that fails, then of
@@ -582,8 +683,8 @@
struct inode *ext4_new_inode(handle_t *handle, struct inode *dir, int mode)
{
struct super_block *sb;
- struct buffer_head *bitmap_bh = NULL;
- struct buffer_head *bh2;
+ struct buffer_head *inode_bitmap_bh = NULL;
+ struct buffer_head *group_desc_bh;
ext4_group_t group = 0;
unsigned long ino = 0;
struct inode *inode;
@@ -602,6 +703,8 @@
return ERR_PTR(-EPERM);
sb = dir->i_sb;
+ trace_mark(ext4_request_inode, "dev %s dir %lu mode %d", sb->s_id,
+ dir->i_ino, mode);
inode = new_inode(sb);
if (!inode)
return ERR_PTR(-ENOMEM);
@@ -631,40 +734,52 @@
for (i = 0; i < sbi->s_groups_count; i++) {
err = -EIO;
- gdp = ext4_get_group_desc(sb, group, &bh2);
+ gdp = ext4_get_group_desc(sb, group, &group_desc_bh);
if (!gdp)
goto fail;
- brelse(bitmap_bh);
- bitmap_bh = ext4_read_inode_bitmap(sb, group);
- if (!bitmap_bh)
+ brelse(inode_bitmap_bh);
+ inode_bitmap_bh = ext4_read_inode_bitmap(sb, group);
+ if (!inode_bitmap_bh)
goto fail;
ino = 0;
repeat_in_this_group:
ino = ext4_find_next_zero_bit((unsigned long *)
- bitmap_bh->b_data, EXT4_INODES_PER_GROUP(sb), ino);
+ inode_bitmap_bh->b_data,
+ EXT4_INODES_PER_GROUP(sb), ino);
+
if (ino < EXT4_INODES_PER_GROUP(sb)) {
- BUFFER_TRACE(bitmap_bh, "get_write_access");
- err = ext4_journal_get_write_access(handle, bitmap_bh);
+ BUFFER_TRACE(inode_bitmap_bh, "get_write_access");
+ err = ext4_journal_get_write_access(handle,
+ inode_bitmap_bh);
if (err)
goto fail;
- if (!ext4_set_bit_atomic(sb_bgl_lock(sbi, group),
- ino, bitmap_bh->b_data)) {
+ BUFFER_TRACE(group_desc_bh, "get_write_access");
+ err = ext4_journal_get_write_access(handle,
+ group_desc_bh);
+ if (err)
+ goto fail;
+ if (!ext4_claim_inode(sb, inode_bitmap_bh,
+ ino, group, mode)) {
/* we won it */
- BUFFER_TRACE(bitmap_bh,
- "call ext4_journal_dirty_metadata");
- err = ext4_journal_dirty_metadata(handle,
- bitmap_bh);
+ BUFFER_TRACE(inode_bitmap_bh,
+ "call ext4_handle_dirty_metadata");
+ err = ext4_handle_dirty_metadata(handle,
+ inode,
+ inode_bitmap_bh);
if (err)
goto fail;
+ /* zero bit is inode number 1*/
+ ino++;
goto got;
}
/* we lost it */
- jbd2_journal_release_buffer(handle, bitmap_bh);
+ ext4_handle_release_buffer(handle, inode_bitmap_bh);
+ ext4_handle_release_buffer(handle, group_desc_bh);
if (++ino < EXT4_INODES_PER_GROUP(sb))
goto repeat_in_this_group;
@@ -684,30 +799,16 @@
goto out;
got:
- ino++;
- if ((group == 0 && ino < EXT4_FIRST_INO(sb)) ||
- ino > EXT4_INODES_PER_GROUP(sb)) {
- ext4_error(sb, __func__,
- "reserved inode or inode > inodes count - "
- "block_group = %lu, inode=%lu", group,
- ino + group * EXT4_INODES_PER_GROUP(sb));
- err = -EIO;
- goto fail;
- }
-
- BUFFER_TRACE(bh2, "get_write_access");
- err = ext4_journal_get_write_access(handle, bh2);
- if (err) goto fail;
-
/* We may have to initialize the block bitmap if it isn't already */
if (EXT4_HAS_RO_COMPAT_FEATURE(sb, EXT4_FEATURE_RO_COMPAT_GDT_CSUM) &&
gdp->bg_flags & cpu_to_le16(EXT4_BG_BLOCK_UNINIT)) {
- struct buffer_head *block_bh = ext4_read_block_bitmap(sb, group);
+ struct buffer_head *block_bitmap_bh;
- BUFFER_TRACE(block_bh, "get block bitmap access");
- err = ext4_journal_get_write_access(handle, block_bh);
+ block_bitmap_bh = ext4_read_block_bitmap(sb, group);
+ BUFFER_TRACE(block_bitmap_bh, "get block bitmap access");
+ err = ext4_journal_get_write_access(handle, block_bitmap_bh);
if (err) {
- brelse(block_bh);
+ brelse(block_bitmap_bh);
goto fail;
}
@@ -715,9 +816,9 @@
spin_lock(sb_bgl_lock(sbi, group));
/* recheck and clear flag under lock if we still need to */
if (gdp->bg_flags & cpu_to_le16(EXT4_BG_BLOCK_UNINIT)) {
- gdp->bg_flags &= cpu_to_le16(~EXT4_BG_BLOCK_UNINIT);
free = ext4_free_blocks_after_init(sb, group, gdp);
- gdp->bg_free_blocks_count = cpu_to_le16(free);
+ gdp->bg_flags &= cpu_to_le16(~EXT4_BG_BLOCK_UNINIT);
+ ext4_free_blks_set(sb, gdp, free);
gdp->bg_checksum = ext4_group_desc_csum(sbi, group,
gdp);
}
@@ -725,55 +826,19 @@
/* Don't need to dirty bitmap block if we didn't change it */
if (free) {
- BUFFER_TRACE(block_bh, "dirty block bitmap");
- err = ext4_journal_dirty_metadata(handle, block_bh);
+ BUFFER_TRACE(block_bitmap_bh, "dirty block bitmap");
+ err = ext4_handle_dirty_metadata(handle,
+ NULL, block_bitmap_bh);
}
- brelse(block_bh);
+ brelse(block_bitmap_bh);
if (err)
goto fail;
}
-
- spin_lock(sb_bgl_lock(sbi, group));
- /* If we didn't allocate from within the initialized part of the inode
- * table then we need to initialize up to this inode. */
- if (EXT4_HAS_RO_COMPAT_FEATURE(sb, EXT4_FEATURE_RO_COMPAT_GDT_CSUM)) {
- if (gdp->bg_flags & cpu_to_le16(EXT4_BG_INODE_UNINIT)) {
- gdp->bg_flags &= cpu_to_le16(~EXT4_BG_INODE_UNINIT);
-
- /* When marking the block group with
- * ~EXT4_BG_INODE_UNINIT we don't want to depend
- * on the value of bg_itable_unused even though
- * mke2fs could have initialized the same for us.
- * Instead we calculated the value below
- */
-
- free = 0;
- } else {
- free = EXT4_INODES_PER_GROUP(sb) -
- le16_to_cpu(gdp->bg_itable_unused);
- }
-
- /*
- * Check the relative inode number against the last used
- * relative inode number in this group. if it is greater
- * we need to update the bg_itable_unused count
- *
- */
- if (ino > free)
- gdp->bg_itable_unused =
- cpu_to_le16(EXT4_INODES_PER_GROUP(sb) - ino);
- }
-
- le16_add_cpu(&gdp->bg_free_inodes_count, -1);
- if (S_ISDIR(mode)) {
- le16_add_cpu(&gdp->bg_used_dirs_count, 1);
- }
- gdp->bg_checksum = ext4_group_desc_csum(sbi, group, gdp);
- spin_unlock(sb_bgl_lock(sbi, group));
- BUFFER_TRACE(bh2, "call ext4_journal_dirty_metadata");
- err = ext4_journal_dirty_metadata(handle, bh2);
- if (err) goto fail;
+ BUFFER_TRACE(group_desc_bh, "call ext4_handle_dirty_metadata");
+ err = ext4_handle_dirty_metadata(handle, NULL, group_desc_bh);
+ if (err)
+ goto fail;
percpu_counter_dec(&sbi->s_freeinodes_counter);
if (S_ISDIR(mode))
@@ -825,7 +890,7 @@
ext4_set_inode_flags(inode);
if (IS_DIRSYNC(inode))
- handle->h_sync = 1;
+ ext4_handle_sync(handle);
if (insert_inode_locked(inode) < 0) {
err = -EINVAL;
goto fail_drop;
@@ -852,7 +917,7 @@
if (err)
goto fail_free_drop;
- if (test_opt(sb, EXTENTS)) {
+ if (EXT4_HAS_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_EXTENTS)) {
/* set extent flag only for directory, file and normal symlink*/
if (S_ISDIR(mode) || S_ISREG(mode) || S_ISLNK(mode)) {
EXT4_I(inode)->i_flags |= EXT4_EXTENTS_FL;
@@ -867,6 +932,8 @@
}
ext4_debug("allocating inode %lu\n", inode->i_ino);
+ trace_mark(ext4_allocate_inode, "dev %s ino %lu dir %lu mode %d",
+ sb->s_id, inode->i_ino, dir->i_ino, mode);
goto really_out;
fail:
ext4_std_error(sb, err);
@@ -874,7 +941,7 @@
iput(inode);
ret = ERR_PTR(err);
really_out:
- brelse(bitmap_bh);
+ brelse(inode_bitmap_bh);
return ret;
fail_free_drop:
@@ -886,7 +953,7 @@
inode->i_nlink = 0;
unlock_new_inode(inode);
iput(inode);
- brelse(bitmap_bh);
+ brelse(inode_bitmap_bh);
return ERR_PTR(err);
}
@@ -985,7 +1052,7 @@
gdp = ext4_get_group_desc(sb, i, NULL);
if (!gdp)
continue;
- desc_count += le16_to_cpu(gdp->bg_free_inodes_count);
+ desc_count += ext4_free_inodes_count(sb, gdp);
brelse(bitmap_bh);
bitmap_bh = ext4_read_inode_bitmap(sb, i);
if (!bitmap_bh)
@@ -993,7 +1060,7 @@
x = ext4_count_free(bitmap_bh, EXT4_INODES_PER_GROUP(sb) / 8);
printk(KERN_DEBUG "group %lu: stored = %d, counted = %lu\n",
- i, le16_to_cpu(gdp->bg_free_inodes_count), x);
+ i, ext4_free_inodes_count(sb, gdp), x);
bitmap_count += x;
}
brelse(bitmap_bh);
@@ -1007,7 +1074,7 @@
gdp = ext4_get_group_desc(sb, i, NULL);
if (!gdp)
continue;
- desc_count += le16_to_cpu(gdp->bg_free_inodes_count);
+ desc_count += ext4_free_inodes_count(sb, gdp);
cond_resched();
}
return desc_count;
@@ -1024,8 +1091,7 @@
struct ext4_group_desc *gdp = ext4_get_group_desc(sb, i, NULL);
if (!gdp)
continue;
- count += le16_to_cpu(gdp->bg_used_dirs_count);
+ count += ext4_used_dirs_count(sb, gdp);
}
return count;
}
-
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 98d3fe7..a6444ce 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -72,12 +72,17 @@
* "bh" may be NULL: a metadata block may have been freed from memory
* but there may still be a record of it in the journal, and that record
* still needs to be revoked.
+ *
+ * If the handle isn't valid we're not journaling so there's nothing to do.
*/
int ext4_forget(handle_t *handle, int is_metadata, struct inode *inode,
struct buffer_head *bh, ext4_fsblk_t blocknr)
{
int err;
+ if (!ext4_handle_valid(handle))
+ return 0;
+
might_sleep();
BUFFER_TRACE(bh, "enter");
@@ -170,7 +175,9 @@
*/
static int try_to_extend_transaction(handle_t *handle, struct inode *inode)
{
- if (handle->h_buffer_credits > EXT4_RESERVE_TRANS_BLOCKS)
+ if (!ext4_handle_valid(handle))
+ return 0;
+ if (ext4_handle_has_enough_credits(handle, EXT4_RESERVE_TRANS_BLOCKS+1))
return 0;
if (!ext4_journal_extend(handle, blocks_for_truncate(inode)))
return 0;
@@ -184,6 +191,7 @@
*/
static int ext4_journal_test_restart(handle_t *handle, struct inode *inode)
{
+ BUG_ON(EXT4_JOURNAL(inode) == NULL);
jbd_debug(2, "restarting handle %p\n", handle);
return ext4_journal_restart(handle, blocks_for_truncate(inode));
}
@@ -216,7 +224,7 @@
}
if (IS_SYNC(inode))
- handle->h_sync = 1;
+ ext4_handle_sync(handle);
inode->i_size = 0;
err = ext4_mark_inode_dirty(handle, inode);
if (err) {
@@ -233,7 +241,7 @@
* enough credits left in the handle to remove the inode from
* the orphan list and set the dtime field.
*/
- if (handle->h_buffer_credits < 3) {
+ if (!ext4_handle_has_enough_credits(handle, 3)) {
err = ext4_journal_extend(handle, 3);
if (err > 0)
err = ext4_journal_restart(handle, 3);
@@ -506,10 +514,10 @@
* return the total number of blocks to be allocate, including the
* direct and indirect blocks.
*/
-static int ext4_blks_to_allocate(Indirect *branch, int k, unsigned long blks,
+static int ext4_blks_to_allocate(Indirect *branch, int k, unsigned int blks,
int blocks_to_boundary)
{
- unsigned long count = 0;
+ unsigned int count = 0;
/*
* Simple case, [t,d]Indirect block(s) has not allocated yet
@@ -547,6 +555,7 @@
int indirect_blks, int blks,
ext4_fsblk_t new_blocks[4], int *err)
{
+ struct ext4_allocation_request ar;
int target, i;
unsigned long count = 0, blk_allocated = 0;
int index = 0;
@@ -595,10 +604,17 @@
if (!target)
goto allocated;
/* Now allocate data blocks */
- count = target;
- /* allocating blocks for data blocks */
- current_block = ext4_new_blocks(handle, inode, iblock,
- goal, &count, err);
+ memset(&ar, 0, sizeof(ar));
+ ar.inode = inode;
+ ar.goal = goal;
+ ar.len = target;
+ ar.logical = iblock;
+ if (S_ISREG(inode->i_mode))
+ /* enable in-core preallocation only for regular files */
+ ar.flags = EXT4_MB_HINT_DATA;
+
+ current_block = ext4_mb_new_blocks(handle, &ar, err);
+
if (*err && (target == blks)) {
/*
* if the allocation failed and we didn't allocate
@@ -614,7 +630,7 @@
*/
new_blocks[index] = current_block;
}
- blk_allocated += count;
+ blk_allocated += ar.len;
}
allocated:
/* total number of blocks allocated for direct blocks */
@@ -709,8 +725,8 @@
set_buffer_uptodate(bh);
unlock_buffer(bh);
- BUFFER_TRACE(bh, "call ext4_journal_dirty_metadata");
- err = ext4_journal_dirty_metadata(handle, bh);
+ BUFFER_TRACE(bh, "call ext4_handle_dirty_metadata");
+ err = ext4_handle_dirty_metadata(handle, inode, bh);
if (err)
goto failed;
}
@@ -792,8 +808,8 @@
* generic_commit_write->__mark_inode_dirty->ext4_dirty_inode.
*/
jbd_debug(5, "splicing indirect only\n");
- BUFFER_TRACE(where->bh, "call ext4_journal_dirty_metadata");
- err = ext4_journal_dirty_metadata(handle, where->bh);
+ BUFFER_TRACE(where->bh, "call ext4_handle_dirty_metadata");
+ err = ext4_handle_dirty_metadata(handle, inode, where->bh);
if (err)
goto err_out;
} else {
@@ -840,10 +856,10 @@
* down_read(&EXT4_I(inode)->i_data_sem) if not allocating file system block
* (ie, create is zero). Otherwise down_write(&EXT4_I(inode)->i_data_sem)
*/
-int ext4_get_blocks_handle(handle_t *handle, struct inode *inode,
- ext4_lblk_t iblock, unsigned long maxblocks,
- struct buffer_head *bh_result,
- int create, int extend_disksize)
+static int ext4_get_blocks_handle(handle_t *handle, struct inode *inode,
+ ext4_lblk_t iblock, unsigned int maxblocks,
+ struct buffer_head *bh_result,
+ int create, int extend_disksize)
{
int err = -EIO;
ext4_lblk_t offsets[4];
@@ -1045,7 +1061,7 @@
* It returns the error in case of allocation failure.
*/
int ext4_get_blocks_wrap(handle_t *handle, struct inode *inode, sector_t block,
- unsigned long max_blocks, struct buffer_head *bh,
+ unsigned int max_blocks, struct buffer_head *bh,
int create, int extend_disksize, int flag)
{
int retval;
@@ -1221,8 +1237,8 @@
set_buffer_uptodate(bh);
}
unlock_buffer(bh);
- BUFFER_TRACE(bh, "call ext4_journal_dirty_metadata");
- err = ext4_journal_dirty_metadata(handle, bh);
+ BUFFER_TRACE(bh, "call ext4_handle_dirty_metadata");
+ err = ext4_handle_dirty_metadata(handle, inode, bh);
if (!fatal)
fatal = err;
} else {
@@ -1335,6 +1351,10 @@
pgoff_t index;
unsigned from, to;
+ trace_mark(ext4_write_begin,
+ "dev %s ino %lu pos %llu len %u flags %u",
+ inode->i_sb->s_id, inode->i_ino,
+ (unsigned long long) pos, len, flags);
index = pos >> PAGE_CACHE_SHIFT;
from = pos & (PAGE_CACHE_SIZE - 1);
to = from + len;
@@ -1387,7 +1407,7 @@
if (!buffer_mapped(bh) || buffer_freed(bh))
return 0;
set_buffer_uptodate(bh);
- return ext4_journal_dirty_metadata(handle, bh);
+ return ext4_handle_dirty_metadata(handle, NULL, bh);
}
/*
@@ -1406,6 +1426,10 @@
struct inode *inode = mapping->host;
int ret = 0, ret2;
+ trace_mark(ext4_ordered_write_end,
+ "dev %s ino %lu pos %llu len %u copied %u",
+ inode->i_sb->s_id, inode->i_ino,
+ (unsigned long long) pos, len, copied);
ret = ext4_jbd2_file_inode(handle, inode);
if (ret == 0) {
@@ -1444,6 +1468,10 @@
int ret = 0, ret2;
loff_t new_i_size;
+ trace_mark(ext4_writeback_write_end,
+ "dev %s ino %lu pos %llu len %u copied %u",
+ inode->i_sb->s_id, inode->i_ino,
+ (unsigned long long) pos, len, copied);
new_i_size = pos + copied;
if (new_i_size > EXT4_I(inode)->i_disksize) {
ext4_update_i_disksize(inode, new_i_size);
@@ -1479,6 +1507,10 @@
unsigned from, to;
loff_t new_i_size;
+ trace_mark(ext4_journalled_write_end,
+ "dev %s ino %lu pos %llu len %u copied %u",
+ inode->i_sb->s_id, inode->i_ino,
+ (unsigned long long) pos, len, copied);
from = pos & (PAGE_CACHE_SIZE - 1);
to = from + len;
@@ -1625,7 +1657,7 @@
get_block_t *get_block;
struct writeback_control *wbc;
int io_done;
- long pages_written;
+ int pages_written;
int retval;
};
@@ -1645,35 +1677,39 @@
*/
static int mpage_da_submit_io(struct mpage_da_data *mpd)
{
- struct address_space *mapping = mpd->inode->i_mapping;
- int ret = 0, err, nr_pages, i;
- unsigned long index, end;
- struct pagevec pvec;
long pages_skipped;
+ struct pagevec pvec;
+ unsigned long index, end;
+ int ret = 0, err, nr_pages, i;
+ struct inode *inode = mpd->inode;
+ struct address_space *mapping = inode->i_mapping;
BUG_ON(mpd->next_page <= mpd->first_page);
- pagevec_init(&pvec, 0);
+ /*
+ * We need to start from the first_page to the next_page - 1
+ * to make sure we also write the mapped dirty buffer_heads.
+ * If we look at mpd->lbh.b_blocknr we would only be looking
+ * at the currently mapped buffer_heads.
+ */
index = mpd->first_page;
end = mpd->next_page - 1;
+ pagevec_init(&pvec, 0);
while (index <= end) {
- /*
- * We can use PAGECACHE_TAG_DIRTY lookup here because
- * even though we have cleared the dirty flag on the page
- * We still keep the page in the radix tree with tag
- * PAGECACHE_TAG_DIRTY. See clear_page_dirty_for_io.
- * The PAGECACHE_TAG_DIRTY is cleared in set_page_writeback
- * which is called via the below writepage callback.
- */
- nr_pages = pagevec_lookup_tag(&pvec, mapping, &index,
- PAGECACHE_TAG_DIRTY,
- min(end - index,
- (pgoff_t)PAGEVEC_SIZE-1) + 1);
+ nr_pages = pagevec_lookup(&pvec, mapping, index, PAGEVEC_SIZE);
if (nr_pages == 0)
break;
for (i = 0; i < nr_pages; i++) {
struct page *page = pvec.pages[i];
+ index = page->index;
+ if (index > end)
+ break;
+ index++;
+
+ BUG_ON(!PageLocked(page));
+ BUG_ON(PageWriteback(page));
+
pages_skipped = mpd->wbc->pages_skipped;
err = mapping->a_ops->writepage(page, mpd->wbc);
if (!err && (pages_skipped == mpd->wbc->pages_skipped))
@@ -1831,13 +1867,13 @@
ext4_count_free_blocks(inode->i_sb));
printk(KERN_EMERG "Free/Dirty block details\n");
printk(KERN_EMERG "free_blocks=%lld\n",
- percpu_counter_sum(&sbi->s_freeblocks_counter));
+ (long long)percpu_counter_sum(&sbi->s_freeblocks_counter));
printk(KERN_EMERG "dirty_blocks=%lld\n",
- percpu_counter_sum(&sbi->s_dirtyblocks_counter));
+ (long long)percpu_counter_sum(&sbi->s_dirtyblocks_counter));
printk(KERN_EMERG "Block reservation details\n");
- printk(KERN_EMERG "i_reserved_data_blocks=%lu\n",
+ printk(KERN_EMERG "i_reserved_data_blocks=%u\n",
EXT4_I(inode)->i_reserved_data_blocks);
- printk(KERN_EMERG "i_reserved_meta_blocks=%lu\n",
+ printk(KERN_EMERG "i_reserved_meta_blocks=%u\n",
EXT4_I(inode)->i_reserved_meta_blocks);
return;
}
@@ -2087,11 +2123,29 @@
bh = head;
do {
BUG_ON(buffer_locked(bh));
+ /*
+ * We need to try to allocate
+ * unmapped blocks in the same page.
+ * Otherwise we won't make progress
+ * with the page in ext4_da_writepage
+ */
if (buffer_dirty(bh) &&
(!buffer_mapped(bh) || buffer_delay(bh))) {
mpage_add_bh_to_extent(mpd, logical, bh);
if (mpd->io_done)
return MPAGE_DA_EXTENT_TAIL;
+ } else if (buffer_dirty(bh) && (buffer_mapped(bh))) {
+ /*
+ * mapped dirty buffer. We need to update
+ * the b_state because we look at
+ * b_state in mpage_da_map_blocks. We don't
+ * update b_size because if we find an
+ * unmapped buffer_head later we need to
+ * use the b_state flag of that buffer_head.
+ */
+ if (mpd->lbh.b_size == 0)
+ mpd->lbh.b_state =
+ bh->b_state & BH_FLAGS;
}
logical++;
} while ((bh = bh->b_this_page) != head);
@@ -2269,10 +2323,13 @@
{
int ret = 0;
loff_t size;
- unsigned long len;
+ unsigned int len;
struct buffer_head *page_bufs;
struct inode *inode = page->mapping->host;
+ trace_mark(ext4_da_writepage,
+ "dev %s ino %lu page_index %lu",
+ inode->i_sb->s_id, inode->i_ino, page->index);
size = i_size_read(inode);
if (page->index == size >> PAGE_CACHE_SHIFT)
len = size & ~PAGE_CACHE_MASK;
@@ -2378,10 +2435,25 @@
struct mpage_da_data mpd;
struct inode *inode = mapping->host;
int no_nrwrite_index_update;
- long pages_written = 0, pages_skipped;
+ int pages_written = 0;
+ long pages_skipped;
int needed_blocks, ret = 0, nr_to_writebump = 0;
struct ext4_sb_info *sbi = EXT4_SB(mapping->host->i_sb);
+ trace_mark(ext4_da_writepages,
+ "dev %s ino %lu nr_t_write %ld "
+ "pages_skipped %ld range_start %llu "
+ "range_end %llu nonblocking %d "
+ "for_kupdate %d for_reclaim %d "
+ "for_writepages %d range_cyclic %d",
+ inode->i_sb->s_id, inode->i_ino,
+ wbc->nr_to_write, wbc->pages_skipped,
+ (unsigned long long) wbc->range_start,
+ (unsigned long long) wbc->range_end,
+ wbc->nonblocking, wbc->for_kupdate,
+ wbc->for_reclaim, wbc->for_writepages,
+ wbc->range_cyclic);
+
/*
* No pages to write? This is mainly a kludge to avoid starting
* a transaction for special inodes like journal inode on last iput()
@@ -2389,6 +2461,20 @@
*/
if (!mapping->nrpages || !mapping_tagged(mapping, PAGECACHE_TAG_DIRTY))
return 0;
+
+ /*
+ * If the filesystem has aborted, it is read-only, so return
+ * right away instead of dumping stack traces later on that
+ * will obscure the real source of the problem. We test
+ * EXT4_MOUNT_ABORT instead of sb->s_flag's MS_RDONLY because
+ * the latter could be true if the filesystem is mounted
+ * read-only, and in that case, ext4_da_writepages should
+ * *never* be called, so if that ever happens, we would want
+ * the stack trace.
+ */
+ if (unlikely(sbi->s_mount_opt & EXT4_MOUNT_ABORT))
+ return -EROFS;
+
/*
* Make sure nr_to_write is >= sbi->s_mb_stream_request
* This make sure small files blocks are allocated in
@@ -2433,7 +2519,7 @@
handle = ext4_journal_start(inode, needed_blocks);
if (IS_ERR(handle)) {
ret = PTR_ERR(handle);
- printk(KERN_EMERG "%s: jbd2_start: "
+ printk(KERN_CRIT "%s: jbd2_start: "
"%ld pages, ino %lu; err %d\n", __func__,
wbc->nr_to_write, inode->i_ino, ret);
dump_stack();
@@ -2486,6 +2572,14 @@
if (!no_nrwrite_index_update)
wbc->no_nrwrite_index_update = 0;
wbc->nr_to_write -= nr_to_writebump;
+ trace_mark(ext4_da_writepage_result,
+ "dev %s ino %lu ret %d pages_written %d "
+ "pages_skipped %ld congestion %d "
+ "more_io %d no_nrwrite_index_update %d",
+ inode->i_sb->s_id, inode->i_ino, ret,
+ pages_written, wbc->pages_skipped,
+ wbc->encountered_congestion, wbc->more_io,
+ wbc->no_nrwrite_index_update);
return ret;
}
@@ -2537,6 +2631,11 @@
len, flags, pagep, fsdata);
}
*fsdata = (void *)0;
+
+ trace_mark(ext4_da_write_begin,
+ "dev %s ino %lu pos %llu len %u flags %u",
+ inode->i_sb->s_id, inode->i_ino,
+ (unsigned long long) pos, len, flags);
retry:
/*
* With delayed allocation, we don't log the i_disksize update
@@ -2626,6 +2725,10 @@
}
}
+ trace_mark(ext4_da_write_end,
+ "dev %s ino %lu pos %llu len %u copied %u",
+ inode->i_sb->s_id, inode->i_ino,
+ (unsigned long long) pos, len, copied);
start = pos & (PAGE_CACHE_SIZE - 1);
end = start + copied - 1;
@@ -2718,7 +2821,10 @@
filemap_write_and_wait(mapping);
}
- if (EXT4_I(inode)->i_state & EXT4_STATE_JDATA) {
+ BUG_ON(!EXT4_JOURNAL(inode) &&
+ EXT4_I(inode)->i_state & EXT4_STATE_JDATA);
+
+ if (EXT4_JOURNAL(inode) && EXT4_I(inode)->i_state & EXT4_STATE_JDATA) {
/*
* This is a REALLY heavyweight approach, but the use of
* bmap on dirty files is expected to be extremely rare:
@@ -2836,6 +2942,9 @@
loff_t size = i_size_read(inode);
loff_t len;
+ trace_mark(ext4_normal_writepage,
+ "dev %s ino %lu page_index %lu",
+ inode->i_sb->s_id, inode->i_ino, page->index);
J_ASSERT(PageLocked(page));
if (page->index == size >> PAGE_CACHE_SHIFT)
len = size & ~PAGE_CACHE_MASK;
@@ -2921,6 +3030,9 @@
loff_t size = i_size_read(inode);
loff_t len;
+ trace_mark(ext4_journalled_writepage,
+ "dev %s ino %lu page_index %lu",
+ inode->i_sb->s_id, inode->i_ino, page->index);
J_ASSERT(PageLocked(page));
if (page->index == size >> PAGE_CACHE_SHIFT)
len = size & ~PAGE_CACHE_MASK;
@@ -2989,7 +3101,10 @@
if (offset == 0)
ClearPageChecked(page);
- jbd2_journal_invalidatepage(journal, page, offset);
+ if (journal)
+ jbd2_journal_invalidatepage(journal, page, offset);
+ else
+ block_invalidatepage(page, offset);
}
static int ext4_releasepage(struct page *page, gfp_t wait)
@@ -2999,7 +3114,10 @@
WARN_ON(PageChecked(page));
if (!page_has_buffers(page))
return 0;
- return jbd2_journal_try_to_free_buffers(journal, page, wait);
+ if (journal)
+ return jbd2_journal_try_to_free_buffers(journal, page, wait);
+ else
+ return try_to_free_buffers(page);
}
/*
@@ -3271,7 +3389,7 @@
err = 0;
if (ext4_should_journal_data(inode)) {
- err = ext4_journal_dirty_metadata(handle, bh);
+ err = ext4_handle_dirty_metadata(handle, inode, bh);
} else {
if (ext4_should_order_data(inode))
err = ext4_jbd2_file_inode(handle, inode);
@@ -3395,8 +3513,8 @@
__le32 *p;
if (try_to_extend_transaction(handle, inode)) {
if (bh) {
- BUFFER_TRACE(bh, "call ext4_journal_dirty_metadata");
- ext4_journal_dirty_metadata(handle, bh);
+ BUFFER_TRACE(bh, "call ext4_handle_dirty_metadata");
+ ext4_handle_dirty_metadata(handle, inode, bh);
}
ext4_mark_inode_dirty(handle, inode);
ext4_journal_test_restart(handle, inode);
@@ -3496,7 +3614,7 @@
count, block_to_free_p, p);
if (this_bh) {
- BUFFER_TRACE(this_bh, "call ext4_journal_dirty_metadata");
+ BUFFER_TRACE(this_bh, "call ext4_handle_dirty_metadata");
/*
* The buffer head should have an attached journal head at this
@@ -3505,7 +3623,7 @@
* the block was cleared. Check for this instead of OOPSing.
*/
if (bh2jh(this_bh))
- ext4_journal_dirty_metadata(handle, this_bh);
+ ext4_handle_dirty_metadata(handle, inode, this_bh);
else
ext4_error(inode->i_sb, __func__,
"circular indirect block detected, "
@@ -3535,7 +3653,7 @@
ext4_fsblk_t nr;
__le32 *p;
- if (is_handle_aborted(handle))
+ if (ext4_handle_is_aborted(handle))
return;
if (depth--) {
@@ -3605,7 +3723,7 @@
* will merely complain about releasing a free block,
* rather than leaking blocks.
*/
- if (is_handle_aborted(handle))
+ if (ext4_handle_is_aborted(handle))
return;
if (try_to_extend_transaction(handle, inode)) {
ext4_mark_inode_dirty(handle, inode);
@@ -3624,9 +3742,10 @@
parent_bh)){
*p = 0;
BUFFER_TRACE(parent_bh,
- "call ext4_journal_dirty_metadata");
- ext4_journal_dirty_metadata(handle,
- parent_bh);
+ "call ext4_handle_dirty_metadata");
+ ext4_handle_dirty_metadata(handle,
+ inode,
+ parent_bh);
}
}
}
@@ -3814,7 +3933,7 @@
* synchronous
*/
if (IS_SYNC(inode))
- handle->h_sync = 1;
+ ext4_handle_sync(handle);
out_stop:
/*
* If this was a simple ftruncate(), and the file will remain alive
@@ -3844,7 +3963,7 @@
ext4_fsblk_t block;
int inodes_per_block, inode_offset;
- iloc->bh = 0;
+ iloc->bh = NULL;
if (!ext4_valid_inum(sb, inode->i_ino))
return -EIO;
@@ -3951,7 +4070,7 @@
num = EXT4_INODES_PER_GROUP(sb);
if (EXT4_HAS_RO_COMPAT_FEATURE(sb,
EXT4_FEATURE_RO_COMPAT_GDT_CSUM))
- num -= le16_to_cpu(gdp->bg_itable_unused);
+ num -= ext4_itable_unused_count(sb, gdp);
table += num / inodes_per_block;
if (end > table)
end = table;
@@ -4313,8 +4432,8 @@
EXT4_SET_RO_COMPAT_FEATURE(sb,
EXT4_FEATURE_RO_COMPAT_LARGE_FILE);
sb->s_dirt = 1;
- handle->h_sync = 1;
- err = ext4_journal_dirty_metadata(handle,
+ ext4_handle_sync(handle);
+ err = ext4_handle_dirty_metadata(handle, inode,
EXT4_SB(sb)->s_sbh);
}
}
@@ -4341,9 +4460,8 @@
raw_inode->i_extra_isize = cpu_to_le16(ei->i_extra_isize);
}
-
- BUFFER_TRACE(bh, "call ext4_journal_dirty_metadata");
- rc = ext4_journal_dirty_metadata(handle, bh);
+ BUFFER_TRACE(bh, "call ext4_handle_dirty_metadata");
+ rc = ext4_handle_dirty_metadata(handle, inode, bh);
if (!err)
err = rc;
ei->i_state &= ~EXT4_STATE_NEW;
@@ -4406,6 +4524,25 @@
return ext4_force_commit(inode->i_sb);
}
+int __ext4_write_dirty_metadata(struct inode *inode, struct buffer_head *bh)
+{
+ int err = 0;
+
+ mark_buffer_dirty(bh);
+ if (inode && inode_needs_sync(inode)) {
+ sync_dirty_buffer(bh);
+ if (buffer_req(bh) && !buffer_uptodate(bh)) {
+ ext4_error(inode->i_sb, __func__,
+ "IO error syncing inode, "
+ "inode=%lu, block=%llu",
+ inode->i_ino,
+ (unsigned long long)bh->b_blocknr);
+ err = -EIO;
+ }
+ }
+ return err;
+}
+
/*
* ext4_setattr()
*
@@ -4710,16 +4847,15 @@
ext4_reserve_inode_write(handle_t *handle, struct inode *inode,
struct ext4_iloc *iloc)
{
- int err = 0;
- if (handle) {
- err = ext4_get_inode_loc(inode, iloc);
- if (!err) {
- BUFFER_TRACE(iloc->bh, "get_write_access");
- err = ext4_journal_get_write_access(handle, iloc->bh);
- if (err) {
- brelse(iloc->bh);
- iloc->bh = NULL;
- }
+ int err;
+
+ err = ext4_get_inode_loc(inode, iloc);
+ if (!err) {
+ BUFFER_TRACE(iloc->bh, "get_write_access");
+ err = ext4_journal_get_write_access(handle, iloc->bh);
+ if (err) {
+ brelse(iloc->bh);
+ iloc->bh = NULL;
}
}
ext4_std_error(inode->i_sb, err);
@@ -4791,7 +4927,8 @@
might_sleep();
err = ext4_reserve_inode_write(handle, inode, &iloc);
- if (EXT4_I(inode)->i_extra_isize < sbi->s_want_extra_isize &&
+ if (ext4_handle_valid(handle) &&
+ EXT4_I(inode)->i_extra_isize < sbi->s_want_extra_isize &&
!(EXT4_I(inode)->i_state & EXT4_STATE_NO_EXPAND)) {
/*
* We need extra buffer credits since we may write into EA block
@@ -4843,6 +4980,11 @@
handle_t *current_handle = ext4_journal_current_handle();
handle_t *handle;
+ if (!ext4_handle_valid(current_handle)) {
+ ext4_mark_inode_dirty(current_handle, inode);
+ return;
+ }
+
handle = ext4_journal_start(inode, 2);
if (IS_ERR(handle))
goto out;
@@ -4880,8 +5022,9 @@
BUFFER_TRACE(iloc.bh, "get_write_access");
err = jbd2_journal_get_write_access(handle, iloc.bh);
if (!err)
- err = ext4_journal_dirty_metadata(handle,
- iloc.bh);
+ err = ext4_handle_dirty_metadata(handle,
+ inode,
+ iloc.bh);
brelse(iloc.bh);
}
}
@@ -4907,6 +5050,8 @@
*/
journal = EXT4_JOURNAL(inode);
+ if (!journal)
+ return 0;
if (is_journal_aborted(journal))
return -EROFS;
@@ -4936,7 +5081,7 @@
return PTR_ERR(handle);
err = ext4_mark_inode_dirty(handle, inode);
- handle->h_sync = 1;
+ ext4_handle_sync(handle);
ext4_journal_stop(handle);
ext4_std_error(inode->i_sb, err);
diff --git a/fs/ext4/ioctl.c b/fs/ext4/ioctl.c
index dc99b47..42dc83f 100644
--- a/fs/ext4/ioctl.c
+++ b/fs/ext4/ioctl.c
@@ -99,7 +99,7 @@
goto flags_out;
}
if (IS_SYNC(inode))
- handle->h_sync = 1;
+ ext4_handle_sync(handle);
err = ext4_reserve_inode_write(handle, inode, &iloc);
if (err)
goto flags_err;
diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
index 444ad99..918aec0 100644
--- a/fs/ext4/mballoc.c
+++ b/fs/ext4/mballoc.c
@@ -100,7 +100,7 @@
* inode as:
*
* { page }
- * [ group 0 buddy][ group 0 bitmap] [group 1][ group 1]...
+ * [ group 0 bitmap][ group 0 buddy] [group 1][ group 1]...
*
*
* one block each for bitmap and buddy information. So for each group we
@@ -330,6 +330,18 @@
* object
*
*/
+static struct kmem_cache *ext4_pspace_cachep;
+static struct kmem_cache *ext4_ac_cachep;
+static struct kmem_cache *ext4_free_ext_cachep;
+static void ext4_mb_generate_from_pa(struct super_block *sb, void *bitmap,
+ ext4_group_t group);
+static void ext4_mb_generate_from_freelist(struct super_block *sb, void *bitmap,
+ ext4_group_t group);
+static int ext4_mb_init_per_dev_proc(struct super_block *sb);
+static int ext4_mb_destroy_per_dev_proc(struct super_block *sb);
+static void release_blocks_on_commit(journal_t *journal, transaction_t *txn);
+
+
static inline void *mb_correct_addr_and_bit(int *bit, void *addr)
{
@@ -445,9 +457,9 @@
blocknr += first + i;
blocknr +=
le32_to_cpu(EXT4_SB(sb)->s_es->s_first_data_block);
-
- ext4_error(sb, __func__, "double-free of inode"
- " %lu's block %llu(bit %u in group %lu)\n",
+ ext4_grp_locked_error(sb, e4b->bd_group,
+ __func__, "double-free of inode"
+ " %lu's block %llu(bit %u in group %u)",
inode ? inode->i_ino : 0, blocknr,
first + i, e4b->bd_group);
}
@@ -477,7 +489,7 @@
b2 = (unsigned char *) bitmap;
for (i = 0; i < e4b->bd_sb->s_blocksize; i++) {
if (b1[i] != b2[i]) {
- printk(KERN_ERR "corruption in group %lu "
+ printk(KERN_ERR "corruption in group %u "
"at byte %u(%u): %x in copy != %x "
"on disk/prealloc\n",
e4b->bd_group, i, i * 8, b1[i], b2[i]);
@@ -690,8 +702,8 @@
grp->bb_fragments = fragments;
if (free != grp->bb_free) {
- ext4_error(sb, __func__,
- "EXT4-fs: group %lu: %u blocks in bitmap, %u in gd\n",
+ ext4_grp_locked_error(sb, group, __func__,
+ "EXT4-fs: group %u: %u blocks in bitmap, %u in gd",
group, free, grp->bb_free);
/*
* If we intent to continue, we consider group descritor
@@ -716,7 +728,7 @@
* stored in the inode as
*
* { page }
- * [ group 0 buddy][ group 0 bitmap] [group 1][ group 1]...
+ * [ group 0 bitmap][ group 0 buddy] [group 1][ group 1]...
*
*
* one block each for bitmap and buddy information.
@@ -782,25 +794,45 @@
if (bh[i] == NULL)
goto out;
- if (buffer_uptodate(bh[i]) &&
- !(desc->bg_flags & cpu_to_le16(EXT4_BG_BLOCK_UNINIT)))
+ if (bitmap_uptodate(bh[i]))
continue;
lock_buffer(bh[i]);
+ if (bitmap_uptodate(bh[i])) {
+ unlock_buffer(bh[i]);
+ continue;
+ }
spin_lock(sb_bgl_lock(EXT4_SB(sb), first_group + i));
if (desc->bg_flags & cpu_to_le16(EXT4_BG_BLOCK_UNINIT)) {
ext4_init_block_bitmap(sb, bh[i],
first_group + i, desc);
+ set_bitmap_uptodate(bh[i]);
set_buffer_uptodate(bh[i]);
- unlock_buffer(bh[i]);
spin_unlock(sb_bgl_lock(EXT4_SB(sb), first_group + i));
+ unlock_buffer(bh[i]);
continue;
}
spin_unlock(sb_bgl_lock(EXT4_SB(sb), first_group + i));
+ if (buffer_uptodate(bh[i])) {
+ /*
+ * if not uninit if bh is uptodate,
+ * bitmap is also uptodate
+ */
+ set_bitmap_uptodate(bh[i]);
+ unlock_buffer(bh[i]);
+ continue;
+ }
get_bh(bh[i]);
+ /*
+ * submit the buffer_head for read. We can
+ * safely mark the bitmap as uptodate now.
+ * We do it here so the bitmap uptodate bit
+ * get set with buffer lock held.
+ */
+ set_bitmap_uptodate(bh[i]);
bh[i]->b_end_io = end_buffer_read_sync;
submit_bh(READ, bh[i]);
- mb_debug("read bitmap for group %lu\n", first_group + i);
+ mb_debug("read bitmap for group %u\n", first_group + i);
}
/* wait for I/O completion */
@@ -814,6 +846,8 @@
err = 0;
first_block = page->index * blocks_per_page;
+ /* init the page */
+ memset(page_address(page), 0xff, PAGE_CACHE_SIZE);
for (i = 0; i < blocks_per_page; i++) {
int group;
struct ext4_group_info *grinfo;
@@ -840,7 +874,6 @@
BUG_ON(incore == NULL);
mb_debug("put buddy for group %u in page %lu/%x\n",
group, page->index, i * blocksize);
- memset(data, 0xff, blocksize);
grinfo = ext4_get_group_info(sb, group);
grinfo->bb_fragments = 0;
memset(grinfo->bb_counters, 0,
@@ -848,7 +881,9 @@
/*
* incore got set to the group block bitmap below
*/
+ ext4_lock_group(sb, group);
ext4_mb_generate_buddy(sb, data, incore, group);
+ ext4_unlock_group(sb, group);
incore = NULL;
} else {
/* this is block of bitmap */
@@ -862,6 +897,7 @@
/* mark all preallocated blks used in in-core bitmap */
ext4_mb_generate_from_pa(sb, data, group);
+ ext4_mb_generate_from_freelist(sb, data, group);
ext4_unlock_group(sb, group);
/* set incore so that the buddy information can be
@@ -886,18 +922,20 @@
ext4_mb_load_buddy(struct super_block *sb, ext4_group_t group,
struct ext4_buddy *e4b)
{
- struct ext4_sb_info *sbi = EXT4_SB(sb);
- struct inode *inode = sbi->s_buddy_cache;
int blocks_per_page;
int block;
int pnum;
int poff;
struct page *page;
int ret;
+ struct ext4_group_info *grp;
+ struct ext4_sb_info *sbi = EXT4_SB(sb);
+ struct inode *inode = sbi->s_buddy_cache;
- mb_debug("load group %lu\n", group);
+ mb_debug("load group %u\n", group);
blocks_per_page = PAGE_CACHE_SIZE / sb->s_blocksize;
+ grp = ext4_get_group_info(sb, group);
e4b->bd_blkbits = sb->s_blocksize_bits;
e4b->bd_info = ext4_get_group_info(sb, group);
@@ -905,6 +943,15 @@
e4b->bd_group = group;
e4b->bd_buddy_page = NULL;
e4b->bd_bitmap_page = NULL;
+ e4b->alloc_semp = &grp->alloc_sem;
+
+ /* Take the read lock on the group alloc
+ * sem. This would make sure a parallel
+ * ext4_mb_init_group happening on other
+ * groups mapped by the page is blocked
+ * till we are done with allocation
+ */
+ down_read(e4b->alloc_semp);
/*
* the buddy cache inode stores the block bitmap
@@ -920,6 +967,14 @@
page = find_get_page(inode->i_mapping, pnum);
if (page == NULL || !PageUptodate(page)) {
if (page)
+ /*
+ * drop the page reference and try
+ * to get the page with lock. If we
+ * are not uptodate that implies
+ * somebody just created the page but
+ * is yet to initialize the same. So
+ * wait for it to initialize.
+ */
page_cache_release(page);
page = find_or_create_page(inode->i_mapping, pnum, GFP_NOFS);
if (page) {
@@ -985,6 +1040,9 @@
page_cache_release(e4b->bd_buddy_page);
e4b->bd_buddy = NULL;
e4b->bd_bitmap = NULL;
+
+ /* Done with the buddy cache */
+ up_read(e4b->alloc_semp);
return ret;
}
@@ -994,6 +1052,9 @@
page_cache_release(e4b->bd_bitmap_page);
if (e4b->bd_buddy_page)
page_cache_release(e4b->bd_buddy_page);
+ /* Done with the buddy cache */
+ if (e4b->alloc_semp)
+ up_read(e4b->alloc_semp);
}
@@ -1031,7 +1092,10 @@
cur += 32;
continue;
}
- mb_clear_bit_atomic(lock, cur, bm);
+ if (lock)
+ mb_clear_bit_atomic(lock, cur, bm);
+ else
+ mb_clear_bit(cur, bm);
cur++;
}
}
@@ -1049,7 +1113,10 @@
cur += 32;
continue;
}
- mb_set_bit_atomic(lock, cur, bm);
+ if (lock)
+ mb_set_bit_atomic(lock, cur, bm);
+ else
+ mb_set_bit(cur, bm);
cur++;
}
}
@@ -1094,12 +1161,11 @@
blocknr += block;
blocknr +=
le32_to_cpu(EXT4_SB(sb)->s_es->s_first_data_block);
- ext4_unlock_group(sb, e4b->bd_group);
- ext4_error(sb, __func__, "double-free of inode"
- " %lu's block %llu(bit %u in group %lu)\n",
+ ext4_grp_locked_error(sb, e4b->bd_group,
+ __func__, "double-free of inode"
+ " %lu's block %llu(bit %u in group %u)",
inode ? inode->i_ino : 0, blocknr, block,
e4b->bd_group);
- ext4_lock_group(sb, e4b->bd_group);
}
mb_clear_bit(block, EXT4_MB_BITMAP(e4b));
e4b->bd_info->bb_counters[order]++;
@@ -1296,13 +1362,20 @@
ac->ac_tail = ret & 0xffff;
ac->ac_buddy = ret >> 16;
- /* XXXXXXX: SUCH A HORRIBLE **CK */
- /*FIXME!! Why ? */
+ /*
+ * take the page reference. We want the page to be pinned
+ * so that we don't get a ext4_mb_init_cache_call for this
+ * group until we update the bitmap. That would mean we
+ * double allocate blocks. The reference is dropped
+ * in ext4_mb_release_context
+ */
ac->ac_bitmap_page = e4b->bd_bitmap_page;
get_page(ac->ac_bitmap_page);
ac->ac_buddy_page = e4b->bd_buddy_page;
get_page(ac->ac_buddy_page);
-
+ /* on allocation we use ac to track the held semaphore */
+ ac->alloc_semp = e4b->alloc_semp;
+ e4b->alloc_semp = NULL;
/* store last allocated for subsequent stream allocation */
if ((ac->ac_flags & EXT4_MB_HINT_DATA)) {
spin_lock(&sbi->s_md_lock);
@@ -1326,6 +1399,8 @@
struct ext4_free_extent ex;
int max;
+ if (ac->ac_status == AC_STATUS_FOUND)
+ return;
/*
* We don't want to scan for a whole year
*/
@@ -1575,8 +1650,9 @@
* free blocks even though group info says we
* we have free blocks
*/
- ext4_error(sb, __func__, "%d free blocks as per "
- "group info. But bitmap says 0\n",
+ ext4_grp_locked_error(sb, e4b->bd_group,
+ __func__, "%d free blocks as per "
+ "group info. But bitmap says 0",
free);
break;
}
@@ -1584,8 +1660,9 @@
mb_find_extent(e4b, 0, i, ac->ac_g_ex.fe_len, &ex);
BUG_ON(ex.fe_len <= 0);
if (free < ex.fe_len) {
- ext4_error(sb, __func__, "%d free blocks as per "
- "group info. But got %d blocks\n",
+ ext4_grp_locked_error(sb, e4b->bd_group,
+ __func__, "%d free blocks as per "
+ "group info. But got %d blocks",
free, ex.fe_len);
/*
* The number of free blocks differs. This mostly
@@ -1692,6 +1769,173 @@
return 0;
}
+/*
+ * lock the group_info alloc_sem of all the groups
+ * belonging to the same buddy cache page. This
+ * make sure other parallel operation on the buddy
+ * cache doesn't happen whild holding the buddy cache
+ * lock
+ */
+int ext4_mb_get_buddy_cache_lock(struct super_block *sb, ext4_group_t group)
+{
+ int i;
+ int block, pnum;
+ int blocks_per_page;
+ int groups_per_page;
+ ext4_group_t first_group;
+ struct ext4_group_info *grp;
+
+ blocks_per_page = PAGE_CACHE_SIZE / sb->s_blocksize;
+ /*
+ * the buddy cache inode stores the block bitmap
+ * and buddy information in consecutive blocks.
+ * So for each group we need two blocks.
+ */
+ block = group * 2;
+ pnum = block / blocks_per_page;
+ first_group = pnum * blocks_per_page / 2;
+
+ groups_per_page = blocks_per_page >> 1;
+ if (groups_per_page == 0)
+ groups_per_page = 1;
+ /* read all groups the page covers into the cache */
+ for (i = 0; i < groups_per_page; i++) {
+
+ if ((first_group + i) >= EXT4_SB(sb)->s_groups_count)
+ break;
+ grp = ext4_get_group_info(sb, first_group + i);
+ /* take all groups write allocation
+ * semaphore. This make sure there is
+ * no block allocation going on in any
+ * of that groups
+ */
+ down_write_nested(&grp->alloc_sem, i);
+ }
+ return i;
+}
+
+void ext4_mb_put_buddy_cache_lock(struct super_block *sb,
+ ext4_group_t group, int locked_group)
+{
+ int i;
+ int block, pnum;
+ int blocks_per_page;
+ ext4_group_t first_group;
+ struct ext4_group_info *grp;
+
+ blocks_per_page = PAGE_CACHE_SIZE / sb->s_blocksize;
+ /*
+ * the buddy cache inode stores the block bitmap
+ * and buddy information in consecutive blocks.
+ * So for each group we need two blocks.
+ */
+ block = group * 2;
+ pnum = block / blocks_per_page;
+ first_group = pnum * blocks_per_page / 2;
+ /* release locks on all the groups */
+ for (i = 0; i < locked_group; i++) {
+
+ grp = ext4_get_group_info(sb, first_group + i);
+ /* take all groups write allocation
+ * semaphore. This make sure there is
+ * no block allocation going on in any
+ * of that groups
+ */
+ up_write(&grp->alloc_sem);
+ }
+
+}
+
+static int ext4_mb_init_group(struct super_block *sb, ext4_group_t group)
+{
+
+ int ret;
+ void *bitmap;
+ int blocks_per_page;
+ int block, pnum, poff;
+ int num_grp_locked = 0;
+ struct ext4_group_info *this_grp;
+ struct ext4_sb_info *sbi = EXT4_SB(sb);
+ struct inode *inode = sbi->s_buddy_cache;
+ struct page *page = NULL, *bitmap_page = NULL;
+
+ mb_debug("init group %lu\n", group);
+ blocks_per_page = PAGE_CACHE_SIZE / sb->s_blocksize;
+ this_grp = ext4_get_group_info(sb, group);
+ /*
+ * This ensures we don't add group
+ * to this buddy cache via resize
+ */
+ num_grp_locked = ext4_mb_get_buddy_cache_lock(sb, group);
+ if (!EXT4_MB_GRP_NEED_INIT(this_grp)) {
+ /*
+ * somebody initialized the group
+ * return without doing anything
+ */
+ ret = 0;
+ goto err;
+ }
+ /*
+ * the buddy cache inode stores the block bitmap
+ * and buddy information in consecutive blocks.
+ * So for each group we need two blocks.
+ */
+ block = group * 2;
+ pnum = block / blocks_per_page;
+ poff = block % blocks_per_page;
+ page = find_or_create_page(inode->i_mapping, pnum, GFP_NOFS);
+ if (page) {
+ BUG_ON(page->mapping != inode->i_mapping);
+ ret = ext4_mb_init_cache(page, NULL);
+ if (ret) {
+ unlock_page(page);
+ goto err;
+ }
+ unlock_page(page);
+ }
+ if (page == NULL || !PageUptodate(page)) {
+ ret = -EIO;
+ goto err;
+ }
+ mark_page_accessed(page);
+ bitmap_page = page;
+ bitmap = page_address(page) + (poff * sb->s_blocksize);
+
+ /* init buddy cache */
+ block++;
+ pnum = block / blocks_per_page;
+ poff = block % blocks_per_page;
+ page = find_or_create_page(inode->i_mapping, pnum, GFP_NOFS);
+ if (page == bitmap_page) {
+ /*
+ * If both the bitmap and buddy are in
+ * the same page we don't need to force
+ * init the buddy
+ */
+ unlock_page(page);
+ } else if (page) {
+ BUG_ON(page->mapping != inode->i_mapping);
+ ret = ext4_mb_init_cache(page, bitmap);
+ if (ret) {
+ unlock_page(page);
+ goto err;
+ }
+ unlock_page(page);
+ }
+ if (page == NULL || !PageUptodate(page)) {
+ ret = -EIO;
+ goto err;
+ }
+ mark_page_accessed(page);
+err:
+ ext4_mb_put_buddy_cache_lock(sb, group, num_grp_locked);
+ if (bitmap_page)
+ page_cache_release(bitmap_page);
+ if (page)
+ page_cache_release(page);
+ return ret;
+}
+
static noinline_for_stack int
ext4_mb_regular_allocator(struct ext4_allocation_context *ac)
{
@@ -1775,7 +2019,7 @@
group = 0;
/* quick check to skip empty groups */
- grp = ext4_get_group_info(ac->ac_sb, group);
+ grp = ext4_get_group_info(sb, group);
if (grp->bb_free == 0)
continue;
@@ -1788,10 +2032,9 @@
* we need full data about the group
* to make a good selection
*/
- err = ext4_mb_load_buddy(sb, group, &e4b);
+ err = ext4_mb_init_group(sb, group);
if (err)
goto out;
- ext4_mb_release_desc(&e4b);
}
/*
@@ -1932,13 +2175,13 @@
if (hs->op == EXT4_MB_HISTORY_ALLOC) {
fmt = "%-5u %-8u %-23s %-23s %-23s %-5u %-5u %-2u "
"%-5u %-5s %-5u %-6u\n";
- sprintf(buf2, "%lu/%d/%u@%u", hs->result.fe_group,
+ sprintf(buf2, "%u/%d/%u@%u", hs->result.fe_group,
hs->result.fe_start, hs->result.fe_len,
hs->result.fe_logical);
- sprintf(buf, "%lu/%d/%u@%u", hs->orig.fe_group,
+ sprintf(buf, "%u/%d/%u@%u", hs->orig.fe_group,
hs->orig.fe_start, hs->orig.fe_len,
hs->orig.fe_logical);
- sprintf(buf3, "%lu/%d/%u@%u", hs->goal.fe_group,
+ sprintf(buf3, "%u/%d/%u@%u", hs->goal.fe_group,
hs->goal.fe_start, hs->goal.fe_len,
hs->goal.fe_logical);
seq_printf(seq, fmt, hs->pid, hs->ino, buf, buf3, buf2,
@@ -1947,20 +2190,20 @@
hs->buddy ? 1 << hs->buddy : 0);
} else if (hs->op == EXT4_MB_HISTORY_PREALLOC) {
fmt = "%-5u %-8u %-23s %-23s %-23s\n";
- sprintf(buf2, "%lu/%d/%u@%u", hs->result.fe_group,
+ sprintf(buf2, "%u/%d/%u@%u", hs->result.fe_group,
hs->result.fe_start, hs->result.fe_len,
hs->result.fe_logical);
- sprintf(buf, "%lu/%d/%u@%u", hs->orig.fe_group,
+ sprintf(buf, "%u/%d/%u@%u", hs->orig.fe_group,
hs->orig.fe_start, hs->orig.fe_len,
hs->orig.fe_logical);
seq_printf(seq, fmt, hs->pid, hs->ino, buf, "", buf2);
} else if (hs->op == EXT4_MB_HISTORY_DISCARD) {
- sprintf(buf2, "%lu/%d/%u", hs->result.fe_group,
+ sprintf(buf2, "%u/%d/%u", hs->result.fe_group,
hs->result.fe_start, hs->result.fe_len);
seq_printf(seq, "%-5u %-8u %-23s discard\n",
hs->pid, hs->ino, buf2);
} else if (hs->op == EXT4_MB_HISTORY_FREE) {
- sprintf(buf2, "%lu/%d/%u", hs->result.fe_group,
+ sprintf(buf2, "%u/%d/%u", hs->result.fe_group,
hs->result.fe_start, hs->result.fe_len);
seq_printf(seq, "%-5u %-8u %-23s free\n",
hs->pid, hs->ino, buf2);
@@ -2073,7 +2316,7 @@
return NULL;
group = *pos + 1;
- return (void *) group;
+ return (void *) ((unsigned long) group);
}
static void *ext4_mb_seq_groups_next(struct seq_file *seq, void *v, loff_t *pos)
@@ -2086,13 +2329,13 @@
if (*pos < 0 || *pos >= sbi->s_groups_count)
return NULL;
group = *pos + 1;
- return (void *) group;;
+ return (void *) ((unsigned long) group);
}
static int ext4_mb_seq_groups_show(struct seq_file *seq, void *v)
{
struct super_block *sb = seq->private;
- long group = (long) v;
+ ext4_group_t group = (ext4_group_t) ((unsigned long) v);
int i;
int err;
struct ext4_buddy e4b;
@@ -2114,7 +2357,7 @@
sizeof(struct ext4_group_info);
err = ext4_mb_load_buddy(sb, group, &e4b);
if (err) {
- seq_printf(seq, "#%-5lu: I/O error\n", group);
+ seq_printf(seq, "#%-5u: I/O error\n", group);
return 0;
}
ext4_lock_group(sb, group);
@@ -2122,7 +2365,7 @@
ext4_unlock_group(sb, group);
ext4_mb_release_desc(&e4b);
- seq_printf(seq, "#%-5lu: %-5u %-5u %-5u [", group, sg.info.bb_free,
+ seq_printf(seq, "#%-5u: %-5u %-5u %-5u [", group, sg.info.bb_free,
sg.info.bb_fragments, sg.info.bb_first_free);
for (i = 0; i <= 13; i++)
seq_printf(seq, " %-5u", i <= sb->s_blocksize_bits + 1 ?
@@ -2296,10 +2539,11 @@
ext4_free_blocks_after_init(sb, group, desc);
} else {
meta_group_info[i]->bb_free =
- le16_to_cpu(desc->bg_free_blocks_count);
+ ext4_free_blks_count(sb, desc);
}
INIT_LIST_HEAD(&meta_group_info[i]->bb_prealloc_list);
+ init_rwsem(&meta_group_info[i]->alloc_sem);
meta_group_info[i]->bb_free_root.rb_node = NULL;;
#ifdef DOUBLE_CHECK
@@ -2327,54 +2571,6 @@
} /* ext4_mb_add_groupinfo */
/*
- * Add a group to the existing groups.
- * This function is used for online resize
- */
-int ext4_mb_add_more_groupinfo(struct super_block *sb, ext4_group_t group,
- struct ext4_group_desc *desc)
-{
- struct ext4_sb_info *sbi = EXT4_SB(sb);
- struct inode *inode = sbi->s_buddy_cache;
- int blocks_per_page;
- int block;
- int pnum;
- struct page *page;
- int err;
-
- /* Add group based on group descriptor*/
- err = ext4_mb_add_groupinfo(sb, group, desc);
- if (err)
- return err;
-
- /*
- * Cache pages containing dynamic mb_alloc datas (buddy and bitmap
- * datas) are set not up to date so that they will be re-initilaized
- * during the next call to ext4_mb_load_buddy
- */
-
- /* Set buddy page as not up to date */
- blocks_per_page = PAGE_CACHE_SIZE / sb->s_blocksize;
- block = group * 2;
- pnum = block / blocks_per_page;
- page = find_get_page(inode->i_mapping, pnum);
- if (page != NULL) {
- ClearPageUptodate(page);
- page_cache_release(page);
- }
-
- /* Set bitmap page as not up to date */
- block++;
- pnum = block / blocks_per_page;
- page = find_get_page(inode->i_mapping, pnum);
- if (page != NULL) {
- ClearPageUptodate(page);
- page_cache_release(page);
- }
-
- return 0;
-}
-
-/*
* Update an existing group.
* This function is used for online resize
*/
@@ -2457,7 +2653,7 @@
desc = ext4_get_group_desc(sb, i, NULL);
if (desc == NULL) {
printk(KERN_ERR
- "EXT4-fs: can't read descriptor %lu\n", i);
+ "EXT4-fs: can't read descriptor %u\n", i);
goto err_freebuddy;
}
if (ext4_mb_add_groupinfo(sb, i, desc) != 0)
@@ -2493,6 +2689,8 @@
if (sbi->s_mb_offsets == NULL) {
return -ENOMEM;
}
+
+ i = (sb->s_blocksize_bits + 2) * sizeof(unsigned int);
sbi->s_mb_maxs = kmalloc(i, GFP_KERNEL);
if (sbi->s_mb_maxs == NULL) {
kfree(sbi->s_mb_maxs);
@@ -2551,7 +2749,8 @@
ext4_mb_init_per_dev_proc(sb);
ext4_mb_history_init(sb);
- sbi->s_journal->j_commit_callback = release_blocks_on_commit;
+ if (sbi->s_journal)
+ sbi->s_journal->j_commit_callback = release_blocks_on_commit;
printk(KERN_INFO "EXT4-fs: mballoc enabled\n");
return 0;
@@ -2652,7 +2851,7 @@
list_for_each_safe(l, ltmp, &txn->t_private_list) {
entry = list_entry(l, struct ext4_free_data, list);
- mb_debug("gonna free %u blocks in group %lu (0x%p):",
+ mb_debug("gonna free %u blocks in group %u (0x%p):",
entry->count, entry->group, entry);
err = ext4_mb_load_buddy(sb, entry->group, &e4b);
@@ -2679,8 +2878,9 @@
discard_block = (ext4_fsblk_t) entry->group * EXT4_BLOCKS_PER_GROUP(sb)
+ entry->start_blk
+ le32_to_cpu(EXT4_SB(sb)->s_es->s_first_data_block);
- trace_mark(ext4_discard_blocks, "dev %s blk %llu count %u", sb->s_id,
- (unsigned long long) discard_block, entry->count);
+ trace_mark(ext4_discard_blocks, "dev %s blk %llu count %u",
+ sb->s_id, (unsigned long long) discard_block,
+ entry->count);
sb_issue_discard(sb, discard_block, entry->count);
kmem_cache_free(ext4_free_ext_cachep, entry);
@@ -2791,7 +2991,7 @@
*/
static noinline_for_stack int
ext4_mb_mark_diskspace_used(struct ext4_allocation_context *ac,
- handle_t *handle, unsigned long reserv_blks)
+ handle_t *handle, unsigned int reserv_blks)
{
struct buffer_head *bitmap_bh = NULL;
struct ext4_super_block *es;
@@ -2824,7 +3024,7 @@
if (!gdp)
goto out_err;
- ext4_debug("using block group %lu(%d)\n", ac->ac_b_ex.fe_group,
+ ext4_debug("using block group %u(%d)\n", ac->ac_b_ex.fe_group,
gdp->bg_free_blocks_count);
err = ext4_journal_get_write_access(handle, gdp_bh);
@@ -2843,8 +3043,8 @@
in_range(block + len - 1, ext4_inode_table(sb, gdp),
EXT4_SB(sb)->s_itb_per_group)) {
ext4_error(sb, __func__,
- "Allocating block in system zone - block = %llu",
- block);
+ "Allocating block %llu in system zone of %d group\n",
+ block, ac->ac_b_ex.fe_group);
/* File system mounted not to panic on error
* Fix the bitmap and repeat the block allocation
* We leak some of the blocks here.
@@ -2852,7 +3052,7 @@
mb_set_bits(sb_bgl_lock(sbi, ac->ac_b_ex.fe_group),
bitmap_bh->b_data, ac->ac_b_ex.fe_start,
ac->ac_b_ex.fe_len);
- err = ext4_journal_dirty_metadata(handle, bitmap_bh);
+ err = ext4_handle_dirty_metadata(handle, NULL, bitmap_bh);
if (!err)
err = -EAGAIN;
goto out_err;
@@ -2866,18 +3066,17 @@
}
}
#endif
- mb_set_bits(sb_bgl_lock(sbi, ac->ac_b_ex.fe_group), bitmap_bh->b_data,
- ac->ac_b_ex.fe_start, ac->ac_b_ex.fe_len);
-
spin_lock(sb_bgl_lock(sbi, ac->ac_b_ex.fe_group));
+ mb_set_bits(NULL, bitmap_bh->b_data,
+ ac->ac_b_ex.fe_start, ac->ac_b_ex.fe_len);
if (gdp->bg_flags & cpu_to_le16(EXT4_BG_BLOCK_UNINIT)) {
gdp->bg_flags &= cpu_to_le16(~EXT4_BG_BLOCK_UNINIT);
- gdp->bg_free_blocks_count =
- cpu_to_le16(ext4_free_blocks_after_init(sb,
- ac->ac_b_ex.fe_group,
- gdp));
+ ext4_free_blks_set(sb, gdp,
+ ext4_free_blocks_after_init(sb,
+ ac->ac_b_ex.fe_group, gdp));
}
- le16_add_cpu(&gdp->bg_free_blocks_count, -ac->ac_b_ex.fe_len);
+ len = ext4_free_blks_count(sb, gdp) - ac->ac_b_ex.fe_len;
+ ext4_free_blks_set(sb, gdp, len);
gdp->bg_checksum = ext4_group_desc_csum(sbi, ac->ac_b_ex.fe_group, gdp);
spin_unlock(sb_bgl_lock(sbi, ac->ac_b_ex.fe_group));
percpu_counter_sub(&sbi->s_freeblocks_counter, ac->ac_b_ex.fe_len);
@@ -2899,10 +3098,10 @@
spin_unlock(sb_bgl_lock(sbi, flex_group));
}
- err = ext4_journal_dirty_metadata(handle, bitmap_bh);
+ err = ext4_handle_dirty_metadata(handle, NULL, bitmap_bh);
if (err)
goto out_err;
- err = ext4_journal_dirty_metadata(handle, gdp_bh);
+ err = ext4_handle_dirty_metadata(handle, NULL, gdp_bh);
out_err:
sb->s_dirt = 1;
@@ -3031,7 +3230,7 @@
/* check we don't cross already preallocated blocks */
rcu_read_lock();
list_for_each_entry_rcu(pa, &ei->i_prealloc_list, pa_inode_list) {
- unsigned long pa_end;
+ ext4_lblk_t pa_end;
if (pa->pa_deleted)
continue;
@@ -3075,7 +3274,7 @@
/* XXX: extra loop to check we really don't overlap preallocations */
rcu_read_lock();
list_for_each_entry_rcu(pa, &ei->i_prealloc_list, pa_inode_list) {
- unsigned long pa_end;
+ ext4_lblk_t pa_end;
spin_lock(&pa->pa_lock);
if (pa->pa_deleted == 0) {
pa_end = pa->pa_lstart + pa->pa_len;
@@ -3307,6 +3506,32 @@
}
/*
+ * the function goes through all block freed in the group
+ * but not yet committed and marks them used in in-core bitmap.
+ * buddy must be generated from this bitmap
+ * Need to be called with ext4 group lock (ext4_lock_group)
+ */
+static void ext4_mb_generate_from_freelist(struct super_block *sb, void *bitmap,
+ ext4_group_t group)
+{
+ struct rb_node *n;
+ struct ext4_group_info *grp;
+ struct ext4_free_data *entry;
+
+ grp = ext4_get_group_info(sb, group);
+ n = rb_first(&(grp->bb_free_root));
+
+ while (n) {
+ entry = rb_entry(n, struct ext4_free_data, node);
+ mb_set_bits(sb_bgl_lock(EXT4_SB(sb), group),
+ bitmap, entry->start_blk,
+ entry->count);
+ n = rb_next(n);
+ }
+ return;
+}
+
+/*
* the function goes through all preallocation in this group and marks them
* used in in-core bitmap. buddy must be generated from this bitmap
* Need to be called with ext4 group lock (ext4_lock_group)
@@ -3346,7 +3571,7 @@
preallocated += len;
count++;
}
- mb_debug("prellocated %u for group %lu\n", preallocated, group);
+ mb_debug("prellocated %u for group %u\n", preallocated, group);
}
static void ext4_mb_pa_callback(struct rcu_head *head)
@@ -3363,7 +3588,7 @@
static void ext4_mb_put_pa(struct ext4_allocation_context *ac,
struct super_block *sb, struct ext4_prealloc_space *pa)
{
- unsigned long grp;
+ ext4_group_t grp;
if (!atomic_dec_and_test(&pa->pa_count) || pa->pa_free != 0)
return;
@@ -3473,6 +3698,10 @@
mb_debug("new inode pa %p: %llu/%u for %u\n", pa,
pa->pa_pstart, pa->pa_len, pa->pa_lstart);
+ trace_mark(ext4_mb_new_inode_pa,
+ "dev %s ino %lu pstart %llu len %u lstart %u",
+ sb->s_id, ac->ac_inode->i_ino,
+ pa->pa_pstart, pa->pa_len, pa->pa_lstart);
ext4_mb_use_inode_pa(ac, pa);
atomic_add(pa->pa_free, &EXT4_SB(sb)->s_mb_preallocated);
@@ -3530,7 +3759,9 @@
pa->pa_linear = 1;
mb_debug("new group pa %p: %llu/%u for %u\n", pa,
- pa->pa_pstart, pa->pa_len, pa->pa_lstart);
+ pa->pa_pstart, pa->pa_len, pa->pa_lstart);
+ trace_mark(ext4_mb_new_group_pa, "dev %s pstart %llu len %u lstart %u",
+ sb->s_id, pa->pa_pstart, pa->pa_len, pa->pa_lstart);
ext4_mb_use_group_pa(ac, pa);
atomic_add(pa->pa_free, &EXT4_SB(sb)->s_mb_preallocated);
@@ -3579,16 +3810,18 @@
{
struct super_block *sb = e4b->bd_sb;
struct ext4_sb_info *sbi = EXT4_SB(sb);
- unsigned long end;
- unsigned long next;
+ unsigned int end;
+ unsigned int next;
ext4_group_t group;
ext4_grpblk_t bit;
+ unsigned long long grp_blk_start;
sector_t start;
int err = 0;
int free = 0;
BUG_ON(pa->pa_deleted == 0);
ext4_get_group_no_and_offset(sb, pa->pa_pstart, &group, &bit);
+ grp_blk_start = pa->pa_pstart - bit;
BUG_ON(group != e4b->bd_group && pa->pa_len != 0);
end = bit + pa->pa_len;
@@ -3618,6 +3851,10 @@
ext4_mb_store_history(ac);
}
+ trace_mark(ext4_mb_release_inode_pa,
+ "dev %s ino %lu block %llu count %u",
+ sb->s_id, pa->pa_inode->i_ino, grp_blk_start + bit,
+ next - bit);
mb_free_blocks(pa->pa_inode, e4b, bit, next - bit);
bit = next + 1;
}
@@ -3626,8 +3863,9 @@
pa, (unsigned long) pa->pa_lstart,
(unsigned long) pa->pa_pstart,
(unsigned long) pa->pa_len);
- ext4_error(sb, __func__, "free %u, pa_free %u\n",
- free, pa->pa_free);
+ ext4_grp_locked_error(sb, group,
+ __func__, "free %u, pa_free %u",
+ free, pa->pa_free);
/*
* pa is already deleted so we use the value obtained
* from the bitmap and continue.
@@ -3650,6 +3888,8 @@
if (ac)
ac->ac_op = EXT4_MB_HISTORY_DISCARD;
+ trace_mark(ext4_mb_release_group_pa, "dev %s pstart %llu len %d",
+ sb->s_id, pa->pa_pstart, pa->pa_len);
BUG_ON(pa->pa_deleted == 0);
ext4_get_group_no_and_offset(sb, pa->pa_pstart, &group, &bit);
BUG_ON(group != e4b->bd_group && pa->pa_len != 0);
@@ -3692,7 +3932,7 @@
int busy = 0;
int free = 0;
- mb_debug("discard preallocation for group %lu\n", group);
+ mb_debug("discard preallocation for group %u\n", group);
if (list_empty(&grp->bb_prealloc_list))
return 0;
@@ -3700,14 +3940,14 @@
bitmap_bh = ext4_read_block_bitmap(sb, group);
if (bitmap_bh == NULL) {
ext4_error(sb, __func__, "Error in reading block "
- "bitmap for %lu\n", group);
+ "bitmap for %u", group);
return 0;
}
err = ext4_mb_load_buddy(sb, group, &e4b);
if (err) {
ext4_error(sb, __func__, "Error in loading buddy "
- "information for %lu\n", group);
+ "information for %u", group);
put_bh(bitmap_bh);
return 0;
}
@@ -3815,6 +4055,8 @@
}
mb_debug("discard preallocation for inode %lu\n", inode->i_ino);
+ trace_mark(ext4_discard_preallocations, "dev %s ino %lu", sb->s_id,
+ inode->i_ino);
INIT_LIST_HEAD(&list);
@@ -3874,14 +4116,14 @@
err = ext4_mb_load_buddy(sb, group, &e4b);
if (err) {
ext4_error(sb, __func__, "Error in loading buddy "
- "information for %lu\n", group);
+ "information for %u", group);
continue;
}
bitmap_bh = ext4_read_block_bitmap(sb, group);
if (bitmap_bh == NULL) {
ext4_error(sb, __func__, "Error in reading block "
- "bitmap for %lu\n", group);
+ "bitmap for %u", group);
ext4_mb_release_desc(&e4b);
continue;
}
@@ -4024,8 +4266,8 @@
struct ext4_sb_info *sbi = EXT4_SB(sb);
struct ext4_super_block *es = sbi->s_es;
ext4_group_t group;
- unsigned long len;
- unsigned long goal;
+ unsigned int len;
+ ext4_fsblk_t goal;
ext4_grpblk_t block;
/* we can't allocate > group size */
@@ -4068,6 +4310,7 @@
ac->ac_pa = NULL;
ac->ac_bitmap_page = NULL;
ac->ac_buddy_page = NULL;
+ ac->alloc_semp = NULL;
ac->ac_lg = NULL;
/* we have to define context: we'll we work with a file or
@@ -4146,7 +4389,7 @@
ext4_get_group_no_and_offset(sb, pa->pa_pstart, &group, NULL);
if (ext4_mb_load_buddy(sb, group, &e4b)) {
ext4_error(sb, __func__, "Error in loading buddy "
- "information for %lu\n", group);
+ "information for %u", group);
continue;
}
ext4_lock_group(sb, group);
@@ -4248,6 +4491,8 @@
}
ext4_mb_put_pa(ac, ac->ac_sb, pa);
}
+ if (ac->alloc_semp)
+ up_read(ac->alloc_semp);
if (ac->ac_bitmap_page)
page_cache_release(ac->ac_bitmap_page);
if (ac->ac_buddy_page)
@@ -4264,6 +4509,8 @@
int ret;
int freed = 0;
+ trace_mark(ext4_mb_discard_preallocations, "dev %s needed %d",
+ sb->s_id, needed);
for (i = 0; i < EXT4_SB(sb)->s_groups_count && needed > 0; i++) {
ret = ext4_mb_discard_group_preallocations(sb, i, needed);
freed += ret;
@@ -4286,12 +4533,24 @@
struct ext4_sb_info *sbi;
struct super_block *sb;
ext4_fsblk_t block = 0;
- unsigned long inquota;
- unsigned long reserv_blks = 0;
+ unsigned int inquota;
+ unsigned int reserv_blks = 0;
sb = ar->inode->i_sb;
sbi = EXT4_SB(sb);
+ trace_mark(ext4_request_blocks, "dev %s flags %u len %u ino %lu "
+ "lblk %llu goal %llu lleft %llu lright %llu "
+ "pleft %llu pright %llu ",
+ sb->s_id, ar->flags, ar->len,
+ ar->inode ? ar->inode->i_ino : 0,
+ (unsigned long long) ar->logical,
+ (unsigned long long) ar->goal,
+ (unsigned long long) ar->lleft,
+ (unsigned long long) ar->lright,
+ (unsigned long long) ar->pleft,
+ (unsigned long long) ar->pright);
+
if (!EXT4_I(ar->inode)->i_delalloc_reserved_flag) {
/*
* With delalloc we already reserved the blocks
@@ -4313,7 +4572,7 @@
}
if (ar->len == 0) {
*errp = -EDQUOT;
- return 0;
+ goto out3;
}
inquota = ar->len;
@@ -4348,10 +4607,14 @@
ac->ac_o_ex.fe_len < ac->ac_b_ex.fe_len)
ext4_mb_new_preallocation(ac);
}
-
if (likely(ac->ac_status == AC_STATUS_FOUND)) {
*errp = ext4_mb_mark_diskspace_used(ac, handle, reserv_blks);
if (*errp == -EAGAIN) {
+ /*
+ * drop the reference that we took
+ * in ext4_mb_use_best_found
+ */
+ ext4_mb_release_context(ac);
ac->ac_b_ex.fe_group = 0;
ac->ac_b_ex.fe_start = 0;
ac->ac_b_ex.fe_len = 0;
@@ -4382,6 +4645,26 @@
out1:
if (ar->len < inquota)
DQUOT_FREE_BLOCK(ar->inode, inquota - ar->len);
+out3:
+ if (!ar->len) {
+ if (!EXT4_I(ar->inode)->i_delalloc_reserved_flag)
+ /* release all the reserved blocks if non delalloc */
+ percpu_counter_sub(&sbi->s_dirtyblocks_counter,
+ reserv_blks);
+ }
+
+ trace_mark(ext4_allocate_blocks,
+ "dev %s block %llu flags %u len %u ino %lu "
+ "logical %llu goal %llu lleft %llu lright %llu "
+ "pleft %llu pright %llu ",
+ sb->s_id, (unsigned long long) block,
+ ar->flags, ar->len, ar->inode ? ar->inode->i_ino : 0,
+ (unsigned long long) ar->logical,
+ (unsigned long long) ar->goal,
+ (unsigned long long) ar->lleft,
+ (unsigned long long) ar->lright,
+ (unsigned long long) ar->pleft,
+ (unsigned long long) ar->pright);
return block;
}
@@ -4403,27 +4686,23 @@
static noinline_for_stack int
ext4_mb_free_metadata(handle_t *handle, struct ext4_buddy *e4b,
- ext4_group_t group, ext4_grpblk_t block, int count)
+ struct ext4_free_data *new_entry)
{
+ ext4_grpblk_t block;
+ struct ext4_free_data *entry;
struct ext4_group_info *db = e4b->bd_info;
struct super_block *sb = e4b->bd_sb;
struct ext4_sb_info *sbi = EXT4_SB(sb);
- struct ext4_free_data *entry, *new_entry;
struct rb_node **n = &db->bb_free_root.rb_node, *node;
struct rb_node *parent = NULL, *new_node;
-
+ BUG_ON(!ext4_handle_valid(handle));
BUG_ON(e4b->bd_bitmap_page == NULL);
BUG_ON(e4b->bd_buddy_page == NULL);
- new_entry = kmem_cache_alloc(ext4_free_ext_cachep, GFP_NOFS);
- new_entry->start_blk = block;
- new_entry->group = group;
- new_entry->count = count;
- new_entry->t_tid = handle->h_transaction->t_tid;
new_node = &new_entry->node;
+ block = new_entry->start_blk;
- ext4_lock_group(sb, group);
if (!*n) {
/* first free block exent. We need to
protect buddy cache from being freed,
@@ -4441,10 +4720,9 @@
else if (block >= (entry->start_blk + entry->count))
n = &(*n)->rb_right;
else {
- ext4_unlock_group(sb, group);
- ext4_error(sb, __func__,
- "Double free of blocks %d (%d %d)\n",
- block, entry->start_blk, entry->count);
+ ext4_grp_locked_error(sb, e4b->bd_group, __func__,
+ "Double free of blocks %d (%d %d)",
+ block, entry->start_blk, entry->count);
return 0;
}
}
@@ -4483,7 +4761,6 @@
spin_lock(&sbi->s_md_lock);
list_add(&new_entry->list, &handle->h_transaction->t_private_list);
spin_unlock(&sbi->s_md_lock);
- ext4_unlock_group(sb, group);
return 0;
}
@@ -4499,7 +4776,7 @@
struct ext4_allocation_context *ac = NULL;
struct ext4_group_desc *gdp;
struct ext4_super_block *es;
- unsigned long overflow;
+ unsigned int overflow;
ext4_grpblk_t bit;
struct buffer_head *gd_bh;
ext4_group_t block_group;
@@ -4522,6 +4799,10 @@
}
ext4_debug("freeing block %lu\n", block);
+ trace_mark(ext4_free_blocks,
+ "dev %s block %llu count %lu metadata %d ino %lu",
+ sb->s_id, (unsigned long long) block, count, metadata,
+ inode ? inode->i_ino : 0);
ac = kmem_cache_alloc(ext4_ac_cachep, GFP_NOFS);
if (ac) {
@@ -4581,11 +4862,6 @@
err = ext4_journal_get_write_access(handle, gd_bh);
if (err)
goto error_return;
-
- err = ext4_mb_load_buddy(sb, block_group, &e4b);
- if (err)
- goto error_return;
-
#ifdef AGGRESSIVE_CHECK
{
int i;
@@ -4593,13 +4869,6 @@
BUG_ON(!mb_test_bit(bit + i, bitmap_bh->b_data));
}
#endif
- mb_clear_bits(sb_bgl_lock(sbi, block_group), bitmap_bh->b_data,
- bit, count);
-
- /* We dirtied the bitmap block */
- BUFFER_TRACE(bitmap_bh, "dirtied bitmap block");
- err = ext4_journal_dirty_metadata(handle, bitmap_bh);
-
if (ac) {
ac->ac_b_ex.fe_group = block_group;
ac->ac_b_ex.fe_start = bit;
@@ -4607,19 +4876,41 @@
ext4_mb_store_history(ac);
}
- if (metadata) {
- /* blocks being freed are metadata. these blocks shouldn't
- * be used until this transaction is committed */
- ext4_mb_free_metadata(handle, &e4b, block_group, bit, count);
+ err = ext4_mb_load_buddy(sb, block_group, &e4b);
+ if (err)
+ goto error_return;
+ if (metadata && ext4_handle_valid(handle)) {
+ struct ext4_free_data *new_entry;
+ /*
+ * blocks being freed are metadata. these blocks shouldn't
+ * be used until this transaction is committed
+ */
+ new_entry = kmem_cache_alloc(ext4_free_ext_cachep, GFP_NOFS);
+ new_entry->start_blk = bit;
+ new_entry->group = block_group;
+ new_entry->count = count;
+ new_entry->t_tid = handle->h_transaction->t_tid;
+ ext4_lock_group(sb, block_group);
+ mb_clear_bits(sb_bgl_lock(sbi, block_group), bitmap_bh->b_data,
+ bit, count);
+ ext4_mb_free_metadata(handle, &e4b, new_entry);
+ ext4_unlock_group(sb, block_group);
} else {
ext4_lock_group(sb, block_group);
+ /* need to update group_info->bb_free and bitmap
+ * with group lock held. generate_buddy look at
+ * them with group lock_held
+ */
+ mb_clear_bits(sb_bgl_lock(sbi, block_group), bitmap_bh->b_data,
+ bit, count);
mb_free_blocks(inode, &e4b, bit, count);
ext4_mb_return_to_preallocation(inode, &e4b, block, count);
ext4_unlock_group(sb, block_group);
}
spin_lock(sb_bgl_lock(sbi, block_group));
- le16_add_cpu(&gdp->bg_free_blocks_count, count);
+ ret = ext4_free_blks_count(sb, gdp) + count;
+ ext4_free_blks_set(sb, gdp, ret);
gdp->bg_checksum = ext4_group_desc_csum(sbi, block_group, gdp);
spin_unlock(sb_bgl_lock(sbi, block_group));
percpu_counter_add(&sbi->s_freeblocks_counter, count);
@@ -4635,9 +4926,13 @@
*freed += count;
+ /* We dirtied the bitmap block */
+ BUFFER_TRACE(bitmap_bh, "dirtied bitmap block");
+ err = ext4_handle_dirty_metadata(handle, NULL, bitmap_bh);
+
/* And the group descriptor block */
BUFFER_TRACE(gd_bh, "dirtied group descriptor block");
- ret = ext4_journal_dirty_metadata(handle, gd_bh);
+ ret = ext4_handle_dirty_metadata(handle, NULL, gd_bh);
if (!err)
err = ret;
diff --git a/fs/ext4/mballoc.h b/fs/ext4/mballoc.h
index b5dff1f..10a2921 100644
--- a/fs/ext4/mballoc.h
+++ b/fs/ext4/mballoc.h
@@ -20,6 +20,7 @@
#include <linux/version.h>
#include <linux/blkdev.h>
#include <linux/marker.h>
+#include <linux/mutex.h>
#include "ext4_jbd2.h"
#include "ext4.h"
#include "group.h"
@@ -98,9 +99,6 @@
*/
#define MB_DEFAULT_GROUP_PREALLOC 512
-static struct kmem_cache *ext4_pspace_cachep;
-static struct kmem_cache *ext4_ac_cachep;
-static struct kmem_cache *ext4_free_ext_cachep;
struct ext4_free_data {
/* this links the free block information from group_info */
@@ -120,26 +118,6 @@
tid_t t_tid;
};
-struct ext4_group_info {
- unsigned long bb_state;
- struct rb_root bb_free_root;
- unsigned short bb_first_free;
- unsigned short bb_free;
- unsigned short bb_fragments;
- struct list_head bb_prealloc_list;
-#ifdef DOUBLE_CHECK
- void *bb_bitmap;
-#endif
- unsigned short bb_counters[];
-};
-
-#define EXT4_GROUP_INFO_NEED_INIT_BIT 0
-#define EXT4_GROUP_INFO_LOCKED_BIT 1
-
-#define EXT4_MB_GRP_NEED_INIT(grp) \
- (test_bit(EXT4_GROUP_INFO_NEED_INIT_BIT, &((grp)->bb_state)))
-
-
struct ext4_prealloc_space {
struct list_head pa_inode_list;
struct list_head pa_group_list;
@@ -217,6 +195,11 @@
__u8 ac_op; /* operation, for history only */
struct page *ac_bitmap_page;
struct page *ac_buddy_page;
+ /*
+ * pointer to the held semaphore upon successful
+ * block allocation
+ */
+ struct rw_semaphore *alloc_semp;
struct ext4_prealloc_space *ac_pa;
struct ext4_locality_group *ac_lg;
};
@@ -250,6 +233,7 @@
struct super_block *bd_sb;
__u16 bd_blkbits;
ext4_group_t bd_group;
+ struct rw_semaphore *alloc_semp;
};
#define EXT4_MB_BITMAP(e4b) ((e4b)->bd_bitmap)
#define EXT4_MB_BUDDY(e4b) ((e4b)->bd_buddy)
@@ -259,51 +243,12 @@
{
return;
}
-#else
-static void ext4_mb_store_history(struct ext4_allocation_context *ac);
#endif
#define in_range(b, first, len) ((b) >= (first) && (b) <= (first) + (len) - 1)
struct buffer_head *read_block_bitmap(struct super_block *, ext4_group_t);
-
-static void ext4_mb_generate_from_pa(struct super_block *sb, void *bitmap,
- ext4_group_t group);
-static void ext4_mb_return_to_preallocation(struct inode *inode,
- struct ext4_buddy *e4b, sector_t block,
- int count);
-static void ext4_mb_put_pa(struct ext4_allocation_context *,
- struct super_block *, struct ext4_prealloc_space *pa);
-static int ext4_mb_init_per_dev_proc(struct super_block *sb);
-static int ext4_mb_destroy_per_dev_proc(struct super_block *sb);
-static void release_blocks_on_commit(journal_t *journal, transaction_t *txn);
-
-
-static inline void ext4_lock_group(struct super_block *sb, ext4_group_t group)
-{
- struct ext4_group_info *grinfo = ext4_get_group_info(sb, group);
-
- bit_spin_lock(EXT4_GROUP_INFO_LOCKED_BIT, &(grinfo->bb_state));
-}
-
-static inline void ext4_unlock_group(struct super_block *sb,
- ext4_group_t group)
-{
- struct ext4_group_info *grinfo = ext4_get_group_info(sb, group);
-
- bit_spin_unlock(EXT4_GROUP_INFO_LOCKED_BIT, &(grinfo->bb_state));
-}
-
-static inline int ext4_is_group_locked(struct super_block *sb,
- ext4_group_t group)
-{
- struct ext4_group_info *grinfo = ext4_get_group_info(sb, group);
-
- return bit_spin_is_locked(EXT4_GROUP_INFO_LOCKED_BIT,
- &(grinfo->bb_state));
-}
-
-static ext4_fsblk_t ext4_grp_offs_to_block(struct super_block *sb,
+static inline ext4_fsblk_t ext4_grp_offs_to_block(struct super_block *sb,
struct ext4_free_extent *fex)
{
ext4_fsblk_t block;
diff --git a/fs/ext4/migrate.c b/fs/ext4/migrate.c
index f2a9cf4..734abca 100644
--- a/fs/ext4/migrate.c
+++ b/fs/ext4/migrate.c
@@ -59,7 +59,8 @@
/*
* Make sure the credit we accumalated is not really high
*/
- if (needed && handle->h_buffer_credits >= EXT4_RESERVE_TRANS_BLOCKS) {
+ if (needed && ext4_handle_has_enough_credits(handle,
+ EXT4_RESERVE_TRANS_BLOCKS)) {
retval = ext4_journal_restart(handle, needed);
if (retval)
goto err_out;
@@ -229,7 +230,7 @@
{
int retval = 0, needed;
- if (handle->h_buffer_credits > EXT4_RESERVE_TRANS_BLOCKS)
+ if (ext4_handle_has_enough_credits(handle, EXT4_RESERVE_TRANS_BLOCKS+1))
return 0;
/*
* We are freeing a blocks. During this we touch
@@ -458,13 +459,13 @@
struct list_blocks_struct lb;
unsigned long max_entries;
- if (!test_opt(inode->i_sb, EXTENTS))
- /*
- * if mounted with noextents we don't allow the migrate
- */
- return -EINVAL;
-
- if ((EXT4_I(inode)->i_flags & EXT4_EXTENTS_FL))
+ /*
+ * If the filesystem does not support extents, or the inode
+ * already is extent-based, error out.
+ */
+ if (!EXT4_HAS_INCOMPAT_FEATURE(inode->i_sb,
+ EXT4_FEATURE_INCOMPAT_EXTENTS) ||
+ (EXT4_I(inode)->i_flags & EXT4_EXTENTS_FL))
return -EINVAL;
if (S_ISLNK(inode->i_mode) && inode->i_blocks == 0)
diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c
index 9fd2a5e..fec0b4c 100644
--- a/fs/ext4/namei.c
+++ b/fs/ext4/namei.c
@@ -74,10 +74,6 @@
#define assert(test) J_ASSERT(test)
#endif
-#ifndef swap
-#define swap(x, y) do { typeof(x) z = x; x = y; y = z; } while (0)
-#endif
-
#ifdef DX_DEBUG
#define dxtrace(command) command
#else
@@ -372,6 +368,8 @@
goto fail;
}
hinfo->hash_version = root->info.hash_version;
+ if (hinfo->hash_version <= DX_HASH_TEA)
+ hinfo->hash_version += EXT4_SB(dir->i_sb)->s_hash_unsigned;
hinfo->seed = EXT4_SB(dir->i_sb)->s_hash_seed;
if (d_name)
ext4fs_dirhash(d_name->name, d_name->len, hinfo);
@@ -641,6 +639,9 @@
dir = dir_file->f_path.dentry->d_inode;
if (!(EXT4_I(dir)->i_flags & EXT4_INDEX_FL)) {
hinfo.hash_version = EXT4_SB(dir->i_sb)->s_def_hash_version;
+ if (hinfo.hash_version <= DX_HASH_TEA)
+ hinfo.hash_version +=
+ EXT4_SB(dir->i_sb)->s_hash_unsigned;
hinfo.seed = EXT4_SB(dir->i_sb)->s_hash_seed;
count = htree_dirblock_to_tree(dir_file, dir, 0, &hinfo,
start_hash, start_minor_hash);
@@ -806,7 +807,7 @@
static inline int search_dirblock(struct buffer_head *bh,
struct inode *dir,
const struct qstr *d_name,
- unsigned long offset,
+ unsigned int offset,
struct ext4_dir_entry_2 ** res_dir)
{
struct ext4_dir_entry_2 * de;
@@ -1043,11 +1044,11 @@
bh = ext4_find_entry(dir, &dentry->d_name, &de);
inode = NULL;
if (bh) {
- unsigned long ino = le32_to_cpu(de->inode);
+ __u32 ino = le32_to_cpu(de->inode);
brelse(bh);
if (!ext4_valid_inum(dir->i_sb, ino)) {
ext4_error(dir->i_sb, "ext4_lookup",
- "bad inode number: %lu", ino);
+ "bad inode number: %u", ino);
return ERR_PTR(-EIO);
}
inode = ext4_iget(dir->i_sb, ino);
@@ -1060,7 +1061,7 @@
struct dentry *ext4_get_parent(struct dentry *child)
{
- unsigned long ino;
+ __u32 ino;
struct inode *inode;
static const struct qstr dotdot = {
.name = "..",
@@ -1078,7 +1079,7 @@
if (!ext4_valid_inum(child->d_inode->i_sb, ino)) {
ext4_error(child->d_inode->i_sb, "ext4_get_parent",
- "bad inode number: %lu", ino);
+ "bad inode number: %u", ino);
return ERR_PTR(-EIO);
}
@@ -1166,9 +1167,9 @@
u32 hash2;
struct dx_map_entry *map;
char *data1 = (*bh)->b_data, *data2;
- unsigned split, move, size, i;
+ unsigned split, move, size;
struct ext4_dir_entry_2 *de = NULL, *de2;
- int err = 0;
+ int err = 0, i;
bh2 = ext4_append (handle, dir, &newblock, &err);
if (!(bh2)) {
@@ -1228,10 +1229,10 @@
de = de2;
}
dx_insert_block(frame, hash2 + continued, newblock);
- err = ext4_journal_dirty_metadata(handle, bh2);
+ err = ext4_handle_dirty_metadata(handle, dir, bh2);
if (err)
goto journal_error;
- err = ext4_journal_dirty_metadata(handle, frame->bh);
+ err = ext4_handle_dirty_metadata(handle, dir, frame->bh);
if (err)
goto journal_error;
brelse(bh2);
@@ -1266,7 +1267,7 @@
struct inode *dir = dentry->d_parent->d_inode;
const char *name = dentry->d_name.name;
int namelen = dentry->d_name.len;
- unsigned long offset = 0;
+ unsigned int offset = 0;
unsigned short reclen;
int nlen, rlen, err;
char *top;
@@ -1335,8 +1336,8 @@
ext4_update_dx_flag(dir);
dir->i_version++;
ext4_mark_inode_dirty(handle, dir);
- BUFFER_TRACE(bh, "call ext4_journal_dirty_metadata");
- err = ext4_journal_dirty_metadata(handle, bh);
+ BUFFER_TRACE(bh, "call ext4_handle_dirty_metadata");
+ err = ext4_handle_dirty_metadata(handle, dir, bh);
if (err)
ext4_std_error(dir->i_sb, err);
brelse(bh);
@@ -1408,6 +1409,8 @@
/* Initialize as for dx_probe */
hinfo.hash_version = root->info.hash_version;
+ if (hinfo.hash_version <= DX_HASH_TEA)
+ hinfo.hash_version += EXT4_SB(dir->i_sb)->s_hash_unsigned;
hinfo.seed = EXT4_SB(dir->i_sb)->s_hash_seed;
ext4fs_dirhash(name, namelen, &hinfo);
frame = frames;
@@ -1437,7 +1440,6 @@
struct inode *inode)
{
struct inode *dir = dentry->d_parent->d_inode;
- unsigned long offset;
struct buffer_head *bh;
struct ext4_dir_entry_2 *de;
struct super_block *sb;
@@ -1459,7 +1461,7 @@
ext4_mark_inode_dirty(handle, dir);
}
blocks = dir->i_size >> sb->s_blocksize_bits;
- for (block = 0, offset = 0; block < blocks; block++) {
+ for (block = 0; block < blocks; block++) {
bh = ext4_bread(handle, dir, block, 0, &retval);
if(!bh)
return retval;
@@ -1574,7 +1576,7 @@
dxtrace(dx_show_index("node", frames[1].entries));
dxtrace(dx_show_index("node",
((struct dx_node *) bh2->b_data)->entries));
- err = ext4_journal_dirty_metadata(handle, bh2);
+ err = ext4_handle_dirty_metadata(handle, inode, bh2);
if (err)
goto journal_error;
brelse (bh2);
@@ -1600,7 +1602,7 @@
if (err)
goto journal_error;
}
- ext4_journal_dirty_metadata(handle, frames[0].bh);
+ ext4_handle_dirty_metadata(handle, inode, frames[0].bh);
}
de = do_split(handle, dir, &bh, frame, &hinfo, &err);
if (!de)
@@ -1646,8 +1648,8 @@
else
de->inode = 0;
dir->i_version++;
- BUFFER_TRACE(bh, "call ext4_journal_dirty_metadata");
- ext4_journal_dirty_metadata(handle, bh);
+ BUFFER_TRACE(bh, "call ext4_handle_dirty_metadata");
+ ext4_handle_dirty_metadata(handle, dir, bh);
return 0;
}
i += ext4_rec_len_from_disk(de->rec_len);
@@ -1725,7 +1727,7 @@
return PTR_ERR(handle);
if (IS_DIRSYNC(dir))
- handle->h_sync = 1;
+ ext4_handle_sync(handle);
inode = ext4_new_inode (handle, dir, mode);
err = PTR_ERR(inode);
@@ -1759,7 +1761,7 @@
return PTR_ERR(handle);
if (IS_DIRSYNC(dir))
- handle->h_sync = 1;
+ ext4_handle_sync(handle);
inode = ext4_new_inode(handle, dir, mode);
err = PTR_ERR(inode);
@@ -1795,7 +1797,7 @@
return PTR_ERR(handle);
if (IS_DIRSYNC(dir))
- handle->h_sync = 1;
+ ext4_handle_sync(handle);
inode = ext4_new_inode(handle, dir, S_IFDIR | mode);
err = PTR_ERR(inode);
@@ -1824,8 +1826,8 @@
strcpy(de->name, "..");
ext4_set_de_type(dir->i_sb, de, S_IFDIR);
inode->i_nlink = 2;
- BUFFER_TRACE(dir_block, "call ext4_journal_dirty_metadata");
- ext4_journal_dirty_metadata(handle, dir_block);
+ BUFFER_TRACE(dir_block, "call ext4_handle_dirty_metadata");
+ ext4_handle_dirty_metadata(handle, dir, dir_block);
brelse(dir_block);
ext4_mark_inode_dirty(handle, inode);
err = ext4_add_entry(handle, dentry, inode);
@@ -1854,7 +1856,7 @@
*/
static int empty_dir(struct inode *inode)
{
- unsigned long offset;
+ unsigned int offset;
struct buffer_head *bh;
struct ext4_dir_entry_2 *de, *de1;
struct super_block *sb;
@@ -1899,7 +1901,7 @@
if (err)
ext4_error(sb, __func__,
"error %d reading directory"
- " #%lu offset %lu",
+ " #%lu offset %u",
err, inode->i_ino, offset);
offset += sb->s_blocksize;
continue;
@@ -1937,6 +1939,9 @@
struct ext4_iloc iloc;
int err = 0, rc;
+ if (!ext4_handle_valid(handle))
+ return 0;
+
lock_super(sb);
if (!list_empty(&EXT4_I(inode)->i_orphan))
goto out_unlock;
@@ -1965,7 +1970,7 @@
/* Insert this inode at the head of the on-disk orphan list... */
NEXT_ORPHAN(inode) = le32_to_cpu(EXT4_SB(sb)->s_es->s_last_orphan);
EXT4_SB(sb)->s_es->s_last_orphan = cpu_to_le32(inode->i_ino);
- err = ext4_journal_dirty_metadata(handle, EXT4_SB(sb)->s_sbh);
+ err = ext4_handle_dirty_metadata(handle, inode, EXT4_SB(sb)->s_sbh);
rc = ext4_mark_iloc_dirty(handle, inode, &iloc);
if (!err)
err = rc;
@@ -1999,10 +2004,13 @@
struct list_head *prev;
struct ext4_inode_info *ei = EXT4_I(inode);
struct ext4_sb_info *sbi;
- unsigned long ino_next;
+ __u32 ino_next;
struct ext4_iloc iloc;
int err = 0;
+ if (!ext4_handle_valid(handle))
+ return 0;
+
lock_super(inode->i_sb);
if (list_empty(&ei->i_orphan)) {
unlock_super(inode->i_sb);
@@ -2021,7 +2029,7 @@
* transaction handle with which to update the orphan list on
* disk, but we still need to remove the inode from the linked
* list in memory. */
- if (!handle)
+ if (sbi->s_journal && !handle)
goto out;
err = ext4_reserve_inode_write(handle, inode, &iloc);
@@ -2029,19 +2037,19 @@
goto out_err;
if (prev == &sbi->s_orphan) {
- jbd_debug(4, "superblock will point to %lu\n", ino_next);
+ jbd_debug(4, "superblock will point to %u\n", ino_next);
BUFFER_TRACE(sbi->s_sbh, "get_write_access");
err = ext4_journal_get_write_access(handle, sbi->s_sbh);
if (err)
goto out_brelse;
sbi->s_es->s_last_orphan = cpu_to_le32(ino_next);
- err = ext4_journal_dirty_metadata(handle, sbi->s_sbh);
+ err = ext4_handle_dirty_metadata(handle, inode, sbi->s_sbh);
} else {
struct ext4_iloc iloc2;
struct inode *i_prev =
&list_entry(prev, struct ext4_inode_info, i_orphan)->vfs_inode;
- jbd_debug(4, "orphan inode %lu will point to %lu\n",
+ jbd_debug(4, "orphan inode %lu will point to %u\n",
i_prev->i_ino, ino_next);
err = ext4_reserve_inode_write(handle, i_prev, &iloc2);
if (err)
@@ -2086,7 +2094,7 @@
goto end_rmdir;
if (IS_DIRSYNC(dir))
- handle->h_sync = 1;
+ ext4_handle_sync(handle);
inode = dentry->d_inode;
@@ -2140,7 +2148,7 @@
return PTR_ERR(handle);
if (IS_DIRSYNC(dir))
- handle->h_sync = 1;
+ ext4_handle_sync(handle);
retval = -ENOENT;
bh = ext4_find_entry(dir, &dentry->d_name, &de);
@@ -2197,7 +2205,7 @@
return PTR_ERR(handle);
if (IS_DIRSYNC(dir))
- handle->h_sync = 1;
+ ext4_handle_sync(handle);
inode = ext4_new_inode(handle, dir, S_IFLNK|S_IRWXUGO);
err = PTR_ERR(inode);
@@ -2260,7 +2268,7 @@
return PTR_ERR(handle);
if (IS_DIRSYNC(dir))
- handle->h_sync = 1;
+ ext4_handle_sync(handle);
inode->i_ctime = ext4_current_time(inode);
ext4_inc_count(handle, inode);
@@ -2309,7 +2317,7 @@
return PTR_ERR(handle);
if (IS_DIRSYNC(old_dir) || IS_DIRSYNC(new_dir))
- handle->h_sync = 1;
+ ext4_handle_sync(handle);
old_bh = ext4_find_entry(old_dir, &old_dentry->d_name, &old_de);
/*
@@ -2363,8 +2371,8 @@
new_dir->i_ctime = new_dir->i_mtime =
ext4_current_time(new_dir);
ext4_mark_inode_dirty(handle, new_dir);
- BUFFER_TRACE(new_bh, "call ext4_journal_dirty_metadata");
- ext4_journal_dirty_metadata(handle, new_bh);
+ BUFFER_TRACE(new_bh, "call ext4_handle_dirty_metadata");
+ ext4_handle_dirty_metadata(handle, new_dir, new_bh);
brelse(new_bh);
new_bh = NULL;
}
@@ -2414,8 +2422,8 @@
BUFFER_TRACE(dir_bh, "get_write_access");
ext4_journal_get_write_access(handle, dir_bh);
PARENT_INO(dir_bh->b_data) = cpu_to_le32(new_dir->i_ino);
- BUFFER_TRACE(dir_bh, "call ext4_journal_dirty_metadata");
- ext4_journal_dirty_metadata(handle, dir_bh);
+ BUFFER_TRACE(dir_bh, "call ext4_handle_dirty_metadata");
+ ext4_handle_dirty_metadata(handle, old_dir, dir_bh);
ext4_dec_count(handle, old_dir);
if (new_inode) {
/* checked empty_dir above, can't have another parent,
diff --git a/fs/ext4/resize.c b/fs/ext4/resize.c
index b6ec184..c328be5 100644
--- a/fs/ext4/resize.c
+++ b/fs/ext4/resize.c
@@ -50,7 +50,7 @@
ext4_get_group_no_and_offset(sb, start, NULL, &offset);
if (group != sbi->s_groups_count)
ext4_warning(sb, __func__,
- "Cannot add at group %u (only %lu groups)",
+ "Cannot add at group %u (only %u groups)",
input->group, sbi->s_groups_count);
else if (offset != 0)
ext4_warning(sb, __func__, "Last group not full");
@@ -149,7 +149,7 @@
{
int err;
- if (handle->h_buffer_credits >= thresh)
+ if (ext4_handle_has_enough_credits(handle, thresh))
return 0;
err = ext4_journal_extend(handle, EXT4_MAX_TRANS_DATA);
@@ -232,7 +232,7 @@
memcpy(gdb->b_data, sbi->s_group_desc[i]->b_data, gdb->b_size);
set_buffer_uptodate(gdb);
unlock_buffer(gdb);
- ext4_journal_dirty_metadata(handle, gdb);
+ ext4_handle_dirty_metadata(handle, NULL, gdb);
ext4_set_bit(bit, bh->b_data);
brelse(gdb);
}
@@ -251,7 +251,7 @@
err = PTR_ERR(bh);
goto exit_bh;
}
- ext4_journal_dirty_metadata(handle, gdb);
+ ext4_handle_dirty_metadata(handle, NULL, gdb);
ext4_set_bit(bit, bh->b_data);
brelse(gdb);
}
@@ -276,7 +276,7 @@
err = PTR_ERR(it);
goto exit_bh;
}
- ext4_journal_dirty_metadata(handle, it);
+ ext4_handle_dirty_metadata(handle, NULL, it);
brelse(it);
ext4_set_bit(bit, bh->b_data);
}
@@ -284,11 +284,9 @@
if ((err = extend_or_restart_transaction(handle, 2, bh)))
goto exit_bh;
- mark_bitmap_end(input->blocks_count, EXT4_BLOCKS_PER_GROUP(sb),
- bh->b_data);
- ext4_journal_dirty_metadata(handle, bh);
+ mark_bitmap_end(input->blocks_count, sb->s_blocksize * 8, bh->b_data);
+ ext4_handle_dirty_metadata(handle, NULL, bh);
brelse(bh);
-
/* Mark unused entries in inode bitmap used */
ext4_debug("clear inode bitmap %#04llx (+%llu)\n",
input->inode_bitmap, input->inode_bitmap - start);
@@ -297,9 +295,9 @@
goto exit_journal;
}
- mark_bitmap_end(EXT4_INODES_PER_GROUP(sb), EXT4_BLOCKS_PER_GROUP(sb),
+ mark_bitmap_end(EXT4_INODES_PER_GROUP(sb), sb->s_blocksize * 8,
bh->b_data);
- ext4_journal_dirty_metadata(handle, bh);
+ ext4_handle_dirty_metadata(handle, NULL, bh);
exit_bh:
brelse(bh);
@@ -486,12 +484,12 @@
* reserved inode, and will become GDT blocks (primary and backup).
*/
data[gdb_num % EXT4_ADDR_PER_BLOCK(sb)] = 0;
- ext4_journal_dirty_metadata(handle, dind);
+ ext4_handle_dirty_metadata(handle, NULL, dind);
brelse(dind);
inode->i_blocks -= (gdbackups + 1) * sb->s_blocksize >> 9;
ext4_mark_iloc_dirty(handle, inode, &iloc);
memset((*primary)->b_data, 0, sb->s_blocksize);
- ext4_journal_dirty_metadata(handle, *primary);
+ ext4_handle_dirty_metadata(handle, NULL, *primary);
o_group_desc = EXT4_SB(sb)->s_group_desc;
memcpy(n_group_desc, o_group_desc,
@@ -502,7 +500,7 @@
kfree(o_group_desc);
le16_add_cpu(&es->s_reserved_gdt_blocks, -1);
- ext4_journal_dirty_metadata(handle, EXT4_SB(sb)->s_sbh);
+ ext4_handle_dirty_metadata(handle, NULL, EXT4_SB(sb)->s_sbh);
return 0;
@@ -618,7 +616,7 @@
primary[i]->b_blocknr, gdbackups,
blk + primary[i]->b_blocknr); */
data[gdbackups] = cpu_to_le32(blk + primary[i]->b_blocknr);
- err2 = ext4_journal_dirty_metadata(handle, primary[i]);
+ err2 = ext4_handle_dirty_metadata(handle, NULL, primary[i]);
if (!err)
err = err2;
}
@@ -676,7 +674,8 @@
struct buffer_head *bh;
/* Out of journal space, and can't get more - abort - so sad */
- if (handle->h_buffer_credits == 0 &&
+ if (ext4_handle_valid(handle) &&
+ handle->h_buffer_credits == 0 &&
ext4_journal_extend(handle, EXT4_MAX_TRANS_DATA) &&
(err = ext4_journal_restart(handle, EXT4_MAX_TRANS_DATA)))
break;
@@ -696,7 +695,7 @@
memset(bh->b_data + size, 0, rest);
set_buffer_uptodate(bh);
unlock_buffer(bh);
- ext4_journal_dirty_metadata(handle, bh);
+ ext4_handle_dirty_metadata(handle, NULL, bh);
brelse(bh);
}
if ((err2 = ext4_journal_stop(handle)) && !err)
@@ -715,7 +714,7 @@
exit_err:
if (err) {
ext4_warning(sb, __func__,
- "can't update backup for group %lu (err %d), "
+ "can't update backup for group %u (err %d), "
"forcing fsck on next reboot", group, err);
sbi->s_mount_state &= ~EXT4_VALID_FS;
sbi->s_es->s_state &= cpu_to_le16(~EXT4_VALID_FS);
@@ -747,6 +746,7 @@
struct inode *inode = NULL;
handle_t *handle;
int gdb_off, gdb_num;
+ int num_grp_locked = 0;
int err, err2;
gdb_num = input->group / EXT4_DESC_PER_BLOCK(sb);
@@ -761,13 +761,13 @@
if (ext4_blocks_count(es) + input->blocks_count <
ext4_blocks_count(es)) {
- ext4_warning(sb, __func__, "blocks_count overflow\n");
+ ext4_warning(sb, __func__, "blocks_count overflow");
return -EINVAL;
}
if (le32_to_cpu(es->s_inodes_count) + EXT4_INODES_PER_GROUP(sb) <
le32_to_cpu(es->s_inodes_count)) {
- ext4_warning(sb, __func__, "inodes_count overflow\n");
+ ext4_warning(sb, __func__, "inodes_count overflow");
return -EINVAL;
}
@@ -787,6 +787,7 @@
}
}
+
if ((err = verify_group_input(sb, input)))
goto exit_put;
@@ -855,6 +856,7 @@
* using the new disk blocks.
*/
+ num_grp_locked = ext4_mb_get_buddy_cache_lock(sb, input->group);
/* Update group descriptor block for new group */
gdp = (struct ext4_group_desc *)((char *)primary->b_data +
gdb_off * EXT4_DESC_SIZE(sb));
@@ -862,17 +864,20 @@
ext4_block_bitmap_set(sb, gdp, input->block_bitmap); /* LV FIXME */
ext4_inode_bitmap_set(sb, gdp, input->inode_bitmap); /* LV FIXME */
ext4_inode_table_set(sb, gdp, input->inode_table); /* LV FIXME */
- gdp->bg_free_blocks_count = cpu_to_le16(input->free_blocks_count);
- gdp->bg_free_inodes_count = cpu_to_le16(EXT4_INODES_PER_GROUP(sb));
+ ext4_free_blks_set(sb, gdp, input->free_blocks_count);
+ ext4_free_inodes_set(sb, gdp, EXT4_INODES_PER_GROUP(sb));
+ gdp->bg_flags |= cpu_to_le16(EXT4_BG_INODE_ZEROED);
gdp->bg_checksum = ext4_group_desc_csum(sbi, input->group, gdp);
/*
* We can allocate memory for mb_alloc based on the new group
* descriptor
*/
- err = ext4_mb_add_more_groupinfo(sb, input->group, gdp);
- if (err)
+ err = ext4_mb_add_groupinfo(sb, input->group, gdp);
+ if (err) {
+ ext4_mb_put_buddy_cache_lock(sb, input->group, num_grp_locked);
goto exit_journal;
+ }
/*
* Make the new blocks and inodes valid next. We do this before
@@ -914,8 +919,9 @@
/* Update the global fs size fields */
sbi->s_groups_count++;
+ ext4_mb_put_buddy_cache_lock(sb, input->group, num_grp_locked);
- ext4_journal_dirty_metadata(handle, primary);
+ ext4_handle_dirty_metadata(handle, NULL, primary);
/* Update the reserved block counts only once the new group is
* active. */
@@ -937,7 +943,7 @@
EXT4_INODES_PER_GROUP(sb);
}
- ext4_journal_dirty_metadata(handle, sbi->s_sbh);
+ ext4_handle_dirty_metadata(handle, NULL, sbi->s_sbh);
sb->s_dirt = 1;
exit_journal:
@@ -975,9 +981,7 @@
struct buffer_head *bh;
handle_t *handle;
int err;
- unsigned long freed_blocks;
ext4_group_t group;
- struct ext4_group_info *grp;
/* We don't need to worry about locking wrt other resizers just
* yet: we're going to revalidate es->s_blocks_count after
@@ -997,8 +1001,7 @@
" too large to resize to %llu blocks safely\n",
sb->s_id, n_blocks_count);
if (sizeof(sector_t) < 8)
- ext4_warning(sb, __func__,
- "CONFIG_LBD not enabled\n");
+ ext4_warning(sb, __func__, "CONFIG_LBD not enabled");
return -EINVAL;
}
@@ -1071,62 +1074,18 @@
goto exit_put;
}
ext4_blocks_count_set(es, o_blocks_count + add);
- ext4_journal_dirty_metadata(handle, EXT4_SB(sb)->s_sbh);
+ ext4_handle_dirty_metadata(handle, NULL, EXT4_SB(sb)->s_sbh);
sb->s_dirt = 1;
unlock_super(sb);
ext4_debug("freeing blocks %llu through %llu\n", o_blocks_count,
o_blocks_count + add);
- ext4_free_blocks_sb(handle, sb, o_blocks_count, add, &freed_blocks);
+ /* We add the blocks to the bitmap and set the group need init bit */
+ ext4_add_groupblocks(handle, sb, o_blocks_count, add);
ext4_debug("freed blocks %llu through %llu\n", o_blocks_count,
o_blocks_count + add);
if ((err = ext4_journal_stop(handle)))
goto exit_put;
- /*
- * Mark mballoc pages as not up to date so that they will be updated
- * next time they are loaded by ext4_mb_load_buddy.
- *
- * XXX Bad, Bad, BAD!!! We should not be overloading the
- * Uptodate flag, particularly on thte bitmap bh, as way of
- * hinting to ext4_mb_load_buddy() that it needs to be
- * overloaded. A user could take a LVM snapshot, then do an
- * on-line fsck, and clear the uptodate flag, and this would
- * not be a bug in userspace, but a bug in the kernel. FIXME!!!
- */
- {
- struct ext4_sb_info *sbi = EXT4_SB(sb);
- struct inode *inode = sbi->s_buddy_cache;
- int blocks_per_page;
- int block;
- int pnum;
- struct page *page;
-
- /* Set buddy page as not up to date */
- blocks_per_page = PAGE_CACHE_SIZE / sb->s_blocksize;
- block = group * 2;
- pnum = block / blocks_per_page;
- page = find_get_page(inode->i_mapping, pnum);
- if (page != NULL) {
- ClearPageUptodate(page);
- page_cache_release(page);
- }
-
- /* Set bitmap page as not up to date */
- block++;
- pnum = block / blocks_per_page;
- page = find_get_page(inode->i_mapping, pnum);
- if (page != NULL) {
- ClearPageUptodate(page);
- page_cache_release(page);
- }
-
- /* Get the info on the last group */
- grp = ext4_get_group_info(sb, group);
-
- /* Update free blocks in group info */
- ext4_mb_update_group_info(grp, add);
- }
-
if (test_opt(sb, DEBUG))
printk(KERN_DEBUG "EXT4-fs: extended group to %llu blocks\n",
ext4_blocks_count(es));
diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index 9494bb2..8f7e0be 100644
--- a/fs/ext4/super.c
+++ b/fs/ext4/super.c
@@ -51,8 +51,6 @@
static int ext4_load_journal(struct super_block *, struct ext4_super_block *,
unsigned long journal_devnum);
-static int ext4_create_journal(struct super_block *, struct ext4_super_block *,
- unsigned int);
static void ext4_commit_super(struct super_block *sb,
struct ext4_super_block *es, int sync);
static void ext4_mark_recovery_complete(struct super_block *sb,
@@ -93,6 +91,38 @@
(ext4_fsblk_t)le32_to_cpu(bg->bg_inode_table_hi) << 32 : 0);
}
+__u32 ext4_free_blks_count(struct super_block *sb,
+ struct ext4_group_desc *bg)
+{
+ return le16_to_cpu(bg->bg_free_blocks_count_lo) |
+ (EXT4_DESC_SIZE(sb) >= EXT4_MIN_DESC_SIZE_64BIT ?
+ (__u32)le16_to_cpu(bg->bg_free_blocks_count_hi) << 16 : 0);
+}
+
+__u32 ext4_free_inodes_count(struct super_block *sb,
+ struct ext4_group_desc *bg)
+{
+ return le16_to_cpu(bg->bg_free_inodes_count_lo) |
+ (EXT4_DESC_SIZE(sb) >= EXT4_MIN_DESC_SIZE_64BIT ?
+ (__u32)le16_to_cpu(bg->bg_free_inodes_count_hi) << 16 : 0);
+}
+
+__u32 ext4_used_dirs_count(struct super_block *sb,
+ struct ext4_group_desc *bg)
+{
+ return le16_to_cpu(bg->bg_used_dirs_count_lo) |
+ (EXT4_DESC_SIZE(sb) >= EXT4_MIN_DESC_SIZE_64BIT ?
+ (__u32)le16_to_cpu(bg->bg_used_dirs_count_hi) << 16 : 0);
+}
+
+__u32 ext4_itable_unused_count(struct super_block *sb,
+ struct ext4_group_desc *bg)
+{
+ return le16_to_cpu(bg->bg_itable_unused_lo) |
+ (EXT4_DESC_SIZE(sb) >= EXT4_MIN_DESC_SIZE_64BIT ?
+ (__u32)le16_to_cpu(bg->bg_itable_unused_hi) << 16 : 0);
+}
+
void ext4_block_bitmap_set(struct super_block *sb,
struct ext4_group_desc *bg, ext4_fsblk_t blk)
{
@@ -117,6 +147,38 @@
bg->bg_inode_table_hi = cpu_to_le32(blk >> 32);
}
+void ext4_free_blks_set(struct super_block *sb,
+ struct ext4_group_desc *bg, __u32 count)
+{
+ bg->bg_free_blocks_count_lo = cpu_to_le16((__u16)count);
+ if (EXT4_DESC_SIZE(sb) >= EXT4_MIN_DESC_SIZE_64BIT)
+ bg->bg_free_blocks_count_hi = cpu_to_le16(count >> 16);
+}
+
+void ext4_free_inodes_set(struct super_block *sb,
+ struct ext4_group_desc *bg, __u32 count)
+{
+ bg->bg_free_inodes_count_lo = cpu_to_le16((__u16)count);
+ if (EXT4_DESC_SIZE(sb) >= EXT4_MIN_DESC_SIZE_64BIT)
+ bg->bg_free_inodes_count_hi = cpu_to_le16(count >> 16);
+}
+
+void ext4_used_dirs_set(struct super_block *sb,
+ struct ext4_group_desc *bg, __u32 count)
+{
+ bg->bg_used_dirs_count_lo = cpu_to_le16((__u16)count);
+ if (EXT4_DESC_SIZE(sb) >= EXT4_MIN_DESC_SIZE_64BIT)
+ bg->bg_used_dirs_count_hi = cpu_to_le16(count >> 16);
+}
+
+void ext4_itable_unused_set(struct super_block *sb,
+ struct ext4_group_desc *bg, __u32 count)
+{
+ bg->bg_itable_unused_lo = cpu_to_le16((__u16)count);
+ if (EXT4_DESC_SIZE(sb) >= EXT4_MIN_DESC_SIZE_64BIT)
+ bg->bg_itable_unused_hi = cpu_to_le16(count >> 16);
+}
+
/*
* Wrappers for jbd2_journal_start/end.
*
@@ -136,13 +198,19 @@
* backs (eg. EIO in the commit thread), then we still need to
* take the FS itself readonly cleanly. */
journal = EXT4_SB(sb)->s_journal;
- if (is_journal_aborted(journal)) {
- ext4_abort(sb, __func__,
- "Detected aborted journal");
- return ERR_PTR(-EROFS);
+ if (journal) {
+ if (is_journal_aborted(journal)) {
+ ext4_abort(sb, __func__,
+ "Detected aborted journal");
+ return ERR_PTR(-EROFS);
+ }
+ return jbd2_journal_start(journal, nblocks);
}
-
- return jbd2_journal_start(journal, nblocks);
+ /*
+ * We're not journaling, return the appropriate indication.
+ */
+ current->journal_info = EXT4_NOJOURNAL_HANDLE;
+ return current->journal_info;
}
/*
@@ -157,6 +225,14 @@
int err;
int rc;
+ if (!ext4_handle_valid(handle)) {
+ /*
+ * Do this here since we don't call jbd2_journal_stop() in
+ * no-journal mode.
+ */
+ current->journal_info = NULL;
+ return 0;
+ }
sb = handle->h_transaction->t_journal->j_private;
err = handle->h_err;
rc = jbd2_journal_stop(handle);
@@ -174,6 +250,8 @@
char nbuf[16];
const char *errstr = ext4_decode_error(NULL, err, nbuf);
+ BUG_ON(!ext4_handle_valid(handle));
+
if (bh)
BUFFER_TRACE(bh, "abort");
@@ -350,6 +428,44 @@
va_end(args);
}
+void ext4_grp_locked_error(struct super_block *sb, ext4_group_t grp,
+ const char *function, const char *fmt, ...)
+__releases(bitlock)
+__acquires(bitlock)
+{
+ va_list args;
+ struct ext4_super_block *es = EXT4_SB(sb)->s_es;
+
+ va_start(args, fmt);
+ printk(KERN_CRIT "EXT4-fs error (device %s): %s: ", sb->s_id, function);
+ vprintk(fmt, args);
+ printk("\n");
+ va_end(args);
+
+ if (test_opt(sb, ERRORS_CONT)) {
+ EXT4_SB(sb)->s_mount_state |= EXT4_ERROR_FS;
+ es->s_state |= cpu_to_le16(EXT4_ERROR_FS);
+ ext4_commit_super(sb, es, 0);
+ return;
+ }
+ ext4_unlock_group(sb, grp);
+ ext4_handle_error(sb);
+ /*
+ * We only get here in the ERRORS_RO case; relocking the group
+ * may be dangerous, but nothing bad will happen since the
+ * filesystem will have already been marked read/only and the
+ * journal has been aborted. We return 1 as a hint to callers
+ * who might what to use the return value from
+ * ext4_grp_locked_error() to distinguish beween the
+ * ERRORS_CONT and ERRORS_RO case, and perhaps return more
+ * aggressively from the ext4 function in question, with a
+ * more appropriate error code.
+ */
+ ext4_lock_group(sb, grp);
+ return;
+}
+
+
void ext4_update_dynamic_rev(struct super_block *sb)
{
struct ext4_super_block *es = EXT4_SB(sb)->s_es;
@@ -389,7 +505,7 @@
return bdev;
fail:
- printk(KERN_ERR "EXT4: failed to open journal device %s: %ld\n",
+ printk(KERN_ERR "EXT4-fs: failed to open journal device %s: %ld\n",
__bdevname(dev, b), PTR_ERR(bdev));
return NULL;
}
@@ -448,11 +564,13 @@
ext4_mb_release(sb);
ext4_ext_release(sb);
ext4_xattr_put_super(sb);
- err = jbd2_journal_destroy(sbi->s_journal);
- sbi->s_journal = NULL;
- if (err < 0)
- ext4_abort(sb, __func__, "Couldn't clean up the journal");
-
+ if (sbi->s_journal) {
+ err = jbd2_journal_destroy(sbi->s_journal);
+ sbi->s_journal = NULL;
+ if (err < 0)
+ ext4_abort(sb, __func__,
+ "Couldn't clean up the journal");
+ }
if (!(sb->s_flags & MS_RDONLY)) {
EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER);
es->s_state = cpu_to_le16(sbi->s_mount_state);
@@ -522,6 +640,11 @@
memset(&ei->i_cached_extent, 0, sizeof(struct ext4_ext_cache));
INIT_LIST_HEAD(&ei->i_prealloc_list);
spin_lock_init(&ei->i_prealloc_lock);
+ /*
+ * Note: We can be called before EXT4_SB(sb)->s_journal is set,
+ * therefore it can be null here. Don't check it, just initialize
+ * jinode.
+ */
jbd2_journal_init_jbd_inode(&ei->jinode, &ei->vfs_inode);
ei->i_reserved_data_blocks = 0;
ei->i_reserved_meta_blocks = 0;
@@ -588,7 +711,8 @@
}
#endif
ext4_discard_preallocations(inode);
- jbd2_journal_release_jbd_inode(EXT4_SB(inode->i_sb)->s_journal,
+ if (EXT4_JOURNAL(inode))
+ jbd2_journal_release_jbd_inode(EXT4_SB(inode->i_sb)->s_journal,
&EXT4_I(inode)->jinode);
}
@@ -681,10 +805,19 @@
#endif
if (!test_opt(sb, RESERVATION))
seq_puts(seq, ",noreservation");
- if (sbi->s_commit_interval) {
+ if (sbi->s_commit_interval != JBD2_DEFAULT_MAX_COMMIT_AGE*HZ) {
seq_printf(seq, ",commit=%u",
(unsigned) (sbi->s_commit_interval / HZ));
}
+ if (sbi->s_min_batch_time != EXT4_DEF_MIN_BATCH_TIME) {
+ seq_printf(seq, ",min_batch_time=%u",
+ (unsigned) sbi->s_min_batch_time);
+ }
+ if (sbi->s_max_batch_time != EXT4_DEF_MAX_BATCH_TIME) {
+ seq_printf(seq, ",max_batch_time=%u",
+ (unsigned) sbi->s_min_batch_time);
+ }
+
/*
* We're changing the default of barrier mount option, so
* let's always display its mount state so it's clear what its
@@ -696,8 +829,6 @@
seq_puts(seq, ",journal_async_commit");
if (test_opt(sb, NOBH))
seq_puts(seq, ",nobh");
- if (!test_opt(sb, EXTENTS))
- seq_puts(seq, ",noextents");
if (test_opt(sb, I_VERSION))
seq_puts(seq, ",i_version");
if (!test_opt(sb, DELALLOC))
@@ -772,6 +903,25 @@
ext4_nfs_get_inode);
}
+/*
+ * Try to release metadata pages (indirect blocks, directories) which are
+ * mapped via the block device. Since these pages could have journal heads
+ * which would prevent try_to_free_buffers() from freeing them, we must use
+ * jbd2 layer's try_to_free_buffers() function to release them.
+ */
+static int bdev_try_to_free_page(struct super_block *sb, struct page *page, gfp_t wait)
+{
+ journal_t *journal = EXT4_SB(sb)->s_journal;
+
+ WARN_ON(PageChecked(page));
+ if (!page_has_buffers(page))
+ return 0;
+ if (journal)
+ return jbd2_journal_try_to_free_buffers(journal, page,
+ wait & ~__GFP_WAIT);
+ return try_to_free_buffers(page);
+}
+
#ifdef CONFIG_QUOTA
#define QTYPE2NAME(t) ((t) == USRQUOTA ? "user" : "group")
#define QTYPE2MOPT(on, t) ((t) == USRQUOTA?((on)##USRJQUOTA):((on)##GRPJQUOTA))
@@ -838,6 +988,7 @@
.quota_read = ext4_quota_read,
.quota_write = ext4_quota_write,
#endif
+ .bdev_try_to_free_page = bdev_try_to_free_page,
};
static const struct export_operations ext4_export_ops = {
@@ -852,16 +1003,17 @@
Opt_nouid32, Opt_debug, Opt_oldalloc, Opt_orlov,
Opt_user_xattr, Opt_nouser_xattr, Opt_acl, Opt_noacl,
Opt_reservation, Opt_noreservation, Opt_noload, Opt_nobh, Opt_bh,
- Opt_commit, Opt_journal_update, Opt_journal_inum, Opt_journal_dev,
+ Opt_commit, Opt_min_batch_time, Opt_max_batch_time,
+ Opt_journal_update, Opt_journal_dev,
Opt_journal_checksum, Opt_journal_async_commit,
Opt_abort, Opt_data_journal, Opt_data_ordered, Opt_data_writeback,
Opt_data_err_abort, Opt_data_err_ignore,
Opt_usrjquota, Opt_grpjquota, Opt_offusrjquota, Opt_offgrpjquota,
Opt_jqfmt_vfsold, Opt_jqfmt_vfsv0, Opt_quota, Opt_noquota,
Opt_ignore, Opt_barrier, Opt_err, Opt_resize, Opt_usrquota,
- Opt_grpquota, Opt_extents, Opt_noextents, Opt_i_version,
+ Opt_grpquota, Opt_i_version,
Opt_stripe, Opt_delalloc, Opt_nodelalloc,
- Opt_inode_readahead_blks
+ Opt_inode_readahead_blks, Opt_journal_ioprio
};
static const match_table_t tokens = {
@@ -891,8 +1043,9 @@
{Opt_nobh, "nobh"},
{Opt_bh, "bh"},
{Opt_commit, "commit=%u"},
+ {Opt_min_batch_time, "min_batch_time=%u"},
+ {Opt_max_batch_time, "max_batch_time=%u"},
{Opt_journal_update, "journal=update"},
- {Opt_journal_inum, "journal=%u"},
{Opt_journal_dev, "journal_dev=%u"},
{Opt_journal_checksum, "journal_checksum"},
{Opt_journal_async_commit, "journal_async_commit"},
@@ -913,14 +1066,13 @@
{Opt_quota, "quota"},
{Opt_usrquota, "usrquota"},
{Opt_barrier, "barrier=%u"},
- {Opt_extents, "extents"},
- {Opt_noextents, "noextents"},
{Opt_i_version, "i_version"},
{Opt_stripe, "stripe=%u"},
{Opt_resize, "resize"},
{Opt_delalloc, "delalloc"},
{Opt_nodelalloc, "nodelalloc"},
{Opt_inode_readahead_blks, "inode_readahead_blks=%u"},
+ {Opt_journal_ioprio, "journal_ioprio=%u"},
{Opt_err, NULL},
};
@@ -945,8 +1097,11 @@
return sb_block;
}
+#define DEFAULT_JOURNAL_IOPRIO (IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, 3))
+
static int parse_options(char *options, struct super_block *sb,
- unsigned int *inum, unsigned long *journal_devnum,
+ unsigned long *journal_devnum,
+ unsigned int *journal_ioprio,
ext4_fsblk_t *n_blocks_count, int is_remount)
{
struct ext4_sb_info *sbi = EXT4_SB(sb);
@@ -958,7 +1113,6 @@
int qtype, qfmt;
char *qname;
#endif
- ext4_fsblk_t last_block;
if (!options)
return 1;
@@ -1070,16 +1224,6 @@
}
set_opt(sbi->s_mount_opt, UPDATE_JOURNAL);
break;
- case Opt_journal_inum:
- if (is_remount) {
- printk(KERN_ERR "EXT4-fs: cannot specify "
- "journal on remount\n");
- return 0;
- }
- if (match_int(&args[0], &option))
- return 0;
- *inum = option;
- break;
case Opt_journal_dev:
if (is_remount) {
printk(KERN_ERR "EXT4-fs: cannot specify "
@@ -1109,6 +1253,22 @@
option = JBD2_DEFAULT_MAX_COMMIT_AGE;
sbi->s_commit_interval = HZ * option;
break;
+ case Opt_max_batch_time:
+ if (match_int(&args[0], &option))
+ return 0;
+ if (option < 0)
+ return 0;
+ if (option == 0)
+ option = EXT4_DEF_MAX_BATCH_TIME;
+ sbi->s_max_batch_time = option;
+ break;
+ case Opt_min_batch_time:
+ if (match_int(&args[0], &option))
+ return 0;
+ if (option < 0)
+ return 0;
+ sbi->s_min_batch_time = option;
+ break;
case Opt_data_journal:
data_opt = EXT4_MOUNT_JOURNAL_DATA;
goto datacheck;
@@ -1279,33 +1439,6 @@
case Opt_bh:
clear_opt(sbi->s_mount_opt, NOBH);
break;
- case Opt_extents:
- if (!EXT4_HAS_INCOMPAT_FEATURE(sb,
- EXT4_FEATURE_INCOMPAT_EXTENTS)) {
- ext4_warning(sb, __func__,
- "extents feature not enabled "
- "on this filesystem, use tune2fs\n");
- return 0;
- }
- set_opt(sbi->s_mount_opt, EXTENTS);
- break;
- case Opt_noextents:
- /*
- * When e2fsprogs support resizing an already existing
- * ext3 file system to greater than 2**32 we need to
- * add support to block allocator to handle growing
- * already existing block mapped inode so that blocks
- * allocated for them fall within 2**32
- */
- last_block = ext4_blocks_count(sbi->s_es) - 1;
- if (last_block > 0xffffffffULL) {
- printk(KERN_ERR "EXT4-fs: Filesystem too "
- "large to mount with "
- "-o noextents options\n");
- return 0;
- }
- clear_opt(sbi->s_mount_opt, EXTENTS);
- break;
case Opt_i_version:
set_opt(sbi->s_mount_opt, I_VERSION);
sb->s_flags |= MS_I_VERSION;
@@ -1330,6 +1463,14 @@
return 0;
sbi->s_inode_readahead_blks = option;
break;
+ case Opt_journal_ioprio:
+ if (match_int(&args[0], &option))
+ return 0;
+ if (option < 0 || option > 7)
+ break;
+ *journal_ioprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE,
+ option);
+ break;
default:
printk(KERN_ERR
"EXT4-fs: Unrecognized mount option \"%s\" "
@@ -1405,24 +1546,19 @@
printk(KERN_WARNING
"EXT4-fs warning: checktime reached, "
"running e2fsck is recommended\n");
-#if 0
- /* @@@ We _will_ want to clear the valid bit if we find
- * inconsistencies, to force a fsck at reboot. But for
- * a plain journaled filesystem we can keep it set as
- * valid forever! :)
- */
- es->s_state &= cpu_to_le16(~EXT4_VALID_FS);
-#endif
+ if (!sbi->s_journal)
+ es->s_state &= cpu_to_le16(~EXT4_VALID_FS);
if (!(__s16) le16_to_cpu(es->s_max_mnt_count))
es->s_max_mnt_count = cpu_to_le16(EXT4_DFL_MAX_MNT_COUNT);
le16_add_cpu(&es->s_mnt_count, 1);
es->s_mtime = cpu_to_le32(get_seconds());
ext4_update_dynamic_rev(sb);
- EXT4_SET_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER);
+ if (sbi->s_journal)
+ EXT4_SET_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER);
ext4_commit_super(sb, es, 1);
if (test_opt(sb, DEBUG))
- printk(KERN_INFO "[EXT4 FS bs=%lu, gc=%lu, "
+ printk(KERN_INFO "[EXT4 FS bs=%lu, gc=%u, "
"bpg=%lu, ipg=%lu, mo=%04lx]\n",
sb->s_blocksize,
sbi->s_groups_count,
@@ -1430,9 +1566,13 @@
EXT4_INODES_PER_GROUP(sb),
sbi->s_mount_opt);
- printk(KERN_INFO "EXT4 FS on %s, %s journal on %s\n",
- sb->s_id, EXT4_SB(sb)->s_journal->j_inode ? "internal" :
- "external", EXT4_SB(sb)->s_journal->j_devname);
+ if (EXT4_SB(sb)->s_journal) {
+ printk(KERN_INFO "EXT4 FS on %s, %s journal on %s\n",
+ sb->s_id, EXT4_SB(sb)->s_journal->j_inode ? "internal" :
+ "external", EXT4_SB(sb)->s_journal->j_devname);
+ } else {
+ printk(KERN_INFO "EXT4 FS on %s, no journal\n", sb->s_id);
+ }
return res;
}
@@ -1444,7 +1584,6 @@
ext4_group_t flex_group_count;
ext4_group_t flex_group;
int groups_per_flex = 0;
- __u64 block_bitmap = 0;
int i;
if (!sbi->s_es->s_log_groups_per_flex) {
@@ -1463,21 +1602,18 @@
sizeof(struct flex_groups), GFP_KERNEL);
if (sbi->s_flex_groups == NULL) {
printk(KERN_ERR "EXT4-fs: not enough memory for "
- "%lu flex groups\n", flex_group_count);
+ "%u flex groups\n", flex_group_count);
goto failed;
}
- gdp = ext4_get_group_desc(sb, 1, &bh);
- block_bitmap = ext4_block_bitmap(sb, gdp) - 1;
-
for (i = 0; i < sbi->s_groups_count; i++) {
gdp = ext4_get_group_desc(sb, i, &bh);
flex_group = ext4_flex_group(sbi, i);
sbi->s_flex_groups[flex_group].free_inodes +=
- le16_to_cpu(gdp->bg_free_inodes_count);
+ ext4_free_inodes_count(sb, gdp);
sbi->s_flex_groups[flex_group].free_blocks +=
- le16_to_cpu(gdp->bg_free_blocks_count);
+ ext4_free_blks_count(sb, gdp);
}
return 1;
@@ -1551,14 +1687,14 @@
block_bitmap = ext4_block_bitmap(sb, gdp);
if (block_bitmap < first_block || block_bitmap > last_block) {
printk(KERN_ERR "EXT4-fs: ext4_check_descriptors: "
- "Block bitmap for group %lu not in group "
+ "Block bitmap for group %u not in group "
"(block %llu)!\n", i, block_bitmap);
return 0;
}
inode_bitmap = ext4_inode_bitmap(sb, gdp);
if (inode_bitmap < first_block || inode_bitmap > last_block) {
printk(KERN_ERR "EXT4-fs: ext4_check_descriptors: "
- "Inode bitmap for group %lu not in group "
+ "Inode bitmap for group %u not in group "
"(block %llu)!\n", i, inode_bitmap);
return 0;
}
@@ -1566,14 +1702,14 @@
if (inode_table < first_block ||
inode_table + sbi->s_itb_per_group - 1 > last_block) {
printk(KERN_ERR "EXT4-fs: ext4_check_descriptors: "
- "Inode table for group %lu not in group "
+ "Inode table for group %u not in group "
"(block %llu)!\n", i, inode_table);
return 0;
}
spin_lock(sb_bgl_lock(sbi, i));
if (!ext4_group_desc_csum_verify(sbi, i, gdp)) {
printk(KERN_ERR "EXT4-fs: ext4_check_descriptors: "
- "Checksum for group %lu failed (%u!=%u)\n",
+ "Checksum for group %u failed (%u!=%u)\n",
i, le16_to_cpu(ext4_group_desc_csum(sbi, i,
gdp)), le16_to_cpu(gdp->bg_checksum));
if (!(sb->s_flags & MS_RDONLY)) {
@@ -1865,19 +2001,20 @@
ext4_fsblk_t sb_block = get_sb_block(&data);
ext4_fsblk_t logical_sb_block;
unsigned long offset = 0;
- unsigned int journal_inum = 0;
unsigned long journal_devnum = 0;
unsigned long def_mount_opts;
struct inode *root;
char *cp;
+ const char *descr;
int ret = -EINVAL;
int blocksize;
- int db_count;
- int i;
+ unsigned int db_count;
+ unsigned int i;
int needs_recovery, has_huge_files;
- __le32 features;
+ int features;
__u64 blocks_count;
int err;
+ unsigned int journal_ioprio = DEFAULT_JOURNAL_IOPRIO;
sbi = kzalloc(sizeof(*sbi), GFP_KERNEL);
if (!sbi)
@@ -1958,31 +2095,22 @@
sbi->s_resuid = le16_to_cpu(es->s_def_resuid);
sbi->s_resgid = le16_to_cpu(es->s_def_resgid);
+ sbi->s_commit_interval = JBD2_DEFAULT_MAX_COMMIT_AGE * HZ;
+ sbi->s_min_batch_time = EXT4_DEF_MIN_BATCH_TIME;
+ sbi->s_max_batch_time = EXT4_DEF_MAX_BATCH_TIME;
set_opt(sbi->s_mount_opt, RESERVATION);
set_opt(sbi->s_mount_opt, BARRIER);
/*
- * turn on extents feature by default in ext4 filesystem
- * only if feature flag already set by mkfs or tune2fs.
- * Use -o noextents to turn it off
- */
- if (EXT4_HAS_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_EXTENTS))
- set_opt(sbi->s_mount_opt, EXTENTS);
- else
- ext4_warning(sb, __func__,
- "extents feature not enabled on this filesystem, "
- "use tune2fs.\n");
-
- /*
* enable delayed allocation by default
* Use -o nodelalloc to turn it off
*/
set_opt(sbi->s_mount_opt, DELALLOC);
- if (!parse_options((char *) data, sb, &journal_inum, &journal_devnum,
- NULL, 0))
+ if (!parse_options((char *) data, sb, &journal_devnum,
+ &journal_ioprio, NULL, 0))
goto failed_mount;
sb->s_flags = (sb->s_flags & ~MS_POSIXACL) |
@@ -2004,15 +2132,17 @@
features = EXT4_HAS_INCOMPAT_FEATURE(sb, ~EXT4_FEATURE_INCOMPAT_SUPP);
if (features) {
printk(KERN_ERR "EXT4-fs: %s: couldn't mount because of "
- "unsupported optional features (%x).\n",
- sb->s_id, le32_to_cpu(features));
+ "unsupported optional features (%x).\n", sb->s_id,
+ (le32_to_cpu(EXT4_SB(sb)->s_es->s_feature_incompat) &
+ ~EXT4_FEATURE_INCOMPAT_SUPP));
goto failed_mount;
}
features = EXT4_HAS_RO_COMPAT_FEATURE(sb, ~EXT4_FEATURE_RO_COMPAT_SUPP);
if (!(sb->s_flags & MS_RDONLY) && features) {
printk(KERN_ERR "EXT4-fs: %s: couldn't mount RDWR because of "
- "unsupported optional features (%x).\n",
- sb->s_id, le32_to_cpu(features));
+ "unsupported optional features (%x).\n", sb->s_id,
+ (le32_to_cpu(EXT4_SB(sb)->s_es->s_feature_ro_compat) &
+ ~EXT4_FEATURE_RO_COMPAT_SUPP));
goto failed_mount;
}
has_huge_files = EXT4_HAS_RO_COMPAT_FEATURE(sb,
@@ -2117,6 +2247,18 @@
for (i = 0; i < 4; i++)
sbi->s_hash_seed[i] = le32_to_cpu(es->s_hash_seed[i]);
sbi->s_def_hash_version = es->s_def_hash_version;
+ i = le32_to_cpu(es->s_flags);
+ if (i & EXT2_FLAGS_UNSIGNED_HASH)
+ sbi->s_hash_unsigned = 3;
+ else if ((i & EXT2_FLAGS_SIGNED_HASH) == 0) {
+#ifdef __CHAR_UNSIGNED__
+ es->s_flags |= cpu_to_le32(EXT2_FLAGS_UNSIGNED_HASH);
+ sbi->s_hash_unsigned = 3;
+#else
+ es->s_flags |= cpu_to_le32(EXT2_FLAGS_SIGNED_HASH);
+#endif
+ sb->s_dirt = 1;
+ }
if (sbi->s_blocks_per_group > blocksize * 8) {
printk(KERN_ERR
@@ -2144,20 +2286,30 @@
if (EXT4_BLOCKS_PER_GROUP(sb) == 0)
goto cantfind_ext4;
- /* ensure blocks_count calculation below doesn't sign-extend */
- if (ext4_blocks_count(es) + EXT4_BLOCKS_PER_GROUP(sb) <
- le32_to_cpu(es->s_first_data_block) + 1) {
- printk(KERN_WARNING "EXT4-fs: bad geometry: block count %llu, "
- "first data block %u, blocks per group %lu\n",
- ext4_blocks_count(es),
- le32_to_cpu(es->s_first_data_block),
- EXT4_BLOCKS_PER_GROUP(sb));
+ /*
+ * It makes no sense for the first data block to be beyond the end
+ * of the filesystem.
+ */
+ if (le32_to_cpu(es->s_first_data_block) >= ext4_blocks_count(es)) {
+ printk(KERN_WARNING "EXT4-fs: bad geometry: first data"
+ "block %u is beyond end of filesystem (%llu)\n",
+ le32_to_cpu(es->s_first_data_block),
+ ext4_blocks_count(es));
goto failed_mount;
}
blocks_count = (ext4_blocks_count(es) -
le32_to_cpu(es->s_first_data_block) +
EXT4_BLOCKS_PER_GROUP(sb) - 1);
do_div(blocks_count, EXT4_BLOCKS_PER_GROUP(sb));
+ if (blocks_count > ((uint64_t)1<<32) - EXT4_DESC_PER_BLOCK(sb)) {
+ printk(KERN_WARNING "EXT4-fs: groups count too large: %u "
+ "(block count %llu, first data block %u, "
+ "blocks per group %lu)\n", sbi->s_groups_count,
+ ext4_blocks_count(es),
+ le32_to_cpu(es->s_first_data_block),
+ EXT4_BLOCKS_PER_GROUP(sb));
+ goto failed_mount;
+ }
sbi->s_groups_count = blocks_count;
db_count = (sbi->s_groups_count + EXT4_DESC_PER_BLOCK(sb) - 1) /
EXT4_DESC_PER_BLOCK(sb);
@@ -2269,27 +2421,26 @@
EXT4_SB(sb)->s_mount_state |= EXT4_ERROR_FS;
es->s_state |= cpu_to_le16(EXT4_ERROR_FS);
ext4_commit_super(sb, es, 1);
- printk(KERN_CRIT
- "EXT4-fs (device %s): mount failed\n",
- sb->s_id);
goto failed_mount4;
}
}
- } else if (journal_inum) {
- if (ext4_create_journal(sb, es, journal_inum))
- goto failed_mount3;
+ } else if (test_opt(sb, NOLOAD) && !(sb->s_flags & MS_RDONLY) &&
+ EXT4_HAS_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER)) {
+ printk(KERN_ERR "EXT4-fs: required journal recovery "
+ "suppressed and not mounted read-only\n");
+ goto failed_mount4;
} else {
- if (!silent)
- printk(KERN_ERR
- "ext4: No journal on filesystem on %s\n",
- sb->s_id);
- goto failed_mount3;
+ clear_opt(sbi->s_mount_opt, DATA_FLAGS);
+ set_opt(sbi->s_mount_opt, WRITEBACK_DATA);
+ sbi->s_journal = NULL;
+ needs_recovery = 0;
+ goto no_journal;
}
if (ext4_blocks_count(es) > 0xffffffffULL &&
!jbd2_journal_set_features(EXT4_SB(sb)->s_journal, 0, 0,
JBD2_FEATURE_INCOMPAT_64BIT)) {
- printk(KERN_ERR "ext4: Failed to set 64-bit journal feature\n");
+ printk(KERN_ERR "EXT4-fs: Failed to set 64-bit journal feature\n");
goto failed_mount4;
}
@@ -2334,6 +2485,9 @@
default:
break;
}
+ set_task_ioprio(sbi->s_journal->j_task, journal_ioprio);
+
+no_journal:
if (test_opt(sb, NOBH)) {
if (!(test_opt(sb, DATA_FLAGS) == EXT4_MOUNT_WRITEBACK_DATA)) {
@@ -2419,13 +2573,22 @@
EXT4_SB(sb)->s_mount_state |= EXT4_ORPHAN_FS;
ext4_orphan_cleanup(sb, es);
EXT4_SB(sb)->s_mount_state &= ~EXT4_ORPHAN_FS;
- if (needs_recovery)
+ if (needs_recovery) {
printk(KERN_INFO "EXT4-fs: recovery complete.\n");
- ext4_mark_recovery_complete(sb, es);
- printk(KERN_INFO "EXT4-fs: mounted filesystem with %s data mode.\n",
- test_opt(sb, DATA_FLAGS) == EXT4_MOUNT_JOURNAL_DATA ? "journal":
- test_opt(sb, DATA_FLAGS) == EXT4_MOUNT_ORDERED_DATA ? "ordered":
- "writeback");
+ ext4_mark_recovery_complete(sb, es);
+ }
+ if (EXT4_SB(sb)->s_journal) {
+ if (test_opt(sb, DATA_FLAGS) == EXT4_MOUNT_JOURNAL_DATA)
+ descr = " journalled data mode";
+ else if (test_opt(sb, DATA_FLAGS) == EXT4_MOUNT_ORDERED_DATA)
+ descr = " ordered data mode";
+ else
+ descr = " writeback data mode";
+ } else
+ descr = "out journal";
+
+ printk(KERN_INFO "EXT4-fs: mounted filesystem %s with%s\n",
+ sb->s_id, descr);
lock_kernel();
return 0;
@@ -2437,8 +2600,11 @@
goto failed_mount;
failed_mount4:
- jbd2_journal_destroy(sbi->s_journal);
- sbi->s_journal = NULL;
+ printk(KERN_ERR "EXT4-fs (device %s): mount failed\n", sb->s_id);
+ if (sbi->s_journal) {
+ jbd2_journal_destroy(sbi->s_journal);
+ sbi->s_journal = NULL;
+ }
failed_mount3:
percpu_counter_destroy(&sbi->s_freeblocks_counter);
percpu_counter_destroy(&sbi->s_freeinodes_counter);
@@ -2475,11 +2641,9 @@
{
struct ext4_sb_info *sbi = EXT4_SB(sb);
- if (sbi->s_commit_interval)
- journal->j_commit_interval = sbi->s_commit_interval;
- /* We could also set up an ext4-specific default for the commit
- * interval here, but for now we'll just fall back to the jbd
- * default. */
+ journal->j_commit_interval = sbi->s_commit_interval;
+ journal->j_min_batch_time = sbi->s_min_batch_time;
+ journal->j_max_batch_time = sbi->s_max_batch_time;
spin_lock(&journal->j_state_lock);
if (test_opt(sb, BARRIER))
@@ -2499,6 +2663,8 @@
struct inode *journal_inode;
journal_t *journal;
+ BUG_ON(!EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_HAS_JOURNAL));
+
/* First, test for the existence of a valid inode on disk. Bad
* things happen if we iget() an unused inode, as the subsequent
* iput() will try to delete it. */
@@ -2547,13 +2713,15 @@
struct ext4_super_block *es;
struct block_device *bdev;
+ BUG_ON(!EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_HAS_JOURNAL));
+
bdev = ext4_blkdev_get(j_dev);
if (bdev == NULL)
return NULL;
if (bd_claim(bdev, sb)) {
printk(KERN_ERR
- "EXT4: failed to claim external journal device.\n");
+ "EXT4-fs: failed to claim external journal device.\n");
blkdev_put(bdev, FMODE_READ|FMODE_WRITE);
return NULL;
}
@@ -2634,6 +2802,8 @@
int err = 0;
int really_read_only;
+ BUG_ON(!EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_HAS_JOURNAL));
+
if (journal_devnum &&
journal_devnum != le32_to_cpu(es->s_journal_dev)) {
printk(KERN_INFO "EXT4-fs: external journal device major/minor "
@@ -2718,48 +2888,6 @@
return 0;
}
-static int ext4_create_journal(struct super_block *sb,
- struct ext4_super_block *es,
- unsigned int journal_inum)
-{
- journal_t *journal;
- int err;
-
- if (sb->s_flags & MS_RDONLY) {
- printk(KERN_ERR "EXT4-fs: readonly filesystem when trying to "
- "create journal.\n");
- return -EROFS;
- }
-
- journal = ext4_get_journal(sb, journal_inum);
- if (!journal)
- return -EINVAL;
-
- printk(KERN_INFO "EXT4-fs: creating new journal on inode %u\n",
- journal_inum);
-
- err = jbd2_journal_create(journal);
- if (err) {
- printk(KERN_ERR "EXT4-fs: error creating journal.\n");
- jbd2_journal_destroy(journal);
- return -EIO;
- }
-
- EXT4_SB(sb)->s_journal = journal;
-
- ext4_update_dynamic_rev(sb);
- EXT4_SET_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER);
- EXT4_SET_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_HAS_JOURNAL);
-
- es->s_journal_inum = cpu_to_le32(journal_inum);
- sb->s_dirt = 1;
-
- /* Make sure we flush the recovery flag to disk. */
- ext4_commit_super(sb, es, 1);
-
- return 0;
-}
-
static void ext4_commit_super(struct super_block *sb,
struct ext4_super_block *es, int sync)
{
@@ -2776,20 +2904,23 @@
* be remapped. Nothing we can do but to retry the
* write and hope for the best.
*/
- printk(KERN_ERR "ext4: previous I/O error to "
+ printk(KERN_ERR "EXT4-fs: previous I/O error to "
"superblock detected for %s.\n", sb->s_id);
clear_buffer_write_io_error(sbh);
set_buffer_uptodate(sbh);
}
es->s_wtime = cpu_to_le32(get_seconds());
- ext4_free_blocks_count_set(es, ext4_count_free_blocks(sb));
- es->s_free_inodes_count = cpu_to_le32(ext4_count_free_inodes(sb));
+ ext4_free_blocks_count_set(es, percpu_counter_sum_positive(
+ &EXT4_SB(sb)->s_freeblocks_counter));
+ es->s_free_inodes_count = cpu_to_le32(percpu_counter_sum_positive(
+ &EXT4_SB(sb)->s_freeinodes_counter));
+
BUFFER_TRACE(sbh, "marking dirty");
mark_buffer_dirty(sbh);
if (sync) {
sync_dirty_buffer(sbh);
if (buffer_write_io_error(sbh)) {
- printk(KERN_ERR "ext4: I/O error while writing "
+ printk(KERN_ERR "EXT4-fs: I/O error while writing "
"superblock for %s.\n", sb->s_id);
clear_buffer_write_io_error(sbh);
set_buffer_uptodate(sbh);
@@ -2808,6 +2939,10 @@
{
journal_t *journal = EXT4_SB(sb)->s_journal;
+ if (!EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_HAS_JOURNAL)) {
+ BUG_ON(journal != NULL);
+ return;
+ }
jbd2_journal_lock_updates(journal);
if (jbd2_journal_flush(journal) < 0)
goto out;
@@ -2837,6 +2972,8 @@
int j_errno;
const char *errstr;
+ BUG_ON(!EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_HAS_JOURNAL));
+
journal = EXT4_SB(sb)->s_journal;
/*
@@ -2869,14 +3006,17 @@
int ext4_force_commit(struct super_block *sb)
{
journal_t *journal;
- int ret;
+ int ret = 0;
if (sb->s_flags & MS_RDONLY)
return 0;
journal = EXT4_SB(sb)->s_journal;
- sb->s_dirt = 0;
- ret = ext4_journal_force_commit(journal);
+ if (journal) {
+ sb->s_dirt = 0;
+ ret = ext4_journal_force_commit(journal);
+ }
+
return ret;
}
@@ -2888,9 +3028,13 @@
*/
static void ext4_write_super(struct super_block *sb)
{
- if (mutex_trylock(&sb->s_lock) != 0)
- BUG();
- sb->s_dirt = 0;
+ if (EXT4_SB(sb)->s_journal) {
+ if (mutex_trylock(&sb->s_lock) != 0)
+ BUG();
+ sb->s_dirt = 0;
+ } else {
+ ext4_commit_super(sb, EXT4_SB(sb)->s_es, 1);
+ }
}
static int ext4_sync_fs(struct super_block *sb, int wait)
@@ -2899,10 +3043,14 @@
trace_mark(ext4_sync_fs, "dev %s wait %d", sb->s_id, wait);
sb->s_dirt = 0;
- if (wait)
- ret = ext4_force_commit(sb);
- else
- jbd2_journal_start_commit(EXT4_SB(sb)->s_journal, NULL);
+ if (EXT4_SB(sb)->s_journal) {
+ if (wait)
+ ret = ext4_force_commit(sb);
+ else
+ jbd2_journal_start_commit(EXT4_SB(sb)->s_journal, NULL);
+ } else {
+ ext4_commit_super(sb, EXT4_SB(sb)->s_es, wait);
+ }
return ret;
}
@@ -2917,15 +3065,17 @@
if (!(sb->s_flags & MS_RDONLY)) {
journal_t *journal = EXT4_SB(sb)->s_journal;
- /* Now we set up the journal barrier. */
- jbd2_journal_lock_updates(journal);
+ if (journal) {
+ /* Now we set up the journal barrier. */
+ jbd2_journal_lock_updates(journal);
- /*
- * We don't want to clear needs_recovery flag when we failed
- * to flush the journal.
- */
- if (jbd2_journal_flush(journal) < 0)
- return;
+ /*
+ * We don't want to clear needs_recovery flag when we
+ * failed to flush the journal.
+ */
+ if (jbd2_journal_flush(journal) < 0)
+ return;
+ }
/* Journal blocked and flushed, clear needs_recovery flag. */
EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER);
@@ -2939,7 +3089,7 @@
*/
static void ext4_unlockfs(struct super_block *sb)
{
- if (!(sb->s_flags & MS_RDONLY)) {
+ if (EXT4_SB(sb)->s_journal && !(sb->s_flags & MS_RDONLY)) {
lock_super(sb);
/* Reser the needs_recovery flag before the fs is unlocked. */
EXT4_SET_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER);
@@ -2957,6 +3107,7 @@
unsigned long old_sb_flags;
struct ext4_mount_options old_opts;
ext4_group_t g;
+ unsigned int journal_ioprio = DEFAULT_JOURNAL_IOPRIO;
int err;
#ifdef CONFIG_QUOTA
int i;
@@ -2968,16 +3119,21 @@
old_opts.s_resuid = sbi->s_resuid;
old_opts.s_resgid = sbi->s_resgid;
old_opts.s_commit_interval = sbi->s_commit_interval;
+ old_opts.s_min_batch_time = sbi->s_min_batch_time;
+ old_opts.s_max_batch_time = sbi->s_max_batch_time;
#ifdef CONFIG_QUOTA
old_opts.s_jquota_fmt = sbi->s_jquota_fmt;
for (i = 0; i < MAXQUOTAS; i++)
old_opts.s_qf_names[i] = sbi->s_qf_names[i];
#endif
+ if (sbi->s_journal && sbi->s_journal->j_task->io_context)
+ journal_ioprio = sbi->s_journal->j_task->io_context->ioprio;
/*
* Allow the "check" option to be passed as a remount option.
*/
- if (!parse_options(data, sb, NULL, NULL, &n_blocks_count, 1)) {
+ if (!parse_options(data, sb, NULL, &journal_ioprio,
+ &n_blocks_count, 1)) {
err = -EINVAL;
goto restore_opts;
}
@@ -2990,7 +3146,10 @@
es = sbi->s_es;
- ext4_init_journal_params(sb, sbi->s_journal);
+ if (sbi->s_journal) {
+ ext4_init_journal_params(sb, sbi->s_journal);
+ set_task_ioprio(sbi->s_journal->j_task, journal_ioprio);
+ }
if ((*flags & MS_RDONLY) != (sb->s_flags & MS_RDONLY) ||
n_blocks_count > ext4_blocks_count(es)) {
@@ -3019,17 +3178,20 @@
* We have to unlock super so that we can wait for
* transactions.
*/
- unlock_super(sb);
- ext4_mark_recovery_complete(sb, es);
- lock_super(sb);
+ if (sbi->s_journal) {
+ unlock_super(sb);
+ ext4_mark_recovery_complete(sb, es);
+ lock_super(sb);
+ }
} else {
- __le32 ret;
+ int ret;
if ((ret = EXT4_HAS_RO_COMPAT_FEATURE(sb,
~EXT4_FEATURE_RO_COMPAT_SUPP))) {
printk(KERN_WARNING "EXT4-fs: %s: couldn't "
"remount RDWR because of unsupported "
- "optional features (%x).\n",
- sb->s_id, le32_to_cpu(ret));
+ "optional features (%x).\n", sb->s_id,
+ (le32_to_cpu(sbi->s_es->s_feature_ro_compat) &
+ ~EXT4_FEATURE_RO_COMPAT_SUPP));
err = -EROFS;
goto restore_opts;
}
@@ -3046,7 +3208,7 @@
if (!ext4_group_desc_csum_verify(sbi, g, gdp)) {
printk(KERN_ERR
"EXT4-fs: ext4_remount: "
- "Checksum for group %lu failed (%u!=%u)\n",
+ "Checksum for group %u failed (%u!=%u)\n",
g, le16_to_cpu(ext4_group_desc_csum(sbi, g, gdp)),
le16_to_cpu(gdp->bg_checksum));
err = -EINVAL;
@@ -3075,7 +3237,8 @@
* been changed by e2fsck since we originally mounted
* the partition.)
*/
- ext4_clear_journal_err(sb, es);
+ if (sbi->s_journal)
+ ext4_clear_journal_err(sb, es);
sbi->s_mount_state = le16_to_cpu(es->s_state);
if ((err = ext4_group_extend(sb, es, n_blocks_count)))
goto restore_opts;
@@ -3083,6 +3246,9 @@
sb->s_flags &= ~MS_RDONLY;
}
}
+ if (sbi->s_journal == NULL)
+ ext4_commit_super(sb, es, 1);
+
#ifdef CONFIG_QUOTA
/* Release old quota file names */
for (i = 0; i < MAXQUOTAS; i++)
@@ -3097,6 +3263,8 @@
sbi->s_resuid = old_opts.s_resuid;
sbi->s_resgid = old_opts.s_resgid;
sbi->s_commit_interval = old_opts.s_commit_interval;
+ sbi->s_min_batch_time = old_opts.s_min_batch_time;
+ sbi->s_max_batch_time = old_opts.s_max_batch_time;
#ifdef CONFIG_QUOTA
sbi->s_jquota_fmt = old_opts.s_jquota_fmt;
for (i = 0; i < MAXQUOTAS; i++) {
@@ -3359,7 +3527,8 @@
* When we journal data on quota file, we have to flush journal to see
* all updates to the file when we bypass pagecache...
*/
- if (ext4_should_journal_data(path.dentry->d_inode)) {
+ if (EXT4_SB(sb)->s_journal &&
+ ext4_should_journal_data(path.dentry->d_inode)) {
/*
* We don't need to lock updates but journal_flush() could
* otherwise be livelocked...
@@ -3433,7 +3602,7 @@
struct buffer_head *bh;
handle_t *handle = journal_current_handle();
- if (!handle) {
+ if (EXT4_SB(sb)->s_journal && !handle) {
printk(KERN_WARNING "EXT4-fs: Quota write (off=%llu, len=%llu)"
" cancelled because transaction is not started.\n",
(unsigned long long)off, (unsigned long long)len);
@@ -3458,7 +3627,7 @@
flush_dcache_page(bh->b_page);
unlock_buffer(bh);
if (journal_quota)
- err = ext4_journal_dirty_metadata(handle, bh);
+ err = ext4_handle_dirty_metadata(handle, NULL, bh);
else {
/* Always do at least ordered writes for quotas */
err = ext4_jbd2_file_inode(handle, inode);
@@ -3512,18 +3681,15 @@
static ssize_t ext4_ui_proc_write(struct file *file, const char __user *buf,
size_t cnt, loff_t *ppos)
{
- unsigned int *p = PDE(file->f_path.dentry->d_inode)->data;
+ unsigned long *p = PDE(file->f_path.dentry->d_inode)->data;
char str[32];
- unsigned long value;
if (cnt >= sizeof(str))
return -EINVAL;
if (copy_from_user(str, buf, cnt))
return -EFAULT;
- value = simple_strtol(str, NULL, 0);
- if (value < 0)
- return -ERANGE;
- *p = value;
+
+ *p = simple_strtoul(str, NULL, 0);
return cnt;
}
@@ -3614,7 +3780,7 @@
}
MODULE_AUTHOR("Remy Card, Stephen Tweedie, Andrew Morton, Andreas Dilger, Theodore Ts'o and others");
-MODULE_DESCRIPTION("Fourth Extended Filesystem with extents");
+MODULE_DESCRIPTION("Fourth Extended Filesystem");
MODULE_LICENSE("GPL");
module_init(init_ext4_fs)
module_exit(exit_ext4_fs)
diff --git a/fs/ext4/xattr.c b/fs/ext4/xattr.c
index 80626d5..157ce65 100644
--- a/fs/ext4/xattr.c
+++ b/fs/ext4/xattr.c
@@ -457,7 +457,7 @@
if (ext4_journal_get_write_access(handle, EXT4_SB(sb)->s_sbh) == 0) {
EXT4_SET_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_EXT_ATTR);
sb->s_dirt = 1;
- ext4_journal_dirty_metadata(handle, EXT4_SB(sb)->s_sbh);
+ ext4_handle_dirty_metadata(handle, NULL, EXT4_SB(sb)->s_sbh);
}
}
@@ -487,9 +487,9 @@
ext4_forget(handle, 1, inode, bh, bh->b_blocknr);
} else {
le32_add_cpu(&BHDR(bh)->h_refcount, -1);
- error = ext4_journal_dirty_metadata(handle, bh);
+ error = ext4_handle_dirty_metadata(handle, inode, bh);
if (IS_SYNC(inode))
- handle->h_sync = 1;
+ ext4_handle_sync(handle);
DQUOT_FREE_BLOCK(inode, 1);
ea_bdebug(bh, "refcount now=%d; releasing",
le32_to_cpu(BHDR(bh)->h_refcount));
@@ -724,8 +724,9 @@
if (error == -EIO)
goto bad_block;
if (!error)
- error = ext4_journal_dirty_metadata(handle,
- bs->bh);
+ error = ext4_handle_dirty_metadata(handle,
+ inode,
+ bs->bh);
if (error)
goto cleanup;
goto inserted;
@@ -794,8 +795,9 @@
ea_bdebug(new_bh, "reusing; refcount now=%d",
le32_to_cpu(BHDR(new_bh)->h_refcount));
unlock_buffer(new_bh);
- error = ext4_journal_dirty_metadata(handle,
- new_bh);
+ error = ext4_handle_dirty_metadata(handle,
+ inode,
+ new_bh);
if (error)
goto cleanup_dquot;
}
@@ -810,8 +812,8 @@
/* We need to allocate a new block */
ext4_fsblk_t goal = ext4_group_first_block_no(sb,
EXT4_I(inode)->i_block_group);
- ext4_fsblk_t block = ext4_new_meta_block(handle, inode,
- goal, &error);
+ ext4_fsblk_t block = ext4_new_meta_blocks(handle, inode,
+ goal, NULL, &error);
if (error)
goto cleanup;
ea_idebug(inode, "creating block %d", block);
@@ -833,7 +835,8 @@
set_buffer_uptodate(new_bh);
unlock_buffer(new_bh);
ext4_xattr_cache_insert(new_bh);
- error = ext4_journal_dirty_metadata(handle, new_bh);
+ error = ext4_handle_dirty_metadata(handle,
+ inode, new_bh);
if (error)
goto cleanup;
}
@@ -1040,7 +1043,7 @@
*/
is.iloc.bh = NULL;
if (IS_SYNC(inode))
- handle->h_sync = 1;
+ ext4_handle_sync(handle);
}
cleanup:
diff --git a/fs/ioprio.c b/fs/ioprio.c
index 3569e0a..1a39ac3 100644
--- a/fs/ioprio.c
+++ b/fs/ioprio.c
@@ -27,7 +27,7 @@
#include <linux/security.h>
#include <linux/pid_namespace.h>
-static int set_task_ioprio(struct task_struct *task, int ioprio)
+int set_task_ioprio(struct task_struct *task, int ioprio)
{
int err;
struct io_context *ioc;
@@ -70,6 +70,7 @@
task_unlock(task);
return err;
}
+EXPORT_SYMBOL_GPL(set_task_ioprio);
asmlinkage long sys_ioprio_set(int which, int who, int ioprio)
{
diff --git a/fs/jbd/commit.c b/fs/jbd/commit.c
index 25719d9..3fbffb1 100644
--- a/fs/jbd/commit.c
+++ b/fs/jbd/commit.c
@@ -306,6 +306,8 @@
int flags;
int err;
unsigned long blocknr;
+ ktime_t start_time;
+ u64 commit_time;
char *tagp = NULL;
journal_header_t *header;
journal_block_tag_t *tag = NULL;
@@ -418,6 +420,7 @@
commit_transaction->t_state = T_FLUSH;
journal->j_committing_transaction = commit_transaction;
journal->j_running_transaction = NULL;
+ start_time = ktime_get();
commit_transaction->t_log_start = journal->j_head;
wake_up(&journal->j_wait_transaction_locked);
spin_unlock(&journal->j_state_lock);
@@ -913,6 +916,18 @@
J_ASSERT(commit_transaction == journal->j_committing_transaction);
journal->j_commit_sequence = commit_transaction->t_tid;
journal->j_committing_transaction = NULL;
+ commit_time = ktime_to_ns(ktime_sub(ktime_get(), start_time));
+
+ /*
+ * weight the commit time higher than the average time so we don't
+ * react too strongly to vast changes in commit time
+ */
+ if (likely(journal->j_average_commit_time))
+ journal->j_average_commit_time = (commit_time*3 +
+ journal->j_average_commit_time) / 4;
+ else
+ journal->j_average_commit_time = commit_time;
+
spin_unlock(&journal->j_state_lock);
if (commit_transaction->t_checkpoint_list == NULL &&
diff --git a/fs/jbd/transaction.c b/fs/jbd/transaction.c
index 60d4c32..e6a1174 100644
--- a/fs/jbd/transaction.c
+++ b/fs/jbd/transaction.c
@@ -25,6 +25,7 @@
#include <linux/timer.h>
#include <linux/mm.h>
#include <linux/highmem.h>
+#include <linux/hrtimer.h>
static void __journal_temp_unlink_buffer(struct journal_head *jh);
@@ -49,6 +50,7 @@
{
transaction->t_journal = journal;
transaction->t_state = T_RUNNING;
+ transaction->t_start_time = ktime_get();
transaction->t_tid = journal->j_transaction_sequence++;
transaction->t_expires = jiffies + journal->j_commit_interval;
spin_lock_init(&transaction->t_handle_lock);
@@ -752,7 +754,6 @@
* int journal_get_write_access() - notify intent to modify a buffer for metadata (not data) update.
* @handle: transaction to add buffer modifications to
* @bh: bh to be used for metadata writes
- * @credits: variable that will receive credits for the buffer
*
* Returns an error code or 0 on success.
*
@@ -1370,7 +1371,7 @@
{
transaction_t *transaction = handle->h_transaction;
journal_t *journal = transaction->t_journal;
- int old_handle_count, err;
+ int err;
pid_t pid;
J_ASSERT(journal_current_handle() == handle);
@@ -1399,6 +1400,17 @@
* on IO anyway. Speeds up many-threaded, many-dir operations
* by 30x or more...
*
+ * We try and optimize the sleep time against what the underlying disk
+ * can do, instead of having a static sleep time. This is usefull for
+ * the case where our storage is so fast that it is more optimal to go
+ * ahead and force a flush and wait for the transaction to be committed
+ * than it is to wait for an arbitrary amount of time for new writers to
+ * join the transaction. We acheive this by measuring how long it takes
+ * to commit a transaction, and compare it with how long this
+ * transaction has been running, and if run time < commit time then we
+ * sleep for the delta and commit. This greatly helps super fast disks
+ * that would see slowdowns as more threads started doing fsyncs.
+ *
* But don't do this if this process was the most recent one to
* perform a synchronous write. We do this to detect the case where a
* single process is doing a stream of sync writes. No point in waiting
@@ -1406,11 +1418,26 @@
*/
pid = current->pid;
if (handle->h_sync && journal->j_last_sync_writer != pid) {
+ u64 commit_time, trans_time;
+
journal->j_last_sync_writer = pid;
- do {
- old_handle_count = transaction->t_handle_count;
- schedule_timeout_uninterruptible(1);
- } while (old_handle_count != transaction->t_handle_count);
+
+ spin_lock(&journal->j_state_lock);
+ commit_time = journal->j_average_commit_time;
+ spin_unlock(&journal->j_state_lock);
+
+ trans_time = ktime_to_ns(ktime_sub(ktime_get(),
+ transaction->t_start_time));
+
+ commit_time = min_t(u64, commit_time,
+ 1000*jiffies_to_usecs(1));
+
+ if (trans_time < commit_time) {
+ ktime_t expires = ktime_add_ns(ktime_get(),
+ commit_time);
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ schedule_hrtimeout(&expires, HRTIMER_MODE_ABS);
+ }
}
current->journal_info = NULL;
diff --git a/fs/jbd2/checkpoint.c b/fs/jbd2/checkpoint.c
index 9497718..17159ca 100644
--- a/fs/jbd2/checkpoint.c
+++ b/fs/jbd2/checkpoint.c
@@ -249,16 +249,14 @@
return ret;
}
-#define NR_BATCH 64
-
static void
-__flush_batch(journal_t *journal, struct buffer_head **bhs, int *batch_count)
+__flush_batch(journal_t *journal, int *batch_count)
{
int i;
- ll_rw_block(SWRITE, *batch_count, bhs);
+ ll_rw_block(SWRITE, *batch_count, journal->j_chkpt_bhs);
for (i = 0; i < *batch_count; i++) {
- struct buffer_head *bh = bhs[i];
+ struct buffer_head *bh = journal->j_chkpt_bhs[i];
clear_buffer_jwrite(bh);
BUFFER_TRACE(bh, "brelse");
__brelse(bh);
@@ -277,8 +275,7 @@
* Called under jbd_lock_bh_state(jh2bh(jh)), and drops it
*/
static int __process_buffer(journal_t *journal, struct journal_head *jh,
- struct buffer_head **bhs, int *batch_count,
- transaction_t *transaction)
+ int *batch_count, transaction_t *transaction)
{
struct buffer_head *bh = jh2bh(jh);
int ret = 0;
@@ -325,14 +322,14 @@
get_bh(bh);
J_ASSERT_BH(bh, !buffer_jwrite(bh));
set_buffer_jwrite(bh);
- bhs[*batch_count] = bh;
+ journal->j_chkpt_bhs[*batch_count] = bh;
__buffer_relink_io(jh);
jbd_unlock_bh_state(bh);
transaction->t_chp_stats.cs_written++;
(*batch_count)++;
- if (*batch_count == NR_BATCH) {
+ if (*batch_count == JBD2_NR_BATCH) {
spin_unlock(&journal->j_list_lock);
- __flush_batch(journal, bhs, batch_count);
+ __flush_batch(journal, batch_count);
ret = 1;
}
}
@@ -388,7 +385,6 @@
if (journal->j_checkpoint_transactions == transaction &&
transaction->t_tid == this_tid) {
int batch_count = 0;
- struct buffer_head *bhs[NR_BATCH];
struct journal_head *jh;
int retry = 0, err;
@@ -402,7 +398,7 @@
retry = 1;
break;
}
- retry = __process_buffer(journal, jh, bhs, &batch_count,
+ retry = __process_buffer(journal, jh, &batch_count,
transaction);
if (retry < 0 && !result)
result = retry;
@@ -419,7 +415,7 @@
spin_unlock(&journal->j_list_lock);
retry = 1;
}
- __flush_batch(journal, bhs, &batch_count);
+ __flush_batch(journal, &batch_count);
}
if (retry) {
@@ -686,6 +682,7 @@
safely remove this transaction from the log */
__jbd2_journal_drop_transaction(journal, transaction);
+ kfree(transaction);
/* Just in case anybody was waiting for more transactions to be
checkpointed... */
@@ -760,5 +757,4 @@
J_ASSERT(journal->j_running_transaction != transaction);
jbd_debug(1, "Dropping transaction %d, all done\n", transaction->t_tid);
- kfree(transaction);
}
diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c
index c8a1bac..62804e5 100644
--- a/fs/jbd2/commit.c
+++ b/fs/jbd2/commit.c
@@ -25,6 +25,7 @@
#include <linux/crc32.h>
#include <linux/writeback.h>
#include <linux/backing-dev.h>
+#include <linux/bio.h>
/*
* Default IO end handler for temporary BJ_IO buffer_heads.
@@ -137,7 +138,7 @@
set_buffer_ordered(bh);
barrier_done = 1;
}
- ret = submit_bh(WRITE, bh);
+ ret = submit_bh(WRITE_SYNC, bh);
if (barrier_done)
clear_buffer_ordered(bh);
@@ -158,7 +159,7 @@
lock_buffer(bh);
set_buffer_uptodate(bh);
clear_buffer_dirty(bh);
- ret = submit_bh(WRITE, bh);
+ ret = submit_bh(WRITE_SYNC, bh);
}
*cbh = bh;
return ret;
@@ -168,12 +169,34 @@
* This function along with journal_submit_commit_record
* allows to write the commit record asynchronously.
*/
-static int journal_wait_on_commit_record(struct buffer_head *bh)
+static int journal_wait_on_commit_record(journal_t *journal,
+ struct buffer_head *bh)
{
int ret = 0;
+retry:
clear_buffer_dirty(bh);
wait_on_buffer(bh);
+ if (buffer_eopnotsupp(bh) && (journal->j_flags & JBD2_BARRIER)) {
+ printk(KERN_WARNING
+ "JBD2: wait_on_commit_record: sync failed on %s - "
+ "disabling barriers\n", journal->j_devname);
+ spin_lock(&journal->j_state_lock);
+ journal->j_flags &= ~JBD2_BARRIER;
+ spin_unlock(&journal->j_state_lock);
+
+ lock_buffer(bh);
+ clear_buffer_dirty(bh);
+ set_buffer_uptodate(bh);
+ bh->b_end_io = journal_end_buffer_io_sync;
+
+ ret = submit_bh(WRITE_SYNC, bh);
+ if (ret) {
+ unlock_buffer(bh);
+ return ret;
+ }
+ goto retry;
+ }
if (unlikely(!buffer_uptodate(bh)))
ret = -EIO;
@@ -332,13 +355,15 @@
int flags;
int err;
unsigned long long blocknr;
+ ktime_t start_time;
+ u64 commit_time;
char *tagp = NULL;
journal_header_t *header;
journal_block_tag_t *tag = NULL;
int space_left = 0;
int first_tag = 0;
int tag_flag;
- int i;
+ int i, to_free = 0;
int tag_bytes = journal_tag_bytes(journal);
struct buffer_head *cbh = NULL; /* For transactional checksums */
__u32 crc32_sum = ~0;
@@ -458,6 +483,7 @@
commit_transaction->t_state = T_FLUSH;
journal->j_committing_transaction = commit_transaction;
journal->j_running_transaction = NULL;
+ start_time = ktime_get();
commit_transaction->t_log_start = journal->j_head;
wake_up(&journal->j_wait_transaction_locked);
spin_unlock(&journal->j_state_lock);
@@ -803,7 +829,7 @@
__jbd2_journal_abort_hard(journal);
}
if (!err && !is_journal_aborted(journal))
- err = journal_wait_on_commit_record(cbh);
+ err = journal_wait_on_commit_record(journal, cbh);
if (err)
jbd2_journal_abort(journal, err);
@@ -981,14 +1007,23 @@
J_ASSERT(commit_transaction == journal->j_committing_transaction);
journal->j_commit_sequence = commit_transaction->t_tid;
journal->j_committing_transaction = NULL;
- spin_unlock(&journal->j_state_lock);
+ commit_time = ktime_to_ns(ktime_sub(ktime_get(), start_time));
- if (journal->j_commit_callback)
- journal->j_commit_callback(journal, commit_transaction);
+ /*
+ * weight the commit time higher than the average time so we don't
+ * react too strongly to vast changes in the commit time
+ */
+ if (likely(journal->j_average_commit_time))
+ journal->j_average_commit_time = (commit_time +
+ journal->j_average_commit_time*3) / 4;
+ else
+ journal->j_average_commit_time = commit_time;
+ spin_unlock(&journal->j_state_lock);
if (commit_transaction->t_checkpoint_list == NULL &&
commit_transaction->t_checkpoint_io_list == NULL) {
__jbd2_journal_drop_transaction(journal, commit_transaction);
+ to_free = 1;
} else {
if (journal->j_checkpoint_transactions == NULL) {
journal->j_checkpoint_transactions = commit_transaction;
@@ -1007,11 +1042,16 @@
}
spin_unlock(&journal->j_list_lock);
+ if (journal->j_commit_callback)
+ journal->j_commit_callback(journal, commit_transaction);
+
trace_mark(jbd2_end_commit, "dev %s transaction %d head %d",
- journal->j_devname, journal->j_commit_sequence,
+ journal->j_devname, commit_transaction->t_tid,
journal->j_tail_sequence);
jbd_debug(1, "JBD: commit %d complete, head %d\n",
journal->j_commit_sequence, journal->j_tail_sequence);
+ if (to_free)
+ kfree(commit_transaction);
wake_up(&journal->j_wait_done_commit);
}
diff --git a/fs/jbd2/journal.c b/fs/jbd2/journal.c
index f6bff9d..5667530 100644
--- a/fs/jbd2/journal.c
+++ b/fs/jbd2/journal.c
@@ -40,6 +40,7 @@
#include <asm/uaccess.h>
#include <asm/page.h>
+#include <asm/div64.h>
EXPORT_SYMBOL(jbd2_journal_start);
EXPORT_SYMBOL(jbd2_journal_restart);
@@ -66,7 +67,6 @@
EXPORT_SYMBOL(jbd2_journal_check_used_features);
EXPORT_SYMBOL(jbd2_journal_check_available_features);
EXPORT_SYMBOL(jbd2_journal_set_features);
-EXPORT_SYMBOL(jbd2_journal_create);
EXPORT_SYMBOL(jbd2_journal_load);
EXPORT_SYMBOL(jbd2_journal_destroy);
EXPORT_SYMBOL(jbd2_journal_abort);
@@ -132,8 +132,9 @@
journal->j_task = current;
wake_up(&journal->j_wait_done_commit);
- printk(KERN_INFO "kjournald2 starting. Commit interval %ld seconds\n",
- journal->j_commit_interval / HZ);
+ printk(KERN_INFO "kjournald2 starting: pid %d, dev %s, "
+ "commit interval %ld seconds\n", current->pid,
+ journal->j_devname, journal->j_commit_interval / HZ);
/*
* And now, wait forever for commit wakeup events.
@@ -650,6 +651,8 @@
return NULL;
bh = __getblk(journal->j_dev, blocknr, journal->j_blocksize);
+ if (!bh)
+ return NULL;
lock_buffer(bh);
memset(bh->b_data, 0, journal->j_blocksize);
set_buffer_uptodate(bh);
@@ -843,6 +846,8 @@
jiffies_to_msecs(s->stats->u.run.rs_flushing / s->stats->ts_tid));
seq_printf(seq, " %ums logging transaction\n",
jiffies_to_msecs(s->stats->u.run.rs_logging / s->stats->ts_tid));
+ seq_printf(seq, " %luus average transaction commit time\n",
+ do_div(s->journal->j_average_commit_time, 1000));
seq_printf(seq, " %lu handles per transaction\n",
s->stats->u.run.rs_handle_count / s->stats->ts_tid);
seq_printf(seq, " %lu blocks per transaction\n",
@@ -980,6 +985,8 @@
spin_lock_init(&journal->j_state_lock);
journal->j_commit_interval = (HZ * JBD2_DEFAULT_MAX_COMMIT_AGE);
+ journal->j_min_batch_time = 0;
+ journal->j_max_batch_time = 15000; /* 15ms */
/* The journal is marked for error until we succeed with recovery! */
journal->j_flags = JBD2_ABORT;
@@ -1035,15 +1042,14 @@
/* journal descriptor can store up to n blocks -bzzz */
journal->j_blocksize = blocksize;
+ jbd2_stats_proc_init(journal);
n = journal->j_blocksize / sizeof(journal_block_tag_t);
journal->j_wbufsize = n;
journal->j_wbuf = kmalloc(n * sizeof(struct buffer_head*), GFP_KERNEL);
if (!journal->j_wbuf) {
printk(KERN_ERR "%s: Cant allocate bhs for commit thread\n",
__func__);
- kfree(journal);
- journal = NULL;
- goto out;
+ goto out_err;
}
journal->j_dev = bdev;
journal->j_fs_dev = fs_dev;
@@ -1053,14 +1059,22 @@
p = journal->j_devname;
while ((p = strchr(p, '/')))
*p = '!';
- jbd2_stats_proc_init(journal);
bh = __getblk(journal->j_dev, start, journal->j_blocksize);
- J_ASSERT(bh != NULL);
+ if (!bh) {
+ printk(KERN_ERR
+ "%s: Cannot get buffer for journal superblock\n",
+ __func__);
+ goto out_err;
+ }
journal->j_sb_buffer = bh;
journal->j_superblock = (journal_superblock_t *)bh->b_data;
-out:
+
return journal;
+out_err:
+ jbd2_stats_proc_exit(journal);
+ kfree(journal);
+ return NULL;
}
/**
@@ -1108,9 +1122,7 @@
if (!journal->j_wbuf) {
printk(KERN_ERR "%s: Cant allocate bhs for commit thread\n",
__func__);
- jbd2_stats_proc_exit(journal);
- kfree(journal);
- return NULL;
+ goto out_err;
}
err = jbd2_journal_bmap(journal, 0, &blocknr);
@@ -1118,17 +1130,24 @@
if (err) {
printk(KERN_ERR "%s: Cannnot locate journal superblock\n",
__func__);
- jbd2_stats_proc_exit(journal);
- kfree(journal);
- return NULL;
+ goto out_err;
}
bh = __getblk(journal->j_dev, blocknr, journal->j_blocksize);
- J_ASSERT(bh != NULL);
+ if (!bh) {
+ printk(KERN_ERR
+ "%s: Cannot get buffer for journal superblock\n",
+ __func__);
+ goto out_err;
+ }
journal->j_sb_buffer = bh;
journal->j_superblock = (journal_superblock_t *)bh->b_data;
return journal;
+out_err:
+ jbd2_stats_proc_exit(journal);
+ kfree(journal);
+ return NULL;
}
/*
@@ -1177,77 +1196,6 @@
}
/**
- * int jbd2_journal_create() - Initialise the new journal file
- * @journal: Journal to create. This structure must have been initialised
- *
- * Given a journal_t structure which tells us which disk blocks we can
- * use, create a new journal superblock and initialise all of the
- * journal fields from scratch.
- **/
-int jbd2_journal_create(journal_t *journal)
-{
- unsigned long long blocknr;
- struct buffer_head *bh;
- journal_superblock_t *sb;
- int i, err;
-
- if (journal->j_maxlen < JBD2_MIN_JOURNAL_BLOCKS) {
- printk (KERN_ERR "Journal length (%d blocks) too short.\n",
- journal->j_maxlen);
- journal_fail_superblock(journal);
- return -EINVAL;
- }
-
- if (journal->j_inode == NULL) {
- /*
- * We don't know what block to start at!
- */
- printk(KERN_EMERG
- "%s: creation of journal on external device!\n",
- __func__);
- BUG();
- }
-
- /* Zero out the entire journal on disk. We cannot afford to
- have any blocks on disk beginning with JBD2_MAGIC_NUMBER. */
- jbd_debug(1, "JBD: Zeroing out journal blocks...\n");
- for (i = 0; i < journal->j_maxlen; i++) {
- err = jbd2_journal_bmap(journal, i, &blocknr);
- if (err)
- return err;
- bh = __getblk(journal->j_dev, blocknr, journal->j_blocksize);
- lock_buffer(bh);
- memset (bh->b_data, 0, journal->j_blocksize);
- BUFFER_TRACE(bh, "marking dirty");
- mark_buffer_dirty(bh);
- BUFFER_TRACE(bh, "marking uptodate");
- set_buffer_uptodate(bh);
- unlock_buffer(bh);
- __brelse(bh);
- }
-
- sync_blockdev(journal->j_dev);
- jbd_debug(1, "JBD: journal cleared.\n");
-
- /* OK, fill in the initial static fields in the new superblock */
- sb = journal->j_superblock;
-
- sb->s_header.h_magic = cpu_to_be32(JBD2_MAGIC_NUMBER);
- sb->s_header.h_blocktype = cpu_to_be32(JBD2_SUPERBLOCK_V2);
-
- sb->s_blocksize = cpu_to_be32(journal->j_blocksize);
- sb->s_maxlen = cpu_to_be32(journal->j_maxlen);
- sb->s_first = cpu_to_be32(1);
-
- journal->j_transaction_sequence = 1;
-
- journal->j_flags &= ~JBD2_ABORT;
- journal->j_format_version = 2;
-
- return journal_reset(journal);
-}
-
-/**
* void jbd2_journal_update_superblock() - Update journal sb on disk.
* @journal: The journal to update.
* @wait: Set to '0' if you don't want to wait for IO completion.
@@ -1491,7 +1439,9 @@
spin_lock(&journal->j_list_lock);
while (journal->j_checkpoint_transactions != NULL) {
spin_unlock(&journal->j_list_lock);
+ mutex_lock(&journal->j_checkpoint_mutex);
jbd2_log_do_checkpoint(journal);
+ mutex_unlock(&journal->j_checkpoint_mutex);
spin_lock(&journal->j_list_lock);
}
diff --git a/fs/jbd2/transaction.c b/fs/jbd2/transaction.c
index 4f925a4..46b4e34 100644
--- a/fs/jbd2/transaction.c
+++ b/fs/jbd2/transaction.c
@@ -25,6 +25,7 @@
#include <linux/timer.h>
#include <linux/mm.h>
#include <linux/highmem.h>
+#include <linux/hrtimer.h>
static void __jbd2_journal_temp_unlink_buffer(struct journal_head *jh);
@@ -48,6 +49,7 @@
{
transaction->t_journal = journal;
transaction->t_state = T_RUNNING;
+ transaction->t_start_time = ktime_get();
transaction->t_tid = journal->j_transaction_sequence++;
transaction->t_expires = jiffies + journal->j_commit_interval;
spin_lock_init(&transaction->t_handle_lock);
@@ -1240,7 +1242,7 @@
{
transaction_t *transaction = handle->h_transaction;
journal_t *journal = transaction->t_journal;
- int old_handle_count, err;
+ int err;
pid_t pid;
J_ASSERT(journal_current_handle() == handle);
@@ -1263,24 +1265,54 @@
/*
* Implement synchronous transaction batching. If the handle
* was synchronous, don't force a commit immediately. Let's
- * yield and let another thread piggyback onto this transaction.
- * Keep doing that while new threads continue to arrive.
- * It doesn't cost much - we're about to run a commit and sleep
- * on IO anyway. Speeds up many-threaded, many-dir operations
- * by 30x or more...
+ * yield and let another thread piggyback onto this
+ * transaction. Keep doing that while new threads continue to
+ * arrive. It doesn't cost much - we're about to run a commit
+ * and sleep on IO anyway. Speeds up many-threaded, many-dir
+ * operations by 30x or more...
*
- * But don't do this if this process was the most recent one to
- * perform a synchronous write. We do this to detect the case where a
- * single process is doing a stream of sync writes. No point in waiting
- * for joiners in that case.
+ * We try and optimize the sleep time against what the
+ * underlying disk can do, instead of having a static sleep
+ * time. This is useful for the case where our storage is so
+ * fast that it is more optimal to go ahead and force a flush
+ * and wait for the transaction to be committed than it is to
+ * wait for an arbitrary amount of time for new writers to
+ * join the transaction. We achieve this by measuring how
+ * long it takes to commit a transaction, and compare it with
+ * how long this transaction has been running, and if run time
+ * < commit time then we sleep for the delta and commit. This
+ * greatly helps super fast disks that would see slowdowns as
+ * more threads started doing fsyncs.
+ *
+ * But don't do this if this process was the most recent one
+ * to perform a synchronous write. We do this to detect the
+ * case where a single process is doing a stream of sync
+ * writes. No point in waiting for joiners in that case.
*/
pid = current->pid;
if (handle->h_sync && journal->j_last_sync_writer != pid) {
+ u64 commit_time, trans_time;
+
journal->j_last_sync_writer = pid;
- do {
- old_handle_count = transaction->t_handle_count;
- schedule_timeout_uninterruptible(1);
- } while (old_handle_count != transaction->t_handle_count);
+
+ spin_lock(&journal->j_state_lock);
+ commit_time = journal->j_average_commit_time;
+ spin_unlock(&journal->j_state_lock);
+
+ trans_time = ktime_to_ns(ktime_sub(ktime_get(),
+ transaction->t_start_time));
+
+ commit_time = max_t(u64, commit_time,
+ 1000*journal->j_min_batch_time);
+ commit_time = min_t(u64, commit_time,
+ 1000*journal->j_max_batch_time);
+
+ if (trans_time < commit_time) {
+ ktime_t expires = ktime_add_ns(ktime_get(),
+ commit_time);
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ schedule_hrtimeout(&expires, HRTIMER_MODE_ABS);
+ }
}
current->journal_info = NULL;
diff --git a/fs/jffs2/compr_rubin.c b/fs/jffs2/compr_rubin.c
index c73fa89..170d289 100644
--- a/fs/jffs2/compr_rubin.c
+++ b/fs/jffs2/compr_rubin.c
@@ -22,9 +22,7 @@
#define BIT_DIVIDER_MIPS 1043
-static int bits_mips[8] = { 277,249,290,267,229,341,212,241}; /* mips32 */
-
-#include <linux/errno.h>
+static int bits_mips[8] = { 277, 249, 290, 267, 229, 341, 212, 241};
struct pushpull {
unsigned char *buf;
@@ -43,7 +41,9 @@
int bits[8];
};
-static inline void init_pushpull(struct pushpull *pp, char *buf, unsigned buflen, unsigned ofs, unsigned reserve)
+static inline void init_pushpull(struct pushpull *pp, char *buf,
+ unsigned buflen, unsigned ofs,
+ unsigned reserve)
{
pp->buf = buf;
pp->buflen = buflen;
@@ -53,16 +53,14 @@
static inline int pushbit(struct pushpull *pp, int bit, int use_reserved)
{
- if (pp->ofs >= pp->buflen - (use_reserved?0:pp->reserve)) {
+ if (pp->ofs >= pp->buflen - (use_reserved?0:pp->reserve))
return -ENOSPC;
- }
- if (bit) {
- pp->buf[pp->ofs >> 3] |= (1<<(7-(pp->ofs &7)));
- }
- else {
- pp->buf[pp->ofs >> 3] &= ~(1<<(7-(pp->ofs &7)));
- }
+ if (bit)
+ pp->buf[pp->ofs >> 3] |= (1<<(7-(pp->ofs & 7)));
+ else
+ pp->buf[pp->ofs >> 3] &= ~(1<<(7-(pp->ofs & 7)));
+
pp->ofs++;
return 0;
@@ -97,6 +95,7 @@
rs->p = (long) (2 * UPPER_BIT_RUBIN);
rs->bit_number = (long) 0;
rs->bit_divider = div;
+
for (c=0; c<8; c++)
rs->bits[c] = bits[c];
}
@@ -108,7 +107,8 @@
long i0, i1;
int ret;
- while ((rs->q >= UPPER_BIT_RUBIN) || ((rs->p + rs->q) <= UPPER_BIT_RUBIN)) {
+ while ((rs->q >= UPPER_BIT_RUBIN) ||
+ ((rs->p + rs->q) <= UPPER_BIT_RUBIN)) {
rs->bit_number++;
ret = pushbit(&rs->pp, (rs->q & UPPER_BIT_RUBIN) ? 1 : 0, 0);
@@ -119,12 +119,12 @@
rs->p <<= 1;
}
i0 = A * rs->p / (A + B);
- if (i0 <= 0) {
+ if (i0 <= 0)
i0 = 1;
- }
- if (i0 >= rs->p) {
+
+ if (i0 >= rs->p)
i0 = rs->p - 1;
- }
+
i1 = rs->p - i0;
if (symbol == 0)
@@ -157,11 +157,13 @@
/* behalve lower */
rs->rec_q = 0;
- for (rs->bit_number = 0; rs->bit_number++ < RUBIN_REG_SIZE; rs->rec_q = rs->rec_q * 2 + (long) (pullbit(&rs->pp)))
+ for (rs->bit_number = 0; rs->bit_number++ < RUBIN_REG_SIZE;
+ rs->rec_q = rs->rec_q * 2 + (long) (pullbit(&rs->pp)))
;
}
-static void __do_decode(struct rubin_state *rs, unsigned long p, unsigned long q)
+static void __do_decode(struct rubin_state *rs, unsigned long p,
+ unsigned long q)
{
register unsigned long lower_bits_rubin = LOWER_BITS_RUBIN;
unsigned long rec_q;
@@ -207,12 +209,11 @@
__do_decode(rs, p, q);
i0 = A * rs->p / (A + B);
- if (i0 <= 0) {
+ if (i0 <= 0)
i0 = 1;
- }
- if (i0 >= rs->p) {
+
+ if (i0 >= rs->p)
i0 = rs->p - 1;
- }
threshold = rs->q + i0;
symbol = rs->rec_q >= threshold;
@@ -234,14 +235,15 @@
struct rubin_state rs_copy;
rs_copy = *rs;
- for (i=0;i<8;i++) {
- ret = encode(rs, rs->bit_divider-rs->bits[i],rs->bits[i],byte&1);
+ for (i=0; i<8; i++) {
+ ret = encode(rs, rs->bit_divider-rs->bits[i],
+ rs->bits[i], byte & 1);
if (ret) {
/* Failed. Restore old state */
*rs = rs_copy;
return ret;
}
- byte=byte>>1;
+ byte >>= 1 ;
}
return 0;
}
@@ -251,7 +253,8 @@
int i, result = 0, bit_divider = rs->bit_divider;
for (i = 0; i < 8; i++)
- result |= decode(rs, bit_divider - rs->bits[i], rs->bits[i]) << i;
+ result |= decode(rs, bit_divider - rs->bits[i],
+ rs->bits[i]) << i;
return result;
}
@@ -259,7 +262,8 @@
static int rubin_do_compress(int bit_divider, int *bits, unsigned char *data_in,
- unsigned char *cpage_out, uint32_t *sourcelen, uint32_t *dstlen)
+ unsigned char *cpage_out, uint32_t *sourcelen,
+ uint32_t *dstlen)
{
int outpos = 0;
int pos=0;
@@ -295,7 +299,8 @@
int jffs2_rubinmips_compress(unsigned char *data_in, unsigned char *cpage_out,
uint32_t *sourcelen, uint32_t *dstlen, void *model)
{
- return rubin_do_compress(BIT_DIVIDER_MIPS, bits_mips, data_in, cpage_out, sourcelen, dstlen);
+ return rubin_do_compress(BIT_DIVIDER_MIPS, bits_mips, data_in,
+ cpage_out, sourcelen, dstlen);
}
#endif
static int jffs2_dynrubin_compress(unsigned char *data_in,
@@ -316,9 +321,8 @@
return -1;
memset(histo, 0, 256);
- for (i=0; i<mysrclen; i++) {
+ for (i=0; i<mysrclen; i++)
histo[data_in[i]]++;
- }
memset(bits, 0, sizeof(int)*8);
for (i=0; i<256; i++) {
if (i&128)
@@ -346,7 +350,8 @@
cpage_out[i] = bits[i];
}
- ret = rubin_do_compress(256, bits, data_in, cpage_out+8, &mysrclen, &mydstlen);
+ ret = rubin_do_compress(256, bits, data_in, cpage_out+8, &mysrclen,
+ &mydstlen);
if (ret)
return ret;
@@ -363,8 +368,10 @@
return 0;
}
-static void rubin_do_decompress(int bit_divider, int *bits, unsigned char *cdata_in,
- unsigned char *page_out, uint32_t srclen, uint32_t destlen)
+static void rubin_do_decompress(int bit_divider, int *bits,
+ unsigned char *cdata_in,
+ unsigned char *page_out, uint32_t srclen,
+ uint32_t destlen)
{
int outpos = 0;
struct rubin_state rs;
@@ -372,9 +379,8 @@
init_pushpull(&rs.pp, cdata_in, srclen, 0, 0);
init_decode(&rs, bit_divider, bits);
- while (outpos < destlen) {
+ while (outpos < destlen)
page_out[outpos++] = in_byte(&rs);
- }
}
@@ -383,7 +389,8 @@
uint32_t sourcelen, uint32_t dstlen,
void *model)
{
- rubin_do_decompress(BIT_DIVIDER_MIPS, bits_mips, data_in, cpage_out, sourcelen, dstlen);
+ rubin_do_decompress(BIT_DIVIDER_MIPS, bits_mips, data_in,
+ cpage_out, sourcelen, dstlen);
return 0;
}
@@ -398,52 +405,53 @@
for (c=0; c<8; c++)
bits[c] = data_in[c];
- rubin_do_decompress(256, bits, data_in+8, cpage_out, sourcelen-8, dstlen);
+ rubin_do_decompress(256, bits, data_in+8, cpage_out, sourcelen-8,
+ dstlen);
return 0;
}
static struct jffs2_compressor jffs2_rubinmips_comp = {
- .priority = JFFS2_RUBINMIPS_PRIORITY,
- .name = "rubinmips",
- .compr = JFFS2_COMPR_DYNRUBIN,
- .compress = NULL, /*&jffs2_rubinmips_compress,*/
- .decompress = &jffs2_rubinmips_decompress,
+ .priority = JFFS2_RUBINMIPS_PRIORITY,
+ .name = "rubinmips",
+ .compr = JFFS2_COMPR_DYNRUBIN,
+ .compress = NULL, /*&jffs2_rubinmips_compress,*/
+ .decompress = &jffs2_rubinmips_decompress,
#ifdef JFFS2_RUBINMIPS_DISABLED
- .disabled = 1,
+ .disabled = 1,
#else
- .disabled = 0,
+ .disabled = 0,
#endif
};
int jffs2_rubinmips_init(void)
{
- return jffs2_register_compressor(&jffs2_rubinmips_comp);
+ return jffs2_register_compressor(&jffs2_rubinmips_comp);
}
void jffs2_rubinmips_exit(void)
{
- jffs2_unregister_compressor(&jffs2_rubinmips_comp);
+ jffs2_unregister_compressor(&jffs2_rubinmips_comp);
}
static struct jffs2_compressor jffs2_dynrubin_comp = {
- .priority = JFFS2_DYNRUBIN_PRIORITY,
- .name = "dynrubin",
- .compr = JFFS2_COMPR_RUBINMIPS,
- .compress = jffs2_dynrubin_compress,
- .decompress = &jffs2_dynrubin_decompress,
+ .priority = JFFS2_DYNRUBIN_PRIORITY,
+ .name = "dynrubin",
+ .compr = JFFS2_COMPR_RUBINMIPS,
+ .compress = jffs2_dynrubin_compress,
+ .decompress = &jffs2_dynrubin_decompress,
#ifdef JFFS2_DYNRUBIN_DISABLED
- .disabled = 1,
+ .disabled = 1,
#else
- .disabled = 0,
+ .disabled = 0,
#endif
};
int jffs2_dynrubin_init(void)
{
- return jffs2_register_compressor(&jffs2_dynrubin_comp);
+ return jffs2_register_compressor(&jffs2_dynrubin_comp);
}
void jffs2_dynrubin_exit(void)
{
- jffs2_unregister_compressor(&jffs2_dynrubin_comp);
+ jffs2_unregister_compressor(&jffs2_dynrubin_comp);
}
diff --git a/fs/jffs2/erase.c b/fs/jffs2/erase.c
index 259461b..c32b4a1 100644
--- a/fs/jffs2/erase.c
+++ b/fs/jffs2/erase.c
@@ -175,7 +175,7 @@
{
/* For NAND, if the failure did not occur at the device level for a
specific physical page, don't bother updating the bad block table. */
- if (jffs2_cleanmarker_oob(c) && (bad_offset != MTD_FAIL_ADDR_UNKNOWN)) {
+ if (jffs2_cleanmarker_oob(c) && (bad_offset != (uint32_t)MTD_FAIL_ADDR_UNKNOWN)) {
/* We had a device-level failure to erase. Let's see if we've
failed too many times. */
if (!jffs2_write_nand_badblock(c, jeb, bad_offset)) {
@@ -209,7 +209,8 @@
struct erase_priv_struct *priv = (void *)instr->priv;
if(instr->state != MTD_ERASE_DONE) {
- printk(KERN_WARNING "Erase at 0x%08x finished, but state != MTD_ERASE_DONE. State is 0x%x instead.\n", instr->addr, instr->state);
+ printk(KERN_WARNING "Erase at 0x%08llx finished, but state != MTD_ERASE_DONE. State is 0x%x instead.\n",
+ (unsigned long long)instr->addr, instr->state);
jffs2_erase_failed(priv->c, priv->jeb, instr->fail_addr);
} else {
jffs2_erase_succeeded(priv->c, priv->jeb);
diff --git a/fs/ocfs2/alloc.c b/fs/ocfs2/alloc.c
index 54ff4c7..d861096 100644
--- a/fs/ocfs2/alloc.c
+++ b/fs/ocfs2/alloc.c
@@ -3868,7 +3868,7 @@
struct ocfs2_extent_list *left_el = NULL, *right_el, *insert_el, *el;
struct ocfs2_extent_rec *rec, *tmprec;
- right_el = path_leaf_el(right_path);;
+ right_el = path_leaf_el(right_path);
if (left_path)
left_el = path_leaf_el(left_path);
diff --git a/fs/ocfs2/dlmglue.c b/fs/ocfs2/dlmglue.c
index f731ab4..b0c4cad 100644
--- a/fs/ocfs2/dlmglue.c
+++ b/fs/ocfs2/dlmglue.c
@@ -1324,7 +1324,7 @@
goto out;
}
- mlog(0, "lock %s, successfull return from ocfs2_dlm_lock\n",
+ mlog(0, "lock %s, successful return from ocfs2_dlm_lock\n",
lockres->l_name);
/* At this point we've gone inside the dlm and need to
@@ -2951,7 +2951,7 @@
ocfs2_dlm_dump_lksb(&lockres->l_lksb);
BUG();
}
- mlog(0, "lock %s, successfull return from ocfs2_dlm_unlock\n",
+ mlog(0, "lock %s, successful return from ocfs2_dlm_unlock\n",
lockres->l_name);
ocfs2_wait_on_busy_lock(lockres);
diff --git a/fs/ocfs2/file.c b/fs/ocfs2/file.c
index e8f795f..a5887df 100644
--- a/fs/ocfs2/file.c
+++ b/fs/ocfs2/file.c
@@ -1605,7 +1605,7 @@
struct ocfs2_space_resv *sr)
{
struct inode *inode = file->f_path.dentry->d_inode;
- struct ocfs2_super *osb = OCFS2_SB(inode->i_sb);;
+ struct ocfs2_super *osb = OCFS2_SB(inode->i_sb);
if ((cmd == OCFS2_IOC_RESVSP || cmd == OCFS2_IOC_RESVSP64) &&
!ocfs2_writes_unwritten_extents(osb))
diff --git a/fs/partitions/check.c b/fs/partitions/check.c
index 5198ada..6d72024 100644
--- a/fs/partitions/check.c
+++ b/fs/partitions/check.c
@@ -334,6 +334,7 @@
blk_free_devt(part_devt(part));
rcu_assign_pointer(ptbl->part[partno], NULL);
+ rcu_assign_pointer(ptbl->last_lookup, NULL);
kobject_put(part->holder_dir);
device_del(part_to_dev(part));
diff --git a/fs/proc/vmcore.c b/fs/proc/vmcore.c
index 03ec595..5edcc3f 100644
--- a/fs/proc/vmcore.c
+++ b/fs/proc/vmcore.c
@@ -47,8 +47,6 @@
offset = (unsigned long)(*ppos % PAGE_SIZE);
pfn = (unsigned long)(*ppos / PAGE_SIZE);
- if (pfn > saved_max_pfn)
- return -EINVAL;
do {
if (count > (PAGE_SIZE - offset))
diff --git a/fs/romfs/inode.c b/fs/romfs/inode.c
index c97d4c9..98a232f 100644
--- a/fs/romfs/inode.c
+++ b/fs/romfs/inode.c
@@ -490,7 +490,7 @@
static struct inode *
romfs_iget(struct super_block *sb, unsigned long ino)
{
- int nextfh;
+ int nextfh, ret;
struct romfs_inode ri;
struct inode *i;
@@ -526,11 +526,11 @@
i->i_mtime.tv_nsec = i->i_atime.tv_nsec = i->i_ctime.tv_nsec = 0;
/* Precalculate the data offset */
- ino = romfs_strnlen(i, ino+ROMFH_SIZE, ROMFS_MAXFN);
- if (ino >= 0)
- ino = ((ROMFH_SIZE+ino+1+ROMFH_PAD)&ROMFH_MASK);
- else
- ino = 0;
+ ret = romfs_strnlen(i, ino + ROMFH_SIZE, ROMFS_MAXFN);
+ if (ret >= 0)
+ ino = (ROMFH_SIZE + ret + 1 + ROMFH_PAD) & ROMFH_MASK;
+ else
+ ino = 0;
ROMFS_I(i)->i_metasize = ino;
ROMFS_I(i)->i_dataoffset = ino+(i->i_ino&ROMFH_MASK);
diff --git a/fs/splice.c b/fs/splice.c
index 1abab5c..a54b3e3 100644
--- a/fs/splice.c
+++ b/fs/splice.c
@@ -21,6 +21,7 @@
#include <linux/file.h>
#include <linux/pagemap.h>
#include <linux/splice.h>
+#include <linux/memcontrol.h>
#include <linux/mm_inline.h>
#include <linux/swap.h>
#include <linux/writeback.h>
diff --git a/fs/super.c b/fs/super.c
index cb20744..ed080c4 100644
--- a/fs/super.c
+++ b/fs/super.c
@@ -458,7 +458,6 @@
if (sb->s_flags & MS_RDONLY)
continue;
sb->s_need_sync_fs = 1;
- async_synchronize_full_special(&sb->s_async_list);
}
restart:
@@ -471,6 +470,7 @@
sb->s_count++;
spin_unlock(&sb_lock);
down_read(&sb->s_umount);
+ async_synchronize_full_special(&sb->s_async_list);
if (sb->s_root && (wait || sb->s_dirt))
sb->s_op->sync_fs(sb, wait);
up_read(&sb->s_umount);
@@ -810,6 +810,7 @@
}
s->s_flags |= MS_ACTIVE;
+ bdev->bd_super = s;
}
return simple_set_mnt(mnt, s);
@@ -829,6 +830,7 @@
struct block_device *bdev = sb->s_bdev;
fmode_t mode = sb->s_mode;
+ bdev->bd_super = 0;
generic_shutdown_super(sb);
sync_blockdev(bdev);
close_bdev_exclusive(bdev, mode);
diff --git a/include/acpi/acdisasm.h b/include/acpi/acdisasm.h
deleted file mode 100644
index 0c1ed38..0000000
--- a/include/acpi/acdisasm.h
+++ /dev/null
@@ -1,445 +0,0 @@
-/******************************************************************************
- *
- * Name: acdisasm.h - AML disassembler
- *
- *****************************************************************************/
-
-/*
- * Copyright (C) 2000 - 2008, Intel Corp.
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- * notice, this list of conditions, and the following disclaimer,
- * without modification.
- * 2. Redistributions in binary form must reproduce at minimum a disclaimer
- * substantially similar to the "NO WARRANTY" disclaimer below
- * ("Disclaimer") and any redistribution must be conditioned upon
- * including a substantially similar Disclaimer requirement for further
- * binary redistribution.
- * 3. Neither the names of the above-listed copyright holders nor the names
- * of any contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
- *
- * Alternatively, this software may be distributed under the terms of the
- * GNU General Public License ("GPL") version 2 as published by the Free
- * Software Foundation.
- *
- * NO WARRANTY
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
- * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
- * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGES.
- */
-
-#ifndef __ACDISASM_H__
-#define __ACDISASM_H__
-
-#include "amlresrc.h"
-
-#define BLOCK_NONE 0
-#define BLOCK_PAREN 1
-#define BLOCK_BRACE 2
-#define BLOCK_COMMA_LIST 4
-#define ACPI_DEFAULT_RESNAME *(u32 *) "__RD"
-
-struct acpi_external_list {
- char *path;
- char *internal_path;
- struct acpi_external_list *next;
- u32 value;
- u16 length;
- u8 type;
-};
-
-extern struct acpi_external_list *acpi_gbl_external_list;
-
-typedef const struct acpi_dmtable_info {
- u8 opcode;
- u8 offset;
- char *name;
-
-} acpi_dmtable_info;
-
-/*
- * Values for Opcode above.
- * Note: 0-7 must not change, used as a flag shift value
- */
-#define ACPI_DMT_FLAG0 0
-#define ACPI_DMT_FLAG1 1
-#define ACPI_DMT_FLAG2 2
-#define ACPI_DMT_FLAG3 3
-#define ACPI_DMT_FLAG4 4
-#define ACPI_DMT_FLAG5 5
-#define ACPI_DMT_FLAG6 6
-#define ACPI_DMT_FLAG7 7
-#define ACPI_DMT_FLAGS0 8
-#define ACPI_DMT_FLAGS2 9
-#define ACPI_DMT_UINT8 10
-#define ACPI_DMT_UINT16 11
-#define ACPI_DMT_UINT24 12
-#define ACPI_DMT_UINT32 13
-#define ACPI_DMT_UINT56 14
-#define ACPI_DMT_UINT64 15
-#define ACPI_DMT_STRING 16
-#define ACPI_DMT_NAME4 17
-#define ACPI_DMT_NAME6 18
-#define ACPI_DMT_NAME8 19
-#define ACPI_DMT_CHKSUM 20
-#define ACPI_DMT_SPACEID 21
-#define ACPI_DMT_GAS 22
-#define ACPI_DMT_ASF 23
-#define ACPI_DMT_DMAR 24
-#define ACPI_DMT_HEST 25
-#define ACPI_DMT_HESTNTFY 26
-#define ACPI_DMT_HESTNTYP 27
-#define ACPI_DMT_MADT 28
-#define ACPI_DMT_SRAT 29
-#define ACPI_DMT_EXIT 30
-#define ACPI_DMT_SIG 31
-
-typedef
-void (*acpi_dmtable_handler) (struct acpi_table_header * table);
-
-struct acpi_dmtable_data {
- char *signature;
- struct acpi_dmtable_info *table_info;
- acpi_dmtable_handler table_handler;
- char *name;
-};
-
-struct acpi_op_walk_info {
- u32 level;
- u32 last_level;
- u32 count;
- u32 bit_offset;
- u32 flags;
- struct acpi_walk_state *walk_state;
-};
-
-typedef
-acpi_status(*asl_walk_callback) (union acpi_parse_object * op,
- u32 level, void *context);
-
-struct acpi_resource_tag {
- u32 bit_index;
- char *tag;
-};
-
-/* Strings used for decoding flags to ASL keywords */
-
-extern const char *acpi_gbl_word_decode[];
-extern const char *acpi_gbl_irq_decode[];
-extern const char *acpi_gbl_lock_rule[];
-extern const char *acpi_gbl_access_types[];
-extern const char *acpi_gbl_update_rules[];
-extern const char *acpi_gbl_match_ops[];
-
-extern struct acpi_dmtable_info acpi_dm_table_info_asf0[];
-extern struct acpi_dmtable_info acpi_dm_table_info_asf1[];
-extern struct acpi_dmtable_info acpi_dm_table_info_asf1a[];
-extern struct acpi_dmtable_info acpi_dm_table_info_asf2[];
-extern struct acpi_dmtable_info acpi_dm_table_info_asf2a[];
-extern struct acpi_dmtable_info acpi_dm_table_info_asf3[];
-extern struct acpi_dmtable_info acpi_dm_table_info_asf4[];
-extern struct acpi_dmtable_info acpi_dm_table_info_asf_hdr[];
-extern struct acpi_dmtable_info acpi_dm_table_info_boot[];
-extern struct acpi_dmtable_info acpi_dm_table_info_bert[];
-extern struct acpi_dmtable_info acpi_dm_table_info_cpep[];
-extern struct acpi_dmtable_info acpi_dm_table_info_cpep0[];
-extern struct acpi_dmtable_info acpi_dm_table_info_dbgp[];
-extern struct acpi_dmtable_info acpi_dm_table_info_dmar[];
-extern struct acpi_dmtable_info acpi_dm_table_info_dmar_hdr[];
-extern struct acpi_dmtable_info acpi_dm_table_info_dmar_scope[];
-extern struct acpi_dmtable_info acpi_dm_table_info_dmar0[];
-extern struct acpi_dmtable_info acpi_dm_table_info_dmar1[];
-extern struct acpi_dmtable_info acpi_dm_table_info_dmar2[];
-extern struct acpi_dmtable_info acpi_dm_table_info_ecdt[];
-extern struct acpi_dmtable_info acpi_dm_table_info_einj[];
-extern struct acpi_dmtable_info acpi_dm_table_info_einj0[];
-extern struct acpi_dmtable_info acpi_dm_table_info_erst[];
-extern struct acpi_dmtable_info acpi_dm_table_info_facs[];
-extern struct acpi_dmtable_info acpi_dm_table_info_fadt1[];
-extern struct acpi_dmtable_info acpi_dm_table_info_fadt2[];
-extern struct acpi_dmtable_info acpi_dm_table_info_gas[];
-extern struct acpi_dmtable_info acpi_dm_table_info_header[];
-extern struct acpi_dmtable_info acpi_dm_table_info_hest[];
-extern struct acpi_dmtable_info acpi_dm_table_info_hest9[];
-extern struct acpi_dmtable_info acpi_dm_table_info_hest_notify[];
-extern struct acpi_dmtable_info acpi_dm_table_info_hpet[];
-extern struct acpi_dmtable_info acpi_dm_table_info_madt[];
-extern struct acpi_dmtable_info acpi_dm_table_info_madt0[];
-extern struct acpi_dmtable_info acpi_dm_table_info_madt1[];
-extern struct acpi_dmtable_info acpi_dm_table_info_madt2[];
-extern struct acpi_dmtable_info acpi_dm_table_info_madt3[];
-extern struct acpi_dmtable_info acpi_dm_table_info_madt4[];
-extern struct acpi_dmtable_info acpi_dm_table_info_madt5[];
-extern struct acpi_dmtable_info acpi_dm_table_info_madt6[];
-extern struct acpi_dmtable_info acpi_dm_table_info_madt7[];
-extern struct acpi_dmtable_info acpi_dm_table_info_madt8[];
-extern struct acpi_dmtable_info acpi_dm_table_info_madt9[];
-extern struct acpi_dmtable_info acpi_dm_table_info_madt10[];
-extern struct acpi_dmtable_info acpi_dm_table_info_madt_hdr[];
-extern struct acpi_dmtable_info acpi_dm_table_info_mcfg[];
-extern struct acpi_dmtable_info acpi_dm_table_info_mcfg0[];
-extern struct acpi_dmtable_info acpi_dm_table_info_rsdp1[];
-extern struct acpi_dmtable_info acpi_dm_table_info_rsdp2[];
-extern struct acpi_dmtable_info acpi_dm_table_info_sbst[];
-extern struct acpi_dmtable_info acpi_dm_table_info_slic[];
-extern struct acpi_dmtable_info acpi_dm_table_info_slit[];
-extern struct acpi_dmtable_info acpi_dm_table_info_spcr[];
-extern struct acpi_dmtable_info acpi_dm_table_info_spmi[];
-extern struct acpi_dmtable_info acpi_dm_table_info_srat[];
-extern struct acpi_dmtable_info acpi_dm_table_info_srat_hdr[];
-extern struct acpi_dmtable_info acpi_dm_table_info_srat0[];
-extern struct acpi_dmtable_info acpi_dm_table_info_srat1[];
-extern struct acpi_dmtable_info acpi_dm_table_info_srat2[];
-extern struct acpi_dmtable_info acpi_dm_table_info_tcpa[];
-extern struct acpi_dmtable_info acpi_dm_table_info_wdrt[];
-
-/*
- * dmtable
- */
-void acpi_dm_dump_data_table(struct acpi_table_header *table);
-
-acpi_status
-acpi_dm_dump_table(u32 table_length,
- u32 table_offset,
- void *table,
- u32 sub_table_length, struct acpi_dmtable_info *info);
-
-void acpi_dm_line_header(u32 offset, u32 byte_length, char *name);
-
-void acpi_dm_line_header2(u32 offset, u32 byte_length, char *name, u32 value);
-
-/*
- * dmtbdump
- */
-void acpi_dm_dump_asf(struct acpi_table_header *table);
-
-void acpi_dm_dump_cpep(struct acpi_table_header *table);
-
-void acpi_dm_dump_dmar(struct acpi_table_header *table);
-
-void acpi_dm_dump_einj(struct acpi_table_header *table);
-
-void acpi_dm_dump_erst(struct acpi_table_header *table);
-
-void acpi_dm_dump_fadt(struct acpi_table_header *table);
-
-void acpi_dm_dump_hest(struct acpi_table_header *table);
-
-void acpi_dm_dump_mcfg(struct acpi_table_header *table);
-
-void acpi_dm_dump_madt(struct acpi_table_header *table);
-
-u32 acpi_dm_dump_rsdp(struct acpi_table_header *table);
-
-void acpi_dm_dump_rsdt(struct acpi_table_header *table);
-
-void acpi_dm_dump_slit(struct acpi_table_header *table);
-
-void acpi_dm_dump_srat(struct acpi_table_header *table);
-
-void acpi_dm_dump_xsdt(struct acpi_table_header *table);
-
-/*
- * dmwalk
- */
-void
-acpi_dm_disassemble(struct acpi_walk_state *walk_state,
- union acpi_parse_object *origin, u32 num_opcodes);
-
-void
-acpi_dm_walk_parse_tree(union acpi_parse_object *op,
- asl_walk_callback descending_callback,
- asl_walk_callback ascending_callback, void *context);
-
-/*
- * dmopcode
- */
-void
-acpi_dm_disassemble_one_op(struct acpi_walk_state *walk_state,
- struct acpi_op_walk_info *info,
- union acpi_parse_object *op);
-
-void acpi_dm_decode_internal_object(union acpi_operand_object *obj_desc);
-
-u32 acpi_dm_list_type(union acpi_parse_object *op);
-
-void acpi_dm_method_flags(union acpi_parse_object *op);
-
-void acpi_dm_field_flags(union acpi_parse_object *op);
-
-void acpi_dm_address_space(u8 space_id);
-
-void acpi_dm_region_flags(union acpi_parse_object *op);
-
-void acpi_dm_match_op(union acpi_parse_object *op);
-
-u8 acpi_dm_comma_if_list_member(union acpi_parse_object *op);
-
-void acpi_dm_comma_if_field_member(union acpi_parse_object *op);
-
-/*
- * dmnames
- */
-u32 acpi_dm_dump_name(char *name);
-
-acpi_status
-acpi_ps_display_object_pathname(struct acpi_walk_state *walk_state,
- union acpi_parse_object *op);
-
-void acpi_dm_namestring(char *name);
-
-/*
- * dmobject
- */
-void
-acpi_dm_display_internal_object(union acpi_operand_object *obj_desc,
- struct acpi_walk_state *walk_state);
-
-void acpi_dm_display_arguments(struct acpi_walk_state *walk_state);
-
-void acpi_dm_display_locals(struct acpi_walk_state *walk_state);
-
-void
-acpi_dm_dump_method_info(acpi_status status,
- struct acpi_walk_state *walk_state,
- union acpi_parse_object *op);
-
-/*
- * dmbuffer
- */
-void acpi_dm_disasm_byte_list(u32 level, u8 * byte_data, u32 byte_count);
-
-void
-acpi_dm_byte_list(struct acpi_op_walk_info *info, union acpi_parse_object *op);
-
-void acpi_dm_is_eisa_id(union acpi_parse_object *op);
-
-void acpi_dm_eisa_id(u32 encoded_id);
-
-u8 acpi_dm_is_unicode_buffer(union acpi_parse_object *op);
-
-u8 acpi_dm_is_string_buffer(union acpi_parse_object *op);
-
-/*
- * dmresrc
- */
-void acpi_dm_dump_integer8(u8 value, char *name);
-
-void acpi_dm_dump_integer16(u16 value, char *name);
-
-void acpi_dm_dump_integer32(u32 value, char *name);
-
-void acpi_dm_dump_integer64(u64 value, char *name);
-
-void
-acpi_dm_resource_template(struct acpi_op_walk_info *info,
- union acpi_parse_object *op,
- u8 * byte_data, u32 byte_count);
-
-acpi_status acpi_dm_is_resource_template(union acpi_parse_object *op);
-
-void acpi_dm_indent(u32 level);
-
-void acpi_dm_bit_list(u16 mask);
-
-void acpi_dm_decode_attribute(u8 attribute);
-
-void acpi_dm_descriptor_name(void);
-
-/*
- * dmresrcl
- */
-void
-acpi_dm_word_descriptor(union aml_resource *resource, u32 length, u32 level);
-
-void
-acpi_dm_dword_descriptor(union aml_resource *resource, u32 length, u32 level);
-
-void
-acpi_dm_extended_descriptor(union aml_resource *resource,
- u32 length, u32 level);
-
-void
-acpi_dm_qword_descriptor(union aml_resource *resource, u32 length, u32 level);
-
-void
-acpi_dm_memory24_descriptor(union aml_resource *resource,
- u32 length, u32 level);
-
-void
-acpi_dm_memory32_descriptor(union aml_resource *resource,
- u32 length, u32 level);
-
-void
-acpi_dm_fixed_memory32_descriptor(union aml_resource *resource,
- u32 length, u32 level);
-
-void
-acpi_dm_generic_register_descriptor(union aml_resource *resource,
- u32 length, u32 level);
-
-void
-acpi_dm_interrupt_descriptor(union aml_resource *resource,
- u32 length, u32 level);
-
-void
-acpi_dm_vendor_large_descriptor(union aml_resource *resource,
- u32 length, u32 level);
-
-void acpi_dm_vendor_common(char *name, u8 * byte_data, u32 length, u32 level);
-
-/*
- * dmresrcs
- */
-void
-acpi_dm_irq_descriptor(union aml_resource *resource, u32 length, u32 level);
-
-void
-acpi_dm_dma_descriptor(union aml_resource *resource, u32 length, u32 level);
-
-void acpi_dm_io_descriptor(union aml_resource *resource, u32 length, u32 level);
-
-void
-acpi_dm_fixed_io_descriptor(union aml_resource *resource,
- u32 length, u32 level);
-
-void
-acpi_dm_start_dependent_descriptor(union aml_resource *resource,
- u32 length, u32 level);
-
-void
-acpi_dm_end_dependent_descriptor(union aml_resource *resource,
- u32 length, u32 level);
-
-void
-acpi_dm_vendor_small_descriptor(union aml_resource *resource,
- u32 length, u32 level);
-
-/*
- * dmutils
- */
-void acpi_dm_add_to_external_list(char *path, u8 type, u32 value);
-
-/*
- * dmrestag
- */
-void acpi_dm_find_resources(union acpi_parse_object *root);
-
-void
-acpi_dm_check_resource_reference(union acpi_parse_object *op,
- struct acpi_walk_state *walk_state);
-
-#endif /* __ACDISASM_H__ */
diff --git a/include/acpi/acexcep.h b/include/acpi/acexcep.h
index 84f5cb2..eda0454 100644
--- a/include/acpi/acexcep.h
+++ b/include/acpi/acexcep.h
@@ -153,8 +153,9 @@
#define AE_AML_CIRCULAR_REFERENCE (acpi_status) (0x001E | AE_CODE_AML)
#define AE_AML_BAD_RESOURCE_LENGTH (acpi_status) (0x001F | AE_CODE_AML)
#define AE_AML_ILLEGAL_ADDRESS (acpi_status) (0x0020 | AE_CODE_AML)
+#define AE_AML_INFINITE_LOOP (acpi_status) (0x0021 | AE_CODE_AML)
-#define AE_CODE_AML_MAX 0x0020
+#define AE_CODE_AML_MAX 0x0021
/*
* Internal exceptions used for control
@@ -175,6 +176,8 @@
#define AE_CODE_CTRL_MAX 0x000D
+/* Exception strings for acpi_format_exception */
+
#ifdef DEFINE_ACPI_GLOBALS
/*
@@ -267,6 +270,7 @@
"AE_AML_CIRCULAR_REFERENCE",
"AE_AML_BAD_RESOURCE_LENGTH",
"AE_AML_ILLEGAL_ADDRESS",
+ "AE_AML_INFINITE_LOOP"
};
char const *acpi_gbl_exception_names_ctrl[] = {
diff --git a/include/acpi/acoutput.h b/include/acpi/acoutput.h
index db8852d..5c823d5 100644
--- a/include/acpi/acoutput.h
+++ b/include/acpi/acoutput.h
@@ -45,9 +45,9 @@
#define __ACOUTPUT_H__
/*
- * Debug levels and component IDs. These are used to control the
- * granularity of the output of the DEBUG_PRINT macro -- on a per-
- * component basis and a per-exception-type basis.
+ * Debug levels and component IDs. These are used to control the
+ * granularity of the output of the ACPI_DEBUG_PRINT macro -- on a
+ * per-component basis and a per-exception-type basis.
*/
/* Component IDs are used in the global "DebugLayer" */
@@ -69,8 +69,10 @@
#define ACPI_COMPILER 0x00001000
#define ACPI_TOOLS 0x00002000
+#define ACPI_EXAMPLE 0x00004000
+#define ACPI_DRIVER 0x00008000
-#define ACPI_ALL_COMPONENTS 0x00003FFF
+#define ACPI_ALL_COMPONENTS 0x0000FFFF
#define ACPI_COMPONENT_DEFAULT (ACPI_ALL_COMPONENTS)
/* Component IDs reserved for ACPI drivers */
@@ -78,7 +80,7 @@
#define ACPI_ALL_DRIVERS 0xFFFF0000
/*
- * Raw debug output levels, do not use these in the DEBUG_PRINT macros
+ * Raw debug output levels, do not use these in the ACPI_DEBUG_PRINT macros
*/
#define ACPI_LV_INIT 0x00000001
#define ACPI_LV_DEBUG_OBJECT 0x00000002
@@ -176,4 +178,95 @@
#define ACPI_NORMAL_DEFAULT (ACPI_LV_INIT | ACPI_LV_DEBUG_OBJECT)
#define ACPI_DEBUG_ALL (ACPI_LV_AML_DISASSEMBLE | ACPI_LV_ALL_EXCEPTIONS | ACPI_LV_ALL)
+#if defined (ACPI_DEBUG_OUTPUT) || !defined (ACPI_NO_ERROR_MESSAGES)
+/*
+ * Module name is included in both debug and non-debug versions primarily for
+ * error messages. The __FILE__ macro is not very useful for this, because it
+ * often includes the entire pathname to the module
+ */
+#define ACPI_MODULE_NAME(name) static const char ACPI_UNUSED_VAR _acpi_module_name[] = name;
+#else
+#define ACPI_MODULE_NAME(name)
+#endif
+
+/*
+ * Ascii error messages can be configured out
+ */
+#ifndef ACPI_NO_ERROR_MESSAGES
+#define AE_INFO _acpi_module_name, __LINE__
+
+/*
+ * Error reporting. Callers module and line number are inserted by AE_INFO,
+ * the plist contains a set of parens to allow variable-length lists.
+ * These macros are used for both the debug and non-debug versions of the code.
+ */
+#define ACPI_INFO(plist) acpi_info plist
+#define ACPI_WARNING(plist) acpi_warning plist
+#define ACPI_EXCEPTION(plist) acpi_exception plist
+#define ACPI_ERROR(plist) acpi_error plist
+
+#else
+
+/* No error messages */
+
+#define ACPI_INFO(plist)
+#define ACPI_WARNING(plist)
+#define ACPI_EXCEPTION(plist)
+#define ACPI_ERROR(plist)
+
+#endif /* ACPI_NO_ERROR_MESSAGES */
+
+/*
+ * Debug macros that are conditionally compiled
+ */
+#ifdef ACPI_DEBUG_OUTPUT
+
+/*
+ * If ACPI_GET_FUNCTION_NAME was not defined in the compiler-dependent header,
+ * define it now. This is the case where there the compiler does not support
+ * a __FUNCTION__ macro or equivalent.
+ */
+#ifndef ACPI_GET_FUNCTION_NAME
+#define ACPI_GET_FUNCTION_NAME _acpi_function_name
+
+/*
+ * The Name parameter should be the procedure name as a quoted string.
+ * The function name is also used by the function exit macros below.
+ * Note: (const char) is used to be compatible with the debug interfaces
+ * and macros such as __FUNCTION__.
+ */
+#define ACPI_FUNCTION_NAME(name) static const char _acpi_function_name[] = #name;
+
+#else
+/* Compiler supports __FUNCTION__ (or equivalent) -- Ignore this macro */
+
+#define ACPI_FUNCTION_NAME(name)
+#endif /* ACPI_GET_FUNCTION_NAME */
+
+/*
+ * Common parameters used for debug output functions:
+ * line number, function name, module(file) name, component ID
+ */
+#define ACPI_DEBUG_PARAMETERS __LINE__, ACPI_GET_FUNCTION_NAME, _acpi_module_name, _COMPONENT
+
+/*
+ * Master debug print macros
+ * Print message if and only if:
+ * 1) Debug print for the current component is enabled
+ * 2) Debug error level or trace level for the print statement is enabled
+ */
+#define ACPI_DEBUG_PRINT(plist) acpi_debug_print plist
+#define ACPI_DEBUG_PRINT_RAW(plist) acpi_debug_print_raw plist
+
+#else
+/*
+ * This is the non-debug case -- make everything go away,
+ * leaving no executable debug code!
+ */
+#define ACPI_FUNCTION_NAME(a)
+#define ACPI_DEBUG_PRINT(pl)
+#define ACPI_DEBUG_PRINT_RAW(pl)
+
+#endif /* ACPI_DEBUG_OUTPUT */
+
#endif /* __ACOUTPUT_H__ */
diff --git a/include/acpi/acpi.h b/include/acpi/acpi.h
index c515ef6..472b7bf 100644
--- a/include/acpi/acpi.h
+++ b/include/acpi/acpi.h
@@ -1,6 +1,6 @@
/******************************************************************************
*
- * Name: acpi.h - Master include file, Publics and external data.
+ * Name: acpi.h - Master public include file used to interface to ACPICA
*
*****************************************************************************/
@@ -45,25 +45,22 @@
#define __ACPI_H__
/*
- * Common includes for all ACPI driver files
- * We put them here because we don't want to duplicate them
- * in the rest of the source code again and again.
+ * Public include files for use by code that will interface to ACPICA.
+ *
+ * Information includes the ACPICA data types, names, exceptions, and
+ * external interface prototypes. Also included are the definitions for
+ * all ACPI tables (FADT, MADT, etc.)
+ *
+ * Note: The order of these include files is important.
*/
-#include "acnames.h" /* Global ACPI names and strings */
-#include "acconfig.h" /* Configuration constants */
-#include "platform/acenv.h" /* Target environment specific items */
-#include "actypes.h" /* Fundamental common data types */
-#include "acexcep.h" /* ACPI exception codes */
-#include "acmacros.h" /* C macros */
+#include "platform/acenv.h" /* Environment-specific items */
+#include "acnames.h" /* Common ACPI names and strings */
+#include "actypes.h" /* ACPICA data types and structures */
+#include "acexcep.h" /* ACPICA exceptions */
#include "actbl.h" /* ACPI table definitions */
-#include "aclocal.h" /* Internal data types */
#include "acoutput.h" /* Error output and Debug macros */
-#include "acpiosxf.h" /* Interfaces to the ACPI-to-OS layer */
+#include "acrestyp.h" /* Resource Descriptor structs */
+#include "acpiosxf.h" /* OSL interfaces (ACPICA-to-OS) */
#include "acpixf.h" /* ACPI core subsystem external interfaces */
-#include "acobject.h" /* ACPI internal object */
-#include "acstruct.h" /* Common structures */
-#include "acglobal.h" /* All global variables */
-#include "achware.h" /* Hardware defines and interfaces */
-#include "acutils.h" /* Utility interfaces */
#endif /* __ACPI_H__ */
diff --git a/include/acpi/acpiosxf.h b/include/acpi/acpiosxf.h
index b91440a..a62720a 100644
--- a/include/acpi/acpiosxf.h
+++ b/include/acpi/acpiosxf.h
@@ -121,8 +121,11 @@
acpi_status acpi_os_signal_semaphore(acpi_semaphore handle, u32 units);
/*
- * Mutex primitives
+ * Mutex primitives. May be configured to use semaphores instead via
+ * ACPI_MUTEX_TYPE (see platform/acenv.h)
*/
+#if (ACPI_MUTEX_TYPE != ACPI_BINARY_SEMAPHORE)
+
acpi_status acpi_os_create_mutex(acpi_mutex * out_handle);
void acpi_os_delete_mutex(acpi_mutex handle);
@@ -130,13 +133,7 @@
acpi_status acpi_os_acquire_mutex(acpi_mutex handle, u16 timeout);
void acpi_os_release_mutex(acpi_mutex handle);
-
-/* Temporary macros for Mutex* interfaces, map to existing semaphore xfaces */
-
-#define acpi_os_create_mutex(out_handle) acpi_os_create_semaphore (1, 1, out_handle)
-#define acpi_os_delete_mutex(handle) (void) acpi_os_delete_semaphore (handle)
-#define acpi_os_acquire_mutex(handle,time) acpi_os_wait_semaphore (handle, 1, time)
-#define acpi_os_release_mutex(handle) (void) acpi_os_signal_semaphore (handle, 1)
+#endif
/*
* Memory allocation and mapping
diff --git a/include/acpi/acpixf.h b/include/acpi/acpixf.h
index 33bc0e3b1..c8e8cf4 100644
--- a/include/acpi/acpixf.h
+++ b/include/acpi/acpixf.h
@@ -45,9 +45,32 @@
#ifndef __ACXFACE_H__
#define __ACXFACE_H__
+/* Current ACPICA subsystem version in YYYYMMDD format */
+
+#define ACPI_CA_VERSION 0x20081204
+
#include "actypes.h"
#include "actbl.h"
+extern u8 acpi_gbl_permanent_mmap;
+
+/*
+ * Globals that are publically available, allowing for
+ * run time configuration
+ */
+extern u32 acpi_dbg_level;
+extern u32 acpi_dbg_layer;
+extern u8 acpi_gbl_enable_interpreter_slack;
+extern u8 acpi_gbl_all_methods_serialized;
+extern u8 acpi_gbl_create_osi_method;
+extern u8 acpi_gbl_leave_wake_gpes_disabled;
+extern acpi_name acpi_gbl_trace_method_name;
+extern u32 acpi_gbl_trace_flags;
+
+extern u32 acpi_current_gpe_count;
+extern struct acpi_table_fadt acpi_gbl_FADT;
+
+extern u32 acpi_rsdt_forced;
/*
* Global interfaces
*/
@@ -79,11 +102,6 @@
acpi_status acpi_purge_cached_objects(void);
-#ifdef ACPI_FUTURE_USAGE
-acpi_status
-acpi_install_initialization_handler(acpi_init_handler handler, u32 function);
-#endif
-
/*
* ACPI Memory management
*/
@@ -193,9 +211,12 @@
acpi_status acpi_get_parent(acpi_handle object, acpi_handle * out_handle);
/*
- * Event handler interfaces
+ * Handler interfaces
*/
acpi_status
+acpi_install_initialization_handler(acpi_init_handler handler, u32 function);
+
+acpi_status
acpi_install_fixed_event_handler(u32 acpi_event,
acpi_event_handler handler, void *context);
@@ -227,6 +248,10 @@
u32 gpe_number,
u32 type, acpi_event_handler address, void *context);
+acpi_status
+acpi_remove_gpe_handler(acpi_handle gpe_device,
+ u32 gpe_number, acpi_event_handler address);
+
#ifdef ACPI_FUTURE_USAGE
acpi_status acpi_install_exception_handler(acpi_exception_handler handler);
#endif
@@ -238,10 +263,6 @@
acpi_status acpi_release_global_lock(u32 handle);
-acpi_status
-acpi_remove_gpe_handler(acpi_handle gpe_device,
- u32 gpe_number, acpi_event_handler address);
-
acpi_status acpi_enable_event(u32 event, u32 flags);
acpi_status acpi_disable_event(u32 event, u32 flags);
@@ -250,6 +271,9 @@
acpi_status acpi_get_event_status(u32 event, acpi_event_status * event_status);
+/*
+ * GPE Interfaces
+ */
acpi_status acpi_set_gpe_type(acpi_handle gpe_device, u32 gpe_number, u8 type);
acpi_status acpi_enable_gpe(acpi_handle gpe_device, u32 gpe_number);
@@ -263,6 +287,12 @@
u32 gpe_number,
u32 flags, acpi_event_status * event_status);
+acpi_status acpi_disable_all_gpes(void);
+
+acpi_status acpi_enable_all_runtime_gpes(void);
+
+acpi_status acpi_get_gpe_device(u32 gpe_index, acpi_handle *gpe_device);
+
acpi_status
acpi_install_gpe_block(acpi_handle gpe_device,
struct acpi_generic_address *gpe_block_address,
@@ -313,6 +343,8 @@
/*
* Hardware (ACPI device) interfaces
*/
+acpi_status acpi_reset(void);
+
acpi_status acpi_get_register(u32 register_id, u32 * return_value);
acpi_status acpi_get_register_unlocked(u32 register_id, u32 *return_value);
@@ -320,12 +352,14 @@
acpi_status acpi_set_register(u32 register_id, u32 value);
acpi_status
-acpi_set_firmware_waking_vector(acpi_physical_address physical_address);
+acpi_set_firmware_waking_vector(u32 physical_address);
-#ifdef ACPI_FUTURE_USAGE
acpi_status
-acpi_get_firmware_waking_vector(acpi_physical_address * physical_address);
-#endif
+acpi_set_firmware_waking_vector64(u64 physical_address);
+
+acpi_status acpi_read(u32 *value, struct acpi_generic_address *reg);
+
+acpi_status acpi_write(u32 value, struct acpi_generic_address *reg);
acpi_status
acpi_get_sleep_type_data(u8 sleep_state, u8 * slp_typ_a, u8 * slp_typ_b);
@@ -340,4 +374,42 @@
acpi_status acpi_leave_sleep_state(u8 sleep_state);
+/*
+ * Debug output
+ */
+void ACPI_INTERNAL_VAR_XFACE
+acpi_error(const char *module_name,
+ u32 line_number, const char *format, ...) ACPI_PRINTF_LIKE(3);
+
+void ACPI_INTERNAL_VAR_XFACE
+acpi_exception(const char *module_name,
+ u32 line_number,
+ acpi_status status, const char *format, ...) ACPI_PRINTF_LIKE(4);
+
+void ACPI_INTERNAL_VAR_XFACE
+acpi_warning(const char *module_name,
+ u32 line_number, const char *format, ...) ACPI_PRINTF_LIKE(3);
+
+void ACPI_INTERNAL_VAR_XFACE
+acpi_info(const char *module_name,
+ u32 line_number, const char *format, ...) ACPI_PRINTF_LIKE(3);
+
+#ifdef ACPI_DEBUG_OUTPUT
+
+void ACPI_INTERNAL_VAR_XFACE
+acpi_debug_print(u32 requested_debug_level,
+ u32 line_number,
+ const char *function_name,
+ const char *module_name,
+ u32 component_id, const char *format, ...) ACPI_PRINTF_LIKE(6);
+
+void ACPI_INTERNAL_VAR_XFACE
+acpi_debug_print_raw(u32 requested_debug_level,
+ u32 line_number,
+ const char *function_name,
+ const char *module_name,
+ u32 component_id,
+ const char *format, ...) ACPI_PRINTF_LIKE(6);
+#endif
+
#endif /* __ACXFACE_H__ */
diff --git a/include/acpi/acrestyp.h b/include/acpi/acrestyp.h
new file mode 100644
index 0000000..9ffe00f
--- /dev/null
+++ b/include/acpi/acrestyp.h
@@ -0,0 +1,405 @@
+/******************************************************************************
+ *
+ * Name: acrestyp.h - Defines, types, and structures for resource descriptors
+ *
+ *****************************************************************************/
+
+/*
+ * Copyright (C) 2000 - 2008, Intel Corp.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions, and the following disclaimer,
+ * without modification.
+ * 2. Redistributions in binary form must reproduce at minimum a disclaimer
+ * substantially similar to the "NO WARRANTY" disclaimer below
+ * ("Disclaimer") and any redistribution must be conditioned upon
+ * including a substantially similar Disclaimer requirement for further
+ * binary redistribution.
+ * 3. Neither the names of the above-listed copyright holders nor the names
+ * of any contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * Alternatively, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") version 2 as published by the Free
+ * Software Foundation.
+ *
+ * NO WARRANTY
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGES.
+ */
+
+#ifndef __ACRESTYP_H__
+#define __ACRESTYP_H__
+
+/*
+ * Definitions for Resource Attributes
+ */
+typedef u16 acpi_rs_length; /* Resource Length field is fixed at 16 bits */
+typedef u32 acpi_rsdesc_size; /* Max Resource Descriptor size is (Length+3) = (64_k-1)+3 */
+
+/*
+ * Memory Attributes
+ */
+#define ACPI_READ_ONLY_MEMORY (u8) 0x00
+#define ACPI_READ_WRITE_MEMORY (u8) 0x01
+
+#define ACPI_NON_CACHEABLE_MEMORY (u8) 0x00
+#define ACPI_CACHABLE_MEMORY (u8) 0x01
+#define ACPI_WRITE_COMBINING_MEMORY (u8) 0x02
+#define ACPI_PREFETCHABLE_MEMORY (u8) 0x03
+
+/*
+ * IO Attributes
+ * The ISA IO ranges are: n000-n0_fFh, n400-n4_fFh, n800-n8_fFh, n_c00-n_cFFh.
+ * The non-ISA IO ranges are: n100-n3_fFh, n500-n7_fFh, n900-n_bFFh, n_cd0-n_fFFh.
+ */
+#define ACPI_NON_ISA_ONLY_RANGES (u8) 0x01
+#define ACPI_ISA_ONLY_RANGES (u8) 0x02
+#define ACPI_ENTIRE_RANGE (ACPI_NON_ISA_ONLY_RANGES | ACPI_ISA_ONLY_RANGES)
+
+/* Type of translation - 1=Sparse, 0=Dense */
+
+#define ACPI_SPARSE_TRANSLATION (u8) 0x01
+
+/*
+ * IO Port Descriptor Decode
+ */
+#define ACPI_DECODE_10 (u8) 0x00 /* 10-bit IO address decode */
+#define ACPI_DECODE_16 (u8) 0x01 /* 16-bit IO address decode */
+
+/*
+ * IRQ Attributes
+ */
+#define ACPI_LEVEL_SENSITIVE (u8) 0x00
+#define ACPI_EDGE_SENSITIVE (u8) 0x01
+
+#define ACPI_ACTIVE_HIGH (u8) 0x00
+#define ACPI_ACTIVE_LOW (u8) 0x01
+
+#define ACPI_EXCLUSIVE (u8) 0x00
+#define ACPI_SHARED (u8) 0x01
+
+/*
+ * DMA Attributes
+ */
+#define ACPI_COMPATIBILITY (u8) 0x00
+#define ACPI_TYPE_A (u8) 0x01
+#define ACPI_TYPE_B (u8) 0x02
+#define ACPI_TYPE_F (u8) 0x03
+
+#define ACPI_NOT_BUS_MASTER (u8) 0x00
+#define ACPI_BUS_MASTER (u8) 0x01
+
+#define ACPI_TRANSFER_8 (u8) 0x00
+#define ACPI_TRANSFER_8_16 (u8) 0x01
+#define ACPI_TRANSFER_16 (u8) 0x02
+
+/*
+ * Start Dependent Functions Priority definitions
+ */
+#define ACPI_GOOD_CONFIGURATION (u8) 0x00
+#define ACPI_ACCEPTABLE_CONFIGURATION (u8) 0x01
+#define ACPI_SUB_OPTIMAL_CONFIGURATION (u8) 0x02
+
+/*
+ * 16, 32 and 64-bit Address Descriptor resource types
+ */
+#define ACPI_MEMORY_RANGE (u8) 0x00
+#define ACPI_IO_RANGE (u8) 0x01
+#define ACPI_BUS_NUMBER_RANGE (u8) 0x02
+
+#define ACPI_ADDRESS_NOT_FIXED (u8) 0x00
+#define ACPI_ADDRESS_FIXED (u8) 0x01
+
+#define ACPI_POS_DECODE (u8) 0x00
+#define ACPI_SUB_DECODE (u8) 0x01
+
+#define ACPI_PRODUCER (u8) 0x00
+#define ACPI_CONSUMER (u8) 0x01
+
+/*
+ * If possible, pack the following structures to byte alignment
+ */
+#ifndef ACPI_MISALIGNMENT_NOT_SUPPORTED
+#pragma pack(1)
+#endif
+
+/* UUID data structures for use in vendor-defined resource descriptors */
+
+struct acpi_uuid {
+ u8 data[ACPI_UUID_LENGTH];
+};
+
+struct acpi_vendor_uuid {
+ u8 subtype;
+ u8 data[ACPI_UUID_LENGTH];
+};
+
+/*
+ * Structures used to describe device resources
+ */
+struct acpi_resource_irq {
+ u8 descriptor_length;
+ u8 triggering;
+ u8 polarity;
+ u8 sharable;
+ u8 interrupt_count;
+ u8 interrupts[1];
+};
+
+struct acpi_resource_dma {
+ u8 type;
+ u8 bus_master;
+ u8 transfer;
+ u8 channel_count;
+ u8 channels[1];
+};
+
+struct acpi_resource_start_dependent {
+ u8 descriptor_length;
+ u8 compatibility_priority;
+ u8 performance_robustness;
+};
+
+/*
+ * The END_DEPENDENT_FUNCTIONS_RESOURCE struct is not
+ * needed because it has no fields
+ */
+
+struct acpi_resource_io {
+ u8 io_decode;
+ u8 alignment;
+ u8 address_length;
+ u16 minimum;
+ u16 maximum;
+};
+
+struct acpi_resource_fixed_io {
+ u16 address;
+ u8 address_length;
+};
+
+struct acpi_resource_vendor {
+ u16 byte_length;
+ u8 byte_data[1];
+};
+
+/* Vendor resource with UUID info (introduced in ACPI 3.0) */
+
+struct acpi_resource_vendor_typed {
+ u16 byte_length;
+ u8 uuid_subtype;
+ u8 uuid[ACPI_UUID_LENGTH];
+ u8 byte_data[1];
+};
+
+struct acpi_resource_end_tag {
+ u8 checksum;
+};
+
+struct acpi_resource_memory24 {
+ u8 write_protect;
+ u16 minimum;
+ u16 maximum;
+ u16 alignment;
+ u16 address_length;
+};
+
+struct acpi_resource_memory32 {
+ u8 write_protect;
+ u32 minimum;
+ u32 maximum;
+ u32 alignment;
+ u32 address_length;
+};
+
+struct acpi_resource_fixed_memory32 {
+ u8 write_protect;
+ u32 address;
+ u32 address_length;
+};
+
+struct acpi_memory_attribute {
+ u8 write_protect;
+ u8 caching;
+ u8 range_type;
+ u8 translation;
+};
+
+struct acpi_io_attribute {
+ u8 range_type;
+ u8 translation;
+ u8 translation_type;
+ u8 reserved1;
+};
+
+union acpi_resource_attribute {
+ struct acpi_memory_attribute mem;
+ struct acpi_io_attribute io;
+
+ /* Used for the *word_space macros */
+
+ u8 type_specific;
+};
+
+struct acpi_resource_source {
+ u8 index;
+ u16 string_length;
+ char *string_ptr;
+};
+
+/* Fields common to all address descriptors, 16/32/64 bit */
+
+#define ACPI_RESOURCE_ADDRESS_COMMON \
+ u8 resource_type; \
+ u8 producer_consumer; \
+ u8 decode; \
+ u8 min_address_fixed; \
+ u8 max_address_fixed; \
+ union acpi_resource_attribute info;
+
+struct acpi_resource_address {
+ACPI_RESOURCE_ADDRESS_COMMON};
+
+struct acpi_resource_address16 {
+ ACPI_RESOURCE_ADDRESS_COMMON u16 granularity;
+ u16 minimum;
+ u16 maximum;
+ u16 translation_offset;
+ u16 address_length;
+ struct acpi_resource_source resource_source;
+};
+
+struct acpi_resource_address32 {
+ ACPI_RESOURCE_ADDRESS_COMMON u32 granularity;
+ u32 minimum;
+ u32 maximum;
+ u32 translation_offset;
+ u32 address_length;
+ struct acpi_resource_source resource_source;
+};
+
+struct acpi_resource_address64 {
+ ACPI_RESOURCE_ADDRESS_COMMON u64 granularity;
+ u64 minimum;
+ u64 maximum;
+ u64 translation_offset;
+ u64 address_length;
+ struct acpi_resource_source resource_source;
+};
+
+struct acpi_resource_extended_address64 {
+ ACPI_RESOURCE_ADDRESS_COMMON u8 revision_iD;
+ u64 granularity;
+ u64 minimum;
+ u64 maximum;
+ u64 translation_offset;
+ u64 address_length;
+ u64 type_specific;
+};
+
+struct acpi_resource_extended_irq {
+ u8 producer_consumer;
+ u8 triggering;
+ u8 polarity;
+ u8 sharable;
+ u8 interrupt_count;
+ struct acpi_resource_source resource_source;
+ u32 interrupts[1];
+};
+
+struct acpi_resource_generic_register {
+ u8 space_id;
+ u8 bit_width;
+ u8 bit_offset;
+ u8 access_size;
+ u64 address;
+};
+
+/* ACPI_RESOURCE_TYPEs */
+
+#define ACPI_RESOURCE_TYPE_IRQ 0
+#define ACPI_RESOURCE_TYPE_DMA 1
+#define ACPI_RESOURCE_TYPE_START_DEPENDENT 2
+#define ACPI_RESOURCE_TYPE_END_DEPENDENT 3
+#define ACPI_RESOURCE_TYPE_IO 4
+#define ACPI_RESOURCE_TYPE_FIXED_IO 5
+#define ACPI_RESOURCE_TYPE_VENDOR 6
+#define ACPI_RESOURCE_TYPE_END_TAG 7
+#define ACPI_RESOURCE_TYPE_MEMORY24 8
+#define ACPI_RESOURCE_TYPE_MEMORY32 9
+#define ACPI_RESOURCE_TYPE_FIXED_MEMORY32 10
+#define ACPI_RESOURCE_TYPE_ADDRESS16 11
+#define ACPI_RESOURCE_TYPE_ADDRESS32 12
+#define ACPI_RESOURCE_TYPE_ADDRESS64 13
+#define ACPI_RESOURCE_TYPE_EXTENDED_ADDRESS64 14 /* ACPI 3.0 */
+#define ACPI_RESOURCE_TYPE_EXTENDED_IRQ 15
+#define ACPI_RESOURCE_TYPE_GENERIC_REGISTER 16
+#define ACPI_RESOURCE_TYPE_MAX 16
+
+/* Master union for resource descriptors */
+
+union acpi_resource_data {
+ struct acpi_resource_irq irq;
+ struct acpi_resource_dma dma;
+ struct acpi_resource_start_dependent start_dpf;
+ struct acpi_resource_io io;
+ struct acpi_resource_fixed_io fixed_io;
+ struct acpi_resource_vendor vendor;
+ struct acpi_resource_vendor_typed vendor_typed;
+ struct acpi_resource_end_tag end_tag;
+ struct acpi_resource_memory24 memory24;
+ struct acpi_resource_memory32 memory32;
+ struct acpi_resource_fixed_memory32 fixed_memory32;
+ struct acpi_resource_address16 address16;
+ struct acpi_resource_address32 address32;
+ struct acpi_resource_address64 address64;
+ struct acpi_resource_extended_address64 ext_address64;
+ struct acpi_resource_extended_irq extended_irq;
+ struct acpi_resource_generic_register generic_reg;
+
+ /* Common fields */
+
+ struct acpi_resource_address address; /* Common 16/32/64 address fields */
+};
+
+/* Common resource header */
+
+struct acpi_resource {
+ u32 type;
+ u32 length;
+ union acpi_resource_data data;
+};
+
+/* restore default alignment */
+
+#pragma pack()
+
+#define ACPI_RS_SIZE_NO_DATA 8 /* Id + Length fields */
+#define ACPI_RS_SIZE_MIN (u32) ACPI_ROUND_UP_TO_NATIVE_WORD (12)
+#define ACPI_RS_SIZE(type) (u32) (ACPI_RS_SIZE_NO_DATA + sizeof (type))
+
+#define ACPI_NEXT_RESOURCE(res) (struct acpi_resource *)((u8 *) res + res->length)
+
+struct acpi_pci_routing_table {
+ u32 length;
+ u32 pin;
+ acpi_integer address; /* here for 64-bit alignment */
+ u32 source_index;
+ char source[4]; /* pad to 64 bits so sizeof() works in all cases */
+};
+
+#endif /* __ACRESTYP_H__ */
diff --git a/include/acpi/actbl.h b/include/acpi/actbl.h
index 13a3d9a..813e4b6 100644
--- a/include/acpi/actbl.h
+++ b/include/acpi/actbl.h
@@ -288,6 +288,31 @@
#define ACPI_FADT_OFFSET(f) (u8) ACPI_OFFSET (struct acpi_table_fadt, f)
+union acpi_name_union {
+ u32 integer;
+ char ascii[4];
+};
+
+/*
+ * Internal ACPI Table Descriptor. One per ACPI table
+ */
+struct acpi_table_desc {
+ acpi_physical_address address;
+ struct acpi_table_header *pointer;
+ u32 length; /* Length fixed at 32 bits */
+ union acpi_name_union signature;
+ acpi_owner_id owner_id;
+ u8 flags;
+};
+
+/* Flags for above */
+
+#define ACPI_TABLE_ORIGIN_UNKNOWN (0)
+#define ACPI_TABLE_ORIGIN_MAPPED (1)
+#define ACPI_TABLE_ORIGIN_ALLOCATED (2)
+#define ACPI_TABLE_ORIGIN_MASK (3)
+#define ACPI_TABLE_IS_LOADED (4)
+
/*
* Get the remaining ACPI tables
*/
diff --git a/include/acpi/actbl1.h b/include/acpi/actbl1.h
index 63f5b4c..18963b9 100644
--- a/include/acpi/actbl1.h
+++ b/include/acpi/actbl1.h
@@ -627,7 +627,7 @@
u32 uncorrectable_error_mask;
u32 uncorrectable_error_severity;
u32 correctable_error_mask;
- u32 advanced_error_cababilities;
+ u32 advanced_error_capabilities;
};
/* Hardware Error Notification */
diff --git a/include/acpi/actypes.h b/include/acpi/actypes.h
index 8222e8d..a20aab5 100644
--- a/include/acpi/actypes.h
+++ b/include/acpi/actypes.h
@@ -204,11 +204,10 @@
/*******************************************************************************
*
- * OS-dependent and compiler-dependent types
+ * OS-dependent types
*
* If the defaults below are not appropriate for the host system, they can
- * be defined in the compiler-specific or OS-specific header, and this will
- * take precedence.
+ * be defined in the OS-specific header, and this will take precedence.
*
******************************************************************************/
@@ -218,12 +217,6 @@
#define acpi_thread_id acpi_size
#endif
-/* Object returned from acpi_os_create_lock */
-
-#ifndef acpi_spinlock
-#define acpi_spinlock void *
-#endif
-
/* Flags for acpi_os_acquire_lock/acpi_os_release_lock */
#ifndef acpi_cpu_flags
@@ -233,8 +226,50 @@
/* Object returned from acpi_os_create_cache */
#ifndef acpi_cache_t
+#ifdef ACPI_USE_LOCAL_CACHE
#define acpi_cache_t struct acpi_memory_list
+#else
+#define acpi_cache_t void *
#endif
+#endif
+
+/*
+ * Synchronization objects - Mutexes, Semaphores, and spin_locks
+ */
+#if (ACPI_MUTEX_TYPE == ACPI_BINARY_SEMAPHORE)
+/*
+ * These macros are used if the host OS does not support a mutex object.
+ * Map the OSL Mutex interfaces to binary semaphores.
+ */
+#define acpi_mutex acpi_semaphore
+#define acpi_os_create_mutex(out_handle) acpi_os_create_semaphore (1, 1, out_handle)
+#define acpi_os_delete_mutex(handle) (void) acpi_os_delete_semaphore (handle)
+#define acpi_os_acquire_mutex(handle,time) acpi_os_wait_semaphore (handle, 1, time)
+#define acpi_os_release_mutex(handle) (void) acpi_os_signal_semaphore (handle, 1)
+#endif
+
+/* Configurable types for synchronization objects */
+
+#ifndef acpi_spinlock
+#define acpi_spinlock void *
+#endif
+
+#ifndef acpi_semaphore
+#define acpi_semaphore void *
+#endif
+
+#ifndef acpi_mutex
+#define acpi_mutex void *
+#endif
+
+/*******************************************************************************
+ *
+ * Compiler-dependent types
+ *
+ * If the defaults below are not appropriate for the host compiler, they can
+ * be defined in the compiler-specific header, and this will take precedence.
+ *
+ ******************************************************************************/
/* Use C99 uintptr_t for pointer casting if available, "void *" otherwise */
@@ -268,6 +303,43 @@
#define ACPI_EXPORT_SYMBOL(symbol)
#endif
+/******************************************************************************
+ *
+ * ACPI Specification constants (Do not change unless the specification changes)
+ *
+ *****************************************************************************/
+
+/* Number of distinct FADT-based GPE register blocks (GPE0 and GPE1) */
+
+#define ACPI_MAX_GPE_BLOCKS 2
+
+/* Default ACPI register widths */
+
+#define ACPI_GPE_REGISTER_WIDTH 8
+#define ACPI_PM1_REGISTER_WIDTH 16
+#define ACPI_PM2_REGISTER_WIDTH 8
+#define ACPI_PM_TIMER_WIDTH 32
+
+/* Names within the namespace are 4 bytes long */
+
+#define ACPI_NAME_SIZE 4
+#define ACPI_PATH_SEGMENT_LENGTH 5 /* 4 chars for name + 1 char for separator */
+#define ACPI_PATH_SEPARATOR '.'
+
+/* Sizes for ACPI table headers */
+
+#define ACPI_OEM_ID_SIZE 6
+#define ACPI_OEM_TABLE_ID_SIZE 8
+
+/* ACPI/PNP hardware IDs */
+
+#define PCI_ROOT_HID_STRING "PNP0A03"
+#define PCI_EXPRESS_ROOT_HID_STRING "PNP0A08"
+
+/* PM Timer ticks per second (HZ) */
+
+#define PM_TIMER_FREQUENCY 3579545
+
/*******************************************************************************
*
* Independent types
@@ -291,13 +363,18 @@
#endif
/*
- * Mescellaneous types
+ * Miscellaneous types
*/
typedef u32 acpi_status; /* All ACPI Exceptions */
typedef u32 acpi_name; /* 4-byte ACPI name */
typedef char *acpi_string; /* Null terminated ASCII string */
typedef void *acpi_handle; /* Actually a ptr to a NS Node */
+/* Owner IDs are used to track namespace nodes for selective deletion */
+
+typedef u8 acpi_owner_id;
+#define ACPI_OWNER_ID_MAX 0xFF
+
struct uint64_struct {
u32 lo;
u32 hi;
@@ -313,13 +390,8 @@
u32 hi;
};
-/* Synchronization objects */
-
-#define acpi_mutex void *
-#define acpi_semaphore void *
-
/*
- * Acpi integer width. In ACPI version 1, integers are 32 bits. In ACPI
+ * Acpi integer width. In ACPI version 1, integers are 32 bits. In ACPI
* version 2, integers are 64 bits. Note that this pertains to the ACPI integer
* type only, not other integers used in the implementation of the ACPI CA
* subsystem.
@@ -338,10 +410,75 @@
#define ACPI_MAX16_DECIMAL_DIGITS 5
#define ACPI_MAX8_DECIMAL_DIGITS 3
+/* PM Timer ticks per second (HZ) */
+
+#define PM_TIMER_FREQUENCY 3579545
+
/*
* Constants with special meanings
*/
#define ACPI_ROOT_OBJECT ACPI_ADD_PTR (acpi_handle, NULL, ACPI_MAX_PTR)
+#define ACPI_WAIT_FOREVER 0xFFFF /* u16, as per ACPI spec */
+#define ACPI_DO_NOT_WAIT 0
+
+/*******************************************************************************
+ *
+ * Commonly used macros
+ *
+ ******************************************************************************/
+
+/* Data manipulation */
+
+#define ACPI_LOWORD(l) ((u16)(u32)(l))
+#define ACPI_HIWORD(l) ((u16)((((u32)(l)) >> 16) & 0xFFFF))
+#define ACPI_LOBYTE(l) ((u8)(u16)(l))
+#define ACPI_HIBYTE(l) ((u8)((((u16)(l)) >> 8) & 0xFF))
+
+/* Full 64-bit integer must be available on both 32-bit and 64-bit platforms */
+
+struct acpi_integer_overlay {
+ u32 lo_dword;
+ u32 hi_dword;
+};
+
+#define ACPI_LODWORD(integer) (ACPI_CAST_PTR (struct acpi_integer_overlay, &integer)->lo_dword)
+#define ACPI_HIDWORD(integer) (ACPI_CAST_PTR (struct acpi_integer_overlay, &integer)->hi_dword)
+
+#define ACPI_SET_BIT(target,bit) ((target) |= (bit))
+#define ACPI_CLEAR_BIT(target,bit) ((target) &= ~(bit))
+#define ACPI_MIN(a,b) (((a)<(b))?(a):(b))
+#define ACPI_MAX(a,b) (((a)>(b))?(a):(b))
+
+/* Size calculation */
+
+#define ACPI_ARRAY_LENGTH(x) (sizeof(x) / sizeof((x)[0]))
+
+/* Pointer manipulation */
+
+#define ACPI_CAST_PTR(t, p) ((t *) (acpi_uintptr_t) (p))
+#define ACPI_CAST_INDIRECT_PTR(t, p) ((t **) (acpi_uintptr_t) (p))
+#define ACPI_ADD_PTR(t, a, b) ACPI_CAST_PTR (t, (ACPI_CAST_PTR (u8, (a)) + (acpi_size)(b)))
+#define ACPI_PTR_DIFF(a, b) (acpi_size) (ACPI_CAST_PTR (u8, (a)) - ACPI_CAST_PTR (u8, (b)))
+
+/* Pointer/Integer type conversions */
+
+#define ACPI_TO_POINTER(i) ACPI_ADD_PTR (void, (void *) NULL,(acpi_size) i)
+#define ACPI_TO_INTEGER(p) ACPI_PTR_DIFF (p, (void *) NULL)
+#define ACPI_OFFSET(d, f) (acpi_size) ACPI_PTR_DIFF (&(((d *)0)->f), (void *) NULL)
+#define ACPI_PHYSADDR_TO_PTR(i) ACPI_TO_POINTER(i)
+#define ACPI_PTR_TO_PHYSADDR(i) ACPI_TO_INTEGER(i)
+
+#ifndef ACPI_MISALIGNMENT_NOT_SUPPORTED
+#define ACPI_COMPARE_NAME(a,b) (*ACPI_CAST_PTR (u32, (a)) == *ACPI_CAST_PTR (u32, (b)))
+#else
+#define ACPI_COMPARE_NAME(a,b) (!ACPI_STRNCMP (ACPI_CAST_PTR (char, (a)), ACPI_CAST_PTR (char, (b)), ACPI_NAME_SIZE))
+#endif
+
+/*******************************************************************************
+ *
+ * Miscellaneous constants
+ *
+ ******************************************************************************/
/*
* Initialization sequence
@@ -414,7 +551,7 @@
#define ACPI_NOTIFY_MAX 0x0B
/*
- * Types associated with ACPI names and objects. The first group of
+ * Types associated with ACPI names and objects. The first group of
* values (up to ACPI_TYPE_EXTERNAL_MAX) correspond to the definition
* of the ACPI object_type() operator (See the ACPI Spec). Therefore,
* only add to the first group if the spec changes.
@@ -732,6 +869,15 @@
#define ACPI_NAME_TYPE_MAX 1
/*
+ * Predefined Namespace items
+ */
+struct acpi_predefined_names {
+ char *name;
+ u8 type;
+ char *val;
+};
+
+/*
* Structure and flags for acpi_get_system_info
*/
#define ACPI_SYS_MODE_UNKNOWN 0x0000
@@ -787,7 +933,7 @@
u16 opcode,
u32 aml_offset, void *context);
-/* Table Event handler (Load, load_table etc) and types */
+/* Table Event handler (Load, load_table, etc.) and types */
typedef
acpi_status(*acpi_tbl_handler) (u32 event, void *table, void *context);
@@ -823,6 +969,12 @@
#define ACPI_INTERRUPT_NOT_HANDLED 0x00
#define ACPI_INTERRUPT_HANDLED 0x01
+/* Length of _HID, _UID, _CID, and UUID values */
+
+#define ACPI_DEVICE_ID_LENGTH 0x09
+#define ACPI_MAX_CID_LENGTH 48
+#define ACPI_UUID_LENGTH 16
+
/* Common string version of device HIDs and UIDs */
struct acpica_device_id {
@@ -900,357 +1052,28 @@
};
/*
- * Definitions for Resource Attributes
+ * struct acpi_memory_list is used only if the ACPICA local cache is enabled
*/
-typedef u16 acpi_rs_length; /* Resource Length field is fixed at 16 bits */
-typedef u32 acpi_rsdesc_size; /* Max Resource Descriptor size is (Length+3) = (64_k-1)+3 */
+struct acpi_memory_list {
+ char *list_name;
+ void *list_head;
+ u16 object_size;
+ u16 max_depth;
+ u16 current_depth;
+ u16 link_offset;
-/*
- * Memory Attributes
- */
-#define ACPI_READ_ONLY_MEMORY (u8) 0x00
-#define ACPI_READ_WRITE_MEMORY (u8) 0x01
+#ifdef ACPI_DBG_TRACK_ALLOCATIONS
-#define ACPI_NON_CACHEABLE_MEMORY (u8) 0x00
-#define ACPI_CACHABLE_MEMORY (u8) 0x01
-#define ACPI_WRITE_COMBINING_MEMORY (u8) 0x02
-#define ACPI_PREFETCHABLE_MEMORY (u8) 0x03
+ /* Statistics for debug memory tracking only */
-/*
- * IO Attributes
- * The ISA IO ranges are: n000-n0_fFh, n400-n4_fFh, n800-n8_fFh, n_c00-n_cFFh.
- * The non-ISA IO ranges are: n100-n3_fFh, n500-n7_fFh, n900-n_bFFh, n_cd0-n_fFFh.
- */
-#define ACPI_NON_ISA_ONLY_RANGES (u8) 0x01
-#define ACPI_ISA_ONLY_RANGES (u8) 0x02
-#define ACPI_ENTIRE_RANGE (ACPI_NON_ISA_ONLY_RANGES | ACPI_ISA_ONLY_RANGES)
-
-/* Type of translation - 1=Sparse, 0=Dense */
-
-#define ACPI_SPARSE_TRANSLATION (u8) 0x01
-
-/*
- * IO Port Descriptor Decode
- */
-#define ACPI_DECODE_10 (u8) 0x00 /* 10-bit IO address decode */
-#define ACPI_DECODE_16 (u8) 0x01 /* 16-bit IO address decode */
-
-/*
- * IRQ Attributes
- */
-#define ACPI_LEVEL_SENSITIVE (u8) 0x00
-#define ACPI_EDGE_SENSITIVE (u8) 0x01
-
-#define ACPI_ACTIVE_HIGH (u8) 0x00
-#define ACPI_ACTIVE_LOW (u8) 0x01
-
-#define ACPI_EXCLUSIVE (u8) 0x00
-#define ACPI_SHARED (u8) 0x01
-
-/*
- * DMA Attributes
- */
-#define ACPI_COMPATIBILITY (u8) 0x00
-#define ACPI_TYPE_A (u8) 0x01
-#define ACPI_TYPE_B (u8) 0x02
-#define ACPI_TYPE_F (u8) 0x03
-
-#define ACPI_NOT_BUS_MASTER (u8) 0x00
-#define ACPI_BUS_MASTER (u8) 0x01
-
-#define ACPI_TRANSFER_8 (u8) 0x00
-#define ACPI_TRANSFER_8_16 (u8) 0x01
-#define ACPI_TRANSFER_16 (u8) 0x02
-
-/*
- * Start Dependent Functions Priority definitions
- */
-#define ACPI_GOOD_CONFIGURATION (u8) 0x00
-#define ACPI_ACCEPTABLE_CONFIGURATION (u8) 0x01
-#define ACPI_SUB_OPTIMAL_CONFIGURATION (u8) 0x02
-
-/*
- * 16, 32 and 64-bit Address Descriptor resource types
- */
-#define ACPI_MEMORY_RANGE (u8) 0x00
-#define ACPI_IO_RANGE (u8) 0x01
-#define ACPI_BUS_NUMBER_RANGE (u8) 0x02
-
-#define ACPI_ADDRESS_NOT_FIXED (u8) 0x00
-#define ACPI_ADDRESS_FIXED (u8) 0x01
-
-#define ACPI_POS_DECODE (u8) 0x00
-#define ACPI_SUB_DECODE (u8) 0x01
-
-#define ACPI_PRODUCER (u8) 0x00
-#define ACPI_CONSUMER (u8) 0x01
-
-/*
- * If possible, pack the following structures to byte alignment
- */
-#ifndef ACPI_MISALIGNMENT_NOT_SUPPORTED
-#pragma pack(1)
+ u32 total_allocated;
+ u32 total_freed;
+ u32 max_occupied;
+ u32 total_size;
+ u32 current_total_size;
+ u32 requests;
+ u32 hits;
#endif
-
-/* UUID data structures for use in vendor-defined resource descriptors */
-
-struct acpi_uuid {
- u8 data[ACPI_UUID_LENGTH];
-};
-
-struct acpi_vendor_uuid {
- u8 subtype;
- u8 data[ACPI_UUID_LENGTH];
-};
-
-/*
- * Structures used to describe device resources
- */
-struct acpi_resource_irq {
- u8 descriptor_length;
- u8 triggering;
- u8 polarity;
- u8 sharable;
- u8 interrupt_count;
- u8 interrupts[1];
-};
-
-struct acpi_resource_dma {
- u8 type;
- u8 bus_master;
- u8 transfer;
- u8 channel_count;
- u8 channels[1];
-};
-
-struct acpi_resource_start_dependent {
- u8 descriptor_length;
- u8 compatibility_priority;
- u8 performance_robustness;
-};
-
-/*
- * END_DEPENDENT_FUNCTIONS_RESOURCE struct is not
- * needed because it has no fields
- */
-
-struct acpi_resource_io {
- u8 io_decode;
- u8 alignment;
- u8 address_length;
- u16 minimum;
- u16 maximum;
-};
-
-struct acpi_resource_fixed_io {
- u16 address;
- u8 address_length;
-};
-
-struct acpi_resource_vendor {
- u16 byte_length;
- u8 byte_data[1];
-};
-
-/* Vendor resource with UUID info (introduced in ACPI 3.0) */
-
-struct acpi_resource_vendor_typed {
- u16 byte_length;
- u8 uuid_subtype;
- u8 uuid[ACPI_UUID_LENGTH];
- u8 byte_data[1];
-};
-
-struct acpi_resource_end_tag {
- u8 checksum;
-};
-
-struct acpi_resource_memory24 {
- u8 write_protect;
- u16 minimum;
- u16 maximum;
- u16 alignment;
- u16 address_length;
-};
-
-struct acpi_resource_memory32 {
- u8 write_protect;
- u32 minimum;
- u32 maximum;
- u32 alignment;
- u32 address_length;
-};
-
-struct acpi_resource_fixed_memory32 {
- u8 write_protect;
- u32 address;
- u32 address_length;
-};
-
-struct acpi_memory_attribute {
- u8 write_protect;
- u8 caching;
- u8 range_type;
- u8 translation;
-};
-
-struct acpi_io_attribute {
- u8 range_type;
- u8 translation;
- u8 translation_type;
- u8 reserved1;
-};
-
-union acpi_resource_attribute {
- struct acpi_memory_attribute mem;
- struct acpi_io_attribute io;
-
- /* Used for the *word_space macros */
-
- u8 type_specific;
-};
-
-struct acpi_resource_source {
- u8 index;
- u16 string_length;
- char *string_ptr;
-};
-
-/* Fields common to all address descriptors, 16/32/64 bit */
-
-#define ACPI_RESOURCE_ADDRESS_COMMON \
- u8 resource_type; \
- u8 producer_consumer; \
- u8 decode; \
- u8 min_address_fixed; \
- u8 max_address_fixed; \
- union acpi_resource_attribute info;
-
-struct acpi_resource_address {
-ACPI_RESOURCE_ADDRESS_COMMON};
-
-struct acpi_resource_address16 {
- ACPI_RESOURCE_ADDRESS_COMMON u16 granularity;
- u16 minimum;
- u16 maximum;
- u16 translation_offset;
- u16 address_length;
- struct acpi_resource_source resource_source;
-};
-
-struct acpi_resource_address32 {
- ACPI_RESOURCE_ADDRESS_COMMON u32 granularity;
- u32 minimum;
- u32 maximum;
- u32 translation_offset;
- u32 address_length;
- struct acpi_resource_source resource_source;
-};
-
-struct acpi_resource_address64 {
- ACPI_RESOURCE_ADDRESS_COMMON u64 granularity;
- u64 minimum;
- u64 maximum;
- u64 translation_offset;
- u64 address_length;
- struct acpi_resource_source resource_source;
-};
-
-struct acpi_resource_extended_address64 {
- ACPI_RESOURCE_ADDRESS_COMMON u8 revision_iD;
- u64 granularity;
- u64 minimum;
- u64 maximum;
- u64 translation_offset;
- u64 address_length;
- u64 type_specific;
-};
-
-struct acpi_resource_extended_irq {
- u8 producer_consumer;
- u8 triggering;
- u8 polarity;
- u8 sharable;
- u8 interrupt_count;
- struct acpi_resource_source resource_source;
- u32 interrupts[1];
-};
-
-struct acpi_resource_generic_register {
- u8 space_id;
- u8 bit_width;
- u8 bit_offset;
- u8 access_size;
- u64 address;
-};
-
-/* ACPI_RESOURCE_TYPEs */
-
-#define ACPI_RESOURCE_TYPE_IRQ 0
-#define ACPI_RESOURCE_TYPE_DMA 1
-#define ACPI_RESOURCE_TYPE_START_DEPENDENT 2
-#define ACPI_RESOURCE_TYPE_END_DEPENDENT 3
-#define ACPI_RESOURCE_TYPE_IO 4
-#define ACPI_RESOURCE_TYPE_FIXED_IO 5
-#define ACPI_RESOURCE_TYPE_VENDOR 6
-#define ACPI_RESOURCE_TYPE_END_TAG 7
-#define ACPI_RESOURCE_TYPE_MEMORY24 8
-#define ACPI_RESOURCE_TYPE_MEMORY32 9
-#define ACPI_RESOURCE_TYPE_FIXED_MEMORY32 10
-#define ACPI_RESOURCE_TYPE_ADDRESS16 11
-#define ACPI_RESOURCE_TYPE_ADDRESS32 12
-#define ACPI_RESOURCE_TYPE_ADDRESS64 13
-#define ACPI_RESOURCE_TYPE_EXTENDED_ADDRESS64 14 /* ACPI 3.0 */
-#define ACPI_RESOURCE_TYPE_EXTENDED_IRQ 15
-#define ACPI_RESOURCE_TYPE_GENERIC_REGISTER 16
-#define ACPI_RESOURCE_TYPE_MAX 16
-
-union acpi_resource_data {
- struct acpi_resource_irq irq;
- struct acpi_resource_dma dma;
- struct acpi_resource_start_dependent start_dpf;
- struct acpi_resource_io io;
- struct acpi_resource_fixed_io fixed_io;
- struct acpi_resource_vendor vendor;
- struct acpi_resource_vendor_typed vendor_typed;
- struct acpi_resource_end_tag end_tag;
- struct acpi_resource_memory24 memory24;
- struct acpi_resource_memory32 memory32;
- struct acpi_resource_fixed_memory32 fixed_memory32;
- struct acpi_resource_address16 address16;
- struct acpi_resource_address32 address32;
- struct acpi_resource_address64 address64;
- struct acpi_resource_extended_address64 ext_address64;
- struct acpi_resource_extended_irq extended_irq;
- struct acpi_resource_generic_register generic_reg;
-
- /* Common fields */
-
- struct acpi_resource_address address; /* Common 16/32/64 address fields */
-};
-
-struct acpi_resource {
- u32 type;
- u32 length;
- union acpi_resource_data data;
-};
-
-/* restore default alignment */
-
-#pragma pack()
-
-#define ACPI_RS_SIZE_NO_DATA 8 /* Id + Length fields */
-#define ACPI_RS_SIZE_MIN (u32) ACPI_ROUND_UP_TO_NATIVE_WORD (12)
-#define ACPI_RS_SIZE(type) (u32) (ACPI_RS_SIZE_NO_DATA + sizeof (type))
-
-#define ACPI_NEXT_RESOURCE(res) (struct acpi_resource *)((u8 *) res + res->length)
-
-struct acpi_pci_routing_table {
- u32 length;
- u32 pin;
- acpi_integer address; /* here for 64-bit alignment */
- u32 source_index;
- char source[4]; /* pad to 64 bits so sizeof() works in all cases */
};
#endif /* __ACTYPES_H__ */
diff --git a/include/acpi/platform/acenv.h b/include/acpi/platform/acenv.h
index fcd2572..e62f10d 100644
--- a/include/acpi/platform/acenv.h
+++ b/include/acpi/platform/acenv.h
@@ -44,14 +44,26 @@
#ifndef __ACENV_H__
#define __ACENV_H__
-/*
+/* Types for ACPI_MUTEX_TYPE */
+
+#define ACPI_BINARY_SEMAPHORE 0
+#define ACPI_OSL_MUTEX 1
+
+/* Types for DEBUGGER_THREADING */
+
+#define DEBUGGER_SINGLE_THREADED 0
+#define DEBUGGER_MULTI_THREADED 1
+
+/******************************************************************************
+ *
* Configuration for ACPI tools and utilities
- */
+ *
+ *****************************************************************************/
#ifdef ACPI_LIBRARY
/*
* Note: The non-debug version of the acpi_library does not contain any
- * debug support, for minimimal size. The debug version uses ACPI_FULL_DEBUG
+ * debug support, for minimal size. The debug version uses ACPI_FULL_DEBUG
*/
#define ACPI_USE_LOCAL_CACHE
#endif
@@ -75,17 +87,6 @@
#define ACPI_DBG_TRACK_ALLOCATIONS
#endif
-#ifdef ACPI_DASM_APP
-#ifndef MSDOS
-#define ACPI_DEBUG_OUTPUT
-#endif
-#define ACPI_APPLICATION
-#define ACPI_DISASSEMBLER
-#define ACPI_NO_METHOD_EXECUTION
-#define ACPI_LARGE_NAMESPACE_NODE
-#define ACPI_DATA_TABLE_DISASSEMBLY
-#endif
-
#ifdef ACPI_APPLICATION
#define ACPI_USE_SYSTEM_CLIBRARY
#define ACPI_USE_LOCAL_CACHE
@@ -179,6 +180,19 @@
/*! [End] no source code translation !*/
+/******************************************************************************
+ *
+ * Miscellaneous configuration
+ *
+ *****************************************************************************/
+
+/*
+ * Are mutexes supported by the host? default is no, use binary semaphores.
+ */
+#ifndef ACPI_MUTEX_TYPE
+#define ACPI_MUTEX_TYPE ACPI_BINARY_SEMAPHORE
+#endif
+
/*
* Debugger threading model
* Use single threaded if the entire subsystem is contained in an application
@@ -187,9 +201,6 @@
* By default the model is single threaded if ACPI_APPLICATION is set,
* multi-threaded if ACPI_APPLICATION is not set.
*/
-#define DEBUGGER_SINGLE_THREADED 0
-#define DEBUGGER_MULTI_THREADED 1
-
#ifndef DEBUGGER_THREADING
#ifdef ACPI_APPLICATION
#define DEBUGGER_THREADING DEBUGGER_SINGLE_THREADED
diff --git a/include/acpi/platform/aclinux.h b/include/acpi/platform/aclinux.h
index 0515e75..6d49b2a 100644
--- a/include/acpi/platform/aclinux.h
+++ b/include/acpi/platform/aclinux.h
@@ -46,6 +46,7 @@
#define ACPI_USE_SYSTEM_CLIBRARY
#define ACPI_USE_DO_WHILE_0
+#define ACPI_MUTEX_TYPE ACPI_BINARY_SEMAPHORE
#ifdef __KERNEL__
@@ -70,9 +71,6 @@
#define ACPI_EXPORT_SYMBOL(symbol) EXPORT_SYMBOL(symbol);
#define strtoul simple_strtoul
-/* Full namespace pathname length limit - arbitrary */
-#define ACPI_PATHNAME_MAX 256
-
#else /* !__KERNEL__ */
#include <stdarg.h>
diff --git a/include/linux/acpi.h b/include/linux/acpi.h
index fba8051..6fce2fc 100644
--- a/include/linux/acpi.h
+++ b/include/linux/acpi.h
@@ -131,22 +131,6 @@
*/
void acpi_unregister_gsi (u32 gsi);
-struct acpi_prt_entry {
- struct list_head node;
- struct acpi_pci_id id;
- u8 pin;
- struct {
- acpi_handle handle;
- u32 index;
- } link;
- u32 irq;
-};
-
-struct acpi_prt_list {
- int count;
- struct list_head entries;
-};
-
struct pci_dev;
int acpi_pci_irq_enable (struct pci_dev *dev);
@@ -270,6 +254,7 @@
#ifdef CONFIG_PM_SLEEP
void __init acpi_no_s4_hw_signature(void);
void __init acpi_old_suspend_ordering(void);
+void __init acpi_s4_no_nvs(void);
#endif /* CONFIG_PM_SLEEP */
#else /* CONFIG_ACPI */
diff --git a/include/linux/async_tx.h b/include/linux/async_tx.h
index 0f50d4c..45f6297 100644
--- a/include/linux/async_tx.h
+++ b/include/linux/async_tx.h
@@ -59,9 +59,7 @@
};
#ifdef CONFIG_DMA_ENGINE
-void async_tx_issue_pending_all(void);
-enum dma_status dma_wait_for_async_tx(struct dma_async_tx_descriptor *tx);
-void async_tx_run_dependencies(struct dma_async_tx_descriptor *tx);
+#define async_tx_issue_pending_all dma_issue_pending_all
#ifdef CONFIG_ARCH_HAS_ASYNC_TX_FIND_CHANNEL
#include <asm/async_tx.h>
#else
@@ -77,19 +75,6 @@
do { } while (0);
}
-static inline enum dma_status
-dma_wait_for_async_tx(struct dma_async_tx_descriptor *tx)
-{
- return DMA_SUCCESS;
-}
-
-static inline void
-async_tx_run_dependencies(struct dma_async_tx_descriptor *tx,
- struct dma_chan *host_chan)
-{
- do { } while (0);
-}
-
static inline struct dma_chan *
async_tx_find_channel(struct dma_async_tx_descriptor *depend_tx,
enum dma_transaction_type tx_type, struct page **dst, int dst_count,
diff --git a/include/linux/atmel-mci.h b/include/linux/atmel-mci.h
index 2a2213e..2f1f957 100644
--- a/include/linux/atmel-mci.h
+++ b/include/linux/atmel-mci.h
@@ -3,7 +3,7 @@
#define ATMEL_MCI_MAX_NR_SLOTS 2
-struct dma_slave;
+#include <linux/dw_dmac.h>
/**
* struct mci_slot_pdata - board-specific per-slot configuration
@@ -28,11 +28,11 @@
/**
* struct mci_platform_data - board-specific MMC/SDcard configuration
- * @dma_slave: DMA slave interface to use in data transfers, or NULL.
+ * @dma_slave: DMA slave interface to use in data transfers.
* @slot: Per-slot configuration data.
*/
struct mci_platform_data {
- struct dma_slave *dma_slave;
+ struct dw_dma_slave dma_slave;
struct mci_slot_pdata slot[ATMEL_MCI_MAX_NR_SLOTS];
};
diff --git a/include/linux/auxvec.h b/include/linux/auxvec.h
index d7afa9d..f3b5d4e 100644
--- a/include/linux/auxvec.h
+++ b/include/linux/auxvec.h
@@ -23,16 +23,16 @@
#define AT_PLATFORM 15 /* string identifying CPU for optimizations */
#define AT_HWCAP 16 /* arch dependent hints at CPU capabilities */
#define AT_CLKTCK 17 /* frequency at which times() increments */
-
+/* AT_* values 18 through 22 are reserved */
#define AT_SECURE 23 /* secure mode boolean */
-
#define AT_BASE_PLATFORM 24 /* string identifying real platform, may
* differ from AT_PLATFORM. */
+#define AT_RANDOM 25 /* address of 16 random bytes */
#define AT_EXECFN 31 /* filename of program */
#ifdef __KERNEL__
-#define AT_VECTOR_SIZE_BASE 18 /* NEW_AUX_ENT entries in auxiliary table */
+#define AT_VECTOR_SIZE_BASE 19 /* NEW_AUX_ENT entries in auxiliary table */
/* number of "#define AT_.*" above, minus {AT_NULL, AT_IGNORE, AT_NOTELF} */
#endif
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 7035cec..044467e 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -690,6 +690,8 @@
struct page **pages;
int page_order;
int nr_entries;
+ unsigned long offset;
+ int null_mapped;
};
struct req_iterator {
diff --git a/include/linux/can/core.h b/include/linux/can/core.h
index f50785a..25085cb 100644
--- a/include/linux/can/core.h
+++ b/include/linux/can/core.h
@@ -19,7 +19,7 @@
#include <linux/skbuff.h>
#include <linux/netdevice.h>
-#define CAN_VERSION "20081130"
+#define CAN_VERSION "20090105"
/* increment this number each time you change some user-space interface */
#define CAN_ABI_VERSION "8"
diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h
index 08b78c0..e267e62 100644
--- a/include/linux/cgroup.h
+++ b/include/linux/cgroup.h
@@ -52,9 +52,9 @@
* hierarchy structure */
struct cgroup *cgroup;
- /* State maintained by the cgroup system to allow
- * subsystems to be "busy". Should be accessed via css_get()
- * and css_put() */
+ /* State maintained by the cgroup system to allow subsystems
+ * to be "busy". Should be accessed via css_get(),
+ * css_tryget() and and css_put(). */
atomic_t refcnt;
@@ -64,11 +64,14 @@
/* bits in struct cgroup_subsys_state flags field */
enum {
CSS_ROOT, /* This CSS is the root of the subsystem */
+ CSS_REMOVED, /* This CSS is dead */
};
/*
- * Call css_get() to hold a reference on the cgroup;
- *
+ * Call css_get() to hold a reference on the css; it can be used
+ * for a reference obtained via:
+ * - an existing ref-counted reference to the css
+ * - task->cgroups for a locked task
*/
static inline void css_get(struct cgroup_subsys_state *css)
@@ -77,9 +80,32 @@
if (!test_bit(CSS_ROOT, &css->flags))
atomic_inc(&css->refcnt);
}
+
+static inline bool css_is_removed(struct cgroup_subsys_state *css)
+{
+ return test_bit(CSS_REMOVED, &css->flags);
+}
+
+/*
+ * Call css_tryget() to take a reference on a css if your existing
+ * (known-valid) reference isn't already ref-counted. Returns false if
+ * the css has been destroyed.
+ */
+
+static inline bool css_tryget(struct cgroup_subsys_state *css)
+{
+ if (test_bit(CSS_ROOT, &css->flags))
+ return true;
+ while (!atomic_inc_not_zero(&css->refcnt)) {
+ if (test_bit(CSS_REMOVED, &css->flags))
+ return false;
+ }
+ return true;
+}
+
/*
* css_put() should be called to release a reference taken by
- * css_get()
+ * css_get() or css_tryget()
*/
extern void __css_put(struct cgroup_subsys_state *css);
@@ -116,7 +142,7 @@
struct list_head children; /* my children */
struct cgroup *parent; /* my parent */
- struct dentry *dentry; /* cgroup fs entry */
+ struct dentry *dentry; /* cgroup fs entry, RCU protected */
/* Private pointers for each registered subsystem */
struct cgroup_subsys_state *subsys[CGROUP_SUBSYS_COUNT];
@@ -145,6 +171,9 @@
int pids_use_count;
/* Length of the current tasks_pids array */
int pids_length;
+
+ /* For RCU-protected deletion */
+ struct rcu_head rcu_head;
};
/* A css_set is a structure holding pointers to a set of
@@ -337,9 +366,23 @@
#define MAX_CGROUP_TYPE_NAMELEN 32
const char *name;
- /* Protected by RCU */
- struct cgroupfs_root *root;
+ /*
+ * Protects sibling/children links of cgroups in this
+ * hierarchy, plus protects which hierarchy (or none) the
+ * subsystem is a part of (i.e. root/sibling). To avoid
+ * potential deadlocks, the following operations should not be
+ * undertaken while holding any hierarchy_mutex:
+ *
+ * - allocating memory
+ * - initiating hotplug events
+ */
+ struct mutex hierarchy_mutex;
+ /*
+ * Link to parent, and list entry in parent's children.
+ * Protected by this->hierarchy_mutex and cgroup_lock()
+ */
+ struct cgroupfs_root *root;
struct list_head sibling;
};
diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h
index 51ea2bd..90c6074 100644
--- a/include/linux/cpuset.h
+++ b/include/linux/cpuset.h
@@ -20,8 +20,9 @@
extern int cpuset_init_early(void);
extern int cpuset_init(void);
extern void cpuset_init_smp(void);
-extern void cpuset_cpus_allowed(struct task_struct *p, cpumask_t *mask);
-extern void cpuset_cpus_allowed_locked(struct task_struct *p, cpumask_t *mask);
+extern void cpuset_cpus_allowed(struct task_struct *p, struct cpumask *mask);
+extern void cpuset_cpus_allowed_locked(struct task_struct *p,
+ struct cpumask *mask);
extern nodemask_t cpuset_mems_allowed(struct task_struct *p);
#define cpuset_current_mems_allowed (current->mems_allowed)
void cpuset_init_current_mems_allowed(void);
@@ -86,12 +87,13 @@
static inline int cpuset_init(void) { return 0; }
static inline void cpuset_init_smp(void) {}
-static inline void cpuset_cpus_allowed(struct task_struct *p, cpumask_t *mask)
+static inline void cpuset_cpus_allowed(struct task_struct *p,
+ struct cpumask *mask)
{
*mask = cpu_possible_map;
}
static inline void cpuset_cpus_allowed_locked(struct task_struct *p,
- cpumask_t *mask)
+ struct cpumask *mask)
{
*mask = cpu_possible_map;
}
diff --git a/include/linux/dmaengine.h b/include/linux/dmaengine.h
index adb0b08..64dea2a 100644
--- a/include/linux/dmaengine.h
+++ b/include/linux/dmaengine.h
@@ -29,32 +29,6 @@
#include <linux/dma-mapping.h>
/**
- * enum dma_state - resource PNP/power management state
- * @DMA_RESOURCE_SUSPEND: DMA device going into low power state
- * @DMA_RESOURCE_RESUME: DMA device returning to full power
- * @DMA_RESOURCE_AVAILABLE: DMA device available to the system
- * @DMA_RESOURCE_REMOVED: DMA device removed from the system
- */
-enum dma_state {
- DMA_RESOURCE_SUSPEND,
- DMA_RESOURCE_RESUME,
- DMA_RESOURCE_AVAILABLE,
- DMA_RESOURCE_REMOVED,
-};
-
-/**
- * enum dma_state_client - state of the channel in the client
- * @DMA_ACK: client would like to use, or was using this channel
- * @DMA_DUP: client has already seen this channel, or is not using this channel
- * @DMA_NAK: client does not want to see any more channels
- */
-enum dma_state_client {
- DMA_ACK,
- DMA_DUP,
- DMA_NAK,
-};
-
-/**
* typedef dma_cookie_t - an opaque DMA cookie
*
* if dma_cookie_t is >0 it's a DMA request cookie, <0 it's an error code
@@ -89,23 +63,13 @@
DMA_MEMSET,
DMA_MEMCPY_CRC32C,
DMA_INTERRUPT,
+ DMA_PRIVATE,
DMA_SLAVE,
};
/* last transaction type for creation of the capabilities mask */
#define DMA_TX_TYPE_END (DMA_SLAVE + 1)
-/**
- * enum dma_slave_width - DMA slave register access width.
- * @DMA_SLAVE_WIDTH_8BIT: Do 8-bit slave register accesses
- * @DMA_SLAVE_WIDTH_16BIT: Do 16-bit slave register accesses
- * @DMA_SLAVE_WIDTH_32BIT: Do 32-bit slave register accesses
- */
-enum dma_slave_width {
- DMA_SLAVE_WIDTH_8BIT,
- DMA_SLAVE_WIDTH_16BIT,
- DMA_SLAVE_WIDTH_32BIT,
-};
/**
* enum dma_ctrl_flags - DMA flags to augment operation preparation,
@@ -132,32 +96,6 @@
typedef struct { DECLARE_BITMAP(bits, DMA_TX_TYPE_END); } dma_cap_mask_t;
/**
- * struct dma_slave - Information about a DMA slave
- * @dev: device acting as DMA slave
- * @dma_dev: required DMA master device. If non-NULL, the client can not be
- * bound to other masters than this.
- * @tx_reg: physical address of data register used for
- * memory-to-peripheral transfers
- * @rx_reg: physical address of data register used for
- * peripheral-to-memory transfers
- * @reg_width: peripheral register width
- *
- * If dma_dev is non-NULL, the client can not be bound to other DMA
- * masters than the one corresponding to this device. The DMA master
- * driver may use this to determine if there is controller-specific
- * data wrapped around this struct. Drivers of platform code that sets
- * the dma_dev field must therefore make sure to use an appropriate
- * controller-specific dma slave structure wrapping this struct.
- */
-struct dma_slave {
- struct device *dev;
- struct device *dma_dev;
- dma_addr_t tx_reg;
- dma_addr_t rx_reg;
- enum dma_slave_width reg_width;
-};
-
-/**
* struct dma_chan_percpu - the per-CPU part of struct dma_chan
* @refcount: local_t used for open-coded "bigref" counting
* @memcpy_count: transaction counter
@@ -165,7 +103,6 @@
*/
struct dma_chan_percpu {
- local_t refcount;
/* stats */
unsigned long memcpy_count;
unsigned long bytes_transferred;
@@ -176,13 +113,14 @@
* @device: ptr to the dma device who supplies this channel, always !%NULL
* @cookie: last cookie value returned to client
* @chan_id: channel ID for sysfs
- * @class_dev: class device for sysfs
+ * @dev: class device for sysfs
* @refcount: kref, used in "bigref" slow-mode
* @slow_ref: indicates that the DMA channel is free
* @rcu: the DMA channel's RCU head
* @device_node: used to add this to the device chan list
* @local: per-cpu pointer to a struct dma_chan_percpu
* @client-count: how many clients are using this channel
+ * @table_count: number of appearances in the mem-to-mem allocation table
*/
struct dma_chan {
struct dma_device *device;
@@ -190,73 +128,47 @@
/* sysfs */
int chan_id;
- struct device dev;
-
- struct kref refcount;
- int slow_ref;
- struct rcu_head rcu;
+ struct dma_chan_dev *dev;
struct list_head device_node;
struct dma_chan_percpu *local;
int client_count;
+ int table_count;
};
-#define to_dma_chan(p) container_of(p, struct dma_chan, dev)
+/**
+ * struct dma_chan_dev - relate sysfs device node to backing channel device
+ * @chan - driver channel device
+ * @device - sysfs device
+ * @dev_id - parent dma_device dev_id
+ * @idr_ref - reference count to gate release of dma_device dev_id
+ */
+struct dma_chan_dev {
+ struct dma_chan *chan;
+ struct device device;
+ int dev_id;
+ atomic_t *idr_ref;
+};
+
+static inline const char *dma_chan_name(struct dma_chan *chan)
+{
+ return dev_name(&chan->dev->device);
+}
void dma_chan_cleanup(struct kref *kref);
-static inline void dma_chan_get(struct dma_chan *chan)
-{
- if (unlikely(chan->slow_ref))
- kref_get(&chan->refcount);
- else {
- local_inc(&(per_cpu_ptr(chan->local, get_cpu())->refcount));
- put_cpu();
- }
-}
-
-static inline void dma_chan_put(struct dma_chan *chan)
-{
- if (unlikely(chan->slow_ref))
- kref_put(&chan->refcount, dma_chan_cleanup);
- else {
- local_dec(&(per_cpu_ptr(chan->local, get_cpu())->refcount));
- put_cpu();
- }
-}
-
-/*
- * typedef dma_event_callback - function pointer to a DMA event callback
- * For each channel added to the system this routine is called for each client.
- * If the client would like to use the channel it returns '1' to signal (ack)
- * the dmaengine core to take out a reference on the channel and its
- * corresponding device. A client must not 'ack' an available channel more
- * than once. When a channel is removed all clients are notified. If a client
- * is using the channel it must 'ack' the removal. A client must not 'ack' a
- * removed channel more than once.
- * @client - 'this' pointer for the client context
- * @chan - channel to be acted upon
- * @state - available or removed
- */
-struct dma_client;
-typedef enum dma_state_client (*dma_event_callback) (struct dma_client *client,
- struct dma_chan *chan, enum dma_state state);
-
/**
- * struct dma_client - info on the entity making use of DMA services
- * @event_callback: func ptr to call when something happens
- * @cap_mask: only return channels that satisfy the requested capabilities
- * a value of zero corresponds to any capability
- * @slave: data for preparing slave transfer. Must be non-NULL iff the
- * DMA_SLAVE capability is requested.
- * @global_node: list_head for global dma_client_list
+ * typedef dma_filter_fn - callback filter for dma_request_channel
+ * @chan: channel to be reviewed
+ * @filter_param: opaque parameter passed through dma_request_channel
+ *
+ * When this optional parameter is specified in a call to dma_request_channel a
+ * suitable channel is passed to this routine for further dispositioning before
+ * being returned. Where 'suitable' indicates a non-busy channel that
+ * satisfies the given capability mask. It returns 'true' to indicate that the
+ * channel is suitable.
*/
-struct dma_client {
- dma_event_callback event_callback;
- dma_cap_mask_t cap_mask;
- struct dma_slave *slave;
- struct list_head global_node;
-};
+typedef bool (*dma_filter_fn)(struct dma_chan *chan, void *filter_param);
typedef void (*dma_async_tx_callback)(void *dma_async_param);
/**
@@ -323,14 +235,10 @@
dma_cap_mask_t cap_mask;
int max_xor;
- struct kref refcount;
- struct completion done;
-
int dev_id;
struct device *dev;
- int (*device_alloc_chan_resources)(struct dma_chan *chan,
- struct dma_client *client);
+ int (*device_alloc_chan_resources)(struct dma_chan *chan);
void (*device_free_chan_resources)(struct dma_chan *chan);
struct dma_async_tx_descriptor *(*device_prep_dma_memcpy)(
@@ -362,9 +270,8 @@
/* --- public DMA engine API --- */
-void dma_async_client_register(struct dma_client *client);
-void dma_async_client_unregister(struct dma_client *client);
-void dma_async_client_chan_request(struct dma_client *client);
+void dmaengine_get(void);
+void dmaengine_put(void);
dma_cookie_t dma_async_memcpy_buf_to_buf(struct dma_chan *chan,
void *dest, void *src, size_t len);
dma_cookie_t dma_async_memcpy_buf_to_pg(struct dma_chan *chan,
@@ -406,6 +313,12 @@
set_bit(tx_type, dstp->bits);
}
+#define dma_cap_zero(mask) __dma_cap_zero(&(mask))
+static inline void __dma_cap_zero(dma_cap_mask_t *dstp)
+{
+ bitmap_zero(dstp->bits, DMA_TX_TYPE_END);
+}
+
#define dma_has_cap(tx, mask) __dma_has_cap((tx), &(mask))
static inline int
__dma_has_cap(enum dma_transaction_type tx_type, dma_cap_mask_t *srcp)
@@ -475,11 +388,25 @@
}
enum dma_status dma_sync_wait(struct dma_chan *chan, dma_cookie_t cookie);
+#ifdef CONFIG_DMA_ENGINE
+enum dma_status dma_wait_for_async_tx(struct dma_async_tx_descriptor *tx);
+#else
+static inline enum dma_status dma_wait_for_async_tx(struct dma_async_tx_descriptor *tx)
+{
+ return DMA_SUCCESS;
+}
+#endif
/* --- DMA device --- */
int dma_async_device_register(struct dma_device *device);
void dma_async_device_unregister(struct dma_device *device);
+void dma_run_dependencies(struct dma_async_tx_descriptor *tx);
+struct dma_chan *dma_find_channel(enum dma_transaction_type tx_type);
+void dma_issue_pending_all(void);
+#define dma_request_channel(mask, x, y) __dma_request_channel(&(mask), x, y)
+struct dma_chan *__dma_request_channel(dma_cap_mask_t *mask, dma_filter_fn fn, void *fn_param);
+void dma_release_channel(struct dma_chan *chan);
/* --- Helper iov-locking functions --- */
diff --git a/include/linux/dw_dmac.h b/include/linux/dw_dmac.h
index 04d217b..d797dde 100644
--- a/include/linux/dw_dmac.h
+++ b/include/linux/dw_dmac.h
@@ -22,14 +22,34 @@
};
/**
+ * enum dw_dma_slave_width - DMA slave register access width.
+ * @DMA_SLAVE_WIDTH_8BIT: Do 8-bit slave register accesses
+ * @DMA_SLAVE_WIDTH_16BIT: Do 16-bit slave register accesses
+ * @DMA_SLAVE_WIDTH_32BIT: Do 32-bit slave register accesses
+ */
+enum dw_dma_slave_width {
+ DW_DMA_SLAVE_WIDTH_8BIT,
+ DW_DMA_SLAVE_WIDTH_16BIT,
+ DW_DMA_SLAVE_WIDTH_32BIT,
+};
+
+/**
* struct dw_dma_slave - Controller-specific information about a slave
- * @slave: Generic information about the slave
- * @ctl_lo: Platform-specific initializer for the CTL_LO register
+ *
+ * @dma_dev: required DMA master device
+ * @tx_reg: physical address of data register used for
+ * memory-to-peripheral transfers
+ * @rx_reg: physical address of data register used for
+ * peripheral-to-memory transfers
+ * @reg_width: peripheral register width
* @cfg_hi: Platform-specific initializer for the CFG_HI register
* @cfg_lo: Platform-specific initializer for the CFG_LO register
*/
struct dw_dma_slave {
- struct dma_slave slave;
+ struct device *dma_dev;
+ dma_addr_t tx_reg;
+ dma_addr_t rx_reg;
+ enum dw_dma_slave_width reg_width;
u32 cfg_hi;
u32 cfg_lo;
};
@@ -54,9 +74,4 @@
#define DWC_CFGL_HS_DST_POL (1 << 18) /* dst handshake active low */
#define DWC_CFGL_HS_SRC_POL (1 << 19) /* src handshake active low */
-static inline struct dw_dma_slave *to_dw_dma_slave(struct dma_slave *slave)
-{
- return container_of(slave, struct dw_dma_slave, slave);
-}
-
#endif /* DW_DMAC_H */
diff --git a/include/linux/ext2_fs.h b/include/linux/ext2_fs.h
index 78c775a..121720d 100644
--- a/include/linux/ext2_fs.h
+++ b/include/linux/ext2_fs.h
@@ -194,6 +194,30 @@
#define EXT2_FL_USER_VISIBLE FS_FL_USER_VISIBLE /* User visible flags */
#define EXT2_FL_USER_MODIFIABLE FS_FL_USER_MODIFIABLE /* User modifiable flags */
+/* Flags that should be inherited by new inodes from their parent. */
+#define EXT2_FL_INHERITED (EXT2_SECRM_FL | EXT2_UNRM_FL | EXT2_COMPR_FL |\
+ EXT2_SYNC_FL | EXT2_IMMUTABLE_FL | EXT2_APPEND_FL |\
+ EXT2_NODUMP_FL | EXT2_NOATIME_FL | EXT2_COMPRBLK_FL|\
+ EXT2_NOCOMP_FL | EXT2_JOURNAL_DATA_FL |\
+ EXT2_NOTAIL_FL | EXT2_DIRSYNC_FL)
+
+/* Flags that are appropriate for regular files (all but dir-specific ones). */
+#define EXT2_REG_FLMASK (~(EXT2_DIRSYNC_FL | EXT2_TOPDIR_FL))
+
+/* Flags that are appropriate for non-directories/regular files. */
+#define EXT2_OTHER_FLMASK (EXT2_NODUMP_FL | EXT2_NOATIME_FL)
+
+/* Mask out flags that are inappropriate for the given type of inode. */
+static inline __u32 ext2_mask_flags(umode_t mode, __u32 flags)
+{
+ if (S_ISDIR(mode))
+ return flags;
+ else if (S_ISREG(mode))
+ return flags & EXT2_REG_FLMASK;
+ else
+ return flags & EXT2_OTHER_FLMASK;
+}
+
/*
* ioctl commands
*/
diff --git a/include/linux/ext2_fs_sb.h b/include/linux/ext2_fs_sb.h
index dc541f3..1cdb663 100644
--- a/include/linux/ext2_fs_sb.h
+++ b/include/linux/ext2_fs_sb.h
@@ -101,7 +101,7 @@
struct percpu_counter s_freeblocks_counter;
struct percpu_counter s_freeinodes_counter;
struct percpu_counter s_dirs_counter;
- struct blockgroup_lock s_blockgroup_lock;
+ struct blockgroup_lock *s_blockgroup_lock;
/* root of the per fs reservation window tree */
spinlock_t s_rsv_window_lock;
struct rb_root s_rsv_window_root;
@@ -111,7 +111,7 @@
static inline spinlock_t *
sb_bgl_lock(struct ext2_sb_info *sbi, unsigned int block_group)
{
- return bgl_lock_ptr(&sbi->s_blockgroup_lock, block_group);
+ return bgl_lock_ptr(sbi->s_blockgroup_lock, block_group);
}
#endif /* _LINUX_EXT2_FS_SB */
diff --git a/include/linux/ext3_fs.h b/include/linux/ext3_fs.h
index d14f029..dd495b8 100644
--- a/include/linux/ext3_fs.h
+++ b/include/linux/ext3_fs.h
@@ -178,6 +178,30 @@
#define EXT3_FL_USER_VISIBLE 0x0003DFFF /* User visible flags */
#define EXT3_FL_USER_MODIFIABLE 0x000380FF /* User modifiable flags */
+/* Flags that should be inherited by new inodes from their parent. */
+#define EXT3_FL_INHERITED (EXT3_SECRM_FL | EXT3_UNRM_FL | EXT3_COMPR_FL |\
+ EXT3_SYNC_FL | EXT3_IMMUTABLE_FL | EXT3_APPEND_FL |\
+ EXT3_NODUMP_FL | EXT3_NOATIME_FL | EXT3_COMPRBLK_FL|\
+ EXT3_NOCOMPR_FL | EXT3_JOURNAL_DATA_FL |\
+ EXT3_NOTAIL_FL | EXT3_DIRSYNC_FL)
+
+/* Flags that are appropriate for regular files (all but dir-specific ones). */
+#define EXT3_REG_FLMASK (~(EXT3_DIRSYNC_FL | EXT3_TOPDIR_FL))
+
+/* Flags that are appropriate for non-directories/regular files. */
+#define EXT3_OTHER_FLMASK (EXT3_NODUMP_FL | EXT3_NOATIME_FL)
+
+/* Mask out flags that are inappropriate for the given type of inode. */
+static inline __u32 ext3_mask_flags(umode_t mode, __u32 flags)
+{
+ if (S_ISDIR(mode))
+ return flags;
+ else if (S_ISREG(mode))
+ return flags & EXT3_REG_FLMASK;
+ else
+ return flags & EXT3_OTHER_FLMASK;
+}
+
/*
* Inode dynamic state flags
*/
@@ -354,6 +378,13 @@
#define EXT3_ORPHAN_FS 0x0004 /* Orphans being recovered */
/*
+ * Misc. filesystem flags
+ */
+#define EXT2_FLAGS_SIGNED_HASH 0x0001 /* Signed dirhash in use */
+#define EXT2_FLAGS_UNSIGNED_HASH 0x0002 /* Unsigned dirhash in use */
+#define EXT2_FLAGS_TEST_FILESYS 0x0004 /* to test development code */
+
+/*
* Mount flags
*/
#define EXT3_MOUNT_CHECK 0x00001 /* Do mount-time checks */
@@ -489,7 +520,23 @@
__u16 s_reserved_word_pad;
__le32 s_default_mount_opts;
__le32 s_first_meta_bg; /* First metablock block group */
- __u32 s_reserved[190]; /* Padding to the end of the block */
+ __le32 s_mkfs_time; /* When the filesystem was created */
+ __le32 s_jnl_blocks[17]; /* Backup of the journal inode */
+ /* 64bit support valid if EXT4_FEATURE_COMPAT_64BIT */
+/*150*/ __le32 s_blocks_count_hi; /* Blocks count */
+ __le32 s_r_blocks_count_hi; /* Reserved blocks count */
+ __le32 s_free_blocks_count_hi; /* Free blocks count */
+ __le16 s_min_extra_isize; /* All inodes have at least # bytes */
+ __le16 s_want_extra_isize; /* New inodes should reserve # bytes */
+ __le32 s_flags; /* Miscellaneous flags */
+ __le16 s_raid_stride; /* RAID stride */
+ __le16 s_mmp_interval; /* # seconds to wait in MMP checking */
+ __le64 s_mmp_block; /* Block for multi-mount protection */
+ __le32 s_raid_stripe_width; /* blocks on all data disks (N*stride)*/
+ __u8 s_log_groups_per_flex; /* FLEX_BG group size */
+ __u8 s_reserved_char_pad2;
+ __le16 s_reserved_pad;
+ __u32 s_reserved[162]; /* Padding to the end of the block */
};
#ifdef __KERNEL__
@@ -694,6 +741,9 @@
#define DX_HASH_LEGACY 0
#define DX_HASH_HALF_MD4 1
#define DX_HASH_TEA 2
+#define DX_HASH_LEGACY_UNSIGNED 3
+#define DX_HASH_HALF_MD4_UNSIGNED 4
+#define DX_HASH_TEA_UNSIGNED 5
#ifdef __KERNEL__
diff --git a/include/linux/ext3_fs_sb.h b/include/linux/ext3_fs_sb.h
index e024e38..f07f34d 100644
--- a/include/linux/ext3_fs_sb.h
+++ b/include/linux/ext3_fs_sb.h
@@ -57,10 +57,11 @@
u32 s_next_generation;
u32 s_hash_seed[4];
int s_def_hash_version;
+ int s_hash_unsigned; /* 3 if hash should be signed, 0 if not */
struct percpu_counter s_freeblocks_counter;
struct percpu_counter s_freeinodes_counter;
struct percpu_counter s_dirs_counter;
- struct blockgroup_lock s_blockgroup_lock;
+ struct blockgroup_lock *s_blockgroup_lock;
/* root of the per fs reservation window tree */
spinlock_t s_rsv_window_lock;
@@ -86,7 +87,7 @@
static inline spinlock_t *
sb_bgl_lock(struct ext3_sb_info *sbi, unsigned int block_group)
{
- return bgl_lock_ptr(&sbi->s_blockgroup_lock, block_group);
+ return bgl_lock_ptr(sbi->s_blockgroup_lock, block_group);
}
#endif /* _LINUX_EXT3_FS_SB */
diff --git a/include/linux/fs.h b/include/linux/fs.h
index e38a64d..0b87b29 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -565,6 +565,7 @@
struct block_device {
dev_t bd_dev; /* not a kdev_t - it's a search key */
struct inode * bd_inode; /* will die */
+ struct super_block * bd_super;
int bd_openers;
struct mutex bd_mutex; /* open/close mutex */
struct semaphore bd_mount_sem;
@@ -1389,6 +1390,7 @@
ssize_t (*quota_read)(struct super_block *, int, char *, size_t, loff_t);
ssize_t (*quota_write)(struct super_block *, int, const char *, size_t, loff_t);
#endif
+ int (*bdev_try_to_free_page)(struct super_block*, struct page*, gfp_t);
};
/*
diff --git a/include/linux/if_vlan.h b/include/linux/if_vlan.h
index a5cb0c3..f8ff918 100644
--- a/include/linux/if_vlan.h
+++ b/include/linux/if_vlan.h
@@ -115,6 +115,11 @@
extern int __vlan_hwaccel_rx(struct sk_buff *skb, struct vlan_group *grp,
u16 vlan_tci, int polling);
extern int vlan_hwaccel_do_receive(struct sk_buff *skb);
+extern int vlan_gro_receive(struct napi_struct *napi, struct vlan_group *grp,
+ unsigned int vlan_tci, struct sk_buff *skb);
+extern int vlan_gro_frags(struct napi_struct *napi, struct vlan_group *grp,
+ unsigned int vlan_tci,
+ struct napi_gro_fraginfo *info);
#else
static inline struct net_device *vlan_dev_real_dev(const struct net_device *dev)
@@ -140,6 +145,20 @@
{
return 0;
}
+
+static inline int vlan_gro_receive(struct napi_struct *napi,
+ struct vlan_group *grp,
+ unsigned int vlan_tci, struct sk_buff *skb)
+{
+ return NET_RX_DROP;
+}
+
+static inline int vlan_gro_frags(struct napi_struct *napi,
+ struct vlan_group *grp, unsigned int vlan_tci,
+ struct napi_gro_fraginfo *info)
+{
+ return NET_RX_DROP;
+}
#endif
/**
diff --git a/include/linux/ioprio.h b/include/linux/ioprio.h
index f98a656..76dad48 100644
--- a/include/linux/ioprio.h
+++ b/include/linux/ioprio.h
@@ -86,4 +86,6 @@
*/
extern int ioprio_best(unsigned short aprio, unsigned short bprio);
+extern int set_task_ioprio(struct task_struct *task, int ioprio);
+
#endif
diff --git a/include/linux/jbd.h b/include/linux/jbd.h
index 346e2b8..6384b19 100644
--- a/include/linux/jbd.h
+++ b/include/linux/jbd.h
@@ -543,6 +543,11 @@
unsigned long t_expires;
/*
+ * When this transaction started, in nanoseconds [no locking]
+ */
+ ktime_t t_start_time;
+
+ /*
* How many handles used this transaction? [t_handle_lock]
*/
int t_handle_count;
@@ -798,9 +803,19 @@
struct buffer_head **j_wbuf;
int j_wbufsize;
+ /*
+ * this is the pid of the last person to run a synchronous operation
+ * through the journal.
+ */
pid_t j_last_sync_writer;
/*
+ * the average amount of time in nanoseconds it takes to commit a
+ * transaction to the disk. [j_state_lock]
+ */
+ u64 j_average_commit_time;
+
+ /*
* An opaque pointer to fs-private information. ext3 puts its
* superblock pointer here
*/
diff --git a/include/linux/jbd2.h b/include/linux/jbd2.h
index 3445647..b45109c 100644
--- a/include/linux/jbd2.h
+++ b/include/linux/jbd2.h
@@ -638,6 +638,11 @@
unsigned long t_expires;
/*
+ * When this transaction started, in nanoseconds [no locking]
+ */
+ ktime_t t_start_time;
+
+ /*
* How many handles used this transaction? [t_handle_lock]
*/
int t_handle_count;
@@ -682,6 +687,8 @@
return end + (MAX_JIFFY_OFFSET - start);
}
+#define JBD2_NR_BATCH 64
+
/**
* struct journal_s - The journal_s type is the concrete type associated with
* journal_t.
@@ -826,6 +833,14 @@
struct mutex j_checkpoint_mutex;
/*
+ * List of buffer heads used by the checkpoint routine. This
+ * was moved from jbd2_log_do_checkpoint() to reduce stack
+ * usage. Access to this array is controlled by the
+ * j_checkpoint_mutex. [j_checkpoint_mutex]
+ */
+ struct buffer_head *j_chkpt_bhs[JBD2_NR_BATCH];
+
+ /*
* Journal head: identifies the first unused block in the journal.
* [j_state_lock]
*/
@@ -939,8 +954,26 @@
struct buffer_head **j_wbuf;
int j_wbufsize;
+ /*
+ * this is the pid of hte last person to run a synchronous operation
+ * through the journal
+ */
pid_t j_last_sync_writer;
+ /*
+ * the average amount of time in nanoseconds it takes to commit a
+ * transaction to disk. [j_state_lock]
+ */
+ u64 j_average_commit_time;
+
+ /*
+ * minimum and maximum times that we should wait for
+ * additional filesystem operations to get batched into a
+ * synchronous handle in microseconds
+ */
+ u32 j_min_batch_time;
+ u32 j_max_batch_time;
+
/* This function is called when a transaction is closed */
void (*j_commit_callback)(journal_t *,
transaction_t *);
@@ -1102,7 +1135,6 @@
(journal_t *, unsigned long, unsigned long, unsigned long);
extern void jbd2_journal_clear_features
(journal_t *, unsigned long, unsigned long, unsigned long);
-extern int jbd2_journal_create (journal_t *);
extern int jbd2_journal_load (journal_t *journal);
extern int jbd2_journal_destroy (journal_t *);
extern int jbd2_journal_recover (journal_t *journal);
@@ -1177,8 +1209,8 @@
int jbd2_log_do_checkpoint(journal_t *journal);
void __jbd2_log_wait_for_space(journal_t *journal);
-extern void __jbd2_journal_drop_transaction(journal_t *, transaction_t *);
-extern int jbd2_cleanup_journal_tail(journal_t *);
+extern void __jbd2_journal_drop_transaction(journal_t *, transaction_t *);
+extern int jbd2_cleanup_journal_tail(journal_t *);
/* Debugging code only: */
diff --git a/include/linux/kernel.h b/include/linux/kernel.h
index 6b8e202..343df9e 100644
--- a/include/linux/kernel.h
+++ b/include/linux/kernel.h
@@ -476,6 +476,12 @@
__val = __val < __min ? __min: __val; \
__val > __max ? __max: __val; })
+
+/*
+ * swap - swap value of @a and @b
+ */
+#define swap(a, b) ({ typeof(a) __tmp = (a); (a) = (b); (b) = __tmp; })
+
/**
* container_of - cast a member of a structure out to the containing structure
* @ptr: the pointer to the member.
diff --git a/include/linux/libata.h b/include/linux/libata.h
index 3449de5..4f7c8fb 100644
--- a/include/linux/libata.h
+++ b/include/linux/libata.h
@@ -1518,6 +1518,7 @@
extern const struct ata_port_operations ata_sff_port_ops;
extern const struct ata_port_operations ata_bmdma_port_ops;
+extern const struct ata_port_operations ata_bmdma32_port_ops;
/* PIO only, sg_tablesize and dma_boundary limits can be removed */
#define ATA_PIO_SHT(drv_name) \
@@ -1545,6 +1546,8 @@
const struct ata_taskfile *tf);
extern unsigned int ata_sff_data_xfer(struct ata_device *dev,
unsigned char *buf, unsigned int buflen, int rw);
+extern unsigned int ata_sff_data_xfer32(struct ata_device *dev,
+ unsigned char *buf, unsigned int buflen, int rw);
extern unsigned int ata_sff_data_xfer_noirq(struct ata_device *dev,
unsigned char *buf, unsigned int buflen, int rw);
extern u8 ata_sff_irq_on(struct ata_port *ap);
diff --git a/include/linux/magic.h b/include/linux/magic.h
index f7f3fdd..439f6f3 100644
--- a/include/linux/magic.h
+++ b/include/linux/magic.h
@@ -13,6 +13,7 @@
#define EFS_SUPER_MAGIC 0x414A53
#define EXT2_SUPER_MAGIC 0xEF53
#define EXT3_SUPER_MAGIC 0xEF53
+#define XENFS_SUPER_MAGIC 0xabba1974
#define EXT4_SUPER_MAGIC 0xEF53
#define HPFS_SUPER_MAGIC 0xf995e849
#define ISOFS_SUPER_MAGIC 0x9660
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 1fbe14d..326f45c 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -19,22 +19,45 @@
#ifndef _LINUX_MEMCONTROL_H
#define _LINUX_MEMCONTROL_H
-
+#include <linux/cgroup.h>
struct mem_cgroup;
struct page_cgroup;
struct page;
struct mm_struct;
#ifdef CONFIG_CGROUP_MEM_RES_CTLR
+/*
+ * All "charge" functions with gfp_mask should use GFP_KERNEL or
+ * (gfp_mask & GFP_RECLAIM_MASK). In current implementatin, memcg doesn't
+ * alloc memory but reclaims memory from all available zones. So, "where I want
+ * memory from" bits of gfp_mask has no meaning. So any bits of that field is
+ * available but adding a rule is better. charge functions' gfp_mask should
+ * be set to GFP_KERNEL or gfp_mask & GFP_RECLAIM_MASK for avoiding ambiguous
+ * codes.
+ * (Of course, if memcg does memory allocation in future, GFP_KERNEL is sane.)
+ */
-extern int mem_cgroup_charge(struct page *page, struct mm_struct *mm,
+extern int mem_cgroup_newpage_charge(struct page *page, struct mm_struct *mm,
gfp_t gfp_mask);
+/* for swap handling */
+extern int mem_cgroup_try_charge_swapin(struct mm_struct *mm,
+ struct page *page, gfp_t mask, struct mem_cgroup **ptr);
+extern void mem_cgroup_commit_charge_swapin(struct page *page,
+ struct mem_cgroup *ptr);
+extern void mem_cgroup_cancel_charge_swapin(struct mem_cgroup *ptr);
+
extern int mem_cgroup_cache_charge(struct page *page, struct mm_struct *mm,
gfp_t gfp_mask);
-extern void mem_cgroup_move_lists(struct page *page, enum lru_list lru);
+extern void mem_cgroup_add_lru_list(struct page *page, enum lru_list lru);
+extern void mem_cgroup_del_lru_list(struct page *page, enum lru_list lru);
+extern void mem_cgroup_rotate_lru_list(struct page *page, enum lru_list lru);
+extern void mem_cgroup_del_lru(struct page *page);
+extern void mem_cgroup_move_lists(struct page *page,
+ enum lru_list from, enum lru_list to);
extern void mem_cgroup_uncharge_page(struct page *page);
extern void mem_cgroup_uncharge_cache_page(struct page *page);
-extern int mem_cgroup_shrink_usage(struct mm_struct *mm, gfp_t gfp_mask);
+extern int mem_cgroup_shrink_usage(struct page *page,
+ struct mm_struct *mm, gfp_t gfp_mask);
extern unsigned long mem_cgroup_isolate_pages(unsigned long nr_to_scan,
struct list_head *dst,
@@ -47,12 +70,20 @@
extern struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p);
-#define mm_match_cgroup(mm, cgroup) \
- ((cgroup) == mem_cgroup_from_task((mm)->owner))
+static inline
+int mm_match_cgroup(const struct mm_struct *mm, const struct mem_cgroup *cgroup)
+{
+ struct mem_cgroup *mem;
+ rcu_read_lock();
+ mem = mem_cgroup_from_task((mm)->owner);
+ rcu_read_unlock();
+ return cgroup == mem;
+}
extern int
-mem_cgroup_prepare_migration(struct page *page, struct page *newpage);
-extern void mem_cgroup_end_migration(struct page *page);
+mem_cgroup_prepare_migration(struct page *page, struct mem_cgroup **ptr);
+extern void mem_cgroup_end_migration(struct mem_cgroup *mem,
+ struct page *oldpage, struct page *newpage);
/*
* For memory reclaim.
@@ -65,13 +96,32 @@
int priority);
extern void mem_cgroup_record_reclaim_priority(struct mem_cgroup *mem,
int priority);
+int mem_cgroup_inactive_anon_is_low(struct mem_cgroup *memcg);
+unsigned long mem_cgroup_zone_nr_pages(struct mem_cgroup *memcg,
+ struct zone *zone,
+ enum lru_list lru);
+struct zone_reclaim_stat *mem_cgroup_get_reclaim_stat(struct mem_cgroup *memcg,
+ struct zone *zone);
+struct zone_reclaim_stat*
+mem_cgroup_get_reclaim_stat_from_page(struct page *page);
-extern long mem_cgroup_calc_reclaim(struct mem_cgroup *mem, struct zone *zone,
- int priority, enum lru_list lru);
+#ifdef CONFIG_CGROUP_MEM_RES_CTLR_SWAP
+extern int do_swap_account;
+#endif
+static inline bool mem_cgroup_disabled(void)
+{
+ if (mem_cgroup_subsys.disabled)
+ return true;
+ return false;
+}
+
+extern bool mem_cgroup_oom_called(struct task_struct *task);
#else /* CONFIG_CGROUP_MEM_RES_CTLR */
-static inline int mem_cgroup_charge(struct page *page,
+struct mem_cgroup;
+
+static inline int mem_cgroup_newpage_charge(struct page *page,
struct mm_struct *mm, gfp_t gfp_mask)
{
return 0;
@@ -83,6 +133,21 @@
return 0;
}
+static inline int mem_cgroup_try_charge_swapin(struct mm_struct *mm,
+ struct page *page, gfp_t gfp_mask, struct mem_cgroup **ptr)
+{
+ return 0;
+}
+
+static inline void mem_cgroup_commit_charge_swapin(struct page *page,
+ struct mem_cgroup *ptr)
+{
+}
+
+static inline void mem_cgroup_cancel_charge_swapin(struct mem_cgroup *ptr)
+{
+}
+
static inline void mem_cgroup_uncharge_page(struct page *page)
{
}
@@ -91,12 +156,33 @@
{
}
-static inline int mem_cgroup_shrink_usage(struct mm_struct *mm, gfp_t gfp_mask)
+static inline int mem_cgroup_shrink_usage(struct page *page,
+ struct mm_struct *mm, gfp_t gfp_mask)
{
return 0;
}
-static inline void mem_cgroup_move_lists(struct page *page, bool active)
+static inline void mem_cgroup_add_lru_list(struct page *page, int lru)
+{
+}
+
+static inline void mem_cgroup_del_lru_list(struct page *page, int lru)
+{
+ return ;
+}
+
+static inline void mem_cgroup_rotate_lru_list(struct page *page, int lru)
+{
+ return ;
+}
+
+static inline void mem_cgroup_del_lru(struct page *page)
+{
+ return ;
+}
+
+static inline void
+mem_cgroup_move_lists(struct page *page, enum lru_list from, enum lru_list to)
{
}
@@ -112,12 +198,14 @@
}
static inline int
-mem_cgroup_prepare_migration(struct page *page, struct page *newpage)
+mem_cgroup_prepare_migration(struct page *page, struct mem_cgroup **ptr)
{
return 0;
}
-static inline void mem_cgroup_end_migration(struct page *page)
+static inline void mem_cgroup_end_migration(struct mem_cgroup *mem,
+ struct page *oldpage,
+ struct page *newpage)
{
}
@@ -146,12 +234,42 @@
{
}
-static inline long mem_cgroup_calc_reclaim(struct mem_cgroup *mem,
- struct zone *zone, int priority,
- enum lru_list lru)
+static inline bool mem_cgroup_disabled(void)
+{
+ return true;
+}
+
+static inline bool mem_cgroup_oom_called(struct task_struct *task)
+{
+ return false;
+}
+
+static inline int
+mem_cgroup_inactive_anon_is_low(struct mem_cgroup *memcg)
+{
+ return 1;
+}
+
+static inline unsigned long
+mem_cgroup_zone_nr_pages(struct mem_cgroup *memcg, struct zone *zone,
+ enum lru_list lru)
{
return 0;
}
+
+
+static inline struct zone_reclaim_stat*
+mem_cgroup_get_reclaim_stat(struct mem_cgroup *memcg, struct zone *zone)
+{
+ return NULL;
+}
+
+static inline struct zone_reclaim_stat*
+mem_cgroup_get_reclaim_stat_from_page(struct page *page)
+{
+ return NULL;
+}
+
#endif /* CONFIG_CGROUP_MEM_CONT */
#endif /* _LINUX_MEMCONTROL_H */
diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h
index c948350..7fbb972 100644
--- a/include/linux/mm_inline.h
+++ b/include/linux/mm_inline.h
@@ -28,6 +28,7 @@
{
list_add(&page->lru, &zone->lru[l].list);
__inc_zone_state(zone, NR_LRU_BASE + l);
+ mem_cgroup_add_lru_list(page, l);
}
static inline void
@@ -35,6 +36,7 @@
{
list_del(&page->lru);
__dec_zone_state(zone, NR_LRU_BASE + l);
+ mem_cgroup_del_lru_list(page, l);
}
static inline void
@@ -54,6 +56,7 @@
l += page_is_file_cache(page);
}
__dec_zone_state(zone, NR_LRU_BASE + l);
+ mem_cgroup_del_lru_list(page, l);
}
/**
@@ -78,23 +81,4 @@
return lru;
}
-/**
- * inactive_anon_is_low - check if anonymous pages need to be deactivated
- * @zone: zone to check
- *
- * Returns true if the zone does not have enough inactive anon pages,
- * meaning some active anon pages need to be deactivated.
- */
-static inline int inactive_anon_is_low(struct zone *zone)
-{
- unsigned long active, inactive;
-
- active = zone_page_state(zone, NR_ACTIVE_ANON);
- inactive = zone_page_state(zone, NR_INACTIVE_ANON);
-
- if (inactive * zone->inactive_ratio < active)
- return 1;
-
- return 0;
-}
#endif
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 35a7b5e..09c14e2 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -263,6 +263,19 @@
#error ZONES_SHIFT -- too many zones configured adjust calculation
#endif
+struct zone_reclaim_stat {
+ /*
+ * The pageout code in vmscan.c keeps track of how many of the
+ * mem/swap backed and file backed pages are refeferenced.
+ * The higher the rotated/scanned ratio, the more valuable
+ * that cache is.
+ *
+ * The anon LRU stats live in [0], file LRU stats in [1]
+ */
+ unsigned long recent_rotated[2];
+ unsigned long recent_scanned[2];
+};
+
struct zone {
/* Fields commonly accessed by the page allocator */
unsigned long pages_min, pages_low, pages_high;
@@ -315,16 +328,7 @@
unsigned long nr_scan;
} lru[NR_LRU_LISTS];
- /*
- * The pageout code in vmscan.c keeps track of how many of the
- * mem/swap backed and file backed pages are refeferenced.
- * The higher the rotated/scanned ratio, the more valuable
- * that cache is.
- *
- * The anon LRU stats live in [0], file LRU stats in [1]
- */
- unsigned long recent_rotated[2];
- unsigned long recent_scanned[2];
+ struct zone_reclaim_stat reclaim_stat;
unsigned long pages_scanned; /* since last reclaim */
unsigned long flags; /* zone flags, see below */
diff --git a/include/linux/mtd/cfi.h b/include/linux/mtd/cfi.h
index 00e2b57..88d3d8f 100644
--- a/include/linux/mtd/cfi.h
+++ b/include/linux/mtd/cfi.h
@@ -520,6 +520,7 @@
#define CFI_MFR_AMD 0x0001
#define CFI_MFR_ATMEL 0x001F
+#define CFI_MFR_SAMSUNG 0x00EC
#define CFI_MFR_ST 0x0020 /* STMicroelectronics */
void cfi_fixup(struct mtd_info *mtd, struct cfi_fixup* fixups);
diff --git a/include/linux/mtd/ftl.h b/include/linux/mtd/ftl.h
index 0be442f..0555f7a 100644
--- a/include/linux/mtd/ftl.h
+++ b/include/linux/mtd/ftl.h
@@ -32,25 +32,25 @@
#define _LINUX_FTL_H
typedef struct erase_unit_header_t {
- u_int8_t LinkTargetTuple[5];
- u_int8_t DataOrgTuple[10];
- u_int8_t NumTransferUnits;
- u_int32_t EraseCount;
- u_int16_t LogicalEUN;
- u_int8_t BlockSize;
- u_int8_t EraseUnitSize;
- u_int16_t FirstPhysicalEUN;
- u_int16_t NumEraseUnits;
- u_int32_t FormattedSize;
- u_int32_t FirstVMAddress;
- u_int16_t NumVMPages;
- u_int8_t Flags;
- u_int8_t Code;
- u_int32_t SerialNumber;
- u_int32_t AltEUHOffset;
- u_int32_t BAMOffset;
- u_int8_t Reserved[12];
- u_int8_t EndTuple[2];
+ uint8_t LinkTargetTuple[5];
+ uint8_t DataOrgTuple[10];
+ uint8_t NumTransferUnits;
+ uint32_t EraseCount;
+ uint16_t LogicalEUN;
+ uint8_t BlockSize;
+ uint8_t EraseUnitSize;
+ uint16_t FirstPhysicalEUN;
+ uint16_t NumEraseUnits;
+ uint32_t FormattedSize;
+ uint32_t FirstVMAddress;
+ uint16_t NumVMPages;
+ uint8_t Flags;
+ uint8_t Code;
+ uint32_t SerialNumber;
+ uint32_t AltEUHOffset;
+ uint32_t BAMOffset;
+ uint8_t Reserved[12];
+ uint8_t EndTuple[2];
} erase_unit_header_t;
/* Flags in erase_unit_header_t */
diff --git a/include/linux/mtd/map.h b/include/linux/mtd/map.h
index aa30244..b981b87 100644
--- a/include/linux/mtd/map.h
+++ b/include/linux/mtd/map.h
@@ -223,6 +223,7 @@
must leave it enabled. */
void (*set_vpp)(struct map_info *, int);
+ unsigned long pfow_base;
unsigned long map_priv_1;
unsigned long map_priv_2;
void *fldrv_priv;
diff --git a/include/linux/mtd/mtd.h b/include/linux/mtd/mtd.h
index 64433eb..3aa5d77 100644
--- a/include/linux/mtd/mtd.h
+++ b/include/linux/mtd/mtd.h
@@ -15,6 +15,8 @@
#include <linux/mtd/compatmac.h>
#include <mtd/mtd-abi.h>
+#include <asm/div64.h>
+
#define MTD_CHAR_MAJOR 90
#define MTD_BLOCK_MAJOR 31
#define MAX_MTD_DEVICES 32
@@ -25,20 +27,20 @@
#define MTD_ERASE_DONE 0x08
#define MTD_ERASE_FAILED 0x10
-#define MTD_FAIL_ADDR_UNKNOWN 0xffffffff
+#define MTD_FAIL_ADDR_UNKNOWN -1LL
/* If the erase fails, fail_addr might indicate exactly which block failed. If
fail_addr = MTD_FAIL_ADDR_UNKNOWN, the failure was not at the device level or was not
specific to any particular block. */
struct erase_info {
struct mtd_info *mtd;
- u_int32_t addr;
- u_int32_t len;
- u_int32_t fail_addr;
+ uint64_t addr;
+ uint64_t len;
+ uint64_t fail_addr;
u_long time;
u_long retries;
- u_int dev;
- u_int cell;
+ unsigned dev;
+ unsigned cell;
void (*callback) (struct erase_info *self);
u_long priv;
u_char state;
@@ -46,9 +48,9 @@
};
struct mtd_erase_region_info {
- u_int32_t offset; /* At which this region starts, from the beginning of the MTD */
- u_int32_t erasesize; /* For this region */
- u_int32_t numblocks; /* Number of blocks of erasesize in this region */
+ uint64_t offset; /* At which this region starts, from the beginning of the MTD */
+ uint32_t erasesize; /* For this region */
+ uint32_t numblocks; /* Number of blocks of erasesize in this region */
unsigned long *lockmap; /* If keeping bitmap of locks */
};
@@ -100,14 +102,14 @@
struct mtd_info {
u_char type;
- u_int32_t flags;
- u_int32_t size; // Total size of the MTD
+ uint32_t flags;
+ uint64_t size; // Total size of the MTD
/* "Major" erase size for the device. Naïve users may take this
* to be the only erase size available, or may use the more detailed
* information below if they desire
*/
- u_int32_t erasesize;
+ uint32_t erasesize;
/* Minimal writable flash unit size. In case of NOR flash it is 1 (even
* though individual bits can be cleared), in case of NAND flash it is
* one NAND page (or half, or one-fourths of it), in case of ECC-ed NOR
@@ -115,10 +117,20 @@
* Any driver registering a struct mtd_info must ensure a writesize of
* 1 or larger.
*/
- u_int32_t writesize;
+ uint32_t writesize;
- u_int32_t oobsize; // Amount of OOB data per block (e.g. 16)
- u_int32_t oobavail; // Available OOB bytes per block
+ uint32_t oobsize; // Amount of OOB data per block (e.g. 16)
+ uint32_t oobavail; // Available OOB bytes per block
+
+ /*
+ * If erasesize is a power of 2 then the shift is stored in
+ * erasesize_shift otherwise erasesize_shift is zero. Ditto writesize.
+ */
+ unsigned int erasesize_shift;
+ unsigned int writesize_shift;
+ /* Masks based on erasesize_shift and writesize_shift */
+ unsigned int erasesize_mask;
+ unsigned int writesize_mask;
// Kernel-only stuff starts here.
const char *name;
@@ -190,8 +202,8 @@
void (*sync) (struct mtd_info *mtd);
/* Chip-supported device locking */
- int (*lock) (struct mtd_info *mtd, loff_t ofs, size_t len);
- int (*unlock) (struct mtd_info *mtd, loff_t ofs, size_t len);
+ int (*lock) (struct mtd_info *mtd, loff_t ofs, uint64_t len);
+ int (*unlock) (struct mtd_info *mtd, loff_t ofs, uint64_t len);
/* Power Management functions */
int (*suspend) (struct mtd_info *mtd);
@@ -221,6 +233,35 @@
void (*put_device) (struct mtd_info *mtd);
};
+static inline uint32_t mtd_div_by_eb(uint64_t sz, struct mtd_info *mtd)
+{
+ if (mtd->erasesize_shift)
+ return sz >> mtd->erasesize_shift;
+ do_div(sz, mtd->erasesize);
+ return sz;
+}
+
+static inline uint32_t mtd_mod_by_eb(uint64_t sz, struct mtd_info *mtd)
+{
+ if (mtd->erasesize_shift)
+ return sz & mtd->erasesize_mask;
+ return do_div(sz, mtd->erasesize);
+}
+
+static inline uint32_t mtd_div_by_ws(uint64_t sz, struct mtd_info *mtd)
+{
+ if (mtd->writesize_shift)
+ return sz >> mtd->writesize_shift;
+ do_div(sz, mtd->writesize);
+ return sz;
+}
+
+static inline uint32_t mtd_mod_by_ws(uint64_t sz, struct mtd_info *mtd)
+{
+ if (mtd->writesize_shift)
+ return sz & mtd->writesize_mask;
+ return do_div(sz, mtd->writesize);
+}
/* Kernel-side ioctl definitions */
diff --git a/include/linux/mtd/nand.h b/include/linux/mtd/nand.h
index 733d3f3..db5b63d 100644
--- a/include/linux/mtd/nand.h
+++ b/include/linux/mtd/nand.h
@@ -335,17 +335,12 @@
* @erase_cmd: [INTERN] erase command write function, selectable due to AND support
* @scan_bbt: [REPLACEABLE] function to scan bad block table
* @chip_delay: [BOARDSPECIFIC] chip dependent delay for transfering data from array to read regs (tR)
- * @wq: [INTERN] wait queue to sleep on if a NAND operation is in progress
* @state: [INTERN] the current state of the NAND device
* @oob_poi: poison value buffer
* @page_shift: [INTERN] number of address bits in a page (column address bits)
* @phys_erase_shift: [INTERN] number of address bits in a physical eraseblock
* @bbt_erase_shift: [INTERN] number of address bits in a bbt entry
* @chip_shift: [INTERN] number of address bits in one chip
- * @datbuf: [INTERN] internal buffer for one page + oob
- * @oobbuf: [INTERN] oob buffer for one eraseblock
- * @oobdirty: [INTERN] indicates that oob_buf must be reinitialized
- * @data_poi: [INTERN] pointer to a data buffer
* @options: [BOARDSPECIFIC] various chip options. They can partly be set to inform nand_scan about
* special functionality. See the defines for further explanation
* @badblockpos: [INTERN] position of the bad block marker in the oob area
@@ -399,7 +394,7 @@
int bbt_erase_shift;
int chip_shift;
int numchips;
- unsigned long chipsize;
+ uint64_t chipsize;
int pagemask;
int pagebuf;
int subpagesize;
diff --git a/include/linux/mtd/partitions.h b/include/linux/mtd/partitions.h
index c92b4d4..a45dd83 100644
--- a/include/linux/mtd/partitions.h
+++ b/include/linux/mtd/partitions.h
@@ -36,9 +36,9 @@
struct mtd_partition {
char *name; /* identifier string */
- u_int32_t size; /* partition size */
- u_int32_t offset; /* offset within the master MTD space */
- u_int32_t mask_flags; /* master MTD flags to mask out for this partition */
+ uint64_t size; /* partition size */
+ uint64_t offset; /* offset within the master MTD space */
+ uint32_t mask_flags; /* master MTD flags to mask out for this partition */
struct nand_ecclayout *ecclayout; /* out of band layout for this partition (NAND only)*/
struct mtd_info **mtdp; /* pointer to store the MTD object */
};
diff --git a/include/linux/mtd/pfow.h b/include/linux/mtd/pfow.h
new file mode 100644
index 0000000..b730d4f
--- /dev/null
+++ b/include/linux/mtd/pfow.h
@@ -0,0 +1,159 @@
+/* Primary function overlay window definitions
+ * and service functions used by LPDDR chips
+ */
+#ifndef __LINUX_MTD_PFOW_H
+#define __LINUX_MTD_PFOW_H
+
+#include <linux/mtd/qinfo.h>
+
+/* PFOW registers addressing */
+/* Address of symbol "P" */
+#define PFOW_QUERY_STRING_P 0x0000
+/* Address of symbol "F" */
+#define PFOW_QUERY_STRING_F 0x0002
+/* Address of symbol "O" */
+#define PFOW_QUERY_STRING_O 0x0004
+/* Address of symbol "W" */
+#define PFOW_QUERY_STRING_W 0x0006
+/* Identification info for LPDDR chip */
+#define PFOW_MANUFACTURER_ID 0x0020
+#define PFOW_DEVICE_ID 0x0022
+/* Address in PFOW where prog buffer can can be found */
+#define PFOW_PROGRAM_BUFFER_OFFSET 0x0040
+/* Size of program buffer in words */
+#define PFOW_PROGRAM_BUFFER_SIZE 0x0042
+/* Address command code register */
+#define PFOW_COMMAND_CODE 0x0080
+/* command data register */
+#define PFOW_COMMAND_DATA 0x0084
+/* command address register lower address bits */
+#define PFOW_COMMAND_ADDRESS_L 0x0088
+/* command address register upper address bits */
+#define PFOW_COMMAND_ADDRESS_H 0x008a
+/* number of bytes to be proggrammed lower address bits */
+#define PFOW_DATA_COUNT_L 0x0090
+/* number of bytes to be proggrammed higher address bits */
+#define PFOW_DATA_COUNT_H 0x0092
+/* command execution register, the only possible value is 0x01 */
+#define PFOW_COMMAND_EXECUTE 0x00c0
+/* 0x01 should be written at this address to clear buffer */
+#define PFOW_CLEAR_PROGRAM_BUFFER 0x00c4
+/* device program/erase suspend register */
+#define PFOW_PROGRAM_ERASE_SUSPEND 0x00c8
+/* device status register */
+#define PFOW_DSR 0x00cc
+
+/* LPDDR memory device command codes */
+/* They are possible values of PFOW command code register */
+#define LPDDR_WORD_PROGRAM 0x0041
+#define LPDDR_BUFF_PROGRAM 0x00E9
+#define LPDDR_BLOCK_ERASE 0x0020
+#define LPDDR_LOCK_BLOCK 0x0061
+#define LPDDR_UNLOCK_BLOCK 0x0062
+#define LPDDR_READ_BLOCK_LOCK_STATUS 0x0065
+#define LPDDR_INFO_QUERY 0x0098
+#define LPDDR_READ_OTP 0x0097
+#define LPDDR_PROG_OTP 0x00C0
+#define LPDDR_RESUME 0x00D0
+
+/* Defines possible value of PFOW command execution register */
+#define LPDDR_START_EXECUTION 0x0001
+
+/* Defines possible value of PFOW program/erase suspend register */
+#define LPDDR_SUSPEND 0x0001
+
+/* Possible values of PFOW device status register */
+/* access R - read; RC read & clearable */
+#define DSR_DPS (1<<1) /* RC; device protect status
+ * 0 - not protected 1 - locked */
+#define DSR_PSS (1<<2) /* R; program suspend status;
+ * 0-prog in progress/completed,
+ * 1- prog suspended */
+#define DSR_VPPS (1<<3) /* RC; 0-Vpp OK, * 1-Vpp low */
+#define DSR_PROGRAM_STATUS (1<<4) /* RC; 0-successful, 1-error */
+#define DSR_ERASE_STATUS (1<<5) /* RC; erase or blank check status;
+ * 0-success erase/blank check,
+ * 1 blank check error */
+#define DSR_ESS (1<<6) /* R; erase suspend status;
+ * 0-erase in progress/complete,
+ * 1 erase suspended */
+#define DSR_READY_STATUS (1<<7) /* R; Device status
+ * 0-busy,
+ * 1-ready */
+#define DSR_RPS (0x3<<8) /* RC; region program status
+ * 00 - Success,
+ * 01-re-program attempt in region with
+ * object mode data,
+ * 10-object mode program w attempt in
+ * region with control mode data
+ * 11-attempt to program invalid half
+ * with 0x41 command */
+#define DSR_AOS (1<<12) /* RC; 1- AO related failure */
+#define DSR_AVAILABLE (1<<15) /* R; Device availbility
+ * 1 - Device available
+ * 0 - not available */
+
+/* The superset of all possible error bits in DSR */
+#define DSR_ERR 0x133A
+
+static inline void send_pfow_command(struct map_info *map,
+ unsigned long cmd_code, unsigned long adr,
+ unsigned long len, map_word *datum)
+{
+ int bits_per_chip = map_bankwidth(map) * 8;
+ int chipnum;
+ struct lpddr_private *lpddr = map->fldrv_priv;
+ chipnum = adr >> lpddr->chipshift;
+
+ map_write(map, CMD(cmd_code), map->pfow_base + PFOW_COMMAND_CODE);
+ map_write(map, CMD(adr & ((1<<bits_per_chip) - 1)),
+ map->pfow_base + PFOW_COMMAND_ADDRESS_L);
+ map_write(map, CMD(adr>>bits_per_chip),
+ map->pfow_base + PFOW_COMMAND_ADDRESS_H);
+ if (len) {
+ map_write(map, CMD(len & ((1<<bits_per_chip) - 1)),
+ map->pfow_base + PFOW_DATA_COUNT_L);
+ map_write(map, CMD(len>>bits_per_chip),
+ map->pfow_base + PFOW_DATA_COUNT_H);
+ }
+ if (datum)
+ map_write(map, *datum, map->pfow_base + PFOW_COMMAND_DATA);
+
+ /* Command execution start */
+ map_write(map, CMD(LPDDR_START_EXECUTION),
+ map->pfow_base + PFOW_COMMAND_EXECUTE);
+}
+
+static inline void print_drs_error(unsigned dsr)
+{
+ int prog_status = (dsr & DSR_RPS) >> 8;
+
+ if (!(dsr & DSR_AVAILABLE))
+ printk(KERN_NOTICE"DSR.15: (0) Device not Available\n");
+ if (prog_status & 0x03)
+ printk(KERN_NOTICE"DSR.9,8: (11) Attempt to program invalid "
+ "half with 41h command\n");
+ else if (prog_status & 0x02)
+ printk(KERN_NOTICE"DSR.9,8: (10) Object Mode Program attempt "
+ "in region with Control Mode data\n");
+ else if (prog_status & 0x01)
+ printk(KERN_NOTICE"DSR.9,8: (01) Program attempt in region "
+ "with Object Mode data\n");
+ if (!(dsr & DSR_READY_STATUS))
+ printk(KERN_NOTICE"DSR.7: (0) Device is Busy\n");
+ if (dsr & DSR_ESS)
+ printk(KERN_NOTICE"DSR.6: (1) Erase Suspended\n");
+ if (dsr & DSR_ERASE_STATUS)
+ printk(KERN_NOTICE"DSR.5: (1) Erase/Blank check error\n");
+ if (dsr & DSR_PROGRAM_STATUS)
+ printk(KERN_NOTICE"DSR.4: (1) Program Error\n");
+ if (dsr & DSR_VPPS)
+ printk(KERN_NOTICE"DSR.3: (1) Vpp low detect, operation "
+ "aborted\n");
+ if (dsr & DSR_PSS)
+ printk(KERN_NOTICE"DSR.2: (1) Program suspended\n");
+ if (dsr & DSR_DPS)
+ printk(KERN_NOTICE"DSR.1: (1) Aborted Erase/Program attempt "
+ "on locked block\n");
+}
+#endif /* __LINUX_MTD_PFOW_H */
diff --git a/include/linux/mtd/physmap.h b/include/linux/mtd/physmap.h
index c8e63a5..76f7cab 100644
--- a/include/linux/mtd/physmap.h
+++ b/include/linux/mtd/physmap.h
@@ -24,6 +24,7 @@
unsigned int width;
void (*set_vpp)(struct map_info *, int);
unsigned int nr_parts;
+ unsigned int pfow_base;
struct mtd_partition *parts;
};
diff --git a/include/linux/mtd/qinfo.h b/include/linux/mtd/qinfo.h
new file mode 100644
index 0000000..7b3d487
--- /dev/null
+++ b/include/linux/mtd/qinfo.h
@@ -0,0 +1,91 @@
+#ifndef __LINUX_MTD_QINFO_H
+#define __LINUX_MTD_QINFO_H
+
+#include <linux/mtd/map.h>
+#include <linux/wait.h>
+#include <linux/spinlock.h>
+#include <linux/delay.h>
+#include <linux/mtd/mtd.h>
+#include <linux/mtd/flashchip.h>
+#include <linux/mtd/partitions.h>
+
+/* lpddr_private describes lpddr flash chip in memory map
+ * @ManufactId - Chip Manufacture ID
+ * @DevId - Chip Device ID
+ * @qinfo - pointer to qinfo records describing the chip
+ * @numchips - number of chips including virual RWW partitions
+ * @chipshift - Chip/partiton size 2^chipshift
+ * @chips - per-chip data structure
+ */
+struct lpddr_private {
+ uint16_t ManufactId;
+ uint16_t DevId;
+ struct qinfo_chip *qinfo;
+ int numchips;
+ unsigned long chipshift;
+ struct flchip chips[0];
+};
+
+/* qinfo_query_info structure contains request information for
+ * each qinfo record
+ * @major - major number of qinfo record
+ * @major - minor number of qinfo record
+ * @id_str - descriptive string to access the record
+ * @desc - detailed description for the qinfo record
+ */
+struct qinfo_query_info {
+ uint8_t major;
+ uint8_t minor;
+ char *id_str;
+ char *desc;
+};
+
+/*
+ * qinfo_chip structure contains necessary qinfo records data
+ * @DevSizeShift - Device size 2^n bytes
+ * @BufSizeShift - Program buffer size 2^n bytes
+ * @TotalBlocksNum - Total number of blocks
+ * @UniformBlockSizeShift - Uniform block size 2^UniformBlockSizeShift bytes
+ * @HWPartsNum - Number of hardware partitions
+ * @SuspEraseSupp - Suspend erase supported
+ * @SingleWordProgTime - Single word program 2^SingleWordProgTime u-sec
+ * @ProgBufferTime - Program buffer write 2^ProgBufferTime u-sec
+ * @BlockEraseTime - Block erase 2^BlockEraseTime m-sec
+ */
+struct qinfo_chip {
+ /* General device info */
+ uint16_t DevSizeShift;
+ uint16_t BufSizeShift;
+ /* Erase block information */
+ uint16_t TotalBlocksNum;
+ uint16_t UniformBlockSizeShift;
+ /* Partition information */
+ uint16_t HWPartsNum;
+ /* Optional features */
+ uint16_t SuspEraseSupp;
+ /* Operation typical time */
+ uint16_t SingleWordProgTime;
+ uint16_t ProgBufferTime;
+ uint16_t BlockEraseTime;
+};
+
+/* defines for fixup usage */
+#define LPDDR_MFR_ANY 0xffff
+#define LPDDR_ID_ANY 0xffff
+#define NUMONYX_MFGR_ID 0x0089
+#define R18_DEVICE_ID_1G 0x893c
+
+static inline map_word lpddr_build_cmd(u_long cmd, struct map_info *map)
+{
+ map_word val = { {0} };
+ val.x[0] = cmd;
+ return val;
+}
+
+#define CMD(x) lpddr_build_cmd(x, map)
+#define CMDVAL(cmd) cmd.x[0]
+
+struct mtd_info *lpddr_cmdset(struct map_info *);
+
+#endif
+
diff --git a/include/linux/mtd/sharpsl.h b/include/linux/mtd/sharpsl.h
new file mode 100644
index 0000000..25f4d2a
--- /dev/null
+++ b/include/linux/mtd/sharpsl.h
@@ -0,0 +1,20 @@
+/*
+ * SharpSL NAND support
+ *
+ * Copyright (C) 2008 Dmitry Baryshkov
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/mtd/nand.h>
+#include <linux/mtd/nand_ecc.h>
+#include <linux/mtd/partitions.h>
+
+struct sharpsl_nand_platform_data {
+ struct nand_bbt_descr *badblock_pattern;
+ struct nand_ecclayout *ecc_layout;
+ struct mtd_partition *partitions;
+ unsigned int nr_partitions;
+};
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index c28bbba..f245568 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -1125,9 +1125,6 @@
struct sk_buff *completion_queue;
struct napi_struct backlog;
-#ifdef CONFIG_NET_DMA
- struct dma_chan *net_dma;
-#endif
};
DECLARE_PER_CPU(struct softnet_data,softnet_data);
@@ -1373,8 +1370,14 @@
#define HAVE_NETIF_RECEIVE_SKB 1
extern int netif_receive_skb(struct sk_buff *skb);
extern void napi_gro_flush(struct napi_struct *napi);
+extern int dev_gro_receive(struct napi_struct *napi,
+ struct sk_buff *skb);
extern int napi_gro_receive(struct napi_struct *napi,
struct sk_buff *skb);
+extern void napi_reuse_skb(struct napi_struct *napi,
+ struct sk_buff *skb);
+extern struct sk_buff * napi_fraginfo_skb(struct napi_struct *napi,
+ struct napi_gro_fraginfo *info);
extern int napi_gro_frags(struct napi_struct *napi,
struct napi_gro_fraginfo *info);
extern void netif_nit_deliver(struct sk_buff *skb);
diff --git a/include/linux/nwpserial.h b/include/linux/nwpserial.h
new file mode 100644
index 0000000..9acb215
--- /dev/null
+++ b/include/linux/nwpserial.h
@@ -0,0 +1,18 @@
+/*
+ * Serial Port driver for a NWP uart device
+ *
+ * Copyright (C) 2008 IBM Corp., Benjamin Krill <ben@codiert.org>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ *
+ */
+#ifndef _NWPSERIAL_H
+#define _NWPSERIAL_H
+
+int nwpserial_register_port(struct uart_port *port);
+void nwpserial_unregister_port(int line);
+
+#endif /* _NWPSERIAL_H */
diff --git a/include/linux/oprofile.h b/include/linux/oprofile.h
index 1ce9fe5..1d9518b 100644
--- a/include/linux/oprofile.h
+++ b/include/linux/oprofile.h
@@ -164,4 +164,22 @@
unsigned long oprofile_get_cpu_buffer_size(void);
void oprofile_cpu_buffer_inc_smpl_lost(void);
+/* cpu buffer functions */
+
+struct op_sample;
+
+struct op_entry {
+ struct ring_buffer_event *event;
+ struct op_sample *sample;
+ unsigned long irq_flags;
+ unsigned long size;
+ unsigned long *data;
+};
+
+void oprofile_write_reserve(struct op_entry *entry,
+ struct pt_regs * const regs,
+ unsigned long pc, int code, int size);
+int oprofile_add_data(struct op_entry *entry, unsigned long val);
+int oprofile_write_commit(struct op_entry *entry);
+
#endif /* OPROFILE_H */
diff --git a/include/linux/page_cgroup.h b/include/linux/page_cgroup.h
index 1e6d34b..602cc1f 100644
--- a/include/linux/page_cgroup.h
+++ b/include/linux/page_cgroup.h
@@ -26,10 +26,6 @@
PCG_LOCK, /* page cgroup is locked */
PCG_CACHE, /* charged as cache */
PCG_USED, /* this object is in use. */
- /* flags for LRU placement */
- PCG_ACTIVE, /* page is active in this cgroup */
- PCG_FILE, /* page is file system backed */
- PCG_UNEVICTABLE, /* page is unevictableable */
};
#define TESTPCGFLAG(uname, lname) \
@@ -50,19 +46,6 @@
TESTPCGFLAG(Used, USED)
CLEARPCGFLAG(Used, USED)
-/* LRU management flags (from global-lru definition) */
-TESTPCGFLAG(File, FILE)
-SETPCGFLAG(File, FILE)
-CLEARPCGFLAG(File, FILE)
-
-TESTPCGFLAG(Active, ACTIVE)
-SETPCGFLAG(Active, ACTIVE)
-CLEARPCGFLAG(Active, ACTIVE)
-
-TESTPCGFLAG(Unevictable, UNEVICTABLE)
-SETPCGFLAG(Unevictable, UNEVICTABLE)
-CLEARPCGFLAG(Unevictable, UNEVICTABLE)
-
static inline int page_cgroup_nid(struct page_cgroup *pc)
{
return page_to_nid(pc->page);
@@ -105,4 +88,39 @@
}
#endif
+
+#ifdef CONFIG_CGROUP_MEM_RES_CTLR_SWAP
+#include <linux/swap.h>
+extern struct mem_cgroup *
+swap_cgroup_record(swp_entry_t ent, struct mem_cgroup *mem);
+extern struct mem_cgroup *lookup_swap_cgroup(swp_entry_t ent);
+extern int swap_cgroup_swapon(int type, unsigned long max_pages);
+extern void swap_cgroup_swapoff(int type);
+#else
+#include <linux/swap.h>
+
+static inline
+struct mem_cgroup *swap_cgroup_record(swp_entry_t ent, struct mem_cgroup *mem)
+{
+ return NULL;
+}
+
+static inline
+struct mem_cgroup *lookup_swap_cgroup(swp_entry_t ent)
+{
+ return NULL;
+}
+
+static inline int
+swap_cgroup_swapon(int type, unsigned long max_pages)
+{
+ return 0;
+}
+
+static inline void swap_cgroup_swapoff(int type)
+{
+ return;
+}
+
+#endif
#endif
diff --git a/include/linux/pci_hotplug.h b/include/linux/pci_hotplug.h
index f7cc204..2099874 100644
--- a/include/linux/pci_hotplug.h
+++ b/include/linux/pci_hotplug.h
@@ -223,7 +223,6 @@
#ifdef CONFIG_ACPI
#include <acpi/acpi.h>
#include <acpi/acpi_bus.h>
-#include <acpi/actypes.h>
extern acpi_status acpi_get_hp_params_from_firmware(struct pci_bus *bus,
struct hotplug_params *hpp);
int acpi_get_hp_hw_control_from_firmware(struct pci_dev *dev, u32 flags);
diff --git a/include/linux/pid.h b/include/linux/pid.h
index bb206c5..49f1c2f 100644
--- a/include/linux/pid.h
+++ b/include/linux/pid.h
@@ -123,6 +123,24 @@
extern void free_pid(struct pid *pid);
/*
+ * ns_of_pid() returns the pid namespace in which the specified pid was
+ * allocated.
+ *
+ * NOTE:
+ * ns_of_pid() is expected to be called for a process (task) that has
+ * an attached 'struct pid' (see attach_pid(), detach_pid()) i.e @pid
+ * is expected to be non-NULL. If @pid is NULL, caller should handle
+ * the resulting NULL pid-ns.
+ */
+static inline struct pid_namespace *ns_of_pid(struct pid *pid)
+{
+ struct pid_namespace *ns = NULL;
+ if (pid)
+ ns = pid->numbers[pid->level].ns;
+ return ns;
+}
+
+/*
* the helpers to get the pid's id seen from different namespaces
*
* pid_nr() : global id, i.e. the id seen from the init namespace;
diff --git a/include/linux/pid_namespace.h b/include/linux/pid_namespace.h
index d82fe82..38d1032 100644
--- a/include/linux/pid_namespace.h
+++ b/include/linux/pid_namespace.h
@@ -79,11 +79,7 @@
}
#endif /* CONFIG_PID_NS */
-static inline struct pid_namespace *task_active_pid_ns(struct task_struct *tsk)
-{
- return tsk->nsproxy->pid_ns;
-}
-
+extern struct pid_namespace *task_active_pid_ns(struct task_struct *tsk);
void pidhash_init(void);
void pidmap_init(void);
diff --git a/include/linux/raid/md_k.h b/include/linux/raid/md_k.h
index 8fc909e..9743e4d 100644
--- a/include/linux/raid/md_k.h
+++ b/include/linux/raid/md_k.h
@@ -137,6 +137,9 @@
struct gendisk *gendisk;
struct kobject kobj;
+ int hold_active;
+#define UNTIL_IOCTL 1
+#define UNTIL_STOP 2
/* Superblock information */
int major_version,
@@ -215,6 +218,9 @@
#define MD_RECOVERY_FROZEN 9
unsigned long recovery;
+ int recovery_disabled; /* if we detect that recovery
+ * will always fail, set this
+ * so we don't loop trying */
int in_sync; /* know to not need resync */
struct mutex reconfig_mutex;
@@ -244,6 +250,9 @@
struct sysfs_dirent *sysfs_state; /* handle for 'array_state'
* file in sysfs.
*/
+ struct sysfs_dirent *sysfs_action; /* handle for 'sync_action' */
+
+ struct work_struct del_work; /* used for delayed sysfs removal */
spinlock_t write_lock;
wait_queue_head_t sb_wait; /* for waiting on superblock updates */
@@ -334,17 +343,14 @@
* iterates through some rdev ringlist. It's safe to remove the
* current 'rdev'. Dont touch 'tmp' though.
*/
-#define rdev_for_each_list(rdev, tmp, list) \
- \
- for ((tmp) = (list).next; \
- (rdev) = (list_entry((tmp), mdk_rdev_t, same_set)), \
- (tmp) = (tmp)->next, (tmp)->prev != &(list) \
- ; )
+#define rdev_for_each_list(rdev, tmp, head) \
+ list_for_each_entry_safe(rdev, tmp, head, same_set)
+
/*
* iterates through the 'same array disks' ringlist
*/
#define rdev_for_each(rdev, tmp, mddev) \
- rdev_for_each_list(rdev, tmp, (mddev)->disks)
+ list_for_each_entry_safe(rdev, tmp, &((mddev)->disks), same_set)
#define rdev_for_each_rcu(rdev, mddev) \
list_for_each_entry_rcu(rdev, &((mddev)->disks), same_set)
diff --git a/include/linux/raid/md_p.h b/include/linux/raid/md_p.h
index 8b4de4a..9491026 100644
--- a/include/linux/raid/md_p.h
+++ b/include/linux/raid/md_p.h
@@ -194,6 +194,8 @@
return (ev<<32)| sb->events_lo;
}
+#define MD_SUPERBLOCK_1_TIME_SEC_MASK ((1ULL<<40) - 1)
+
/*
* The version-1 superblock :
* All numeric fields are little-endian.
diff --git a/include/linux/raid/raid0.h b/include/linux/raid/raid0.h
index 1b2dda0..fd42aa8 100644
--- a/include/linux/raid/raid0.h
+++ b/include/linux/raid/raid0.h
@@ -5,9 +5,9 @@
struct strip_zone
{
- sector_t zone_offset; /* Zone offset in md_dev */
- sector_t dev_offset; /* Zone offset in real dev */
- sector_t size; /* Zone size */
+ sector_t zone_start; /* Zone offset in md_dev (in sectors) */
+ sector_t dev_start; /* Zone offset in real dev (in sectors) */
+ sector_t sectors; /* Zone size in sectors */
int nb_dev; /* # of devices attached to the zone */
mdk_rdev_t **dev; /* Devices attached to the zone */
};
@@ -19,8 +19,8 @@
mdk_rdev_t **devlist; /* lists of rdevs, pointed to by strip_zone->dev */
int nr_strip_zones;
- sector_t hash_spacing;
- int preshift; /* shift this before divide by hash_spacing */
+ sector_t spacing;
+ int sector_shift; /* shift this before divide by spacing */
};
typedef struct raid0_private_data raid0_conf_t;
diff --git a/include/linux/regulator/consumer.h b/include/linux/regulator/consumer.h
index afdc455..801bf77 100644
--- a/include/linux/regulator/consumer.h
+++ b/include/linux/regulator/consumer.h
@@ -104,10 +104,10 @@
/**
* struct regulator_bulk_data - Data used for bulk regulator operations.
*
- * @supply The name of the supply. Initialised by the user before
- * using the bulk regulator APIs.
- * @consumer The regulator consumer for the supply. This will be managed
- * by the bulk API.
+ * @supply: The name of the supply. Initialised by the user before
+ * using the bulk regulator APIs.
+ * @consumer: The regulator consumer for the supply. This will be managed
+ * by the bulk API.
*
* The regulator APIs provide a series of regulator_bulk_() API calls as
* a convenience to consumers which require multiple supplies. This
diff --git a/include/linux/regulator/driver.h b/include/linux/regulator/driver.h
index e37d805..2dae057 100644
--- a/include/linux/regulator/driver.h
+++ b/include/linux/regulator/driver.h
@@ -24,7 +24,33 @@
/**
* struct regulator_ops - regulator operations.
*
- * This struct describes regulator operations.
+ * This struct describes regulator operations which can be implemented by
+ * regulator chip drivers.
+ *
+ * @enable: Enable the regulator.
+ * @disable: Disable the regulator.
+ * @is_enabled: Return 1 if the regulator is enabled, 0 otherwise.
+ *
+ * @set_voltage: Set the voltage for the regulator within the range specified.
+ * The driver should select the voltage closest to min_uV.
+ * @get_voltage: Return the currently configured voltage for the regulator.
+ *
+ * @set_current_limit: Configure a limit for a current-limited regulator.
+ * @get_current_limit: Get the limit for a current-limited regulator.
+ *
+ * @set_mode: Set the operating mode for the regulator.
+ * @get_mode: Get the current operating mode for the regulator.
+ * @get_optimum_mode: Get the most efficient operating mode for the regulator
+ * when running with the specified parameters.
+ *
+ * @set_suspend_voltage: Set the voltage for the regulator when the system
+ * is suspended.
+ * @set_suspend_enable: Mark the regulator as enabled when the system is
+ * suspended.
+ * @set_suspend_disable: Mark the regulator as disabled when the system is
+ * suspended.
+ * @set_suspend_mode: Set the operating mode for the regulator when the
+ * system is suspended.
*/
struct regulator_ops {
@@ -75,6 +101,15 @@
/**
* struct regulator_desc - Regulator descriptor
*
+ * Each regulator registered with the core is described with a structure of
+ * this type.
+ *
+ * @name: Identifying name for the regulator.
+ * @id: Numerical identifier for the regulator.
+ * @ops: Regulator operations table.
+ * @irq: Interrupt number for the regulator.
+ * @type: Indicates if the regulator is a voltage or current regulator.
+ * @owner: Module providing the regulator, used for refcounting.
*/
struct regulator_desc {
const char *name;
diff --git a/include/linux/regulator/machine.h b/include/linux/regulator/machine.h
index c6d6933..3794773 100644
--- a/include/linux/regulator/machine.h
+++ b/include/linux/regulator/machine.h
@@ -44,6 +44,10 @@
* struct regulator_state - regulator state during low power syatem states
*
* This describes a regulators state during a system wide low power state.
+ *
+ * @uV: Operating voltage during suspend.
+ * @mode: Operating mode during suspend.
+ * @enabled: Enabled during suspend.
*/
struct regulator_state {
int uV; /* suspend voltage */
@@ -55,6 +59,30 @@
* struct regulation_constraints - regulator operating constraints.
*
* This struct describes regulator and board/machine specific constraints.
+ *
+ * @name: Descriptive name for the constraints, used for display purposes.
+ *
+ * @min_uV: Smallest voltage consumers may set.
+ * @max_uV: Largest voltage consumers may set.
+ *
+ * @min_uA: Smallest consumers consumers may set.
+ * @max_uA: Largest current consumers may set.
+ *
+ * @valid_modes_mask: Mask of modes which may be configured by consumers.
+ * @valid_ops_mask: Operations which may be performed by consumers.
+ *
+ * @always_on: Set if the regulator should never be disabled.
+ * @boot_on: Set if the regulator is enabled when the system is initially
+ * started.
+ * @apply_uV: Apply the voltage constraint when initialising.
+ *
+ * @input_uV: Input voltage for regulator when supplied by another regulator.
+ *
+ * @state_disk: State for regulator when system is suspended in disk mode.
+ * @state_mem: State for regulator when system is suspended in mem mode.
+ * @state_standby: State for regulator when system is suspended in standby
+ * mode.
+ * @initial_state: Suspend state to set by default.
*/
struct regulation_constraints {
@@ -93,6 +121,9 @@
* struct regulator_consumer_supply - supply -> device mapping
*
* This maps a supply name to a device.
+ *
+ * @dev: Device structure for the consumer.
+ * @supply: Name for the supply.
*/
struct regulator_consumer_supply {
struct device *dev; /* consumer */
@@ -103,6 +134,16 @@
* struct regulator_init_data - regulator platform initialisation data.
*
* Initialisation constraints, our supply and consumers supplies.
+ *
+ * @supply_regulator_dev: Parent regulator (if any).
+ *
+ * @constraints: Constraints. These must be specified for the regulator to
+ * be usable.
+ * @num_consumer_supplies: Number of consumer device supplies.
+ * @consumer_supplies: Consumer device supply configuration.
+ *
+ * @regulator_init: Callback invoked when the regulator has been registered.
+ * @driver_data: Data passed to regulator_init.
*/
struct regulator_init_data {
struct device *supply_regulator_dev; /* or NULL for LINE */
diff --git a/include/linux/res_counter.h b/include/linux/res_counter.h
index 271c1c2..dede0a2 100644
--- a/include/linux/res_counter.h
+++ b/include/linux/res_counter.h
@@ -43,6 +43,10 @@
* the routines below consider this to be IRQ-safe
*/
spinlock_t lock;
+ /*
+ * Parent counter, used for hierarchial resource accounting
+ */
+ struct res_counter *parent;
};
/**
@@ -87,7 +91,7 @@
* helpers for accounting
*/
-void res_counter_init(struct res_counter *counter);
+void res_counter_init(struct res_counter *counter, struct res_counter *parent);
/*
* charge - try to consume more resource.
@@ -103,7 +107,7 @@
int __must_check res_counter_charge_locked(struct res_counter *counter,
unsigned long val);
int __must_check res_counter_charge(struct res_counter *counter,
- unsigned long val);
+ unsigned long val, struct res_counter **limit_fail_at);
/*
* uncharge - tell that some portion of the resource is released
diff --git a/include/linux/serial_core.h b/include/linux/serial_core.h
index b419984..90bbbf0 100644
--- a/include/linux/serial_core.h
+++ b/include/linux/serial_core.h
@@ -161,6 +161,9 @@
#define PORT_S3C6400 84
+/* NWPSERIAL */
+#define PORT_NWPSERIAL 85
+
#ifdef __KERNEL__
#include <linux/compiler.h>
diff --git a/include/linux/suspend.h b/include/linux/suspend.h
index 2ce8207..2b409c4 100644
--- a/include/linux/suspend.h
+++ b/include/linux/suspend.h
@@ -232,6 +232,11 @@
extern void hibernation_set_ops(struct platform_hibernation_ops *ops);
extern int hibernate(void);
+extern int hibernate_nvs_register(unsigned long start, unsigned long size);
+extern int hibernate_nvs_alloc(void);
+extern void hibernate_nvs_free(void);
+extern void hibernate_nvs_save(void);
+extern void hibernate_nvs_restore(void);
#else /* CONFIG_HIBERNATION */
static inline int swsusp_page_is_forbidden(struct page *p) { return 0; }
static inline void swsusp_set_page_free(struct page *p) {}
@@ -239,6 +244,14 @@
static inline void hibernation_set_ops(struct platform_hibernation_ops *ops) {}
static inline int hibernate(void) { return -ENOSYS; }
+static inline int hibernate_nvs_register(unsigned long a, unsigned long b)
+{
+ return 0;
+}
+static inline int hibernate_nvs_alloc(void) { return 0; }
+static inline void hibernate_nvs_free(void) {}
+static inline void hibernate_nvs_save(void) {}
+static inline void hibernate_nvs_restore(void) {}
#endif /* CONFIG_HIBERNATION */
#ifdef CONFIG_PM_SLEEP
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 91dee50..d302155 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -214,7 +214,8 @@
extern unsigned long try_to_free_pages(struct zonelist *zonelist, int order,
gfp_t gfp_mask);
extern unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *mem,
- gfp_t gfp_mask);
+ gfp_t gfp_mask, bool noswap,
+ unsigned int swappiness);
extern int __isolate_lru_page(struct page *page, int mode, int file);
extern unsigned long shrink_all_memory(unsigned long nr_pages);
extern int vm_swappiness;
@@ -333,6 +334,22 @@
put_swap_token(swap_token_mm);
}
+#ifdef CONFIG_CGROUP_MEM_RES_CTLR
+extern void mem_cgroup_uncharge_swapcache(struct page *page, swp_entry_t ent);
+#else
+static inline void
+mem_cgroup_uncharge_swapcache(struct page *page, swp_entry_t ent)
+{
+}
+#endif
+#ifdef CONFIG_CGROUP_MEM_RES_CTLR_SWAP
+extern void mem_cgroup_uncharge_swap(swp_entry_t ent);
+#else
+static inline void mem_cgroup_uncharge_swap(swp_entry_t ent)
+{
+}
+#endif
+
#else /* CONFIG_SWAP */
#define nr_swap_pages 0L
@@ -409,6 +426,12 @@
#define has_swap_token(x) 0
#define disable_swap_token() do { } while(0)
+static inline int mem_cgroup_cache_charge_swapin(struct page *page,
+ struct mm_struct *mm, gfp_t mask, bool locked)
+{
+ return 0;
+}
+
#endif /* CONFIG_SWAP */
#endif /* __KERNEL__*/
#endif /* _LINUX_SWAP_H */
diff --git a/include/net/netdma.h b/include/net/netdma.h
index f28c6e0..8ba8ce2 100644
--- a/include/net/netdma.h
+++ b/include/net/netdma.h
@@ -24,17 +24,6 @@
#include <linux/dmaengine.h>
#include <linux/skbuff.h>
-static inline struct dma_chan *get_softnet_dma(void)
-{
- struct dma_chan *chan;
- rcu_read_lock();
- chan = rcu_dereference(__get_cpu_var(softnet_data).net_dma);
- if (chan)
- dma_chan_get(chan);
- rcu_read_unlock();
- return chan;
-}
-
int dma_skb_copy_datagram_iovec(struct dma_chan* chan,
struct sk_buff *skb, int offset, struct iovec *to,
size_t len, struct dma_pinned_list *pinned_list);
diff --git a/include/net/protocol.h b/include/net/protocol.h
index cb2965a..ffa5b8b 100644
--- a/include/net/protocol.h
+++ b/include/net/protocol.h
@@ -59,6 +59,9 @@
int (*gso_send_check)(struct sk_buff *skb);
struct sk_buff *(*gso_segment)(struct sk_buff *skb,
int features);
+ struct sk_buff **(*gro_receive)(struct sk_buff **head,
+ struct sk_buff *skb);
+ int (*gro_complete)(struct sk_buff *skb);
unsigned int flags; /* INET6_PROTO_xxx */
};
diff --git a/include/net/wimax.h b/include/net/wimax.h
index 1602614..073809c 100644
--- a/include/net/wimax.h
+++ b/include/net/wimax.h
@@ -323,6 +323,9 @@
*
* @rf_hw: [private] State of the hardware radio switch (OFF/ON)
*
+ * @debufs_dentry: [private] Used to hook up a debugfs entry. This
+ * shows up in the debugfs root as wimax:DEVICENAME.
+ *
* Description:
* This structure defines a common interface to access all WiMAX
* devices from different vendors and provides a common API as well as
diff --git a/include/scsi/scsi_transport_fc.h b/include/scsi/scsi_transport_fc.h
index 6e04e6f..c9184f7 100644
--- a/include/scsi/scsi_transport_fc.h
+++ b/include/scsi/scsi_transport_fc.h
@@ -358,6 +358,7 @@
#define FC_RPORT_DEVLOSS_PENDING 0x01
#define FC_RPORT_SCAN_PENDING 0x02
#define FC_RPORT_FAST_FAIL_TIMEDOUT 0x04
+#define FC_RPORT_DEVLOSS_CALLBK_DONE 0x08
#define dev_to_rport(d) \
container_of(d, struct fc_rport, dev)
diff --git a/include/xen/xenbus.h b/include/xen/xenbus.h
index 6369d89..f87f961 100644
--- a/include/xen/xenbus.h
+++ b/include/xen/xenbus.h
@@ -136,8 +136,6 @@
/* Nil transaction ID. */
#define XBT_NIL ((struct xenbus_transaction) { 0 })
-int __init xenbus_dev_init(void);
-
char **xenbus_directory(struct xenbus_transaction t,
const char *dir, const char *node, unsigned int *num);
void *xenbus_read(struct xenbus_transaction t,
diff --git a/init/Kconfig b/init/Kconfig
index e7893b1..a724a14 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -271,11 +271,75 @@
13 => 8 KB
12 => 4 KB
+#
+# Architectures with an unreliable sched_clock() should select this:
+#
+config HAVE_UNSTABLE_SCHED_CLOCK
+ bool
+
+config GROUP_SCHED
+ bool "Group CPU scheduler"
+ depends on EXPERIMENTAL
+ default n
+ help
+ This feature lets CPU scheduler recognize task groups and control CPU
+ bandwidth allocation to such task groups.
+ In order to create a group from arbitrary set of processes, use
+ CONFIG_CGROUPS. (See Control Group support.)
+
+config FAIR_GROUP_SCHED
+ bool "Group scheduling for SCHED_OTHER"
+ depends on GROUP_SCHED
+ default GROUP_SCHED
+
+config RT_GROUP_SCHED
+ bool "Group scheduling for SCHED_RR/FIFO"
+ depends on EXPERIMENTAL
+ depends on GROUP_SCHED
+ default n
+ help
+ This feature lets you explicitly allocate real CPU bandwidth
+ to users or control groups (depending on the "Basis for grouping tasks"
+ setting below. If enabled, it will also make it impossible to
+ schedule realtime tasks for non-root users until you allocate
+ realtime bandwidth for them.
+ See Documentation/scheduler/sched-rt-group.txt for more information.
+
+choice
+ depends on GROUP_SCHED
+ prompt "Basis for grouping tasks"
+ default USER_SCHED
+
+config USER_SCHED
+ bool "user id"
+ help
+ This option will choose userid as the basis for grouping
+ tasks, thus providing equal CPU bandwidth to each user.
+
+config CGROUP_SCHED
+ bool "Control groups"
+ depends on CGROUPS
+ help
+ This option allows you to create arbitrary task groups
+ using the "cgroup" pseudo filesystem and control
+ the cpu bandwidth allocated to each such task group.
+ Refer to Documentation/cgroups.txt for more information
+ on "cgroup" pseudo filesystem.
+
+endchoice
+
+menu "Control Group support"
config CGROUPS
bool "Control Group support"
help
- This option will let you use process cgroup subsystems
- such as Cpusets
+ This option add support for grouping sets of processes together, for
+ use with process control subsystems such as Cpusets, CFS, memory
+ controls or device isolation.
+ See
+ - Documentation/cpusets.txt (Cpusets)
+ - Documentation/scheduler/sched-design-CFS.txt (CFS)
+ - Documentation/cgroups/ (features for grouping, isolation)
+ - Documentation/controllers/ (features for resource control)
Say N if unsure.
@@ -324,61 +388,6 @@
Say N if unsure.
-#
-# Architectures with an unreliable sched_clock() should select this:
-#
-config HAVE_UNSTABLE_SCHED_CLOCK
- bool
-
-config GROUP_SCHED
- bool "Group CPU scheduler"
- depends on EXPERIMENTAL
- default n
- help
- This feature lets CPU scheduler recognize task groups and control CPU
- bandwidth allocation to such task groups.
-
-config FAIR_GROUP_SCHED
- bool "Group scheduling for SCHED_OTHER"
- depends on GROUP_SCHED
- default GROUP_SCHED
-
-config RT_GROUP_SCHED
- bool "Group scheduling for SCHED_RR/FIFO"
- depends on EXPERIMENTAL
- depends on GROUP_SCHED
- default n
- help
- This feature lets you explicitly allocate real CPU bandwidth
- to users or control groups (depending on the "Basis for grouping tasks"
- setting below. If enabled, it will also make it impossible to
- schedule realtime tasks for non-root users until you allocate
- realtime bandwidth for them.
- See Documentation/scheduler/sched-rt-group.txt for more information.
-
-choice
- depends on GROUP_SCHED
- prompt "Basis for grouping tasks"
- default USER_SCHED
-
-config USER_SCHED
- bool "user id"
- help
- This option will choose userid as the basis for grouping
- tasks, thus providing equal CPU bandwidth to each user.
-
-config CGROUP_SCHED
- bool "Control groups"
- depends on CGROUPS
- help
- This option allows you to create arbitrary task groups
- using the "cgroup" pseudo filesystem and control
- the cpu bandwidth allocated to each such task group.
- Refer to Documentation/cgroups.txt for more information
- on "cgroup" pseudo filesystem.
-
-endchoice
-
config CGROUP_CPUACCT
bool "Simple CPU accounting cgroup subsystem"
depends on CGROUPS
@@ -393,9 +402,6 @@
infrastructure that works with cgroups
depends on CGROUPS
-config MM_OWNER
- bool
-
config CGROUP_MEM_RES_CTLR
bool "Memory Resource Controller for Control Groups"
depends on CGROUPS && RESOURCE_COUNTERS
@@ -414,11 +420,33 @@
sure you need the memory resource controller. Even when you enable
this, you can set "cgroup_disable=memory" at your boot option to
disable memory resource controller and you can avoid overheads.
- (and lose benefits of memory resource contoller)
+ (and lose benefits of memory resource controller)
This config option also selects MM_OWNER config option, which
could in turn add some fork/exit overhead.
+config MM_OWNER
+ bool
+
+config CGROUP_MEM_RES_CTLR_SWAP
+ bool "Memory Resource Controller Swap Extension(EXPERIMENTAL)"
+ depends on CGROUP_MEM_RES_CTLR && SWAP && EXPERIMENTAL
+ help
+ Add swap management feature to memory resource controller. When you
+ enable this, you can limit mem+swap usage per cgroup. In other words,
+ when you disable this, memory resource controller has no cares to
+ usage of swap...a process can exhaust all of the swap. This extension
+ is useful when you want to avoid exhaustion swap but this itself
+ adds more overheads and consumes memory for remembering information.
+ Especially if you use 32bit system or small memory system, please
+ be careful about enabling this. When memory resource controller
+ is disabled by boot option, this will be automatically disabled and
+ there will be no overhead from this. Even when you set this config=y,
+ if boot option "noswapaccount" is set, swap will not be accounted.
+
+
+endmenu
+
config SYSFS_DEPRECATED
bool
diff --git a/ipc/mqueue.c b/ipc/mqueue.c
index eddb624..23fdb84 100644
--- a/ipc/mqueue.c
+++ b/ipc/mqueue.c
@@ -505,7 +505,8 @@
sig_i.si_errno = 0;
sig_i.si_code = SI_MESGQ;
sig_i.si_value = info->notify.sigev_value;
- sig_i.si_pid = task_tgid_vnr(current);
+ sig_i.si_pid = task_tgid_nr_ns(current,
+ ns_of_pid(info->notify_owner));
sig_i.si_uid = current_uid();
kill_pid_info(info->notify.sigev_signo,
diff --git a/kernel/async.c b/kernel/async.c
index 9737338..64cc916 100644
--- a/kernel/async.c
+++ b/kernel/async.c
@@ -206,7 +206,9 @@
void async_synchronize_full(void)
{
- async_synchronize_cookie(next_cookie);
+ do {
+ async_synchronize_cookie(next_cookie);
+ } while (!list_empty(&async_running) || !list_empty(&async_pending));
}
EXPORT_SYMBOL_GPL(async_synchronize_full);
diff --git a/kernel/cgroup.c b/kernel/cgroup.c
index f221446..c298310 100644
--- a/kernel/cgroup.c
+++ b/kernel/cgroup.c
@@ -84,7 +84,7 @@
/* Tracks how many cgroups are currently defined in hierarchy.*/
int number_of_cgroups;
- /* A list running through the mounted hierarchies */
+ /* A list running through the active hierarchies */
struct list_head root_list;
/* Hierarchy-specific flags */
@@ -148,8 +148,8 @@
#define for_each_subsys(_root, _ss) \
list_for_each_entry(_ss, &_root->subsys_list, sibling)
-/* for_each_root() allows you to iterate across the active hierarchies */
-#define for_each_root(_root) \
+/* for_each_active_root() allows you to iterate across the active hierarchies */
+#define for_each_active_root(_root) \
list_for_each_entry(_root, &roots, root_list)
/* the list of cgroups eligible for automatic release. Protected by
@@ -271,7 +271,7 @@
rcu_read_lock();
for (i = 0; i < CGROUP_SUBSYS_COUNT; i++) {
- struct cgroup *cgrp = cg->subsys[i]->cgroup;
+ struct cgroup *cgrp = rcu_dereference(cg->subsys[i]->cgroup);
if (atomic_dec_and_test(&cgrp->count) &&
notify_on_release(cgrp)) {
if (taskexit)
@@ -384,6 +384,25 @@
return 0;
}
+/**
+ * link_css_set - a helper function to link a css_set to a cgroup
+ * @tmp_cg_links: cg_cgroup_link objects allocated by allocate_cg_links()
+ * @cg: the css_set to be linked
+ * @cgrp: the destination cgroup
+ */
+static void link_css_set(struct list_head *tmp_cg_links,
+ struct css_set *cg, struct cgroup *cgrp)
+{
+ struct cg_cgroup_link *link;
+
+ BUG_ON(list_empty(tmp_cg_links));
+ link = list_first_entry(tmp_cg_links, struct cg_cgroup_link,
+ cgrp_link_list);
+ link->cg = cg;
+ list_move(&link->cgrp_link_list, &cgrp->css_sets);
+ list_add(&link->cg_link_list, &cg->cg_links);
+}
+
/*
* find_css_set() takes an existing cgroup group and a
* cgroup object, and returns a css_set object that's
@@ -399,7 +418,6 @@
int i;
struct list_head tmp_cg_links;
- struct cg_cgroup_link *link;
struct hlist_head *hhead;
@@ -444,26 +462,11 @@
* only do it for the first subsystem in each
* hierarchy
*/
- if (ss->root->subsys_list.next == &ss->sibling) {
- BUG_ON(list_empty(&tmp_cg_links));
- link = list_entry(tmp_cg_links.next,
- struct cg_cgroup_link,
- cgrp_link_list);
- list_del(&link->cgrp_link_list);
- list_add(&link->cgrp_link_list, &cgrp->css_sets);
- link->cg = res;
- list_add(&link->cg_link_list, &res->cg_links);
- }
+ if (ss->root->subsys_list.next == &ss->sibling)
+ link_css_set(&tmp_cg_links, res, cgrp);
}
- if (list_empty(&rootnode.subsys_list)) {
- link = list_entry(tmp_cg_links.next,
- struct cg_cgroup_link,
- cgrp_link_list);
- list_del(&link->cgrp_link_list);
- list_add(&link->cgrp_link_list, &dummytop->css_sets);
- link->cg = res;
- list_add(&link->cg_link_list, &res->cg_links);
- }
+ if (list_empty(&rootnode.subsys_list))
+ link_css_set(&tmp_cg_links, res, dummytop);
BUG_ON(!list_empty(&tmp_cg_links));
@@ -586,11 +589,18 @@
{
struct cgroup_subsys *ss;
for_each_subsys(cgrp->root, ss)
- if (ss->pre_destroy && cgrp->subsys[ss->subsys_id])
+ if (ss->pre_destroy)
ss->pre_destroy(ss, cgrp);
return;
}
+static void free_cgroup_rcu(struct rcu_head *obj)
+{
+ struct cgroup *cgrp = container_of(obj, struct cgroup, rcu_head);
+
+ kfree(cgrp);
+}
+
static void cgroup_diput(struct dentry *dentry, struct inode *inode)
{
/* is dentry a directory ? if so, kfree() associated cgroup */
@@ -610,19 +620,19 @@
/*
* Release the subsystem state objects.
*/
- for_each_subsys(cgrp->root, ss) {
- if (cgrp->subsys[ss->subsys_id])
- ss->destroy(ss, cgrp);
- }
+ for_each_subsys(cgrp->root, ss)
+ ss->destroy(ss, cgrp);
cgrp->root->number_of_cgroups--;
mutex_unlock(&cgroup_mutex);
- /* Drop the active superblock reference that we took when we
- * created the cgroup */
+ /*
+ * Drop the active superblock reference that we took when we
+ * created the cgroup
+ */
deactivate_super(cgrp->root->sb);
- kfree(cgrp);
+ call_rcu(&cgrp->rcu_head, free_cgroup_rcu);
}
iput(inode);
}
@@ -712,23 +722,26 @@
BUG_ON(cgrp->subsys[i]);
BUG_ON(!dummytop->subsys[i]);
BUG_ON(dummytop->subsys[i]->cgroup != dummytop);
+ mutex_lock(&ss->hierarchy_mutex);
cgrp->subsys[i] = dummytop->subsys[i];
cgrp->subsys[i]->cgroup = cgrp;
- list_add(&ss->sibling, &root->subsys_list);
- rcu_assign_pointer(ss->root, root);
+ list_move(&ss->sibling, &root->subsys_list);
+ ss->root = root;
if (ss->bind)
ss->bind(ss, cgrp);
-
+ mutex_unlock(&ss->hierarchy_mutex);
} else if (bit & removed_bits) {
/* We're removing this subsystem */
BUG_ON(cgrp->subsys[i] != dummytop->subsys[i]);
BUG_ON(cgrp->subsys[i]->cgroup != cgrp);
+ mutex_lock(&ss->hierarchy_mutex);
if (ss->bind)
ss->bind(ss, dummytop);
dummytop->subsys[i]->cgroup = dummytop;
cgrp->subsys[i] = NULL;
- rcu_assign_pointer(subsys[i]->root, &rootnode);
- list_del(&ss->sibling);
+ subsys[i]->root = &rootnode;
+ list_move(&ss->sibling, &rootnode.subsys_list);
+ mutex_unlock(&ss->hierarchy_mutex);
} else if (bit & final_bits) {
/* Subsystem state should already exist */
BUG_ON(!cgrp->subsys[i]);
@@ -990,7 +1003,7 @@
root = NULL;
} else {
/* New superblock */
- struct cgroup *cgrp = &root->top_cgroup;
+ struct cgroup *root_cgrp = &root->top_cgroup;
struct inode *inode;
int i;
@@ -1031,7 +1044,7 @@
list_add(&root->root_list, &roots);
root_count++;
- sb->s_root->d_fsdata = &root->top_cgroup;
+ sb->s_root->d_fsdata = root_cgrp;
root->top_cgroup.dentry = sb->s_root;
/* Link the top cgroup in this hierarchy into all
@@ -1042,29 +1055,18 @@
struct hlist_node *node;
struct css_set *cg;
- hlist_for_each_entry(cg, node, hhead, hlist) {
- struct cg_cgroup_link *link;
-
- BUG_ON(list_empty(&tmp_cg_links));
- link = list_entry(tmp_cg_links.next,
- struct cg_cgroup_link,
- cgrp_link_list);
- list_del(&link->cgrp_link_list);
- link->cg = cg;
- list_add(&link->cgrp_link_list,
- &root->top_cgroup.css_sets);
- list_add(&link->cg_link_list, &cg->cg_links);
- }
+ hlist_for_each_entry(cg, node, hhead, hlist)
+ link_css_set(&tmp_cg_links, cg, root_cgrp);
}
write_unlock(&css_set_lock);
free_cg_links(&tmp_cg_links);
- BUG_ON(!list_empty(&cgrp->sibling));
- BUG_ON(!list_empty(&cgrp->children));
+ BUG_ON(!list_empty(&root_cgrp->sibling));
+ BUG_ON(!list_empty(&root_cgrp->children));
BUG_ON(root->number_of_cgroups != 1);
- cgroup_populate_dir(cgrp);
+ cgroup_populate_dir(root_cgrp);
mutex_unlock(&inode->i_mutex);
mutex_unlock(&cgroup_mutex);
}
@@ -1113,10 +1115,9 @@
}
write_unlock(&css_set_lock);
- if (!list_empty(&root->root_list)) {
- list_del(&root->root_list);
- root_count--;
- }
+ list_del(&root->root_list);
+ root_count--;
+
mutex_unlock(&cgroup_mutex);
kfree(root);
@@ -1145,14 +1146,16 @@
* @buf: the buffer to write the path into
* @buflen: the length of the buffer
*
- * Called with cgroup_mutex held. Writes path of cgroup into buf.
- * Returns 0 on success, -errno on error.
+ * Called with cgroup_mutex held or else with an RCU-protected cgroup
+ * reference. Writes path of cgroup into buf. Returns 0 on success,
+ * -errno on error.
*/
int cgroup_path(const struct cgroup *cgrp, char *buf, int buflen)
{
char *start;
+ struct dentry *dentry = rcu_dereference(cgrp->dentry);
- if (cgrp == dummytop) {
+ if (!dentry || cgrp == dummytop) {
/*
* Inactive subsystems have no dentry for their root
* cgroup
@@ -1165,13 +1168,14 @@
*--start = '\0';
for (;;) {
- int len = cgrp->dentry->d_name.len;
+ int len = dentry->d_name.len;
if ((start -= len) < buf)
return -ENAMETOOLONG;
memcpy(start, cgrp->dentry->d_name.name, len);
cgrp = cgrp->parent;
if (!cgrp)
break;
+ dentry = rcu_dereference(cgrp->dentry);
if (!cgrp->parent)
continue;
if (--start < buf)
@@ -1216,7 +1220,7 @@
int retval = 0;
struct cgroup_subsys *ss;
struct cgroup *oldcgrp;
- struct css_set *cg = tsk->cgroups;
+ struct css_set *cg;
struct css_set *newcg;
struct cgroupfs_root *root = cgrp->root;
int subsys_id;
@@ -1236,11 +1240,16 @@
}
}
+ task_lock(tsk);
+ cg = tsk->cgroups;
+ get_css_set(cg);
+ task_unlock(tsk);
/*
* Locate or allocate a new css_set for this task,
* based on its final set of cgroups
*/
newcg = find_css_set(cg, cgrp);
+ put_css_set(cg);
if (!newcg)
return -ENOMEM;
@@ -1445,7 +1454,7 @@
struct cftype *cft = __d_cft(file->f_dentry);
struct cgroup *cgrp = __d_cgrp(file->f_dentry->d_parent);
- if (!cft || cgroup_is_removed(cgrp))
+ if (cgroup_is_removed(cgrp))
return -ENODEV;
if (cft->write)
return cft->write(cgrp, cft, file, buf, nbytes, ppos);
@@ -1490,7 +1499,7 @@
struct cftype *cft = __d_cft(file->f_dentry);
struct cgroup *cgrp = __d_cgrp(file->f_dentry->d_parent);
- if (!cft || cgroup_is_removed(cgrp))
+ if (cgroup_is_removed(cgrp))
return -ENODEV;
if (cft->read)
@@ -1554,10 +1563,8 @@
err = generic_file_open(inode, file);
if (err)
return err;
-
cft = __d_cft(file->f_dentry);
- if (!cft)
- return -ENODEV;
+
if (cft->read_map || cft->read_seq_string) {
struct cgroup_seqfile_state *state =
kzalloc(sizeof(*state), GFP_USER);
@@ -1671,7 +1678,7 @@
if (!error) {
dentry->d_fsdata = cgrp;
inc_nlink(parent->d_inode);
- cgrp->dentry = dentry;
+ rcu_assign_pointer(cgrp->dentry, dentry);
dget(dentry);
}
dput(dentry);
@@ -1812,6 +1819,7 @@
{
struct task_struct *res;
struct list_head *l = it->task;
+ struct cg_cgroup_link *link;
/* If the iterator cg is NULL, we have no tasks */
if (!it->cg_link)
@@ -1819,7 +1827,8 @@
res = list_entry(l, struct task_struct, cg_list);
/* Advance iterator to find next entry */
l = l->next;
- if (l == &res->cgroups->tasks) {
+ link = list_entry(it->cg_link, struct cg_cgroup_link, cgrp_link_list);
+ if (l == &link->cg->tasks) {
/* We reached the end of this task list - move on to
* the next cg_cgroup_link */
cgroup_advance_iter(cgrp, it);
@@ -2013,14 +2022,16 @@
*/
static int pid_array_load(pid_t *pidarray, int npids, struct cgroup *cgrp)
{
- int n = 0;
+ int n = 0, pid;
struct cgroup_iter it;
struct task_struct *tsk;
cgroup_iter_start(cgrp, &it);
while ((tsk = cgroup_iter_next(cgrp, &it))) {
if (unlikely(n == npids))
break;
- pidarray[n++] = task_pid_vnr(tsk);
+ pid = task_pid_vnr(tsk);
+ if (pid > 0)
+ pidarray[n++] = pid;
}
cgroup_iter_end(cgrp, &it);
return n;
@@ -2052,7 +2063,6 @@
ret = 0;
cgrp = dentry->d_fsdata;
- rcu_read_lock();
cgroup_iter_start(cgrp, &it);
while ((tsk = cgroup_iter_next(cgrp, &it))) {
@@ -2077,7 +2087,6 @@
}
cgroup_iter_end(cgrp, &it);
- rcu_read_unlock();
err:
return ret;
}
@@ -2324,7 +2333,7 @@
struct cgroup *cgrp)
{
css->cgroup = cgrp;
- atomic_set(&css->refcnt, 0);
+ atomic_set(&css->refcnt, 1);
css->flags = 0;
if (cgrp == dummytop)
set_bit(CSS_ROOT, &css->flags);
@@ -2332,6 +2341,29 @@
cgrp->subsys[ss->subsys_id] = css;
}
+static void cgroup_lock_hierarchy(struct cgroupfs_root *root)
+{
+ /* We need to take each hierarchy_mutex in a consistent order */
+ int i;
+
+ for (i = 0; i < CGROUP_SUBSYS_COUNT; i++) {
+ struct cgroup_subsys *ss = subsys[i];
+ if (ss->root == root)
+ mutex_lock_nested(&ss->hierarchy_mutex, i);
+ }
+}
+
+static void cgroup_unlock_hierarchy(struct cgroupfs_root *root)
+{
+ int i;
+
+ for (i = 0; i < CGROUP_SUBSYS_COUNT; i++) {
+ struct cgroup_subsys *ss = subsys[i];
+ if (ss->root == root)
+ mutex_unlock(&ss->hierarchy_mutex);
+ }
+}
+
/*
* cgroup_create - create a cgroup
* @parent: cgroup that will be parent of the new cgroup
@@ -2380,7 +2412,9 @@
init_cgroup_css(css, ss, cgrp);
}
+ cgroup_lock_hierarchy(root);
list_add(&cgrp->sibling, &cgrp->parent->children);
+ cgroup_unlock_hierarchy(root);
root->number_of_cgroups++;
err = cgroup_create_dir(cgrp, dentry, mode);
@@ -2431,7 +2465,7 @@
{
/* Check the reference count on each subsystem. Since we
* already established that there are no tasks in the
- * cgroup, if the css refcount is also 0, then there should
+ * cgroup, if the css refcount is also 1, then there should
* be no outstanding references, so the subsystem is safe to
* destroy. We scan across all subsystems rather than using
* the per-hierarchy linked list of mounted subsystems since
@@ -2452,19 +2486,67 @@
* matter, since it can only happen if the cgroup
* has been deleted and hence no longer needs the
* release agent to be called anyway. */
- if (css && atomic_read(&css->refcnt))
+ if (css && (atomic_read(&css->refcnt) > 1))
return 1;
}
return 0;
}
+/*
+ * Atomically mark all (or else none) of the cgroup's CSS objects as
+ * CSS_REMOVED. Return true on success, or false if the cgroup has
+ * busy subsystems. Call with cgroup_mutex held
+ */
+
+static int cgroup_clear_css_refs(struct cgroup *cgrp)
+{
+ struct cgroup_subsys *ss;
+ unsigned long flags;
+ bool failed = false;
+ local_irq_save(flags);
+ for_each_subsys(cgrp->root, ss) {
+ struct cgroup_subsys_state *css = cgrp->subsys[ss->subsys_id];
+ int refcnt;
+ do {
+ /* We can only remove a CSS with a refcnt==1 */
+ refcnt = atomic_read(&css->refcnt);
+ if (refcnt > 1) {
+ failed = true;
+ goto done;
+ }
+ BUG_ON(!refcnt);
+ /*
+ * Drop the refcnt to 0 while we check other
+ * subsystems. This will cause any racing
+ * css_tryget() to spin until we set the
+ * CSS_REMOVED bits or abort
+ */
+ } while (atomic_cmpxchg(&css->refcnt, refcnt, 0) != refcnt);
+ }
+ done:
+ for_each_subsys(cgrp->root, ss) {
+ struct cgroup_subsys_state *css = cgrp->subsys[ss->subsys_id];
+ if (failed) {
+ /*
+ * Restore old refcnt if we previously managed
+ * to clear it from 1 to 0
+ */
+ if (!atomic_read(&css->refcnt))
+ atomic_set(&css->refcnt, 1);
+ } else {
+ /* Commit the fact that the CSS is removed */
+ set_bit(CSS_REMOVED, &css->flags);
+ }
+ }
+ local_irq_restore(flags);
+ return !failed;
+}
+
static int cgroup_rmdir(struct inode *unused_dir, struct dentry *dentry)
{
struct cgroup *cgrp = dentry->d_fsdata;
struct dentry *d;
struct cgroup *parent;
- struct super_block *sb;
- struct cgroupfs_root *root;
/* the vfs holds both inode->i_mutex already */
@@ -2487,12 +2569,10 @@
mutex_lock(&cgroup_mutex);
parent = cgrp->parent;
- root = cgrp->root;
- sb = root->sb;
if (atomic_read(&cgrp->count)
|| !list_empty(&cgrp->children)
- || cgroup_has_css_refs(cgrp)) {
+ || !cgroup_clear_css_refs(cgrp)) {
mutex_unlock(&cgroup_mutex);
return -EBUSY;
}
@@ -2502,8 +2582,12 @@
if (!list_empty(&cgrp->release_list))
list_del(&cgrp->release_list);
spin_unlock(&release_list_lock);
- /* delete my sibling from parent->children */
+
+ cgroup_lock_hierarchy(cgrp->root);
+ /* delete this cgroup from parent->children */
list_del(&cgrp->sibling);
+ cgroup_unlock_hierarchy(cgrp->root);
+
spin_lock(&cgrp->dentry->d_lock);
d = dget(cgrp->dentry);
spin_unlock(&d->d_lock);
@@ -2525,6 +2609,7 @@
printk(KERN_INFO "Initializing cgroup subsys %s\n", ss->name);
/* Create the top cgroup state for this subsystem */
+ list_add(&ss->sibling, &rootnode.subsys_list);
ss->root = &rootnode;
css = ss->create(ss, dummytop);
/* We don't handle early failures gracefully */
@@ -2544,6 +2629,7 @@
* need to invoke fork callbacks here. */
BUG_ON(!list_empty(&init_task.tasks));
+ mutex_init(&ss->hierarchy_mutex);
ss->active = 1;
}
@@ -2562,7 +2648,6 @@
INIT_HLIST_NODE(&init_css_set.hlist);
css_set_count = 1;
init_cgroup_root(&rootnode);
- list_add(&rootnode.root_list, &roots);
root_count = 1;
init_task.cgroups = &init_css_set;
@@ -2669,15 +2754,12 @@
mutex_lock(&cgroup_mutex);
- for_each_root(root) {
+ for_each_active_root(root) {
struct cgroup_subsys *ss;
struct cgroup *cgrp;
int subsys_id;
int count = 0;
- /* Skip this hierarchy if it has no active subsystems */
- if (!root->actual_subsys_bits)
- continue;
seq_printf(m, "%lu:", root->subsys_bits);
for_each_subsys(root, ss)
seq_printf(m, "%s%s", count++ ? "," : "", ss->name);
@@ -2800,8 +2882,10 @@
{
if (use_task_css_set_links) {
write_lock(&css_set_lock);
+ task_lock(child);
if (list_empty(&child->cg_list))
list_add(&child->cg_list, &child->cgroups->tasks);
+ task_unlock(child);
write_unlock(&css_set_lock);
}
}
@@ -2907,6 +2991,7 @@
mutex_unlock(&cgroup_mutex);
return 0;
}
+ task_lock(tsk);
cg = tsk->cgroups;
parent = task_cgroup(tsk, subsys->subsys_id);
@@ -2919,6 +3004,7 @@
/* Keep the cgroup alive */
get_css_set(cg);
+ task_unlock(tsk);
mutex_unlock(&cgroup_mutex);
/* Now do the VFS work to create a cgroup */
@@ -2937,7 +3023,7 @@
}
/* Create the cgroup directory, which also creates the cgroup */
- ret = vfs_mkdir(inode, dentry, S_IFDIR | 0755);
+ ret = vfs_mkdir(inode, dentry, 0755);
child = __d_cgrp(dentry);
dput(dentry);
if (ret) {
@@ -2947,13 +3033,6 @@
goto out_release;
}
- if (!child) {
- printk(KERN_INFO
- "Couldn't find new cgroup %s\n", nodename);
- ret = -ENOMEM;
- goto out_release;
- }
-
/* The cgroup now exists. Retake cgroup_mutex and check
* that we're still in the same state that we thought we
* were. */
@@ -3049,7 +3128,8 @@
{
struct cgroup *cgrp = css->cgroup;
rcu_read_lock();
- if (atomic_dec_and_test(&css->refcnt) && notify_on_release(cgrp)) {
+ if ((atomic_dec_return(&css->refcnt) == 1) &&
+ notify_on_release(cgrp)) {
set_bit(CGRP_RELEASABLE, &cgrp->flags);
check_for_release(cgrp);
}
diff --git a/kernel/cpuset.c b/kernel/cpuset.c
index 345ace5..647c77a 100644
--- a/kernel/cpuset.c
+++ b/kernel/cpuset.c
@@ -84,7 +84,7 @@
struct cgroup_subsys_state css;
unsigned long flags; /* "unsigned long" so bitops work */
- cpumask_t cpus_allowed; /* CPUs allowed to tasks in cpuset */
+ cpumask_var_t cpus_allowed; /* CPUs allowed to tasks in cpuset */
nodemask_t mems_allowed; /* Memory Nodes allowed to tasks */
struct cpuset *parent; /* my parent */
@@ -195,8 +195,6 @@
static struct cpuset top_cpuset = {
.flags = ((1 << CS_CPU_EXCLUSIVE) | (1 << CS_MEM_EXCLUSIVE)),
- .cpus_allowed = CPU_MASK_ALL,
- .mems_allowed = NODE_MASK_ALL,
};
/*
@@ -278,7 +276,7 @@
};
/*
- * Return in *pmask the portion of a cpusets's cpus_allowed that
+ * Return in pmask the portion of a cpusets's cpus_allowed that
* are online. If none are online, walk up the cpuset hierarchy
* until we find one that does have some online cpus. If we get
* all the way to the top and still haven't found any online cpus,
@@ -291,15 +289,16 @@
* Call with callback_mutex held.
*/
-static void guarantee_online_cpus(const struct cpuset *cs, cpumask_t *pmask)
+static void guarantee_online_cpus(const struct cpuset *cs,
+ struct cpumask *pmask)
{
- while (cs && !cpus_intersects(cs->cpus_allowed, cpu_online_map))
+ while (cs && !cpumask_intersects(cs->cpus_allowed, cpu_online_mask))
cs = cs->parent;
if (cs)
- cpus_and(*pmask, cs->cpus_allowed, cpu_online_map);
+ cpumask_and(pmask, cs->cpus_allowed, cpu_online_mask);
else
- *pmask = cpu_online_map;
- BUG_ON(!cpus_intersects(*pmask, cpu_online_map));
+ cpumask_copy(pmask, cpu_online_mask);
+ BUG_ON(!cpumask_intersects(pmask, cpu_online_mask));
}
/*
@@ -375,14 +374,9 @@
struct task_struct *tsk = current;
struct cpuset *cs;
- if (task_cs(tsk) == &top_cpuset) {
- /* Don't need rcu for top_cpuset. It's never freed. */
- my_cpusets_mem_gen = top_cpuset.mems_generation;
- } else {
- rcu_read_lock();
- my_cpusets_mem_gen = task_cs(tsk)->mems_generation;
- rcu_read_unlock();
- }
+ rcu_read_lock();
+ my_cpusets_mem_gen = task_cs(tsk)->mems_generation;
+ rcu_read_unlock();
if (my_cpusets_mem_gen != tsk->cpuset_mems_generation) {
mutex_lock(&callback_mutex);
@@ -414,12 +408,43 @@
static int is_cpuset_subset(const struct cpuset *p, const struct cpuset *q)
{
- return cpus_subset(p->cpus_allowed, q->cpus_allowed) &&
+ return cpumask_subset(p->cpus_allowed, q->cpus_allowed) &&
nodes_subset(p->mems_allowed, q->mems_allowed) &&
is_cpu_exclusive(p) <= is_cpu_exclusive(q) &&
is_mem_exclusive(p) <= is_mem_exclusive(q);
}
+/**
+ * alloc_trial_cpuset - allocate a trial cpuset
+ * @cs: the cpuset that the trial cpuset duplicates
+ */
+static struct cpuset *alloc_trial_cpuset(const struct cpuset *cs)
+{
+ struct cpuset *trial;
+
+ trial = kmemdup(cs, sizeof(*cs), GFP_KERNEL);
+ if (!trial)
+ return NULL;
+
+ if (!alloc_cpumask_var(&trial->cpus_allowed, GFP_KERNEL)) {
+ kfree(trial);
+ return NULL;
+ }
+ cpumask_copy(trial->cpus_allowed, cs->cpus_allowed);
+
+ return trial;
+}
+
+/**
+ * free_trial_cpuset - free the trial cpuset
+ * @trial: the trial cpuset to be freed
+ */
+static void free_trial_cpuset(struct cpuset *trial)
+{
+ free_cpumask_var(trial->cpus_allowed);
+ kfree(trial);
+}
+
/*
* validate_change() - Used to validate that any proposed cpuset change
* follows the structural rules for cpusets.
@@ -469,7 +494,7 @@
c = cgroup_cs(cont);
if ((is_cpu_exclusive(trial) || is_cpu_exclusive(c)) &&
c != cur &&
- cpus_intersects(trial->cpus_allowed, c->cpus_allowed))
+ cpumask_intersects(trial->cpus_allowed, c->cpus_allowed))
return -EINVAL;
if ((is_mem_exclusive(trial) || is_mem_exclusive(c)) &&
c != cur &&
@@ -479,7 +504,7 @@
/* Cpusets with tasks can't have empty cpus_allowed or mems_allowed */
if (cgroup_task_count(cur->css.cgroup)) {
- if (cpus_empty(trial->cpus_allowed) ||
+ if (cpumask_empty(trial->cpus_allowed) ||
nodes_empty(trial->mems_allowed)) {
return -ENOSPC;
}
@@ -494,7 +519,7 @@
*/
static int cpusets_overlap(struct cpuset *a, struct cpuset *b)
{
- return cpus_intersects(a->cpus_allowed, b->cpus_allowed);
+ return cpumask_intersects(a->cpus_allowed, b->cpus_allowed);
}
static void
@@ -519,7 +544,7 @@
cp = list_first_entry(&q, struct cpuset, stack_list);
list_del(q.next);
- if (cpus_empty(cp->cpus_allowed))
+ if (cpumask_empty(cp->cpus_allowed))
continue;
if (is_sched_load_balance(cp))
@@ -586,7 +611,8 @@
* element of the partition (one sched domain) to be passed to
* partition_sched_domains().
*/
-static int generate_sched_domains(cpumask_t **domains,
+/* FIXME: see the FIXME in partition_sched_domains() */
+static int generate_sched_domains(struct cpumask **domains,
struct sched_domain_attr **attributes)
{
LIST_HEAD(q); /* queue of cpusets to be scanned */
@@ -594,10 +620,10 @@
struct cpuset **csa; /* array of all cpuset ptrs */
int csn; /* how many cpuset ptrs in csa so far */
int i, j, k; /* indices for partition finding loops */
- cpumask_t *doms; /* resulting partition; i.e. sched domains */
+ struct cpumask *doms; /* resulting partition; i.e. sched domains */
struct sched_domain_attr *dattr; /* attributes for custom domains */
int ndoms = 0; /* number of sched domains in result */
- int nslot; /* next empty doms[] cpumask_t slot */
+ int nslot; /* next empty doms[] struct cpumask slot */
doms = NULL;
dattr = NULL;
@@ -605,7 +631,7 @@
/* Special case for the 99% of systems with one, full, sched domain */
if (is_sched_load_balance(&top_cpuset)) {
- doms = kmalloc(sizeof(cpumask_t), GFP_KERNEL);
+ doms = kmalloc(cpumask_size(), GFP_KERNEL);
if (!doms)
goto done;
@@ -614,7 +640,7 @@
*dattr = SD_ATTR_INIT;
update_domain_attr_tree(dattr, &top_cpuset);
}
- *doms = top_cpuset.cpus_allowed;
+ cpumask_copy(doms, top_cpuset.cpus_allowed);
ndoms = 1;
goto done;
@@ -633,7 +659,7 @@
cp = list_first_entry(&q, struct cpuset, stack_list);
list_del(q.next);
- if (cpus_empty(cp->cpus_allowed))
+ if (cpumask_empty(cp->cpus_allowed))
continue;
/*
@@ -684,7 +710,7 @@
* Now we know how many domains to create.
* Convert <csn, csa> to <ndoms, doms> and populate cpu masks.
*/
- doms = kmalloc(ndoms * sizeof(cpumask_t), GFP_KERNEL);
+ doms = kmalloc(ndoms * cpumask_size(), GFP_KERNEL);
if (!doms)
goto done;
@@ -696,7 +722,7 @@
for (nslot = 0, i = 0; i < csn; i++) {
struct cpuset *a = csa[i];
- cpumask_t *dp;
+ struct cpumask *dp;
int apn = a->pn;
if (apn < 0) {
@@ -719,14 +745,14 @@
continue;
}
- cpus_clear(*dp);
+ cpumask_clear(dp);
if (dattr)
*(dattr + nslot) = SD_ATTR_INIT;
for (j = i; j < csn; j++) {
struct cpuset *b = csa[j];
if (apn == b->pn) {
- cpus_or(*dp, *dp, b->cpus_allowed);
+ cpumask_or(dp, dp, b->cpus_allowed);
if (dattr)
update_domain_attr_tree(dattr + nslot, b);
@@ -766,7 +792,7 @@
static void do_rebuild_sched_domains(struct work_struct *unused)
{
struct sched_domain_attr *attr;
- cpumask_t *doms;
+ struct cpumask *doms;
int ndoms;
get_online_cpus();
@@ -835,7 +861,7 @@
static int cpuset_test_cpumask(struct task_struct *tsk,
struct cgroup_scanner *scan)
{
- return !cpus_equal(tsk->cpus_allowed,
+ return !cpumask_equal(&tsk->cpus_allowed,
(cgroup_cs(scan->cg))->cpus_allowed);
}
@@ -853,7 +879,7 @@
static void cpuset_change_cpumask(struct task_struct *tsk,
struct cgroup_scanner *scan)
{
- set_cpus_allowed_ptr(tsk, &((cgroup_cs(scan->cg))->cpus_allowed));
+ set_cpus_allowed_ptr(tsk, ((cgroup_cs(scan->cg))->cpus_allowed));
}
/**
@@ -885,10 +911,10 @@
* @cs: the cpuset to consider
* @buf: buffer of cpu numbers written to this cpuset
*/
-static int update_cpumask(struct cpuset *cs, const char *buf)
+static int update_cpumask(struct cpuset *cs, struct cpuset *trialcs,
+ const char *buf)
{
struct ptr_heap heap;
- struct cpuset trialcs;
int retval;
int is_load_balanced;
@@ -896,8 +922,6 @@
if (cs == &top_cpuset)
return -EACCES;
- trialcs = *cs;
-
/*
* An empty cpus_allowed is ok only if the cpuset has no tasks.
* Since cpulist_parse() fails on an empty mask, we special case
@@ -905,31 +929,31 @@
* with tasks have cpus.
*/
if (!*buf) {
- cpus_clear(trialcs.cpus_allowed);
+ cpumask_clear(trialcs->cpus_allowed);
} else {
- retval = cpulist_parse(buf, &trialcs.cpus_allowed);
+ retval = cpulist_parse(buf, trialcs->cpus_allowed);
if (retval < 0)
return retval;
- if (!cpus_subset(trialcs.cpus_allowed, cpu_online_map))
+ if (!cpumask_subset(trialcs->cpus_allowed, cpu_online_mask))
return -EINVAL;
}
- retval = validate_change(cs, &trialcs);
+ retval = validate_change(cs, trialcs);
if (retval < 0)
return retval;
/* Nothing to do if the cpus didn't change */
- if (cpus_equal(cs->cpus_allowed, trialcs.cpus_allowed))
+ if (cpumask_equal(cs->cpus_allowed, trialcs->cpus_allowed))
return 0;
retval = heap_init(&heap, PAGE_SIZE, GFP_KERNEL, NULL);
if (retval)
return retval;
- is_load_balanced = is_sched_load_balance(&trialcs);
+ is_load_balanced = is_sched_load_balance(trialcs);
mutex_lock(&callback_mutex);
- cs->cpus_allowed = trialcs.cpus_allowed;
+ cpumask_copy(cs->cpus_allowed, trialcs->cpus_allowed);
mutex_unlock(&callback_mutex);
/*
@@ -1017,7 +1041,7 @@
cpuset_being_rebound = cs; /* causes mpol_dup() rebind */
fudge = 10; /* spare mmarray[] slots */
- fudge += cpus_weight(cs->cpus_allowed); /* imagine one fork-bomb/cpu */
+ fudge += cpumask_weight(cs->cpus_allowed);/* imagine 1 fork-bomb/cpu */
retval = -ENOMEM;
/*
@@ -1104,9 +1128,9 @@
* lock each such tasks mm->mmap_sem, scan its vma's and rebind
* their mempolicies to the cpusets new mems_allowed.
*/
-static int update_nodemask(struct cpuset *cs, const char *buf)
+static int update_nodemask(struct cpuset *cs, struct cpuset *trialcs,
+ const char *buf)
{
- struct cpuset trialcs;
nodemask_t oldmem;
int retval;
@@ -1117,8 +1141,6 @@
if (cs == &top_cpuset)
return -EACCES;
- trialcs = *cs;
-
/*
* An empty mems_allowed is ok iff there are no tasks in the cpuset.
* Since nodelist_parse() fails on an empty mask, we special case
@@ -1126,27 +1148,27 @@
* with tasks have memory.
*/
if (!*buf) {
- nodes_clear(trialcs.mems_allowed);
+ nodes_clear(trialcs->mems_allowed);
} else {
- retval = nodelist_parse(buf, trialcs.mems_allowed);
+ retval = nodelist_parse(buf, trialcs->mems_allowed);
if (retval < 0)
goto done;
- if (!nodes_subset(trialcs.mems_allowed,
+ if (!nodes_subset(trialcs->mems_allowed,
node_states[N_HIGH_MEMORY]))
return -EINVAL;
}
oldmem = cs->mems_allowed;
- if (nodes_equal(oldmem, trialcs.mems_allowed)) {
+ if (nodes_equal(oldmem, trialcs->mems_allowed)) {
retval = 0; /* Too easy - nothing to do */
goto done;
}
- retval = validate_change(cs, &trialcs);
+ retval = validate_change(cs, trialcs);
if (retval < 0)
goto done;
mutex_lock(&callback_mutex);
- cs->mems_allowed = trialcs.mems_allowed;
+ cs->mems_allowed = trialcs->mems_allowed;
cs->mems_generation = cpuset_mems_generation++;
mutex_unlock(&callback_mutex);
@@ -1167,7 +1189,8 @@
if (val != cs->relax_domain_level) {
cs->relax_domain_level = val;
- if (!cpus_empty(cs->cpus_allowed) && is_sched_load_balance(cs))
+ if (!cpumask_empty(cs->cpus_allowed) &&
+ is_sched_load_balance(cs))
async_rebuild_sched_domains();
}
@@ -1186,31 +1209,36 @@
static int update_flag(cpuset_flagbits_t bit, struct cpuset *cs,
int turning_on)
{
- struct cpuset trialcs;
+ struct cpuset *trialcs;
int err;
int balance_flag_changed;
- trialcs = *cs;
- if (turning_on)
- set_bit(bit, &trialcs.flags);
- else
- clear_bit(bit, &trialcs.flags);
+ trialcs = alloc_trial_cpuset(cs);
+ if (!trialcs)
+ return -ENOMEM;
- err = validate_change(cs, &trialcs);
+ if (turning_on)
+ set_bit(bit, &trialcs->flags);
+ else
+ clear_bit(bit, &trialcs->flags);
+
+ err = validate_change(cs, trialcs);
if (err < 0)
- return err;
+ goto out;
balance_flag_changed = (is_sched_load_balance(cs) !=
- is_sched_load_balance(&trialcs));
+ is_sched_load_balance(trialcs));
mutex_lock(&callback_mutex);
- cs->flags = trialcs.flags;
+ cs->flags = trialcs->flags;
mutex_unlock(&callback_mutex);
- if (!cpus_empty(trialcs.cpus_allowed) && balance_flag_changed)
+ if (!cpumask_empty(trialcs->cpus_allowed) && balance_flag_changed)
async_rebuild_sched_domains();
- return 0;
+out:
+ free_trial_cpuset(trialcs);
+ return err;
}
/*
@@ -1311,42 +1339,47 @@
return val;
}
+/* Protected by cgroup_lock */
+static cpumask_var_t cpus_attach;
+
/* Called by cgroups to determine if a cpuset is usable; cgroup_mutex held */
static int cpuset_can_attach(struct cgroup_subsys *ss,
struct cgroup *cont, struct task_struct *tsk)
{
struct cpuset *cs = cgroup_cs(cont);
+ int ret = 0;
- if (cpus_empty(cs->cpus_allowed) || nodes_empty(cs->mems_allowed))
+ if (cpumask_empty(cs->cpus_allowed) || nodes_empty(cs->mems_allowed))
return -ENOSPC;
- if (tsk->flags & PF_THREAD_BOUND) {
- cpumask_t mask;
+ if (tsk->flags & PF_THREAD_BOUND) {
mutex_lock(&callback_mutex);
- mask = cs->cpus_allowed;
+ if (!cpumask_equal(&tsk->cpus_allowed, cs->cpus_allowed))
+ ret = -EINVAL;
mutex_unlock(&callback_mutex);
- if (!cpus_equal(tsk->cpus_allowed, mask))
- return -EINVAL;
}
- return security_task_setscheduler(tsk, 0, NULL);
+ return ret < 0 ? ret : security_task_setscheduler(tsk, 0, NULL);
}
static void cpuset_attach(struct cgroup_subsys *ss,
struct cgroup *cont, struct cgroup *oldcont,
struct task_struct *tsk)
{
- cpumask_t cpus;
nodemask_t from, to;
struct mm_struct *mm;
struct cpuset *cs = cgroup_cs(cont);
struct cpuset *oldcs = cgroup_cs(oldcont);
int err;
- mutex_lock(&callback_mutex);
- guarantee_online_cpus(cs, &cpus);
- err = set_cpus_allowed_ptr(tsk, &cpus);
- mutex_unlock(&callback_mutex);
+ if (cs == &top_cpuset) {
+ cpumask_copy(cpus_attach, cpu_possible_mask);
+ } else {
+ mutex_lock(&callback_mutex);
+ guarantee_online_cpus(cs, cpus_attach);
+ mutex_unlock(&callback_mutex);
+ }
+ err = set_cpus_allowed_ptr(tsk, cpus_attach);
if (err)
return;
@@ -1359,7 +1392,6 @@
cpuset_migrate_mm(mm, &from, &to);
mmput(mm);
}
-
}
/* The various types of files and directories in a cpuset file system */
@@ -1454,21 +1486,29 @@
const char *buf)
{
int retval = 0;
+ struct cpuset *cs = cgroup_cs(cgrp);
+ struct cpuset *trialcs;
if (!cgroup_lock_live_group(cgrp))
return -ENODEV;
+ trialcs = alloc_trial_cpuset(cs);
+ if (!trialcs)
+ return -ENOMEM;
+
switch (cft->private) {
case FILE_CPULIST:
- retval = update_cpumask(cgroup_cs(cgrp), buf);
+ retval = update_cpumask(cs, trialcs, buf);
break;
case FILE_MEMLIST:
- retval = update_nodemask(cgroup_cs(cgrp), buf);
+ retval = update_nodemask(cs, trialcs, buf);
break;
default:
retval = -EINVAL;
break;
}
+
+ free_trial_cpuset(trialcs);
cgroup_unlock();
return retval;
}
@@ -1487,13 +1527,13 @@
static int cpuset_sprintf_cpulist(char *page, struct cpuset *cs)
{
- cpumask_t mask;
+ int ret;
mutex_lock(&callback_mutex);
- mask = cs->cpus_allowed;
+ ret = cpulist_scnprintf(page, PAGE_SIZE, cs->cpus_allowed);
mutex_unlock(&callback_mutex);
- return cpulist_scnprintf(page, PAGE_SIZE, &mask);
+ return ret;
}
static int cpuset_sprintf_memlist(char *page, struct cpuset *cs)
@@ -1729,7 +1769,7 @@
parent_cs = cgroup_cs(parent);
cs->mems_allowed = parent_cs->mems_allowed;
- cs->cpus_allowed = parent_cs->cpus_allowed;
+ cpumask_copy(cs->cpus_allowed, parent_cs->cpus_allowed);
return;
}
@@ -1755,6 +1795,10 @@
cs = kmalloc(sizeof(*cs), GFP_KERNEL);
if (!cs)
return ERR_PTR(-ENOMEM);
+ if (!alloc_cpumask_var(&cs->cpus_allowed, GFP_KERNEL)) {
+ kfree(cs);
+ return ERR_PTR(-ENOMEM);
+ }
cpuset_update_task_memory_state();
cs->flags = 0;
@@ -1763,7 +1807,7 @@
if (is_spread_slab(parent))
set_bit(CS_SPREAD_SLAB, &cs->flags);
set_bit(CS_SCHED_LOAD_BALANCE, &cs->flags);
- cpus_clear(cs->cpus_allowed);
+ cpumask_clear(cs->cpus_allowed);
nodes_clear(cs->mems_allowed);
cs->mems_generation = cpuset_mems_generation++;
fmeter_init(&cs->fmeter);
@@ -1790,6 +1834,7 @@
update_flag(CS_SCHED_LOAD_BALANCE, cs, 0);
number_of_cpusets--;
+ free_cpumask_var(cs->cpus_allowed);
kfree(cs);
}
@@ -1813,6 +1858,8 @@
int __init cpuset_init_early(void)
{
+ alloc_bootmem_cpumask_var(&top_cpuset.cpus_allowed);
+
top_cpuset.mems_generation = cpuset_mems_generation++;
return 0;
}
@@ -1828,7 +1875,7 @@
{
int err = 0;
- cpus_setall(top_cpuset.cpus_allowed);
+ cpumask_setall(top_cpuset.cpus_allowed);
nodes_setall(top_cpuset.mems_allowed);
fmeter_init(&top_cpuset.fmeter);
@@ -1840,6 +1887,9 @@
if (err < 0)
return err;
+ if (!alloc_cpumask_var(&cpus_attach, GFP_KERNEL))
+ BUG();
+
number_of_cpusets = 1;
return 0;
}
@@ -1914,7 +1964,7 @@
* has online cpus, so can't be empty).
*/
parent = cs->parent;
- while (cpus_empty(parent->cpus_allowed) ||
+ while (cpumask_empty(parent->cpus_allowed) ||
nodes_empty(parent->mems_allowed))
parent = parent->parent;
@@ -1955,7 +2005,7 @@
}
/* Continue past cpusets with all cpus, mems online */
- if (cpus_subset(cp->cpus_allowed, cpu_online_map) &&
+ if (cpumask_subset(cp->cpus_allowed, cpu_online_mask) &&
nodes_subset(cp->mems_allowed, node_states[N_HIGH_MEMORY]))
continue;
@@ -1963,13 +2013,14 @@
/* Remove offline cpus and mems from this cpuset. */
mutex_lock(&callback_mutex);
- cpus_and(cp->cpus_allowed, cp->cpus_allowed, cpu_online_map);
+ cpumask_and(cp->cpus_allowed, cp->cpus_allowed,
+ cpu_online_mask);
nodes_and(cp->mems_allowed, cp->mems_allowed,
node_states[N_HIGH_MEMORY]);
mutex_unlock(&callback_mutex);
/* Move tasks from the empty cpuset to a parent */
- if (cpus_empty(cp->cpus_allowed) ||
+ if (cpumask_empty(cp->cpus_allowed) ||
nodes_empty(cp->mems_allowed))
remove_tasks_in_empty_cpuset(cp);
else {
@@ -1995,7 +2046,7 @@
unsigned long phase, void *unused_cpu)
{
struct sched_domain_attr *attr;
- cpumask_t *doms;
+ struct cpumask *doms;
int ndoms;
switch (phase) {
@@ -2010,7 +2061,7 @@
}
cgroup_lock();
- top_cpuset.cpus_allowed = cpu_online_map;
+ cpumask_copy(top_cpuset.cpus_allowed, cpu_online_mask);
scan_for_empty_cpusets(&top_cpuset);
ndoms = generate_sched_domains(&doms, &attr);
cgroup_unlock();
@@ -2055,7 +2106,7 @@
void __init cpuset_init_smp(void)
{
- top_cpuset.cpus_allowed = cpu_online_map;
+ cpumask_copy(top_cpuset.cpus_allowed, cpu_online_mask);
top_cpuset.mems_allowed = node_states[N_HIGH_MEMORY];
hotcpu_notifier(cpuset_track_online_cpus, 0);
@@ -2065,15 +2116,15 @@
/**
* cpuset_cpus_allowed - return cpus_allowed mask from a tasks cpuset.
* @tsk: pointer to task_struct from which to obtain cpuset->cpus_allowed.
- * @pmask: pointer to cpumask_t variable to receive cpus_allowed set.
+ * @pmask: pointer to struct cpumask variable to receive cpus_allowed set.
*
- * Description: Returns the cpumask_t cpus_allowed of the cpuset
+ * Description: Returns the cpumask_var_t cpus_allowed of the cpuset
* attached to the specified @tsk. Guaranteed to return some non-empty
* subset of cpu_online_map, even if this means going outside the
* tasks cpuset.
**/
-void cpuset_cpus_allowed(struct task_struct *tsk, cpumask_t *pmask)
+void cpuset_cpus_allowed(struct task_struct *tsk, struct cpumask *pmask)
{
mutex_lock(&callback_mutex);
cpuset_cpus_allowed_locked(tsk, pmask);
@@ -2084,7 +2135,7 @@
* cpuset_cpus_allowed_locked - return cpus_allowed mask from a tasks cpuset.
* Must be called with callback_mutex held.
**/
-void cpuset_cpus_allowed_locked(struct task_struct *tsk, cpumask_t *pmask)
+void cpuset_cpus_allowed_locked(struct task_struct *tsk, struct cpumask *pmask)
{
task_lock(tsk);
guarantee_online_cpus(task_cs(tsk), pmask);
diff --git a/kernel/cred.c b/kernel/cred.c
index ff7bc07..043f78c 100644
--- a/kernel/cred.c
+++ b/kernel/cred.c
@@ -506,6 +506,7 @@
else
old = get_cred(&init_cred);
+ *new = *old;
get_uid(new->user);
get_group_info(new->group_info);
@@ -529,6 +530,7 @@
error:
put_cred(new);
+ put_cred(old);
return NULL;
}
EXPORT_SYMBOL(prepare_kernel_cred);
diff --git a/kernel/fork.c b/kernel/fork.c
index 7b8f2a7..4018308 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -1126,12 +1126,12 @@
if (pid != &init_struct_pid) {
retval = -ENOMEM;
- pid = alloc_pid(task_active_pid_ns(p));
+ pid = alloc_pid(p->nsproxy->pid_ns);
if (!pid)
goto bad_fork_cleanup_io;
if (clone_flags & CLONE_NEWPID) {
- retval = pid_ns_prepare_proc(task_active_pid_ns(p));
+ retval = pid_ns_prepare_proc(p->nsproxy->pid_ns);
if (retval < 0)
goto bad_fork_free_pid;
}
diff --git a/kernel/ns_cgroup.c b/kernel/ns_cgroup.c
index 43c2111..78bc3fd 100644
--- a/kernel/ns_cgroup.c
+++ b/kernel/ns_cgroup.c
@@ -13,7 +13,6 @@
struct ns_cgroup {
struct cgroup_subsys_state css;
- spinlock_t lock;
};
struct cgroup_subsys ns_subsys;
@@ -84,7 +83,6 @@
ns_cgroup = kzalloc(sizeof(*ns_cgroup), GFP_KERNEL);
if (!ns_cgroup)
return ERR_PTR(-ENOMEM);
- spin_lock_init(&ns_cgroup->lock);
return &ns_cgroup->css;
}
diff --git a/kernel/pid.c b/kernel/pid.c
index af9224c..1b3586f 100644
--- a/kernel/pid.c
+++ b/kernel/pid.c
@@ -474,6 +474,12 @@
}
EXPORT_SYMBOL(task_session_nr_ns);
+struct pid_namespace *task_active_pid_ns(struct task_struct *tsk)
+{
+ return ns_of_pid(task_pid(tsk));
+}
+EXPORT_SYMBOL_GPL(task_active_pid_ns);
+
/*
* Used by proc to find the first pid that is greater than or equal to nr.
*
diff --git a/kernel/power/disk.c b/kernel/power/disk.c
index f77d381..45e8541 100644
--- a/kernel/power/disk.c
+++ b/kernel/power/disk.c
@@ -258,12 +258,12 @@
{
int error;
- /* Free memory before shutting down devices. */
- error = swsusp_shrink_memory();
+ error = platform_begin(platform_mode);
if (error)
return error;
- error = platform_begin(platform_mode);
+ /* Free memory before shutting down devices. */
+ error = swsusp_shrink_memory();
if (error)
goto Close;
diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c
index 5d2ab83..f5fc2d7 100644
--- a/kernel/power/snapshot.c
+++ b/kernel/power/snapshot.c
@@ -25,6 +25,7 @@
#include <linux/syscalls.h>
#include <linux/console.h>
#include <linux/highmem.h>
+#include <linux/list.h>
#include <asm/uaccess.h>
#include <asm/mmu_context.h>
@@ -192,12 +193,6 @@
return ret;
}
-static void chain_free(struct chain_allocator *ca, int clear_page_nosave)
-{
- free_list_of_pages(ca->chain, clear_page_nosave);
- memset(ca, 0, sizeof(struct chain_allocator));
-}
-
/**
* Data types related to memory bitmaps.
*
@@ -233,7 +228,7 @@
#define BM_BITS_PER_BLOCK (PAGE_SIZE << 3)
struct bm_block {
- struct bm_block *next; /* next element of the list */
+ struct list_head hook; /* hook into a list of bitmap blocks */
unsigned long start_pfn; /* pfn represented by the first bit */
unsigned long end_pfn; /* pfn represented by the last bit plus 1 */
unsigned long *data; /* bitmap representing pages */
@@ -244,24 +239,15 @@
return bb->end_pfn - bb->start_pfn;
}
-struct zone_bitmap {
- struct zone_bitmap *next; /* next element of the list */
- unsigned long start_pfn; /* minimal pfn in this zone */
- unsigned long end_pfn; /* maximal pfn in this zone plus 1 */
- struct bm_block *bm_blocks; /* list of bitmap blocks */
- struct bm_block *cur_block; /* recently used bitmap block */
-};
-
/* strcut bm_position is used for browsing memory bitmaps */
struct bm_position {
- struct zone_bitmap *zone_bm;
struct bm_block *block;
int bit;
};
struct memory_bitmap {
- struct zone_bitmap *zone_bm_list; /* list of zone bitmaps */
+ struct list_head blocks; /* list of bitmap blocks */
struct linked_page *p_list; /* list of pages used to store zone
* bitmap objects and bitmap block
* objects
@@ -273,11 +259,7 @@
static void memory_bm_position_reset(struct memory_bitmap *bm)
{
- struct zone_bitmap *zone_bm;
-
- zone_bm = bm->zone_bm_list;
- bm->cur.zone_bm = zone_bm;
- bm->cur.block = zone_bm->bm_blocks;
+ bm->cur.block = list_entry(bm->blocks.next, struct bm_block, hook);
bm->cur.bit = 0;
}
@@ -285,151 +267,184 @@
/**
* create_bm_block_list - create a list of block bitmap objects
+ * @nr_blocks - number of blocks to allocate
+ * @list - list to put the allocated blocks into
+ * @ca - chain allocator to be used for allocating memory
*/
-
-static inline struct bm_block *
-create_bm_block_list(unsigned int nr_blocks, struct chain_allocator *ca)
+static int create_bm_block_list(unsigned long pages,
+ struct list_head *list,
+ struct chain_allocator *ca)
{
- struct bm_block *bblist = NULL;
+ unsigned int nr_blocks = DIV_ROUND_UP(pages, BM_BITS_PER_BLOCK);
while (nr_blocks-- > 0) {
struct bm_block *bb;
bb = chain_alloc(ca, sizeof(struct bm_block));
if (!bb)
- return NULL;
-
- bb->next = bblist;
- bblist = bb;
+ return -ENOMEM;
+ list_add(&bb->hook, list);
}
- return bblist;
+
+ return 0;
+}
+
+struct mem_extent {
+ struct list_head hook;
+ unsigned long start;
+ unsigned long end;
+};
+
+/**
+ * free_mem_extents - free a list of memory extents
+ * @list - list of extents to empty
+ */
+static void free_mem_extents(struct list_head *list)
+{
+ struct mem_extent *ext, *aux;
+
+ list_for_each_entry_safe(ext, aux, list, hook) {
+ list_del(&ext->hook);
+ kfree(ext);
+ }
}
/**
- * create_zone_bm_list - create a list of zone bitmap objects
+ * create_mem_extents - create a list of memory extents representing
+ * contiguous ranges of PFNs
+ * @list - list to put the extents into
+ * @gfp_mask - mask to use for memory allocations
*/
-
-static inline struct zone_bitmap *
-create_zone_bm_list(unsigned int nr_zones, struct chain_allocator *ca)
+static int create_mem_extents(struct list_head *list, gfp_t gfp_mask)
{
- struct zone_bitmap *zbmlist = NULL;
+ struct zone *zone;
- while (nr_zones-- > 0) {
- struct zone_bitmap *zbm;
+ INIT_LIST_HEAD(list);
- zbm = chain_alloc(ca, sizeof(struct zone_bitmap));
- if (!zbm)
- return NULL;
+ for_each_zone(zone) {
+ unsigned long zone_start, zone_end;
+ struct mem_extent *ext, *cur, *aux;
- zbm->next = zbmlist;
- zbmlist = zbm;
+ if (!populated_zone(zone))
+ continue;
+
+ zone_start = zone->zone_start_pfn;
+ zone_end = zone->zone_start_pfn + zone->spanned_pages;
+
+ list_for_each_entry(ext, list, hook)
+ if (zone_start <= ext->end)
+ break;
+
+ if (&ext->hook == list || zone_end < ext->start) {
+ /* New extent is necessary */
+ struct mem_extent *new_ext;
+
+ new_ext = kzalloc(sizeof(struct mem_extent), gfp_mask);
+ if (!new_ext) {
+ free_mem_extents(list);
+ return -ENOMEM;
+ }
+ new_ext->start = zone_start;
+ new_ext->end = zone_end;
+ list_add_tail(&new_ext->hook, &ext->hook);
+ continue;
+ }
+
+ /* Merge this zone's range of PFNs with the existing one */
+ if (zone_start < ext->start)
+ ext->start = zone_start;
+ if (zone_end > ext->end)
+ ext->end = zone_end;
+
+ /* More merging may be possible */
+ cur = ext;
+ list_for_each_entry_safe_continue(cur, aux, list, hook) {
+ if (zone_end < cur->start)
+ break;
+ if (zone_end < cur->end)
+ ext->end = cur->end;
+ list_del(&cur->hook);
+ kfree(cur);
+ }
}
- return zbmlist;
+
+ return 0;
}
/**
* memory_bm_create - allocate memory for a memory bitmap
*/
-
static int
memory_bm_create(struct memory_bitmap *bm, gfp_t gfp_mask, int safe_needed)
{
struct chain_allocator ca;
- struct zone *zone;
- struct zone_bitmap *zone_bm;
- struct bm_block *bb;
- unsigned int nr;
+ struct list_head mem_extents;
+ struct mem_extent *ext;
+ int error;
chain_init(&ca, gfp_mask, safe_needed);
+ INIT_LIST_HEAD(&bm->blocks);
- /* Compute the number of zones */
- nr = 0;
- for_each_zone(zone)
- if (populated_zone(zone))
- nr++;
+ error = create_mem_extents(&mem_extents, gfp_mask);
+ if (error)
+ return error;
- /* Allocate the list of zones bitmap objects */
- zone_bm = create_zone_bm_list(nr, &ca);
- bm->zone_bm_list = zone_bm;
- if (!zone_bm) {
- chain_free(&ca, PG_UNSAFE_CLEAR);
- return -ENOMEM;
- }
+ list_for_each_entry(ext, &mem_extents, hook) {
+ struct bm_block *bb;
+ unsigned long pfn = ext->start;
+ unsigned long pages = ext->end - ext->start;
- /* Initialize the zone bitmap objects */
- for_each_zone(zone) {
- unsigned long pfn;
+ bb = list_entry(bm->blocks.prev, struct bm_block, hook);
- if (!populated_zone(zone))
- continue;
+ error = create_bm_block_list(pages, bm->blocks.prev, &ca);
+ if (error)
+ goto Error;
- zone_bm->start_pfn = zone->zone_start_pfn;
- zone_bm->end_pfn = zone->zone_start_pfn + zone->spanned_pages;
- /* Allocate the list of bitmap block objects */
- nr = DIV_ROUND_UP(zone->spanned_pages, BM_BITS_PER_BLOCK);
- bb = create_bm_block_list(nr, &ca);
- zone_bm->bm_blocks = bb;
- zone_bm->cur_block = bb;
- if (!bb)
- goto Free;
-
- nr = zone->spanned_pages;
- pfn = zone->zone_start_pfn;
- /* Initialize the bitmap block objects */
- while (bb) {
- unsigned long *ptr;
-
- ptr = get_image_page(gfp_mask, safe_needed);
- bb->data = ptr;
- if (!ptr)
- goto Free;
+ list_for_each_entry_continue(bb, &bm->blocks, hook) {
+ bb->data = get_image_page(gfp_mask, safe_needed);
+ if (!bb->data) {
+ error = -ENOMEM;
+ goto Error;
+ }
bb->start_pfn = pfn;
- if (nr >= BM_BITS_PER_BLOCK) {
+ if (pages >= BM_BITS_PER_BLOCK) {
pfn += BM_BITS_PER_BLOCK;
- nr -= BM_BITS_PER_BLOCK;
+ pages -= BM_BITS_PER_BLOCK;
} else {
/* This is executed only once in the loop */
- pfn += nr;
+ pfn += pages;
}
bb->end_pfn = pfn;
- bb = bb->next;
}
- zone_bm = zone_bm->next;
}
+
bm->p_list = ca.chain;
memory_bm_position_reset(bm);
- return 0;
+ Exit:
+ free_mem_extents(&mem_extents);
+ return error;
- Free:
+ Error:
bm->p_list = ca.chain;
memory_bm_free(bm, PG_UNSAFE_CLEAR);
- return -ENOMEM;
+ goto Exit;
}
/**
* memory_bm_free - free memory occupied by the memory bitmap @bm
*/
-
static void memory_bm_free(struct memory_bitmap *bm, int clear_nosave_free)
{
- struct zone_bitmap *zone_bm;
+ struct bm_block *bb;
- /* Free the list of bit blocks for each zone_bitmap object */
- zone_bm = bm->zone_bm_list;
- while (zone_bm) {
- struct bm_block *bb;
+ list_for_each_entry(bb, &bm->blocks, hook)
+ if (bb->data)
+ free_image_page(bb->data, clear_nosave_free);
- bb = zone_bm->bm_blocks;
- while (bb) {
- if (bb->data)
- free_image_page(bb->data, clear_nosave_free);
- bb = bb->next;
- }
- zone_bm = zone_bm->next;
- }
free_list_of_pages(bm->p_list, clear_nosave_free);
- bm->zone_bm_list = NULL;
+
+ INIT_LIST_HEAD(&bm->blocks);
}
/**
@@ -437,38 +452,33 @@
* to given pfn. The cur_zone_bm member of @bm and the cur_block member
* of @bm->cur_zone_bm are updated.
*/
-
static int memory_bm_find_bit(struct memory_bitmap *bm, unsigned long pfn,
void **addr, unsigned int *bit_nr)
{
- struct zone_bitmap *zone_bm;
struct bm_block *bb;
- /* Check if the pfn is from the current zone */
- zone_bm = bm->cur.zone_bm;
- if (pfn < zone_bm->start_pfn || pfn >= zone_bm->end_pfn) {
- zone_bm = bm->zone_bm_list;
- /* We don't assume that the zones are sorted by pfns */
- while (pfn < zone_bm->start_pfn || pfn >= zone_bm->end_pfn) {
- zone_bm = zone_bm->next;
-
- if (!zone_bm)
- return -EFAULT;
- }
- bm->cur.zone_bm = zone_bm;
- }
- /* Check if the pfn corresponds to the current bitmap block */
- bb = zone_bm->cur_block;
+ /*
+ * Check if the pfn corresponds to the current bitmap block and find
+ * the block where it fits if this is not the case.
+ */
+ bb = bm->cur.block;
if (pfn < bb->start_pfn)
- bb = zone_bm->bm_blocks;
+ list_for_each_entry_continue_reverse(bb, &bm->blocks, hook)
+ if (pfn >= bb->start_pfn)
+ break;
- while (pfn >= bb->end_pfn) {
- bb = bb->next;
+ if (pfn >= bb->end_pfn)
+ list_for_each_entry_continue(bb, &bm->blocks, hook)
+ if (pfn >= bb->start_pfn && pfn < bb->end_pfn)
+ break;
- BUG_ON(!bb);
- }
- zone_bm->cur_block = bb;
+ if (&bb->hook == &bm->blocks)
+ return -EFAULT;
+
+ /* The block has been found */
+ bm->cur.block = bb;
pfn -= bb->start_pfn;
+ bm->cur.bit = pfn + 1;
*bit_nr = pfn;
*addr = bb->data;
return 0;
@@ -519,6 +529,14 @@
return test_bit(bit, addr);
}
+static bool memory_bm_pfn_present(struct memory_bitmap *bm, unsigned long pfn)
+{
+ void *addr;
+ unsigned int bit;
+
+ return !memory_bm_find_bit(bm, pfn, &addr, &bit);
+}
+
/**
* memory_bm_next_pfn - find the pfn that corresponds to the next set bit
* in the bitmap @bm. If the pfn cannot be found, BM_END_OF_MAP is
@@ -530,29 +548,21 @@
static unsigned long memory_bm_next_pfn(struct memory_bitmap *bm)
{
- struct zone_bitmap *zone_bm;
struct bm_block *bb;
int bit;
+ bb = bm->cur.block;
do {
- bb = bm->cur.block;
- do {
- bit = bm->cur.bit;
- bit = find_next_bit(bb->data, bm_block_bits(bb), bit);
- if (bit < bm_block_bits(bb))
- goto Return_pfn;
+ bit = bm->cur.bit;
+ bit = find_next_bit(bb->data, bm_block_bits(bb), bit);
+ if (bit < bm_block_bits(bb))
+ goto Return_pfn;
- bb = bb->next;
- bm->cur.block = bb;
- bm->cur.bit = 0;
- } while (bb);
- zone_bm = bm->cur.zone_bm->next;
- if (zone_bm) {
- bm->cur.zone_bm = zone_bm;
- bm->cur.block = zone_bm->bm_blocks;
- bm->cur.bit = 0;
- }
- } while (zone_bm);
+ bb = list_entry(bb->hook.next, struct bm_block, hook);
+ bm->cur.block = bb;
+ bm->cur.bit = 0;
+ } while (&bb->hook != &bm->blocks);
+
memory_bm_position_reset(bm);
return BM_END_OF_MAP;
@@ -808,8 +818,7 @@
* We should save the page if it isn't Nosave or NosaveFree, or Reserved,
* and it isn't a part of a free chunk of pages.
*/
-
-static struct page *saveable_highmem_page(unsigned long pfn)
+static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn)
{
struct page *page;
@@ -817,6 +826,8 @@
return NULL;
page = pfn_to_page(pfn);
+ if (page_zone(page) != zone)
+ return NULL;
BUG_ON(!PageHighMem(page));
@@ -846,13 +857,16 @@
mark_free_pages(zone);
max_zone_pfn = zone->zone_start_pfn + zone->spanned_pages;
for (pfn = zone->zone_start_pfn; pfn < max_zone_pfn; pfn++)
- if (saveable_highmem_page(pfn))
+ if (saveable_highmem_page(zone, pfn))
n++;
}
return n;
}
#else
-static inline void *saveable_highmem_page(unsigned long pfn) { return NULL; }
+static inline void *saveable_highmem_page(struct zone *z, unsigned long p)
+{
+ return NULL;
+}
#endif /* CONFIG_HIGHMEM */
/**
@@ -863,8 +877,7 @@
* of pages statically defined as 'unsaveable', and it isn't a part of
* a free chunk of pages.
*/
-
-static struct page *saveable_page(unsigned long pfn)
+static struct page *saveable_page(struct zone *zone, unsigned long pfn)
{
struct page *page;
@@ -872,6 +885,8 @@
return NULL;
page = pfn_to_page(pfn);
+ if (page_zone(page) != zone)
+ return NULL;
BUG_ON(PageHighMem(page));
@@ -903,7 +918,7 @@
mark_free_pages(zone);
max_zone_pfn = zone->zone_start_pfn + zone->spanned_pages;
for (pfn = zone->zone_start_pfn; pfn < max_zone_pfn; pfn++)
- if(saveable_page(pfn))
+ if (saveable_page(zone, pfn))
n++;
}
return n;
@@ -944,7 +959,7 @@
page_is_saveable(struct zone *zone, unsigned long pfn)
{
return is_highmem(zone) ?
- saveable_highmem_page(pfn) : saveable_page(pfn);
+ saveable_highmem_page(zone, pfn) : saveable_page(zone, pfn);
}
static void copy_data_page(unsigned long dst_pfn, unsigned long src_pfn)
@@ -966,7 +981,7 @@
* data modified by kmap_atomic()
*/
safe_copy_page(buffer, s_page);
- dst = kmap_atomic(pfn_to_page(dst_pfn), KM_USER0);
+ dst = kmap_atomic(d_page, KM_USER0);
memcpy(dst, buffer, PAGE_SIZE);
kunmap_atomic(dst, KM_USER0);
} else {
@@ -975,7 +990,7 @@
}
}
#else
-#define page_is_saveable(zone, pfn) saveable_page(pfn)
+#define page_is_saveable(zone, pfn) saveable_page(zone, pfn)
static inline void copy_data_page(unsigned long dst_pfn, unsigned long src_pfn)
{
@@ -1459,9 +1474,7 @@
* unpack_orig_pfns - for each element of @buf[] (1 page at a time) set
* the corresponding bit in the memory bitmap @bm
*/
-
-static inline void
-unpack_orig_pfns(unsigned long *buf, struct memory_bitmap *bm)
+static int unpack_orig_pfns(unsigned long *buf, struct memory_bitmap *bm)
{
int j;
@@ -1469,8 +1482,13 @@
if (unlikely(buf[j] == BM_END_OF_MAP))
break;
- memory_bm_set_bit(bm, buf[j]);
+ if (memory_bm_pfn_present(bm, buf[j]))
+ memory_bm_set_bit(bm, buf[j]);
+ else
+ return -EFAULT;
}
+
+ return 0;
}
/* List of "safe" pages that may be used to store data loaded from the suspend
@@ -1608,7 +1626,7 @@
pbe = chain_alloc(ca, sizeof(struct highmem_pbe));
if (!pbe) {
swsusp_free();
- return NULL;
+ return ERR_PTR(-ENOMEM);
}
pbe->orig_page = page;
if (safe_highmem_pages > 0) {
@@ -1677,7 +1695,7 @@
static inline void *
get_highmem_page_buffer(struct page *page, struct chain_allocator *ca)
{
- return NULL;
+ return ERR_PTR(-EINVAL);
}
static inline void copy_last_highmem_page(void) {}
@@ -1788,8 +1806,13 @@
static void *get_buffer(struct memory_bitmap *bm, struct chain_allocator *ca)
{
struct pbe *pbe;
- struct page *page = pfn_to_page(memory_bm_next_pfn(bm));
+ struct page *page;
+ unsigned long pfn = memory_bm_next_pfn(bm);
+ if (pfn == BM_END_OF_MAP)
+ return ERR_PTR(-EFAULT);
+
+ page = pfn_to_page(pfn);
if (PageHighMem(page))
return get_highmem_page_buffer(page, ca);
@@ -1805,7 +1828,7 @@
pbe = chain_alloc(ca, sizeof(struct pbe));
if (!pbe) {
swsusp_free();
- return NULL;
+ return ERR_PTR(-ENOMEM);
}
pbe->orig_address = page_address(page);
pbe->address = safe_pages_list;
@@ -1868,7 +1891,10 @@
return error;
} else if (handle->prev <= nr_meta_pages) {
- unpack_orig_pfns(buffer, ©_bm);
+ error = unpack_orig_pfns(buffer, ©_bm);
+ if (error)
+ return error;
+
if (handle->prev == nr_meta_pages) {
error = prepare_image(&orig_bm, ©_bm);
if (error)
@@ -1879,12 +1905,14 @@
restore_pblist = NULL;
handle->buffer = get_buffer(&orig_bm, &ca);
handle->sync_read = 0;
- if (!handle->buffer)
- return -ENOMEM;
+ if (IS_ERR(handle->buffer))
+ return PTR_ERR(handle->buffer);
}
} else {
copy_last_highmem_page();
handle->buffer = get_buffer(&orig_bm, &ca);
+ if (IS_ERR(handle->buffer))
+ return PTR_ERR(handle->buffer);
if (handle->buffer != buffer)
handle->sync_read = 0;
}
diff --git a/kernel/power/swsusp.c b/kernel/power/swsusp.c
index 023ff2a..a92c914 100644
--- a/kernel/power/swsusp.c
+++ b/kernel/power/swsusp.c
@@ -262,3 +262,125 @@
return 0;
}
+
+/*
+ * Platforms, like ACPI, may want us to save some memory used by them during
+ * hibernation and to restore the contents of this memory during the subsequent
+ * resume. The code below implements a mechanism allowing us to do that.
+ */
+
+struct nvs_page {
+ unsigned long phys_start;
+ unsigned int size;
+ void *kaddr;
+ void *data;
+ struct list_head node;
+};
+
+static LIST_HEAD(nvs_list);
+
+/**
+ * hibernate_nvs_register - register platform NVS memory region to save
+ * @start - physical address of the region
+ * @size - size of the region
+ *
+ * The NVS region need not be page-aligned (both ends) and we arrange
+ * things so that the data from page-aligned addresses in this region will
+ * be copied into separate RAM pages.
+ */
+int hibernate_nvs_register(unsigned long start, unsigned long size)
+{
+ struct nvs_page *entry, *next;
+
+ while (size > 0) {
+ unsigned int nr_bytes;
+
+ entry = kzalloc(sizeof(struct nvs_page), GFP_KERNEL);
+ if (!entry)
+ goto Error;
+
+ list_add_tail(&entry->node, &nvs_list);
+ entry->phys_start = start;
+ nr_bytes = PAGE_SIZE - (start & ~PAGE_MASK);
+ entry->size = (size < nr_bytes) ? size : nr_bytes;
+
+ start += entry->size;
+ size -= entry->size;
+ }
+ return 0;
+
+ Error:
+ list_for_each_entry_safe(entry, next, &nvs_list, node) {
+ list_del(&entry->node);
+ kfree(entry);
+ }
+ return -ENOMEM;
+}
+
+/**
+ * hibernate_nvs_free - free data pages allocated for saving NVS regions
+ */
+void hibernate_nvs_free(void)
+{
+ struct nvs_page *entry;
+
+ list_for_each_entry(entry, &nvs_list, node)
+ if (entry->data) {
+ free_page((unsigned long)entry->data);
+ entry->data = NULL;
+ if (entry->kaddr) {
+ iounmap(entry->kaddr);
+ entry->kaddr = NULL;
+ }
+ }
+}
+
+/**
+ * hibernate_nvs_alloc - allocate memory necessary for saving NVS regions
+ */
+int hibernate_nvs_alloc(void)
+{
+ struct nvs_page *entry;
+
+ list_for_each_entry(entry, &nvs_list, node) {
+ entry->data = (void *)__get_free_page(GFP_KERNEL);
+ if (!entry->data) {
+ hibernate_nvs_free();
+ return -ENOMEM;
+ }
+ }
+ return 0;
+}
+
+/**
+ * hibernate_nvs_save - save NVS memory regions
+ */
+void hibernate_nvs_save(void)
+{
+ struct nvs_page *entry;
+
+ printk(KERN_INFO "PM: Saving platform NVS memory\n");
+
+ list_for_each_entry(entry, &nvs_list, node)
+ if (entry->data) {
+ entry->kaddr = ioremap(entry->phys_start, entry->size);
+ memcpy(entry->data, entry->kaddr, entry->size);
+ }
+}
+
+/**
+ * hibernate_nvs_restore - restore NVS memory regions
+ *
+ * This function is going to be called with interrupts disabled, so it
+ * cannot iounmap the virtual addresses used to access the NVS region.
+ */
+void hibernate_nvs_restore(void)
+{
+ struct nvs_page *entry;
+
+ printk(KERN_INFO "PM: Restoring platform NVS memory\n");
+
+ list_for_each_entry(entry, &nvs_list, node)
+ if (entry->data)
+ memcpy(entry->kaddr, entry->data, entry->size);
+}
diff --git a/kernel/res_counter.c b/kernel/res_counter.c
index f275c8e..bf8e753 100644
--- a/kernel/res_counter.c
+++ b/kernel/res_counter.c
@@ -15,10 +15,11 @@
#include <linux/uaccess.h>
#include <linux/mm.h>
-void res_counter_init(struct res_counter *counter)
+void res_counter_init(struct res_counter *counter, struct res_counter *parent)
{
spin_lock_init(&counter->lock);
counter->limit = (unsigned long long)LLONG_MAX;
+ counter->parent = parent;
}
int res_counter_charge_locked(struct res_counter *counter, unsigned long val)
@@ -34,14 +35,34 @@
return 0;
}
-int res_counter_charge(struct res_counter *counter, unsigned long val)
+int res_counter_charge(struct res_counter *counter, unsigned long val,
+ struct res_counter **limit_fail_at)
{
int ret;
unsigned long flags;
+ struct res_counter *c, *u;
- spin_lock_irqsave(&counter->lock, flags);
- ret = res_counter_charge_locked(counter, val);
- spin_unlock_irqrestore(&counter->lock, flags);
+ *limit_fail_at = NULL;
+ local_irq_save(flags);
+ for (c = counter; c != NULL; c = c->parent) {
+ spin_lock(&c->lock);
+ ret = res_counter_charge_locked(c, val);
+ spin_unlock(&c->lock);
+ if (ret < 0) {
+ *limit_fail_at = c;
+ goto undo;
+ }
+ }
+ ret = 0;
+ goto done;
+undo:
+ for (u = counter; u != c; u = u->parent) {
+ spin_lock(&u->lock);
+ res_counter_uncharge_locked(u, val);
+ spin_unlock(&u->lock);
+ }
+done:
+ local_irq_restore(flags);
return ret;
}
@@ -56,10 +77,15 @@
void res_counter_uncharge(struct res_counter *counter, unsigned long val)
{
unsigned long flags;
+ struct res_counter *c;
- spin_lock_irqsave(&counter->lock, flags);
- res_counter_uncharge_locked(counter, val);
- spin_unlock_irqrestore(&counter->lock, flags);
+ local_irq_save(flags);
+ for (c = counter; c != NULL; c = c->parent) {
+ spin_lock(&c->lock);
+ res_counter_uncharge_locked(c, val);
+ spin_unlock(&c->lock);
+ }
+ local_irq_restore(flags);
}
diff --git a/kernel/sched_fair.c b/kernel/sched_fair.c
index e0c0b4b..8e1352c 100644
--- a/kernel/sched_fair.c
+++ b/kernel/sched_fair.c
@@ -1617,8 +1617,6 @@
}
}
-#define swap(a, b) do { typeof(a) tmp = (a); (a) = (b); (b) = tmp; } while (0)
-
/*
* Share the fairness runtime between parent and child, thus the
* total amount of pressure for CPU stays equal - new tasks
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index a9d9760..8b0daf0 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -168,7 +168,13 @@
*/
unsigned ring_buffer_event_length(struct ring_buffer_event *event)
{
- return rb_event_length(event);
+ unsigned length = rb_event_length(event);
+ if (event->type != RINGBUF_TYPE_DATA)
+ return length;
+ length -= RB_EVNT_HDR_SIZE;
+ if (length > RB_MAX_SMALL_DATA + sizeof(event->array[0]))
+ length -= sizeof(event->array[0]);
+ return length;
}
EXPORT_SYMBOL_GPL(ring_buffer_event_length);
diff --git a/lib/sort.c b/lib/sort.c
index 6abbaf3..926d004 100644
--- a/lib/sort.c
+++ b/lib/sort.c
@@ -32,11 +32,11 @@
* @base: pointer to data to sort
* @num: number of elements
* @size: size of each element
- * @cmp: pointer to comparison function
- * @swap: pointer to swap function or NULL
+ * @cmp_func: pointer to comparison function
+ * @swap_func: pointer to swap function or NULL
*
* This function does a heapsort on the given array. You may provide a
- * swap function optimized to your element type.
+ * swap_func function optimized to your element type.
*
* Sorting time is O(n log n) both on average and worst-case. While
* qsort is about 20% faster on average, it suffers from exploitable
@@ -45,37 +45,39 @@
*/
void sort(void *base, size_t num, size_t size,
- int (*cmp)(const void *, const void *),
- void (*swap)(void *, void *, int size))
+ int (*cmp_func)(const void *, const void *),
+ void (*swap_func)(void *, void *, int size))
{
/* pre-scale counters for performance */
int i = (num/2 - 1) * size, n = num * size, c, r;
- if (!swap)
- swap = (size == 4 ? u32_swap : generic_swap);
+ if (!swap_func)
+ swap_func = (size == 4 ? u32_swap : generic_swap);
/* heapify */
for ( ; i >= 0; i -= size) {
for (r = i; r * 2 + size < n; r = c) {
c = r * 2 + size;
- if (c < n - size && cmp(base + c, base + c + size) < 0)
+ if (c < n - size &&
+ cmp_func(base + c, base + c + size) < 0)
c += size;
- if (cmp(base + r, base + c) >= 0)
+ if (cmp_func(base + r, base + c) >= 0)
break;
- swap(base + r, base + c, size);
+ swap_func(base + r, base + c, size);
}
}
/* sort */
for (i = n - size; i > 0; i -= size) {
- swap(base, base + i, size);
+ swap_func(base, base + i, size);
for (r = 0; r * 2 + size < i; r = c) {
c = r * 2 + size;
- if (c < i - size && cmp(base + c, base + c + size) < 0)
+ if (c < i - size &&
+ cmp_func(base + c, base + c + size) < 0)
c += size;
- if (cmp(base + r, base + c) >= 0)
+ if (cmp_func(base + r, base + c) >= 0)
break;
- swap(base + r, base + c, size);
+ swap_func(base + r, base + c, size);
}
}
}
diff --git a/mm/filemap.c b/mm/filemap.c
index 2f55a1e..ceba0bd 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -460,7 +460,7 @@
VM_BUG_ON(!PageLocked(page));
error = mem_cgroup_cache_charge(page, current->mm,
- gfp_mask & ~__GFP_HIGHMEM);
+ gfp_mask & GFP_RECLAIM_MASK);
if (error)
goto out;
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 51ee965..e2996b8 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -21,11 +21,13 @@
#include <linux/memcontrol.h>
#include <linux/cgroup.h>
#include <linux/mm.h>
+#include <linux/pagemap.h>
#include <linux/smp.h>
#include <linux/page-flags.h>
#include <linux/backing-dev.h>
#include <linux/bit_spinlock.h>
#include <linux/rcupdate.h>
+#include <linux/mutex.h>
#include <linux/slab.h>
#include <linux/swap.h>
#include <linux/spinlock.h>
@@ -34,12 +36,23 @@
#include <linux/vmalloc.h>
#include <linux/mm_inline.h>
#include <linux/page_cgroup.h>
+#include "internal.h"
#include <asm/uaccess.h>
struct cgroup_subsys mem_cgroup_subsys __read_mostly;
#define MEM_CGROUP_RECLAIM_RETRIES 5
+#ifdef CONFIG_CGROUP_MEM_RES_CTLR_SWAP
+/* Turned on only when memory cgroup is enabled && really_do_swap_account = 0 */
+int do_swap_account __read_mostly;
+static int really_do_swap_account __initdata = 1; /* for remember boot option*/
+#else
+#define do_swap_account (0)
+#endif
+
+static DEFINE_MUTEX(memcg_tasklist); /* can be hold under cgroup_mutex */
+
/*
* Statistics for memory cgroup.
*/
@@ -60,7 +73,7 @@
} ____cacheline_aligned_in_smp;
struct mem_cgroup_stat {
- struct mem_cgroup_stat_cpu cpustat[NR_CPUS];
+ struct mem_cgroup_stat_cpu cpustat[0];
};
/*
@@ -89,9 +102,10 @@
/*
* spin_lock to protect the per cgroup LRU
*/
- spinlock_t lru_lock;
struct list_head lists[NR_LRU_LISTS];
unsigned long count[NR_LRU_LISTS];
+
+ struct zone_reclaim_stat reclaim_stat;
};
/* Macro for accessing counter */
#define MEM_CGROUP_ZSTAT(mz, idx) ((mz)->count[(idx)])
@@ -122,44 +136,73 @@
*/
struct res_counter res;
/*
+ * the counter to account for mem+swap usage.
+ */
+ struct res_counter memsw;
+ /*
* Per cgroup active and inactive list, similar to the
* per zone LRU lists.
*/
struct mem_cgroup_lru_info info;
- int prev_priority; /* for recording reclaim priority */
/*
- * statistics.
+ protect against reclaim related member.
+ */
+ spinlock_t reclaim_param_lock;
+
+ int prev_priority; /* for recording reclaim priority */
+
+ /*
+ * While reclaiming in a hiearchy, we cache the last child we
+ * reclaimed from. Protected by hierarchy_mutex
+ */
+ struct mem_cgroup *last_scanned_child;
+ /*
+ * Should the accounting and control be hierarchical, per subtree?
+ */
+ bool use_hierarchy;
+ unsigned long last_oom_jiffies;
+ atomic_t refcnt;
+
+ unsigned int swappiness;
+
+ /*
+ * statistics. This must be placed at the end of memcg.
*/
struct mem_cgroup_stat stat;
};
-static struct mem_cgroup init_mem_cgroup;
enum charge_type {
MEM_CGROUP_CHARGE_TYPE_CACHE = 0,
MEM_CGROUP_CHARGE_TYPE_MAPPED,
MEM_CGROUP_CHARGE_TYPE_SHMEM, /* used by page migration of shmem */
MEM_CGROUP_CHARGE_TYPE_FORCE, /* used by force_empty */
+ MEM_CGROUP_CHARGE_TYPE_SWAPOUT, /* for accounting swapcache */
NR_CHARGE_TYPE,
};
/* only for here (for easy reading.) */
#define PCGF_CACHE (1UL << PCG_CACHE)
#define PCGF_USED (1UL << PCG_USED)
-#define PCGF_ACTIVE (1UL << PCG_ACTIVE)
#define PCGF_LOCK (1UL << PCG_LOCK)
-#define PCGF_FILE (1UL << PCG_FILE)
static const unsigned long
pcg_default_flags[NR_CHARGE_TYPE] = {
- PCGF_CACHE | PCGF_FILE | PCGF_USED | PCGF_LOCK, /* File Cache */
- PCGF_ACTIVE | PCGF_USED | PCGF_LOCK, /* Anon */
- PCGF_ACTIVE | PCGF_CACHE | PCGF_USED | PCGF_LOCK, /* Shmem */
+ PCGF_CACHE | PCGF_USED | PCGF_LOCK, /* File Cache */
+ PCGF_USED | PCGF_LOCK, /* Anon */
+ PCGF_CACHE | PCGF_USED | PCGF_LOCK, /* Shmem */
0, /* FORCE */
};
-/*
- * Always modified under lru lock. Then, not necessary to preempt_disable()
- */
+/* for encoding cft->private value on file */
+#define _MEM (0)
+#define _MEMSWAP (1)
+#define MEMFILE_PRIVATE(x, val) (((x) << 16) | (val))
+#define MEMFILE_TYPE(val) (((val) >> 16) & 0xffff)
+#define MEMFILE_ATTR(val) ((val) & 0xffff)
+
+static void mem_cgroup_get(struct mem_cgroup *mem);
+static void mem_cgroup_put(struct mem_cgroup *mem);
+
static void mem_cgroup_charge_statistics(struct mem_cgroup *mem,
struct page_cgroup *pc,
bool charge)
@@ -167,10 +210,9 @@
int val = (charge)? 1 : -1;
struct mem_cgroup_stat *stat = &mem->stat;
struct mem_cgroup_stat_cpu *cpustat;
+ int cpu = get_cpu();
- VM_BUG_ON(!irqs_disabled());
-
- cpustat = &stat->cpustat[smp_processor_id()];
+ cpustat = &stat->cpustat[cpu];
if (PageCgroupCache(pc))
__mem_cgroup_stat_add_safe(cpustat, MEM_CGROUP_STAT_CACHE, val);
else
@@ -182,6 +224,7 @@
else
__mem_cgroup_stat_add_safe(cpustat,
MEM_CGROUP_STAT_PGPGOUT_COUNT, 1);
+ put_cpu();
}
static struct mem_cgroup_per_zone *
@@ -197,6 +240,9 @@
int nid = page_cgroup_nid(pc);
int zid = page_cgroup_zid(pc);
+ if (!mem)
+ return NULL;
+
return mem_cgroup_zoneinfo(mem, nid, zid);
}
@@ -236,79 +282,154 @@
struct mem_cgroup, css);
}
-static void __mem_cgroup_remove_list(struct mem_cgroup_per_zone *mz,
- struct page_cgroup *pc)
+static struct mem_cgroup *try_get_mem_cgroup_from_mm(struct mm_struct *mm)
{
- int lru = LRU_BASE;
+ struct mem_cgroup *mem = NULL;
+ /*
+ * Because we have no locks, mm->owner's may be being moved to other
+ * cgroup. We use css_tryget() here even if this looks
+ * pessimistic (rather than adding locks here).
+ */
+ rcu_read_lock();
+ do {
+ mem = mem_cgroup_from_task(rcu_dereference(mm->owner));
+ if (unlikely(!mem))
+ break;
+ } while (!css_tryget(&mem->css));
+ rcu_read_unlock();
+ return mem;
+}
- if (PageCgroupUnevictable(pc))
- lru = LRU_UNEVICTABLE;
- else {
- if (PageCgroupActive(pc))
- lru += LRU_ACTIVE;
- if (PageCgroupFile(pc))
- lru += LRU_FILE;
- }
+static bool mem_cgroup_is_obsolete(struct mem_cgroup *mem)
+{
+ if (!mem)
+ return true;
+ return css_is_removed(&mem->css);
+}
+/*
+ * Following LRU functions are allowed to be used without PCG_LOCK.
+ * Operations are called by routine of global LRU independently from memcg.
+ * What we have to take care of here is validness of pc->mem_cgroup.
+ *
+ * Changes to pc->mem_cgroup happens when
+ * 1. charge
+ * 2. moving account
+ * In typical case, "charge" is done before add-to-lru. Exception is SwapCache.
+ * It is added to LRU before charge.
+ * If PCG_USED bit is not set, page_cgroup is not added to this private LRU.
+ * When moving account, the page is not on LRU. It's isolated.
+ */
+
+void mem_cgroup_del_lru_list(struct page *page, enum lru_list lru)
+{
+ struct page_cgroup *pc;
+ struct mem_cgroup *mem;
+ struct mem_cgroup_per_zone *mz;
+
+ if (mem_cgroup_disabled())
+ return;
+ pc = lookup_page_cgroup(page);
+ /* can happen while we handle swapcache. */
+ if (list_empty(&pc->lru) || !pc->mem_cgroup)
+ return;
+ /*
+ * We don't check PCG_USED bit. It's cleared when the "page" is finally
+ * removed from global LRU.
+ */
+ mz = page_cgroup_zoneinfo(pc);
+ mem = pc->mem_cgroup;
MEM_CGROUP_ZSTAT(mz, lru) -= 1;
-
- mem_cgroup_charge_statistics(pc->mem_cgroup, pc, false);
- list_del(&pc->lru);
+ list_del_init(&pc->lru);
+ return;
}
-static void __mem_cgroup_add_list(struct mem_cgroup_per_zone *mz,
- struct page_cgroup *pc)
+void mem_cgroup_del_lru(struct page *page)
{
- int lru = LRU_BASE;
-
- if (PageCgroupUnevictable(pc))
- lru = LRU_UNEVICTABLE;
- else {
- if (PageCgroupActive(pc))
- lru += LRU_ACTIVE;
- if (PageCgroupFile(pc))
- lru += LRU_FILE;
- }
-
- MEM_CGROUP_ZSTAT(mz, lru) += 1;
- list_add(&pc->lru, &mz->lists[lru]);
-
- mem_cgroup_charge_statistics(pc->mem_cgroup, pc, true);
+ mem_cgroup_del_lru_list(page, page_lru(page));
}
-static void __mem_cgroup_move_lists(struct page_cgroup *pc, enum lru_list lru)
+void mem_cgroup_rotate_lru_list(struct page *page, enum lru_list lru)
{
- struct mem_cgroup_per_zone *mz = page_cgroup_zoneinfo(pc);
- int active = PageCgroupActive(pc);
- int file = PageCgroupFile(pc);
- int unevictable = PageCgroupUnevictable(pc);
- enum lru_list from = unevictable ? LRU_UNEVICTABLE :
- (LRU_FILE * !!file + !!active);
+ struct mem_cgroup_per_zone *mz;
+ struct page_cgroup *pc;
- if (lru == from)
+ if (mem_cgroup_disabled())
return;
- MEM_CGROUP_ZSTAT(mz, from) -= 1;
- /*
- * However this is done under mz->lru_lock, another flags, which
- * are not related to LRU, will be modified from out-of-lock.
- * We have to use atomic set/clear flags.
- */
- if (is_unevictable_lru(lru)) {
- ClearPageCgroupActive(pc);
- SetPageCgroupUnevictable(pc);
- } else {
- if (is_active_lru(lru))
- SetPageCgroupActive(pc);
- else
- ClearPageCgroupActive(pc);
- ClearPageCgroupUnevictable(pc);
- }
-
- MEM_CGROUP_ZSTAT(mz, lru) += 1;
+ pc = lookup_page_cgroup(page);
+ smp_rmb();
+ /* unused page is not rotated. */
+ if (!PageCgroupUsed(pc))
+ return;
+ mz = page_cgroup_zoneinfo(pc);
list_move(&pc->lru, &mz->lists[lru]);
}
+void mem_cgroup_add_lru_list(struct page *page, enum lru_list lru)
+{
+ struct page_cgroup *pc;
+ struct mem_cgroup_per_zone *mz;
+
+ if (mem_cgroup_disabled())
+ return;
+ pc = lookup_page_cgroup(page);
+ /* barrier to sync with "charge" */
+ smp_rmb();
+ if (!PageCgroupUsed(pc))
+ return;
+
+ mz = page_cgroup_zoneinfo(pc);
+ MEM_CGROUP_ZSTAT(mz, lru) += 1;
+ list_add(&pc->lru, &mz->lists[lru]);
+}
+
+/*
+ * At handling SwapCache, pc->mem_cgroup may be changed while it's linked to
+ * lru because the page may.be reused after it's fully uncharged (because of
+ * SwapCache behavior).To handle that, unlink page_cgroup from LRU when charge
+ * it again. This function is only used to charge SwapCache. It's done under
+ * lock_page and expected that zone->lru_lock is never held.
+ */
+static void mem_cgroup_lru_del_before_commit_swapcache(struct page *page)
+{
+ unsigned long flags;
+ struct zone *zone = page_zone(page);
+ struct page_cgroup *pc = lookup_page_cgroup(page);
+
+ spin_lock_irqsave(&zone->lru_lock, flags);
+ /*
+ * Forget old LRU when this page_cgroup is *not* used. This Used bit
+ * is guarded by lock_page() because the page is SwapCache.
+ */
+ if (!PageCgroupUsed(pc))
+ mem_cgroup_del_lru_list(page, page_lru(page));
+ spin_unlock_irqrestore(&zone->lru_lock, flags);
+}
+
+static void mem_cgroup_lru_add_after_commit_swapcache(struct page *page)
+{
+ unsigned long flags;
+ struct zone *zone = page_zone(page);
+ struct page_cgroup *pc = lookup_page_cgroup(page);
+
+ spin_lock_irqsave(&zone->lru_lock, flags);
+ /* link when the page is linked to LRU but page_cgroup isn't */
+ if (PageLRU(page) && list_empty(&pc->lru))
+ mem_cgroup_add_lru_list(page, page_lru(page));
+ spin_unlock_irqrestore(&zone->lru_lock, flags);
+}
+
+
+void mem_cgroup_move_lists(struct page *page,
+ enum lru_list from, enum lru_list to)
+{
+ if (mem_cgroup_disabled())
+ return;
+ mem_cgroup_del_lru_list(page, from);
+ mem_cgroup_add_lru_list(page, to);
+}
+
int task_in_mem_cgroup(struct task_struct *task, const struct mem_cgroup *mem)
{
int ret;
@@ -320,37 +441,6 @@
}
/*
- * This routine assumes that the appropriate zone's lru lock is already held
- */
-void mem_cgroup_move_lists(struct page *page, enum lru_list lru)
-{
- struct page_cgroup *pc;
- struct mem_cgroup_per_zone *mz;
- unsigned long flags;
-
- if (mem_cgroup_subsys.disabled)
- return;
-
- /*
- * We cannot lock_page_cgroup while holding zone's lru_lock,
- * because other holders of lock_page_cgroup can be interrupted
- * with an attempt to rotate_reclaimable_page. But we cannot
- * safely get to page_cgroup without it, so just try_lock it:
- * mem_cgroup_isolate_pages allows for page left on wrong list.
- */
- pc = lookup_page_cgroup(page);
- if (!trylock_page_cgroup(pc))
- return;
- if (pc && PageCgroupUsed(pc)) {
- mz = page_cgroup_zoneinfo(pc);
- spin_lock_irqsave(&mz->lru_lock, flags);
- __mem_cgroup_move_lists(pc, lru);
- spin_unlock_irqrestore(&mz->lru_lock, flags);
- }
- unlock_page_cgroup(pc);
-}
-
-/*
* Calculate mapped_ratio under memory controller. This will be used in
* vmscan.c for deteremining we have to reclaim mapped pages.
*/
@@ -372,39 +462,108 @@
*/
int mem_cgroup_get_reclaim_priority(struct mem_cgroup *mem)
{
- return mem->prev_priority;
+ int prev_priority;
+
+ spin_lock(&mem->reclaim_param_lock);
+ prev_priority = mem->prev_priority;
+ spin_unlock(&mem->reclaim_param_lock);
+
+ return prev_priority;
}
void mem_cgroup_note_reclaim_priority(struct mem_cgroup *mem, int priority)
{
+ spin_lock(&mem->reclaim_param_lock);
if (priority < mem->prev_priority)
mem->prev_priority = priority;
+ spin_unlock(&mem->reclaim_param_lock);
}
void mem_cgroup_record_reclaim_priority(struct mem_cgroup *mem, int priority)
{
+ spin_lock(&mem->reclaim_param_lock);
mem->prev_priority = priority;
+ spin_unlock(&mem->reclaim_param_lock);
}
-/*
- * Calculate # of pages to be scanned in this priority/zone.
- * See also vmscan.c
- *
- * priority starts from "DEF_PRIORITY" and decremented in each loop.
- * (see include/linux/mmzone.h)
- */
-
-long mem_cgroup_calc_reclaim(struct mem_cgroup *mem, struct zone *zone,
- int priority, enum lru_list lru)
+static int calc_inactive_ratio(struct mem_cgroup *memcg, unsigned long *present_pages)
{
- long nr_pages;
+ unsigned long active;
+ unsigned long inactive;
+ unsigned long gb;
+ unsigned long inactive_ratio;
+
+ inactive = mem_cgroup_get_all_zonestat(memcg, LRU_INACTIVE_ANON);
+ active = mem_cgroup_get_all_zonestat(memcg, LRU_ACTIVE_ANON);
+
+ gb = (inactive + active) >> (30 - PAGE_SHIFT);
+ if (gb)
+ inactive_ratio = int_sqrt(10 * gb);
+ else
+ inactive_ratio = 1;
+
+ if (present_pages) {
+ present_pages[0] = inactive;
+ present_pages[1] = active;
+ }
+
+ return inactive_ratio;
+}
+
+int mem_cgroup_inactive_anon_is_low(struct mem_cgroup *memcg)
+{
+ unsigned long active;
+ unsigned long inactive;
+ unsigned long present_pages[2];
+ unsigned long inactive_ratio;
+
+ inactive_ratio = calc_inactive_ratio(memcg, present_pages);
+
+ inactive = present_pages[0];
+ active = present_pages[1];
+
+ if (inactive * inactive_ratio < active)
+ return 1;
+
+ return 0;
+}
+
+unsigned long mem_cgroup_zone_nr_pages(struct mem_cgroup *memcg,
+ struct zone *zone,
+ enum lru_list lru)
+{
int nid = zone->zone_pgdat->node_id;
int zid = zone_idx(zone);
- struct mem_cgroup_per_zone *mz = mem_cgroup_zoneinfo(mem, nid, zid);
+ struct mem_cgroup_per_zone *mz = mem_cgroup_zoneinfo(memcg, nid, zid);
- nr_pages = MEM_CGROUP_ZSTAT(mz, lru);
+ return MEM_CGROUP_ZSTAT(mz, lru);
+}
- return (nr_pages >> priority);
+struct zone_reclaim_stat *mem_cgroup_get_reclaim_stat(struct mem_cgroup *memcg,
+ struct zone *zone)
+{
+ int nid = zone->zone_pgdat->node_id;
+ int zid = zone_idx(zone);
+ struct mem_cgroup_per_zone *mz = mem_cgroup_zoneinfo(memcg, nid, zid);
+
+ return &mz->reclaim_stat;
+}
+
+struct zone_reclaim_stat *
+mem_cgroup_get_reclaim_stat_from_page(struct page *page)
+{
+ struct page_cgroup *pc;
+ struct mem_cgroup_per_zone *mz;
+
+ if (mem_cgroup_disabled())
+ return NULL;
+
+ pc = lookup_page_cgroup(page);
+ mz = page_cgroup_zoneinfo(pc);
+ if (!mz)
+ return NULL;
+
+ return &mz->reclaim_stat;
}
unsigned long mem_cgroup_isolate_pages(unsigned long nr_to_scan,
@@ -429,44 +588,463 @@
mz = mem_cgroup_zoneinfo(mem_cont, nid, zid);
src = &mz->lists[lru];
- spin_lock(&mz->lru_lock);
scan = 0;
list_for_each_entry_safe_reverse(pc, tmp, src, lru) {
if (scan >= nr_to_scan)
break;
+
+ page = pc->page;
if (unlikely(!PageCgroupUsed(pc)))
continue;
- page = pc->page;
-
if (unlikely(!PageLRU(page)))
continue;
- /*
- * TODO: play better with lumpy reclaim, grabbing anything.
- */
- if (PageUnevictable(page) ||
- (PageActive(page) && !active) ||
- (!PageActive(page) && active)) {
- __mem_cgroup_move_lists(pc, page_lru(page));
- continue;
- }
-
scan++;
- list_move(&pc->lru, &pc_list);
-
if (__isolate_lru_page(page, mode, file) == 0) {
list_move(&page->lru, dst);
nr_taken++;
}
}
- list_splice(&pc_list, src);
- spin_unlock(&mz->lru_lock);
-
*scanned = scan;
return nr_taken;
}
+#define mem_cgroup_from_res_counter(counter, member) \
+ container_of(counter, struct mem_cgroup, member)
+
+/*
+ * This routine finds the DFS walk successor. This routine should be
+ * called with hierarchy_mutex held
+ */
+static struct mem_cgroup *
+mem_cgroup_get_next_node(struct mem_cgroup *curr, struct mem_cgroup *root_mem)
+{
+ struct cgroup *cgroup, *curr_cgroup, *root_cgroup;
+
+ curr_cgroup = curr->css.cgroup;
+ root_cgroup = root_mem->css.cgroup;
+
+ if (!list_empty(&curr_cgroup->children)) {
+ /*
+ * Walk down to children
+ */
+ mem_cgroup_put(curr);
+ cgroup = list_entry(curr_cgroup->children.next,
+ struct cgroup, sibling);
+ curr = mem_cgroup_from_cont(cgroup);
+ mem_cgroup_get(curr);
+ goto done;
+ }
+
+visit_parent:
+ if (curr_cgroup == root_cgroup) {
+ mem_cgroup_put(curr);
+ curr = root_mem;
+ mem_cgroup_get(curr);
+ goto done;
+ }
+
+ /*
+ * Goto next sibling
+ */
+ if (curr_cgroup->sibling.next != &curr_cgroup->parent->children) {
+ mem_cgroup_put(curr);
+ cgroup = list_entry(curr_cgroup->sibling.next, struct cgroup,
+ sibling);
+ curr = mem_cgroup_from_cont(cgroup);
+ mem_cgroup_get(curr);
+ goto done;
+ }
+
+ /*
+ * Go up to next parent and next parent's sibling if need be
+ */
+ curr_cgroup = curr_cgroup->parent;
+ goto visit_parent;
+
+done:
+ root_mem->last_scanned_child = curr;
+ return curr;
+}
+
+/*
+ * Visit the first child (need not be the first child as per the ordering
+ * of the cgroup list, since we track last_scanned_child) of @mem and use
+ * that to reclaim free pages from.
+ */
+static struct mem_cgroup *
+mem_cgroup_get_first_node(struct mem_cgroup *root_mem)
+{
+ struct cgroup *cgroup;
+ struct mem_cgroup *ret;
+ bool obsolete;
+
+ obsolete = mem_cgroup_is_obsolete(root_mem->last_scanned_child);
+
+ /*
+ * Scan all children under the mem_cgroup mem
+ */
+ mutex_lock(&mem_cgroup_subsys.hierarchy_mutex);
+ if (list_empty(&root_mem->css.cgroup->children)) {
+ ret = root_mem;
+ goto done;
+ }
+
+ if (!root_mem->last_scanned_child || obsolete) {
+
+ if (obsolete && root_mem->last_scanned_child)
+ mem_cgroup_put(root_mem->last_scanned_child);
+
+ cgroup = list_first_entry(&root_mem->css.cgroup->children,
+ struct cgroup, sibling);
+ ret = mem_cgroup_from_cont(cgroup);
+ mem_cgroup_get(ret);
+ } else
+ ret = mem_cgroup_get_next_node(root_mem->last_scanned_child,
+ root_mem);
+
+done:
+ root_mem->last_scanned_child = ret;
+ mutex_unlock(&mem_cgroup_subsys.hierarchy_mutex);
+ return ret;
+}
+
+static bool mem_cgroup_check_under_limit(struct mem_cgroup *mem)
+{
+ if (do_swap_account) {
+ if (res_counter_check_under_limit(&mem->res) &&
+ res_counter_check_under_limit(&mem->memsw))
+ return true;
+ } else
+ if (res_counter_check_under_limit(&mem->res))
+ return true;
+ return false;
+}
+
+static unsigned int get_swappiness(struct mem_cgroup *memcg)
+{
+ struct cgroup *cgrp = memcg->css.cgroup;
+ unsigned int swappiness;
+
+ /* root ? */
+ if (cgrp->parent == NULL)
+ return vm_swappiness;
+
+ spin_lock(&memcg->reclaim_param_lock);
+ swappiness = memcg->swappiness;
+ spin_unlock(&memcg->reclaim_param_lock);
+
+ return swappiness;
+}
+
+/*
+ * Dance down the hierarchy if needed to reclaim memory. We remember the
+ * last child we reclaimed from, so that we don't end up penalizing
+ * one child extensively based on its position in the children list.
+ *
+ * root_mem is the original ancestor that we've been reclaim from.
+ */
+static int mem_cgroup_hierarchical_reclaim(struct mem_cgroup *root_mem,
+ gfp_t gfp_mask, bool noswap)
+{
+ struct mem_cgroup *next_mem;
+ int ret = 0;
+
+ /*
+ * Reclaim unconditionally and don't check for return value.
+ * We need to reclaim in the current group and down the tree.
+ * One might think about checking for children before reclaiming,
+ * but there might be left over accounting, even after children
+ * have left.
+ */
+ ret = try_to_free_mem_cgroup_pages(root_mem, gfp_mask, noswap,
+ get_swappiness(root_mem));
+ if (mem_cgroup_check_under_limit(root_mem))
+ return 0;
+ if (!root_mem->use_hierarchy)
+ return ret;
+
+ next_mem = mem_cgroup_get_first_node(root_mem);
+
+ while (next_mem != root_mem) {
+ if (mem_cgroup_is_obsolete(next_mem)) {
+ mem_cgroup_put(next_mem);
+ next_mem = mem_cgroup_get_first_node(root_mem);
+ continue;
+ }
+ ret = try_to_free_mem_cgroup_pages(next_mem, gfp_mask, noswap,
+ get_swappiness(next_mem));
+ if (mem_cgroup_check_under_limit(root_mem))
+ return 0;
+ mutex_lock(&mem_cgroup_subsys.hierarchy_mutex);
+ next_mem = mem_cgroup_get_next_node(next_mem, root_mem);
+ mutex_unlock(&mem_cgroup_subsys.hierarchy_mutex);
+ }
+ return ret;
+}
+
+bool mem_cgroup_oom_called(struct task_struct *task)
+{
+ bool ret = false;
+ struct mem_cgroup *mem;
+ struct mm_struct *mm;
+
+ rcu_read_lock();
+ mm = task->mm;
+ if (!mm)
+ mm = &init_mm;
+ mem = mem_cgroup_from_task(rcu_dereference(mm->owner));
+ if (mem && time_before(jiffies, mem->last_oom_jiffies + HZ/10))
+ ret = true;
+ rcu_read_unlock();
+ return ret;
+}
+/*
+ * Unlike exported interface, "oom" parameter is added. if oom==true,
+ * oom-killer can be invoked.
+ */
+static int __mem_cgroup_try_charge(struct mm_struct *mm,
+ gfp_t gfp_mask, struct mem_cgroup **memcg,
+ bool oom)
+{
+ struct mem_cgroup *mem, *mem_over_limit;
+ int nr_retries = MEM_CGROUP_RECLAIM_RETRIES;
+ struct res_counter *fail_res;
+
+ if (unlikely(test_thread_flag(TIF_MEMDIE))) {
+ /* Don't account this! */
+ *memcg = NULL;
+ return 0;
+ }
+
+ /*
+ * We always charge the cgroup the mm_struct belongs to.
+ * The mm_struct's mem_cgroup changes on task migration if the
+ * thread group leader migrates. It's possible that mm is not
+ * set, if so charge the init_mm (happens for pagecache usage).
+ */
+ mem = *memcg;
+ if (likely(!mem)) {
+ mem = try_get_mem_cgroup_from_mm(mm);
+ *memcg = mem;
+ } else {
+ css_get(&mem->css);
+ }
+ if (unlikely(!mem))
+ return 0;
+
+ VM_BUG_ON(mem_cgroup_is_obsolete(mem));
+
+ while (1) {
+ int ret;
+ bool noswap = false;
+
+ ret = res_counter_charge(&mem->res, PAGE_SIZE, &fail_res);
+ if (likely(!ret)) {
+ if (!do_swap_account)
+ break;
+ ret = res_counter_charge(&mem->memsw, PAGE_SIZE,
+ &fail_res);
+ if (likely(!ret))
+ break;
+ /* mem+swap counter fails */
+ res_counter_uncharge(&mem->res, PAGE_SIZE);
+ noswap = true;
+ mem_over_limit = mem_cgroup_from_res_counter(fail_res,
+ memsw);
+ } else
+ /* mem counter fails */
+ mem_over_limit = mem_cgroup_from_res_counter(fail_res,
+ res);
+
+ if (!(gfp_mask & __GFP_WAIT))
+ goto nomem;
+
+ ret = mem_cgroup_hierarchical_reclaim(mem_over_limit, gfp_mask,
+ noswap);
+
+ /*
+ * try_to_free_mem_cgroup_pages() might not give us a full
+ * picture of reclaim. Some pages are reclaimed and might be
+ * moved to swap cache or just unmapped from the cgroup.
+ * Check the limit again to see if the reclaim reduced the
+ * current usage of the cgroup before giving up
+ *
+ */
+ if (mem_cgroup_check_under_limit(mem_over_limit))
+ continue;
+
+ if (!nr_retries--) {
+ if (oom) {
+ mutex_lock(&memcg_tasklist);
+ mem_cgroup_out_of_memory(mem_over_limit, gfp_mask);
+ mutex_unlock(&memcg_tasklist);
+ mem_over_limit->last_oom_jiffies = jiffies;
+ }
+ goto nomem;
+ }
+ }
+ return 0;
+nomem:
+ css_put(&mem->css);
+ return -ENOMEM;
+}
+
+static struct mem_cgroup *try_get_mem_cgroup_from_swapcache(struct page *page)
+{
+ struct mem_cgroup *mem;
+ swp_entry_t ent;
+
+ if (!PageSwapCache(page))
+ return NULL;
+
+ ent.val = page_private(page);
+ mem = lookup_swap_cgroup(ent);
+ if (!mem)
+ return NULL;
+ if (!css_tryget(&mem->css))
+ return NULL;
+ return mem;
+}
+
+/*
+ * commit a charge got by __mem_cgroup_try_charge() and makes page_cgroup to be
+ * USED state. If already USED, uncharge and return.
+ */
+
+static void __mem_cgroup_commit_charge(struct mem_cgroup *mem,
+ struct page_cgroup *pc,
+ enum charge_type ctype)
+{
+ /* try_charge() can return NULL to *memcg, taking care of it. */
+ if (!mem)
+ return;
+
+ lock_page_cgroup(pc);
+ if (unlikely(PageCgroupUsed(pc))) {
+ unlock_page_cgroup(pc);
+ res_counter_uncharge(&mem->res, PAGE_SIZE);
+ if (do_swap_account)
+ res_counter_uncharge(&mem->memsw, PAGE_SIZE);
+ css_put(&mem->css);
+ return;
+ }
+ pc->mem_cgroup = mem;
+ smp_wmb();
+ pc->flags = pcg_default_flags[ctype];
+
+ mem_cgroup_charge_statistics(mem, pc, true);
+
+ unlock_page_cgroup(pc);
+}
+
+/**
+ * mem_cgroup_move_account - move account of the page
+ * @pc: page_cgroup of the page.
+ * @from: mem_cgroup which the page is moved from.
+ * @to: mem_cgroup which the page is moved to. @from != @to.
+ *
+ * The caller must confirm following.
+ * - page is not on LRU (isolate_page() is useful.)
+ *
+ * returns 0 at success,
+ * returns -EBUSY when lock is busy or "pc" is unstable.
+ *
+ * This function does "uncharge" from old cgroup but doesn't do "charge" to
+ * new cgroup. It should be done by a caller.
+ */
+
+static int mem_cgroup_move_account(struct page_cgroup *pc,
+ struct mem_cgroup *from, struct mem_cgroup *to)
+{
+ struct mem_cgroup_per_zone *from_mz, *to_mz;
+ int nid, zid;
+ int ret = -EBUSY;
+
+ VM_BUG_ON(from == to);
+ VM_BUG_ON(PageLRU(pc->page));
+
+ nid = page_cgroup_nid(pc);
+ zid = page_cgroup_zid(pc);
+ from_mz = mem_cgroup_zoneinfo(from, nid, zid);
+ to_mz = mem_cgroup_zoneinfo(to, nid, zid);
+
+ if (!trylock_page_cgroup(pc))
+ return ret;
+
+ if (!PageCgroupUsed(pc))
+ goto out;
+
+ if (pc->mem_cgroup != from)
+ goto out;
+
+ css_put(&from->css);
+ res_counter_uncharge(&from->res, PAGE_SIZE);
+ mem_cgroup_charge_statistics(from, pc, false);
+ if (do_swap_account)
+ res_counter_uncharge(&from->memsw, PAGE_SIZE);
+ pc->mem_cgroup = to;
+ mem_cgroup_charge_statistics(to, pc, true);
+ css_get(&to->css);
+ ret = 0;
+out:
+ unlock_page_cgroup(pc);
+ return ret;
+}
+
+/*
+ * move charges to its parent.
+ */
+
+static int mem_cgroup_move_parent(struct page_cgroup *pc,
+ struct mem_cgroup *child,
+ gfp_t gfp_mask)
+{
+ struct page *page = pc->page;
+ struct cgroup *cg = child->css.cgroup;
+ struct cgroup *pcg = cg->parent;
+ struct mem_cgroup *parent;
+ int ret;
+
+ /* Is ROOT ? */
+ if (!pcg)
+ return -EINVAL;
+
+
+ parent = mem_cgroup_from_cont(pcg);
+
+
+ ret = __mem_cgroup_try_charge(NULL, gfp_mask, &parent, false);
+ if (ret || !parent)
+ return ret;
+
+ if (!get_page_unless_zero(page))
+ return -EBUSY;
+
+ ret = isolate_lru_page(page);
+
+ if (ret)
+ goto cancel;
+
+ ret = mem_cgroup_move_account(pc, child, parent);
+
+ /* drop extra refcnt by try_charge() (move_account increment one) */
+ css_put(&parent->css);
+ putback_lru_page(page);
+ if (!ret) {
+ put_page(page);
+ return 0;
+ }
+ /* uncharge if move fails */
+cancel:
+ res_counter_uncharge(&parent->res, PAGE_SIZE);
+ if (do_swap_account)
+ res_counter_uncharge(&parent->memsw, PAGE_SIZE);
+ put_page(page);
+ return ret;
+}
+
/*
* Charge the memory controller for page usage.
* Return
@@ -479,95 +1057,27 @@
{
struct mem_cgroup *mem;
struct page_cgroup *pc;
- unsigned long nr_retries = MEM_CGROUP_RECLAIM_RETRIES;
- struct mem_cgroup_per_zone *mz;
- unsigned long flags;
+ int ret;
pc = lookup_page_cgroup(page);
/* can happen at boot */
if (unlikely(!pc))
return 0;
prefetchw(pc);
- /*
- * We always charge the cgroup the mm_struct belongs to.
- * The mm_struct's mem_cgroup changes on task migration if the
- * thread group leader migrates. It's possible that mm is not
- * set, if so charge the init_mm (happens for pagecache usage).
- */
- if (likely(!memcg)) {
- rcu_read_lock();
- mem = mem_cgroup_from_task(rcu_dereference(mm->owner));
- if (unlikely(!mem)) {
- rcu_read_unlock();
- return 0;
- }
- /*
- * For every charge from the cgroup, increment reference count
- */
- css_get(&mem->css);
- rcu_read_unlock();
- } else {
- mem = memcg;
- css_get(&memcg->css);
- }
+ mem = memcg;
+ ret = __mem_cgroup_try_charge(mm, gfp_mask, &mem, true);
+ if (ret || !mem)
+ return ret;
- while (unlikely(res_counter_charge(&mem->res, PAGE_SIZE))) {
- if (!(gfp_mask & __GFP_WAIT))
- goto out;
-
- if (try_to_free_mem_cgroup_pages(mem, gfp_mask))
- continue;
-
- /*
- * try_to_free_mem_cgroup_pages() might not give us a full
- * picture of reclaim. Some pages are reclaimed and might be
- * moved to swap cache or just unmapped from the cgroup.
- * Check the limit again to see if the reclaim reduced the
- * current usage of the cgroup before giving up
- */
- if (res_counter_check_under_limit(&mem->res))
- continue;
-
- if (!nr_retries--) {
- mem_cgroup_out_of_memory(mem, gfp_mask);
- goto out;
- }
- }
-
-
- lock_page_cgroup(pc);
- if (unlikely(PageCgroupUsed(pc))) {
- unlock_page_cgroup(pc);
- res_counter_uncharge(&mem->res, PAGE_SIZE);
- css_put(&mem->css);
-
- goto done;
- }
- pc->mem_cgroup = mem;
- /*
- * If a page is accounted as a page cache, insert to inactive list.
- * If anon, insert to active list.
- */
- pc->flags = pcg_default_flags[ctype];
-
- mz = page_cgroup_zoneinfo(pc);
-
- spin_lock_irqsave(&mz->lru_lock, flags);
- __mem_cgroup_add_list(mz, pc);
- spin_unlock_irqrestore(&mz->lru_lock, flags);
- unlock_page_cgroup(pc);
-
-done:
+ __mem_cgroup_commit_charge(mem, pc, ctype);
return 0;
-out:
- css_put(&mem->css);
- return -ENOMEM;
}
-int mem_cgroup_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask)
+int mem_cgroup_newpage_charge(struct page *page,
+ struct mm_struct *mm, gfp_t gfp_mask)
{
- if (mem_cgroup_subsys.disabled)
+ if (mem_cgroup_disabled())
return 0;
if (PageCompound(page))
return 0;
@@ -589,7 +1099,10 @@
int mem_cgroup_cache_charge(struct page *page, struct mm_struct *mm,
gfp_t gfp_mask)
{
- if (mem_cgroup_subsys.disabled)
+ struct mem_cgroup *mem = NULL;
+ int ret;
+
+ if (mem_cgroup_disabled())
return 0;
if (PageCompound(page))
return 0;
@@ -601,6 +1114,8 @@
* For GFP_NOWAIT case, the page may be pre-charged before calling
* add_to_page_cache(). (See shmem.c) check it here and avoid to call
* charge twice. (It works but has to pay a bit larger cost.)
+ * And when the page is SwapCache, it should take swap information
+ * into account. This is under lock_page() now.
*/
if (!(gfp_mask & __GFP_WAIT)) {
struct page_cgroup *pc;
@@ -617,58 +1132,198 @@
unlock_page_cgroup(pc);
}
- if (unlikely(!mm))
+ if (do_swap_account && PageSwapCache(page)) {
+ mem = try_get_mem_cgroup_from_swapcache(page);
+ if (mem)
+ mm = NULL;
+ else
+ mem = NULL;
+ /* SwapCache may be still linked to LRU now. */
+ mem_cgroup_lru_del_before_commit_swapcache(page);
+ }
+
+ if (unlikely(!mm && !mem))
mm = &init_mm;
if (page_is_file_cache(page))
return mem_cgroup_charge_common(page, mm, gfp_mask,
MEM_CGROUP_CHARGE_TYPE_CACHE, NULL);
- else
- return mem_cgroup_charge_common(page, mm, gfp_mask,
- MEM_CGROUP_CHARGE_TYPE_SHMEM, NULL);
+
+ ret = mem_cgroup_charge_common(page, mm, gfp_mask,
+ MEM_CGROUP_CHARGE_TYPE_SHMEM, mem);
+ if (mem)
+ css_put(&mem->css);
+ if (PageSwapCache(page))
+ mem_cgroup_lru_add_after_commit_swapcache(page);
+
+ if (do_swap_account && !ret && PageSwapCache(page)) {
+ swp_entry_t ent = {.val = page_private(page)};
+ /* avoid double counting */
+ mem = swap_cgroup_record(ent, NULL);
+ if (mem) {
+ res_counter_uncharge(&mem->memsw, PAGE_SIZE);
+ mem_cgroup_put(mem);
+ }
+ }
+ return ret;
}
/*
+ * While swap-in, try_charge -> commit or cancel, the page is locked.
+ * And when try_charge() successfully returns, one refcnt to memcg without
+ * struct page_cgroup is aquired. This refcnt will be cumsumed by
+ * "commit()" or removed by "cancel()"
+ */
+int mem_cgroup_try_charge_swapin(struct mm_struct *mm,
+ struct page *page,
+ gfp_t mask, struct mem_cgroup **ptr)
+{
+ struct mem_cgroup *mem;
+ int ret;
+
+ if (mem_cgroup_disabled())
+ return 0;
+
+ if (!do_swap_account)
+ goto charge_cur_mm;
+ /*
+ * A racing thread's fault, or swapoff, may have already updated
+ * the pte, and even removed page from swap cache: return success
+ * to go on to do_swap_page()'s pte_same() test, which should fail.
+ */
+ if (!PageSwapCache(page))
+ return 0;
+ mem = try_get_mem_cgroup_from_swapcache(page);
+ if (!mem)
+ goto charge_cur_mm;
+ *ptr = mem;
+ ret = __mem_cgroup_try_charge(NULL, mask, ptr, true);
+ /* drop extra refcnt from tryget */
+ css_put(&mem->css);
+ return ret;
+charge_cur_mm:
+ if (unlikely(!mm))
+ mm = &init_mm;
+ return __mem_cgroup_try_charge(mm, mask, ptr, true);
+}
+
+void mem_cgroup_commit_charge_swapin(struct page *page, struct mem_cgroup *ptr)
+{
+ struct page_cgroup *pc;
+
+ if (mem_cgroup_disabled())
+ return;
+ if (!ptr)
+ return;
+ pc = lookup_page_cgroup(page);
+ mem_cgroup_lru_del_before_commit_swapcache(page);
+ __mem_cgroup_commit_charge(ptr, pc, MEM_CGROUP_CHARGE_TYPE_MAPPED);
+ mem_cgroup_lru_add_after_commit_swapcache(page);
+ /*
+ * Now swap is on-memory. This means this page may be
+ * counted both as mem and swap....double count.
+ * Fix it by uncharging from memsw. Basically, this SwapCache is stable
+ * under lock_page(). But in do_swap_page()::memory.c, reuse_swap_page()
+ * may call delete_from_swap_cache() before reach here.
+ */
+ if (do_swap_account && PageSwapCache(page)) {
+ swp_entry_t ent = {.val = page_private(page)};
+ struct mem_cgroup *memcg;
+ memcg = swap_cgroup_record(ent, NULL);
+ if (memcg) {
+ res_counter_uncharge(&memcg->memsw, PAGE_SIZE);
+ mem_cgroup_put(memcg);
+ }
+
+ }
+ /* add this page(page_cgroup) to the LRU we want. */
+
+}
+
+void mem_cgroup_cancel_charge_swapin(struct mem_cgroup *mem)
+{
+ if (mem_cgroup_disabled())
+ return;
+ if (!mem)
+ return;
+ res_counter_uncharge(&mem->res, PAGE_SIZE);
+ if (do_swap_account)
+ res_counter_uncharge(&mem->memsw, PAGE_SIZE);
+ css_put(&mem->css);
+}
+
+
+/*
* uncharge if !page_mapped(page)
*/
-static void
+static struct mem_cgroup *
__mem_cgroup_uncharge_common(struct page *page, enum charge_type ctype)
{
struct page_cgroup *pc;
- struct mem_cgroup *mem;
+ struct mem_cgroup *mem = NULL;
struct mem_cgroup_per_zone *mz;
- unsigned long flags;
- if (mem_cgroup_subsys.disabled)
- return;
+ if (mem_cgroup_disabled())
+ return NULL;
+
+ if (PageSwapCache(page))
+ return NULL;
/*
* Check if our page_cgroup is valid
*/
pc = lookup_page_cgroup(page);
if (unlikely(!pc || !PageCgroupUsed(pc)))
- return;
+ return NULL;
lock_page_cgroup(pc);
- if ((ctype == MEM_CGROUP_CHARGE_TYPE_MAPPED && page_mapped(page))
- || !PageCgroupUsed(pc)) {
- /* This happens at race in zap_pte_range() and do_swap_page()*/
- unlock_page_cgroup(pc);
- return;
- }
- ClearPageCgroupUsed(pc);
+
mem = pc->mem_cgroup;
- mz = page_cgroup_zoneinfo(pc);
- spin_lock_irqsave(&mz->lru_lock, flags);
- __mem_cgroup_remove_list(mz, pc);
- spin_unlock_irqrestore(&mz->lru_lock, flags);
- unlock_page_cgroup(pc);
+ if (!PageCgroupUsed(pc))
+ goto unlock_out;
+
+ switch (ctype) {
+ case MEM_CGROUP_CHARGE_TYPE_MAPPED:
+ if (page_mapped(page))
+ goto unlock_out;
+ break;
+ case MEM_CGROUP_CHARGE_TYPE_SWAPOUT:
+ if (!PageAnon(page)) { /* Shared memory */
+ if (page->mapping && !page_is_file_cache(page))
+ goto unlock_out;
+ } else if (page_mapped(page)) /* Anon */
+ goto unlock_out;
+ break;
+ default:
+ break;
+ }
res_counter_uncharge(&mem->res, PAGE_SIZE);
- css_put(&mem->css);
+ if (do_swap_account && (ctype != MEM_CGROUP_CHARGE_TYPE_SWAPOUT))
+ res_counter_uncharge(&mem->memsw, PAGE_SIZE);
- return;
+ mem_cgroup_charge_statistics(mem, pc, false);
+ ClearPageCgroupUsed(pc);
+ /*
+ * pc->mem_cgroup is not cleared here. It will be accessed when it's
+ * freed from LRU. This is safe because uncharged page is expected not
+ * to be reused (freed soon). Exception is SwapCache, it's handled by
+ * special functions.
+ */
+
+ mz = page_cgroup_zoneinfo(pc);
+ unlock_page_cgroup(pc);
+
+ /* at swapout, this memcg will be accessed to record to swap */
+ if (ctype != MEM_CGROUP_CHARGE_TYPE_SWAPOUT)
+ css_put(&mem->css);
+
+ return mem;
+
+unlock_out:
+ unlock_page_cgroup(pc);
+ return NULL;
}
void mem_cgroup_uncharge_page(struct page *page)
@@ -689,16 +1344,55 @@
}
/*
- * Before starting migration, account against new page.
+ * called from __delete_from_swap_cache() and drop "page" account.
+ * memcg information is recorded to swap_cgroup of "ent"
*/
-int mem_cgroup_prepare_migration(struct page *page, struct page *newpage)
+void mem_cgroup_uncharge_swapcache(struct page *page, swp_entry_t ent)
+{
+ struct mem_cgroup *memcg;
+
+ memcg = __mem_cgroup_uncharge_common(page,
+ MEM_CGROUP_CHARGE_TYPE_SWAPOUT);
+ /* record memcg information */
+ if (do_swap_account && memcg) {
+ swap_cgroup_record(ent, memcg);
+ mem_cgroup_get(memcg);
+ }
+ if (memcg)
+ css_put(&memcg->css);
+}
+
+#ifdef CONFIG_CGROUP_MEM_RES_CTLR_SWAP
+/*
+ * called from swap_entry_free(). remove record in swap_cgroup and
+ * uncharge "memsw" account.
+ */
+void mem_cgroup_uncharge_swap(swp_entry_t ent)
+{
+ struct mem_cgroup *memcg;
+
+ if (!do_swap_account)
+ return;
+
+ memcg = swap_cgroup_record(ent, NULL);
+ if (memcg) {
+ res_counter_uncharge(&memcg->memsw, PAGE_SIZE);
+ mem_cgroup_put(memcg);
+ }
+}
+#endif
+
+/*
+ * Before starting migration, account PAGE_SIZE to mem_cgroup that the old
+ * page belongs to.
+ */
+int mem_cgroup_prepare_migration(struct page *page, struct mem_cgroup **ptr)
{
struct page_cgroup *pc;
struct mem_cgroup *mem = NULL;
- enum charge_type ctype = MEM_CGROUP_CHARGE_TYPE_MAPPED;
int ret = 0;
- if (mem_cgroup_subsys.disabled)
+ if (mem_cgroup_disabled())
return 0;
pc = lookup_page_cgroup(page);
@@ -706,41 +1400,67 @@
if (PageCgroupUsed(pc)) {
mem = pc->mem_cgroup;
css_get(&mem->css);
- if (PageCgroupCache(pc)) {
- if (page_is_file_cache(page))
- ctype = MEM_CGROUP_CHARGE_TYPE_CACHE;
- else
- ctype = MEM_CGROUP_CHARGE_TYPE_SHMEM;
- }
}
unlock_page_cgroup(pc);
+
if (mem) {
- ret = mem_cgroup_charge_common(newpage, NULL, GFP_KERNEL,
- ctype, mem);
+ ret = __mem_cgroup_try_charge(NULL, GFP_KERNEL, &mem, false);
css_put(&mem->css);
}
+ *ptr = mem;
return ret;
}
/* remove redundant charge if migration failed*/
-void mem_cgroup_end_migration(struct page *newpage)
+void mem_cgroup_end_migration(struct mem_cgroup *mem,
+ struct page *oldpage, struct page *newpage)
{
+ struct page *target, *unused;
+ struct page_cgroup *pc;
+ enum charge_type ctype;
+
+ if (!mem)
+ return;
+
+ /* at migration success, oldpage->mapping is NULL. */
+ if (oldpage->mapping) {
+ target = oldpage;
+ unused = NULL;
+ } else {
+ target = newpage;
+ unused = oldpage;
+ }
+
+ if (PageAnon(target))
+ ctype = MEM_CGROUP_CHARGE_TYPE_MAPPED;
+ else if (page_is_file_cache(target))
+ ctype = MEM_CGROUP_CHARGE_TYPE_CACHE;
+ else
+ ctype = MEM_CGROUP_CHARGE_TYPE_SHMEM;
+
+ /* unused page is not on radix-tree now. */
+ if (unused)
+ __mem_cgroup_uncharge_common(unused, ctype);
+
+ pc = lookup_page_cgroup(target);
/*
- * At success, page->mapping is not NULL.
- * special rollback care is necessary when
- * 1. at migration failure. (newpage->mapping is cleared in this case)
- * 2. the newpage was moved but not remapped again because the task
- * exits and the newpage is obsolete. In this case, the new page
- * may be a swapcache. So, we just call mem_cgroup_uncharge_page()
- * always for avoiding mess. The page_cgroup will be removed if
- * unnecessary. File cache pages is still on radix-tree. Don't
- * care it.
+ * __mem_cgroup_commit_charge() check PCG_USED bit of page_cgroup.
+ * So, double-counting is effectively avoided.
*/
- if (!newpage->mapping)
- __mem_cgroup_uncharge_common(newpage,
- MEM_CGROUP_CHARGE_TYPE_FORCE);
- else if (PageAnon(newpage))
- mem_cgroup_uncharge_page(newpage);
+ __mem_cgroup_commit_charge(mem, pc, ctype);
+
+ /*
+ * Both of oldpage and newpage are still under lock_page().
+ * Then, we don't have to care about race in radix-tree.
+ * But we have to be careful that this page is unmapped or not.
+ *
+ * There is a case for !page_mapped(). At the start of
+ * migration, oldpage was mapped. But now, it's zapped.
+ * But we know *target* page is not freed/reused under us.
+ * mem_cgroup_uncharge_page() does all necessary checks.
+ */
+ if (ctype == MEM_CGROUP_CHARGE_TYPE_MAPPED)
+ mem_cgroup_uncharge_page(target);
}
/*
@@ -748,29 +1468,26 @@
* This is typically used for page reclaiming for shmem for reducing side
* effect of page allocation from shmem, which is used by some mem_cgroup.
*/
-int mem_cgroup_shrink_usage(struct mm_struct *mm, gfp_t gfp_mask)
+int mem_cgroup_shrink_usage(struct page *page,
+ struct mm_struct *mm,
+ gfp_t gfp_mask)
{
- struct mem_cgroup *mem;
+ struct mem_cgroup *mem = NULL;
int progress = 0;
int retry = MEM_CGROUP_RECLAIM_RETRIES;
- if (mem_cgroup_subsys.disabled)
+ if (mem_cgroup_disabled())
return 0;
- if (!mm)
+ if (page)
+ mem = try_get_mem_cgroup_from_swapcache(page);
+ if (!mem && mm)
+ mem = try_get_mem_cgroup_from_mm(mm);
+ if (unlikely(!mem))
return 0;
- rcu_read_lock();
- mem = mem_cgroup_from_task(rcu_dereference(mm->owner));
- if (unlikely(!mem)) {
- rcu_read_unlock();
- return 0;
- }
- css_get(&mem->css);
- rcu_read_unlock();
-
do {
- progress = try_to_free_mem_cgroup_pages(mem, gfp_mask);
- progress += res_counter_check_under_limit(&mem->res);
+ progress = mem_cgroup_hierarchical_reclaim(mem, gfp_mask, true);
+ progress += mem_cgroup_check_under_limit(mem);
} while (!progress && --retry);
css_put(&mem->css);
@@ -779,117 +1496,295 @@
return 0;
}
+static DEFINE_MUTEX(set_limit_mutex);
+
static int mem_cgroup_resize_limit(struct mem_cgroup *memcg,
- unsigned long long val)
+ unsigned long long val)
{
int retry_count = MEM_CGROUP_RECLAIM_RETRIES;
int progress;
+ u64 memswlimit;
int ret = 0;
- while (res_counter_set_limit(&memcg->res, val)) {
+ while (retry_count) {
if (signal_pending(current)) {
ret = -EINTR;
break;
}
- if (!retry_count) {
- ret = -EBUSY;
+ /*
+ * Rather than hide all in some function, I do this in
+ * open coded manner. You see what this really does.
+ * We have to guarantee mem->res.limit < mem->memsw.limit.
+ */
+ mutex_lock(&set_limit_mutex);
+ memswlimit = res_counter_read_u64(&memcg->memsw, RES_LIMIT);
+ if (memswlimit < val) {
+ ret = -EINVAL;
+ mutex_unlock(&set_limit_mutex);
break;
}
- progress = try_to_free_mem_cgroup_pages(memcg, GFP_KERNEL);
- if (!progress)
+ ret = res_counter_set_limit(&memcg->res, val);
+ mutex_unlock(&set_limit_mutex);
+
+ if (!ret)
+ break;
+
+ progress = mem_cgroup_hierarchical_reclaim(memcg, GFP_KERNEL,
+ false);
+ if (!progress) retry_count--;
+ }
+
+ return ret;
+}
+
+int mem_cgroup_resize_memsw_limit(struct mem_cgroup *memcg,
+ unsigned long long val)
+{
+ int retry_count = MEM_CGROUP_RECLAIM_RETRIES;
+ u64 memlimit, oldusage, curusage;
+ int ret;
+
+ if (!do_swap_account)
+ return -EINVAL;
+
+ while (retry_count) {
+ if (signal_pending(current)) {
+ ret = -EINTR;
+ break;
+ }
+ /*
+ * Rather than hide all in some function, I do this in
+ * open coded manner. You see what this really does.
+ * We have to guarantee mem->res.limit < mem->memsw.limit.
+ */
+ mutex_lock(&set_limit_mutex);
+ memlimit = res_counter_read_u64(&memcg->res, RES_LIMIT);
+ if (memlimit > val) {
+ ret = -EINVAL;
+ mutex_unlock(&set_limit_mutex);
+ break;
+ }
+ ret = res_counter_set_limit(&memcg->memsw, val);
+ mutex_unlock(&set_limit_mutex);
+
+ if (!ret)
+ break;
+
+ oldusage = res_counter_read_u64(&memcg->memsw, RES_USAGE);
+ mem_cgroup_hierarchical_reclaim(memcg, GFP_KERNEL, true);
+ curusage = res_counter_read_u64(&memcg->memsw, RES_USAGE);
+ if (curusage >= oldusage)
retry_count--;
}
return ret;
}
-
/*
* This routine traverse page_cgroup in given list and drop them all.
* *And* this routine doesn't reclaim page itself, just removes page_cgroup.
*/
-#define FORCE_UNCHARGE_BATCH (128)
-static void mem_cgroup_force_empty_list(struct mem_cgroup *mem,
- struct mem_cgroup_per_zone *mz,
- enum lru_list lru)
+static int mem_cgroup_force_empty_list(struct mem_cgroup *mem,
+ int node, int zid, enum lru_list lru)
{
- struct page_cgroup *pc;
- struct page *page;
- int count = FORCE_UNCHARGE_BATCH;
- unsigned long flags;
+ struct zone *zone;
+ struct mem_cgroup_per_zone *mz;
+ struct page_cgroup *pc, *busy;
+ unsigned long flags, loop;
struct list_head *list;
+ int ret = 0;
+ zone = &NODE_DATA(node)->node_zones[zid];
+ mz = mem_cgroup_zoneinfo(mem, node, zid);
list = &mz->lists[lru];
- spin_lock_irqsave(&mz->lru_lock, flags);
- while (!list_empty(list)) {
- pc = list_entry(list->prev, struct page_cgroup, lru);
- page = pc->page;
- if (!PageCgroupUsed(pc))
- break;
- get_page(page);
- spin_unlock_irqrestore(&mz->lru_lock, flags);
- /*
- * Check if this page is on LRU. !LRU page can be found
- * if it's under page migration.
- */
- if (PageLRU(page)) {
- __mem_cgroup_uncharge_common(page,
- MEM_CGROUP_CHARGE_TYPE_FORCE);
- put_page(page);
- if (--count <= 0) {
- count = FORCE_UNCHARGE_BATCH;
- cond_resched();
- }
- } else {
- spin_lock_irqsave(&mz->lru_lock, flags);
+ loop = MEM_CGROUP_ZSTAT(mz, lru);
+ /* give some margin against EBUSY etc...*/
+ loop += 256;
+ busy = NULL;
+ while (loop--) {
+ ret = 0;
+ spin_lock_irqsave(&zone->lru_lock, flags);
+ if (list_empty(list)) {
+ spin_unlock_irqrestore(&zone->lru_lock, flags);
break;
}
- spin_lock_irqsave(&mz->lru_lock, flags);
+ pc = list_entry(list->prev, struct page_cgroup, lru);
+ if (busy == pc) {
+ list_move(&pc->lru, list);
+ busy = 0;
+ spin_unlock_irqrestore(&zone->lru_lock, flags);
+ continue;
+ }
+ spin_unlock_irqrestore(&zone->lru_lock, flags);
+
+ ret = mem_cgroup_move_parent(pc, mem, GFP_KERNEL);
+ if (ret == -ENOMEM)
+ break;
+
+ if (ret == -EBUSY || ret == -EINVAL) {
+ /* found lock contention or "pc" is obsolete. */
+ busy = pc;
+ cond_resched();
+ } else
+ busy = NULL;
}
- spin_unlock_irqrestore(&mz->lru_lock, flags);
+
+ if (!ret && !list_empty(list))
+ return -EBUSY;
+ return ret;
}
/*
* make mem_cgroup's charge to be 0 if there is no task.
* This enables deleting this mem_cgroup.
*/
-static int mem_cgroup_force_empty(struct mem_cgroup *mem)
+static int mem_cgroup_force_empty(struct mem_cgroup *mem, bool free_all)
{
- int ret = -EBUSY;
- int node, zid;
+ int ret;
+ int node, zid, shrink;
+ int nr_retries = MEM_CGROUP_RECLAIM_RETRIES;
+ struct cgroup *cgrp = mem->css.cgroup;
css_get(&mem->css);
- /*
- * page reclaim code (kswapd etc..) will move pages between
- * active_list <-> inactive_list while we don't take a lock.
- * So, we have to do loop here until all lists are empty.
- */
+
+ shrink = 0;
+ /* should free all ? */
+ if (free_all)
+ goto try_to_free;
+move_account:
while (mem->res.usage > 0) {
- if (atomic_read(&mem->css.cgroup->count) > 0)
+ ret = -EBUSY;
+ if (cgroup_task_count(cgrp) || !list_empty(&cgrp->children))
+ goto out;
+ ret = -EINTR;
+ if (signal_pending(current))
goto out;
/* This is for making all *used* pages to be on LRU. */
lru_add_drain_all();
- for_each_node_state(node, N_POSSIBLE)
- for (zid = 0; zid < MAX_NR_ZONES; zid++) {
- struct mem_cgroup_per_zone *mz;
+ ret = 0;
+ for_each_node_state(node, N_POSSIBLE) {
+ for (zid = 0; !ret && zid < MAX_NR_ZONES; zid++) {
enum lru_list l;
- mz = mem_cgroup_zoneinfo(mem, node, zid);
- for_each_lru(l)
- mem_cgroup_force_empty_list(mem, mz, l);
+ for_each_lru(l) {
+ ret = mem_cgroup_force_empty_list(mem,
+ node, zid, l);
+ if (ret)
+ break;
+ }
}
+ if (ret)
+ break;
+ }
+ /* it seems parent cgroup doesn't have enough mem */
+ if (ret == -ENOMEM)
+ goto try_to_free;
cond_resched();
}
ret = 0;
out:
css_put(&mem->css);
return ret;
+
+try_to_free:
+ /* returns EBUSY if there is a task or if we come here twice. */
+ if (cgroup_task_count(cgrp) || !list_empty(&cgrp->children) || shrink) {
+ ret = -EBUSY;
+ goto out;
+ }
+ /* we call try-to-free pages for make this cgroup empty */
+ lru_add_drain_all();
+ /* try to free all pages in this cgroup */
+ shrink = 1;
+ while (nr_retries && mem->res.usage > 0) {
+ int progress;
+
+ if (signal_pending(current)) {
+ ret = -EINTR;
+ goto out;
+ }
+ progress = try_to_free_mem_cgroup_pages(mem, GFP_KERNEL,
+ false, get_swappiness(mem));
+ if (!progress) {
+ nr_retries--;
+ /* maybe some writeback is necessary */
+ congestion_wait(WRITE, HZ/10);
+ }
+
+ }
+ lru_add_drain();
+ /* try move_account...there may be some *locked* pages. */
+ if (mem->res.usage)
+ goto move_account;
+ ret = 0;
+ goto out;
+}
+
+int mem_cgroup_force_empty_write(struct cgroup *cont, unsigned int event)
+{
+ return mem_cgroup_force_empty(mem_cgroup_from_cont(cont), true);
+}
+
+
+static u64 mem_cgroup_hierarchy_read(struct cgroup *cont, struct cftype *cft)
+{
+ return mem_cgroup_from_cont(cont)->use_hierarchy;
+}
+
+static int mem_cgroup_hierarchy_write(struct cgroup *cont, struct cftype *cft,
+ u64 val)
+{
+ int retval = 0;
+ struct mem_cgroup *mem = mem_cgroup_from_cont(cont);
+ struct cgroup *parent = cont->parent;
+ struct mem_cgroup *parent_mem = NULL;
+
+ if (parent)
+ parent_mem = mem_cgroup_from_cont(parent);
+
+ cgroup_lock();
+ /*
+ * If parent's use_hiearchy is set, we can't make any modifications
+ * in the child subtrees. If it is unset, then the change can
+ * occur, provided the current cgroup has no children.
+ *
+ * For the root cgroup, parent_mem is NULL, we allow value to be
+ * set if there are no children.
+ */
+ if ((!parent_mem || !parent_mem->use_hierarchy) &&
+ (val == 1 || val == 0)) {
+ if (list_empty(&cont->children))
+ mem->use_hierarchy = val;
+ else
+ retval = -EBUSY;
+ } else
+ retval = -EINVAL;
+ cgroup_unlock();
+
+ return retval;
}
static u64 mem_cgroup_read(struct cgroup *cont, struct cftype *cft)
{
- return res_counter_read_u64(&mem_cgroup_from_cont(cont)->res,
- cft->private);
+ struct mem_cgroup *mem = mem_cgroup_from_cont(cont);
+ u64 val = 0;
+ int type, name;
+
+ type = MEMFILE_TYPE(cft->private);
+ name = MEMFILE_ATTR(cft->private);
+ switch (type) {
+ case _MEM:
+ val = res_counter_read_u64(&mem->res, name);
+ break;
+ case _MEMSWAP:
+ if (do_swap_account)
+ val = res_counter_read_u64(&mem->memsw, name);
+ break;
+ default:
+ BUG();
+ break;
+ }
+ return val;
}
/*
* The user of this function is...
@@ -899,15 +1794,22 @@
const char *buffer)
{
struct mem_cgroup *memcg = mem_cgroup_from_cont(cont);
+ int type, name;
unsigned long long val;
int ret;
- switch (cft->private) {
+ type = MEMFILE_TYPE(cft->private);
+ name = MEMFILE_ATTR(cft->private);
+ switch (name) {
case RES_LIMIT:
/* This function does all necessary parse...reuse it */
ret = res_counter_memparse_write_strategy(buffer, &val);
- if (!ret)
+ if (ret)
+ break;
+ if (type == _MEM)
ret = mem_cgroup_resize_limit(memcg, val);
+ else
+ ret = mem_cgroup_resize_memsw_limit(memcg, val);
break;
default:
ret = -EINVAL; /* should be BUG() ? */
@@ -916,27 +1818,59 @@
return ret;
}
+static void memcg_get_hierarchical_limit(struct mem_cgroup *memcg,
+ unsigned long long *mem_limit, unsigned long long *memsw_limit)
+{
+ struct cgroup *cgroup;
+ unsigned long long min_limit, min_memsw_limit, tmp;
+
+ min_limit = res_counter_read_u64(&memcg->res, RES_LIMIT);
+ min_memsw_limit = res_counter_read_u64(&memcg->memsw, RES_LIMIT);
+ cgroup = memcg->css.cgroup;
+ if (!memcg->use_hierarchy)
+ goto out;
+
+ while (cgroup->parent) {
+ cgroup = cgroup->parent;
+ memcg = mem_cgroup_from_cont(cgroup);
+ if (!memcg->use_hierarchy)
+ break;
+ tmp = res_counter_read_u64(&memcg->res, RES_LIMIT);
+ min_limit = min(min_limit, tmp);
+ tmp = res_counter_read_u64(&memcg->memsw, RES_LIMIT);
+ min_memsw_limit = min(min_memsw_limit, tmp);
+ }
+out:
+ *mem_limit = min_limit;
+ *memsw_limit = min_memsw_limit;
+ return;
+}
+
static int mem_cgroup_reset(struct cgroup *cont, unsigned int event)
{
struct mem_cgroup *mem;
+ int type, name;
mem = mem_cgroup_from_cont(cont);
- switch (event) {
+ type = MEMFILE_TYPE(event);
+ name = MEMFILE_ATTR(event);
+ switch (name) {
case RES_MAX_USAGE:
- res_counter_reset_max(&mem->res);
+ if (type == _MEM)
+ res_counter_reset_max(&mem->res);
+ else
+ res_counter_reset_max(&mem->memsw);
break;
case RES_FAILCNT:
- res_counter_reset_failcnt(&mem->res);
+ if (type == _MEM)
+ res_counter_reset_failcnt(&mem->res);
+ else
+ res_counter_reset_failcnt(&mem->memsw);
break;
}
return 0;
}
-static int mem_force_empty_write(struct cgroup *cont, unsigned int event)
-{
- return mem_cgroup_force_empty(mem_cgroup_from_cont(cont));
-}
-
static const struct mem_cgroup_stat_desc {
const char *msg;
u64 unit;
@@ -985,43 +1919,163 @@
cb->fill(cb, "unevictable", unevictable * PAGE_SIZE);
}
+ {
+ unsigned long long limit, memsw_limit;
+ memcg_get_hierarchical_limit(mem_cont, &limit, &memsw_limit);
+ cb->fill(cb, "hierarchical_memory_limit", limit);
+ if (do_swap_account)
+ cb->fill(cb, "hierarchical_memsw_limit", memsw_limit);
+ }
+
+#ifdef CONFIG_DEBUG_VM
+ cb->fill(cb, "inactive_ratio", calc_inactive_ratio(mem_cont, NULL));
+
+ {
+ int nid, zid;
+ struct mem_cgroup_per_zone *mz;
+ unsigned long recent_rotated[2] = {0, 0};
+ unsigned long recent_scanned[2] = {0, 0};
+
+ for_each_online_node(nid)
+ for (zid = 0; zid < MAX_NR_ZONES; zid++) {
+ mz = mem_cgroup_zoneinfo(mem_cont, nid, zid);
+
+ recent_rotated[0] +=
+ mz->reclaim_stat.recent_rotated[0];
+ recent_rotated[1] +=
+ mz->reclaim_stat.recent_rotated[1];
+ recent_scanned[0] +=
+ mz->reclaim_stat.recent_scanned[0];
+ recent_scanned[1] +=
+ mz->reclaim_stat.recent_scanned[1];
+ }
+ cb->fill(cb, "recent_rotated_anon", recent_rotated[0]);
+ cb->fill(cb, "recent_rotated_file", recent_rotated[1]);
+ cb->fill(cb, "recent_scanned_anon", recent_scanned[0]);
+ cb->fill(cb, "recent_scanned_file", recent_scanned[1]);
+ }
+#endif
+
return 0;
}
+static u64 mem_cgroup_swappiness_read(struct cgroup *cgrp, struct cftype *cft)
+{
+ struct mem_cgroup *memcg = mem_cgroup_from_cont(cgrp);
+
+ return get_swappiness(memcg);
+}
+
+static int mem_cgroup_swappiness_write(struct cgroup *cgrp, struct cftype *cft,
+ u64 val)
+{
+ struct mem_cgroup *memcg = mem_cgroup_from_cont(cgrp);
+ struct mem_cgroup *parent;
+ if (val > 100)
+ return -EINVAL;
+
+ if (cgrp->parent == NULL)
+ return -EINVAL;
+
+ parent = mem_cgroup_from_cont(cgrp->parent);
+ /* If under hierarchy, only empty-root can set this value */
+ if ((parent->use_hierarchy) ||
+ (memcg->use_hierarchy && !list_empty(&cgrp->children)))
+ return -EINVAL;
+
+ spin_lock(&memcg->reclaim_param_lock);
+ memcg->swappiness = val;
+ spin_unlock(&memcg->reclaim_param_lock);
+
+ return 0;
+}
+
+
static struct cftype mem_cgroup_files[] = {
{
.name = "usage_in_bytes",
- .private = RES_USAGE,
+ .private = MEMFILE_PRIVATE(_MEM, RES_USAGE),
.read_u64 = mem_cgroup_read,
},
{
.name = "max_usage_in_bytes",
- .private = RES_MAX_USAGE,
+ .private = MEMFILE_PRIVATE(_MEM, RES_MAX_USAGE),
.trigger = mem_cgroup_reset,
.read_u64 = mem_cgroup_read,
},
{
.name = "limit_in_bytes",
- .private = RES_LIMIT,
+ .private = MEMFILE_PRIVATE(_MEM, RES_LIMIT),
.write_string = mem_cgroup_write,
.read_u64 = mem_cgroup_read,
},
{
.name = "failcnt",
- .private = RES_FAILCNT,
+ .private = MEMFILE_PRIVATE(_MEM, RES_FAILCNT),
.trigger = mem_cgroup_reset,
.read_u64 = mem_cgroup_read,
},
{
- .name = "force_empty",
- .trigger = mem_force_empty_write,
- },
- {
.name = "stat",
.read_map = mem_control_stat_show,
},
+ {
+ .name = "force_empty",
+ .trigger = mem_cgroup_force_empty_write,
+ },
+ {
+ .name = "use_hierarchy",
+ .write_u64 = mem_cgroup_hierarchy_write,
+ .read_u64 = mem_cgroup_hierarchy_read,
+ },
+ {
+ .name = "swappiness",
+ .read_u64 = mem_cgroup_swappiness_read,
+ .write_u64 = mem_cgroup_swappiness_write,
+ },
};
+#ifdef CONFIG_CGROUP_MEM_RES_CTLR_SWAP
+static struct cftype memsw_cgroup_files[] = {
+ {
+ .name = "memsw.usage_in_bytes",
+ .private = MEMFILE_PRIVATE(_MEMSWAP, RES_USAGE),
+ .read_u64 = mem_cgroup_read,
+ },
+ {
+ .name = "memsw.max_usage_in_bytes",
+ .private = MEMFILE_PRIVATE(_MEMSWAP, RES_MAX_USAGE),
+ .trigger = mem_cgroup_reset,
+ .read_u64 = mem_cgroup_read,
+ },
+ {
+ .name = "memsw.limit_in_bytes",
+ .private = MEMFILE_PRIVATE(_MEMSWAP, RES_LIMIT),
+ .write_string = mem_cgroup_write,
+ .read_u64 = mem_cgroup_read,
+ },
+ {
+ .name = "memsw.failcnt",
+ .private = MEMFILE_PRIVATE(_MEMSWAP, RES_FAILCNT),
+ .trigger = mem_cgroup_reset,
+ .read_u64 = mem_cgroup_read,
+ },
+};
+
+static int register_memsw_files(struct cgroup *cont, struct cgroup_subsys *ss)
+{
+ if (!do_swap_account)
+ return 0;
+ return cgroup_add_files(cont, ss, memsw_cgroup_files,
+ ARRAY_SIZE(memsw_cgroup_files));
+};
+#else
+static int register_memsw_files(struct cgroup *cont, struct cgroup_subsys *ss)
+{
+ return 0;
+}
+#endif
+
static int alloc_mem_cgroup_per_zone_info(struct mem_cgroup *mem, int node)
{
struct mem_cgroup_per_node *pn;
@@ -1047,7 +2101,6 @@
for (zone = 0; zone < MAX_NR_ZONES; zone++) {
mz = &pn->zoneinfo[zone];
- spin_lock_init(&mz->lru_lock);
for_each_lru(l)
INIT_LIST_HEAD(&mz->lists[l]);
}
@@ -1059,55 +2112,113 @@
kfree(mem->info.nodeinfo[node]);
}
+static int mem_cgroup_size(void)
+{
+ int cpustat_size = nr_cpu_ids * sizeof(struct mem_cgroup_stat_cpu);
+ return sizeof(struct mem_cgroup) + cpustat_size;
+}
+
static struct mem_cgroup *mem_cgroup_alloc(void)
{
struct mem_cgroup *mem;
+ int size = mem_cgroup_size();
- if (sizeof(*mem) < PAGE_SIZE)
- mem = kmalloc(sizeof(*mem), GFP_KERNEL);
+ if (size < PAGE_SIZE)
+ mem = kmalloc(size, GFP_KERNEL);
else
- mem = vmalloc(sizeof(*mem));
+ mem = vmalloc(size);
if (mem)
- memset(mem, 0, sizeof(*mem));
+ memset(mem, 0, size);
return mem;
}
-static void mem_cgroup_free(struct mem_cgroup *mem)
+/*
+ * At destroying mem_cgroup, references from swap_cgroup can remain.
+ * (scanning all at force_empty is too costly...)
+ *
+ * Instead of clearing all references at force_empty, we remember
+ * the number of reference from swap_cgroup and free mem_cgroup when
+ * it goes down to 0.
+ *
+ * Removal of cgroup itself succeeds regardless of refs from swap.
+ */
+
+static void __mem_cgroup_free(struct mem_cgroup *mem)
{
- if (sizeof(*mem) < PAGE_SIZE)
+ int node;
+
+ for_each_node_state(node, N_POSSIBLE)
+ free_mem_cgroup_per_zone_info(mem, node);
+
+ if (mem_cgroup_size() < PAGE_SIZE)
kfree(mem);
else
vfree(mem);
}
+static void mem_cgroup_get(struct mem_cgroup *mem)
+{
+ atomic_inc(&mem->refcnt);
+}
+
+static void mem_cgroup_put(struct mem_cgroup *mem)
+{
+ if (atomic_dec_and_test(&mem->refcnt))
+ __mem_cgroup_free(mem);
+}
+
+
+#ifdef CONFIG_CGROUP_MEM_RES_CTLR_SWAP
+static void __init enable_swap_cgroup(void)
+{
+ if (!mem_cgroup_disabled() && really_do_swap_account)
+ do_swap_account = 1;
+}
+#else
+static void __init enable_swap_cgroup(void)
+{
+}
+#endif
static struct cgroup_subsys_state *
mem_cgroup_create(struct cgroup_subsys *ss, struct cgroup *cont)
{
- struct mem_cgroup *mem;
+ struct mem_cgroup *mem, *parent;
int node;
- if (unlikely((cont->parent) == NULL)) {
- mem = &init_mem_cgroup;
- } else {
- mem = mem_cgroup_alloc();
- if (!mem)
- return ERR_PTR(-ENOMEM);
- }
-
- res_counter_init(&mem->res);
+ mem = mem_cgroup_alloc();
+ if (!mem)
+ return ERR_PTR(-ENOMEM);
for_each_node_state(node, N_POSSIBLE)
if (alloc_mem_cgroup_per_zone_info(mem, node))
goto free_out;
+ /* root ? */
+ if (cont->parent == NULL) {
+ enable_swap_cgroup();
+ parent = NULL;
+ } else {
+ parent = mem_cgroup_from_cont(cont->parent);
+ mem->use_hierarchy = parent->use_hierarchy;
+ }
+ if (parent && parent->use_hierarchy) {
+ res_counter_init(&mem->res, &parent->res);
+ res_counter_init(&mem->memsw, &parent->memsw);
+ } else {
+ res_counter_init(&mem->res, NULL);
+ res_counter_init(&mem->memsw, NULL);
+ }
+ mem->last_scanned_child = NULL;
+ spin_lock_init(&mem->reclaim_param_lock);
+
+ if (parent)
+ mem->swappiness = get_swappiness(parent);
+ atomic_set(&mem->refcnt, 1);
return &mem->css;
free_out:
- for_each_node_state(node, N_POSSIBLE)
- free_mem_cgroup_per_zone_info(mem, node);
- if (cont->parent != NULL)
- mem_cgroup_free(mem);
+ __mem_cgroup_free(mem);
return ERR_PTR(-ENOMEM);
}
@@ -1115,26 +2226,26 @@
struct cgroup *cont)
{
struct mem_cgroup *mem = mem_cgroup_from_cont(cont);
- mem_cgroup_force_empty(mem);
+ mem_cgroup_force_empty(mem, false);
}
static void mem_cgroup_destroy(struct cgroup_subsys *ss,
struct cgroup *cont)
{
- int node;
- struct mem_cgroup *mem = mem_cgroup_from_cont(cont);
-
- for_each_node_state(node, N_POSSIBLE)
- free_mem_cgroup_per_zone_info(mem, node);
-
- mem_cgroup_free(mem_cgroup_from_cont(cont));
+ mem_cgroup_put(mem_cgroup_from_cont(cont));
}
static int mem_cgroup_populate(struct cgroup_subsys *ss,
struct cgroup *cont)
{
- return cgroup_add_files(cont, ss, mem_cgroup_files,
- ARRAY_SIZE(mem_cgroup_files));
+ int ret;
+
+ ret = cgroup_add_files(cont, ss, mem_cgroup_files,
+ ARRAY_SIZE(mem_cgroup_files));
+
+ if (!ret)
+ ret = register_memsw_files(cont, ss);
+ return ret;
}
static void mem_cgroup_move_task(struct cgroup_subsys *ss,
@@ -1142,25 +2253,12 @@
struct cgroup *old_cont,
struct task_struct *p)
{
- struct mm_struct *mm;
- struct mem_cgroup *mem, *old_mem;
-
- mm = get_task_mm(p);
- if (mm == NULL)
- return;
-
- mem = mem_cgroup_from_cont(cont);
- old_mem = mem_cgroup_from_cont(old_cont);
-
+ mutex_lock(&memcg_tasklist);
/*
- * Only thread group leaders are allowed to migrate, the mm_struct is
- * in effect owned by the leader
+ * FIXME: It's better to move charges of this process from old
+ * memcg to new memcg. But it's just on TODO-List now.
*/
- if (!thread_group_leader(p))
- goto out;
-
-out:
- mmput(mm);
+ mutex_unlock(&memcg_tasklist);
}
struct cgroup_subsys mem_cgroup_subsys = {
@@ -1173,3 +2271,13 @@
.attach = mem_cgroup_move_task,
.early_init = 0,
};
+
+#ifdef CONFIG_CGROUP_MEM_RES_CTLR_SWAP
+
+static int __init disable_swap_account(char *s)
+{
+ really_do_swap_account = 0;
+ return 1;
+}
+__setup("noswapaccount", disable_swap_account);
+#endif
diff --git a/mm/memory.c b/mm/memory.c
index 3f8fa06..e009ce8 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -2000,7 +2000,7 @@
cow_user_page(new_page, old_page, address, vma);
__SetPageUptodate(new_page);
- if (mem_cgroup_charge(new_page, mm, GFP_KERNEL))
+ if (mem_cgroup_newpage_charge(new_page, mm, GFP_KERNEL))
goto oom_free_new;
/*
@@ -2392,6 +2392,7 @@
struct page *page;
swp_entry_t entry;
pte_t pte;
+ struct mem_cgroup *ptr = NULL;
int ret = 0;
if (!pte_unmap_same(mm, pmd, page_table, orig_pte))
@@ -2430,7 +2431,7 @@
lock_page(page);
delayacct_clear_flag(DELAYACCT_PF_SWAPIN);
- if (mem_cgroup_charge(page, mm, GFP_KERNEL)) {
+ if (mem_cgroup_try_charge_swapin(mm, page, GFP_KERNEL, &ptr)) {
ret = VM_FAULT_OOM;
unlock_page(page);
goto out;
@@ -2448,7 +2449,19 @@
goto out_nomap;
}
- /* The page isn't present yet, go ahead with the fault. */
+ /*
+ * The page isn't present yet, go ahead with the fault.
+ *
+ * Be careful about the sequence of operations here.
+ * To get its accounting right, reuse_swap_page() must be called
+ * while the page is counted on swap but not yet in mapcount i.e.
+ * before page_add_anon_rmap() and swap_free(); try_to_free_swap()
+ * must be called after the swap_free(), or it will never succeed.
+ * Because delete_from_swap_page() may be called by reuse_swap_page(),
+ * mem_cgroup_commit_charge_swapin() may not be able to find swp_entry
+ * in page->private. In this case, a record in swap_cgroup is silently
+ * discarded at swap_free().
+ */
inc_mm_counter(mm, anon_rss);
pte = mk_pte(page, vma->vm_page_prot);
@@ -2456,10 +2469,11 @@
pte = maybe_mkwrite(pte_mkdirty(pte), vma);
write_access = 0;
}
-
flush_icache_page(vma, page);
set_pte_at(mm, address, page_table, pte);
page_add_anon_rmap(page, vma, address);
+ /* It's better to call commit-charge after rmap is established */
+ mem_cgroup_commit_charge_swapin(page, ptr);
swap_free(entry);
if (vm_swap_full() || (vma->vm_flags & VM_LOCKED) || PageMlocked(page))
@@ -2480,7 +2494,7 @@
out:
return ret;
out_nomap:
- mem_cgroup_uncharge_page(page);
+ mem_cgroup_cancel_charge_swapin(ptr);
pte_unmap_unlock(page_table, ptl);
unlock_page(page);
page_cache_release(page);
@@ -2510,7 +2524,7 @@
goto oom;
__SetPageUptodate(page);
- if (mem_cgroup_charge(page, mm, GFP_KERNEL))
+ if (mem_cgroup_newpage_charge(page, mm, GFP_KERNEL))
goto oom_free_page;
entry = mk_pte(page, vma->vm_page_prot);
@@ -2601,7 +2615,7 @@
ret = VM_FAULT_OOM;
goto out;
}
- if (mem_cgroup_charge(page, mm, GFP_KERNEL)) {
+ if (mem_cgroup_newpage_charge(page, mm, GFP_KERNEL)) {
ret = VM_FAULT_OOM;
page_cache_release(page);
goto out;
diff --git a/mm/migrate.c b/mm/migrate.c
index 5537398..a30ea5f 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -121,20 +121,6 @@
if (!is_migration_entry(entry) || migration_entry_to_page(entry) != old)
goto out;
- /*
- * Yes, ignore the return value from a GFP_ATOMIC mem_cgroup_charge.
- * Failure is not an option here: we're now expected to remove every
- * migration pte, and will cause crashes otherwise. Normally this
- * is not an issue: mem_cgroup_prepare_migration bumped up the old
- * page_cgroup count for safety, that's now attached to the new page,
- * so this charge should just be another incrementation of the count,
- * to keep in balance with rmap.c's mem_cgroup_uncharging. But if
- * there's been a force_empty, those reference counts may no longer
- * be reliable, and this charge can actually fail: oh well, we don't
- * make the situation any worse by proceeding as if it had succeeded.
- */
- mem_cgroup_charge(new, mm, GFP_ATOMIC);
-
get_page(new);
pte = pte_mkold(mk_pte(new, vma->vm_page_prot));
if (is_write_migration_entry(entry))
@@ -378,9 +364,6 @@
anon = PageAnon(page);
page->mapping = NULL;
- if (!anon) /* This page was removed from radix-tree. */
- mem_cgroup_uncharge_cache_page(page);
-
/*
* If any waiters have accumulated on the new page then
* wake them up.
@@ -614,6 +597,7 @@
struct page *newpage = get_new_page(page, private, &result);
int rcu_locked = 0;
int charge = 0;
+ struct mem_cgroup *mem;
if (!newpage)
return -ENOMEM;
@@ -623,24 +607,26 @@
goto move_newpage;
}
- charge = mem_cgroup_prepare_migration(page, newpage);
- if (charge == -ENOMEM) {
- rc = -ENOMEM;
- goto move_newpage;
- }
/* prepare cgroup just returns 0 or -ENOMEM */
- BUG_ON(charge);
-
rc = -EAGAIN;
+
if (!trylock_page(page)) {
if (!force)
goto move_newpage;
lock_page(page);
}
+ /* charge against new page */
+ charge = mem_cgroup_prepare_migration(page, &mem);
+ if (charge == -ENOMEM) {
+ rc = -ENOMEM;
+ goto unlock;
+ }
+ BUG_ON(charge);
+
if (PageWriteback(page)) {
if (!force)
- goto unlock;
+ goto uncharge;
wait_on_page_writeback(page);
}
/*
@@ -693,7 +679,9 @@
rcu_unlock:
if (rcu_locked)
rcu_read_unlock();
-
+uncharge:
+ if (!charge)
+ mem_cgroup_end_migration(mem, page, newpage);
unlock:
unlock_page(page);
@@ -709,8 +697,6 @@
}
move_newpage:
- if (!charge)
- mem_cgroup_end_migration(newpage);
/*
* Move the new page to the LRU. If migration was not successful
diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index 6b9e758..40ba050 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -429,7 +429,6 @@
unsigned long points = 0;
struct task_struct *p;
- cgroup_lock();
read_lock(&tasklist_lock);
retry:
p = select_bad_process(&points, mem);
@@ -444,7 +443,6 @@
goto retry;
out:
read_unlock(&tasklist_lock);
- cgroup_unlock();
}
#endif
@@ -560,6 +558,13 @@
/* Got some memory back in the last second. */
return;
+ /*
+ * If this is from memcg, oom-killer is already invoked.
+ * and not worth to go system-wide-oom.
+ */
+ if (mem_cgroup_oom_called(current))
+ goto rest_and_return;
+
if (sysctl_panic_on_oom)
panic("out of memory from page fault. panic_on_oom is selected.\n");
@@ -571,6 +576,7 @@
* Give "p" a good chance of killing itself before we
* retry to allocate memory.
*/
+rest_and_return:
if (!test_thread_flag(TIF_MEMDIE))
schedule_timeout_uninterruptible(1);
}
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 7bf22e0..5675b30 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3523,10 +3523,10 @@
INIT_LIST_HEAD(&zone->lru[l].list);
zone->lru[l].nr_scan = 0;
}
- zone->recent_rotated[0] = 0;
- zone->recent_rotated[1] = 0;
- zone->recent_scanned[0] = 0;
- zone->recent_scanned[1] = 0;
+ zone->reclaim_stat.recent_rotated[0] = 0;
+ zone->reclaim_stat.recent_rotated[1] = 0;
+ zone->reclaim_stat.recent_scanned[0] = 0;
+ zone->reclaim_stat.recent_scanned[1] = 0;
zap_zone_vm_stats(zone);
zone->flags = 0;
if (!size)
diff --git a/mm/page_cgroup.c b/mm/page_cgroup.c
index d6507a6..7006a11 100644
--- a/mm/page_cgroup.c
+++ b/mm/page_cgroup.c
@@ -8,6 +8,7 @@
#include <linux/memory.h>
#include <linux/vmalloc.h>
#include <linux/cgroup.h>
+#include <linux/swapops.h>
static void __meminit
__init_page_cgroup(struct page_cgroup *pc, unsigned long pfn)
@@ -15,6 +16,7 @@
pc->flags = 0;
pc->mem_cgroup = NULL;
pc->page = pfn_to_page(pfn);
+ INIT_LIST_HEAD(&pc->lru);
}
static unsigned long total_usage;
@@ -72,7 +74,7 @@
int nid, fail;
- if (mem_cgroup_subsys.disabled)
+ if (mem_cgroup_disabled())
return;
for_each_online_node(nid) {
@@ -103,13 +105,11 @@
/* __alloc_bootmem...() is protected by !slab_available() */
static int __init_refok init_section_page_cgroup(unsigned long pfn)
{
- struct mem_section *section;
+ struct mem_section *section = __pfn_to_section(pfn);
struct page_cgroup *base, *pc;
unsigned long table_size;
int nid, index;
- section = __pfn_to_section(pfn);
-
if (!section->page_cgroup) {
nid = page_to_nid(pfn_to_page(pfn));
table_size = sizeof(struct page_cgroup) * PAGES_PER_SECTION;
@@ -145,7 +145,6 @@
__init_page_cgroup(pc, pfn + index);
}
- section = __pfn_to_section(pfn);
section->page_cgroup = base - pfn;
total_usage += table_size;
return 0;
@@ -248,7 +247,7 @@
unsigned long pfn;
int fail = 0;
- if (mem_cgroup_subsys.disabled)
+ if (mem_cgroup_disabled())
return;
for (pfn = 0; !fail && pfn < max_pfn; pfn += PAGES_PER_SECTION) {
@@ -273,3 +272,199 @@
}
#endif
+
+
+#ifdef CONFIG_CGROUP_MEM_RES_CTLR_SWAP
+
+static DEFINE_MUTEX(swap_cgroup_mutex);
+struct swap_cgroup_ctrl {
+ struct page **map;
+ unsigned long length;
+};
+
+struct swap_cgroup_ctrl swap_cgroup_ctrl[MAX_SWAPFILES];
+
+/*
+ * This 8bytes seems big..maybe we can reduce this when we can use "id" for
+ * cgroup rather than pointer.
+ */
+struct swap_cgroup {
+ struct mem_cgroup *val;
+};
+#define SC_PER_PAGE (PAGE_SIZE/sizeof(struct swap_cgroup))
+#define SC_POS_MASK (SC_PER_PAGE - 1)
+
+/*
+ * SwapCgroup implements "lookup" and "exchange" operations.
+ * In typical usage, this swap_cgroup is accessed via memcg's charge/uncharge
+ * against SwapCache. At swap_free(), this is accessed directly from swap.
+ *
+ * This means,
+ * - we have no race in "exchange" when we're accessed via SwapCache because
+ * SwapCache(and its swp_entry) is under lock.
+ * - When called via swap_free(), there is no user of this entry and no race.
+ * Then, we don't need lock around "exchange".
+ *
+ * TODO: we can push these buffers out to HIGHMEM.
+ */
+
+/*
+ * allocate buffer for swap_cgroup.
+ */
+static int swap_cgroup_prepare(int type)
+{
+ struct page *page;
+ struct swap_cgroup_ctrl *ctrl;
+ unsigned long idx, max;
+
+ if (!do_swap_account)
+ return 0;
+ ctrl = &swap_cgroup_ctrl[type];
+
+ for (idx = 0; idx < ctrl->length; idx++) {
+ page = alloc_page(GFP_KERNEL | __GFP_ZERO);
+ if (!page)
+ goto not_enough_page;
+ ctrl->map[idx] = page;
+ }
+ return 0;
+not_enough_page:
+ max = idx;
+ for (idx = 0; idx < max; idx++)
+ __free_page(ctrl->map[idx]);
+
+ return -ENOMEM;
+}
+
+/**
+ * swap_cgroup_record - record mem_cgroup for this swp_entry.
+ * @ent: swap entry to be recorded into
+ * @mem: mem_cgroup to be recorded
+ *
+ * Returns old value at success, NULL at failure.
+ * (Of course, old value can be NULL.)
+ */
+struct mem_cgroup *swap_cgroup_record(swp_entry_t ent, struct mem_cgroup *mem)
+{
+ int type = swp_type(ent);
+ unsigned long offset = swp_offset(ent);
+ unsigned long idx = offset / SC_PER_PAGE;
+ unsigned long pos = offset & SC_POS_MASK;
+ struct swap_cgroup_ctrl *ctrl;
+ struct page *mappage;
+ struct swap_cgroup *sc;
+ struct mem_cgroup *old;
+
+ if (!do_swap_account)
+ return NULL;
+
+ ctrl = &swap_cgroup_ctrl[type];
+
+ mappage = ctrl->map[idx];
+ sc = page_address(mappage);
+ sc += pos;
+ old = sc->val;
+ sc->val = mem;
+
+ return old;
+}
+
+/**
+ * lookup_swap_cgroup - lookup mem_cgroup tied to swap entry
+ * @ent: swap entry to be looked up.
+ *
+ * Returns pointer to mem_cgroup at success. NULL at failure.
+ */
+struct mem_cgroup *lookup_swap_cgroup(swp_entry_t ent)
+{
+ int type = swp_type(ent);
+ unsigned long offset = swp_offset(ent);
+ unsigned long idx = offset / SC_PER_PAGE;
+ unsigned long pos = offset & SC_POS_MASK;
+ struct swap_cgroup_ctrl *ctrl;
+ struct page *mappage;
+ struct swap_cgroup *sc;
+ struct mem_cgroup *ret;
+
+ if (!do_swap_account)
+ return NULL;
+
+ ctrl = &swap_cgroup_ctrl[type];
+ mappage = ctrl->map[idx];
+ sc = page_address(mappage);
+ sc += pos;
+ ret = sc->val;
+ return ret;
+}
+
+int swap_cgroup_swapon(int type, unsigned long max_pages)
+{
+ void *array;
+ unsigned long array_size;
+ unsigned long length;
+ struct swap_cgroup_ctrl *ctrl;
+
+ if (!do_swap_account)
+ return 0;
+
+ length = ((max_pages/SC_PER_PAGE) + 1);
+ array_size = length * sizeof(void *);
+
+ array = vmalloc(array_size);
+ if (!array)
+ goto nomem;
+
+ memset(array, 0, array_size);
+ ctrl = &swap_cgroup_ctrl[type];
+ mutex_lock(&swap_cgroup_mutex);
+ ctrl->length = length;
+ ctrl->map = array;
+ if (swap_cgroup_prepare(type)) {
+ /* memory shortage */
+ ctrl->map = NULL;
+ ctrl->length = 0;
+ vfree(array);
+ mutex_unlock(&swap_cgroup_mutex);
+ goto nomem;
+ }
+ mutex_unlock(&swap_cgroup_mutex);
+
+ printk(KERN_INFO
+ "swap_cgroup: uses %ld bytes of vmalloc for pointer array space"
+ " and %ld bytes to hold mem_cgroup pointers on swap\n",
+ array_size, length * PAGE_SIZE);
+ printk(KERN_INFO
+ "swap_cgroup can be disabled by noswapaccount boot option.\n");
+
+ return 0;
+nomem:
+ printk(KERN_INFO "couldn't allocate enough memory for swap_cgroup.\n");
+ printk(KERN_INFO
+ "swap_cgroup can be disabled by noswapaccount boot option\n");
+ return -ENOMEM;
+}
+
+void swap_cgroup_swapoff(int type)
+{
+ int i;
+ struct swap_cgroup_ctrl *ctrl;
+
+ if (!do_swap_account)
+ return;
+
+ mutex_lock(&swap_cgroup_mutex);
+ ctrl = &swap_cgroup_ctrl[type];
+ if (ctrl->map) {
+ for (i = 0; i < ctrl->length; i++) {
+ struct page *page = ctrl->map[i];
+ if (page)
+ __free_page(page);
+ }
+ vfree(ctrl->map);
+ ctrl->map = NULL;
+ ctrl->length = 0;
+ }
+ mutex_unlock(&swap_cgroup_mutex);
+}
+
+#endif
diff --git a/mm/shmem.c b/mm/shmem.c
index 5941f98..5d0de96 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -928,7 +928,11 @@
error = 1;
if (!inode)
goto out;
- /* Precharge page using GFP_KERNEL while we can wait */
+ /*
+ * Charge page using GFP_KERNEL while we can wait.
+ * Charged back to the user(not to caller) when swap account is used.
+ * add_to_page_cache() will be called with GFP_NOWAIT.
+ */
error = mem_cgroup_cache_charge(page, current->mm, GFP_KERNEL);
if (error)
goto out;
@@ -1320,15 +1324,19 @@
} else {
shmem_swp_unmap(entry);
spin_unlock(&info->lock);
- unlock_page(swappage);
- page_cache_release(swappage);
if (error == -ENOMEM) {
/* allow reclaim from this memory cgroup */
- error = mem_cgroup_shrink_usage(current->mm,
+ error = mem_cgroup_shrink_usage(swappage,
+ current->mm,
gfp);
- if (error)
+ if (error) {
+ unlock_page(swappage);
+ page_cache_release(swappage);
goto failed;
+ }
}
+ unlock_page(swappage);
+ page_cache_release(swappage);
goto repeat;
}
} else if (sgp == SGP_READ && !filepage) {
@@ -1379,7 +1387,7 @@
/* Precharge page while we can wait, compensate after */
error = mem_cgroup_cache_charge(filepage, current->mm,
- gfp & ~__GFP_HIGHMEM);
+ GFP_KERNEL);
if (error) {
page_cache_release(filepage);
shmem_unacct_blocks(info->flags, 1);
diff --git a/mm/swap.c b/mm/swap.c
index ba2c0e8..8adb9fe 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -151,6 +151,26 @@
}
}
+static void update_page_reclaim_stat(struct zone *zone, struct page *page,
+ int file, int rotated)
+{
+ struct zone_reclaim_stat *reclaim_stat = &zone->reclaim_stat;
+ struct zone_reclaim_stat *memcg_reclaim_stat;
+
+ memcg_reclaim_stat = mem_cgroup_get_reclaim_stat_from_page(page);
+
+ reclaim_stat->recent_scanned[file]++;
+ if (rotated)
+ reclaim_stat->recent_rotated[file]++;
+
+ if (!memcg_reclaim_stat)
+ return;
+
+ memcg_reclaim_stat->recent_scanned[file]++;
+ if (rotated)
+ memcg_reclaim_stat->recent_rotated[file]++;
+}
+
/*
* FIXME: speed this up?
*/
@@ -168,10 +188,8 @@
lru += LRU_ACTIVE;
add_page_to_lru_list(zone, page, lru);
__count_vm_event(PGACTIVATE);
- mem_cgroup_move_lists(page, lru);
- zone->recent_rotated[!!file]++;
- zone->recent_scanned[!!file]++;
+ update_page_reclaim_stat(zone, page, !!file, 1);
}
spin_unlock_irq(&zone->lru_lock);
}
@@ -386,12 +404,14 @@
{
int i;
struct zone *zone = NULL;
+
VM_BUG_ON(is_unevictable_lru(lru));
for (i = 0; i < pagevec_count(pvec); i++) {
struct page *page = pvec->pages[i];
struct zone *pagezone = page_zone(page);
int file;
+ int active;
if (pagezone != zone) {
if (zone)
@@ -403,12 +423,11 @@
VM_BUG_ON(PageUnevictable(page));
VM_BUG_ON(PageLRU(page));
SetPageLRU(page);
+ active = is_active_lru(lru);
file = is_file_lru(lru);
- zone->recent_scanned[file]++;
- if (is_active_lru(lru)) {
+ if (active)
SetPageActive(page);
- zone->recent_rotated[file]++;
- }
+ update_page_reclaim_stat(zone, page, file, active);
add_page_to_lru_list(zone, page, lru);
}
if (zone)
diff --git a/mm/swap_state.c b/mm/swap_state.c
index 81c825f..3ecea98 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -17,6 +17,7 @@
#include <linux/backing-dev.h>
#include <linux/pagevec.h>
#include <linux/migrate.h>
+#include <linux/page_cgroup.h>
#include <asm/pgtable.h>
@@ -108,6 +109,8 @@
*/
void __delete_from_swap_cache(struct page *page)
{
+ swp_entry_t ent = {.val = page_private(page)};
+
VM_BUG_ON(!PageLocked(page));
VM_BUG_ON(!PageSwapCache(page));
VM_BUG_ON(PageWriteback(page));
@@ -118,6 +121,7 @@
total_swapcache_pages--;
__dec_zone_page_state(page, NR_FILE_PAGES);
INC_CACHE_INFO(del_total);
+ mem_cgroup_uncharge_swapcache(page, ent);
}
/**
diff --git a/mm/swapfile.c b/mm/swapfile.c
index eec5ca7..da422c4 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -33,6 +33,7 @@
#include <asm/pgtable.h>
#include <asm/tlbflush.h>
#include <linux/swapops.h>
+#include <linux/page_cgroup.h>
static DEFINE_SPINLOCK(swap_lock);
static unsigned int nr_swapfiles;
@@ -470,8 +471,9 @@
return NULL;
}
-static int swap_entry_free(struct swap_info_struct *p, unsigned long offset)
+static int swap_entry_free(struct swap_info_struct *p, swp_entry_t ent)
{
+ unsigned long offset = swp_offset(ent);
int count = p->swap_map[offset];
if (count < SWAP_MAP_MAX) {
@@ -486,6 +488,7 @@
swap_list.next = p - swap_info;
nr_swap_pages++;
p->inuse_pages--;
+ mem_cgroup_uncharge_swap(ent);
}
}
return count;
@@ -501,7 +504,7 @@
p = swap_info_get(entry);
if (p) {
- swap_entry_free(p, swp_offset(entry));
+ swap_entry_free(p, entry);
spin_unlock(&swap_lock);
}
}
@@ -581,7 +584,7 @@
p = swap_info_get(entry);
if (p) {
- if (swap_entry_free(p, swp_offset(entry)) == 1) {
+ if (swap_entry_free(p, entry) == 1) {
page = find_get_page(&swapper_space, entry.val);
if (page && !trylock_page(page)) {
page_cache_release(page);
@@ -690,17 +693,18 @@
static int unuse_pte(struct vm_area_struct *vma, pmd_t *pmd,
unsigned long addr, swp_entry_t entry, struct page *page)
{
+ struct mem_cgroup *ptr = NULL;
spinlock_t *ptl;
pte_t *pte;
int ret = 1;
- if (mem_cgroup_charge(page, vma->vm_mm, GFP_KERNEL))
+ if (mem_cgroup_try_charge_swapin(vma->vm_mm, page, GFP_KERNEL, &ptr))
ret = -ENOMEM;
pte = pte_offset_map_lock(vma->vm_mm, pmd, addr, &ptl);
if (unlikely(!pte_same(*pte, swp_entry_to_pte(entry)))) {
if (ret > 0)
- mem_cgroup_uncharge_page(page);
+ mem_cgroup_cancel_charge_swapin(ptr);
ret = 0;
goto out;
}
@@ -710,6 +714,7 @@
set_pte_at(vma->vm_mm, addr, pte,
pte_mkold(mk_pte(page, vma->vm_page_prot)));
page_add_anon_rmap(page, vma, addr);
+ mem_cgroup_commit_charge_swapin(page, ptr);
swap_free(entry);
/*
* Move the page to the active list so it is not
@@ -1492,6 +1497,9 @@
spin_unlock(&swap_lock);
mutex_unlock(&swapon_mutex);
vfree(swap_map);
+ /* Destroy swap account informatin */
+ swap_cgroup_swapoff(type);
+
inode = mapping->host;
if (S_ISBLK(inode->i_mode)) {
struct block_device *bdev = I_BDEV(inode);
@@ -1809,6 +1817,11 @@
}
swap_map[page_nr] = SWAP_MAP_BAD;
}
+
+ error = swap_cgroup_swapon(type, maxpages);
+ if (error)
+ goto bad_swap;
+
nr_good_pages = swap_header->info.last_page -
swap_header->info.nr_badpages -
1 /* header page */;
@@ -1880,6 +1893,7 @@
bd_release(bdev);
}
destroy_swap_extents(p);
+ swap_cgroup_swapoff(type);
bad_swap_2:
spin_lock(&swap_lock);
p->swap_file = NULL;
diff --git a/mm/vmscan.c b/mm/vmscan.c
index b07c48b..9a27c44 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -125,11 +125,30 @@
static DECLARE_RWSEM(shrinker_rwsem);
#ifdef CONFIG_CGROUP_MEM_RES_CTLR
-#define scan_global_lru(sc) (!(sc)->mem_cgroup)
+#define scanning_global_lru(sc) (!(sc)->mem_cgroup)
#else
-#define scan_global_lru(sc) (1)
+#define scanning_global_lru(sc) (1)
#endif
+static struct zone_reclaim_stat *get_reclaim_stat(struct zone *zone,
+ struct scan_control *sc)
+{
+ if (!scanning_global_lru(sc))
+ return mem_cgroup_get_reclaim_stat(sc->mem_cgroup, zone);
+
+ return &zone->reclaim_stat;
+}
+
+static unsigned long zone_nr_pages(struct zone *zone, struct scan_control *sc,
+ enum lru_list lru)
+{
+ if (!scanning_global_lru(sc))
+ return mem_cgroup_zone_nr_pages(sc->mem_cgroup, zone, lru);
+
+ return zone_page_state(zone, NR_LRU_BASE + lru);
+}
+
+
/*
* Add a shrinker callback to be called from the vm
*/
@@ -512,7 +531,6 @@
lru = LRU_UNEVICTABLE;
add_page_to_unevictable_list(page);
}
- mem_cgroup_move_lists(page, lru);
/*
* page's status can change while we move it among lru. If an evictable
@@ -547,7 +565,6 @@
lru = !!TestClearPageActive(page) + page_is_file_cache(page);
lru_cache_add_lru(page, lru);
- mem_cgroup_move_lists(page, lru);
put_page(page);
}
#endif /* CONFIG_UNEVICTABLE_LRU */
@@ -813,6 +830,7 @@
return ret;
ret = -EBUSY;
+
if (likely(get_page_unless_zero(page))) {
/*
* Be careful not to clear PageLRU until after we're
@@ -821,6 +839,7 @@
*/
ClearPageLRU(page);
ret = 0;
+ mem_cgroup_del_lru(page);
}
return ret;
@@ -1029,6 +1048,7 @@
struct pagevec pvec;
unsigned long nr_scanned = 0;
unsigned long nr_reclaimed = 0;
+ struct zone_reclaim_stat *reclaim_stat = get_reclaim_stat(zone, sc);
pagevec_init(&pvec, 1);
@@ -1070,13 +1090,14 @@
__mod_zone_page_state(zone, NR_INACTIVE_ANON,
-count[LRU_INACTIVE_ANON]);
- if (scan_global_lru(sc)) {
+ if (scanning_global_lru(sc))
zone->pages_scanned += nr_scan;
- zone->recent_scanned[0] += count[LRU_INACTIVE_ANON];
- zone->recent_scanned[0] += count[LRU_ACTIVE_ANON];
- zone->recent_scanned[1] += count[LRU_INACTIVE_FILE];
- zone->recent_scanned[1] += count[LRU_ACTIVE_FILE];
- }
+
+ reclaim_stat->recent_scanned[0] += count[LRU_INACTIVE_ANON];
+ reclaim_stat->recent_scanned[0] += count[LRU_ACTIVE_ANON];
+ reclaim_stat->recent_scanned[1] += count[LRU_INACTIVE_FILE];
+ reclaim_stat->recent_scanned[1] += count[LRU_ACTIVE_FILE];
+
spin_unlock_irq(&zone->lru_lock);
nr_scanned += nr_scan;
@@ -1108,7 +1129,7 @@
if (current_is_kswapd()) {
__count_zone_vm_events(PGSCAN_KSWAPD, zone, nr_scan);
__count_vm_events(KSWAPD_STEAL, nr_freed);
- } else if (scan_global_lru(sc))
+ } else if (scanning_global_lru(sc))
__count_zone_vm_events(PGSCAN_DIRECT, zone, nr_scan);
__count_zone_vm_events(PGSTEAL, zone, nr_freed);
@@ -1134,10 +1155,9 @@
SetPageLRU(page);
lru = page_lru(page);
add_page_to_lru_list(zone, page, lru);
- mem_cgroup_move_lists(page, lru);
- if (PageActive(page) && scan_global_lru(sc)) {
+ if (PageActive(page)) {
int file = !!page_is_file_cache(page);
- zone->recent_rotated[file]++;
+ reclaim_stat->recent_rotated[file]++;
}
if (!pagevec_add(&pvec, page)) {
spin_unlock_irq(&zone->lru_lock);
@@ -1197,6 +1217,7 @@
struct page *page;
struct pagevec pvec;
enum lru_list lru;
+ struct zone_reclaim_stat *reclaim_stat = get_reclaim_stat(zone, sc);
lru_add_drain();
spin_lock_irq(&zone->lru_lock);
@@ -1207,10 +1228,10 @@
* zone->pages_scanned is used for detect zone's oom
* mem_cgroup remembers nr_scan by itself.
*/
- if (scan_global_lru(sc)) {
+ if (scanning_global_lru(sc)) {
zone->pages_scanned += pgscanned;
- zone->recent_scanned[!!file] += pgmoved;
}
+ reclaim_stat->recent_scanned[!!file] += pgmoved;
if (file)
__mod_zone_page_state(zone, NR_ACTIVE_FILE, -pgmoved);
@@ -1251,8 +1272,7 @@
* This helps balance scan pressure between file and anonymous
* pages in get_scan_ratio.
*/
- if (scan_global_lru(sc))
- zone->recent_rotated[!!file] += pgmoved;
+ reclaim_stat->recent_rotated[!!file] += pgmoved;
while (!list_empty(&l_inactive)) {
page = lru_to_page(&l_inactive);
@@ -1263,7 +1283,7 @@
ClearPageActive(page);
list_move(&page->lru, &zone->lru[lru].list);
- mem_cgroup_move_lists(page, lru);
+ mem_cgroup_add_lru_list(page, lru);
pgmoved++;
if (!pagevec_add(&pvec, page)) {
__mod_zone_page_state(zone, NR_LRU_BASE + lru, pgmoved);
@@ -1292,6 +1312,38 @@
pagevec_release(&pvec);
}
+static int inactive_anon_is_low_global(struct zone *zone)
+{
+ unsigned long active, inactive;
+
+ active = zone_page_state(zone, NR_ACTIVE_ANON);
+ inactive = zone_page_state(zone, NR_INACTIVE_ANON);
+
+ if (inactive * zone->inactive_ratio < active)
+ return 1;
+
+ return 0;
+}
+
+/**
+ * inactive_anon_is_low - check if anonymous pages need to be deactivated
+ * @zone: zone to check
+ * @sc: scan control of this context
+ *
+ * Returns true if the zone does not have enough inactive anon pages,
+ * meaning some active anon pages need to be deactivated.
+ */
+static int inactive_anon_is_low(struct zone *zone, struct scan_control *sc)
+{
+ int low;
+
+ if (scanning_global_lru(sc))
+ low = inactive_anon_is_low_global(zone);
+ else
+ low = mem_cgroup_inactive_anon_is_low(sc->mem_cgroup);
+ return low;
+}
+
static unsigned long shrink_list(enum lru_list lru, unsigned long nr_to_scan,
struct zone *zone, struct scan_control *sc, int priority)
{
@@ -1302,8 +1354,7 @@
return 0;
}
- if (lru == LRU_ACTIVE_ANON &&
- (!scan_global_lru(sc) || inactive_anon_is_low(zone))) {
+ if (lru == LRU_ACTIVE_ANON && inactive_anon_is_low(zone, sc)) {
shrink_active_list(nr_to_scan, zone, sc, priority, file);
return 0;
}
@@ -1325,6 +1376,7 @@
unsigned long anon, file, free;
unsigned long anon_prio, file_prio;
unsigned long ap, fp;
+ struct zone_reclaim_stat *reclaim_stat = get_reclaim_stat(zone, sc);
/* If we have no swap space, do not bother scanning anon pages. */
if (nr_swap_pages <= 0) {
@@ -1333,17 +1385,20 @@
return;
}
- anon = zone_page_state(zone, NR_ACTIVE_ANON) +
- zone_page_state(zone, NR_INACTIVE_ANON);
- file = zone_page_state(zone, NR_ACTIVE_FILE) +
- zone_page_state(zone, NR_INACTIVE_FILE);
- free = zone_page_state(zone, NR_FREE_PAGES);
+ anon = zone_nr_pages(zone, sc, LRU_ACTIVE_ANON) +
+ zone_nr_pages(zone, sc, LRU_INACTIVE_ANON);
+ file = zone_nr_pages(zone, sc, LRU_ACTIVE_FILE) +
+ zone_nr_pages(zone, sc, LRU_INACTIVE_FILE);
- /* If we have very few page cache pages, force-scan anon pages. */
- if (unlikely(file + free <= zone->pages_high)) {
- percent[0] = 100;
- percent[1] = 0;
- return;
+ if (scanning_global_lru(sc)) {
+ free = zone_page_state(zone, NR_FREE_PAGES);
+ /* If we have very few page cache pages,
+ force-scan anon pages. */
+ if (unlikely(file + free <= zone->pages_high)) {
+ percent[0] = 100;
+ percent[1] = 0;
+ return;
+ }
}
/*
@@ -1357,17 +1412,17 @@
*
* anon in [0], file in [1]
*/
- if (unlikely(zone->recent_scanned[0] > anon / 4)) {
+ if (unlikely(reclaim_stat->recent_scanned[0] > anon / 4)) {
spin_lock_irq(&zone->lru_lock);
- zone->recent_scanned[0] /= 2;
- zone->recent_rotated[0] /= 2;
+ reclaim_stat->recent_scanned[0] /= 2;
+ reclaim_stat->recent_rotated[0] /= 2;
spin_unlock_irq(&zone->lru_lock);
}
- if (unlikely(zone->recent_scanned[1] > file / 4)) {
+ if (unlikely(reclaim_stat->recent_scanned[1] > file / 4)) {
spin_lock_irq(&zone->lru_lock);
- zone->recent_scanned[1] /= 2;
- zone->recent_rotated[1] /= 2;
+ reclaim_stat->recent_scanned[1] /= 2;
+ reclaim_stat->recent_rotated[1] /= 2;
spin_unlock_irq(&zone->lru_lock);
}
@@ -1383,11 +1438,11 @@
* proportional to the fraction of recently scanned pages on
* each list that were recently referenced and in active use.
*/
- ap = (anon_prio + 1) * (zone->recent_scanned[0] + 1);
- ap /= zone->recent_rotated[0] + 1;
+ ap = (anon_prio + 1) * (reclaim_stat->recent_scanned[0] + 1);
+ ap /= reclaim_stat->recent_rotated[0] + 1;
- fp = (file_prio + 1) * (zone->recent_scanned[1] + 1);
- fp /= zone->recent_rotated[1] + 1;
+ fp = (file_prio + 1) * (reclaim_stat->recent_scanned[1] + 1);
+ fp /= reclaim_stat->recent_rotated[1] + 1;
/* Normalize to percentages */
percent[0] = 100 * ap / (ap + fp + 1);
@@ -1411,30 +1466,23 @@
get_scan_ratio(zone, sc, percent);
for_each_evictable_lru(l) {
- if (scan_global_lru(sc)) {
- int file = is_file_lru(l);
- int scan;
+ int file = is_file_lru(l);
+ int scan;
- scan = zone_page_state(zone, NR_LRU_BASE + l);
- if (priority) {
- scan >>= priority;
- scan = (scan * percent[file]) / 100;
- }
+ scan = zone_page_state(zone, NR_LRU_BASE + l);
+ if (priority) {
+ scan >>= priority;
+ scan = (scan * percent[file]) / 100;
+ }
+ if (scanning_global_lru(sc)) {
zone->lru[l].nr_scan += scan;
nr[l] = zone->lru[l].nr_scan;
if (nr[l] >= swap_cluster_max)
zone->lru[l].nr_scan = 0;
else
nr[l] = 0;
- } else {
- /*
- * This reclaim occurs not because zone memory shortage
- * but because memory controller hits its limit.
- * Don't modify zone reclaim related data.
- */
- nr[l] = mem_cgroup_calc_reclaim(sc->mem_cgroup, zone,
- priority, l);
- }
+ } else
+ nr[l] = scan;
}
while (nr[LRU_INACTIVE_ANON] || nr[LRU_ACTIVE_FILE] ||
@@ -1467,9 +1515,7 @@
* Even if we did not try to evict anon pages at all, we want to
* rebalance the anon lru active/inactive ratio.
*/
- if (!scan_global_lru(sc) || inactive_anon_is_low(zone))
- shrink_active_list(SWAP_CLUSTER_MAX, zone, sc, priority, 0);
- else if (!scan_global_lru(sc))
+ if (inactive_anon_is_low(zone, sc))
shrink_active_list(SWAP_CLUSTER_MAX, zone, sc, priority, 0);
throttle_vm_writeout(sc->gfp_mask);
@@ -1504,7 +1550,7 @@
* Take care memory controller reclaiming has small influence
* to global LRU.
*/
- if (scan_global_lru(sc)) {
+ if (scanning_global_lru(sc)) {
if (!cpuset_zone_allowed_hardwall(zone, GFP_KERNEL))
continue;
note_zone_scanning_priority(zone, priority);
@@ -1557,12 +1603,12 @@
delayacct_freepages_start();
- if (scan_global_lru(sc))
+ if (scanning_global_lru(sc))
count_vm_event(ALLOCSTALL);
/*
* mem_cgroup will not do shrink_slab.
*/
- if (scan_global_lru(sc)) {
+ if (scanning_global_lru(sc)) {
for_each_zone_zonelist(zone, z, zonelist, high_zoneidx) {
if (!cpuset_zone_allowed_hardwall(zone, GFP_KERNEL))
@@ -1581,7 +1627,7 @@
* Don't shrink slabs when reclaiming memory from
* over limit cgroups
*/
- if (scan_global_lru(sc)) {
+ if (scanning_global_lru(sc)) {
shrink_slab(sc->nr_scanned, sc->gfp_mask, lru_pages);
if (reclaim_state) {
sc->nr_reclaimed += reclaim_state->reclaimed_slab;
@@ -1612,7 +1658,7 @@
congestion_wait(WRITE, HZ/10);
}
/* top priority shrink_zones still had more to do? don't OOM, then */
- if (!sc->all_unreclaimable && scan_global_lru(sc))
+ if (!sc->all_unreclaimable && scanning_global_lru(sc))
ret = sc->nr_reclaimed;
out:
/*
@@ -1625,7 +1671,7 @@
if (priority < 0)
priority = 0;
- if (scan_global_lru(sc)) {
+ if (scanning_global_lru(sc)) {
for_each_zone_zonelist(zone, z, zonelist, high_zoneidx) {
if (!cpuset_zone_allowed_hardwall(zone, GFP_KERNEL))
@@ -1661,19 +1707,24 @@
#ifdef CONFIG_CGROUP_MEM_RES_CTLR
unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *mem_cont,
- gfp_t gfp_mask)
+ gfp_t gfp_mask,
+ bool noswap,
+ unsigned int swappiness)
{
struct scan_control sc = {
.may_writepage = !laptop_mode,
.may_swap = 1,
.swap_cluster_max = SWAP_CLUSTER_MAX,
- .swappiness = vm_swappiness,
+ .swappiness = swappiness,
.order = 0,
.mem_cgroup = mem_cont,
.isolate_pages = mem_cgroup_isolate_pages,
};
struct zonelist *zonelist;
+ if (noswap)
+ sc.may_swap = 0;
+
sc.gfp_mask = (gfp_mask & GFP_RECLAIM_MASK) |
(GFP_HIGHUSER_MOVABLE & ~GFP_RECLAIM_MASK);
zonelist = NODE_DATA(numa_node_id())->node_zonelists;
@@ -1761,7 +1812,7 @@
* Do some background aging of the anon list, to give
* pages a chance to be referenced before reclaiming.
*/
- if (inactive_anon_is_low(zone))
+ if (inactive_anon_is_low(zone, &sc))
shrink_active_list(SWAP_CLUSTER_MAX, zone,
&sc, priority, 0);
@@ -2404,6 +2455,7 @@
__dec_zone_state(zone, NR_UNEVICTABLE);
list_move(&page->lru, &zone->lru[l].list);
+ mem_cgroup_move_lists(page, LRU_UNEVICTABLE, l);
__inc_zone_state(zone, NR_INACTIVE_ANON + l);
__count_vm_event(UNEVICTABLE_PGRESCUED);
} else {
@@ -2412,6 +2464,7 @@
*/
SetPageUnevictable(page);
list_move(&page->lru, &zone->lru[LRU_UNEVICTABLE].list);
+ mem_cgroup_rotate_lru_list(page, LRU_UNEVICTABLE);
if (page_evictable(page, NULL))
goto retry;
}
diff --git a/net/8021q/vlan_core.c b/net/8021q/vlan_core.c
index dd86a1d..6c13239 100644
--- a/net/8021q/vlan_core.c
+++ b/net/8021q/vlan_core.c
@@ -3,46 +3,35 @@
#include <linux/if_vlan.h>
#include "vlan.h"
-struct vlan_hwaccel_cb {
- struct net_device *dev;
-};
-
-static inline struct vlan_hwaccel_cb *vlan_hwaccel_cb(struct sk_buff *skb)
-{
- return (struct vlan_hwaccel_cb *)skb->cb;
-}
-
/* VLAN rx hw acceleration helper. This acts like netif_{rx,receive_skb}(). */
int __vlan_hwaccel_rx(struct sk_buff *skb, struct vlan_group *grp,
u16 vlan_tci, int polling)
{
- struct vlan_hwaccel_cb *cb = vlan_hwaccel_cb(skb);
-
- if (skb_bond_should_drop(skb)) {
- dev_kfree_skb_any(skb);
- return NET_RX_DROP;
- }
+ if (skb_bond_should_drop(skb))
+ goto drop;
skb->vlan_tci = vlan_tci;
- cb->dev = vlan_group_get_device(grp, vlan_tci & VLAN_VID_MASK);
+ skb->dev = vlan_group_get_device(grp, vlan_tci & VLAN_VID_MASK);
+
+ if (!skb->dev)
+ goto drop;
return (polling ? netif_receive_skb(skb) : netif_rx(skb));
+
+drop:
+ dev_kfree_skb_any(skb);
+ return NET_RX_DROP;
}
EXPORT_SYMBOL(__vlan_hwaccel_rx);
int vlan_hwaccel_do_receive(struct sk_buff *skb)
{
- struct vlan_hwaccel_cb *cb = vlan_hwaccel_cb(skb);
- struct net_device *dev = cb->dev;
+ struct net_device *dev = skb->dev;
struct net_device_stats *stats;
+ skb->dev = vlan_dev_info(dev)->real_dev;
netif_nit_deliver(skb);
- if (dev == NULL) {
- kfree_skb(skb);
- return -1;
- }
-
skb->dev = dev;
skb->priority = vlan_get_ingress_priority(dev, skb->vlan_tci);
skb->vlan_tci = 0;
@@ -80,3 +69,79 @@
return vlan_dev_info(dev)->vlan_id;
}
EXPORT_SYMBOL_GPL(vlan_dev_vlan_id);
+
+static int vlan_gro_common(struct napi_struct *napi, struct vlan_group *grp,
+ unsigned int vlan_tci, struct sk_buff *skb)
+{
+ struct sk_buff *p;
+
+ if (skb_bond_should_drop(skb))
+ goto drop;
+
+ skb->vlan_tci = vlan_tci;
+ skb->dev = vlan_group_get_device(grp, vlan_tci & VLAN_VID_MASK);
+
+ if (!skb->dev)
+ goto drop;
+
+ for (p = napi->gro_list; p; p = p->next) {
+ NAPI_GRO_CB(p)->same_flow = p->dev == skb->dev;
+ NAPI_GRO_CB(p)->flush = 0;
+ }
+
+ return dev_gro_receive(napi, skb);
+
+drop:
+ return 2;
+}
+
+int vlan_gro_receive(struct napi_struct *napi, struct vlan_group *grp,
+ unsigned int vlan_tci, struct sk_buff *skb)
+{
+ int err = NET_RX_SUCCESS;
+
+ switch (vlan_gro_common(napi, grp, vlan_tci, skb)) {
+ case -1:
+ return netif_receive_skb(skb);
+
+ case 2:
+ err = NET_RX_DROP;
+ /* fall through */
+
+ case 1:
+ kfree_skb(skb);
+ break;
+ }
+
+ return err;
+}
+EXPORT_SYMBOL(vlan_gro_receive);
+
+int vlan_gro_frags(struct napi_struct *napi, struct vlan_group *grp,
+ unsigned int vlan_tci, struct napi_gro_fraginfo *info)
+{
+ struct sk_buff *skb = napi_fraginfo_skb(napi, info);
+ int err = NET_RX_DROP;
+
+ if (!skb)
+ goto out;
+
+ err = NET_RX_SUCCESS;
+
+ switch (vlan_gro_common(napi, grp, vlan_tci, skb)) {
+ case -1:
+ return netif_receive_skb(skb);
+
+ case 2:
+ err = NET_RX_DROP;
+ /* fall through */
+
+ case 1:
+ napi_reuse_skb(napi, skb);
+ break;
+ }
+
+out:
+ return err;
+}
+EXPORT_SYMBOL(vlan_gro_frags);
diff --git a/net/8021q/vlan_dev.c b/net/8021q/vlan_dev.c
index 89a3bbd..4a19acd 100644
--- a/net/8021q/vlan_dev.c
+++ b/net/8021q/vlan_dev.c
@@ -546,6 +546,18 @@
return err;
}
+static int vlan_dev_neigh_setup(struct net_device *dev, struct neigh_parms *pa)
+{
+ struct net_device *real_dev = vlan_dev_info(dev)->real_dev;
+ const struct net_device_ops *ops = real_dev->netdev_ops;
+ int err = 0;
+
+ if (netif_device_present(real_dev) && ops->ndo_neigh_setup)
+ err = ops->ndo_neigh_setup(dev, pa);
+
+ return err;
+}
+
static void vlan_dev_change_rx_flags(struct net_device *dev, int change)
{
struct net_device *real_dev = vlan_dev_info(dev)->real_dev;
@@ -713,6 +725,7 @@
.ndo_set_multicast_list = vlan_dev_set_rx_mode,
.ndo_change_rx_flags = vlan_dev_change_rx_flags,
.ndo_do_ioctl = vlan_dev_ioctl,
+ .ndo_neigh_setup = vlan_dev_neigh_setup,
};
static const struct net_device_ops vlan_netdev_accel_ops = {
@@ -728,6 +741,7 @@
.ndo_set_multicast_list = vlan_dev_set_rx_mode,
.ndo_change_rx_flags = vlan_dev_change_rx_flags,
.ndo_do_ioctl = vlan_dev_ioctl,
+ .ndo_neigh_setup = vlan_dev_neigh_setup,
};
void vlan_setup(struct net_device *dev)
diff --git a/net/appletalk/aarp.c b/net/appletalk/aarp.c
index b03ff58..89f99d3 100644
--- a/net/appletalk/aarp.c
+++ b/net/appletalk/aarp.c
@@ -443,13 +443,14 @@
{
struct ifreq atreq;
struct sockaddr_at *sa = (struct sockaddr_at *)&atreq.ifr_addr;
+ const struct net_device_ops *ops = iface->dev->netdev_ops;
sa->sat_addr.s_node = iface->address.s_node;
sa->sat_addr.s_net = ntohs(iface->address.s_net);
/* We pass the Net:Node to the drivers/cards by a Device ioctl. */
- if (!(iface->dev->do_ioctl(iface->dev, &atreq, SIOCSIFADDR))) {
- (void)iface->dev->do_ioctl(iface->dev, &atreq, SIOCGIFADDR);
+ if (!(ops->ndo_do_ioctl(iface->dev, &atreq, SIOCSIFADDR))) {
+ ops->ndo_do_ioctl(iface->dev, &atreq, SIOCGIFADDR);
if (iface->address.s_net != htons(sa->sat_addr.s_net) ||
iface->address.s_node != sa->sat_addr.s_node)
iface->status |= ATIF_PROBE_FAIL;
diff --git a/net/bluetooth/bnep/bnep.h b/net/bluetooth/bnep/bnep.h
index d20f8a4..0d9e506 100644
--- a/net/bluetooth/bnep/bnep.h
+++ b/net/bluetooth/bnep/bnep.h
@@ -165,7 +165,6 @@
struct socket *sock;
struct net_device *dev;
- struct net_device_stats stats;
};
void bnep_net_setup(struct net_device *dev);
diff --git a/net/bluetooth/bnep/core.c b/net/bluetooth/bnep/core.c
index 70fea8b..52a6ce0 100644
--- a/net/bluetooth/bnep/core.c
+++ b/net/bluetooth/bnep/core.c
@@ -306,7 +306,7 @@
struct sk_buff *nskb;
u8 type;
- s->stats.rx_bytes += skb->len;
+ dev->stats.rx_bytes += skb->len;
type = *(u8 *) skb->data; skb_pull(skb, 1);
@@ -343,7 +343,7 @@
* may not be modified and because of the alignment requirements. */
nskb = alloc_skb(2 + ETH_HLEN + skb->len, GFP_KERNEL);
if (!nskb) {
- s->stats.rx_dropped++;
+ dev->stats.rx_dropped++;
kfree_skb(skb);
return -ENOMEM;
}
@@ -378,14 +378,14 @@
skb_copy_from_linear_data(skb, __skb_put(nskb, skb->len), skb->len);
kfree_skb(skb);
- s->stats.rx_packets++;
+ dev->stats.rx_packets++;
nskb->ip_summed = CHECKSUM_NONE;
nskb->protocol = eth_type_trans(nskb, dev);
netif_rx_ni(nskb);
return 0;
badframe:
- s->stats.rx_errors++;
+ dev->stats.rx_errors++;
kfree_skb(skb);
return 0;
}
@@ -448,8 +448,8 @@
kfree_skb(skb);
if (len > 0) {
- s->stats.tx_bytes += len;
- s->stats.tx_packets++;
+ s->dev->stats.tx_bytes += len;
+ s->dev->stats.tx_packets++;
return 0;
}
diff --git a/net/bluetooth/bnep/netdev.c b/net/bluetooth/bnep/netdev.c
index f897da6..d7a0e97 100644
--- a/net/bluetooth/bnep/netdev.c
+++ b/net/bluetooth/bnep/netdev.c
@@ -55,12 +55,6 @@
return 0;
}
-static struct net_device_stats *bnep_net_get_stats(struct net_device *dev)
-{
- struct bnep_session *s = netdev_priv(dev);
- return &s->stats;
-}
-
static void bnep_net_set_mc_list(struct net_device *dev)
{
#ifdef CONFIG_BT_BNEP_MC_FILTER
@@ -128,11 +122,6 @@
netif_wake_queue(dev);
}
-static int bnep_net_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd)
-{
- return -EINVAL;
-}
-
#ifdef CONFIG_BT_BNEP_MC_FILTER
static inline int bnep_net_mc_filter(struct sk_buff *skb, struct bnep_session *s)
{
@@ -217,6 +206,18 @@
return 0;
}
+static const struct net_device_ops bnep_netdev_ops = {
+ .ndo_open = bnep_net_open,
+ .ndo_stop = bnep_net_close,
+ .ndo_start_xmit = bnep_net_xmit,
+ .ndo_validate_addr = eth_validate_addr,
+ .ndo_set_multicast_list = bnep_net_set_mc_list,
+ .ndo_set_mac_address = bnep_net_set_mac_addr,
+ .ndo_tx_timeout = bnep_net_timeout,
+ .ndo_change_mtu = eth_change_mtu,
+
+};
+
void bnep_net_setup(struct net_device *dev)
{
@@ -224,15 +225,7 @@
dev->addr_len = ETH_ALEN;
ether_setup(dev);
-
- dev->open = bnep_net_open;
- dev->stop = bnep_net_close;
- dev->hard_start_xmit = bnep_net_xmit;
- dev->get_stats = bnep_net_get_stats;
- dev->do_ioctl = bnep_net_ioctl;
- dev->set_mac_address = bnep_net_set_mac_addr;
- dev->set_multicast_list = bnep_net_set_mc_list;
+ dev->netdev_ops = &bnep_netdev_ops;
dev->watchdog_timeo = HZ * 2;
- dev->tx_timeout = bnep_net_timeout;
}
diff --git a/net/can/af_can.c b/net/can/af_can.c
index 3dadb33..fa417ca 100644
--- a/net/can/af_can.c
+++ b/net/can/af_can.c
@@ -414,6 +414,12 @@
* The filter can be inverted (CAN_INV_FILTER bit set in can_id) or it can
* filter for error frames (CAN_ERR_FLAG bit set in mask).
*
+ * The provided pointer to the sk_buff is guaranteed to be valid as long as
+ * the callback function is running. The callback function must *not* free
+ * the given sk_buff while processing it's task. When the given sk_buff is
+ * needed after the end of the callback function it must be cloned inside
+ * the callback function with skb_clone().
+ *
* Return:
* 0 on success
* -ENOMEM on missing cache mem to create subscription entry
@@ -569,13 +575,8 @@
static inline void deliver(struct sk_buff *skb, struct receiver *r)
{
- struct sk_buff *clone = skb_clone(skb, GFP_ATOMIC);
-
- if (clone) {
- clone->sk = skb->sk;
- r->func(clone, r->data);
- r->matches++;
- }
+ r->func(skb, r->data);
+ r->matches++;
}
static int can_rcv_filter(struct dev_rcv_lists *d, struct sk_buff *skb)
diff --git a/net/can/bcm.c b/net/can/bcm.c
index 6248ae2..1649c8a 100644
--- a/net/can/bcm.c
+++ b/net/can/bcm.c
@@ -633,7 +633,7 @@
hrtimer_cancel(&op->timer);
if (op->can_id != rxframe->can_id)
- goto rx_freeskb;
+ return;
/* save rx timestamp */
op->rx_stamp = skb->tstamp;
@@ -645,19 +645,19 @@
if (op->flags & RX_RTR_FRAME) {
/* send reply for RTR-request (placed in op->frames[0]) */
bcm_can_tx(op);
- goto rx_freeskb;
+ return;
}
if (op->flags & RX_FILTER_ID) {
/* the easiest case */
bcm_rx_update_and_send(op, &op->last_frames[0], rxframe);
- goto rx_freeskb_starttimer;
+ goto rx_starttimer;
}
if (op->nframes == 1) {
/* simple compare with index 0 */
bcm_rx_cmp_to_index(op, 0, rxframe);
- goto rx_freeskb_starttimer;
+ goto rx_starttimer;
}
if (op->nframes > 1) {
@@ -678,10 +678,8 @@
}
}
-rx_freeskb_starttimer:
+rx_starttimer:
bcm_rx_starttimer(op);
-rx_freeskb:
- kfree_skb(skb);
}
/*
diff --git a/net/can/raw.c b/net/can/raw.c
index 27aab63..0703cba 100644
--- a/net/can/raw.c
+++ b/net/can/raw.c
@@ -99,13 +99,14 @@
struct raw_sock *ro = raw_sk(sk);
struct sockaddr_can *addr;
- if (!ro->recv_own_msgs) {
- /* check the received tx sock reference */
- if (skb->sk == sk) {
- kfree_skb(skb);
- return;
- }
- }
+ /* check the received tx sock reference */
+ if (!ro->recv_own_msgs && skb->sk == sk)
+ return;
+
+ /* clone the given skb to be able to enqueue it into the rcv queue */
+ skb = skb_clone(skb, GFP_ATOMIC);
+ if (!skb)
+ return;
/*
* Put the datagram to the queue so that raw_recvmsg() can
diff --git a/net/core/dev.c b/net/core/dev.c
index 382df6c..5f736f1 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -170,25 +170,6 @@
static struct list_head ptype_base[PTYPE_HASH_SIZE] __read_mostly;
static struct list_head ptype_all __read_mostly; /* Taps */
-#ifdef CONFIG_NET_DMA
-struct net_dma {
- struct dma_client client;
- spinlock_t lock;
- cpumask_t channel_mask;
- struct dma_chan **channels;
-};
-
-static enum dma_state_client
-netdev_dma_event(struct dma_client *client, struct dma_chan *chan,
- enum dma_state state);
-
-static struct net_dma net_dma = {
- .client = {
- .event_callback = netdev_dma_event,
- },
-};
-#endif
-
/*
* The @dev_base_head list is protected by @dev_base_lock and the rtnl
* semaphore.
@@ -2387,7 +2368,7 @@
}
EXPORT_SYMBOL(napi_gro_flush);
-static int __napi_gro_receive(struct napi_struct *napi, struct sk_buff *skb)
+int dev_gro_receive(struct napi_struct *napi, struct sk_buff *skb)
{
struct sk_buff **pp = NULL;
struct packet_type *ptype;
@@ -2417,11 +2398,14 @@
for (p = napi->gro_list; p; p = p->next) {
count++;
- NAPI_GRO_CB(p)->same_flow =
- p->mac_len == mac_len &&
- !memcmp(skb_mac_header(p), skb_mac_header(skb),
- mac_len);
- NAPI_GRO_CB(p)->flush = 0;
+
+ if (!NAPI_GRO_CB(p)->same_flow)
+ continue;
+
+ if (p->mac_len != mac_len ||
+ memcmp(skb_mac_header(p), skb_mac_header(skb),
+ mac_len))
+ NAPI_GRO_CB(p)->same_flow = 0;
}
pp = ptype->gro_receive(&napi->gro_list, skb);
@@ -2463,6 +2447,19 @@
normal:
return -1;
}
+EXPORT_SYMBOL(dev_gro_receive);
+
+static int __napi_gro_receive(struct napi_struct *napi, struct sk_buff *skb)
+{
+ struct sk_buff *p;
+
+ for (p = napi->gro_list; p; p = p->next) {
+ NAPI_GRO_CB(p)->same_flow = 1;
+ NAPI_GRO_CB(p)->flush = 0;
+ }
+
+ return dev_gro_receive(napi, skb);
+}
int napi_gro_receive(struct napi_struct *napi, struct sk_buff *skb)
{
@@ -2479,11 +2476,26 @@
}
EXPORT_SYMBOL(napi_gro_receive);
-int napi_gro_frags(struct napi_struct *napi, struct napi_gro_fraginfo *info)
+void napi_reuse_skb(struct napi_struct *napi, struct sk_buff *skb)
+{
+ skb_shinfo(skb)->nr_frags = 0;
+
+ skb->len -= skb->data_len;
+ skb->truesize -= skb->data_len;
+ skb->data_len = 0;
+
+ __skb_pull(skb, skb_headlen(skb));
+ skb_reserve(skb, NET_IP_ALIGN - skb_headroom(skb));
+
+ napi->skb = skb;
+}
+EXPORT_SYMBOL(napi_reuse_skb);
+
+struct sk_buff *napi_fraginfo_skb(struct napi_struct *napi,
+ struct napi_gro_fraginfo *info)
{
struct net_device *dev = napi->dev;
struct sk_buff *skb = napi->skb;
- int err = NET_RX_DROP;
napi->skb = NULL;
@@ -2503,16 +2515,31 @@
skb->len += info->len;
skb->truesize += info->len;
- if (!pskb_may_pull(skb, ETH_HLEN))
- goto reuse;
-
- err = NET_RX_SUCCESS;
+ if (!pskb_may_pull(skb, ETH_HLEN)) {
+ napi_reuse_skb(napi, skb);
+ goto out;
+ }
skb->protocol = eth_type_trans(skb, dev);
skb->ip_summed = info->ip_summed;
skb->csum = info->csum;
+out:
+ return skb;
+}
+EXPORT_SYMBOL(napi_fraginfo_skb);
+
+int napi_gro_frags(struct napi_struct *napi, struct napi_gro_fraginfo *info)
+{
+ struct sk_buff *skb = napi_fraginfo_skb(napi, info);
+ int err = NET_RX_DROP;
+
+ if (!skb)
+ goto out;
+
+ err = NET_RX_SUCCESS;
+
switch (__napi_gro_receive(napi, skb)) {
case -1:
return netif_receive_skb(skb);
@@ -2521,17 +2548,7 @@
goto out;
}
-reuse:
- skb_shinfo(skb)->nr_frags = 0;
-
- skb->len -= skb->data_len;
- skb->truesize -= skb->data_len;
- skb->data_len = 0;
-
- __skb_pull(skb, skb_headlen(skb));
- skb_reserve(skb, NET_IP_ALIGN - skb_headroom(skb));
-
- napi->skb = skb;
+ napi_reuse_skb(napi, skb);
out:
return err;
@@ -2718,14 +2735,7 @@
* There may not be any more sk_buffs coming right now, so push
* any pending DMA copies to hardware
*/
- if (!cpus_empty(net_dma.channel_mask)) {
- int chan_idx;
- for_each_cpu_mask_nr(chan_idx, net_dma.channel_mask) {
- struct dma_chan *chan = net_dma.channels[chan_idx];
- if (chan)
- dma_async_memcpy_issue_pending(chan);
- }
- }
+ dma_issue_pending_all();
#endif
return;
@@ -4916,122 +4926,6 @@
return NOTIFY_OK;
}
-#ifdef CONFIG_NET_DMA
-/**
- * net_dma_rebalance - try to maintain one DMA channel per CPU
- * @net_dma: DMA client and associated data (lock, channels, channel_mask)
- *
- * This is called when the number of channels allocated to the net_dma client
- * changes. The net_dma client tries to have one DMA channel per CPU.
- */
-
-static void net_dma_rebalance(struct net_dma *net_dma)
-{
- unsigned int cpu, i, n, chan_idx;
- struct dma_chan *chan;
-
- if (cpus_empty(net_dma->channel_mask)) {
- for_each_online_cpu(cpu)
- rcu_assign_pointer(per_cpu(softnet_data, cpu).net_dma, NULL);
- return;
- }
-
- i = 0;
- cpu = first_cpu(cpu_online_map);
-
- for_each_cpu_mask_nr(chan_idx, net_dma->channel_mask) {
- chan = net_dma->channels[chan_idx];
-
- n = ((num_online_cpus() / cpus_weight(net_dma->channel_mask))
- + (i < (num_online_cpus() %
- cpus_weight(net_dma->channel_mask)) ? 1 : 0));
-
- while(n) {
- per_cpu(softnet_data, cpu).net_dma = chan;
- cpu = next_cpu(cpu, cpu_online_map);
- n--;
- }
- i++;
- }
-}
-
-/**
- * netdev_dma_event - event callback for the net_dma_client
- * @client: should always be net_dma_client
- * @chan: DMA channel for the event
- * @state: DMA state to be handled
- */
-static enum dma_state_client
-netdev_dma_event(struct dma_client *client, struct dma_chan *chan,
- enum dma_state state)
-{
- int i, found = 0, pos = -1;
- struct net_dma *net_dma =
- container_of(client, struct net_dma, client);
- enum dma_state_client ack = DMA_DUP; /* default: take no action */
-
- spin_lock(&net_dma->lock);
- switch (state) {
- case DMA_RESOURCE_AVAILABLE:
- for (i = 0; i < nr_cpu_ids; i++)
- if (net_dma->channels[i] == chan) {
- found = 1;
- break;
- } else if (net_dma->channels[i] == NULL && pos < 0)
- pos = i;
-
- if (!found && pos >= 0) {
- ack = DMA_ACK;
- net_dma->channels[pos] = chan;
- cpu_set(pos, net_dma->channel_mask);
- net_dma_rebalance(net_dma);
- }
- break;
- case DMA_RESOURCE_REMOVED:
- for (i = 0; i < nr_cpu_ids; i++)
- if (net_dma->channels[i] == chan) {
- found = 1;
- pos = i;
- break;
- }
-
- if (found) {
- ack = DMA_ACK;
- cpu_clear(pos, net_dma->channel_mask);
- net_dma->channels[i] = NULL;
- net_dma_rebalance(net_dma);
- }
- break;
- default:
- break;
- }
- spin_unlock(&net_dma->lock);
-
- return ack;
-}
-
-/**
- * netdev_dma_register - register the networking subsystem as a DMA client
- */
-static int __init netdev_dma_register(void)
-{
- net_dma.channels = kzalloc(nr_cpu_ids * sizeof(struct net_dma),
- GFP_KERNEL);
- if (unlikely(!net_dma.channels)) {
- printk(KERN_NOTICE
- "netdev_dma: no memory for net_dma.channels\n");
- return -ENOMEM;
- }
- spin_lock_init(&net_dma.lock);
- dma_cap_set(DMA_MEMCPY, net_dma.client.cap_mask);
- dma_async_client_register(&net_dma.client);
- dma_async_client_chan_request(&net_dma.client);
- return 0;
-}
-
-#else
-static int __init netdev_dma_register(void) { return -ENODEV; }
-#endif /* CONFIG_NET_DMA */
/**
* netdev_increment_features - increment feature set by one
@@ -5251,14 +5145,15 @@
if (register_pernet_device(&default_device_ops))
goto out;
- netdev_dma_register();
-
open_softirq(NET_TX_SOFTIRQ, net_tx_action);
open_softirq(NET_RX_SOFTIRQ, net_rx_action);
hotcpu_notifier(dev_cpu_callback, 0);
dst_init();
dev_mcast_init();
+ #ifdef CONFIG_NET_DMA
+ dmaengine_get();
+ #endif
rc = 0;
out:
return rc;
diff --git a/net/dsa/slave.c b/net/dsa/slave.c
index a3a410d..a68fd79 100644
--- a/net/dsa/slave.c
+++ b/net/dsa/slave.c
@@ -286,6 +286,42 @@
.get_sset_count = dsa_slave_get_sset_count,
};
+#ifdef CONFIG_NET_DSA_TAG_DSA
+static const struct net_device_ops dsa_netdev_ops = {
+ .ndo_open = dsa_slave_open,
+ .ndo_stop = dsa_slave_close,
+ .ndo_start_xmit = dsa_xmit,
+ .ndo_change_rx_flags = dsa_slave_change_rx_flags,
+ .ndo_set_rx_mode = dsa_slave_set_rx_mode,
+ .ndo_set_multicast_list = dsa_slave_set_rx_mode,
+ .ndo_set_mac_address = dsa_slave_set_mac_address,
+ .ndo_do_ioctl = dsa_slave_ioctl,
+};
+#endif
+#ifdef CONFIG_NET_DSA_TAG_EDSA
+static const struct net_device_ops edsa_netdev_ops = {
+ .ndo_open = dsa_slave_open,
+ .ndo_stop = dsa_slave_close,
+ .ndo_start_xmit = edsa_xmit,
+ .ndo_change_rx_flags = dsa_slave_change_rx_flags,
+ .ndo_set_rx_mode = dsa_slave_set_rx_mode,
+ .ndo_set_multicast_list = dsa_slave_set_rx_mode,
+ .ndo_set_mac_address = dsa_slave_set_mac_address,
+ .ndo_do_ioctl = dsa_slave_ioctl,
+};
+#endif
+#ifdef CONFIG_NET_DSA_TAG_TRAILER
+static const struct net_device_ops trailer_netdev_ops = {
+ .ndo_open = dsa_slave_open,
+ .ndo_stop = dsa_slave_close,
+ .ndo_start_xmit = trailer_xmit,
+ .ndo_change_rx_flags = dsa_slave_change_rx_flags,
+ .ndo_set_rx_mode = dsa_slave_set_rx_mode,
+ .ndo_set_multicast_list = dsa_slave_set_rx_mode,
+ .ndo_set_mac_address = dsa_slave_set_mac_address,
+ .ndo_do_ioctl = dsa_slave_ioctl,
+};
+#endif
/* slave device setup *******************************************************/
struct net_device *
@@ -306,32 +342,27 @@
SET_ETHTOOL_OPS(slave_dev, &dsa_slave_ethtool_ops);
memcpy(slave_dev->dev_addr, master->dev_addr, ETH_ALEN);
slave_dev->tx_queue_len = 0;
+
switch (ds->tag_protocol) {
#ifdef CONFIG_NET_DSA_TAG_DSA
case htons(ETH_P_DSA):
- slave_dev->hard_start_xmit = dsa_xmit;
+ slave_dev->netdev_ops = &dsa_netdev_ops;
break;
#endif
#ifdef CONFIG_NET_DSA_TAG_EDSA
case htons(ETH_P_EDSA):
- slave_dev->hard_start_xmit = edsa_xmit;
+ slave_dev->netdev_ops = &edsa_netdev_ops;
break;
#endif
#ifdef CONFIG_NET_DSA_TAG_TRAILER
case htons(ETH_P_TRAILER):
- slave_dev->hard_start_xmit = trailer_xmit;
+ slave_dev->netdev_ops = &trailer_netdev_ops;
break;
#endif
default:
BUG();
}
- slave_dev->open = dsa_slave_open;
- slave_dev->stop = dsa_slave_close;
- slave_dev->change_rx_flags = dsa_slave_change_rx_flags;
- slave_dev->set_rx_mode = dsa_slave_set_rx_mode;
- slave_dev->set_multicast_list = dsa_slave_set_rx_mode;
- slave_dev->set_mac_address = dsa_slave_set_mac_address;
- slave_dev->do_ioctl = dsa_slave_ioctl;
+
SET_NETDEV_DEV(slave_dev, parent);
slave_dev->vlan_features = master->vlan_features;
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 35bcddf..ce572f9 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -1313,7 +1313,7 @@
if ((available < target) &&
(len > sysctl_tcp_dma_copybreak) && !(flags & MSG_PEEK) &&
!sysctl_tcp_low_latency &&
- __get_cpu_var(softnet_data).net_dma) {
+ dma_find_channel(DMA_MEMCPY)) {
preempt_enable_no_resched();
tp->ucopy.pinned_list =
dma_pin_iovec_pages(msg->msg_iov, len);
@@ -1523,7 +1523,7 @@
if (!(flags & MSG_TRUNC)) {
#ifdef CONFIG_NET_DMA
if (!tp->ucopy.dma_chan && tp->ucopy.pinned_list)
- tp->ucopy.dma_chan = get_softnet_dma();
+ tp->ucopy.dma_chan = dma_find_channel(DMA_MEMCPY);
if (tp->ucopy.dma_chan) {
tp->ucopy.dma_cookie = dma_skb_copy_datagram_iovec(
@@ -1628,7 +1628,6 @@
/* Safe to free early-copied skbs now */
__skb_queue_purge(&sk->sk_async_wait_queue);
- dma_chan_put(tp->ucopy.dma_chan);
tp->ucopy.dma_chan = NULL;
}
if (tp->ucopy.pinned_list) {
@@ -2542,6 +2541,7 @@
return pp;
}
+EXPORT_SYMBOL(tcp_gro_receive);
int tcp_gro_complete(struct sk_buff *skb)
{
@@ -2558,6 +2558,7 @@
return 0;
}
+EXPORT_SYMBOL(tcp_gro_complete);
#ifdef CONFIG_TCP_MD5SIG
static unsigned long tcp_md5sig_users;
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 99b7ecb..a6961d7 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -5005,7 +5005,7 @@
return 0;
if (!tp->ucopy.dma_chan && tp->ucopy.pinned_list)
- tp->ucopy.dma_chan = get_softnet_dma();
+ tp->ucopy.dma_chan = dma_find_channel(DMA_MEMCPY);
if (tp->ucopy.dma_chan && skb_csum_unnecessary(skb)) {
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 9d839fa..19d7b42 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1594,7 +1594,7 @@
#ifdef CONFIG_NET_DMA
struct tcp_sock *tp = tcp_sk(sk);
if (!tp->ucopy.dma_chan && tp->ucopy.pinned_list)
- tp->ucopy.dma_chan = get_softnet_dma();
+ tp->ucopy.dma_chan = dma_find_channel(DMA_MEMCPY);
if (tp->ucopy.dma_chan)
ret = tcp_v4_do_rcv(sk, skb);
else
diff --git a/net/ipv6/af_inet6.c b/net/ipv6/af_inet6.c
index 437b750..94f74f5 100644
--- a/net/ipv6/af_inet6.c
+++ b/net/ipv6/af_inet6.c
@@ -672,8 +672,7 @@
EXPORT_SYMBOL_GPL(ipv6_opt_accepted);
-static struct inet6_protocol *ipv6_gso_pull_exthdrs(struct sk_buff *skb,
- int proto)
+static int ipv6_gso_pull_exthdrs(struct sk_buff *skb, int proto)
{
struct inet6_protocol *ops = NULL;
@@ -704,7 +703,7 @@
__skb_pull(skb, len);
}
- return ops;
+ return proto;
}
static int ipv6_gso_send_check(struct sk_buff *skb)
@@ -721,7 +720,9 @@
err = -EPROTONOSUPPORT;
rcu_read_lock();
- ops = ipv6_gso_pull_exthdrs(skb, ipv6h->nexthdr);
+ ops = rcu_dereference(inet6_protos[
+ ipv6_gso_pull_exthdrs(skb, ipv6h->nexthdr)]);
+
if (likely(ops && ops->gso_send_check)) {
skb_reset_transport_header(skb);
err = ops->gso_send_check(skb);
@@ -757,7 +758,9 @@
segs = ERR_PTR(-EPROTONOSUPPORT);
rcu_read_lock();
- ops = ipv6_gso_pull_exthdrs(skb, ipv6h->nexthdr);
+ ops = rcu_dereference(inet6_protos[
+ ipv6_gso_pull_exthdrs(skb, ipv6h->nexthdr)]);
+
if (likely(ops && ops->gso_segment)) {
skb_reset_transport_header(skb);
segs = ops->gso_segment(skb, features);
@@ -777,11 +780,105 @@
return segs;
}
+struct ipv6_gro_cb {
+ struct napi_gro_cb napi;
+ int proto;
+};
+
+#define IPV6_GRO_CB(skb) ((struct ipv6_gro_cb *)(skb)->cb)
+
+static struct sk_buff **ipv6_gro_receive(struct sk_buff **head,
+ struct sk_buff *skb)
+{
+ struct inet6_protocol *ops;
+ struct sk_buff **pp = NULL;
+ struct sk_buff *p;
+ struct ipv6hdr *iph;
+ unsigned int nlen;
+ int flush = 1;
+ int proto;
+
+ if (unlikely(!pskb_may_pull(skb, sizeof(*iph))))
+ goto out;
+
+ iph = ipv6_hdr(skb);
+ __skb_pull(skb, sizeof(*iph));
+
+ flush += ntohs(iph->payload_len) != skb->len;
+
+ rcu_read_lock();
+ proto = ipv6_gso_pull_exthdrs(skb, iph->nexthdr);
+ IPV6_GRO_CB(skb)->proto = proto;
+ ops = rcu_dereference(inet6_protos[proto]);
+ if (!ops || !ops->gro_receive)
+ goto out_unlock;
+
+ flush--;
+ skb_reset_transport_header(skb);
+ nlen = skb_network_header_len(skb);
+
+ for (p = *head; p; p = p->next) {
+ struct ipv6hdr *iph2;
+
+ if (!NAPI_GRO_CB(p)->same_flow)
+ continue;
+
+ iph2 = ipv6_hdr(p);
+
+ /* All fields must match except length. */
+ if (nlen != skb_network_header_len(p) ||
+ memcmp(iph, iph2, offsetof(struct ipv6hdr, payload_len)) ||
+ memcmp(&iph->nexthdr, &iph2->nexthdr,
+ nlen - offsetof(struct ipv6hdr, nexthdr))) {
+ NAPI_GRO_CB(p)->same_flow = 0;
+ continue;
+ }
+
+ NAPI_GRO_CB(p)->flush |= flush;
+ }
+
+ NAPI_GRO_CB(skb)->flush |= flush;
+
+ pp = ops->gro_receive(head, skb);
+
+out_unlock:
+ rcu_read_unlock();
+
+out:
+ NAPI_GRO_CB(skb)->flush |= flush;
+
+ return pp;
+}
+
+static int ipv6_gro_complete(struct sk_buff *skb)
+{
+ struct inet6_protocol *ops;
+ struct ipv6hdr *iph = ipv6_hdr(skb);
+ int err = -ENOSYS;
+
+ iph->payload_len = htons(skb->len - skb_network_offset(skb) -
+ sizeof(*iph));
+
+ rcu_read_lock();
+ ops = rcu_dereference(inet6_protos[IPV6_GRO_CB(skb)->proto]);
+ if (WARN_ON(!ops || !ops->gro_complete))
+ goto out_unlock;
+
+ err = ops->gro_complete(skb);
+
+out_unlock:
+ rcu_read_unlock();
+
+ return err;
+}
+
static struct packet_type ipv6_packet_type = {
.type = __constant_htons(ETH_P_IPV6),
.func = ipv6_rcv,
.gso_send_check = ipv6_gso_send_check,
.gso_segment = ipv6_gso_segment,
+ .gro_receive = ipv6_gro_receive,
+ .gro_complete = ipv6_gro_complete,
};
static int __init ipv6_packet_init(void)
diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index 76f06b9..c4a5982 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -2752,7 +2752,7 @@
kmem_cache_create("ip6_dst_cache", sizeof(struct rt6_info), 0,
SLAB_HWCACHE_ALIGN, NULL);
if (!ip6_dst_ops_template.kmem_cachep)
- goto out;;
+ goto out;
ret = register_pernet_subsys(&ip6_route_net_ops);
if (ret)
diff --git a/net/ipv6/sysctl_net_ipv6.c b/net/ipv6/sysctl_net_ipv6.c
index 9048fe7..a031034 100644
--- a/net/ipv6/sysctl_net_ipv6.c
+++ b/net/ipv6/sysctl_net_ipv6.c
@@ -128,7 +128,7 @@
int ipv6_sysctl_register(void)
{
- int err = -ENOMEM;;
+ int err = -ENOMEM;
ip6_header = register_net_sysctl_rotable(net_ipv6_ctl_path, ipv6_table);
if (ip6_header == NULL)
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index e8b8337..e5b85d4 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -101,7 +101,7 @@
}
}
-static __inline__ __sum16 tcp_v6_check(struct tcphdr *th, int len,
+static __inline__ __sum16 tcp_v6_check(int len,
struct in6_addr *saddr,
struct in6_addr *daddr,
__wsum base)
@@ -501,7 +501,7 @@
if (skb) {
struct tcphdr *th = tcp_hdr(skb);
- th->check = tcp_v6_check(th, skb->len,
+ th->check = tcp_v6_check(skb->len,
&treq->loc_addr, &treq->rmt_addr,
csum_partial(th, skb->len, skb->csum));
@@ -942,6 +942,41 @@
return 0;
}
+struct sk_buff **tcp6_gro_receive(struct sk_buff **head, struct sk_buff *skb)
+{
+ struct ipv6hdr *iph = ipv6_hdr(skb);
+
+ switch (skb->ip_summed) {
+ case CHECKSUM_COMPLETE:
+ if (!tcp_v6_check(skb->len, &iph->saddr, &iph->daddr,
+ skb->csum)) {
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
+ break;
+ }
+
+ /* fall through */
+ case CHECKSUM_NONE:
+ NAPI_GRO_CB(skb)->flush = 1;
+ return NULL;
+ }
+
+ return tcp_gro_receive(head, skb);
+}
+EXPORT_SYMBOL(tcp6_gro_receive);
+
+int tcp6_gro_complete(struct sk_buff *skb)
+{
+ struct ipv6hdr *iph = ipv6_hdr(skb);
+ struct tcphdr *th = tcp_hdr(skb);
+
+ th->check = ~tcp_v6_check(skb->len - skb_transport_offset(skb),
+ &iph->saddr, &iph->daddr, 0);
+ skb_shinfo(skb)->gso_type = SKB_GSO_TCPV6;
+
+ return tcp_gro_complete(skb);
+}
+EXPORT_SYMBOL(tcp6_gro_complete);
+
static void tcp_v6_send_response(struct sk_buff *skb, u32 seq, u32 ack, u32 win,
u32 ts, struct tcp_md5sig_key *key, int rst)
{
@@ -1429,14 +1464,14 @@
static __sum16 tcp_v6_checksum_init(struct sk_buff *skb)
{
if (skb->ip_summed == CHECKSUM_COMPLETE) {
- if (!tcp_v6_check(tcp_hdr(skb), skb->len, &ipv6_hdr(skb)->saddr,
+ if (!tcp_v6_check(skb->len, &ipv6_hdr(skb)->saddr,
&ipv6_hdr(skb)->daddr, skb->csum)) {
skb->ip_summed = CHECKSUM_UNNECESSARY;
return 0;
}
}
- skb->csum = ~csum_unfold(tcp_v6_check(tcp_hdr(skb), skb->len,
+ skb->csum = ~csum_unfold(tcp_v6_check(skb->len,
&ipv6_hdr(skb)->saddr,
&ipv6_hdr(skb)->daddr, 0));
@@ -1640,7 +1675,7 @@
#ifdef CONFIG_NET_DMA
struct tcp_sock *tp = tcp_sk(sk);
if (!tp->ucopy.dma_chan && tp->ucopy.pinned_list)
- tp->ucopy.dma_chan = get_softnet_dma();
+ tp->ucopy.dma_chan = dma_find_channel(DMA_MEMCPY);
if (tp->ucopy.dma_chan)
ret = tcp_v6_do_rcv(sk, skb);
else
@@ -2062,6 +2097,8 @@
.err_handler = tcp_v6_err,
.gso_send_check = tcp_v6_gso_send_check,
.gso_segment = tcp_tso_segment,
+ .gro_receive = tcp6_gro_receive,
+ .gro_complete = tcp6_gro_complete,
.flags = INET6_PROTO_NOPOLICY|INET6_PROTO_FINAL,
};
diff --git a/net/phonet/pep-gprs.c b/net/phonet/pep-gprs.c
index b0ceac2..6a91a32 100644
--- a/net/phonet/pep-gprs.c
+++ b/net/phonet/pep-gprs.c
@@ -227,6 +227,13 @@
return 0;
}
+static const struct net_device_ops gprs_netdev_ops = {
+ .ndo_open = gprs_open,
+ .ndo_stop = gprs_close,
+ .ndo_start_xmit = gprs_xmit,
+ .ndo_change_mtu = gprs_set_mtu,
+};
+
static void gprs_setup(struct net_device *dev)
{
dev->features = NETIF_F_FRAGLIST;
@@ -237,11 +244,8 @@
dev->addr_len = 0;
dev->tx_queue_len = 10;
+ dev->netdev_ops = &gprs_netdev_ops;
dev->destructor = free_netdev;
- dev->open = gprs_open;
- dev->stop = gprs_close;
- dev->hard_start_xmit = gprs_xmit; /* mandatory */
- dev->change_mtu = gprs_set_mtu;
}
/*
diff --git a/net/sched/sch_sfq.c b/net/sched/sch_sfq.c
index f3965df..33133d2 100644
--- a/net/sched/sch_sfq.c
+++ b/net/sched/sch_sfq.c
@@ -435,7 +435,7 @@
int i;
q->perturb_timer.function = sfq_perturbation;
- q->perturb_timer.data = (unsigned long)sch;;
+ q->perturb_timer.data = (unsigned long)sch;
init_timer_deferrable(&q->perturb_timer);
for (i = 0; i < SFQ_HASH_DIVISOR; i++)
diff --git a/net/sched/sch_teql.c b/net/sched/sch_teql.c
index cfc8e7c..ec697ce 100644
--- a/net/sched/sch_teql.c
+++ b/net/sched/sch_teql.c
@@ -289,9 +289,9 @@
do {
struct net_device *slave = qdisc_dev(q);
- struct netdev_queue *slave_txq;
+ struct netdev_queue *slave_txq = netdev_get_tx_queue(slave, 0);
+ const struct net_device_ops *slave_ops = slave->netdev_ops;
- slave_txq = netdev_get_tx_queue(slave, 0);
if (slave_txq->qdisc_sleeping != q)
continue;
if (__netif_subqueue_stopped(slave, subq) ||
@@ -305,7 +305,7 @@
if (__netif_tx_trylock(slave_txq)) {
if (!netif_tx_queue_stopped(slave_txq) &&
!netif_tx_queue_frozen(slave_txq) &&
- slave->hard_start_xmit(skb, slave) == 0) {
+ slave_ops->ndo_start_xmit(skb, slave) == 0) {
__netif_tx_unlock(slave_txq);
master->slaves = NEXT_SLAVE(q);
netif_wake_queue(dev);
@@ -420,6 +420,14 @@
return 0;
}
+static const struct net_device_ops teql_netdev_ops = {
+ .ndo_open = teql_master_open,
+ .ndo_stop = teql_master_close,
+ .ndo_start_xmit = teql_master_xmit,
+ .ndo_get_stats = teql_master_stats,
+ .ndo_change_mtu = teql_master_mtu,
+};
+
static __init void teql_master_setup(struct net_device *dev)
{
struct teql_master *master = netdev_priv(dev);
@@ -436,11 +444,7 @@
ops->destroy = teql_destroy;
ops->owner = THIS_MODULE;
- dev->open = teql_master_open;
- dev->hard_start_xmit = teql_master_xmit;
- dev->stop = teql_master_close;
- dev->get_stats = teql_master_stats;
- dev->change_mtu = teql_master_mtu;
+ dev->netdev_ops = &teql_netdev_ops;
dev->type = ARPHRD_VOID;
dev->mtu = 1500;
dev->tx_queue_len = 100;
diff --git a/net/sctp/auth.c b/net/sctp/auth.c
index 20c576f..56935bb 100644
--- a/net/sctp/auth.c
+++ b/net/sctp/auth.c
@@ -489,7 +489,7 @@
return 0;
out_err:
- /* Clean up any successfull allocations */
+ /* Clean up any successful allocations */
sctp_auth_destroy_hmacs(ep->auth_hmacs);
return -ENOMEM;
}
diff --git a/net/wimax/Kconfig b/net/wimax/Kconfig
index 0bdbb69..18495cd 100644
--- a/net/wimax/Kconfig
+++ b/net/wimax/Kconfig
@@ -1,9 +1,23 @@
#
# WiMAX LAN device configuration
#
+# Note the ugly 'depends on' on WIMAX: that disallows RFKILL to be a
+# module if WIMAX is to be linked in. The WiMAX code is done in such a
+# way that it doesn't require and explicit dependency on RFKILL in
+# case an embedded system wants to rip it out.
+#
+# As well, enablement of the RFKILL code means we need the INPUT layer
+# support to inject events coming from hw rfkill switches. That
+# dependency could be killed if input.h provided appropiate means to
+# work when input is disabled.
+
+comment "WiMAX Wireless Broadband support requires CONFIG_INPUT enabled"
+ depends on INPUT = n && RFKILL != n
menuconfig WIMAX
tristate "WiMAX Wireless Broadband support"
+ depends on (y && RFKILL != m) || m
+ depends on (INPUT && RFKILL != n) || RFKILL = n
help
Select to configure support for devices that provide
diff --git a/net/wimax/id-table.c b/net/wimax/id-table.c
index d3b8855..5e685f7 100644
--- a/net/wimax/id-table.c
+++ b/net/wimax/id-table.c
@@ -123,15 +123,17 @@
/*
* Release the gennetlink family id / mapping table
*
- * On debug, verify that the table is empty upon removal.
+ * On debug, verify that the table is empty upon removal. We want the
+ * code always compiled, to ensure it doesn't bit rot. It will be
+ * compiled out if CONFIG_BUG is disabled.
*/
void wimax_id_table_release(void)
{
+ struct wimax_dev *wimax_dev;
+
#ifndef CONFIG_BUG
return;
#endif
- struct wimax_dev *wimax_dev;
-
spin_lock(&wimax_id_table_lock);
list_for_each_entry(wimax_dev, &wimax_id_table, id_table_node) {
printk(KERN_ERR "BUG: %s wimax_dev %p ifindex %d not cleared\n",
diff --git a/net/wimax/op-rfkill.c b/net/wimax/op-rfkill.c
index 8745bac..2b75aee 100644
--- a/net/wimax/op-rfkill.c
+++ b/net/wimax/op-rfkill.c
@@ -71,7 +71,7 @@
#define D_SUBMODULE op_rfkill
#include "debug-levels.h"
-#ifdef CONFIG_RFKILL
+#if defined(CONFIG_RFKILL) || defined(CONFIG_RFKILL_MODULE)
/**
diff --git a/net/wireless/wext.c b/net/wireless/wext.c
index e49a2d1..cb6a5bb 100644
--- a/net/wireless/wext.c
+++ b/net/wireless/wext.c
@@ -1055,8 +1055,8 @@
return private(dev, iwr, cmd, info, handler);
}
/* Old driver API : call driver ioctl handler */
- if (dev->do_ioctl)
- return dev->do_ioctl(dev, ifr, cmd);
+ if (dev->netdev_ops->ndo_do_ioctl)
+ return dev->netdev_ops->ndo_do_ioctl(dev, ifr, cmd);
return -EOPNOTSUPP;
}
diff --git a/security/device_cgroup.c b/security/device_cgroup.c
index 5ba7870..3aacd0f 100644
--- a/security/device_cgroup.c
+++ b/security/device_cgroup.c
@@ -513,11 +513,14 @@
struct dev_cgroup *dev_cgroup;
struct dev_whitelist_item *wh;
+ if (!S_ISBLK(mode) && !S_ISCHR(mode))
+ return 0;
+
rcu_read_lock();
dev_cgroup = task_devcgroup(current);
- list_for_each_entry(wh, &dev_cgroup->whitelist, list) {
+ list_for_each_entry_rcu(wh, &dev_cgroup->whitelist, list) {
if (wh->type & DEV_ALL)
goto acc_check;
if ((wh->type & DEV_BLOCK) && !S_ISBLK(mode))
diff --git a/security/smack/smackfs.c b/security/smack/smackfs.c
index bf107a3..71e2b91 100644
--- a/security/smack/smackfs.c
+++ b/security/smack/smackfs.c
@@ -569,7 +569,7 @@
if (skp == NULL)
goto out;
- rule += SMK_LABELLEN;;
+ rule += SMK_LABELLEN;
ret = sscanf(rule, "%d", &maplevel);
if (ret != 1 || maplevel > SMACK_CIPSO_MAXLEVEL)
goto out;
diff --git a/sound/soc/au1x/dbdma2.c b/sound/soc/au1x/dbdma2.c
index 74c823d..bc8d654 100644
--- a/sound/soc/au1x/dbdma2.c
+++ b/sound/soc/au1x/dbdma2.c
@@ -187,7 +187,7 @@
au1x_pcm_dmatx_cb, (void *)pcd);
if (!pcd->ddma_chan)
- return -ENOMEM;;
+ return -ENOMEM;
au1xxx_dbdma_set_devwidth(pcd->ddma_chan, msbits);
au1xxx_dbdma_ring_alloc(pcd->ddma_chan, 2);
diff --git a/sound/soc/davinci/davinci-pcm.c b/sound/soc/davinci/davinci-pcm.c
index 74abc9b..366049d 100644
--- a/sound/soc/davinci/davinci-pcm.c
+++ b/sound/soc/davinci/davinci-pcm.c
@@ -212,7 +212,7 @@
if (substream->stream == SNDRV_PCM_STREAM_PLAYBACK)
count = src - runtime->dma_addr;
else
- count = dst - runtime->dma_addr;;
+ count = dst - runtime->dma_addr;
spin_unlock(&prtd->lock);