blob: 04dc1cf9d2155a3488d2fe557d7f82b76a06ecc8 [file] [log] [blame]
Rafael J. Wysocki83144182007-07-17 04:03:35 -07001Freezing of tasks
2 (C) 2007 Rafael J. Wysocki <rjw@sisk.pl>, GPL
3
4I. What is the freezing of tasks?
5
6The freezing of tasks is a mechanism by which user space processes and some
7kernel threads are controlled during hibernation or system-wide suspend (on some
8architectures).
9
10II. How does it work?
11
12There are four per-task flags used for that, PF_NOFREEZE, PF_FROZEN, TIF_FREEZE
13and PF_FREEZER_SKIP (the last one is auxiliary). The tasks that have
14PF_NOFREEZE unset (all user space processes and some kernel threads) are
15regarded as 'freezable' and treated in a special way before the system enters a
16suspend state as well as before a hibernation image is created (in what follows
17we only consider hibernation, but the description also applies to suspend).
18
19Namely, as the first step of the hibernation procedure the function
20freeze_processes() (defined in kernel/power/process.c) is called. It executes
21try_to_freeze_tasks() that sets TIF_FREEZE for all of the freezable tasks and
22sends a fake signal to each of them. A task that receives such a signal and has
23TIF_FREEZE set, should react to it by calling the refrigerator() function
24(defined in kernel/power/process.c), which sets the task's PF_FROZEN flag,
25changes its state to TASK_UNINTERRUPTIBLE and makes it loop until PF_FROZEN is
26cleared for it. Then, we say that the task is 'frozen' and therefore the set of
27functions handling this mechanism is called 'the freezer' (these functions are
28defined in kernel/power/process.c and include/linux/freezer.h). User space
29processes are generally frozen before kernel threads.
30
31It is not recommended to call refrigerator() directly. Instead, it is
32recommended to use the try_to_freeze() function (defined in
33include/linux/freezer.h), that checks the task's TIF_FREEZE flag and makes the
34task enter refrigerator() if the flag is set.
35
36For user space processes try_to_freeze() is called automatically from the
37signal-handling code, but the freezable kernel threads need to call it
38explicitly in suitable places. The code to do this may look like the following:
39
40 do {
41 hub_events();
42 wait_event_interruptible(khubd_wait,
43 !list_empty(&hub_event_list));
44 try_to_freeze();
45 } while (!signal_pending(current));
46
47(from drivers/usb/core/hub.c::hub_thread()).
48
49If a freezable kernel thread fails to call try_to_freeze() after the freezer has
50set TIF_FREEZE for it, the freezing of tasks will fail and the entire
51hibernation operation will be cancelled. For this reason, freezable kernel
52threads must call try_to_freeze() somewhere.
53
54After the system memory state has been restored from a hibernation image and
55devices have been reinitialized, the function thaw_processes() is called in
56order to clear the PF_FROZEN flag for each frozen task. Then, the tasks that
57have been frozen leave refrigerator() and continue running.
58
59III. Which kernel threads are freezable?
60
61Kernel threads are not freezable by default. However, a kernel thread may clear
62PF_NOFREEZE for itself by calling set_freezable() (the resetting of PF_NOFREEZE
63directly is strongly discouraged). From this point it is regarded as freezable
64and must call try_to_freeze() in a suitable place.
65
66IV. Why do we do that?
67
68Generally speaking, there is a couple of reasons to use the freezing of tasks:
69
701. The principal reason is to prevent filesystems from being damaged after
71hibernation. At the moment we have no simple means of checkpointing
72filesystems, so if there are any modifications made to filesystem data and/or
73metadata on disks, we cannot bring them back to the state from before the
74modifications. At the same time each hibernation image contains some
75filesystem-related information that must be consistent with the state of the
76on-disk data and metadata after the system memory state has been restored from
77the image (otherwise the filesystems will be damaged in a nasty way, usually
78making them almost impossible to repair). We therefore freeze tasks that might
79cause the on-disk filesystems' data and metadata to be modified after the
80hibernation image has been created and before the system is finally powered off.
81The majority of these are user space processes, but if any of the kernel threads
82may cause something like this to happen, they have to be freezable.
83
842. The second reason is to prevent user space processes and some kernel threads
85from interfering with the suspending and resuming of devices. A user space
86process running on a second CPU while we are suspending devices may, for
87example, be troublesome and without the freezing of tasks we would need some
88safeguards against race conditions that might occur in such a case.
89
90Although Linus Torvalds doesn't like the freezing of tasks, he said this in one
91of the discussions on LKML (http://lkml.org/lkml/2007/4/27/608):
92
93"RJW:> Why we freeze tasks at all or why we freeze kernel threads?
94
95Linus: In many ways, 'at all'.
96
97I _do_ realize the IO request queue issues, and that we cannot actually do
98s2ram with some devices in the middle of a DMA. So we want to be able to
99avoid *that*, there's no question about that. And I suspect that stopping
100user threads and then waiting for a sync is practically one of the easier
101ways to do so.
102
103So in practice, the 'at all' may become a 'why freeze kernel threads?' and
104freezing user threads I don't find really objectionable."
105
106Still, there are kernel threads that may want to be freezable. For example, if
107a kernel that belongs to a device driver accesses the device directly, it in
108principle needs to know when the device is suspended, so that it doesn't try to
109access it at that time. However, if the kernel thread is freezable, it will be
110frozen before the driver's .suspend() callback is executed and it will be
111thawed after the driver's .resume() callback has run, so it won't be accessing
112the device while it's suspended.
113
1143. Another reason for freezing tasks is to prevent user space processes from
115realizing that hibernation (or suspend) operation takes place. Ideally, user
116space processes should not notice that such a system-wide operation has occurred
117and should continue running without any problems after the restore (or resume
118from suspend). Unfortunately, in the most general case this is quite difficult
119to achieve without the freezing of tasks. Consider, for example, a process
120that depends on all CPUs being online while it's running. Since we need to
121disable nonboot CPUs during the hibernation, if this process is not frozen, it
122may notice that the number of CPUs has changed and may start to work incorrectly
123because of that.
124
125V. Are there any problems related to the freezing of tasks?
126
127Yes, there are.
128
129First of all, the freezing of kernel threads may be tricky if they depend one
130on another. For example, if kernel thread A waits for a completion (in the
131TASK_UNINTERRUPTIBLE state) that needs to be done by freezable kernel thread B
132and B is frozen in the meantime, then A will be blocked until B is thawed, which
133may be undesirable. That's why kernel threads are not freezable by default.
134
135Second, there are the following two problems related to the freezing of user
136space processes:
1371. Putting processes into an uninterruptible sleep distorts the load average.
1382. Now that we have FUSE, plus the framework for doing device drivers in
139userspace, it gets even more complicated because some userspace processes are
140now doing the sorts of things that kernel threads do
141(https://lists.linux-foundation.org/pipermail/linux-pm/2007-May/012309.html).
142
143The problem 1. seems to be fixable, although it hasn't been fixed so far. The
144other one is more serious, but it seems that we can work around it by using
145hibernation (and suspend) notifiers (in that case, though, we won't be able to
146avoid the realization by the user space processes that the hibernation is taking
147place).
148
149There are also problems that the freezing of tasks tends to expose, although
150they are not directly related to it. For example, if request_firmware() is
151called from a device driver's .resume() routine, it will timeout and eventually
152fail, because the user land process that should respond to the request is frozen
153at this point. So, seemingly, the failure is due to the freezing of tasks.
154Suppose, however, that the firmware file is located on a filesystem accessible
155only through another device that hasn't been resumed yet. In that case,
156request_firmware() will fail regardless of whether or not the freezing of tasks
157is used. Consequently, the problem is not really related to the freezing of
Oliver Neukumfccdb5a2007-07-21 04:37:43 -0700158tasks, since it generally exists anyway.
159
160A driver must have all firmwares it may need in RAM before suspend() is called.
161If keeping them is not practical, for example due to their size, they must be
162requested early enough using the suspend notifier API described in notifiers.txt.