David Woodhouse | 7d20096 | 2005-04-13 14:22:38 +0100 | [diff] [blame] | 1 | $Id: README.Locking,v 1.12 2005/04/13 13:22:35 dwmw2 Exp $ |
Linus Torvalds | 1da177e | 2005-04-16 15:20:36 -0700 | [diff] [blame] | 2 | |
| 3 | JFFS2 LOCKING DOCUMENTATION |
| 4 | --------------------------- |
| 5 | |
| 6 | At least theoretically, JFFS2 does not require the Big Kernel Lock |
| 7 | (BKL), which was always helpfully obtained for it by Linux 2.4 VFS |
| 8 | code. It has its own locking, as described below. |
| 9 | |
| 10 | This document attempts to describe the existing locking rules for |
| 11 | JFFS2. It is not expected to remain perfectly up to date, but ought to |
| 12 | be fairly close. |
| 13 | |
| 14 | |
| 15 | alloc_sem |
| 16 | --------- |
| 17 | |
| 18 | The alloc_sem is a per-filesystem semaphore, used primarily to ensure |
| 19 | contiguous allocation of space on the medium. It is automatically |
| 20 | obtained during space allocations (jffs2_reserve_space()) and freed |
| 21 | upon write completion (jffs2_complete_reservation()). Note that |
| 22 | the garbage collector will obtain this right at the beginning of |
| 23 | jffs2_garbage_collect_pass() and release it at the end, thereby |
| 24 | preventing any other write activity on the file system during a |
| 25 | garbage collect pass. |
| 26 | |
| 27 | When writing new nodes, the alloc_sem must be held until the new nodes |
| 28 | have been properly linked into the data structures for the inode to |
| 29 | which they belong. This is for the benefit of NAND flash - adding new |
| 30 | nodes to an inode may obsolete old ones, and by holding the alloc_sem |
| 31 | until this happens we ensure that any data in the write-buffer at the |
| 32 | time this happens are part of the new node, not just something that |
| 33 | was written afterwards. Hence, we can ensure the newly-obsoleted nodes |
| 34 | don't actually get erased until the write-buffer has been flushed to |
| 35 | the medium. |
| 36 | |
| 37 | With the introduction of NAND flash support and the write-buffer, |
| 38 | the alloc_sem is also used to protect the wbuf-related members of the |
| 39 | jffs2_sb_info structure. Atomically reading the wbuf_len member to see |
| 40 | if the wbuf is currently holding any data is permitted, though. |
| 41 | |
| 42 | Ordering constraints: See f->sem. |
| 43 | |
| 44 | |
| 45 | File Semaphore f->sem |
| 46 | --------------------- |
| 47 | |
| 48 | This is the JFFS2-internal equivalent of the inode semaphore i->i_sem. |
| 49 | It protects the contents of the jffs2_inode_info private inode data, |
| 50 | including the linked list of node fragments (but see the notes below on |
| 51 | erase_completion_lock), etc. |
| 52 | |
| 53 | The reason that the i_sem itself isn't used for this purpose is to |
| 54 | avoid deadlocks with garbage collection -- the VFS will lock the i_sem |
| 55 | before calling a function which may need to allocate space. The |
| 56 | allocation may trigger garbage-collection, which may need to move a |
| 57 | node belonging to the inode which was locked in the first place by the |
| 58 | VFS. If the garbage collection code were to attempt to lock the i_sem |
| 59 | of the inode from which it's garbage-collecting a physical node, this |
| 60 | lead to deadlock, unless we played games with unlocking the i_sem |
| 61 | before calling the space allocation functions. |
| 62 | |
| 63 | Instead of playing such games, we just have an extra internal |
| 64 | semaphore, which is obtained by the garbage collection code and also |
| 65 | by the normal file system code _after_ allocation of space. |
| 66 | |
| 67 | Ordering constraints: |
| 68 | |
| 69 | 1. Never attempt to allocate space or lock alloc_sem with |
| 70 | any f->sem held. |
| 71 | 2. Never attempt to lock two file semaphores in one thread. |
| 72 | No ordering rules have been made for doing so. |
| 73 | |
| 74 | |
| 75 | erase_completion_lock spinlock |
| 76 | ------------------------------ |
| 77 | |
| 78 | This is used to serialise access to the eraseblock lists, to the |
| 79 | per-eraseblock lists of physical jffs2_raw_node_ref structures, and |
| 80 | (NB) the per-inode list of physical nodes. The latter is a special |
| 81 | case - see below. |
| 82 | |
| 83 | As the MTD API no longer permits erase-completion callback functions |
| 84 | to be called from bottom-half (timer) context (on the basis that nobody |
| 85 | ever actually implemented such a thing), it's now sufficient to use |
| 86 | a simple spin_lock() rather than spin_lock_bh(). |
| 87 | |
| 88 | Note that the per-inode list of physical nodes (f->nodes) is a special |
| 89 | case. Any changes to _valid_ nodes (i.e. ->flash_offset & 1 == 0) in |
| 90 | the list are protected by the file semaphore f->sem. But the erase |
| 91 | code may remove _obsolete_ nodes from the list while holding only the |
| 92 | erase_completion_lock. So you can walk the list only while holding the |
| 93 | erase_completion_lock, and can drop the lock temporarily mid-walk as |
| 94 | long as the pointer you're holding is to a _valid_ node, not an |
| 95 | obsolete one. |
| 96 | |
| 97 | The erase_completion_lock is also used to protect the c->gc_task |
| 98 | pointer when the garbage collection thread exits. The code to kill the |
| 99 | GC thread locks it, sends the signal, then unlocks it - while the GC |
| 100 | thread itself locks it, zeroes c->gc_task, then unlocks on the exit path. |
| 101 | |
| 102 | |
| 103 | inocache_lock spinlock |
| 104 | ---------------------- |
| 105 | |
| 106 | This spinlock protects the hashed list (c->inocache_list) of the |
| 107 | in-core jffs2_inode_cache objects (each inode in JFFS2 has the |
| 108 | correspondent jffs2_inode_cache object). So, the inocache_lock |
| 109 | has to be locked while walking the c->inocache_list hash buckets. |
| 110 | |
David Woodhouse | 7d20096 | 2005-04-13 14:22:38 +0100 | [diff] [blame] | 111 | This spinlock also covers allocation of new inode numbers, which is |
| 112 | currently just '++->highest_ino++', but might one day get more complicated |
| 113 | if we need to deal with wrapping after 4 milliard inode numbers are used. |
| 114 | |
Linus Torvalds | 1da177e | 2005-04-16 15:20:36 -0700 | [diff] [blame] | 115 | Note, the f->sem guarantees that the correspondent jffs2_inode_cache |
| 116 | will not be removed. So, it is allowed to access it without locking |
| 117 | the inocache_lock spinlock. |
| 118 | |
| 119 | Ordering constraints: |
| 120 | |
| 121 | If both erase_completion_lock and inocache_lock are needed, the |
| 122 | c->erase_completion has to be acquired first. |
| 123 | |
| 124 | |
| 125 | erase_free_sem |
| 126 | -------------- |
| 127 | |
| 128 | This semaphore is only used by the erase code which frees obsolete |
| 129 | node references and the jffs2_garbage_collect_deletion_dirent() |
| 130 | function. The latter function on NAND flash must read _obsolete_ nodes |
| 131 | to determine whether the 'deletion dirent' under consideration can be |
| 132 | discarded or whether it is still required to show that an inode has |
| 133 | been unlinked. Because reading from the flash may sleep, the |
| 134 | erase_completion_lock cannot be held, so an alternative, more |
| 135 | heavyweight lock was required to prevent the erase code from freeing |
| 136 | the jffs2_raw_node_ref structures in question while the garbage |
| 137 | collection code is looking at them. |
| 138 | |
| 139 | Suggestions for alternative solutions to this problem would be welcomed. |
| 140 | |
| 141 | |
| 142 | wbuf_sem |
| 143 | -------- |
| 144 | |
| 145 | This read/write semaphore protects against concurrent access to the |
| 146 | write-behind buffer ('wbuf') used for flash chips where we must write |
| 147 | in blocks. It protects both the contents of the wbuf and the metadata |
| 148 | which indicates which flash region (if any) is currently covered by |
| 149 | the buffer. |
| 150 | |
| 151 | Ordering constraints: |
| 152 | Lock wbuf_sem last, after the alloc_sem or and f->sem. |