blob: ced9207bedcfc9d7a27aa1476bee94ddca6fd606 [file] [log] [blame]
Greg Kroah-Hartmand36cc9d2005-11-18 09:31:11 -08001HOWTO do Linux kernel development
2---------------------------------
3
4This is the be-all, end-all document on this topic. It contains
5instructions on how to become a Linux kernel developer and how to learn
6to work with the Linux kernel development community. It tries to not
7contain anything related to the technical aspects of kernel programming,
8but will help point you in the right direction for that.
9
10If anything in this document becomes out of date, please send in patches
11to the maintainer of this file, who is listed at the bottom of the
12document.
13
14
15Introduction
16------------
17
18So, you want to learn how to become a Linux kernel developer? Or you
19have been told by your manager, "Go write a Linux driver for this
20device." This document's goal is to teach you everything you need to
21know to achieve this by describing the process you need to go through,
22and hints on how to work with the community. It will also try to
23explain some of the reasons why the community works like it does.
24
25The kernel is written mostly in C, with some architecture-dependent
26parts written in assembly. A good understanding of C is required for
27kernel development. Assembly (any architecture) is not required unless
28you plan to do low-level development for that architecture. Though they
29are not a good substitute for a solid C education and/or years of
30experience, the following books are good for, if anything, reference:
31 - "The C Programming Language" by Kernighan and Ritchie [Prentice Hall]
32 - "Practical C Programming" by Steve Oualline [O'Reilly]
Robert P. J. Day4de0ca82007-01-17 04:54:07 -050033 - "C: A Reference Manual" by Harbison and Steele [Prentice Hall]
Greg Kroah-Hartmand36cc9d2005-11-18 09:31:11 -080034
35The kernel is written using GNU C and the GNU toolchain. While it
36adheres to the ISO C89 standard, it uses a number of extensions that are
37not featured in the standard. The kernel is a freestanding C
38environment, with no reliance on the standard C library, so some
39portions of the C standard are not supported. Arbitrary long long
40divisions and floating point are not allowed. It can sometimes be
41difficult to understand the assumptions the kernel has on the toolchain
42and the extensions that it uses, and unfortunately there is no
43definitive reference for them. Please check the gcc info pages (`info
44gcc`) for some information on them.
45
46Please remember that you are trying to learn how to work with the
47existing development community. It is a diverse group of people, with
48high standards for coding, style and procedure. These standards have
49been created over time based on what they have found to work best for
50such a large and geographically dispersed team. Try to learn as much as
51possible about these standards ahead of time, as they are well
52documented; do not expect people to adapt to you or your company's way
53of doing things.
54
55
56Legal Issues
57------------
58
59The Linux kernel source code is released under the GPL. Please see the
60file, COPYING, in the main directory of the source tree, for details on
61the license. If you have further questions about the license, please
62contact a lawyer, and do not ask on the Linux kernel mailing list. The
63people on the mailing lists are not lawyers, and you should not rely on
64their statements on legal matters.
65
66For common questions and answers about the GPL, please see:
67 http://www.gnu.org/licenses/gpl-faq.html
68
69
70Documentation
71------------
72
73The Linux kernel source tree has a large range of documents that are
74invaluable for learning how to interact with the kernel community. When
75new features are added to the kernel, it is recommended that new
76documentation files are also added which explain how to use the feature.
77When a kernel change causes the interface that the kernel exposes to
78userspace to change, it is recommended that you send the information or
79a patch to the manual pages explaining the change to the manual pages
80maintainer at mtk-manpages@gmx.net.
81
82Here is a list of files that are in the kernel source tree that are
83required reading:
84 README
85 This file gives a short background on the Linux kernel and describes
86 what is necessary to do to configure and build the kernel. People
87 who are new to the kernel should start here.
88
89 Documentation/Changes
90 This file gives a list of the minimum levels of various software
91 packages that are necessary to build and run the kernel
92 successfully.
93
94 Documentation/CodingStyle
95 This describes the Linux kernel coding style, and some of the
96 rationale behind it. All new code is expected to follow the
97 guidelines in this document. Most maintainers will only accept
98 patches if these rules are followed, and many people will only
99 review code if it is in the proper style.
100
101 Documentation/SubmittingPatches
102 Documentation/SubmittingDrivers
103 These files describe in explicit detail how to successfully create
104 and send a patch, including (but not limited to):
105 - Email contents
106 - Email format
107 - Who to send it to
108 Following these rules will not guarantee success (as all patches are
109 subject to scrutiny for content and style), but not following them
110 will almost always prevent it.
111
112 Other excellent descriptions of how to create patches properly are:
113 "The Perfect Patch"
114 http://www.zip.com.au/~akpm/linux/patches/stuff/tpp.txt
115 "Linux kernel patch submission format"
116 http://linux.yyz.us/patch-format.html
117
118 Documentation/stable_api_nonsense.txt
119 This file describes the rationale behind the conscious decision to
120 not have a stable API within the kernel, including things like:
121 - Subsystem shim-layers (for compatibility?)
122 - Driver portability between Operating Systems.
123 - Mitigating rapid change within the kernel source tree (or
124 preventing rapid change)
125 This document is crucial for understanding the Linux development
126 philosophy and is very important for people moving to Linux from
127 development on other Operating Systems.
128
129 Documentation/SecurityBugs
130 If you feel you have found a security problem in the Linux kernel,
131 please follow the steps in this document to help notify the kernel
132 developers, and help solve the issue.
133
134 Documentation/ManagementStyle
135 This document describes how Linux kernel maintainers operate and the
136 shared ethos behind their methodologies. This is important reading
137 for anyone new to kernel development (or anyone simply curious about
138 it), as it resolves a lot of common misconceptions and confusion
139 about the unique behavior of kernel maintainers.
140
141 Documentation/stable_kernel_rules.txt
142 This file describes the rules on how the stable kernel releases
143 happen, and what to do if you want to get a change into one of these
144 releases.
145
146 Documentation/kernel-docs.txt
147 A list of external documentation that pertains to kernel
148 development. Please consult this list if you do not find what you
149 are looking for within the in-kernel documentation.
150
151 Documentation/applying-patches.txt
152 A good introduction describing exactly what a patch is and how to
153 apply it to the different development branches of the kernel.
154
155The kernel also has a large number of documents that can be
156automatically generated from the source code itself. This includes a
157full description of the in-kernel API, and rules on how to handle
158locking properly. The documents will be created in the
159Documentation/DocBook/ directory and can be generated as PDF,
160Postscript, HTML, and man pages by running:
161 make pdfdocs
162 make psdocs
163 make htmldocs
164 make mandocs
165respectively from the main kernel source directory.
166
167
168Becoming A Kernel Developer
169---------------------------
170
171If you do not know anything about Linux kernel development, you should
172look at the Linux KernelNewbies project:
173 http://kernelnewbies.org
174It consists of a helpful mailing list where you can ask almost any type
175of basic kernel development question (make sure to search the archives
176first, before asking something that has already been answered in the
177past.) It also has an IRC channel that you can use to ask questions in
178real-time, and a lot of helpful documentation that is useful for
179learning about Linux kernel development.
180
181The website has basic information about code organization, subsystems,
182and current projects (both in-tree and out-of-tree). It also describes
183some basic logistical information, like how to compile a kernel and
184apply a patch.
185
186If you do not know where you want to start, but you want to look for
187some task to start doing to join into the kernel development community,
188go to the Linux Kernel Janitor's project:
189 http://janitor.kernelnewbies.org/
190It is a great place to start. It describes a list of relatively simple
191problems that need to be cleaned up and fixed within the Linux kernel
192source tree. Working with the developers in charge of this project, you
193will learn the basics of getting your patch into the Linux kernel tree,
194and possibly be pointed in the direction of what to go work on next, if
195you do not already have an idea.
196
197If you already have a chunk of code that you want to put into the kernel
198tree, but need some help getting it in the proper form, the
199kernel-mentors project was created to help you out with this. It is a
200mailing list, and can be found at:
201 http://selenic.com/mailman/listinfo/kernel-mentors
202
203Before making any actual modifications to the Linux kernel code, it is
204imperative to understand how the code in question works. For this
205purpose, nothing is better than reading through it directly (most tricky
206bits are commented well), perhaps even with the help of specialized
207tools. One such tool that is particularly recommended is the Linux
208Cross-Reference project, which is able to present source code in a
209self-referential, indexed webpage format. An excellent up-to-date
210repository of the kernel code may be found at:
211 http://sosdg.org/~coywolf/lxr/
212
213
214The development process
215-----------------------
216
217Linux kernel development process currently consists of a few different
218main kernel "branches" and lots of different subsystem-specific kernel
219branches. These different branches are:
220 - main 2.6.x kernel tree
221 - 2.6.x.y -stable kernel tree
222 - 2.6.x -git kernel patches
223 - 2.6.x -mm kernel patches
224 - subsystem specific kernel trees and patches
225
2262.6.x kernel tree
227-----------------
2282.6.x kernels are maintained by Linus Torvalds, and can be found on
229kernel.org in the pub/linux/kernel/v2.6/ directory. Its development
230process is as follows:
231 - As soon as a new kernel is released a two weeks window is open,
232 during this period of time maintainers can submit big diffs to
233 Linus, usually the patches that have already been included in the
234 -mm kernel for a few weeks. The preferred way to submit big changes
235 is using git (the kernel's source management tool, more information
236 can be found at http://git.or.cz/) but plain patches are also just
237 fine.
238 - After two weeks a -rc1 kernel is released it is now possible to push
239 only patches that do not include new features that could affect the
240 stability of the whole kernel. Please note that a whole new driver
241 (or filesystem) might be accepted after -rc1 because there is no
242 risk of causing regressions with such a change as long as the change
243 is self-contained and does not affect areas outside of the code that
244 is being added. git can be used to send patches to Linus after -rc1
245 is released, but the patches need to also be sent to a public
246 mailing list for review.
247 - A new -rc is released whenever Linus deems the current git tree to
248 be in a reasonably sane state adequate for testing. The goal is to
249 release a new -rc kernel every week.
250 - Process continues until the kernel is considered "ready", the
251 process should last around 6 weeks.
252
253It is worth mentioning what Andrew Morton wrote on the linux-kernel
254mailing list about kernel releases:
255 "Nobody knows when a kernel will be released, because it's
256 released according to perceived bug status, not according to a
257 preconceived timeline."
258
2592.6.x.y -stable kernel tree
260---------------------------
261Kernels with 4 digit versions are -stable kernels. They contain
262relatively small and critical fixes for security problems or significant
263regressions discovered in a given 2.6.x kernel.
264
265This is the recommended branch for users who want the most recent stable
266kernel and are not interested in helping test development/experimental
267versions.
268
269If no 2.6.x.y kernel is available, then the highest numbered 2.6.x
270kernel is the current stable kernel.
271
2722.6.x.y are maintained by the "stable" team <stable@kernel.org>, and are
273released almost every other week.
274
275The file Documentation/stable_kernel_rules.txt in the kernel tree
276documents what kinds of changes are acceptable for the -stable tree, and
277how the release process works.
278
2792.6.x -git patches
280------------------
281These are daily snapshots of Linus' kernel tree which are managed in a
282git repository (hence the name.) These patches are usually released
283daily and represent the current state of Linus' tree. They are more
284experimental than -rc kernels since they are generated automatically
285without even a cursory glance to see if they are sane.
286
2872.6.x -mm kernel patches
288------------------------
289These are experimental kernel patches released by Andrew Morton. Andrew
290takes all of the different subsystem kernel trees and patches and mushes
291them together, along with a lot of patches that have been plucked from
292the linux-kernel mailing list. This tree serves as a proving ground for
293new features and patches. Once a patch has proved its worth in -mm for
294a while Andrew or the subsystem maintainer pushes it on to Linus for
295inclusion in mainline.
296
297It is heavily encouraged that all new patches get tested in the -mm tree
298before they are sent to Linus for inclusion in the main kernel tree.
299
300These kernels are not appropriate for use on systems that are supposed
301to be stable and they are more risky to run than any of the other
302branches.
303
304If you wish to help out with the kernel development process, please test
305and use these kernel releases and provide feedback to the linux-kernel
306mailing list if you have any problems, and if everything works properly.
307
308In addition to all the other experimental patches, these kernels usually
309also contain any changes in the mainline -git kernels available at the
310time of release.
311
312The -mm kernels are not released on a fixed schedule, but usually a few
313-mm kernels are released in between each -rc kernel (1 to 3 is common).
314
315Subsystem Specific kernel trees and patches
316-------------------------------------------
317A number of the different kernel subsystem developers expose their
318development trees so that others can see what is happening in the
319different areas of the kernel. These trees are pulled into the -mm
320kernel releases as described above.
321
322Here is a list of some of the different kernel trees available:
323 git trees:
324 - Kbuild development tree, Sam Ravnborg <sam@ravnborg.org>
325 kernel.org:/pub/scm/linux/kernel/git/sam/kbuild.git
326
327 - ACPI development tree, Len Brown <len.brown@intel.com>
328 kernel.org:/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6.git
329
330 - Block development tree, Jens Axboe <axboe@suse.de>
331 kernel.org:/pub/scm/linux/kernel/git/axboe/linux-2.6-block.git
332
333 - DRM development tree, Dave Airlie <airlied@linux.ie>
334 kernel.org:/pub/scm/linux/kernel/git/airlied/drm-2.6.git
335
336 - ia64 development tree, Tony Luck <tony.luck@intel.com>
337 kernel.org:/pub/scm/linux/kernel/git/aegl/linux-2.6.git
338
339 - ieee1394 development tree, Jody McIntyre <scjody@modernduck.com>
340 kernel.org:/pub/scm/linux/kernel/git/scjody/ieee1394.git
341
342 - infiniband, Roland Dreier <rolandd@cisco.com>
343 kernel.org:/pub/scm/linux/kernel/git/roland/infiniband.git
344
345 - libata, Jeff Garzik <jgarzik@pobox.com>
346 kernel.org:/pub/scm/linux/kernel/git/jgarzik/libata-dev.git
347
348 - network drivers, Jeff Garzik <jgarzik@pobox.com>
349 kernel.org:/pub/scm/linux/kernel/git/jgarzik/netdev-2.6.git
350
351 - pcmcia, Dominik Brodowski <linux@dominikbrodowski.net>
352 kernel.org:/pub/scm/linux/kernel/git/brodo/pcmcia-2.6.git
353
354 - SCSI, James Bottomley <James.Bottomley@SteelEye.com>
355 kernel.org:/pub/scm/linux/kernel/git/jejb/scsi-misc-2.6.git
356
357 Other git kernel trees can be found listed at http://kernel.org/git
358
359 quilt trees:
360 - USB, PCI, Driver Core, and I2C, Greg Kroah-Hartman <gregkh@suse.de>
361 kernel.org/pub/linux/kernel/people/gregkh/gregkh-2.6/
Andi Kleen027a51c2006-09-26 10:52:38 +0200362 - x86-64, partly i386, Andi Kleen <ak@suse.de>
363 ftp.firstfloor.org:/pub/ak/x86_64/quilt/
Greg Kroah-Hartmand36cc9d2005-11-18 09:31:11 -0800364
365Bug Reporting
366-------------
367
368bugzilla.kernel.org is where the Linux kernel developers track kernel
369bugs. Users are encouraged to report all bugs that they find in this
370tool. For details on how to use the kernel bugzilla, please see:
371 http://test.kernel.org/bugzilla/faq.html
372
373The file REPORTING-BUGS in the main kernel source directory has a good
374template for how to report a possible kernel bug, and details what kind
375of information is needed by the kernel developers to help track down the
376problem.
377
378
Diego Calleja3f271002006-09-30 23:27:49 -0700379Managing bug reports
380--------------------
381
382One of the best ways to put into practice your hacking skills is by fixing
383bugs reported by other people. Not only you will help to make the kernel
384more stable, you'll learn to fix real world problems and you will improve
385your skills, and other developers will be aware of your presence. Fixing
386bugs is one of the best ways to earn merit amongst the developers, because
387not many people like wasting time fixing other people's bugs.
388
389To work in the already reported bug reports, go to http://bugzilla.kernel.org.
390If you want to be advised of the future bug reports, you can subscribe to the
391bugme-new mailing list (only new bug reports are mailed here) or to the
392bugme-janitor mailing list (every change in the bugzilla is mailed here)
393
394 http://lists.osdl.org/mailman/listinfo/bugme-new
395 http://lists.osdl.org/mailman/listinfo/bugme-janitors
396
397
398
Greg Kroah-Hartmand36cc9d2005-11-18 09:31:11 -0800399Mailing lists
400-------------
401
402As some of the above documents describe, the majority of the core kernel
403developers participate on the Linux Kernel Mailing list. Details on how
404to subscribe and unsubscribe from the list can be found at:
405 http://vger.kernel.org/vger-lists.html#linux-kernel
406There are archives of the mailing list on the web in many different
407places. Use a search engine to find these archives. For example:
408 http://dir.gmane.org/gmane.linux.kernel
409It is highly recommended that you search the archives about the topic
410you want to bring up, before you post it to the list. A lot of things
411already discussed in detail are only recorded at the mailing list
412archives.
413
414Most of the individual kernel subsystems also have their own separate
415mailing list where they do their development efforts. See the
416MAINTAINERS file for a list of what these lists are for the different
417groups.
418
419Many of the lists are hosted on kernel.org. Information on them can be
420found at:
421 http://vger.kernel.org/vger-lists.html
422
423Please remember to follow good behavioral habits when using the lists.
424Though a bit cheesy, the following URL has some simple guidelines for
425interacting with the list (or any list):
426 http://www.albion.com/netiquette/
427
428If multiple people respond to your mail, the CC: list of recipients may
429get pretty large. Don't remove anybody from the CC: list without a good
430reason, or don't reply only to the list address. Get used to receiving the
431mail twice, one from the sender and the one from the list, and don't try
432to tune that by adding fancy mail-headers, people will not like it.
433
434Remember to keep the context and the attribution of your replies intact,
435keep the "John Kernelhacker wrote ...:" lines at the top of your reply, and
436add your statements between the individual quoted sections instead of
437writing at the top of the mail.
438
439If you add patches to your mail, make sure they are plain readable text
440as stated in Documentation/SubmittingPatches. Kernel developers don't
441want to deal with attachments or compressed patches; they may want
442to comment on individual lines of your patch, which works only that way.
443Make sure you use a mail program that does not mangle spaces and tab
444characters. A good first test is to send the mail to yourself and try
445to apply your own patch by yourself. If that doesn't work, get your
446mail program fixed or change it until it works.
447
448Above all, please remember to show respect to other subscribers.
449
450
451Working with the community
452--------------------------
453
454The goal of the kernel community is to provide the best possible kernel
455there is. When you submit a patch for acceptance, it will be reviewed
456on its technical merits and those alone. So, what should you be
457expecting?
458 - criticism
459 - comments
460 - requests for change
461 - requests for justification
462 - silence
463
464Remember, this is part of getting your patch into the kernel. You have
465to be able to take criticism and comments about your patches, evaluate
466them at a technical level and either rework your patches or provide
467clear and concise reasoning as to why those changes should not be made.
468If there are no responses to your posting, wait a few days and try
469again, sometimes things get lost in the huge volume.
470
471What should you not do?
472 - expect your patch to be accepted without question
473 - become defensive
474 - ignore comments
475 - resubmit the patch without making any of the requested changes
476
477In a community that is looking for the best technical solution possible,
478there will always be differing opinions on how beneficial a patch is.
479You have to be cooperative, and willing to adapt your idea to fit within
480the kernel. Or at least be willing to prove your idea is worth it.
481Remember, being wrong is acceptable as long as you are willing to work
482toward a solution that is right.
483
484It is normal that the answers to your first patch might simply be a list
485of a dozen things you should correct. This does _not_ imply that your
486patch will not be accepted, and it is _not_ meant against you
487personally. Simply correct all issues raised against your patch and
488resend it.
489
490
491Differences between the kernel community and corporate structures
492-----------------------------------------------------------------
493
494The kernel community works differently than most traditional corporate
495development environments. Here are a list of things that you can try to
496do to try to avoid problems:
497 Good things to say regarding your proposed changes:
498 - "This solves multiple problems."
499 - "This deletes 2000 lines of code."
500 - "Here is a patch that explains what I am trying to describe."
501 - "I tested it on 5 different architectures..."
502 - "Here is a series of small patches that..."
503 - "This increases performance on typical machines..."
504
505 Bad things you should avoid saying:
506 - "We did it this way in AIX/ptx/Solaris, so therefore it must be
507 good..."
508 - "I've being doing this for 20 years, so..."
509 - "This is required for my company to make money"
510 - "This is for our Enterprise product line."
511 - "Here is my 1000 page design document that describes my idea"
512 - "I've been working on this for 6 months..."
513 - "Here's a 5000 line patch that..."
514 - "I rewrote all of the current mess, and here it is..."
515 - "I have a deadline, and this patch needs to be applied now."
516
517Another way the kernel community is different than most traditional
518software engineering work environments is the faceless nature of
519interaction. One benefit of using email and irc as the primary forms of
520communication is the lack of discrimination based on gender or race.
521The Linux kernel work environment is accepting of women and minorities
522because all you are is an email address. The international aspect also
523helps to level the playing field because you can't guess gender based on
524a person's name. A man may be named Andrea and a woman may be named Pat.
525Most women who have worked in the Linux kernel and have expressed an
526opinion have had positive experiences.
527
528The language barrier can cause problems for some people who are not
529comfortable with English. A good grasp of the language can be needed in
530order to get ideas across properly on mailing lists, so it is
531recommended that you check your emails to make sure they make sense in
532English before sending them.
533
534
535Break up your changes
536---------------------
537
538The Linux kernel community does not gladly accept large chunks of code
539dropped on it all at once. The changes need to be properly introduced,
540discussed, and broken up into tiny, individual portions. This is almost
541the exact opposite of what companies are used to doing. Your proposal
542should also be introduced very early in the development process, so that
543you can receive feedback on what you are doing. It also lets the
544community feel that you are working with them, and not simply using them
545as a dumping ground for your feature. However, don't send 50 emails at
546one time to a mailing list, your patch series should be smaller than
547that almost all of the time.
548
549The reasons for breaking things up are the following:
550
5511) Small patches increase the likelihood that your patches will be
552 applied, since they don't take much time or effort to verify for
553 correctness. A 5 line patch can be applied by a maintainer with
554 barely a second glance. However, a 500 line patch may take hours to
555 review for correctness (the time it takes is exponentially
556 proportional to the size of the patch, or something).
557
558 Small patches also make it very easy to debug when something goes
559 wrong. It's much easier to back out patches one by one than it is
560 to dissect a very large patch after it's been applied (and broken
561 something).
562
5632) It's important not only to send small patches, but also to rewrite
564 and simplify (or simply re-order) patches before submitting them.
565
566Here is an analogy from kernel developer Al Viro:
567 "Think of a teacher grading homework from a math student. The
568 teacher does not want to see the student's trials and errors
569 before they came up with the solution. They want to see the
570 cleanest, most elegant answer. A good student knows this, and
571 would never submit her intermediate work before the final
572 solution."
573
574 The same is true of kernel development. The maintainers and
575 reviewers do not want to see the thought process behind the
576 solution to the problem one is solving. They want to see a
577 simple and elegant solution."
578
579It may be challenging to keep the balance between presenting an elegant
580solution and working together with the community and discussing your
581unfinished work. Therefore it is good to get early in the process to
582get feedback to improve your work, but also keep your changes in small
583chunks that they may get already accepted, even when your whole task is
584not ready for inclusion now.
585
586Also realize that it is not acceptable to send patches for inclusion
587that are unfinished and will be "fixed up later."
588
589
590Justify your change
591-------------------
592
593Along with breaking up your patches, it is very important for you to let
594the Linux community know why they should add this change. New features
595must be justified as being needed and useful.
596
597
598Document your change
599--------------------
600
601When sending in your patches, pay special attention to what you say in
602the text in your email. This information will become the ChangeLog
603information for the patch, and will be preserved for everyone to see for
604all time. It should describe the patch completely, containing:
605 - why the change is necessary
606 - the overall design approach in the patch
607 - implementation details
608 - testing results
609
610For more details on what this should all look like, please see the
611ChangeLog section of the document:
612 "The Perfect Patch"
613 http://www.zip.com.au/~akpm/linux/patches/stuff/tpp.txt
614
615
616
617
618All of these things are sometimes very hard to do. It can take years to
619perfect these practices (if at all). It's a continuous process of
620improvement that requires a lot of patience and determination. But
621don't give up, it's possible. Many have done it before, and each had to
622start exactly where you are now.
623
624
625
626
627----------
Paolo Ciarrocchi5bd982e2006-04-25 22:47:51 +0200628Thanks to Paolo Ciarrocchi who allowed the "Development Process"
629(http://linux.tar.bz/articles/2.6-development_process) section
Greg Kroah-Hartmand36cc9d2005-11-18 09:31:11 -0800630to be based on text he had written, and to Randy Dunlap and Gerrit
631Huizenga for some of the list of things you should and should not say.
632Also thanks to Pat Mochel, Hanna Linder, Randy Dunlap, Kay Sievers,
633Vojtech Pavlik, Jan Kara, Josh Boyer, Kees Cook, Andrew Morton, Andi
634Kleen, Vadim Lobanov, Jesper Juhl, Adrian Bunk, Keri Harris, Frans Pop,
635David A. Wheeler, Junio Hamano, Michael Kerrisk, and Alex Shepard for
636their review, comments, and contributions. Without their help, this
637document would not have been possible.
638
639
640
641Maintainer: Greg Kroah-Hartman <greg@kroah.com>