Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 1 | <?xml version="1.0" encoding="UTF-8"?> |
| 2 | <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN" |
| 3 | "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd" []> |
| 4 | |
| 5 | <book id="Generic-IRQ-Guide"> |
| 6 | <bookinfo> |
| 7 | <title>Linux generic IRQ handling</title> |
| 8 | |
| 9 | <authorgroup> |
| 10 | <author> |
| 11 | <firstname>Thomas</firstname> |
| 12 | <surname>Gleixner</surname> |
| 13 | <affiliation> |
| 14 | <address> |
| 15 | <email>tglx@linutronix.de</email> |
| 16 | </address> |
| 17 | </affiliation> |
| 18 | </author> |
| 19 | <author> |
| 20 | <firstname>Ingo</firstname> |
| 21 | <surname>Molnar</surname> |
| 22 | <affiliation> |
| 23 | <address> |
| 24 | <email>mingo@elte.hu</email> |
| 25 | </address> |
| 26 | </affiliation> |
| 27 | </author> |
| 28 | </authorgroup> |
| 29 | |
| 30 | <copyright> |
| 31 | <year>2005-2006</year> |
| 32 | <holder>Thomas Gleixner</holder> |
| 33 | </copyright> |
| 34 | <copyright> |
| 35 | <year>2005-2006</year> |
| 36 | <holder>Ingo Molnar</holder> |
| 37 | </copyright> |
| 38 | |
| 39 | <legalnotice> |
| 40 | <para> |
| 41 | This documentation is free software; you can redistribute |
| 42 | it and/or modify it under the terms of the GNU General Public |
| 43 | License version 2 as published by the Free Software Foundation. |
| 44 | </para> |
| 45 | |
| 46 | <para> |
| 47 | This program is distributed in the hope that it will be |
| 48 | useful, but WITHOUT ANY WARRANTY; without even the implied |
| 49 | warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. |
| 50 | See the GNU General Public License for more details. |
| 51 | </para> |
| 52 | |
| 53 | <para> |
| 54 | You should have received a copy of the GNU General Public |
| 55 | License along with this program; if not, write to the Free |
| 56 | Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, |
| 57 | MA 02111-1307 USA |
| 58 | </para> |
| 59 | |
| 60 | <para> |
| 61 | For more details see the file COPYING in the source |
| 62 | distribution of Linux. |
| 63 | </para> |
| 64 | </legalnotice> |
| 65 | </bookinfo> |
| 66 | |
| 67 | <toc></toc> |
| 68 | |
| 69 | <chapter id="intro"> |
| 70 | <title>Introduction</title> |
| 71 | <para> |
| 72 | The generic interrupt handling layer is designed to provide a |
| 73 | complete abstraction of interrupt handling for device drivers. |
| 74 | It is able to handle all the different types of interrupt controller |
| 75 | hardware. Device drivers use generic API functions to request, enable, |
| 76 | disable and free interrupts. The drivers do not have to know anything |
| 77 | about interrupt hardware details, so they can be used on different |
| 78 | platforms without code changes. |
| 79 | </para> |
| 80 | <para> |
| 81 | This documentation is provided to developers who want to implement |
| 82 | an interrupt subsystem based for their architecture, with the help |
| 83 | of the generic IRQ handling layer. |
| 84 | </para> |
| 85 | </chapter> |
| 86 | |
| 87 | <chapter id="rationale"> |
| 88 | <title>Rationale</title> |
| 89 | <para> |
| 90 | The original implementation of interrupt handling in Linux is using |
| 91 | the __do_IRQ() super-handler, which is able to deal with every |
| 92 | type of interrupt logic. |
| 93 | </para> |
| 94 | <para> |
| 95 | Originally, Russell King identified different types of handlers to |
| 96 | build a quite universal set for the ARM interrupt handler |
| 97 | implementation in Linux 2.5/2.6. He distinguished between: |
| 98 | <itemizedlist> |
| 99 | <listitem><para>Level type</para></listitem> |
| 100 | <listitem><para>Edge type</para></listitem> |
| 101 | <listitem><para>Simple type</para></listitem> |
| 102 | </itemizedlist> |
| 103 | In the SMP world of the __do_IRQ() super-handler another type |
| 104 | was identified: |
| 105 | <itemizedlist> |
| 106 | <listitem><para>Per CPU type</para></listitem> |
| 107 | </itemizedlist> |
| 108 | </para> |
| 109 | <para> |
| 110 | This split implementation of highlevel IRQ handlers allows us to |
| 111 | optimize the flow of the interrupt handling for each specific |
| 112 | interrupt type. This reduces complexity in that particular codepath |
| 113 | and allows the optimized handling of a given type. |
| 114 | </para> |
| 115 | <para> |
| 116 | The original general IRQ implementation used hw_interrupt_type |
| 117 | structures and their ->ack(), ->end() [etc.] callbacks to |
| 118 | differentiate the flow control in the super-handler. This leads to |
| 119 | a mix of flow logic and lowlevel hardware logic, and it also leads |
| 120 | to unnecessary code duplication: for example in i386, there is a |
| 121 | ioapic_level_irq and a ioapic_edge_irq irq-type which share many |
| 122 | of the lowlevel details but have different flow handling. |
| 123 | </para> |
| 124 | <para> |
| 125 | A more natural abstraction is the clean separation of the |
| 126 | 'irq flow' and the 'chip details'. |
| 127 | </para> |
| 128 | <para> |
| 129 | Analysing a couple of architecture's IRQ subsystem implementations |
| 130 | reveals that most of them can use a generic set of 'irq flow' |
| 131 | methods and only need to add the chip level specific code. |
| 132 | The separation is also valuable for (sub)architectures |
| 133 | which need specific quirks in the irq flow itself but not in the |
| 134 | chip-details - and thus provides a more transparent IRQ subsystem |
| 135 | design. |
| 136 | </para> |
| 137 | <para> |
| 138 | Each interrupt descriptor is assigned its own highlevel flow |
| 139 | handler, which is normally one of the generic |
| 140 | implementations. (This highlevel flow handler implementation also |
| 141 | makes it simple to provide demultiplexing handlers which can be |
| 142 | found in embedded platforms on various architectures.) |
| 143 | </para> |
| 144 | <para> |
| 145 | The separation makes the generic interrupt handling layer more |
| 146 | flexible and extensible. For example, an (sub)architecture can |
| 147 | use a generic irq-flow implementation for 'level type' interrupts |
| 148 | and add a (sub)architecture specific 'edge type' implementation. |
| 149 | </para> |
| 150 | <para> |
| 151 | To make the transition to the new model easier and prevent the |
| 152 | breakage of existing implementations, the __do_IRQ() super-handler |
| 153 | is still available. This leads to a kind of duality for the time |
| 154 | being. Over time the new model should be used in more and more |
| 155 | architectures, as it enables smaller and cleaner IRQ subsystems. |
| 156 | </para> |
| 157 | </chapter> |
| 158 | <chapter id="bugs"> |
| 159 | <title>Known Bugs And Assumptions</title> |
| 160 | <para> |
| 161 | None (knock on wood). |
| 162 | </para> |
| 163 | </chapter> |
| 164 | |
| 165 | <chapter id="Abstraction"> |
| 166 | <title>Abstraction layers</title> |
| 167 | <para> |
| 168 | There are three main levels of abstraction in the interrupt code: |
| 169 | <orderedlist> |
| 170 | <listitem><para>Highlevel driver API</para></listitem> |
| 171 | <listitem><para>Highlevel IRQ flow handlers</para></listitem> |
| 172 | <listitem><para>Chiplevel hardware encapsulation</para></listitem> |
| 173 | </orderedlist> |
| 174 | </para> |
Rob Landley | aa9128f | 2008-02-07 00:13:28 -0800 | [diff] [blame] | 175 | <sect1 id="Interrupt_control_flow"> |
Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 176 | <title>Interrupt control flow</title> |
| 177 | <para> |
| 178 | Each interrupt is described by an interrupt descriptor structure |
| 179 | irq_desc. The interrupt is referenced by an 'unsigned int' numeric |
| 180 | value which selects the corresponding interrupt decription structure |
| 181 | in the descriptor structures array. |
| 182 | The descriptor structure contains status information and pointers |
| 183 | to the interrupt flow method and the interrupt chip structure |
| 184 | which are assigned to this interrupt. |
| 185 | </para> |
| 186 | <para> |
| 187 | Whenever an interrupt triggers, the lowlevel arch code calls into |
| 188 | the generic interrupt code by calling desc->handle_irq(). |
| 189 | This highlevel IRQ handling function only uses desc->chip primitives |
| 190 | referenced by the assigned chip descriptor structure. |
| 191 | </para> |
| 192 | </sect1> |
Rob Landley | aa9128f | 2008-02-07 00:13:28 -0800 | [diff] [blame] | 193 | <sect1 id="Highlevel_Driver_API"> |
Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 194 | <title>Highlevel Driver API</title> |
| 195 | <para> |
| 196 | The highlevel Driver API consists of following functions: |
| 197 | <itemizedlist> |
| 198 | <listitem><para>request_irq()</para></listitem> |
| 199 | <listitem><para>free_irq()</para></listitem> |
| 200 | <listitem><para>disable_irq()</para></listitem> |
| 201 | <listitem><para>enable_irq()</para></listitem> |
| 202 | <listitem><para>disable_irq_nosync() (SMP only)</para></listitem> |
| 203 | <listitem><para>synchronize_irq() (SMP only)</para></listitem> |
| 204 | <listitem><para>set_irq_type()</para></listitem> |
| 205 | <listitem><para>set_irq_wake()</para></listitem> |
| 206 | <listitem><para>set_irq_data()</para></listitem> |
| 207 | <listitem><para>set_irq_chip()</para></listitem> |
| 208 | <listitem><para>set_irq_chip_data()</para></listitem> |
| 209 | </itemizedlist> |
| 210 | See the autogenerated function documentation for details. |
| 211 | </para> |
| 212 | </sect1> |
Rob Landley | aa9128f | 2008-02-07 00:13:28 -0800 | [diff] [blame] | 213 | <sect1 id="Highlevel_IRQ_flow_handlers"> |
Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 214 | <title>Highlevel IRQ flow handlers</title> |
| 215 | <para> |
| 216 | The generic layer provides a set of pre-defined irq-flow methods: |
| 217 | <itemizedlist> |
| 218 | <listitem><para>handle_level_irq</para></listitem> |
| 219 | <listitem><para>handle_edge_irq</para></listitem> |
| 220 | <listitem><para>handle_simple_irq</para></listitem> |
| 221 | <listitem><para>handle_percpu_irq</para></listitem> |
| 222 | </itemizedlist> |
| 223 | The interrupt flow handlers (either predefined or architecture |
| 224 | specific) are assigned to specific interrupts by the architecture |
| 225 | either during bootup or during device initialization. |
| 226 | </para> |
Rob Landley | aa9128f | 2008-02-07 00:13:28 -0800 | [diff] [blame] | 227 | <sect2 id="Default_flow_implementations"> |
Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 228 | <title>Default flow implementations</title> |
Rob Landley | aa9128f | 2008-02-07 00:13:28 -0800 | [diff] [blame] | 229 | <sect3 id="Helper_functions"> |
Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 230 | <title>Helper functions</title> |
| 231 | <para> |
| 232 | The helper functions call the chip primitives and |
| 233 | are used by the default flow implementations. |
| 234 | The following helper functions are implemented (simplified excerpt): |
| 235 | <programlisting> |
| 236 | default_enable(irq) |
| 237 | { |
| 238 | desc->chip->unmask(irq); |
| 239 | } |
| 240 | |
| 241 | default_disable(irq) |
| 242 | { |
| 243 | if (!delay_disable(irq)) |
| 244 | desc->chip->mask(irq); |
| 245 | } |
| 246 | |
| 247 | default_ack(irq) |
| 248 | { |
| 249 | chip->ack(irq); |
| 250 | } |
| 251 | |
| 252 | default_mask_ack(irq) |
| 253 | { |
| 254 | if (chip->mask_ack) { |
| 255 | chip->mask_ack(irq); |
| 256 | } else { |
| 257 | chip->mask(irq); |
| 258 | chip->ack(irq); |
| 259 | } |
| 260 | } |
| 261 | |
| 262 | noop(irq) |
| 263 | { |
| 264 | } |
| 265 | |
| 266 | </programlisting> |
| 267 | </para> |
| 268 | </sect3> |
| 269 | </sect2> |
Rob Landley | aa9128f | 2008-02-07 00:13:28 -0800 | [diff] [blame] | 270 | <sect2 id="Default_flow_handler_implementations"> |
Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 271 | <title>Default flow handler implementations</title> |
Rob Landley | aa9128f | 2008-02-07 00:13:28 -0800 | [diff] [blame] | 272 | <sect3 id="Default_Level_IRQ_flow_handler"> |
Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 273 | <title>Default Level IRQ flow handler</title> |
| 274 | <para> |
| 275 | handle_level_irq provides a generic implementation |
| 276 | for level-triggered interrupts. |
| 277 | </para> |
| 278 | <para> |
| 279 | The following control flow is implemented (simplified excerpt): |
| 280 | <programlisting> |
| 281 | desc->chip->start(); |
| 282 | handle_IRQ_event(desc->action); |
| 283 | desc->chip->end(); |
| 284 | </programlisting> |
| 285 | </para> |
| 286 | </sect3> |
Rob Landley | aa9128f | 2008-02-07 00:13:28 -0800 | [diff] [blame] | 287 | <sect3 id="Default_Edge_IRQ_flow_handler"> |
Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 288 | <title>Default Edge IRQ flow handler</title> |
| 289 | <para> |
| 290 | handle_edge_irq provides a generic implementation |
| 291 | for edge-triggered interrupts. |
| 292 | </para> |
| 293 | <para> |
| 294 | The following control flow is implemented (simplified excerpt): |
| 295 | <programlisting> |
| 296 | if (desc->status & running) { |
| 297 | desc->chip->hold(); |
| 298 | desc->status |= pending | masked; |
| 299 | return; |
| 300 | } |
| 301 | desc->chip->start(); |
| 302 | desc->status |= running; |
| 303 | do { |
| 304 | if (desc->status & masked) |
| 305 | desc->chip->enable(); |
David S. Miller | b06824c | 2006-12-12 01:00:06 -0800 | [diff] [blame] | 306 | desc->status &= ~pending; |
Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 307 | handle_IRQ_event(desc->action); |
| 308 | } while (status & pending); |
David S. Miller | b06824c | 2006-12-12 01:00:06 -0800 | [diff] [blame] | 309 | desc->status &= ~running; |
Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 310 | desc->chip->end(); |
| 311 | </programlisting> |
| 312 | </para> |
| 313 | </sect3> |
Rob Landley | aa9128f | 2008-02-07 00:13:28 -0800 | [diff] [blame] | 314 | <sect3 id="Default_simple_IRQ_flow_handler"> |
Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 315 | <title>Default simple IRQ flow handler</title> |
| 316 | <para> |
| 317 | handle_simple_irq provides a generic implementation |
| 318 | for simple interrupts. |
| 319 | </para> |
| 320 | <para> |
| 321 | Note: The simple flow handler does not call any |
| 322 | handler/chip primitives. |
| 323 | </para> |
| 324 | <para> |
| 325 | The following control flow is implemented (simplified excerpt): |
| 326 | <programlisting> |
| 327 | handle_IRQ_event(desc->action); |
| 328 | </programlisting> |
| 329 | </para> |
| 330 | </sect3> |
Rob Landley | aa9128f | 2008-02-07 00:13:28 -0800 | [diff] [blame] | 331 | <sect3 id="Default_per_CPU_flow_handler"> |
Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 332 | <title>Default per CPU flow handler</title> |
| 333 | <para> |
| 334 | handle_percpu_irq provides a generic implementation |
| 335 | for per CPU interrupts. |
| 336 | </para> |
| 337 | <para> |
| 338 | Per CPU interrupts are only available on SMP and |
| 339 | the handler provides a simplified version without |
| 340 | locking. |
| 341 | </para> |
| 342 | <para> |
| 343 | The following control flow is implemented (simplified excerpt): |
| 344 | <programlisting> |
| 345 | desc->chip->start(); |
| 346 | handle_IRQ_event(desc->action); |
| 347 | desc->chip->end(); |
| 348 | </programlisting> |
| 349 | </para> |
| 350 | </sect3> |
| 351 | </sect2> |
Rob Landley | aa9128f | 2008-02-07 00:13:28 -0800 | [diff] [blame] | 352 | <sect2 id="Quirks_and_optimizations"> |
Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 353 | <title>Quirks and optimizations</title> |
| 354 | <para> |
| 355 | The generic functions are intended for 'clean' architectures and chips, |
| 356 | which have no platform-specific IRQ handling quirks. If an architecture |
| 357 | needs to implement quirks on the 'flow' level then it can do so by |
| 358 | overriding the highlevel irq-flow handler. |
| 359 | </para> |
| 360 | </sect2> |
Rob Landley | aa9128f | 2008-02-07 00:13:28 -0800 | [diff] [blame] | 361 | <sect2 id="Delayed_interrupt_disable"> |
Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 362 | <title>Delayed interrupt disable</title> |
| 363 | <para> |
| 364 | This per interrupt selectable feature, which was introduced by Russell |
| 365 | King in the ARM interrupt implementation, does not mask an interrupt |
| 366 | at the hardware level when disable_irq() is called. The interrupt is |
| 367 | kept enabled and is masked in the flow handler when an interrupt event |
| 368 | happens. This prevents losing edge interrupts on hardware which does |
| 369 | not store an edge interrupt event while the interrupt is disabled at |
| 370 | the hardware level. When an interrupt arrives while the IRQ_DISABLED |
| 371 | flag is set, then the interrupt is masked at the hardware level and |
| 372 | the IRQ_PENDING bit is set. When the interrupt is re-enabled by |
| 373 | enable_irq() the pending bit is checked and if it is set, the |
| 374 | interrupt is resent either via hardware or by a software resend |
| 375 | mechanism. (It's necessary to enable CONFIG_HARDIRQS_SW_RESEND when |
| 376 | you want to use the delayed interrupt disable feature and your |
| 377 | hardware is not capable of retriggering an interrupt.) |
| 378 | The delayed interrupt disable can be runtime enabled, per interrupt, |
| 379 | by setting the IRQ_DELAYED_DISABLE flag in the irq_desc status field. |
| 380 | </para> |
| 381 | </sect2> |
| 382 | </sect1> |
Rob Landley | aa9128f | 2008-02-07 00:13:28 -0800 | [diff] [blame] | 383 | <sect1 id="Chiplevel_hardware_encapsulation"> |
Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 384 | <title>Chiplevel hardware encapsulation</title> |
| 385 | <para> |
| 386 | The chip level hardware descriptor structure irq_chip |
| 387 | contains all the direct chip relevant functions, which |
| 388 | can be utilized by the irq flow implementations. |
| 389 | <itemizedlist> |
| 390 | <listitem><para>ack()</para></listitem> |
| 391 | <listitem><para>mask_ack() - Optional, recommended for performance</para></listitem> |
| 392 | <listitem><para>mask()</para></listitem> |
| 393 | <listitem><para>unmask()</para></listitem> |
| 394 | <listitem><para>retrigger() - Optional</para></listitem> |
| 395 | <listitem><para>set_type() - Optional</para></listitem> |
| 396 | <listitem><para>set_wake() - Optional</para></listitem> |
| 397 | </itemizedlist> |
| 398 | These primitives are strictly intended to mean what they say: ack means |
| 399 | ACK, masking means masking of an IRQ line, etc. It is up to the flow |
| 400 | handler(s) to use these basic units of lowlevel functionality. |
| 401 | </para> |
| 402 | </sect1> |
| 403 | </chapter> |
| 404 | |
| 405 | <chapter id="doirq"> |
| 406 | <title>__do_IRQ entry point</title> |
| 407 | <para> |
| 408 | The original implementation __do_IRQ() is an alternative entry |
| 409 | point for all types of interrupts. |
| 410 | </para> |
| 411 | <para> |
| 412 | This handler turned out to be not suitable for all |
| 413 | interrupt hardware and was therefore reimplemented with split |
| 414 | functionality for egde/level/simple/percpu interrupts. This is not |
| 415 | only a functional optimization. It also shortens code paths for |
| 416 | interrupts. |
| 417 | </para> |
| 418 | <para> |
| 419 | To make use of the split implementation, replace the call to |
Krzysztof Halasa | bebd04c | 2009-11-15 18:57:24 +0100 | [diff] [blame] | 420 | __do_IRQ by a call to desc->handle_irq() and associate |
| 421 | the appropriate handler function to desc->handle_irq(). |
Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 422 | In most cases the generic handler implementations should |
| 423 | be sufficient. |
| 424 | </para> |
| 425 | </chapter> |
| 426 | |
| 427 | <chapter id="locking"> |
| 428 | <title>Locking on SMP</title> |
| 429 | <para> |
| 430 | The locking of chip registers is up to the architecture that |
| 431 | defines the chip primitives. There is a chip->lock field that can be used |
| 432 | for serialization, but the generic layer does not touch it. The per-irq |
| 433 | structure is protected via desc->lock, by the generic layer. |
| 434 | </para> |
| 435 | </chapter> |
| 436 | <chapter id="structs"> |
| 437 | <title>Structures</title> |
| 438 | <para> |
| 439 | This chapter contains the autogenerated documentation of the structures which are |
| 440 | used in the generic IRQ layer. |
| 441 | </para> |
| 442 | !Iinclude/linux/irq.h |
Thomas Gleixner | a9d0a1a | 2009-03-03 16:58:16 +0100 | [diff] [blame] | 443 | !Iinclude/linux/interrupt.h |
Thomas Gleixner | 11c869e | 2006-06-29 02:24:47 -0700 | [diff] [blame] | 444 | </chapter> |
| 445 | |
| 446 | <chapter id="pubfunctions"> |
| 447 | <title>Public Functions Provided</title> |
| 448 | <para> |
| 449 | This chapter contains the autogenerated documentation of the kernel API functions |
| 450 | which are exported. |
| 451 | </para> |
| 452 | !Ekernel/irq/manage.c |
| 453 | !Ekernel/irq/chip.c |
| 454 | </chapter> |
| 455 | |
| 456 | <chapter id="intfunctions"> |
| 457 | <title>Internal Functions Provided</title> |
| 458 | <para> |
| 459 | This chapter contains the autogenerated documentation of the internal functions. |
| 460 | </para> |
| 461 | !Ikernel/irq/handle.c |
| 462 | !Ikernel/irq/chip.c |
| 463 | </chapter> |
| 464 | |
| 465 | <chapter id="credits"> |
| 466 | <title>Credits</title> |
| 467 | <para> |
| 468 | The following people have contributed to this document: |
| 469 | <orderedlist> |
| 470 | <listitem><para>Thomas Gleixner<email>tglx@linutronix.de</email></para></listitem> |
| 471 | <listitem><para>Ingo Molnar<email>mingo@elte.hu</email></para></listitem> |
| 472 | </orderedlist> |
| 473 | </para> |
| 474 | </chapter> |
| 475 | </book> |