blob: cb4c2cefd45a452b3dabb2bb55587136643c9efb [file] [log] [blame]
Alan Coxda9bb1d2006-01-18 17:44:13 -08001
2
3EDAC - Error Detection And Correction
4
Doug Thompson87f24c32007-07-19 01:50:34 -07005Written by Doug Thompson <dougthompson@xmission.com>
Alan Coxda9bb1d2006-01-18 17:44:13 -080067 Dec 2005
Doug Thompson87f24c32007-07-19 01:50:34 -0700717 Jul 2007 Updated
Alan Coxda9bb1d2006-01-18 17:44:13 -08008
Mauro Carvalho Chehab37e59f82014-02-07 08:03:07 -02009(c) Mauro Carvalho Chehab
Mauro Carvalho Chehab31983a02009-08-05 21:16:56 -03001005 Aug 2009 Nehalem interface
Alan Coxda9bb1d2006-01-18 17:44:13 -080011
Doug Thompson87f24c32007-07-19 01:50:34 -070012EDAC is maintained and written by:
Alan Coxda9bb1d2006-01-18 17:44:13 -080013
Doug Thompson87f24c32007-07-19 01:50:34 -070014 Doug Thompson, Dave Jiang, Dave Peterson et al,
15 original author: Thayne Harbaugh,
16
17Contact:
18 website: bluesmoke.sourceforge.net
19 mailing list: bluesmoke-devel@lists.sourceforge.net
20
21"bluesmoke" was the name for this device driver when it was "out-of-tree"
22and maintained at sourceforge.net. When it was pushed into 2.6.16 for the
23first time, it was renamed to 'EDAC'.
24
25The bluesmoke project at sourceforge.net is now utilized as a 'staging area'
26for EDAC development, before it is sent upstream to kernel.org
27
Matt LaPlante19f59462009-04-27 15:06:31 +020028At the bluesmoke/EDAC project site is a series of quilt patches against
29recent kernels, stored in a SVN repository. For easier downloading, there
Doug Thompson87f24c32007-07-19 01:50:34 -070030is also a tarball snapshot available.
Alan Coxda9bb1d2006-01-18 17:44:13 -080031
32============================================================================
33EDAC PURPOSE
34
35The 'edac' kernel module goal is to detect and report errors that occur
Doug Thompson87f24c32007-07-19 01:50:34 -070036within the computer system running under linux.
37
38MEMORY
39
40In the initial release, memory Correctable Errors (CE) and Uncorrectable
41Errors (UE) are the primary errors being harvested. These types of errors
42are harvested by the 'edac_mc' class of device.
Alan Coxda9bb1d2006-01-18 17:44:13 -080043
44Detecting CE events, then harvesting those events and reporting them,
45CAN be a predictor of future UE events. With CE events, the system can
Dave Petersonf3479812006-03-26 01:38:53 -080046continue to operate, but with less safety. Preventive maintenance and
Alan Coxda9bb1d2006-01-18 17:44:13 -080047proactive part replacement of memory DIMMs exhibiting CEs can reduce
48the likelihood of the dreaded UE events and system 'panics'.
49
Doug Thompson87f24c32007-07-19 01:50:34 -070050NON-MEMORY
51
52A new feature for EDAC, the edac_device class of device, was added in
53the 2.6.23 version of the kernel.
54
55This new device type allows for non-memory type of ECC hardware detectors
56to have their states harvested and presented to userspace via the sysfs
57interface.
58
59Some architectures have ECC detectors for L1, L2 and L3 caches, along with DMA
60engines, fabric switches, main data path switches, interconnections,
61and various other hardware data paths. If the hardware reports it, then
62a edac_device device probably can be constructed to harvest and present
63that to userspace.
64
65
66PCI BUS SCANNING
Alan Coxda9bb1d2006-01-18 17:44:13 -080067
68In addition, PCI Bus Parity and SERR Errors are scanned for on PCI devices
69in order to determine if errors are occurring on data transfers.
Doug Thompson87f24c32007-07-19 01:50:34 -070070
Alan Coxda9bb1d2006-01-18 17:44:13 -080071The presence of PCI Parity errors must be examined with a grain of salt.
Dave Petersonf3479812006-03-26 01:38:53 -080072There are several add-in adapters that do NOT follow the PCI specification
Alan Coxda9bb1d2006-01-18 17:44:13 -080073with regards to Parity generation and reporting. The specification says
74the vendor should tie the parity status bits to 0 if they do not intend
75to generate parity. Some vendors do not do this, and thus the parity bit
76can "float" giving false positives.
77
Matt LaPlante19f59462009-04-27 15:06:31 +020078In the kernel there is a PCI device attribute located in sysfs that is
Doug Thompson87f24c32007-07-19 01:50:34 -070079checked by the EDAC PCI scanning code. If that attribute is set,
Matt LaPlante19f59462009-04-27 15:06:31 +020080PCI parity/error scanning is skipped for that device. The attribute
Doug Thompson87f24c32007-07-19 01:50:34 -070081is:
82
83 broken_parity_status
84
Davidlohr Buesoe281d752009-11-04 15:30:34 -030085as is located in /sys/devices/pci<XXX>/0000:XX:YY.Z directories for
Doug Thompson87f24c32007-07-19 01:50:34 -070086PCI devices.
87
88FUTURE HARDWARE SCANNING
Alan Coxda9bb1d2006-01-18 17:44:13 -080089
Doug Thompson49c0dab72006-07-10 04:45:19 -070090EDAC will have future error detectors that will be integrated with
91EDAC or added to it, in the following list:
Alan Coxda9bb1d2006-01-18 17:44:13 -080092
93 MCE Machine Check Exception
94 MCA Machine Check Architecture
95 NMI NMI notification of ECC errors
96 MSRs Machine Specific Register error cases
97 and other mechanisms.
98
99These errors are usually bus errors, ECC errors, thermal throttling
100and the like.
101
102
103============================================================================
104EDAC VERSIONING
105
Doug Thompson87f24c32007-07-19 01:50:34 -0700106EDAC is composed of a "core" module (edac_core.ko) and several Memory
Alan Coxda9bb1d2006-01-18 17:44:13 -0800107Controller (MC) driver modules. On a given system, the CORE
108is loaded and one MC driver will be loaded. Both the CORE and
Doug Thompson87f24c32007-07-19 01:50:34 -0700109the MC driver (or edac_device driver) have individual versions that reflect
110current release level of their respective modules.
111
112Thus, to "report" on what version a system is running, one must report both
113the CORE's and the MC driver's versions.
Alan Coxda9bb1d2006-01-18 17:44:13 -0800114
115
116LOADING
117
118If 'edac' was statically linked with the kernel then no loading is
119necessary. If 'edac' was built as modules then simply modprobe the
120'edac' pieces that you need. You should be able to modprobe
121hardware-specific modules and have the dependencies load the necessary core
122modules.
123
124Example:
125
126$> modprobe amd76x_edac
127
128loads both the amd76x_edac.ko memory controller module and the edac_mc.ko
129core module.
130
131
132============================================================================
133EDAC sysfs INTERFACE
134
135EDAC presents a 'sysfs' interface for control, reporting and attribute
136reporting purposes.
137
Doug Thompson87f24c32007-07-19 01:50:34 -0700138EDAC lives in the /sys/devices/system/edac directory.
139
140Within this directory there currently reside 2 'edac' components:
Alan Coxda9bb1d2006-01-18 17:44:13 -0800141
142 mc memory controller(s) system
Doug Thompson49c0dab72006-07-10 04:45:19 -0700143 pci PCI control and status system
Alan Coxda9bb1d2006-01-18 17:44:13 -0800144
145
146============================================================================
147Memory Controller (mc) Model
148
149First a background on the memory controller's model abstracted in EDAC.
Doug Thompson49c0dab72006-07-10 04:45:19 -0700150Each 'mc' device controls a set of DIMM memory modules. These modules are
Dave Petersonf3479812006-03-26 01:38:53 -0800151laid out in a Chip-Select Row (csrowX) and Channel table (chX). There can
Doug Thompson49c0dab72006-07-10 04:45:19 -0700152be multiple csrows and multiple channels.
Alan Coxda9bb1d2006-01-18 17:44:13 -0800153
154Memory controllers allow for several csrows, with 8 csrows being a typical value.
155Yet, the actual number of csrows depends on the electrical "loading"
156of a given motherboard, memory controller and DIMM characteristics.
157
158Dual channels allows for 128 bit data transfers to the CPU from memory.
Doug Thompson49c0dab72006-07-10 04:45:19 -0700159Some newer chipsets allow for more than 2 channels, like Fully Buffered DIMMs
160(FB-DIMMs). The following example will assume 2 channels:
Alan Coxda9bb1d2006-01-18 17:44:13 -0800161
162
163 Channel 0 Channel 1
164 ===================================
165 csrow0 | DIMM_A0 | DIMM_B0 |
166 csrow1 | DIMM_A0 | DIMM_B0 |
167 ===================================
168
169 ===================================
170 csrow2 | DIMM_A1 | DIMM_B1 |
171 csrow3 | DIMM_A1 | DIMM_B1 |
172 ===================================
173
174In the above example table there are 4 physical slots on the motherboard
175for memory DIMMs:
176
177 DIMM_A0
178 DIMM_B0
179 DIMM_A1
180 DIMM_B1
181
182Labels for these slots are usually silk screened on the motherboard. Slots
Dave Petersonf3479812006-03-26 01:38:53 -0800183labeled 'A' are channel 0 in this example. Slots labeled 'B'
Alan Coxda9bb1d2006-01-18 17:44:13 -0800184are channel 1. Notice that there are two csrows possible on a
185physical DIMM. These csrows are allocated their csrow assignment
186based on the slot into which the memory DIMM is placed. Thus, when 1 DIMM
187is placed in each Channel, the csrows cross both DIMMs.
188
189Memory DIMMs come single or dual "ranked". A rank is a populated csrow.
190Thus, 2 single ranked DIMMs, placed in slots DIMM_A0 and DIMM_B0 above
191will have 1 csrow, csrow0. csrow1 will be empty. On the other hand,
Dave Petersonf3479812006-03-26 01:38:53 -0800192when 2 dual ranked DIMMs are similarly placed, then both csrow0 and
Alan Coxda9bb1d2006-01-18 17:44:13 -0800193csrow1 will be populated. The pattern repeats itself for csrow2 and
194csrow3.
195
196The representation of the above is reflected in the directory tree
197in EDAC's sysfs interface. Starting in directory
198/sys/devices/system/edac/mc each memory controller will be represented
Raoul Bhatia5989f112010-11-25 17:32:47 +0100199by its own 'mcX' directory, where 'X' is the index of the MC.
Alan Coxda9bb1d2006-01-18 17:44:13 -0800200
201
202 ..../edac/mc/
203 |
204 |->mc0
205 |->mc1
206 |->mc2
207 ....
208
209Under each 'mcX' directory each 'csrowX' is again represented by a
Raoul Bhatia5989f112010-11-25 17:32:47 +0100210'csrowX', where 'X' is the csrow index:
Alan Coxda9bb1d2006-01-18 17:44:13 -0800211
212
213 .../mc/mc0/
214 |
215 |->csrow0
216 |->csrow2
217 |->csrow3
218 ....
219
220Notice that there is no csrow1, which indicates that csrow0 is
221composed of a single ranked DIMMs. This should also apply in both
222Channels, in order to have dual-channel mode be operational. Since
223both csrow2 and csrow3 are populated, this indicates a dual ranked
224set of DIMMs for channels 0 and 1.
225
226
Arthur Jones327dafb2008-07-25 01:49:10 -0700227Within each of the 'mcX' and 'csrowX' directories are several
Alan Coxda9bb1d2006-01-18 17:44:13 -0800228EDAC control and attribute files.
229
Alan Coxda9bb1d2006-01-18 17:44:13 -0800230============================================================================
231'mcX' DIRECTORIES
232
233
234In 'mcX' directories are EDAC control and attribute files for
Mauro Carvalho Chehab8b6f04c2012-04-17 08:53:34 -0300235this 'X' instance of the memory controllers.
Alan Coxda9bb1d2006-01-18 17:44:13 -0800236
Mauro Carvalho Chehab8b6f04c2012-04-17 08:53:34 -0300237For a description of the sysfs API, please see:
238 Documentation/ABI/testing/sysfs/devices-edac
Alan Coxda9bb1d2006-01-18 17:44:13 -0800239
240
241============================================================================
242'csrowX' DIRECTORIES
243
Mauro Carvalho Chehab8b6f04c2012-04-17 08:53:34 -0300244When CONFIG_EDAC_LEGACY_SYSFS is enabled, the sysfs will contain the
245csrowX directories. As this API doesn't work properly for Rambus, FB-DIMMs
246and modern Intel Memory Controllers, this is being deprecated in favor
247of dimmX directories.
248
Alan Coxda9bb1d2006-01-18 17:44:13 -0800249In the 'csrowX' directories are EDAC control and attribute files for
Raoul Bhatia5989f112010-11-25 17:32:47 +0100250this 'X' instance of csrow:
Alan Coxda9bb1d2006-01-18 17:44:13 -0800251
252
253Total Uncorrectable Errors count attribute file:
254
255 'ue_count'
256
257 This attribute file displays the total count of uncorrectable
258 errors that have occurred on this csrow. If panic_on_ue is set
259 this counter will not have a chance to increment, since EDAC
260 will panic the system.
261
262
263Total Correctable Errors count attribute file:
264
265 'ce_count'
266
267 This attribute file displays the total count of correctable
268 errors that have occurred on this csrow. This
269 count is very important to examine. CEs provide early
270 indications that a DIMM is beginning to fail. This count
271 field should be monitored for non-zero values and report
272 such information to the system administrator.
273
274
275Total memory managed by this csrow attribute file:
276
277 'size_mb'
278
279 This attribute file displays, in count of megabytes, of memory
Dave Petersonf3479812006-03-26 01:38:53 -0800280 that this csrow contains.
Alan Coxda9bb1d2006-01-18 17:44:13 -0800281
282
283Memory Type attribute file:
284
285 'mem_type'
286
287 This attribute file will display what type of memory is currently
288 on this csrow. Normally, either buffered or unbuffered memory.
Doug Thompson49c0dab72006-07-10 04:45:19 -0700289 Examples:
290 Registered-DDR
291 Unbuffered-DDR
Alan Coxda9bb1d2006-01-18 17:44:13 -0800292
293
294EDAC Mode of operation attribute file:
295
296 'edac_mode'
297
298 This attribute file will display what type of Error detection
299 and correction is being utilized.
300
301
302Device type attribute file:
303
304 'dev_type'
305
Doug Thompson49c0dab72006-07-10 04:45:19 -0700306 This attribute file will display what type of DRAM device is
307 being utilized on this DIMM.
308 Examples:
309 x1
310 x2
311 x4
312 x8
Alan Coxda9bb1d2006-01-18 17:44:13 -0800313
314
315Channel 0 CE Count attribute file:
316
317 'ch0_ce_count'
318
319 This attribute file will display the count of CEs on this
320 DIMM located in channel 0.
321
322
323Channel 0 UE Count attribute file:
324
325 'ch0_ue_count'
326
327 This attribute file will display the count of UEs on this
328 DIMM located in channel 0.
329
330
331Channel 0 DIMM Label control file:
332
333 'ch0_dimm_label'
334
335 This control file allows this DIMM to have a label assigned
336 to it. With this label in the module, when errors occur
337 the output can provide the DIMM label in the system log.
338 This becomes vital for panic events to isolate the
339 cause of the UE event.
340
341 DIMM Labels must be assigned after booting, with information
342 that correctly identifies the physical slot with its
343 silk screen label. This information is currently very
344 motherboard specific and determination of this information
345 must occur in userland at this time.
346
347
348Channel 1 CE Count attribute file:
349
350 'ch1_ce_count'
351
352 This attribute file will display the count of CEs on this
353 DIMM located in channel 1.
354
355
356Channel 1 UE Count attribute file:
357
358 'ch1_ue_count'
359
360 This attribute file will display the count of UEs on this
361 DIMM located in channel 0.
362
363
364Channel 1 DIMM Label control file:
365
366 'ch1_dimm_label'
367
368 This control file allows this DIMM to have a label assigned
369 to it. With this label in the module, when errors occur
370 the output can provide the DIMM label in the system log.
371 This becomes vital for panic events to isolate the
372 cause of the UE event.
373
374 DIMM Labels must be assigned after booting, with information
375 that correctly identifies the physical slot with its
376 silk screen label. This information is currently very
377 motherboard specific and determination of this information
378 must occur in userland at this time.
379
Alan Coxda9bb1d2006-01-18 17:44:13 -0800380============================================================================
381SYSTEM LOGGING
382
383If logging for UEs and CEs are enabled then system logs will have
384error notices indicating errors that have been detected:
385
Doug Thompson49c0dab72006-07-10 04:45:19 -0700386EDAC MC0: CE page 0x283, offset 0xce0, grain 8, syndrome 0x6ec3, row 0,
Alan Coxda9bb1d2006-01-18 17:44:13 -0800387channel 1 "DIMM_B1": amd76x_edac
388
Doug Thompson49c0dab72006-07-10 04:45:19 -0700389EDAC MC0: CE page 0x1e5, offset 0xfb0, grain 8, syndrome 0xb741, row 0,
Alan Coxda9bb1d2006-01-18 17:44:13 -0800390channel 1 "DIMM_B1": amd76x_edac
391
392
393The structure of the message is:
394 the memory controller (MC0)
395 Error type (CE)
396 memory page (0x283)
397 offset in the page (0xce0)
398 the byte granularity (grain 8)
399 or resolution of the error
400 the error syndrome (0xb741)
401 memory row (row 0)
402 memory channel (channel 1)
403 DIMM label, if set prior (DIMM B1
404 and then an optional, driver-specific message that may
405 have additional information.
406
407Both UEs and CEs with no info will lack all but memory controller,
408error type, a notice of "no info" and then an optional,
409driver-specific error message.
410
411
Alan Coxda9bb1d2006-01-18 17:44:13 -0800412============================================================================
413PCI Bus Parity Detection
414
415
416On Header Type 00 devices the primary status is looked at
417for any parity error regardless of whether Parity is enabled on the
418device. (The spec indicates parity is generated in some cases).
419On Header Type 01 bridges, the secondary status register is also
Dave Petersonf3479812006-03-26 01:38:53 -0800420looked at to see if parity occurred on the bus on the other side of
Alan Coxda9bb1d2006-01-18 17:44:13 -0800421the bridge.
422
423
424SYSFS CONFIGURATION
425
426Under /sys/devices/system/edac/pci are control and attribute files as follows:
427
428
429Enable/Disable PCI Parity checking control file:
430
431 'check_pci_parity'
432
433
434 This control file enables or disables the PCI Bus Parity scanning
435 operation. Writing a 1 to this file enables the scanning. Writing
436 a 0 to this file disables the scanning.
437
438 Enable:
439 echo "1" >/sys/devices/system/edac/pci/check_pci_parity
440
441 Disable:
442 echo "0" >/sys/devices/system/edac/pci/check_pci_parity
443
444
Arthur Jones327dafb2008-07-25 01:49:10 -0700445Parity Count:
446
447 'pci_parity_count'
448
449 This attribute file will display the number of parity errors that
450 have been detected.
451
452
453============================================================================
454MODULE PARAMETERS
455
456Panic on UE control file:
457
458 'edac_mc_panic_on_ue'
459
460 An uncorrectable error will cause a machine panic. This is usually
461 desirable. It is a bad idea to continue when an uncorrectable error
462 occurs - it is indeterminate what was uncorrected and the operating
463 system context might be so mangled that continuing will lead to further
464 corruption. If the kernel has MCE configured, then EDAC will never
465 notice the UE.
466
467 LOAD TIME: module/kernel parameter: edac_mc_panic_on_ue=[0|1]
468
469 RUN TIME: echo "1" > /sys/module/edac_core/parameters/edac_mc_panic_on_ue
470
471
472Log UE control file:
473
474 'edac_mc_log_ue'
475
476 Generate kernel messages describing uncorrectable errors. These errors
477 are reported through the system message log system. UE statistics
478 will be accumulated even when UE logging is disabled.
479
480 LOAD TIME: module/kernel parameter: edac_mc_log_ue=[0|1]
481
482 RUN TIME: echo "1" > /sys/module/edac_core/parameters/edac_mc_log_ue
483
484
485Log CE control file:
486
487 'edac_mc_log_ce'
488
489 Generate kernel messages describing correctable errors. These
490 errors are reported through the system message log system.
491 CE statistics will be accumulated even when CE logging is disabled.
492
493 LOAD TIME: module/kernel parameter: edac_mc_log_ce=[0|1]
494
495 RUN TIME: echo "1" > /sys/module/edac_core/parameters/edac_mc_log_ce
496
497
498Polling period control file:
499
500 'edac_mc_poll_msec'
501
502 The time period, in milliseconds, for polling for error information.
503 Too small a value wastes resources. Too large a value might delay
504 necessary handling of errors and might loose valuable information for
505 locating the error. 1000 milliseconds (once each second) is the current
506 default. Systems which require all the bandwidth they can get, may
507 increase this.
508
509 LOAD TIME: module/kernel parameter: edac_mc_poll_msec=[0|1]
510
511 RUN TIME: echo "1000" > /sys/module/edac_core/parameters/edac_mc_poll_msec
512
Alan Coxda9bb1d2006-01-18 17:44:13 -0800513
514Panic on PCI PARITY Error:
515
516 'panic_on_pci_parity'
517
518
Dave Petersonf3479812006-03-26 01:38:53 -0800519 This control files enables or disables panicking when a parity
Alan Coxda9bb1d2006-01-18 17:44:13 -0800520 error has been detected.
521
522
Arthur Jones327dafb2008-07-25 01:49:10 -0700523 module/kernel parameter: edac_panic_on_pci_pe=[0|1]
Alan Coxda9bb1d2006-01-18 17:44:13 -0800524
525 Enable:
Arthur Jones327dafb2008-07-25 01:49:10 -0700526 echo "1" > /sys/module/edac_core/parameters/edac_panic_on_pci_pe
Alan Coxda9bb1d2006-01-18 17:44:13 -0800527
528 Disable:
Arthur Jones327dafb2008-07-25 01:49:10 -0700529 echo "0" > /sys/module/edac_core/parameters/edac_panic_on_pci_pe
Alan Coxda9bb1d2006-01-18 17:44:13 -0800530
531
532
Alan Coxda9bb1d2006-01-18 17:44:13 -0800533=======================================================================
Doug Thompson87f24c32007-07-19 01:50:34 -0700534
535
536EDAC_DEVICE type of device
537
538In the header file, edac_core.h, there is a series of edac_device structures
539and APIs for the EDAC_DEVICE.
540
541User space access to an edac_device is through the sysfs interface.
542
543At the location /sys/devices/system/edac (sysfs) new edac_device devices will
544appear.
545
546There is a three level tree beneath the above 'edac' directory. For example,
547the 'test_device_edac' device (found at the bluesmoke.sourceforget.net website)
548installs itself as:
549
550 /sys/devices/systm/edac/test-instance
551
552in this directory are various controls, a symlink and one or more 'instance'
553directorys.
554
555The standard default controls are:
556
557 log_ce boolean to log CE events
558 log_ue boolean to log UE events
559 panic_on_ue boolean to 'panic' the system if an UE is encountered
560 (default off, can be set true via startup script)
561 poll_msec time period between POLL cycles for events
562
563The test_device_edac device adds at least one of its own custom control:
564
565 test_bits which in the current test driver does nothing but
566 show how it is installed. A ported driver can
567 add one or more such controls and/or attributes
568 for specific uses.
569 One out-of-tree driver uses controls here to allow
570 for ERROR INJECTION operations to hardware
571 injection registers
572
573The symlink points to the 'struct dev' that is registered for this edac_device.
574
575INSTANCES
576
577One or more instance directories are present. For the 'test_device_edac' case:
578
579 test-instance0
580
581
582In this directory there are two default counter attributes, which are totals of
583counter in deeper subdirectories.
584
585 ce_count total of CE events of subdirectories
586 ue_count total of UE events of subdirectories
587
588BLOCKS
589
590At the lowest directory level is the 'block' directory. There can be 0, 1
591or more blocks specified in each instance.
592
593 test-block0
594
595
596In this directory the default attributes are:
597
598 ce_count which is counter of CE events for this 'block'
599 of hardware being monitored
600 ue_count which is counter of UE events for this 'block'
601 of hardware being monitored
602
603
604The 'test_device_edac' device adds 4 attributes and 1 control:
605
606 test-block-bits-0 for every POLL cycle this counter
607 is incremented
608 test-block-bits-1 every 10 cycles, this counter is bumped once,
609 and test-block-bits-0 is set to 0
610 test-block-bits-2 every 100 cycles, this counter is bumped once,
611 and test-block-bits-1 is set to 0
612 test-block-bits-3 every 1000 cycles, this counter is bumped once,
613 and test-block-bits-2 is set to 0
614
615
616 reset-counters writing ANY thing to this control will
617 reset all the above counters.
618
619
620Use of the 'test_device_edac' driver should any others to create their own
621unique drivers for their hardware systems.
622
623The 'test_device_edac' sample driver is located at the
624bluesmoke.sourceforge.net project site for EDAC.
625
Mauro Carvalho Chehab31983a02009-08-05 21:16:56 -0300626=======================================================================
627NEHALEM USAGE OF EDAC APIs
628
629This chapter documents some EXPERIMENTAL mappings for EDAC API to handle
630Nehalem EDAC driver. They will likely be changed on future versions
631of the driver.
632
633Due to the way Nehalem exports Memory Controller data, some adjustments
634were done at i7core_edac driver. This chapter will cover those differences
635
6361) On Nehalem, there are one Memory Controller per Quick Patch Interconnect
Mauro Carvalho Chehabc3444362009-09-05 05:10:15 -0300637 (QPI). At the driver, the term "socket" means one QPI. This is
638 associated with a physical CPU socket.
Mauro Carvalho Chehab31983a02009-08-05 21:16:56 -0300639
640 Each MC have 3 physical read channels, 3 physical write channels and
Masanari Iidac94bed8e2012-04-10 00:22:13 +0900641 3 logic channels. The driver currently sees it as just 3 channels.
Mauro Carvalho Chehab31983a02009-08-05 21:16:56 -0300642 Each channel can have up to 3 DIMMs.
643
644 The minimum known unity is DIMMs. There are no information about csrows.
Mauro Carvalho Chehabc3444362009-09-05 05:10:15 -0300645 As EDAC API maps the minimum unity is csrows, the driver sequencially
646 maps channel/dimm into different csrows.
647
Lucas De Marchi25985ed2011-03-30 22:57:33 -0300648 For example, supposing the following layout:
Mauro Carvalho Chehabc3444362009-09-05 05:10:15 -0300649 Ch0 phy rd0, wr0 (0x063f4031): 2 ranks, UDIMMs
650 dimm 0 1024 Mb offset: 0, bank: 8, rank: 1, row: 0x4000, col: 0x400
651 dimm 1 1024 Mb offset: 4, bank: 8, rank: 1, row: 0x4000, col: 0x400
652 Ch1 phy rd1, wr1 (0x063f4031): 2 ranks, UDIMMs
653 dimm 0 1024 Mb offset: 0, bank: 8, rank: 1, row: 0x4000, col: 0x400
654 Ch2 phy rd3, wr3 (0x063f4031): 2 ranks, UDIMMs
655 dimm 0 1024 Mb offset: 0, bank: 8, rank: 1, row: 0x4000, col: 0x400
656 The driver will map it as:
657 csrow0: channel 0, dimm0
658 csrow1: channel 0, dimm1
659 csrow2: channel 1, dimm0
660 csrow3: channel 2, dimm0
661
662exports one
Mauro Carvalho Chehab31983a02009-08-05 21:16:56 -0300663 DIMM per csrow.
664
Mauro Carvalho Chehabc3444362009-09-05 05:10:15 -0300665 Each QPI is exported as a different memory controller.
Mauro Carvalho Chehab31983a02009-08-05 21:16:56 -0300666
6672) Nehalem MC has the hability to generate errors. The driver implements this
668 functionality via some error injection nodes:
669
670 For injecting a memory error, there are some sysfs nodes, under
Mauro Carvalho Chehabc3444362009-09-05 05:10:15 -0300671 /sys/devices/system/edac/mc/mc?/:
Mauro Carvalho Chehab31983a02009-08-05 21:16:56 -0300672
Mauro Carvalho Chehab35be9542009-09-24 17:28:50 -0300673 inject_addrmatch/*:
Mauro Carvalho Chehab31983a02009-08-05 21:16:56 -0300674 Controls the error injection mask register. It is possible to specify
675 several characteristics of the address to match an error code:
676 dimm = the affected dimm. Numbers are relative to a channel;
677 rank = the memory rank;
678 channel = the channel that will generate an error;
679 bank = the affected bank;
680 page = the page address;
681 column (or col) = the address column.
682 each of the above values can be set to "any" to match any valid value.
683
684 At driver init, all values are set to any.
685
686 For example, to generate an error at rank 1 of dimm 2, for any channel,
687 any bank, any page, any column:
Mauro Carvalho Chehab35be9542009-09-24 17:28:50 -0300688 echo 2 >/sys/devices/system/edac/mc/mc0/inject_addrmatch/dimm
689 echo 1 >/sys/devices/system/edac/mc/mc0/inject_addrmatch/rank
Mauro Carvalho Chehab31983a02009-08-05 21:16:56 -0300690
691 To return to the default behaviour of matching any, you can do:
Mauro Carvalho Chehab35be9542009-09-24 17:28:50 -0300692 echo any >/sys/devices/system/edac/mc/mc0/inject_addrmatch/dimm
693 echo any >/sys/devices/system/edac/mc/mc0/inject_addrmatch/rank
Mauro Carvalho Chehab31983a02009-08-05 21:16:56 -0300694
695 inject_eccmask:
696 specifies what bits will have troubles,
697
698 inject_section:
699 specifies what ECC cache section will get the error:
700 3 for both
701 2 for the highest
702 1 for the lowest
703
Mauro Carvalho Chehab31983a02009-08-05 21:16:56 -0300704 inject_type:
705 specifies the type of error, being a combination of the following bits:
706 bit 0 - repeat
707 bit 1 - ecc
708 bit 2 - parity
709
710 inject_enable starts the error generation when something different
711 than 0 is written.
712
713 All inject vars can be read. root permission is needed for write.
714
715 Datasheet states that the error will only be generated after a write on an
716 address that matches inject_addrmatch. It seems, however, that reading will
717 also produce an error.
718
719 For example, the following code will generate an error for any write access
720 at socket 0, on any DIMM/address on channel 2:
721
Mauro Carvalho Chehab35be9542009-09-24 17:28:50 -0300722 echo 2 >/sys/devices/system/edac/mc/mc0/inject_addrmatch/channel
Mauro Carvalho Chehab31983a02009-08-05 21:16:56 -0300723 echo 2 >/sys/devices/system/edac/mc/mc0/inject_type
724 echo 64 >/sys/devices/system/edac/mc/mc0/inject_eccmask
725 echo 3 >/sys/devices/system/edac/mc/mc0/inject_section
Mauro Carvalho Chehab31983a02009-08-05 21:16:56 -0300726 echo 1 >/sys/devices/system/edac/mc/mc0/inject_enable
727 dd if=/dev/mem of=/dev/null seek=16k bs=4k count=1 >& /dev/null
728
Mauro Carvalho Chehabc3444362009-09-05 05:10:15 -0300729 For socket 1, it is needed to replace "mc0" by "mc1" at the above
730 commands.
731
Mauro Carvalho Chehab31983a02009-08-05 21:16:56 -0300732 The generated error message will look like:
733
734 EDAC MC0: UE row 0, channel-a= 0 channel-b= 0 labels "-": NON_FATAL (addr = 0x0075b980, socket=0, Dimm=0, Channel=2, syndrome=0x00000040, count=1, Err=8c0000400001009f:4000080482 (read error: read ECC error))
735
7363) Nehalem specific Corrected Error memory counters
737
Mauro Carvalho Chehab35be9542009-09-24 17:28:50 -0300738 Nehalem have some registers to count memory errors. The driver uses those
739 registers to report Corrected Errors on devices with Registered Dimms.
Mauro Carvalho Chehab31983a02009-08-05 21:16:56 -0300740
Mauro Carvalho Chehab35be9542009-09-24 17:28:50 -0300741 However, those counters don't work with Unregistered Dimms. As the chipset
742 offers some counters that also work with UDIMMS (but with a worse level of
743 granularity than the default ones), the driver exposes those registers for
744 UDIMM memories.
Mauro Carvalho Chehabc3444362009-09-05 05:10:15 -0300745
Mauro Carvalho Chehab35be9542009-09-24 17:28:50 -0300746 They can be read by looking at the contents of all_channel_counts/
Mauro Carvalho Chehab31983a02009-08-05 21:16:56 -0300747
Mauro Carvalho Chehab35be9542009-09-24 17:28:50 -0300748 $ for i in /sys/devices/system/edac/mc/mc0/all_channel_counts/*; do echo $i; cat $i; done
749 /sys/devices/system/edac/mc/mc0/all_channel_counts/udimm0
750 0
751 /sys/devices/system/edac/mc/mc0/all_channel_counts/udimm1
752 0
753 /sys/devices/system/edac/mc/mc0/all_channel_counts/udimm2
754 0
Mauro Carvalho Chehabc3444362009-09-05 05:10:15 -0300755
756 What happens here is that errors on different csrows, but at the same
757 dimm number will increment the same counter.
758 So, in this memory mapping:
759 csrow0: channel 0, dimm0
760 csrow1: channel 0, dimm1
761 csrow2: channel 1, dimm0
762 csrow3: channel 2, dimm0
Mauro Carvalho Chehab35be9542009-09-24 17:28:50 -0300763 The hardware will increment udimm0 for an error at the first dimm at either
764 csrow0, csrow2 or csrow3;
765 The hardware will increment udimm1 for an error at the second dimm at either
766 csrow0, csrow2 or csrow3;
767 The hardware will increment udimm2 for an error at the third dimm at either
768 csrow0, csrow2 or csrow3;
Mauro Carvalho Chehabc3444362009-09-05 05:10:15 -0300769
7704) Standard error counters
771
772 The standard error counters are generated when an mcelog error is received
Mauro Carvalho Chehab35be9542009-09-24 17:28:50 -0300773 by the driver. Since, with udimm, this is counted by software, it is
774 possible that some errors could be lost. With rdimm's, they displays the
775 contents of the registers