blob: 38f9a52d182030eaf801ec494ef10f416c8c1a04 [file] [log] [blame]
Bjorn Helgaas32e62c62006-05-05 17:19:50 -06001 MEMORY ATTRIBUTE ALIASING ON IA-64
2
3 Bjorn Helgaas
4 <bjorn.helgaas@hp.com>
5 May 4, 2006
6
7
8MEMORY ATTRIBUTES
9
10 Itanium supports several attributes for virtual memory references.
11 The attribute is part of the virtual translation, i.e., it is
12 contained in the TLB entry. The ones of most interest to the Linux
13 kernel are:
14
15 WB Write-back (cacheable)
16 UC Uncacheable
17 WC Write-coalescing
18
19 System memory typically uses the WB attribute. The UC attribute is
20 used for memory-mapped I/O devices. The WC attribute is uncacheable
21 like UC is, but writes may be delayed and combined to increase
22 performance for things like frame buffers.
23
24 The Itanium architecture requires that we avoid accessing the same
25 page with both a cacheable mapping and an uncacheable mapping[1].
26
27 The design of the chipset determines which attributes are supported
28 on which regions of the address space. For example, some chipsets
29 support either WB or UC access to main memory, while others support
30 only WB access.
31
32MEMORY MAP
33
34 Platform firmware describes the physical memory map and the
35 supported attributes for each region. At boot-time, the kernel uses
36 the EFI GetMemoryMap() interface. ACPI can also describe memory
37 devices and the attributes they support, but Linux/ia64 currently
38 doesn't use this information.
39
40 The kernel uses the efi_memmap table returned from GetMemoryMap() to
41 learn the attributes supported by each region of physical address
42 space. Unfortunately, this table does not completely describe the
43 address space because some machines omit some or all of the MMIO
44 regions from the map.
45
46 The kernel maintains another table, kern_memmap, which describes the
47 memory Linux is actually using and the attribute for each region.
48 This contains only system memory; it does not contain MMIO space.
49
50 The kern_memmap table typically contains only a subset of the system
51 memory described by the efi_memmap. Linux/ia64 can't use all memory
52 in the system because of constraints imposed by the identity mapping
53 scheme.
54
55 The efi_memmap table is preserved unmodified because the original
56 boot-time information is required for kexec.
57
58KERNEL IDENTITY MAPPINGS
59
60 Linux/ia64 identity mappings are done with large pages, currently
61 either 16MB or 64MB, referred to as "granules." Cacheable mappings
62 are speculative[2], so the processor can read any location in the
63 page at any time, independent of the programmer's intentions. This
64 means that to avoid attribute aliasing, Linux can create a cacheable
65 identity mapping only when the entire granule supports cacheable
66 access.
67
68 Therefore, kern_memmap contains only full granule-sized regions that
69 can referenced safely by an identity mapping.
70
71 Uncacheable mappings are not speculative, so the processor will
72 generate UC accesses only to locations explicitly referenced by
73 software. This allows UC identity mappings to cover granules that
74 are only partially populated, or populated with a combination of UC
75 and WB regions.
76
77USER MAPPINGS
78
79 User mappings are typically done with 16K or 64K pages. The smaller
80 page size allows more flexibility because only 16K or 64K has to be
81 homogeneous with respect to memory attributes.
82
83POTENTIAL ATTRIBUTE ALIASING CASES
84
85 There are several ways the kernel creates new mappings:
86
87 mmap of /dev/mem
88
89 This uses remap_pfn_range(), which creates user mappings. These
90 mappings may be either WB or UC. If the region being mapped
91 happens to be in kern_memmap, meaning that it may also be mapped
92 by a kernel identity mapping, the user mapping must use the same
93 attribute as the kernel mapping.
94
95 If the region is not in kern_memmap, the user mapping should use
96 an attribute reported as being supported in the EFI memory map.
97
98 Since the EFI memory map does not describe MMIO on some
99 machines, this should use an uncacheable mapping as a fallback.
100
101 mmap of /sys/class/pci_bus/.../legacy_mem
102
103 This is very similar to mmap of /dev/mem, except that legacy_mem
104 only allows mmap of the one megabyte "legacy MMIO" area for a
105 specific PCI bus. Typically this is the first megabyte of
106 physical address space, but it may be different on machines with
107 several VGA devices.
108
109 "X" uses this to access VGA frame buffers. Using legacy_mem
110 rather than /dev/mem allows multiple instances of X to talk to
111 different VGA cards.
112
113 The /dev/mem mmap constraints apply.
114
115 However, since this is for mapping legacy MMIO space, WB access
116 does not make sense. This matters on machines without legacy
117 VGA support: these machines may have WB memory for the entire
118 first megabyte (or even the entire first granule).
119
120 On these machines, we could mmap legacy_mem as WB, which would
121 be safe in terms of attribute aliasing, but X has no way of
122 knowing that it is accessing regular memory, not a frame buffer,
123 so the kernel should fail the mmap rather than doing it with WB.
124
125 read/write of /dev/mem
126
127 This uses copy_from_user(), which implicitly uses a kernel
128 identity mapping. This is obviously safe for things in
129 kern_memmap.
130
131 There may be corner cases of things that are not in kern_memmap,
132 but could be accessed this way. For example, registers in MMIO
133 space are not in kern_memmap, but could be accessed with a UC
134 mapping. This would not cause attribute aliasing. But
135 registers typically can be accessed only with four-byte or
136 eight-byte accesses, and the copy_from_user() path doesn't allow
137 any control over the access size, so this would be dangerous.
138
139 ioremap()
140
141 This returns a kernel identity mapping for use inside the
142 kernel.
143
144 If the region is in kern_memmap, we should use the attribute
145 specified there. Otherwise, if the EFI memory map reports that
146 the entire granule supports WB, we should use that (granules
147 that are partially reserved or occupied by firmware do not appear
148 in kern_memmap). Otherwise, we should use a UC mapping.
149
150PAST PROBLEM CASES
151
152 mmap of various MMIO regions from /dev/mem by "X" on Intel platforms
153
154 The EFI memory map may not report these MMIO regions.
155
156 These must be allowed so that X will work. This means that
157 when the EFI memory map is incomplete, every /dev/mem mmap must
158 succeed. It may create either WB or UC user mappings, depending
159 on whether the region is in kern_memmap or the EFI memory map.
160
161 mmap of 0x0-0xA0000 /dev/mem by "hwinfo" on HP sx1000 with VGA enabled
162
163 See https://bugzilla.novell.com/show_bug.cgi?id=140858.
164
165 The EFI memory map reports the following attributes:
166 0x00000-0x9FFFF WB only
167 0xA0000-0xBFFFF UC only (VGA frame buffer)
168 0xC0000-0xFFFFF WB only
169
170 This mmap is done with user pages, not kernel identity mappings,
171 so it is safe to use WB mappings.
172
173 The kernel VGA driver may ioremap the VGA frame buffer at 0xA0000,
174 which will use a granule-sized UC mapping covering 0-0xFFFFF. This
175 granule covers some WB-only memory, but since UC is non-speculative,
176 the processor will never generate an uncacheable reference to the
177 WB-only areas unless the driver explicitly touches them.
178
179 mmap of 0x0-0xFFFFF legacy_mem by "X"
180
181 If the EFI memory map reports this entire range as WB, there
182 is no VGA MMIO hole, and the mmap should fail or be done with
183 a WB mapping.
184
185 There's no easy way for X to determine whether the 0xA0000-0xBFFFF
186 region is a frame buffer or just memory, so I think it's best to
187 just fail this mmap request rather than using a WB mapping. As
188 far as I know, there's no need to map legacy_mem with WB
189 mappings.
190
191 Otherwise, a UC mapping of the entire region is probably safe.
192 The VGA hole means the region will not be in kern_memmap. The
193 HP sx1000 chipset doesn't support UC access to the memory surrounding
194 the VGA hole, but X doesn't need that area anyway and should not
195 reference it.
196
197 mmap of 0xA0000-0xBFFFF legacy_mem by "X" on HP sx1000 with VGA disabled
198
199 The EFI memory map reports the following attributes:
200 0x00000-0xFFFFF WB only (no VGA MMIO hole)
201
202 This is a special case of the previous case, and the mmap should
203 fail for the same reason as above.
204
205NOTES
206
207 [1] SDM rev 2.2, vol 2, sec 4.4.1.
208 [2] SDM rev 2.2, vol 2, sec 4.4.6.