blob: 1a2f2c8ec59ee88c8808374092b3cb755078e7a7 [file] [log] [blame]
Caz Yokoyama8d497512013-09-05 16:42:39 -07001An Intel MIC X100 device is a PCIe form factor add-in coprocessor
2card based on the Intel Many Integrated Core (MIC) architecture
3that runs a Linux OS. It is a PCIe endpoint in a platform and therefore
4implements the three required standard address spaces i.e. configuration,
5memory and I/O. The host OS loads a device driver as is typical for
6PCIe devices. The card itself runs a bootstrap after reset that
Dasaratharaman Chandramouliaf190492013-10-03 18:06:23 -07007transfers control to the card OS downloaded from the host driver. The
8host driver supports OSPM suspend and resume operations. It shuts down
9the card during suspend and reboots the card OS during resume.
Caz Yokoyama8d497512013-09-05 16:42:39 -070010The card OS as shipped by Intel is a Linux kernel with modifications
11for the X100 devices.
12
13Since it is a PCIe card, it does not have the ability to host hardware
14devices for networking, storage and console. We provide these devices
15on X100 coprocessors thus enabling a self-bootable equivalent environment
16for applications. A key benefit of our solution is that it leverages
17the standard virtio framework for network, disk and console devices,
18though in our case the virtio framework is used across a PCIe bus.
19
Siva Yerramreddya8035842014-07-11 14:04:19 -070020MIC PCIe card has a dma controller with 8 channels. These channels are
21shared between the host s/w and the card s/w. 0 to 3 are used by host
22and 4 to 7 by card. As the dma device doesn't show up as PCIe device,
23a virtual bus called mic bus is created and virtual dma devices are
24created on it by the host/card drivers. On host the channels are private
25and used only by the host driver to transfer data for the virtio devices.
26
Sudeep Dutt7df20f22015-04-29 05:32:28 -070027The Symmetric Communication Interface (SCIF (pronounced as skiff)) is a
28low level communications API across PCIe currently implemented for MIC.
29More details are available at scif_overview.txt.
30
Caz Yokoyama8d497512013-09-05 16:42:39 -070031Here is a block diagram of the various components described above. The
32virtio backends are situated on the host rather than the card given better
33single threaded performance for the host compared to MIC, the ability of
34the host to initiate DMA's to/from the card using the MIC DMA engine and
35the fact that the virtio block storage backend can only be on the host.
36
Siva Yerramreddya8035842014-07-11 14:04:19 -070037 |
38 +----------+ | +----------+
39 | Card OS | | | Host OS |
40 +----------+ | +----------+
41 |
42 +-------+ +--------+ +------+ | +---------+ +--------+ +--------+
43 | Virtio| |Virtio | |Virtio| | |Virtio | |Virtio | |Virtio |
44 | Net | |Console | |Block | | |Net | |Console | |Block |
45 | Driver| |Driver | |Driver| | |backend | |backend | |backend |
46 +-------+ +--------+ +------+ | +---------+ +--------+ +--------+
47 | | | | | | |
48 | | | |User | | |
49 | | | |------|------------|---------|-------
50 +-------------------+ |Kernel +--------------------------+
51 | | | Virtio over PCIe IOCTLs |
52 | | +--------------------------+
53+-----------+ | | | +-----------+
Sudeep Dutt7df20f22015-04-29 05:32:28 -070054| MIC DMA | | +----------+ | +-----------+ | | MIC DMA |
55| Driver | | | SCIF | | | SCIF | | | Driver |
56+-----------+ | +----------+ | +-----------+ | +-----------+
57 | | | | | | |
58+---------------+ | +-----+-----+ | +-----+-----+ | +---------------+
59|MIC virtual Bus| | |SCIF HW Bus| | |SCIF HW BUS| | |MIC virtual Bus|
60+---------------+ | +-----------+ | +-----+-----+ | +---------------+
61 | | | | | | |
62 | +--------------+ | | | +---------------+ |
63 | |Intel MIC | | | | |Intel MIC | |
64 +---|Card Driver +----+ | | |Host Driver | |
65 +--------------+ | +----+---------------+-----+
Siva Yerramreddya8035842014-07-11 14:04:19 -070066 | | |
67 +-------------------------------------------------------------+
68 | |
69 | PCIe Bus |
70 +-------------------------------------------------------------+