blob: 86551cc72e034fc48e63c54c6e662954ca857372 [file] [log] [blame]
Yu Zhao15b49be2009-03-20 11:25:18 +08001 PCI Express I/O Virtualization Howto
2 Copyright (C) 2009 Intel Corporation
3 Yu Zhao <yu.zhao@intel.com>
4
Donald Dutile2597ba72012-11-27 22:31:37 -05005 Update: November 2012
6 -- sysfs-based SRIOV enable-/disable-ment
7 Donald Dutile <ddutile@redhat.com>
Yu Zhao15b49be2009-03-20 11:25:18 +08008
91. Overview
10
111.1 What is SR-IOV
12
13Single Root I/O Virtualization (SR-IOV) is a PCI Express Extended
14capability which makes one physical device appear as multiple virtual
15devices. The physical device is referred to as Physical Function (PF)
16while the virtual devices are referred to as Virtual Functions (VF).
17Allocation of the VF can be dynamically controlled by the PF via
18registers encapsulated in the capability. By default, this feature is
19not enabled and the PF behaves as traditional PCIe device. Once it's
20turned on, each VF's PCI configuration space can be accessed by its own
21Bus, Device and Function Number (Routing ID). And each VF also has PCI
22Memory Space, which is used to map its register set. VF device driver
23operates on the register set so it can be functional and appear as a
24real existing PCI device.
25
262. User Guide
27
282.1 How can I enable SR-IOV capability
29
Donald Dutile2597ba72012-11-27 22:31:37 -050030Multiple methods are available for SR-IOV enablement.
31In the first method, the device driver (PF driver) will control the
32enabling and disabling of the capability via API provided by SR-IOV core.
33If the hardware has SR-IOV capability, loading its PF driver would
34enable it and all VFs associated with the PF. Some PF drivers require
35a module parameter to be set to determine the number of VFs to enable.
36In the second method, a write to the sysfs file sriov_numvfs will
37enable and disable the VFs associated with a PCIe PF. This method
38enables per-PF, VF enable/disable values versus the first method,
39which applies to all PFs of the same device. Additionally, the
40PCI SRIOV core support ensures that enable/disable operations are
41valid to reduce duplication in multiple drivers for the same
42checks, e.g., check numvfs == 0 if enabling VFs, ensure
43numvfs <= totalvfs.
44The second method is the recommended method for new/future VF devices.
Yu Zhao15b49be2009-03-20 11:25:18 +080045
462.2 How can I use the Virtual Functions
47
48The VF is treated as hot-plugged PCI devices in the kernel, so they
49should be able to work in the same way as real PCI devices. The VF
50requires device driver that is same as a normal PCI device's.
51
523. Developer Guide
53
543.1 SR-IOV API
55
56To enable SR-IOV capability:
Donald Dutile2597ba72012-11-27 22:31:37 -050057(a) For the first method, in the driver:
Yu Zhao15b49be2009-03-20 11:25:18 +080058 int pci_enable_sriov(struct pci_dev *dev, int nr_virtfn);
59 'nr_virtfn' is number of VFs to be enabled.
Donald Dutile2597ba72012-11-27 22:31:37 -050060(b) For the second method, from sysfs:
61 echo 'nr_virtfn' > \
62 /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_numvfs
Yu Zhao15b49be2009-03-20 11:25:18 +080063
64To disable SR-IOV capability:
Donald Dutile2597ba72012-11-27 22:31:37 -050065(a) For the first method, in the driver:
Yu Zhao15b49be2009-03-20 11:25:18 +080066 void pci_disable_sriov(struct pci_dev *dev);
Donald Dutile2597ba72012-11-27 22:31:37 -050067(b) For the second method, from sysfs:
68 echo 0 > \
69 /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_numvfs
Yu Zhao15b49be2009-03-20 11:25:18 +080070
71To notify SR-IOV core of Virtual Function Migration:
Donald Dutile2597ba72012-11-27 22:31:37 -050072(a) In the driver:
Yu Zhao15b49be2009-03-20 11:25:18 +080073 irqreturn_t pci_sriov_migration(struct pci_dev *dev);
74
753.2 Usage example
76
77Following piece of code illustrates the usage of the SR-IOV API.
78
Greg Kroah-Hartman63a29f72012-12-21 15:15:02 -080079static int dev_probe(struct pci_dev *dev, const struct pci_device_id *id)
Yu Zhao15b49be2009-03-20 11:25:18 +080080{
81 pci_enable_sriov(dev, NR_VIRTFN);
82
83 ...
84
85 return 0;
86}
87
Greg Kroah-Hartman63a29f72012-12-21 15:15:02 -080088static void dev_remove(struct pci_dev *dev)
Yu Zhao15b49be2009-03-20 11:25:18 +080089{
90 pci_disable_sriov(dev);
91
92 ...
93}
94
95static int dev_suspend(struct pci_dev *dev, pm_message_t state)
96{
97 ...
98
99 return 0;
100}
101
102static int dev_resume(struct pci_dev *dev)
103{
104 ...
105
106 return 0;
107}
108
109static void dev_shutdown(struct pci_dev *dev)
110{
111 ...
112}
113
Donald Dutile2597ba72012-11-27 22:31:37 -0500114static int dev_sriov_configure(struct pci_dev *dev, int numvfs)
115{
116 if (numvfs > 0) {
117 ...
118 pci_enable_sriov(dev, numvfs);
119 ...
120 return numvfs;
121 }
122 if (numvfs == 0) {
123 ....
124 pci_disable_sriov(dev);
125 ...
126 return 0;
127 }
128}
129
Yu Zhao15b49be2009-03-20 11:25:18 +0800130static struct pci_driver dev_driver = {
131 .name = "SR-IOV Physical Function driver",
132 .id_table = dev_id_table,
133 .probe = dev_probe,
Greg Kroah-Hartman63a29f72012-12-21 15:15:02 -0800134 .remove = dev_remove,
Yu Zhao15b49be2009-03-20 11:25:18 +0800135 .suspend = dev_suspend,
136 .resume = dev_resume,
137 .shutdown = dev_shutdown,
Donald Dutile2597ba72012-11-27 22:31:37 -0500138 .sriov_configure = dev_sriov_configure,
Yu Zhao15b49be2009-03-20 11:25:18 +0800139};