Mauro Carvalho Chehab | c0d0138 | 2016-07-17 20:44:13 -0300 | [diff] [blame] | 1 | The Soc-Camera Drivers |
| 2 | ====================== |
| 3 | |
| 4 | Author: Guennadi Liakhovetski <g.liakhovetski@gmx.de> |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 5 | |
| 6 | Terminology |
| 7 | ----------- |
| 8 | |
| 9 | The following terms are used in this document: |
| 10 | - camera / camera device / camera sensor - a video-camera sensor chip, capable |
| 11 | of connecting to a variety of systems and interfaces, typically uses i2c for |
| 12 | control and configuration, and a parallel or a serial bus for data. |
| 13 | - camera host - an interface, to which a camera is connected. Typically a |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 14 | specialised interface, present on many SoCs, e.g. PXA27x and PXA3xx, SuperH, |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 15 | AVR32, i.MX27, i.MX31. |
| 16 | - camera host bus - a connection between a camera host and a camera. Can be |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 17 | parallel or serial, consists of data and control lines, e.g. clock, vertical |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 18 | and horizontal synchronization signals. |
| 19 | |
| 20 | Purpose of the soc-camera subsystem |
| 21 | ----------------------------------- |
| 22 | |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 23 | The soc-camera subsystem initially provided a unified API between camera host |
| 24 | drivers and camera sensor drivers. Later the soc-camera sensor API has been |
| 25 | replaced with the V4L2 standard subdev API. This also made camera driver re-use |
| 26 | with non-soc-camera hosts possible. The camera host API to the soc-camera core |
| 27 | has been preserved. |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 28 | |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 29 | Soc-camera implements a V4L2 interface to the user, currently only the "mmap" |
| 30 | method is supported by host drivers. However, the soc-camera core also provides |
| 31 | support for the "read" method. |
| 32 | |
| 33 | The subsystem has been designed to support multiple camera host interfaces and |
| 34 | multiple cameras per interface, although most applications have only one camera |
| 35 | sensor. |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 36 | |
| 37 | Existing drivers |
| 38 | ---------------- |
| 39 | |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 40 | As of 3.7 there are seven host drivers in the mainline: atmel-isi.c, |
| 41 | mx1_camera.c (broken, scheduled for removal), mx2_camera.c, mx3_camera.c, |
| 42 | omap1_camera.c, pxa_camera.c, sh_mobile_ceu_camera.c, and multiple sensor |
| 43 | drivers under drivers/media/i2c/soc_camera/. |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 44 | |
| 45 | Camera host API |
| 46 | --------------- |
| 47 | |
| 48 | A host camera driver is registered using the |
| 49 | |
Mauro Carvalho Chehab | c0d0138 | 2016-07-17 20:44:13 -0300 | [diff] [blame] | 50 | .. code-block:: none |
| 51 | |
| 52 | soc_camera_host_register(struct soc_camera_host *); |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 53 | |
| 54 | function. The host object can be initialized as follows: |
| 55 | |
Mauro Carvalho Chehab | c0d0138 | 2016-07-17 20:44:13 -0300 | [diff] [blame] | 56 | .. code-block:: none |
| 57 | |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 58 | struct soc_camera_host *ici; |
| 59 | ici->drv_name = DRV_NAME; |
| 60 | ici->ops = &camera_host_ops; |
| 61 | ici->priv = pcdev; |
| 62 | ici->v4l2_dev.dev = &pdev->dev; |
| 63 | ici->nr = pdev->id; |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 64 | |
| 65 | All camera host methods are passed in a struct soc_camera_host_ops: |
| 66 | |
Mauro Carvalho Chehab | c0d0138 | 2016-07-17 20:44:13 -0300 | [diff] [blame] | 67 | .. code-block:: none |
| 68 | |
| 69 | static struct soc_camera_host_ops camera_host_ops = { |
| 70 | .owner = THIS_MODULE, |
| 71 | .add = camera_add_device, |
| 72 | .remove = camera_remove_device, |
| 73 | .set_fmt = camera_set_fmt_cap, |
| 74 | .try_fmt = camera_try_fmt_cap, |
| 75 | .init_videobuf2 = camera_init_videobuf2, |
| 76 | .poll = camera_poll, |
| 77 | .querycap = camera_querycap, |
| 78 | .set_bus_param = camera_set_bus_param, |
| 79 | /* The rest of host operations are optional */ |
| 80 | }; |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 81 | |
| 82 | .add and .remove methods are called when a sensor is attached to or detached |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 83 | from the host. .set_bus_param is used to configure physical connection |
| 84 | parameters between the host and the sensor. .init_videobuf2 is called by |
| 85 | soc-camera core when a video-device is opened, the host driver would typically |
| 86 | call vb2_queue_init() in this method. Further video-buffer management is |
| 87 | implemented completely by the specific camera host driver. If the host driver |
| 88 | supports non-standard pixel format conversion, it should implement a |
| 89 | .get_formats and, possibly, a .put_formats operations. See below for more |
| 90 | details about format conversion. The rest of the methods are called from |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 91 | respective V4L2 operations. |
| 92 | |
| 93 | Camera API |
| 94 | ---------- |
| 95 | |
| 96 | Sensor drivers can use struct soc_camera_link, typically provided by the |
| 97 | platform, and used to specify to which camera host bus the sensor is connected, |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 98 | and optionally provide platform .power and .reset methods for the camera. This |
| 99 | struct is provided to the camera driver via the I2C client device platform data |
| 100 | and can be obtained, using the soc_camera_i2c_to_link() macro. Care should be |
| 101 | taken, when using soc_camera_vdev_to_subdev() and when accessing struct |
| 102 | soc_camera_device, using v4l2_get_subdev_hostdata(): both only work, when |
| 103 | running on an soc-camera host. The actual camera driver operation is implemented |
| 104 | using the V4L2 subdev API. Additionally soc-camera camera drivers can use |
| 105 | auxiliary soc-camera helper functions like soc_camera_power_on() and |
| 106 | soc_camera_power_off(), which switch regulators, provided by the platform and call |
| 107 | board-specific power switching methods. soc_camera_apply_board_flags() takes |
| 108 | camera bus configuration capability flags and applies any board transformations, |
| 109 | e.g. signal polarity inversion. soc_mbus_get_fmtdesc() can be used to obtain a |
| 110 | pixel format descriptor, corresponding to a certain media-bus pixel format code. |
| 111 | soc_camera_limit_side() can be used to restrict beginning and length of a frame |
| 112 | side, based on camera capabilities. |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 113 | |
Guennadi Liakhovetski | 6a6c878 | 2009-08-25 11:50:46 -0300 | [diff] [blame] | 114 | VIDIOC_S_CROP and VIDIOC_S_FMT behaviour |
| 115 | ---------------------------------------- |
| 116 | |
| 117 | Above user ioctls modify image geometry as follows: |
| 118 | |
| 119 | VIDIOC_S_CROP: sets location and sizes of the sensor window. Unit is one sensor |
| 120 | pixel. Changing sensor window sizes preserves any scaling factors, therefore |
| 121 | user window sizes change as well. |
| 122 | |
| 123 | VIDIOC_S_FMT: sets user window. Should preserve previously set sensor window as |
| 124 | much as possible by modifying scaling factors. If the sensor window cannot be |
| 125 | preserved precisely, it may be changed too. |
| 126 | |
Stefan Huber | 8bfcb93 | 2013-06-05 12:24:33 +0200 | [diff] [blame] | 127 | In soc-camera there are two locations, where scaling and cropping can take |
Guennadi Liakhovetski | 6a6c878 | 2009-08-25 11:50:46 -0300 | [diff] [blame] | 128 | place: in the camera driver and in the host driver. User ioctls are first passed |
| 129 | to the host driver, which then generally passes them down to the camera driver. |
| 130 | It is more efficient to perform scaling and cropping in the camera driver to |
| 131 | save camera bus bandwidth and maximise the framerate. However, if the camera |
| 132 | driver failed to set the required parameters with sufficient precision, the host |
| 133 | driver may decide to also use its own scaling and cropping to fulfill the user's |
| 134 | request. |
| 135 | |
| 136 | Camera drivers are interfaced to the soc-camera core and to host drivers over |
| 137 | the v4l2-subdev API, which is completely functional, it doesn't pass any data. |
| 138 | Therefore all camera drivers shall reply to .g_fmt() requests with their current |
| 139 | output geometry. This is necessary to correctly configure the camera bus. |
| 140 | .s_fmt() and .try_fmt() have to be implemented too. Sensor window and scaling |
| 141 | factors have to be maintained by camera drivers internally. According to the |
| 142 | V4L2 API all capture drivers must support the VIDIOC_CROPCAP ioctl, hence we |
| 143 | rely on camera drivers implementing .cropcap(). If the camera driver does not |
| 144 | support cropping, it may choose to not implement .s_crop(), but to enable |
| 145 | cropping support by the camera host driver at least the .g_crop method must be |
| 146 | implemented. |
| 147 | |
| 148 | User window geometry is kept in .user_width and .user_height fields in struct |
| 149 | soc_camera_device and used by the soc-camera core and host drivers. The core |
| 150 | updates these fields upon successful completion of a .s_fmt() call, but if these |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 151 | fields change elsewhere, e.g. during .s_crop() processing, the host driver is |
Guennadi Liakhovetski | 6a6c878 | 2009-08-25 11:50:46 -0300 | [diff] [blame] | 152 | responsible for updating them. |
| 153 | |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 154 | Format conversion |
| 155 | ----------------- |
| 156 | |
| 157 | V4L2 distinguishes between pixel formats, as they are stored in memory, and as |
| 158 | they are transferred over a media bus. Soc-camera provides support to |
| 159 | conveniently manage these formats. A table of standard transformations is |
| 160 | maintained by soc-camera core, which describes, what FOURCC pixel format will |
| 161 | be obtained, if a media-bus pixel format is stored in memory according to |
Boris BREZILLON | 27ffaeb | 2014-11-10 14:28:31 -0300 | [diff] [blame] | 162 | certain rules. E.g. if MEDIA_BUS_FMT_YUYV8_2X8 data is sampled with 8 bits per |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 163 | sample and stored in memory in the little-endian order with no gaps between |
| 164 | bytes, data in memory will represent the V4L2_PIX_FMT_YUYV FOURCC format. These |
| 165 | standard transformations will be used by soc-camera or by camera host drivers to |
| 166 | configure camera drivers to produce the FOURCC format, requested by the user, |
| 167 | using the VIDIOC_S_FMT ioctl(). Apart from those standard format conversions, |
| 168 | host drivers can also provide their own conversion rules by implementing a |
| 169 | .get_formats and, if required, a .put_formats methods. |