Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 1 | Soc-Camera Subsystem |
| 2 | ==================== |
| 3 | |
| 4 | Terminology |
| 5 | ----------- |
| 6 | |
| 7 | The following terms are used in this document: |
| 8 | - camera / camera device / camera sensor - a video-camera sensor chip, capable |
| 9 | of connecting to a variety of systems and interfaces, typically uses i2c for |
| 10 | control and configuration, and a parallel or a serial bus for data. |
| 11 | - camera host - an interface, to which a camera is connected. Typically a |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 12 | specialised interface, present on many SoCs, e.g. PXA27x and PXA3xx, SuperH, |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 13 | AVR32, i.MX27, i.MX31. |
| 14 | - camera host bus - a connection between a camera host and a camera. Can be |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 15 | parallel or serial, consists of data and control lines, e.g. clock, vertical |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 16 | and horizontal synchronization signals. |
| 17 | |
| 18 | Purpose of the soc-camera subsystem |
| 19 | ----------------------------------- |
| 20 | |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 21 | The soc-camera subsystem initially provided a unified API between camera host |
| 22 | drivers and camera sensor drivers. Later the soc-camera sensor API has been |
| 23 | replaced with the V4L2 standard subdev API. This also made camera driver re-use |
| 24 | with non-soc-camera hosts possible. The camera host API to the soc-camera core |
| 25 | has been preserved. |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 26 | |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 27 | Soc-camera implements a V4L2 interface to the user, currently only the "mmap" |
| 28 | method is supported by host drivers. However, the soc-camera core also provides |
| 29 | support for the "read" method. |
| 30 | |
| 31 | The subsystem has been designed to support multiple camera host interfaces and |
| 32 | multiple cameras per interface, although most applications have only one camera |
| 33 | sensor. |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 34 | |
| 35 | Existing drivers |
| 36 | ---------------- |
| 37 | |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 38 | As of 3.7 there are seven host drivers in the mainline: atmel-isi.c, |
| 39 | mx1_camera.c (broken, scheduled for removal), mx2_camera.c, mx3_camera.c, |
| 40 | omap1_camera.c, pxa_camera.c, sh_mobile_ceu_camera.c, and multiple sensor |
| 41 | drivers under drivers/media/i2c/soc_camera/. |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 42 | |
| 43 | Camera host API |
| 44 | --------------- |
| 45 | |
| 46 | A host camera driver is registered using the |
| 47 | |
| 48 | soc_camera_host_register(struct soc_camera_host *); |
| 49 | |
| 50 | function. The host object can be initialized as follows: |
| 51 | |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 52 | struct soc_camera_host *ici; |
| 53 | ici->drv_name = DRV_NAME; |
| 54 | ici->ops = &camera_host_ops; |
| 55 | ici->priv = pcdev; |
| 56 | ici->v4l2_dev.dev = &pdev->dev; |
| 57 | ici->nr = pdev->id; |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 58 | |
| 59 | All camera host methods are passed in a struct soc_camera_host_ops: |
| 60 | |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 61 | static struct soc_camera_host_ops camera_host_ops = { |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 62 | .owner = THIS_MODULE, |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 63 | .add = camera_add_device, |
| 64 | .remove = camera_remove_device, |
| 65 | .set_fmt = camera_set_fmt_cap, |
| 66 | .try_fmt = camera_try_fmt_cap, |
| 67 | .init_videobuf2 = camera_init_videobuf2, |
| 68 | .poll = camera_poll, |
| 69 | .querycap = camera_querycap, |
| 70 | .set_bus_param = camera_set_bus_param, |
| 71 | /* The rest of host operations are optional */ |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 72 | }; |
| 73 | |
| 74 | .add and .remove methods are called when a sensor is attached to or detached |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 75 | from the host. .set_bus_param is used to configure physical connection |
| 76 | parameters between the host and the sensor. .init_videobuf2 is called by |
| 77 | soc-camera core when a video-device is opened, the host driver would typically |
| 78 | call vb2_queue_init() in this method. Further video-buffer management is |
| 79 | implemented completely by the specific camera host driver. If the host driver |
| 80 | supports non-standard pixel format conversion, it should implement a |
| 81 | .get_formats and, possibly, a .put_formats operations. See below for more |
| 82 | details about format conversion. The rest of the methods are called from |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 83 | respective V4L2 operations. |
| 84 | |
| 85 | Camera API |
| 86 | ---------- |
| 87 | |
| 88 | Sensor drivers can use struct soc_camera_link, typically provided by the |
| 89 | platform, and used to specify to which camera host bus the sensor is connected, |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 90 | and optionally provide platform .power and .reset methods for the camera. This |
| 91 | struct is provided to the camera driver via the I2C client device platform data |
| 92 | and can be obtained, using the soc_camera_i2c_to_link() macro. Care should be |
| 93 | taken, when using soc_camera_vdev_to_subdev() and when accessing struct |
| 94 | soc_camera_device, using v4l2_get_subdev_hostdata(): both only work, when |
| 95 | running on an soc-camera host. The actual camera driver operation is implemented |
| 96 | using the V4L2 subdev API. Additionally soc-camera camera drivers can use |
| 97 | auxiliary soc-camera helper functions like soc_camera_power_on() and |
| 98 | soc_camera_power_off(), which switch regulators, provided by the platform and call |
| 99 | board-specific power switching methods. soc_camera_apply_board_flags() takes |
| 100 | camera bus configuration capability flags and applies any board transformations, |
| 101 | e.g. signal polarity inversion. soc_mbus_get_fmtdesc() can be used to obtain a |
| 102 | pixel format descriptor, corresponding to a certain media-bus pixel format code. |
| 103 | soc_camera_limit_side() can be used to restrict beginning and length of a frame |
| 104 | side, based on camera capabilities. |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 105 | |
Guennadi Liakhovetski | 6a6c878 | 2009-08-25 11:50:46 -0300 | [diff] [blame] | 106 | VIDIOC_S_CROP and VIDIOC_S_FMT behaviour |
| 107 | ---------------------------------------- |
| 108 | |
| 109 | Above user ioctls modify image geometry as follows: |
| 110 | |
| 111 | VIDIOC_S_CROP: sets location and sizes of the sensor window. Unit is one sensor |
| 112 | pixel. Changing sensor window sizes preserves any scaling factors, therefore |
| 113 | user window sizes change as well. |
| 114 | |
| 115 | VIDIOC_S_FMT: sets user window. Should preserve previously set sensor window as |
| 116 | much as possible by modifying scaling factors. If the sensor window cannot be |
| 117 | preserved precisely, it may be changed too. |
| 118 | |
Stefan Huber | 8bfcb93 | 2013-06-05 12:24:33 +0200 | [diff] [blame^] | 119 | In soc-camera there are two locations, where scaling and cropping can take |
Guennadi Liakhovetski | 6a6c878 | 2009-08-25 11:50:46 -0300 | [diff] [blame] | 120 | place: in the camera driver and in the host driver. User ioctls are first passed |
| 121 | to the host driver, which then generally passes them down to the camera driver. |
| 122 | It is more efficient to perform scaling and cropping in the camera driver to |
| 123 | save camera bus bandwidth and maximise the framerate. However, if the camera |
| 124 | driver failed to set the required parameters with sufficient precision, the host |
| 125 | driver may decide to also use its own scaling and cropping to fulfill the user's |
| 126 | request. |
| 127 | |
| 128 | Camera drivers are interfaced to the soc-camera core and to host drivers over |
| 129 | the v4l2-subdev API, which is completely functional, it doesn't pass any data. |
| 130 | Therefore all camera drivers shall reply to .g_fmt() requests with their current |
| 131 | output geometry. This is necessary to correctly configure the camera bus. |
| 132 | .s_fmt() and .try_fmt() have to be implemented too. Sensor window and scaling |
| 133 | factors have to be maintained by camera drivers internally. According to the |
| 134 | V4L2 API all capture drivers must support the VIDIOC_CROPCAP ioctl, hence we |
| 135 | rely on camera drivers implementing .cropcap(). If the camera driver does not |
| 136 | support cropping, it may choose to not implement .s_crop(), but to enable |
| 137 | cropping support by the camera host driver at least the .g_crop method must be |
| 138 | implemented. |
| 139 | |
| 140 | User window geometry is kept in .user_width and .user_height fields in struct |
| 141 | soc_camera_device and used by the soc-camera core and host drivers. The core |
| 142 | updates these fields upon successful completion of a .s_fmt() call, but if these |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 143 | fields change elsewhere, e.g. during .s_crop() processing, the host driver is |
Guennadi Liakhovetski | 6a6c878 | 2009-08-25 11:50:46 -0300 | [diff] [blame] | 144 | responsible for updating them. |
| 145 | |
Guennadi Liakhovetski | 454547f | 2012-10-05 12:33:45 -0300 | [diff] [blame] | 146 | Format conversion |
| 147 | ----------------- |
| 148 | |
| 149 | V4L2 distinguishes between pixel formats, as they are stored in memory, and as |
| 150 | they are transferred over a media bus. Soc-camera provides support to |
| 151 | conveniently manage these formats. A table of standard transformations is |
| 152 | maintained by soc-camera core, which describes, what FOURCC pixel format will |
| 153 | be obtained, if a media-bus pixel format is stored in memory according to |
| 154 | certain rules. E.g. if V4L2_MBUS_FMT_YUYV8_2X8 data is sampled with 8 bits per |
| 155 | sample and stored in memory in the little-endian order with no gaps between |
| 156 | bytes, data in memory will represent the V4L2_PIX_FMT_YUYV FOURCC format. These |
| 157 | standard transformations will be used by soc-camera or by camera host drivers to |
| 158 | configure camera drivers to produce the FOURCC format, requested by the user, |
| 159 | using the VIDIOC_S_FMT ioctl(). Apart from those standard format conversions, |
| 160 | host drivers can also provide their own conversion rules by implementing a |
| 161 | .get_formats and, if required, a .put_formats methods. |
| 162 | |
Guennadi Liakhovetski | 2853155 | 2008-08-29 15:16:31 -0300 | [diff] [blame] | 163 | -- |
| 164 | Author: Guennadi Liakhovetski <g.liakhovetski@gmx.de> |