Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 1 | .. -*- coding: utf-8; mode: rst -*- |
| 2 | |
| 3 | .. _crop: |
| 4 | |
| 5 | ************************************* |
| 6 | Image Cropping, Insertion and Scaling |
| 7 | ************************************* |
| 8 | |
| 9 | Some video capture devices can sample a subsection of the picture and |
| 10 | shrink or enlarge it to an image of arbitrary size. We call these |
| 11 | abilities cropping and scaling. Some video output devices can scale an |
| 12 | image up or down and insert it at an arbitrary scan line and horizontal |
| 13 | offset into a video signal. |
| 14 | |
| 15 | Applications can use the following API to select an area in the video |
| 16 | signal, query the default area and the hardware limits. *Despite their |
| 17 | name, the :ref:`VIDIOC_CROPCAP <vidioc-cropcap>`, |
| 18 | :ref:`VIDIOC_G_CROP <vidioc-g-crop>` and |
| 19 | :ref:`VIDIOC_S_CROP <vidioc-g-crop>` ioctls apply to input as well |
| 20 | as output devices.* |
| 21 | |
| 22 | Scaling requires a source and a target. On a video capture or overlay |
| 23 | device the source is the video signal, and the cropping ioctls determine |
| 24 | the area actually sampled. The target are images read by the application |
| 25 | or overlaid onto the graphics screen. Their size (and position for an |
| 26 | overlay) is negotiated with the :ref:`VIDIOC_G_FMT <vidioc-g-fmt>` |
| 27 | and :ref:`VIDIOC_S_FMT <vidioc-g-fmt>` ioctls. |
| 28 | |
| 29 | On a video output device the source are the images passed in by the |
| 30 | application, and their size is again negotiated with the |
| 31 | ``VIDIOC_G/S_FMT`` ioctls, or may be encoded in a compressed video |
| 32 | stream. The target is the video signal, and the cropping ioctls |
| 33 | determine the area where the images are inserted. |
| 34 | |
| 35 | Source and target rectangles are defined even if the device does not |
| 36 | support scaling or the ``VIDIOC_G/S_CROP`` ioctls. Their size (and |
| 37 | position where applicable) will be fixed in this case. *All capture and |
| 38 | output device must support the ``VIDIOC_CROPCAP`` ioctl such that |
| 39 | applications can determine if scaling takes place.* |
| 40 | |
| 41 | |
| 42 | Cropping Structures |
| 43 | =================== |
| 44 | |
| 45 | |
| 46 | .. _crop-scale: |
| 47 | |
| 48 | .. figure:: crop_files/crop.* |
| 49 | :alt: crop.pdf / crop.gif |
| 50 | :align: center |
| 51 | |
| 52 | Image Cropping, Insertion and Scaling |
| 53 | |
| 54 | The cropping, insertion and scaling process |
| 55 | |
| 56 | |
| 57 | |
| 58 | For capture devices the coordinates of the top left corner, width and |
| 59 | height of the area which can be sampled is given by the ``bounds`` |
| 60 | substructure of the struct :ref:`v4l2_cropcap <v4l2-cropcap>` |
| 61 | returned by the ``VIDIOC_CROPCAP`` ioctl. To support a wide range of |
| 62 | hardware this specification does not define an origin or units. However |
| 63 | by convention drivers should horizontally count unscaled samples |
| 64 | relative to 0H (the leading edge of the horizontal sync pulse, see |
| 65 | :ref:`vbi-hsync`). Vertically ITU-R line numbers of the first field |
| 66 | (:ref:`vbi-525`, :ref:`vbi-625`), multiplied by two if the driver |
| 67 | can capture both fields. |
| 68 | |
| 69 | The top left corner, width and height of the source rectangle, that is |
| 70 | the area actually sampled, is given by struct |
| 71 | :ref:`v4l2_crop <v4l2-crop>` using the same coordinate system as |
| 72 | struct :ref:`v4l2_cropcap <v4l2-cropcap>`. Applications can use the |
| 73 | ``VIDIOC_G_CROP`` and ``VIDIOC_S_CROP`` ioctls to get and set this |
| 74 | rectangle. It must lie completely within the capture boundaries and the |
| 75 | driver may further adjust the requested size and/or position according |
| 76 | to hardware limitations. |
| 77 | |
| 78 | Each capture device has a default source rectangle, given by the |
| 79 | ``defrect`` substructure of struct |
| 80 | :ref:`v4l2_cropcap <v4l2-cropcap>`. The center of this rectangle |
| 81 | shall align with the center of the active picture area of the video |
| 82 | signal, and cover what the driver writer considers the complete picture. |
| 83 | Drivers shall reset the source rectangle to the default when the driver |
| 84 | is first loaded, but not later. |
| 85 | |
| 86 | For output devices these structures and ioctls are used accordingly, |
| 87 | defining the *target* rectangle where the images will be inserted into |
| 88 | the video signal. |
| 89 | |
| 90 | |
| 91 | Scaling Adjustments |
| 92 | =================== |
| 93 | |
| 94 | Video hardware can have various cropping, insertion and scaling |
| 95 | limitations. It may only scale up or down, support only discrete scaling |
| 96 | factors, or have different scaling abilities in horizontal and vertical |
| 97 | direction. Also it may not support scaling at all. At the same time the |
| 98 | struct :ref:`v4l2_crop <v4l2-crop>` rectangle may have to be aligned, |
| 99 | and both the source and target rectangles may have arbitrary upper and |
| 100 | lower size limits. In particular the maximum ``width`` and ``height`` in |
| 101 | struct :ref:`v4l2_crop <v4l2-crop>` may be smaller than the struct |
| 102 | :ref:`v4l2_cropcap <v4l2-cropcap>`. ``bounds`` area. Therefore, as |
| 103 | usual, drivers are expected to adjust the requested parameters and |
| 104 | return the actual values selected. |
| 105 | |
| 106 | Applications can change the source or the target rectangle first, as |
| 107 | they may prefer a particular image size or a certain area in the video |
| 108 | signal. If the driver has to adjust both to satisfy hardware |
| 109 | limitations, the last requested rectangle shall take priority, and the |
| 110 | driver should preferably adjust the opposite one. The |
| 111 | :ref:`VIDIOC_TRY_FMT <vidioc-g-fmt>` ioctl however shall not change |
| 112 | the driver state and therefore only adjust the requested rectangle. |
| 113 | |
| 114 | Suppose scaling on a video capture device is restricted to a factor 1:1 |
| 115 | or 2:1 in either direction and the target image size must be a multiple |
| 116 | of 16 × 16 pixels. The source cropping rectangle is set to defaults, |
| 117 | which are also the upper limit in this example, of 640 × 400 pixels at |
| 118 | offset 0, 0. An application requests an image size of 300 × 225 pixels, |
| 119 | assuming video will be scaled down from the "full picture" accordingly. |
| 120 | The driver sets the image size to the closest possible values 304 × 224, |
| 121 | then chooses the cropping rectangle closest to the requested size, that |
| 122 | is 608 × 224 (224 × 2:1 would exceed the limit 400). The offset 0, 0 is |
| 123 | still valid, thus unmodified. Given the default cropping rectangle |
| 124 | reported by ``VIDIOC_CROPCAP`` the application can easily propose |
| 125 | another offset to center the cropping rectangle. |
| 126 | |
| 127 | Now the application may insist on covering an area using a picture |
| 128 | aspect ratio closer to the original request, so it asks for a cropping |
| 129 | rectangle of 608 × 456 pixels. The present scaling factors limit |
| 130 | cropping to 640 × 384, so the driver returns the cropping size 608 × 384 |
| 131 | and adjusts the image size to closest possible 304 × 192. |
| 132 | |
| 133 | |
| 134 | Examples |
| 135 | ======== |
| 136 | |
| 137 | Source and target rectangles shall remain unchanged across closing and |
| 138 | reopening a device, such that piping data into or out of a device will |
| 139 | work without special preparations. More advanced applications should |
| 140 | ensure the parameters are suitable before starting I/O. |
| 141 | |
| 142 | (A video capture device is assumed; change |
| 143 | ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` for other devices.) |
| 144 | |
| 145 | |
| 146 | .. code-block:: c |
| 147 | |
| 148 | struct v4l2_cropcap cropcap; |
| 149 | struct v4l2_crop crop; |
| 150 | |
| 151 | memset (&cropcap, 0, sizeof (cropcap)); |
| 152 | cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; |
| 153 | |
| 154 | if (-1 == ioctl (fd, VIDIOC_CROPCAP, &cropcap)) { |
| 155 | perror ("VIDIOC_CROPCAP"); |
| 156 | exit (EXIT_FAILURE); |
| 157 | } |
| 158 | |
| 159 | memset (&crop, 0, sizeof (crop)); |
| 160 | crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; |
| 161 | crop.c = cropcap.defrect; |
| 162 | |
| 163 | /* Ignore if cropping is not supported (EINVAL). */ |
| 164 | |
| 165 | if (-1 == ioctl (fd, VIDIOC_S_CROP, &crop) |
| 166 | && errno != EINVAL) { |
| 167 | perror ("VIDIOC_S_CROP"); |
| 168 | exit (EXIT_FAILURE); |
| 169 | } |
| 170 | |
| 171 | (A video capture device is assumed.) |
| 172 | |
| 173 | |
| 174 | .. code-block:: c |
| 175 | |
| 176 | struct v4l2_cropcap cropcap; |
| 177 | struct v4l2_format format; |
| 178 | |
| 179 | reset_cropping_parameters (); |
| 180 | |
| 181 | /* Scale down to 1/4 size of full picture. */ |
| 182 | |
| 183 | memset (&format, 0, sizeof (format)); /* defaults */ |
| 184 | |
| 185 | format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; |
| 186 | |
| 187 | format.fmt.pix.width = cropcap.defrect.width >> 1; |
| 188 | format.fmt.pix.height = cropcap.defrect.height >> 1; |
| 189 | format.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV; |
| 190 | |
| 191 | if (-1 == ioctl (fd, VIDIOC_S_FMT, &format)) { |
| 192 | perror ("VIDIOC_S_FORMAT"); |
| 193 | exit (EXIT_FAILURE); |
| 194 | } |
| 195 | |
| 196 | /* We could check the actual image size now, the actual scaling factor |
| 197 | or if the driver can scale at all. */ |
| 198 | |
| 199 | |
| 200 | .. code-block:: c |
| 201 | |
| 202 | struct v4l2_cropcap cropcap; |
| 203 | struct v4l2_crop crop; |
| 204 | |
| 205 | memset (&cropcap, 0, sizeof (cropcap)); |
| 206 | cropcap.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; |
| 207 | |
| 208 | if (-1 == ioctl (fd, VIDIOC_CROPCAP;, &cropcap)) { |
| 209 | perror ("VIDIOC_CROPCAP"); |
| 210 | exit (EXIT_FAILURE); |
| 211 | } |
| 212 | |
| 213 | memset (&crop, 0, sizeof (crop)); |
| 214 | |
| 215 | crop.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; |
| 216 | crop.c = cropcap.defrect; |
| 217 | |
| 218 | /* Scale the width and height to 50 % of their original size |
| 219 | and center the output. */ |
| 220 | |
| 221 | crop.c.width /= 2; |
| 222 | crop.c.height /= 2; |
| 223 | crop.c.left += crop.c.width / 2; |
| 224 | crop.c.top += crop.c.height / 2; |
| 225 | |
| 226 | /* Ignore if cropping is not supported (EINVAL). */ |
| 227 | |
| 228 | if (-1 == ioctl (fd, VIDIOC_S_CROP, &crop) |
| 229 | && errno != EINVAL) { |
| 230 | perror ("VIDIOC_S_CROP"); |
| 231 | exit (EXIT_FAILURE); |
| 232 | } |
| 233 | |
| 234 | (A video capture device is assumed.) |
| 235 | |
| 236 | |
| 237 | .. code-block:: c |
| 238 | |
| 239 | struct v4l2_cropcap cropcap; |
| 240 | struct v4l2_crop crop; |
| 241 | struct v4l2_format format; |
| 242 | double hscale, vscale; |
| 243 | double aspect; |
| 244 | int dwidth, dheight; |
| 245 | |
| 246 | memset (&cropcap, 0, sizeof (cropcap)); |
| 247 | cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; |
| 248 | |
| 249 | if (-1 == ioctl (fd, VIDIOC_CROPCAP, &cropcap)) { |
| 250 | perror ("VIDIOC_CROPCAP"); |
| 251 | exit (EXIT_FAILURE); |
| 252 | } |
| 253 | |
| 254 | memset (&crop, 0, sizeof (crop)); |
| 255 | crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; |
| 256 | |
| 257 | if (-1 == ioctl (fd, VIDIOC_G_CROP, &crop)) { |
| 258 | if (errno != EINVAL) { |
| 259 | perror ("VIDIOC_G_CROP"); |
| 260 | exit (EXIT_FAILURE); |
| 261 | } |
| 262 | |
| 263 | /* Cropping not supported. */ |
| 264 | crop.c = cropcap.defrect; |
| 265 | } |
| 266 | |
| 267 | memset (&format, 0, sizeof (format)); |
| 268 | format.fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; |
| 269 | |
| 270 | if (-1 == ioctl (fd, VIDIOC_G_FMT, &format)) { |
| 271 | perror ("VIDIOC_G_FMT"); |
| 272 | exit (EXIT_FAILURE); |
| 273 | } |
| 274 | |
| 275 | /* The scaling applied by the driver. */ |
| 276 | |
| 277 | hscale = format.fmt.pix.width / (double) crop.c.width; |
| 278 | vscale = format.fmt.pix.height / (double) crop.c.height; |
| 279 | |
| 280 | aspect = cropcap.pixelaspect.numerator / |
| 281 | (double) cropcap.pixelaspect.denominator; |
| 282 | aspect = aspect * hscale / vscale; |
| 283 | |
| 284 | /* Devices following ITU-R BT.601 do not capture |
| 285 | square pixels. For playback on a computer monitor |
| 286 | we should scale the images to this size. */ |
| 287 | |
| 288 | dwidth = format.fmt.pix.width / aspect; |
| 289 | dheight = format.fmt.pix.height; |
| 290 | |
| 291 | |
| 292 | |
| 293 | |
| 294 | .. ------------------------------------------------------------------------------ |
| 295 | .. This file was automatically converted from DocBook-XML with the dbxml |
| 296 | .. library (https://github.com/return42/sphkerneldoc). The origin XML comes |
| 297 | .. from the linux kernel, refer to: |
| 298 | .. |
| 299 | .. * https://github.com/torvalds/linux/tree/master/Documentation/DocBook |
| 300 | .. ------------------------------------------------------------------------------ |