Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 1 | .. -*- coding: utf-8; mode: rst -*- |
| 2 | |
| 3 | .. _overlay: |
| 4 | |
| 5 | *********************** |
| 6 | Video Overlay Interface |
| 7 | *********************** |
| 8 | |
| 9 | |
| 10 | **Also known as Framebuffer Overlay or Previewing** |
| 11 | Video overlay devices have the ability to genlock (TV-)video into the |
| 12 | (VGA-)video signal of a graphics card, or to store captured images |
| 13 | directly in video memory of a graphics card, typically with clipping. |
| 14 | This can be considerable more efficient than capturing images and |
| 15 | displaying them by other means. In the old days when only nuclear power |
| 16 | plants needed cooling towers this used to be the only way to put live |
| 17 | video into a window. |
| 18 | |
| 19 | Video overlay devices are accessed through the same character special |
| 20 | files as :ref:`video capture <capture>` devices. Note the default |
| 21 | function of a ``/dev/video`` device is video capturing. The overlay |
| 22 | function is only available after calling the |
Mauro Carvalho Chehab | af4a4d0 | 2016-07-01 13:42:29 -0300 | [diff] [blame^] | 23 | :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. |
Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 24 | |
| 25 | The driver may support simultaneous overlay and capturing using the |
| 26 | read/write and streaming I/O methods. If so, operation at the nominal |
| 27 | frame rate of the video standard is not guaranteed. Frames may be |
| 28 | directed away from overlay to capture, or one field may be used for |
| 29 | overlay and the other for capture if the capture parameters permit this. |
| 30 | |
| 31 | Applications should use different file descriptors for capturing and |
| 32 | overlay. This must be supported by all drivers capable of simultaneous |
| 33 | capturing and overlay. Optionally these drivers may also permit |
| 34 | capturing and overlay with a single file descriptor for compatibility |
| 35 | with V4L and earlier versions of V4L2. [1]_ |
| 36 | |
| 37 | |
| 38 | Querying Capabilities |
| 39 | ===================== |
| 40 | |
| 41 | Devices supporting the video overlay interface set the |
| 42 | ``V4L2_CAP_VIDEO_OVERLAY`` flag in the ``capabilities`` field of struct |
| 43 | :ref:`v4l2_capability <v4l2-capability>` returned by the |
Mauro Carvalho Chehab | af4a4d0 | 2016-07-01 13:42:29 -0300 | [diff] [blame^] | 44 | :ref:`VIDIOC_QUERYCAP <VIDIOC_QUERYCAP>` ioctl. The overlay I/O |
Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 45 | method specified below must be supported. Tuners and audio inputs are |
| 46 | optional. |
| 47 | |
| 48 | |
| 49 | Supplemental Functions |
| 50 | ====================== |
| 51 | |
| 52 | Video overlay devices shall support :ref:`audio input <audio>`, |
| 53 | :ref:`tuner <tuner>`, :ref:`controls <control>`, |
| 54 | :ref:`cropping and scaling <crop>` and |
| 55 | :ref:`streaming parameter <streaming-par>` ioctls as needed. The |
| 56 | :ref:`video input <video>` and :ref:`video standard <standard>` |
| 57 | ioctls must be supported by all video overlay devices. |
| 58 | |
| 59 | |
| 60 | Setup |
| 61 | ===== |
| 62 | |
| 63 | Before overlay can commence applications must program the driver with |
| 64 | frame buffer parameters, namely the address and size of the frame buffer |
| 65 | and the image format, for example RGB 5:6:5. The |
Mauro Carvalho Chehab | af4a4d0 | 2016-07-01 13:42:29 -0300 | [diff] [blame^] | 66 | :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` and |
| 67 | :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctls are available to get and |
Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 68 | set these parameters, respectively. The ``VIDIOC_S_FBUF`` ioctl is |
| 69 | privileged because it allows to set up DMA into physical memory, |
| 70 | bypassing the memory protection mechanisms of the kernel. Only the |
| 71 | superuser can change the frame buffer address and size. Users are not |
| 72 | supposed to run TV applications as root or with SUID bit set. A small |
| 73 | helper application with suitable privileges should query the graphics |
| 74 | system and program the V4L2 driver at the appropriate time. |
| 75 | |
| 76 | Some devices add the video overlay to the output signal of the graphics |
| 77 | card. In this case the frame buffer is not modified by the video device, |
| 78 | and the frame buffer address and pixel format are not needed by the |
| 79 | driver. The ``VIDIOC_S_FBUF`` ioctl is not privileged. An application |
| 80 | can check for this type of device by calling the ``VIDIOC_G_FBUF`` |
| 81 | ioctl. |
| 82 | |
| 83 | A driver may support any (or none) of five clipping/blending methods: |
| 84 | |
| 85 | 1. Chroma-keying displays the overlaid image only where pixels in the |
| 86 | primary graphics surface assume a certain color. |
| 87 | |
| 88 | 2. A bitmap can be specified where each bit corresponds to a pixel in |
| 89 | the overlaid image. When the bit is set, the corresponding video |
| 90 | pixel is displayed, otherwise a pixel of the graphics surface. |
| 91 | |
| 92 | 3. A list of clipping rectangles can be specified. In these regions *no* |
| 93 | video is displayed, so the graphics surface can be seen here. |
| 94 | |
| 95 | 4. The framebuffer has an alpha channel that can be used to clip or |
| 96 | blend the framebuffer with the video. |
| 97 | |
| 98 | 5. A global alpha value can be specified to blend the framebuffer |
| 99 | contents with video images. |
| 100 | |
| 101 | When simultaneous capturing and overlay is supported and the hardware |
| 102 | prohibits different image and frame buffer formats, the format requested |
| 103 | first takes precedence. The attempt to capture |
Mauro Carvalho Chehab | af4a4d0 | 2016-07-01 13:42:29 -0300 | [diff] [blame^] | 104 | (:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) or overlay |
| 105 | (:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`) may fail with an EBUSY error |
Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 106 | code or return accordingly modified parameters.. |
| 107 | |
| 108 | |
| 109 | Overlay Window |
| 110 | ============== |
| 111 | |
| 112 | The overlaid image is determined by cropping and overlay window |
| 113 | parameters. The former select an area of the video picture to capture, |
| 114 | the latter how images are overlaid and clipped. Cropping initialization |
| 115 | at minimum requires to reset the parameters to defaults. An example is |
| 116 | given in :ref:`crop`. |
| 117 | |
| 118 | The overlay window is described by a struct |
| 119 | :ref:`v4l2_window <v4l2-window>`. It defines the size of the image, |
| 120 | its position over the graphics surface and the clipping to be applied. |
| 121 | To get the current parameters applications set the ``type`` field of a |
| 122 | struct :ref:`v4l2_format <v4l2-format>` to |
| 123 | ``V4L2_BUF_TYPE_VIDEO_OVERLAY`` and call the |
Mauro Carvalho Chehab | af4a4d0 | 2016-07-01 13:42:29 -0300 | [diff] [blame^] | 124 | :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` ioctl. The driver fills the |
Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 125 | :c:type:`struct v4l2_window` substructure named ``win``. It is not |
| 126 | possible to retrieve a previously programmed clipping list or bitmap. |
| 127 | |
| 128 | To program the overlay window applications set the ``type`` field of a |
| 129 | struct :ref:`v4l2_format <v4l2-format>` to |
| 130 | ``V4L2_BUF_TYPE_VIDEO_OVERLAY``, initialize the ``win`` substructure and |
Mauro Carvalho Chehab | af4a4d0 | 2016-07-01 13:42:29 -0300 | [diff] [blame^] | 131 | call the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. The driver |
Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 132 | adjusts the parameters against hardware limits and returns the actual |
| 133 | parameters as ``VIDIOC_G_FMT`` does. Like ``VIDIOC_S_FMT``, the |
Mauro Carvalho Chehab | af4a4d0 | 2016-07-01 13:42:29 -0300 | [diff] [blame^] | 134 | :ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl can be used to learn |
Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 135 | about driver capabilities without actually changing driver state. Unlike |
| 136 | ``VIDIOC_S_FMT`` this also works after the overlay has been enabled. |
| 137 | |
| 138 | The scaling factor of the overlaid image is implied by the width and |
| 139 | height given in struct :ref:`v4l2_window <v4l2-window>` and the size |
| 140 | of the cropping rectangle. For more information see :ref:`crop`. |
| 141 | |
| 142 | When simultaneous capturing and overlay is supported and the hardware |
| 143 | prohibits different image and window sizes, the size requested first |
| 144 | takes precedence. The attempt to capture or overlay as well |
Mauro Carvalho Chehab | af4a4d0 | 2016-07-01 13:42:29 -0300 | [diff] [blame^] | 145 | (:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) may fail with an EBUSY error |
Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 146 | code or return accordingly modified parameters. |
| 147 | |
| 148 | |
| 149 | .. _v4l2-window: |
| 150 | |
| 151 | struct v4l2_window |
| 152 | ------------------ |
| 153 | |
| 154 | ``struct v4l2_rect w`` |
| 155 | Size and position of the window relative to the top, left corner of |
| 156 | the frame buffer defined with |
Mauro Carvalho Chehab | af4a4d0 | 2016-07-01 13:42:29 -0300 | [diff] [blame^] | 157 | :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`. The window can extend the |
Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 158 | frame buffer width and height, the ``x`` and ``y`` coordinates can |
| 159 | be negative, and it can lie completely outside the frame buffer. The |
| 160 | driver clips the window accordingly, or if that is not possible, |
| 161 | modifies its size and/or position. |
| 162 | |
| 163 | ``enum v4l2_field field`` |
| 164 | Applications set this field to determine which video field shall be |
| 165 | overlaid, typically one of ``V4L2_FIELD_ANY`` (0), |
| 166 | ``V4L2_FIELD_TOP``, ``V4L2_FIELD_BOTTOM`` or |
| 167 | ``V4L2_FIELD_INTERLACED``. Drivers may have to choose a different |
| 168 | field order and return the actual setting here. |
| 169 | |
| 170 | ``__u32 chromakey`` |
| 171 | When chroma-keying has been negotiated with |
Mauro Carvalho Chehab | af4a4d0 | 2016-07-01 13:42:29 -0300 | [diff] [blame^] | 172 | :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` applications set this field |
Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 173 | to the desired pixel value for the chroma key. The format is the |
| 174 | same as the pixel format of the framebuffer (struct |
| 175 | :ref:`v4l2_framebuffer <v4l2-framebuffer>` ``fmt.pixelformat`` |
| 176 | field), with bytes in host order. E. g. for |
| 177 | :ref:`V4L2_PIX_FMT_BGR24 <V4L2-PIX-FMT-BGR32>` the value should |
| 178 | be 0xRRGGBB on a little endian, 0xBBGGRR on a big endian host. |
| 179 | |
| 180 | ``struct v4l2_clip * clips`` |
| 181 | When chroma-keying has *not* been negotiated and |
Mauro Carvalho Chehab | af4a4d0 | 2016-07-01 13:42:29 -0300 | [diff] [blame^] | 182 | :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability, |
Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 183 | applications can set this field to point to an array of clipping |
| 184 | rectangles. |
| 185 | |
| 186 | Like the window coordinates |
| 187 | w |
| 188 | , clipping rectangles are defined relative to the top, left corner of |
| 189 | the frame buffer. However clipping rectangles must not extend the frame |
| 190 | buffer width and height, and they must not overlap. If possible |
| 191 | applications should merge adjacent rectangles. Whether this must create |
| 192 | x-y or y-x bands, or the order of rectangles, is not defined. When clip |
| 193 | lists are not supported the driver ignores this field. Its contents |
| 194 | after calling |
Mauro Carvalho Chehab | af4a4d0 | 2016-07-01 13:42:29 -0300 | [diff] [blame^] | 195 | !ri!:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` |
Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 196 | are undefined. |
| 197 | |
| 198 | ``__u32 clipcount`` |
| 199 | When the application set the ``clips`` field, this field must |
| 200 | contain the number of clipping rectangles in the list. When clip |
| 201 | lists are not supported the driver ignores this field, its contents |
| 202 | after calling ``VIDIOC_S_FMT`` are undefined. When clip lists are |
| 203 | supported but no clipping is desired this field must be set to zero. |
| 204 | |
| 205 | ``void * bitmap`` |
| 206 | When chroma-keying has *not* been negotiated and |
Mauro Carvalho Chehab | af4a4d0 | 2016-07-01 13:42:29 -0300 | [diff] [blame^] | 207 | :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability, |
Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 208 | applications can set this field to point to a clipping bit mask. |
| 209 | |
| 210 | It must be of the same size as the window, ``w.width`` and ``w.height``. |
| 211 | Each bit corresponds to a pixel in the overlaid image, which is |
| 212 | displayed only when the bit is *set*. Pixel coordinates translate to |
| 213 | bits like: |
| 214 | |
| 215 | |
| 216 | .. code-block:: c |
| 217 | |
| 218 | ((__u8 *) bitmap)[w.width * y + x / 8] & (1 << (x & 7)) |
| 219 | |
| 220 | where ``0`` ≤ x < ``w.width`` and ``0`` ≤ y <``w.height``. [2]_ |
| 221 | |
| 222 | When a clipping bit mask is not supported the driver ignores this field, |
Mauro Carvalho Chehab | af4a4d0 | 2016-07-01 13:42:29 -0300 | [diff] [blame^] | 223 | its contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are |
Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 224 | undefined. When a bit mask is supported but no clipping is desired this |
| 225 | field must be set to ``NULL``. |
| 226 | |
| 227 | Applications need not create a clip list or bit mask. When they pass |
| 228 | both, or despite negotiating chroma-keying, the results are undefined. |
| 229 | Regardless of the chosen method, the clipping abilities of the hardware |
| 230 | may be limited in quantity or quality. The results when these limits are |
| 231 | exceeded are undefined. [3]_ |
| 232 | |
| 233 | ``__u8 global_alpha`` |
| 234 | The global alpha value used to blend the framebuffer with video |
| 235 | images, if global alpha blending has been negotiated |
| 236 | (``V4L2_FBUF_FLAG_GLOBAL_ALPHA``, see |
Mauro Carvalho Chehab | af4a4d0 | 2016-07-01 13:42:29 -0300 | [diff] [blame^] | 237 | :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`, |
Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 238 | :ref:`framebuffer-flags`). |
| 239 | |
| 240 | Note this field was added in Linux 2.6.23, extending the structure. |
| 241 | However the |
Mauro Carvalho Chehab | af4a4d0 | 2016-07-01 13:42:29 -0300 | [diff] [blame^] | 242 | !ri!:ref:`VIDIOC_G/S/TRY_FMT <VIDIOC_G_FMT>` |
Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 243 | ioctls, which take a pointer to a |
| 244 | !ri!:ref:`v4l2_format <v4l2-format>` |
| 245 | parent structure with padding bytes at the end, are not affected. |
| 246 | |
| 247 | |
| 248 | .. _v4l2-clip: |
| 249 | |
| 250 | struct v4l2_clip |
| 251 | ---------------- |
| 252 | |
| 253 | The X Window system defines "regions" which are vectors of struct BoxRec |
| 254 | { short x1, y1, x2, y2; } with width = x2 - x1 and height = y2 - y1, so |
| 255 | one cannot pass X11 clip lists directly. |
| 256 | |
| 257 | ``struct v4l2_rect c`` |
| 258 | Coordinates of the clipping rectangle, relative to the top, left |
| 259 | corner of the frame buffer. Only window pixels *outside* all |
| 260 | clipping rectangles are displayed. |
| 261 | |
| 262 | ``struct v4l2_clip * next`` |
| 263 | Pointer to the next clipping rectangle, NULL when this is the last |
| 264 | rectangle. Drivers ignore this field, it cannot be used to pass a |
| 265 | linked list of clipping rectangles. |
| 266 | |
| 267 | |
| 268 | .. _v4l2-rect: |
| 269 | |
| 270 | struct v4l2_rect |
| 271 | ---------------- |
| 272 | |
| 273 | ``__s32 left`` |
| 274 | Horizontal offset of the top, left corner of the rectangle, in |
| 275 | pixels. |
| 276 | |
| 277 | ``__s32 top`` |
| 278 | Vertical offset of the top, left corner of the rectangle, in pixels. |
| 279 | Offsets increase to the right and down. |
| 280 | |
| 281 | ``__u32 width`` |
| 282 | Width of the rectangle, in pixels. |
| 283 | |
| 284 | ``__u32 height`` |
| 285 | Height of the rectangle, in pixels. |
| 286 | |
| 287 | |
| 288 | Enabling Overlay |
| 289 | ================ |
| 290 | |
| 291 | To start or stop the frame buffer overlay applications call the |
Mauro Carvalho Chehab | af4a4d0 | 2016-07-01 13:42:29 -0300 | [diff] [blame^] | 292 | :ref:`VIDIOC_OVERLAY <VIDIOC_OVERLAY>` ioctl. |
Markus Heiser | 5377d91 | 2016-06-30 15:18:56 +0200 | [diff] [blame] | 293 | |
| 294 | .. [1] |
| 295 | A common application of two file descriptors is the XFree86 |
| 296 | :ref:`Xv/V4L <xvideo>` interface driver and a V4L2 application. |
| 297 | While the X server controls video overlay, the application can take |
| 298 | advantage of memory mapping and DMA. |
| 299 | |
| 300 | In the opinion of the designers of this API, no driver writer taking |
| 301 | the efforts to support simultaneous capturing and overlay will |
| 302 | restrict this ability by requiring a single file descriptor, as in |
| 303 | V4L and earlier versions of V4L2. Making this optional means |
| 304 | applications depending on two file descriptors need backup routines |
| 305 | to be compatible with all drivers, which is considerable more work |
| 306 | than using two fds in applications which do not. Also two fd's fit |
| 307 | the general concept of one file descriptor for each logical stream. |
| 308 | Hence as a complexity trade-off drivers *must* support two file |
| 309 | descriptors and *may* support single fd operation. |
| 310 | |
| 311 | .. [2] |
| 312 | Should we require ``w.width`` to be a multiple of eight? |
| 313 | |
| 314 | .. [3] |
| 315 | When the image is written into frame buffer memory it will be |
| 316 | undesirable if the driver clips out less pixels than expected, |
| 317 | because the application and graphics system are not aware these |
| 318 | regions need to be refreshed. The driver should clip out more pixels |
| 319 | or not write the image at all. |
| 320 | |
| 321 | |
| 322 | .. ------------------------------------------------------------------------------ |
| 323 | .. This file was automatically converted from DocBook-XML with the dbxml |
| 324 | .. library (https://github.com/return42/sphkerneldoc). The origin XML comes |
| 325 | .. from the linux kernel, refer to: |
| 326 | .. |
| 327 | .. * https://github.com/torvalds/linux/tree/master/Documentation/DocBook |
| 328 | .. ------------------------------------------------------------------------------ |