Mauro Carvalho Chehab | 9cbfcd7 | 2016-07-17 19:43:22 -0300 | [diff] [blame] | 1 | Cropping and Scaling algorithm, used in the sh_mobile_ceu_camera driver |
| 2 | ======================================================================= |
| 3 | |
| 4 | Author: Guennadi Liakhovetski <g.liakhovetski@gmx.de> |
Guennadi Liakhovetski | 35b23b5 | 2009-12-11 11:34:20 -0300 | [diff] [blame] | 5 | |
| 6 | Terminology |
| 7 | ----------- |
| 8 | |
| 9 | sensor scales: horizontal and vertical scales, configured by the sensor driver |
| 10 | host scales: -"- host driver |
| 11 | combined scales: sensor_scale * host_scale |
| 12 | |
| 13 | |
| 14 | Generic scaling / cropping scheme |
| 15 | --------------------------------- |
| 16 | |
Mauro Carvalho Chehab | 9cbfcd7 | 2016-07-17 19:43:22 -0300 | [diff] [blame] | 17 | .. code-block:: none |
| 18 | |
| 19 | -1-- |
| 20 | | |
| 21 | -2-- -\ |
| 22 | | --\ |
| 23 | | --\ |
| 24 | +-5-- . -- -3-- -\ |
| 25 | | `... -\ |
| 26 | | `... -4-- . - -7.. |
| 27 | | `. |
| 28 | | `. .6-- |
| 29 | | |
| 30 | | . .6'- |
| 31 | | .´ |
| 32 | | ... -4'- .´ |
| 33 | | ...´ - -7'. |
| 34 | +-5'- .´ -/ |
| 35 | | -- -3'- -/ |
| 36 | | --/ |
| 37 | | --/ |
| 38 | -2'- -/ |
| 39 | | |
| 40 | | |
| 41 | -1'- |
Guennadi Liakhovetski | 35b23b5 | 2009-12-11 11:34:20 -0300 | [diff] [blame] | 42 | |
Guennadi Liakhovetski | f7fc97a | 2010-03-23 11:42:32 -0300 | [diff] [blame] | 43 | In the above chart minuses and slashes represent "real" data amounts, points and |
Guennadi Liakhovetski | d16290d | 2011-03-28 13:16:43 -0300 | [diff] [blame] | 44 | accents represent "useful" data, basically, CEU scaled and cropped output, |
Guennadi Liakhovetski | f7fc97a | 2010-03-23 11:42:32 -0300 | [diff] [blame] | 45 | mapped back onto the client's source plane. |
| 46 | |
| 47 | Such a configuration can be produced by user requests: |
Guennadi Liakhovetski | 35b23b5 | 2009-12-11 11:34:20 -0300 | [diff] [blame] | 48 | |
| 49 | S_CROP(left / top = (5) - (1), width / height = (5') - (5)) |
| 50 | S_FMT(width / height = (6') - (6)) |
| 51 | |
| 52 | Here: |
| 53 | |
| 54 | (1) to (1') - whole max width or height |
| 55 | (1) to (2) - sensor cropped left or top |
| 56 | (2) to (2') - sensor cropped width or height |
| 57 | (3) to (3') - sensor scale |
| 58 | (3) to (4) - CEU cropped left or top |
| 59 | (4) to (4') - CEU cropped width or height |
| 60 | (5) to (5') - reverse sensor scale applied to CEU cropped width or height |
| 61 | (2) to (5) - reverse sensor scale applied to CEU cropped left or top |
| 62 | (6) to (6') - CEU scale - user window |
| 63 | |
| 64 | |
| 65 | S_FMT |
| 66 | ----- |
| 67 | |
| 68 | Do not touch input rectangle - it is already optimal. |
| 69 | |
| 70 | 1. Calculate current sensor scales: |
| 71 | |
Guennadi Liakhovetski | d16290d | 2011-03-28 13:16:43 -0300 | [diff] [blame] | 72 | scale_s = ((2') - (2)) / ((3') - (3)) |
Guennadi Liakhovetski | 35b23b5 | 2009-12-11 11:34:20 -0300 | [diff] [blame] | 73 | |
| 74 | 2. Calculate "effective" input crop (sensor subwindow) - CEU crop scaled back at |
| 75 | current sensor scales onto input window - this is user S_CROP: |
| 76 | |
| 77 | width_u = (5') - (5) = ((4') - (4)) * scale_s |
| 78 | |
| 79 | 3. Calculate new combined scales from "effective" input window to requested user |
| 80 | window: |
| 81 | |
| 82 | scale_comb = width_u / ((6') - (6)) |
| 83 | |
| 84 | 4. Calculate sensor output window by applying combined scales to real input |
| 85 | window: |
| 86 | |
Guennadi Liakhovetski | d16290d | 2011-03-28 13:16:43 -0300 | [diff] [blame] | 87 | width_s_out = ((7') - (7)) = ((2') - (2)) / scale_comb |
Guennadi Liakhovetski | 35b23b5 | 2009-12-11 11:34:20 -0300 | [diff] [blame] | 88 | |
| 89 | 5. Apply iterative sensor S_FMT for sensor output window. |
| 90 | |
| 91 | subdev->video_ops->s_fmt(.width = width_s_out) |
| 92 | |
| 93 | 6. Retrieve sensor output window (g_fmt) |
| 94 | |
| 95 | 7. Calculate new sensor scales: |
| 96 | |
| 97 | scale_s_new = ((3')_new - (3)_new) / ((2') - (2)) |
| 98 | |
| 99 | 8. Calculate new CEU crop - apply sensor scales to previously calculated |
| 100 | "effective" crop: |
| 101 | |
| 102 | width_ceu = (4')_new - (4)_new = width_u / scale_s_new |
| 103 | left_ceu = (4)_new - (3)_new = ((5) - (2)) / scale_s_new |
| 104 | |
| 105 | 9. Use CEU cropping to crop to the new window: |
| 106 | |
| 107 | ceu_crop(.width = width_ceu, .left = left_ceu) |
| 108 | |
| 109 | 10. Use CEU scaling to scale to the requested user window: |
| 110 | |
| 111 | scale_ceu = width_ceu / width |
| 112 | |
| 113 | |
| 114 | S_CROP |
| 115 | ------ |
| 116 | |
Guennadi Liakhovetski | f7fc97a | 2010-03-23 11:42:32 -0300 | [diff] [blame] | 117 | The API at http://v4l2spec.bytesex.org/spec/x1904.htm says: |
Guennadi Liakhovetski | 35b23b5 | 2009-12-11 11:34:20 -0300 | [diff] [blame] | 118 | |
Guennadi Liakhovetski | f7fc97a | 2010-03-23 11:42:32 -0300 | [diff] [blame] | 119 | "...specification does not define an origin or units. However by convention |
| 120 | drivers should horizontally count unscaled samples relative to 0H." |
Guennadi Liakhovetski | 35b23b5 | 2009-12-11 11:34:20 -0300 | [diff] [blame] | 121 | |
Guennadi Liakhovetski | f7fc97a | 2010-03-23 11:42:32 -0300 | [diff] [blame] | 122 | We choose to follow the advise and interpret cropping units as client input |
| 123 | pixels. |
Guennadi Liakhovetski | 35b23b5 | 2009-12-11 11:34:20 -0300 | [diff] [blame] | 124 | |
Guennadi Liakhovetski | f7fc97a | 2010-03-23 11:42:32 -0300 | [diff] [blame] | 125 | Cropping is performed in the following 6 steps: |
Guennadi Liakhovetski | 35b23b5 | 2009-12-11 11:34:20 -0300 | [diff] [blame] | 126 | |
Guennadi Liakhovetski | f7fc97a | 2010-03-23 11:42:32 -0300 | [diff] [blame] | 127 | 1. Request exactly user rectangle from the sensor. |
Guennadi Liakhovetski | 35b23b5 | 2009-12-11 11:34:20 -0300 | [diff] [blame] | 128 | |
Guennadi Liakhovetski | f7fc97a | 2010-03-23 11:42:32 -0300 | [diff] [blame] | 129 | 2. If smaller - iterate until a larger one is obtained. Result: sensor cropped |
| 130 | to 2 : 2', target crop 5 : 5', current output format 6' - 6. |
Guennadi Liakhovetski | 35b23b5 | 2009-12-11 11:34:20 -0300 | [diff] [blame] | 131 | |
Guennadi Liakhovetski | f7fc97a | 2010-03-23 11:42:32 -0300 | [diff] [blame] | 132 | 3. In the previous step the sensor has tried to preserve its output frame as |
| 133 | good as possible, but it could have changed. Retrieve it again. |
Guennadi Liakhovetski | 35b23b5 | 2009-12-11 11:34:20 -0300 | [diff] [blame] | 134 | |
Guennadi Liakhovetski | f7fc97a | 2010-03-23 11:42:32 -0300 | [diff] [blame] | 135 | 4. Sensor scaled to 3 : 3'. Sensor's scale is (2' - 2) / (3' - 3). Calculate |
| 136 | intermediate window: 4' - 4 = (5' - 5) * (3' - 3) / (2' - 2) |
Guennadi Liakhovetski | 35b23b5 | 2009-12-11 11:34:20 -0300 | [diff] [blame] | 137 | |
Guennadi Liakhovetski | f7fc97a | 2010-03-23 11:42:32 -0300 | [diff] [blame] | 138 | 5. Calculate and apply host scale = (6' - 6) / (4' - 4) |
Guennadi Liakhovetski | 35b23b5 | 2009-12-11 11:34:20 -0300 | [diff] [blame] | 139 | |
Guennadi Liakhovetski | f7fc97a | 2010-03-23 11:42:32 -0300 | [diff] [blame] | 140 | 6. Calculate and apply host crop: 6 - 7 = (5 - 2) * (6' - 6) / (5' - 5) |