Documentation/video4linux/omap3isp.txt - kernel/msm-4.9 - Gitiles

 OMAP 3 Image Signal Processor (ISP) driver

 Copyright (C) 2010 Nokia Corporation
 Copyright (C) 2009 Texas Instruments, Inc.

 Contacts: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
 	  Sakari Ailus <sakari.ailus@iki.fi>
 	  David Cohen <dacohen@gmail.com>


 Introduction
 ============

 This file documents the Texas Instruments OMAP 3 Image Signal Processor (ISP)
 driver located under drivers/media/platform/omap3isp. The original driver was
 written by Texas Instruments but since that it has been rewritten (twice) at
 Nokia.

 The driver has been successfully used on the following versions of OMAP 3:

 	3430
 	3530
 	3630

 The driver implements V4L2, Media controller and v4l2_subdev interfaces.
 Sensor, lens and flash drivers using the v4l2_subdev interface in the kernel
 are supported.


 Split to subdevs
 ================

 The OMAP 3 ISP is split into V4L2 subdevs, each of the blocks inside the ISP
 having one subdev to represent it. Each of the subdevs provide a V4L2 subdev
 interface to userspace.

 	OMAP3 ISP CCP2
 	OMAP3 ISP CSI2a
 	OMAP3 ISP CCDC
 	OMAP3 ISP preview
 	OMAP3 ISP resizer
 	OMAP3 ISP AEWB
 	OMAP3 ISP AF
 	OMAP3 ISP histogram

 Each possible link in the ISP is modelled by a link in the Media controller
 interface. For an example program see [2].


 Controlling the OMAP 3 ISP
 ==========================

 In general, the settings given to the OMAP 3 ISP take effect at the beginning
 of the following frame. This is done when the module becomes idle during the
 vertical blanking period on the sensor. In memory-to-memory operation the pipe
 is run one frame at a time. Applying the settings is done between the frames.

 All the blocks in the ISP, excluding the CSI-2 and possibly the CCP2 receiver,
 insist on receiving complete frames. Sensors must thus never send the ISP
 partial frames.

 Autoidle does have issues with some ISP blocks on the 3430, at least.
 Autoidle is only enabled on 3630 when the omap3isp module parameter autoidle
 is non-zero.


 Events
 ======

 The OMAP 3 ISP driver does support the V4L2 event interface on CCDC and
 statistics (AEWB, AF and histogram) subdevs.

 The CCDC subdev produces V4L2_EVENT_FRAME_SYNC type event on HS_VS
 interrupt which is used to signal frame start. Earlier version of this
 driver used V4L2_EVENT_OMAP3ISP_HS_VS for this purpose. The event is
 triggered exactly when the reception of the first line of the frame starts
 in the CCDC module. The event can be subscribed on the CCDC subdev.

 (When using parallel interface one must pay account to correct configuration
 of the VS signal polarity. This is automatically correct when using the serial
 receivers.)

 Each of the statistics subdevs is able to produce events. An event is
 generated whenever a statistics buffer can be dequeued by a user space
 application using the VIDIOC_OMAP3ISP_STAT_REQ IOCTL. The events available
 are:

 	V4L2_EVENT_OMAP3ISP_AEWB
 	V4L2_EVENT_OMAP3ISP_AF
 	V4L2_EVENT_OMAP3ISP_HIST

 The type of the event data is struct omap3isp_stat_event_status for these
 ioctls. If there is an error calculating the statistics, there will be an
 event as usual, but no related statistics buffer. In this case
 omap3isp_stat_event_status.buf_err is set to non-zero.


 Private IOCTLs
 ==============

 The OMAP 3 ISP driver supports standard V4L2 IOCTLs and controls where
 possible and practical. Much of the functions provided by the ISP, however,
 does not fall under the standard IOCTLs --- gamma tables and configuration of
 statistics collection are examples of such.

 In general, there is a private ioctl for configuring each of the blocks
 containing hardware-dependent functions.

 The following private IOCTLs are supported:

 	VIDIOC_OMAP3ISP_CCDC_CFG
 	VIDIOC_OMAP3ISP_PRV_CFG
 	VIDIOC_OMAP3ISP_AEWB_CFG
 	VIDIOC_OMAP3ISP_HIST_CFG
 	VIDIOC_OMAP3ISP_AF_CFG
 	VIDIOC_OMAP3ISP_STAT_REQ
 	VIDIOC_OMAP3ISP_STAT_EN

 The parameter structures used by these ioctls are described in
 include/linux/omap3isp.h. The detailed functions of the ISP itself related to
 a given ISP block is described in the Technical Reference Manuals (TRMs) ---
 see the end of the document for those.

 While it is possible to use the ISP driver without any use of these private
 IOCTLs it is not possible to obtain optimal image quality this way. The AEWB,
 AF and histogram modules cannot be used without configuring them using the
 appropriate private IOCTLs.


 CCDC and preview block IOCTLs
 =============================

 The VIDIOC_OMAP3ISP_CCDC_CFG and VIDIOC_OMAP3ISP_PRV_CFG IOCTLs are used to
 configure, enable and disable functions in the CCDC and preview blocks,
 respectively. Both IOCTLs control several functions in the blocks they
 control. VIDIOC_OMAP3ISP_CCDC_CFG IOCTL accepts a pointer to struct
 omap3isp_ccdc_update_config as its argument. Similarly VIDIOC_OMAP3ISP_PRV_CFG
 accepts a pointer to struct omap3isp_prev_update_config. The definition of
 both structures is available in [1].

 The update field in the structures tells whether to update the configuration
 for the specific function and the flag tells whether to enable or disable the
 function.

 The update and flag bit masks accept the following values. Each separate
 functions in the CCDC and preview blocks is associated with a flag (either
 disable or enable; part of the flag field in the structure) and a pointer to
 configuration data for the function.

 Valid values for the update and flag fields are listed here for
 VIDIOC_OMAP3ISP_CCDC_CFG. Values may be or'ed to configure more than one
 function in the same IOCTL call.

         OMAP3ISP_CCDC_ALAW
         OMAP3ISP_CCDC_LPF
         OMAP3ISP_CCDC_BLCLAMP
         OMAP3ISP_CCDC_BCOMP
         OMAP3ISP_CCDC_FPC
         OMAP3ISP_CCDC_CULL
         OMAP3ISP_CCDC_CONFIG_LSC
         OMAP3ISP_CCDC_TBL_LSC

 The corresponding values for the VIDIOC_OMAP3ISP_PRV_CFG are here:

         OMAP3ISP_PREV_LUMAENH
         OMAP3ISP_PREV_INVALAW
         OMAP3ISP_PREV_HRZ_MED
         OMAP3ISP_PREV_CFA
         OMAP3ISP_PREV_CHROMA_SUPP
         OMAP3ISP_PREV_WB
         OMAP3ISP_PREV_BLKADJ
         OMAP3ISP_PREV_RGB2RGB
         OMAP3ISP_PREV_COLOR_CONV
         OMAP3ISP_PREV_YC_LIMIT
         OMAP3ISP_PREV_DEFECT_COR
         OMAP3ISP_PREV_GAMMABYPASS
         OMAP3ISP_PREV_DRK_FRM_CAPTURE
         OMAP3ISP_PREV_DRK_FRM_SUBTRACT
         OMAP3ISP_PREV_LENS_SHADING
         OMAP3ISP_PREV_NF
         OMAP3ISP_PREV_GAMMA

 The associated configuration pointer for the function may not be NULL when
 enabling the function. When disabling a function the configuration pointer is
 ignored.


 Statistic blocks IOCTLs
 =======================

 The statistics subdevs do offer more dynamic configuration options than the
 other subdevs. They can be enabled, disable and reconfigured when the pipeline
 is in streaming state.

 The statistics blocks always get the input image data from the CCDC (as the
 histogram memory read isn't implemented). The statistics are dequeueable by
 the user from the statistics subdev nodes using private IOCTLs.

 The private IOCTLs offered by the AEWB, AF and histogram subdevs are heavily
 reflected by the register level interface offered by the ISP hardware. There
 are aspects that are purely related to the driver implementation and these are
 discussed next.

 VIDIOC_OMAP3ISP_STAT_EN
 -----------------------

 This private IOCTL enables/disables a statistic module. If this request is
 done before streaming, it will take effect as soon as the pipeline starts to
 stream.  If the pipeline is already streaming, it will take effect as soon as
 the CCDC becomes idle.

 VIDIOC_OMAP3ISP_AEWB_CFG, VIDIOC_OMAP3ISP_HIST_CFG and VIDIOC_OMAP3ISP_AF_CFG
 -----------------------------------------------------------------------------

 Those IOCTLs are used to configure the modules. They require user applications
 to have an in-depth knowledge of the hardware. Most of the fields explanation
 can be found on OMAP's TRMs. The two following fields common to all the above
 configure private IOCTLs require explanation for better understanding as they
 are not part of the TRM.

 omap3isp_[h3a_af/h3a_aewb/hist]_config.buf_size:

 The modules handle their buffers internally. The necessary buffer size for the
 module's data output depends on the requested configuration. Although the
 driver supports reconfiguration while streaming, it does not support a
 reconfiguration which requires bigger buffer size than what is already
 internally allocated if the module is enabled. It will return -EBUSY on this
 case. In order to avoid such condition, either disable/reconfigure/enable the
 module or request the necessary buffer size during the first configuration
 while the module is disabled.

 The internal buffer size allocation considers the requested configuration's
 minimum buffer size and the value set on buf_size field. If buf_size field is
 out of [minimum, maximum] buffer size range, it's clamped to fit in there.
 The driver then selects the biggest value. The corrected buf_size value is
 written back to user application.

 omap3isp_[h3a_af/h3a_aewb/hist]_config.config_counter:

 As the configuration doesn't take effect synchronously to the request, the
 driver must provide a way to track this information to provide more accurate
 data. After a configuration is requested, the config_counter returned to user
 space application will be an unique value associated to that request. When
 user application receives an event for buffer availability or when a new
 buffer is requested, this config_counter is used to match a buffer data and a
 configuration.

 VIDIOC_OMAP3ISP_STAT_REQ
 ------------------------

 Send to user space the oldest data available in the internal buffer queue and
 discards such buffer afterwards. The field omap3isp_stat_data.frame_number
 matches with the video buffer's field_count.


 Technical reference manuals (TRMs) and other documentation
 ==========================================================

 OMAP 3430 TRM:
 <URL:http://focus.ti.com/pdfs/wtbu/OMAP34xx_ES3.1.x_PUBLIC_TRM_vZM.zip>
 Referenced 2011-03-05.

 OMAP 35xx TRM:
 <URL:http://www.ti.com/litv/pdf/spruf98o> Referenced 2011-03-05.

 OMAP 3630 TRM:
 <URL:http://focus.ti.com/pdfs/wtbu/OMAP36xx_ES1.x_PUBLIC_TRM_vQ.zip>
 Referenced 2011-03-05.

 DM 3730 TRM:
 <URL:http://www.ti.com/litv/pdf/sprugn4h> Referenced 2011-03-06.


 References
 ==========

 [1] include/linux/omap3isp.h

 [2] http://git.ideasonboard.org/?p=media-ctl.git;a=summary
	OMAP 3 Image Signal Processor (ISP) driver

	Copyright (C) 2010 Nokia Corporation
	Copyright (C) 2009 Texas Instruments, Inc.

	Contacts: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
	Sakari Ailus <sakari.ailus@iki.fi>
	David Cohen <dacohen@gmail.com>


	Introduction
	============

	This file documents the Texas Instruments OMAP 3 Image Signal Processor (ISP)
	driver located under drivers/media/platform/omap3isp. The original driver was
	written by Texas Instruments but since that it has been rewritten (twice) at
	Nokia.

	The driver has been successfully used on the following versions of OMAP 3:

	3430
	3530
	3630

	The driver implements V4L2, Media controller and v4l2_subdev interfaces.
	Sensor, lens and flash drivers using the v4l2_subdev interface in the kernel
	are supported.


	Split to subdevs
	================

	The OMAP 3 ISP is split into V4L2 subdevs, each of the blocks inside the ISP
	having one subdev to represent it. Each of the subdevs provide a V4L2 subdev
	interface to userspace.

	OMAP3 ISP CCP2
	OMAP3 ISP CSI2a
	OMAP3 ISP CCDC
	OMAP3 ISP preview
	OMAP3 ISP resizer
	OMAP3 ISP AEWB
	OMAP3 ISP AF
	OMAP3 ISP histogram

	Each possible link in the ISP is modelled by a link in the Media controller
	interface. For an example program see [2].


	Controlling the OMAP 3 ISP
	==========================

	In general, the settings given to the OMAP 3 ISP take effect at the beginning
	of the following frame. This is done when the module becomes idle during the
	vertical blanking period on the sensor. In memory-to-memory operation the pipe
	is run one frame at a time. Applying the settings is done between the frames.

	All the blocks in the ISP, excluding the CSI-2 and possibly the CCP2 receiver,
	insist on receiving complete frames. Sensors must thus never send the ISP
	partial frames.

	Autoidle does have issues with some ISP blocks on the 3430, at least.
	Autoidle is only enabled on 3630 when the omap3isp module parameter autoidle
	is non-zero.


	Events
	======

	The OMAP 3 ISP driver does support the V4L2 event interface on CCDC and
	statistics (AEWB, AF and histogram) subdevs.

	The CCDC subdev produces V4L2_EVENT_FRAME_SYNC type event on HS_VS
	interrupt which is used to signal frame start. Earlier version of this
	driver used V4L2_EVENT_OMAP3ISP_HS_VS for this purpose. The event is
	triggered exactly when the reception of the first line of the frame starts
	in the CCDC module. The event can be subscribed on the CCDC subdev.

	(When using parallel interface one must pay account to correct configuration
	of the VS signal polarity. This is automatically correct when using the serial
	receivers.)

	Each of the statistics subdevs is able to produce events. An event is
	generated whenever a statistics buffer can be dequeued by a user space
	application using the VIDIOC_OMAP3ISP_STAT_REQ IOCTL. The events available
	are:

	V4L2_EVENT_OMAP3ISP_AEWB
	V4L2_EVENT_OMAP3ISP_AF
	V4L2_EVENT_OMAP3ISP_HIST

	The type of the event data is struct omap3isp_stat_event_status for these
	ioctls. If there is an error calculating the statistics, there will be an
	event as usual, but no related statistics buffer. In this case
	omap3isp_stat_event_status.buf_err is set to non-zero.


	Private IOCTLs
	==============

	The OMAP 3 ISP driver supports standard V4L2 IOCTLs and controls where
	possible and practical. Much of the functions provided by the ISP, however,
	does not fall under the standard IOCTLs --- gamma tables and configuration of
	statistics collection are examples of such.

	In general, there is a private ioctl for configuring each of the blocks
	containing hardware-dependent functions.

	The following private IOCTLs are supported:

	VIDIOC_OMAP3ISP_CCDC_CFG
	VIDIOC_OMAP3ISP_PRV_CFG
	VIDIOC_OMAP3ISP_AEWB_CFG
	VIDIOC_OMAP3ISP_HIST_CFG
	VIDIOC_OMAP3ISP_AF_CFG
	VIDIOC_OMAP3ISP_STAT_REQ
	VIDIOC_OMAP3ISP_STAT_EN

	The parameter structures used by these ioctls are described in
	include/linux/omap3isp.h. The detailed functions of the ISP itself related to
	a given ISP block is described in the Technical Reference Manuals (TRMs) ---
	see the end of the document for those.

	While it is possible to use the ISP driver without any use of these private
	IOCTLs it is not possible to obtain optimal image quality this way. The AEWB,
	AF and histogram modules cannot be used without configuring them using the
	appropriate private IOCTLs.


	CCDC and preview block IOCTLs
	=============================

	The VIDIOC_OMAP3ISP_CCDC_CFG and VIDIOC_OMAP3ISP_PRV_CFG IOCTLs are used to
	configure, enable and disable functions in the CCDC and preview blocks,
	respectively. Both IOCTLs control several functions in the blocks they
	control. VIDIOC_OMAP3ISP_CCDC_CFG IOCTL accepts a pointer to struct
	omap3isp_ccdc_update_config as its argument. Similarly VIDIOC_OMAP3ISP_PRV_CFG
	accepts a pointer to struct omap3isp_prev_update_config. The definition of
	both structures is available in [1].

	The update field in the structures tells whether to update the configuration
	for the specific function and the flag tells whether to enable or disable the
	function.

	The update and flag bit masks accept the following values. Each separate
	functions in the CCDC and preview blocks is associated with a flag (either
	disable or enable; part of the flag field in the structure) and a pointer to
	configuration data for the function.

	Valid values for the update and flag fields are listed here for
	VIDIOC_OMAP3ISP_CCDC_CFG. Values may be or'ed to configure more than one
	function in the same IOCTL call.

	OMAP3ISP_CCDC_ALAW
	OMAP3ISP_CCDC_LPF
	OMAP3ISP_CCDC_BLCLAMP
	OMAP3ISP_CCDC_BCOMP
	OMAP3ISP_CCDC_FPC
	OMAP3ISP_CCDC_CULL
	OMAP3ISP_CCDC_CONFIG_LSC
	OMAP3ISP_CCDC_TBL_LSC

	The corresponding values for the VIDIOC_OMAP3ISP_PRV_CFG are here:

	OMAP3ISP_PREV_LUMAENH
	OMAP3ISP_PREV_INVALAW
	OMAP3ISP_PREV_HRZ_MED
	OMAP3ISP_PREV_CFA
	OMAP3ISP_PREV_CHROMA_SUPP
	OMAP3ISP_PREV_WB
	OMAP3ISP_PREV_BLKADJ
	OMAP3ISP_PREV_RGB2RGB
	OMAP3ISP_PREV_COLOR_CONV
	OMAP3ISP_PREV_YC_LIMIT
	OMAP3ISP_PREV_DEFECT_COR
	OMAP3ISP_PREV_GAMMABYPASS
	OMAP3ISP_PREV_DRK_FRM_CAPTURE
	OMAP3ISP_PREV_DRK_FRM_SUBTRACT
	OMAP3ISP_PREV_LENS_SHADING
	OMAP3ISP_PREV_NF
	OMAP3ISP_PREV_GAMMA

	The associated configuration pointer for the function may not be NULL when
	enabling the function. When disabling a function the configuration pointer is
	ignored.


	Statistic blocks IOCTLs
	=======================

	The statistics subdevs do offer more dynamic configuration options than the
	other subdevs. They can be enabled, disable and reconfigured when the pipeline
	is in streaming state.

	The statistics blocks always get the input image data from the CCDC (as the
	histogram memory read isn't implemented). The statistics are dequeueable by
	the user from the statistics subdev nodes using private IOCTLs.

	The private IOCTLs offered by the AEWB, AF and histogram subdevs are heavily
	reflected by the register level interface offered by the ISP hardware. There
	are aspects that are purely related to the driver implementation and these are
	discussed next.

	VIDIOC_OMAP3ISP_STAT_EN
	-----------------------

	This private IOCTL enables/disables a statistic module. If this request is
	done before streaming, it will take effect as soon as the pipeline starts to
	stream. If the pipeline is already streaming, it will take effect as soon as
	the CCDC becomes idle.

	VIDIOC_OMAP3ISP_AEWB_CFG, VIDIOC_OMAP3ISP_HIST_CFG and VIDIOC_OMAP3ISP_AF_CFG
	-----------------------------------------------------------------------------

	Those IOCTLs are used to configure the modules. They require user applications
	to have an in-depth knowledge of the hardware. Most of the fields explanation
	can be found on OMAP's TRMs. The two following fields common to all the above
	configure private IOCTLs require explanation for better understanding as they
	are not part of the TRM.

	omap3isp_[h3a_af/h3a_aewb/hist]_config.buf_size:

	The modules handle their buffers internally. The necessary buffer size for the
	module's data output depends on the requested configuration. Although the
	driver supports reconfiguration while streaming, it does not support a
	reconfiguration which requires bigger buffer size than what is already
	internally allocated if the module is enabled. It will return -EBUSY on this
	case. In order to avoid such condition, either disable/reconfigure/enable the
	module or request the necessary buffer size during the first configuration
	while the module is disabled.

	The internal buffer size allocation considers the requested configuration's
	minimum buffer size and the value set on buf_size field. If buf_size field is
	out of [minimum, maximum] buffer size range, it's clamped to fit in there.
	The driver then selects the biggest value. The corrected buf_size value is
	written back to user application.

	omap3isp_[h3a_af/h3a_aewb/hist]_config.config_counter:

	As the configuration doesn't take effect synchronously to the request, the
	driver must provide a way to track this information to provide more accurate
	data. After a configuration is requested, the config_counter returned to user
	space application will be an unique value associated to that request. When
	user application receives an event for buffer availability or when a new
	buffer is requested, this config_counter is used to match a buffer data and a
	configuration.

	VIDIOC_OMAP3ISP_STAT_REQ
	------------------------

	Send to user space the oldest data available in the internal buffer queue and
	discards such buffer afterwards. The field omap3isp_stat_data.frame_number
	matches with the video buffer's field_count.


	Technical reference manuals (TRMs) and other documentation
	==========================================================

	OMAP 3430 TRM:
	<URL:http://focus.ti.com/pdfs/wtbu/OMAP34xx_ES3.1.x_PUBLIC_TRM_vZM.zip>
	Referenced 2011-03-05.

	OMAP 35xx TRM:
	<URL:http://www.ti.com/litv/pdf/spruf98o> Referenced 2011-03-05.

	OMAP 3630 TRM:
	<URL:http://focus.ti.com/pdfs/wtbu/OMAP36xx_ES1.x_PUBLIC_TRM_vQ.zip>
	Referenced 2011-03-05.

	DM 3730 TRM:
	<URL:http://www.ti.com/litv/pdf/sprugn4h> Referenced 2011-03-06.


	References
	==========

	[1] include/linux/omap3isp.h

	[2] http://git.ideasonboard.org/?p=media-ctl.git;a=summary