Blame - docs/dyn/dataproc_v1.projects.regions.jobs.html - platform/external/python/google-api-python-client

blob: 7bacd91db4141df578930eaec64d281bfa0fc292 [file] [log] [blame]

Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1	<html><body>
				2	<style>
				3
				4	body, h1, h2, h3, div, span, p, pre, a {
				5	margin: 0;
				6	padding: 0;
				7	border: 0;
				8	font-weight: inherit;
				9	font-style: inherit;
				10	font-size: 100%;
				11	font-family: inherit;
				12	vertical-align: baseline;
				13	}
				14
				15	body {
				16	font-size: 13px;
				17	padding: 1em;
				18	}
				19
				20	h1 {
				21	font-size: 26px;
				22	margin-bottom: 1em;
				23	}
				24
				25	h2 {
				26	font-size: 24px;
				27	margin-bottom: 1em;
				28	}
				29
				30	h3 {
				31	font-size: 20px;
				32	margin-bottom: 1em;
				33	margin-top: 1em;
				34	}
				35
				36	pre, code {
				37	line-height: 1.5;
				38	font-family: Monaco, 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Lucida Console', monospace;
				39	}
				40
				41	pre {
				42	margin-top: 0.5em;
				43	}
				44
				45	h1, h2, h3, p {
				46	font-family: Arial, sans serif;
				47	}
				48
				49	h1, h2, h3 {
				50	border-bottom: solid #CCC 1px;
				51	}
				52
				53	.toc_element {
				54	margin-top: 0.5em;
				55	}
				56
				57	.firstline {
				58	margin-left: 2 em;
				59	}
				60
				61	.method {
				62	margin-top: 1em;
				63	border: solid 1px #CCC;
				64	padding: 1em;
				65	background: #EEE;
				66	}
				67
				68	.details {
				69	font-weight: bold;
				70	font-size: 14px;
				71	}
				72
				73	</style>
				74
				75	<h1><a href="dataproc_v1.html">Google Cloud Dataproc API</a> . <a href="dataproc_v1.projects.html">projects</a> . <a href="dataproc_v1.projects.regions.html">regions</a> . <a href="dataproc_v1.projects.regions.jobs.html">jobs</a></h1>
				76	<h2>Instance Methods</h2>
				77	<p class="toc_element">
				78	<code><a href="#cancel">cancel(projectId, region, jobId, body, x__xgafv=None)</a></code></p>
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	79	<p class="firstline">Starts a job cancellation request. To access the job resource after cancellation, call regions/{region}/jobs.list or regions/{region}/jobs.get.</p>
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	80	<p class="toc_element">
				81	<code><a href="#delete">delete(projectId, region, jobId, x__xgafv=None)</a></code></p>
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	82	<p class="firstline">Deletes the job from the project. If the job is active, the delete fails, and the response returns FAILED_PRECONDITION.</p>
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	83	<p class="toc_element">
				84	<code><a href="#get">get(projectId, region, jobId, x__xgafv=None)</a></code></p>
				85	<p class="firstline">Gets the resource representation for a job in a project.</p>
				86	<p class="toc_element">
Jon Wayne Parrott	692617a	2017-01-06 09:58:29 -0800	[diff] [blame]	87	<code><a href="#list">list(projectId, region, pageSize=None, x__xgafv=None, pageToken=None, clusterName=None, jobStateMatcher=None, filter=None)</a></code></p>
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	88	<p class="firstline">Lists regions/{region}/jobs in a project.</p>
				89	<p class="toc_element">
				90	<code><a href="#list_next">list_next(previous_request, previous_response)</a></code></p>
				91	<p class="firstline">Retrieves the next page of results.</p>
				92	<p class="toc_element">
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	93	<code><a href="#patch">patch(projectId, region, jobId, body, updateMask=None, x__xgafv=None)</a></code></p>
				94	<p class="firstline">Updates a job in a project.</p>
				95	<p class="toc_element">
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	96	<code><a href="#submit">submit(projectId, region, body, x__xgafv=None)</a></code></p>
				97	<p class="firstline">Submits a job to a cluster.</p>
				98	<h3>Method Details</h3>
				99	<div class="method">
				100	<code class="details" id="cancel">cancel(projectId, region, jobId, body, x__xgafv=None)</code>
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	101	<pre>Starts a job cancellation request. To access the job resource after cancellation, call regions/{region}/jobs.list or regions/{region}/jobs.get.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	102
				103	Args:
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	104	projectId: string, Required. The ID of the Google Cloud Platform project that the job belongs to. (required)
				105	region: string, Required. The Cloud Dataproc region in which to handle the request. (required)
				106	jobId: string, Required. The job ID. (required)
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	107	body: object, The request body. (required)
				108	The object takes the form of:
				109
				110	{ # A request to cancel a job.
				111	}
				112
				113	x__xgafv: string, V1 error format.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	114	Allowed values
				115	1 - v1 error format
				116	2 - v2 error format
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	117
				118	Returns:
				119	An object of the form:
				120
				121	{ # A Cloud Dataproc job resource.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	122	"status": { # Cloud Dataproc job status. # Output-only. The job status. Additional application-specific status information may be contained in the <code>type_job</code> and <code>yarn_applications</code> fields.
				123	"state": "A String", # Output-only. A state message specifying the overall job state.
				124	"stateStartTime": "A String", # Output-only. The time when this state was entered.
				125	"details": "A String", # Output-only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				126	"substate": "A String", # Output-only. Additional state information, which includes status reported by the agent.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	127	},
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	128	"sparkSqlJob": { # A Cloud Dataproc job for running Apache Spark SQL (http://spark.apache.org/sql/) queries. # Job is a SparkSql job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	129	"queryFileUri": "A String", # The HCFS URI of the script that contains SQL queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	130	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	131	"a_key": "A String",
				132	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	133	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	134	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	135	"a_key": "A String",
				136	},
				137	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	138	"jarFileUris": [ # Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	139	"A String",
				140	],
				141	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	142	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	143	# "hiveJob": {
				144	# "queryList": {
				145	# "queries": [
				146	# "query1",
				147	# "query2",
				148	# "query3;query4",
				149	# ]
				150	# }
				151	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	152	"A String",
				153	],
				154	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	155	"properties": { # Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Cloud Dataproc API may be overwritten.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	156	"a_key": "A String",
				157	},
				158	},
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	159	"pigJob": { # A Cloud Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN. # Job is a Pig job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	160	"queryFileUri": "A String", # The HCFS URI of the script that contains the Pig queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	161	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	162	"a_key": "A String",
				163	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	164	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	165	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	166	"a_key": "A String",
				167	},
				168	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	169	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	170	"A String",
				171	],
				172	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	173	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	174	# "hiveJob": {
				175	# "queryList": {
				176	# "queries": [
				177	# "query1",
				178	# "query2",
				179	# "query3;query4",
				180	# ]
				181	# }
				182	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	183	"A String",
				184	],
				185	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	186	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				187	"properties": { # Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	188	"a_key": "A String",
				189	},
				190	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	191	"placement": { # Cloud Dataproc job config. # Required. Job information, including how, when, and where to run the job.
				192	"clusterName": "A String", # Required. The name of the cluster where the job will be submitted.
				193	"clusterUuid": "A String", # Output-only. A cluster UUID generated by the Cloud Dataproc service when the job is submitted.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	194	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	195	"reference": { # Encapsulates the full scoping used to reference a job. # Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a <code>job_id</code>.
				196	"projectId": "A String", # Required. The ID of the Google Cloud Platform project that the job belongs to.
				197	"jobId": "A String", # Optional. The job ID, which must be unique within the project. The job ID is generated by the server upon job submission or provided by the user as a means to perform retries without creating duplicate jobs. The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	198	},
				199	"hadoopJob": { # A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). # Job is a Hadoop job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	200	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	201	"A String",
				202	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	203	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	204	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	205	"a_key": "A String",
				206	},
				207	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	208	"jarFileUris": [ # Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	209	"A String",
				210	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	211	"fileUris": [ # Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	212	"A String",
				213	],
				214	"mainClass": "A String", # The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	215	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	216	"A String",
				217	],
				218	"mainJarFileUri": "A String", # The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	219	"properties": { # Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	220	"a_key": "A String",
				221	},
				222	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	223	"labels": { # Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values may be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	224	"a_key": "A String",
				225	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	226	"driverOutputResourceUri": "A String", # Output-only. A URI pointing to the location of the stdout of the job's driver program.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	227	"pysparkJob": { # A Cloud Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/python-programming-guide.html) applications on YARN. # Job is a Pyspark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	228	"mainPythonFileUri": "A String", # Required. The HCFS URI of the main Python file to use as the driver. Must be a .py file.
				229	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	230	"A String",
				231	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	232	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	233	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				234	"a_key": "A String",
				235	},
				236	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	237	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	238	"A String",
				239	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	240	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	241	"A String",
				242	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	243	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	244	"A String",
				245	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	246	"pythonFileUris": [ # Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	247	"A String",
				248	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	249	"properties": { # Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	250	"a_key": "A String",
				251	},
				252	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	253	"driverControlFilesUri": "A String", # Output-only. If present, the location of miscellaneous control files which may be used as part of job setup and handling. If not present, control files may be placed in the same location as driver_output_uri.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	254	"sparkJob": { # A Cloud Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN. # Job is a Spark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	255	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	256	"A String",
				257	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	258	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	259	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				260	"a_key": "A String",
				261	},
				262	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	263	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	264	"A String",
				265	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	266	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Spark drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	267	"A String",
				268	],
				269	"mainClass": "A String", # The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	270	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Spark drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	271	"A String",
				272	],
Jon Wayne Parrott	0a471d3	2016-05-19 10:54:38 -0700	[diff] [blame]	273	"mainJarFileUri": "A String", # The HCFS URI of the jar file that contains the main class.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	274	"properties": { # Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	275	"a_key": "A String",
				276	},
				277	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	278	"yarnApplications": [ # Output-only. The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	279	{ # A YARN application created by a job. Application information is a subset of <code>org.apache.hadoop.yarn.proto.YarnProtos.ApplicationReportProto</code>.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	280	"progress": 3.14, # Required. The numerical progress of the application, from 1 to 100.
				281	"state": "A String", # Required. The application state.
				282	"name": "A String", # Required. The application name.
				283	"trackingUrl": "A String", # Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
Jon Wayne Parrott	692617a	2017-01-06 09:58:29 -0800	[diff] [blame]	284	},
				285	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	286	"scheduling": { # Job scheduling options.Beta Feature: These options are available for testing purposes only. They may be changed before final release. # Optional. Job scheduling configuration.
				287	"maxFailuresPerHour": 42, # Optional. Maximum number of times per hour a driver may be restarted as a result of driver terminating with non-zero code before job is reported failed.A job may be reported as thrashing if driver exits with non-zero code 4 times within 10 minute window.Maximum value is 10.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	288	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	289	"statusHistory": [ # Output-only. The previous job status.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	290	{ # Cloud Dataproc job status.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	291	"state": "A String", # Output-only. A state message specifying the overall job state.
				292	"stateStartTime": "A String", # Output-only. The time when this state was entered.
				293	"details": "A String", # Output-only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				294	"substate": "A String", # Output-only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	295	},
				296	],
				297	"hiveJob": { # A Cloud Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN. # Job is a Hive job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	298	"queryFileUri": "A String", # The HCFS URI of the script that contains Hive queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	299	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	300	"a_key": "A String",
				301	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	302	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	303	"A String",
				304	],
				305	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	306	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	307	# "hiveJob": {
				308	# "queryList": {
				309	# "queries": [
				310	# "query1",
				311	# "query2",
				312	# "query3;query4",
				313	# ]
				314	# }
				315	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	316	"A String",
				317	],
				318	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	319	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				320	"properties": { # Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	321	"a_key": "A String",
				322	},
				323	},
				324	}</pre>
				325	</div>
				326
				327	<div class="method">
				328	<code class="details" id="delete">delete(projectId, region, jobId, x__xgafv=None)</code>
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	329	<pre>Deletes the job from the project. If the job is active, the delete fails, and the response returns FAILED_PRECONDITION.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	330
				331	Args:
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	332	projectId: string, Required. The ID of the Google Cloud Platform project that the job belongs to. (required)
				333	region: string, Required. The Cloud Dataproc region in which to handle the request. (required)
				334	jobId: string, Required. The job ID. (required)
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	335	x__xgafv: string, V1 error format.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	336	Allowed values
				337	1 - v1 error format
				338	2 - v2 error format
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	339
				340	Returns:
				341	An object of the form:
				342
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	343	{ # A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance:
				344	# service Foo {
				345	# rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty);
				346	# }
				347	# The JSON representation for Empty is empty JSON object {}.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	348	}</pre>
				349	</div>
				350
				351	<div class="method">
				352	<code class="details" id="get">get(projectId, region, jobId, x__xgafv=None)</code>
				353	<pre>Gets the resource representation for a job in a project.
				354
				355	Args:
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	356	projectId: string, Required. The ID of the Google Cloud Platform project that the job belongs to. (required)
				357	region: string, Required. The Cloud Dataproc region in which to handle the request. (required)
				358	jobId: string, Required. The job ID. (required)
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	359	x__xgafv: string, V1 error format.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	360	Allowed values
				361	1 - v1 error format
				362	2 - v2 error format
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	363
				364	Returns:
				365	An object of the form:
				366
				367	{ # A Cloud Dataproc job resource.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	368	"status": { # Cloud Dataproc job status. # Output-only. The job status. Additional application-specific status information may be contained in the <code>type_job</code> and <code>yarn_applications</code> fields.
				369	"state": "A String", # Output-only. A state message specifying the overall job state.
				370	"stateStartTime": "A String", # Output-only. The time when this state was entered.
				371	"details": "A String", # Output-only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				372	"substate": "A String", # Output-only. Additional state information, which includes status reported by the agent.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	373	},
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	374	"sparkSqlJob": { # A Cloud Dataproc job for running Apache Spark SQL (http://spark.apache.org/sql/) queries. # Job is a SparkSql job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	375	"queryFileUri": "A String", # The HCFS URI of the script that contains SQL queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	376	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	377	"a_key": "A String",
				378	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	379	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	380	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	381	"a_key": "A String",
				382	},
				383	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	384	"jarFileUris": [ # Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	385	"A String",
				386	],
				387	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	388	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	389	# "hiveJob": {
				390	# "queryList": {
				391	# "queries": [
				392	# "query1",
				393	# "query2",
				394	# "query3;query4",
				395	# ]
				396	# }
				397	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	398	"A String",
				399	],
				400	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	401	"properties": { # Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Cloud Dataproc API may be overwritten.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	402	"a_key": "A String",
				403	},
				404	},
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	405	"pigJob": { # A Cloud Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN. # Job is a Pig job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	406	"queryFileUri": "A String", # The HCFS URI of the script that contains the Pig queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	407	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	408	"a_key": "A String",
				409	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	410	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	411	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	412	"a_key": "A String",
				413	},
				414	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	415	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	416	"A String",
				417	],
				418	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	419	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	420	# "hiveJob": {
				421	# "queryList": {
				422	# "queries": [
				423	# "query1",
				424	# "query2",
				425	# "query3;query4",
				426	# ]
				427	# }
				428	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	429	"A String",
				430	],
				431	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	432	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				433	"properties": { # Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	434	"a_key": "A String",
				435	},
				436	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	437	"placement": { # Cloud Dataproc job config. # Required. Job information, including how, when, and where to run the job.
				438	"clusterName": "A String", # Required. The name of the cluster where the job will be submitted.
				439	"clusterUuid": "A String", # Output-only. A cluster UUID generated by the Cloud Dataproc service when the job is submitted.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	440	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	441	"reference": { # Encapsulates the full scoping used to reference a job. # Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a <code>job_id</code>.
				442	"projectId": "A String", # Required. The ID of the Google Cloud Platform project that the job belongs to.
				443	"jobId": "A String", # Optional. The job ID, which must be unique within the project. The job ID is generated by the server upon job submission or provided by the user as a means to perform retries without creating duplicate jobs. The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	444	},
				445	"hadoopJob": { # A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). # Job is a Hadoop job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	446	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	447	"A String",
				448	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	449	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	450	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	451	"a_key": "A String",
				452	},
				453	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	454	"jarFileUris": [ # Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	455	"A String",
				456	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	457	"fileUris": [ # Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	458	"A String",
				459	],
				460	"mainClass": "A String", # The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	461	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	462	"A String",
				463	],
				464	"mainJarFileUri": "A String", # The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	465	"properties": { # Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	466	"a_key": "A String",
				467	},
				468	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	469	"labels": { # Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values may be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	470	"a_key": "A String",
				471	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	472	"driverOutputResourceUri": "A String", # Output-only. A URI pointing to the location of the stdout of the job's driver program.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	473	"pysparkJob": { # A Cloud Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/python-programming-guide.html) applications on YARN. # Job is a Pyspark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	474	"mainPythonFileUri": "A String", # Required. The HCFS URI of the main Python file to use as the driver. Must be a .py file.
				475	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	476	"A String",
				477	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	478	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	479	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				480	"a_key": "A String",
				481	},
				482	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	483	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	484	"A String",
				485	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	486	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	487	"A String",
				488	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	489	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	490	"A String",
				491	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	492	"pythonFileUris": [ # Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	493	"A String",
				494	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	495	"properties": { # Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	496	"a_key": "A String",
				497	},
				498	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	499	"driverControlFilesUri": "A String", # Output-only. If present, the location of miscellaneous control files which may be used as part of job setup and handling. If not present, control files may be placed in the same location as driver_output_uri.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	500	"sparkJob": { # A Cloud Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN. # Job is a Spark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	501	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	502	"A String",
				503	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	504	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	505	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				506	"a_key": "A String",
				507	},
				508	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	509	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	510	"A String",
				511	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	512	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Spark drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	513	"A String",
				514	],
				515	"mainClass": "A String", # The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	516	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Spark drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	517	"A String",
				518	],
Jon Wayne Parrott	0a471d3	2016-05-19 10:54:38 -0700	[diff] [blame]	519	"mainJarFileUri": "A String", # The HCFS URI of the jar file that contains the main class.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	520	"properties": { # Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	521	"a_key": "A String",
				522	},
				523	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	524	"yarnApplications": [ # Output-only. The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	525	{ # A YARN application created by a job. Application information is a subset of <code>org.apache.hadoop.yarn.proto.YarnProtos.ApplicationReportProto</code>.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	526	"progress": 3.14, # Required. The numerical progress of the application, from 1 to 100.
				527	"state": "A String", # Required. The application state.
				528	"name": "A String", # Required. The application name.
				529	"trackingUrl": "A String", # Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
Jon Wayne Parrott	692617a	2017-01-06 09:58:29 -0800	[diff] [blame]	530	},
				531	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	532	"scheduling": { # Job scheduling options.Beta Feature: These options are available for testing purposes only. They may be changed before final release. # Optional. Job scheduling configuration.
				533	"maxFailuresPerHour": 42, # Optional. Maximum number of times per hour a driver may be restarted as a result of driver terminating with non-zero code before job is reported failed.A job may be reported as thrashing if driver exits with non-zero code 4 times within 10 minute window.Maximum value is 10.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	534	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	535	"statusHistory": [ # Output-only. The previous job status.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	536	{ # Cloud Dataproc job status.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	537	"state": "A String", # Output-only. A state message specifying the overall job state.
				538	"stateStartTime": "A String", # Output-only. The time when this state was entered.
				539	"details": "A String", # Output-only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				540	"substate": "A String", # Output-only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	541	},
				542	],
				543	"hiveJob": { # A Cloud Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN. # Job is a Hive job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	544	"queryFileUri": "A String", # The HCFS URI of the script that contains Hive queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	545	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	546	"a_key": "A String",
				547	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	548	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	549	"A String",
				550	],
				551	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	552	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	553	# "hiveJob": {
				554	# "queryList": {
				555	# "queries": [
				556	# "query1",
				557	# "query2",
				558	# "query3;query4",
				559	# ]
				560	# }
				561	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	562	"A String",
				563	],
				564	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	565	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				566	"properties": { # Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	567	"a_key": "A String",
				568	},
				569	},
				570	}</pre>
				571	</div>
				572
				573	<div class="method">
Jon Wayne Parrott	692617a	2017-01-06 09:58:29 -0800	[diff] [blame]	574	<code class="details" id="list">list(projectId, region, pageSize=None, x__xgafv=None, pageToken=None, clusterName=None, jobStateMatcher=None, filter=None)</code>
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	575	<pre>Lists regions/{region}/jobs in a project.
				576
				577	Args:
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	578	projectId: string, Required. The ID of the Google Cloud Platform project that the job belongs to. (required)
				579	region: string, Required. The Cloud Dataproc region in which to handle the request. (required)
				580	pageSize: integer, Optional. The number of results to return in each response.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	581	x__xgafv: string, V1 error format.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	582	Allowed values
				583	1 - v1 error format
				584	2 - v2 error format
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	585	pageToken: string, Optional. The page token, returned by a previous call, to request the next page of results.
				586	clusterName: string, Optional. If set, the returned jobs list includes only jobs that were submitted to the named cluster.
				587	jobStateMatcher: string, Optional. Specifies enumerated categories of jobs to list (default = match ALL jobs).
				588	filter: string, Optional. A filter constraining the jobs to list. Filters are case-sensitive and have the following syntax:field = value AND field = value ...where field is status.state or labels.[KEY], and [KEY] is a label key. value can be * to match all values. status.state can be either ACTIVE or INACTIVE. Only the logical AND operator is supported; space-separated items are treated as having an implicit AND operator.Example filter:status.state = ACTIVE AND labels.env = staging AND labels.starred = *
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	589
				590	Returns:
				591	An object of the form:
				592
				593	{ # A list of jobs in a project.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	594	"nextPageToken": "A String", # Optional. This token is included in the response if there are more results to fetch. To fetch additional results, provide this value as the page_token in a subsequent <code>ListJobsRequest</code>.
				595	"jobs": [ # Output-only. Jobs list.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	596	{ # A Cloud Dataproc job resource.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	597	"status": { # Cloud Dataproc job status. # Output-only. The job status. Additional application-specific status information may be contained in the <code>type_job</code> and <code>yarn_applications</code> fields.
				598	"state": "A String", # Output-only. A state message specifying the overall job state.
				599	"stateStartTime": "A String", # Output-only. The time when this state was entered.
				600	"details": "A String", # Output-only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				601	"substate": "A String", # Output-only. Additional state information, which includes status reported by the agent.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	602	},
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	603	"sparkSqlJob": { # A Cloud Dataproc job for running Apache Spark SQL (http://spark.apache.org/sql/) queries. # Job is a SparkSql job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	604	"queryFileUri": "A String", # The HCFS URI of the script that contains SQL queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	605	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	606	"a_key": "A String",
				607	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	608	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	609	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	610	"a_key": "A String",
				611	},
				612	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	613	"jarFileUris": [ # Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	614	"A String",
				615	],
				616	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	617	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	618	# "hiveJob": {
				619	# "queryList": {
				620	# "queries": [
				621	# "query1",
				622	# "query2",
				623	# "query3;query4",
				624	# ]
				625	# }
				626	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	627	"A String",
				628	],
				629	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	630	"properties": { # Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Cloud Dataproc API may be overwritten.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	631	"a_key": "A String",
				632	},
				633	},
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	634	"pigJob": { # A Cloud Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN. # Job is a Pig job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	635	"queryFileUri": "A String", # The HCFS URI of the script that contains the Pig queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	636	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	637	"a_key": "A String",
				638	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	639	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	640	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	641	"a_key": "A String",
				642	},
				643	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	644	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	645	"A String",
				646	],
				647	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	648	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	649	# "hiveJob": {
				650	# "queryList": {
				651	# "queries": [
				652	# "query1",
				653	# "query2",
				654	# "query3;query4",
				655	# ]
				656	# }
				657	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	658	"A String",
				659	],
				660	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	661	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				662	"properties": { # Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	663	"a_key": "A String",
				664	},
				665	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	666	"placement": { # Cloud Dataproc job config. # Required. Job information, including how, when, and where to run the job.
				667	"clusterName": "A String", # Required. The name of the cluster where the job will be submitted.
				668	"clusterUuid": "A String", # Output-only. A cluster UUID generated by the Cloud Dataproc service when the job is submitted.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	669	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	670	"reference": { # Encapsulates the full scoping used to reference a job. # Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a <code>job_id</code>.
				671	"projectId": "A String", # Required. The ID of the Google Cloud Platform project that the job belongs to.
				672	"jobId": "A String", # Optional. The job ID, which must be unique within the project. The job ID is generated by the server upon job submission or provided by the user as a means to perform retries without creating duplicate jobs. The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	673	},
				674	"hadoopJob": { # A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). # Job is a Hadoop job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	675	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	676	"A String",
				677	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	678	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	679	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	680	"a_key": "A String",
				681	},
				682	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	683	"jarFileUris": [ # Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	684	"A String",
				685	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	686	"fileUris": [ # Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	687	"A String",
				688	],
				689	"mainClass": "A String", # The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	690	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	691	"A String",
				692	],
				693	"mainJarFileUri": "A String", # The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	694	"properties": { # Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	695	"a_key": "A String",
				696	},
				697	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	698	"labels": { # Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values may be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	699	"a_key": "A String",
				700	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	701	"driverOutputResourceUri": "A String", # Output-only. A URI pointing to the location of the stdout of the job's driver program.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	702	"pysparkJob": { # A Cloud Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/python-programming-guide.html) applications on YARN. # Job is a Pyspark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	703	"mainPythonFileUri": "A String", # Required. The HCFS URI of the main Python file to use as the driver. Must be a .py file.
				704	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	705	"A String",
				706	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	707	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	708	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				709	"a_key": "A String",
				710	},
				711	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	712	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	713	"A String",
				714	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	715	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	716	"A String",
				717	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	718	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	719	"A String",
				720	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	721	"pythonFileUris": [ # Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	722	"A String",
				723	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	724	"properties": { # Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	725	"a_key": "A String",
				726	},
				727	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	728	"driverControlFilesUri": "A String", # Output-only. If present, the location of miscellaneous control files which may be used as part of job setup and handling. If not present, control files may be placed in the same location as driver_output_uri.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	729	"sparkJob": { # A Cloud Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN. # Job is a Spark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	730	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	731	"A String",
				732	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	733	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	734	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				735	"a_key": "A String",
				736	},
				737	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	738	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	739	"A String",
				740	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	741	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Spark drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	742	"A String",
				743	],
				744	"mainClass": "A String", # The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	745	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Spark drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	746	"A String",
				747	],
Jon Wayne Parrott	0a471d3	2016-05-19 10:54:38 -0700	[diff] [blame]	748	"mainJarFileUri": "A String", # The HCFS URI of the jar file that contains the main class.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	749	"properties": { # Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	750	"a_key": "A String",
				751	},
				752	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	753	"yarnApplications": [ # Output-only. The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	754	{ # A YARN application created by a job. Application information is a subset of <code>org.apache.hadoop.yarn.proto.YarnProtos.ApplicationReportProto</code>.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	755	"progress": 3.14, # Required. The numerical progress of the application, from 1 to 100.
				756	"state": "A String", # Required. The application state.
				757	"name": "A String", # Required. The application name.
				758	"trackingUrl": "A String", # Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
Jon Wayne Parrott	692617a	2017-01-06 09:58:29 -0800	[diff] [blame]	759	},
				760	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	761	"scheduling": { # Job scheduling options.Beta Feature: These options are available for testing purposes only. They may be changed before final release. # Optional. Job scheduling configuration.
				762	"maxFailuresPerHour": 42, # Optional. Maximum number of times per hour a driver may be restarted as a result of driver terminating with non-zero code before job is reported failed.A job may be reported as thrashing if driver exits with non-zero code 4 times within 10 minute window.Maximum value is 10.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	763	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	764	"statusHistory": [ # Output-only. The previous job status.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	765	{ # Cloud Dataproc job status.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	766	"state": "A String", # Output-only. A state message specifying the overall job state.
				767	"stateStartTime": "A String", # Output-only. The time when this state was entered.
				768	"details": "A String", # Output-only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				769	"substate": "A String", # Output-only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	770	},
				771	],
				772	"hiveJob": { # A Cloud Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN. # Job is a Hive job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	773	"queryFileUri": "A String", # The HCFS URI of the script that contains Hive queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	774	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	775	"a_key": "A String",
				776	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	777	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	778	"A String",
				779	],
				780	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	781	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	782	# "hiveJob": {
				783	# "queryList": {
				784	# "queries": [
				785	# "query1",
				786	# "query2",
				787	# "query3;query4",
				788	# ]
				789	# }
				790	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	791	"A String",
				792	],
				793	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	794	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				795	"properties": { # Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	796	"a_key": "A String",
				797	},
				798	},
				799	},
				800	],
				801	}</pre>
				802	</div>
				803
				804	<div class="method">
				805	<code class="details" id="list_next">list_next(previous_request, previous_response)</code>
				806	<pre>Retrieves the next page of results.
				807
				808	Args:
				809	previous_request: The request for the previous page. (required)
				810	previous_response: The response from the request for the previous page. (required)
				811
				812	Returns:
				813	A request object that you can call 'execute()' on to request the next
				814	page. Returns None if there are no more items in the collection.
				815	</pre>
				816	</div>
				817
				818	<div class="method">
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	819	<code class="details" id="patch">patch(projectId, region, jobId, body, updateMask=None, x__xgafv=None)</code>
				820	<pre>Updates a job in a project.
				821
				822	Args:
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	823	projectId: string, Required. The ID of the Google Cloud Platform project that the job belongs to. (required)
				824	region: string, Required. The Cloud Dataproc region in which to handle the request. (required)
				825	jobId: string, Required. The job ID. (required)
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	826	body: object, The request body. (required)
				827	The object takes the form of:
				828
				829	{ # A Cloud Dataproc job resource.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	830	"status": { # Cloud Dataproc job status. # Output-only. The job status. Additional application-specific status information may be contained in the <code>type_job</code> and <code>yarn_applications</code> fields.
				831	"state": "A String", # Output-only. A state message specifying the overall job state.
				832	"stateStartTime": "A String", # Output-only. The time when this state was entered.
				833	"details": "A String", # Output-only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				834	"substate": "A String", # Output-only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	835	},
				836	"sparkSqlJob": { # A Cloud Dataproc job for running Apache Spark SQL (http://spark.apache.org/sql/) queries. # Job is a SparkSql job.
				837	"queryFileUri": "A String", # The HCFS URI of the script that contains SQL queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	838	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	839	"a_key": "A String",
				840	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	841	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	842	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				843	"a_key": "A String",
				844	},
				845	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	846	"jarFileUris": [ # Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	847	"A String",
				848	],
				849	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	850	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	851	# "hiveJob": {
				852	# "queryList": {
				853	# "queries": [
				854	# "query1",
				855	# "query2",
				856	# "query3;query4",
				857	# ]
				858	# }
				859	# }
				860	"A String",
				861	],
				862	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	863	"properties": { # Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Cloud Dataproc API may be overwritten.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	864	"a_key": "A String",
				865	},
				866	},
				867	"pigJob": { # A Cloud Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN. # Job is a Pig job.
				868	"queryFileUri": "A String", # The HCFS URI of the script that contains the Pig queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	869	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	870	"a_key": "A String",
				871	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	872	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	873	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				874	"a_key": "A String",
				875	},
				876	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	877	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	878	"A String",
				879	],
				880	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	881	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	882	# "hiveJob": {
				883	# "queryList": {
				884	# "queries": [
				885	# "query1",
				886	# "query2",
				887	# "query3;query4",
				888	# ]
				889	# }
				890	# }
				891	"A String",
				892	],
				893	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	894	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				895	"properties": { # Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	896	"a_key": "A String",
				897	},
				898	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	899	"placement": { # Cloud Dataproc job config. # Required. Job information, including how, when, and where to run the job.
				900	"clusterName": "A String", # Required. The name of the cluster where the job will be submitted.
				901	"clusterUuid": "A String", # Output-only. A cluster UUID generated by the Cloud Dataproc service when the job is submitted.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	902	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	903	"reference": { # Encapsulates the full scoping used to reference a job. # Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a <code>job_id</code>.
				904	"projectId": "A String", # Required. The ID of the Google Cloud Platform project that the job belongs to.
				905	"jobId": "A String", # Optional. The job ID, which must be unique within the project. The job ID is generated by the server upon job submission or provided by the user as a means to perform retries without creating duplicate jobs. The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	906	},
				907	"hadoopJob": { # A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). # Job is a Hadoop job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	908	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	909	"A String",
				910	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	911	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	912	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				913	"a_key": "A String",
				914	},
				915	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	916	"jarFileUris": [ # Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	917	"A String",
				918	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	919	"fileUris": [ # Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	920	"A String",
				921	],
				922	"mainClass": "A String", # The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	923	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	924	"A String",
				925	],
				926	"mainJarFileUri": "A String", # The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	927	"properties": { # Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	928	"a_key": "A String",
				929	},
				930	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	931	"labels": { # Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values may be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	932	"a_key": "A String",
				933	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	934	"driverOutputResourceUri": "A String", # Output-only. A URI pointing to the location of the stdout of the job's driver program.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	935	"pysparkJob": { # A Cloud Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/python-programming-guide.html) applications on YARN. # Job is a Pyspark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	936	"mainPythonFileUri": "A String", # Required. The HCFS URI of the main Python file to use as the driver. Must be a .py file.
				937	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	938	"A String",
				939	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	940	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	941	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				942	"a_key": "A String",
				943	},
				944	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	945	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	946	"A String",
				947	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	948	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	949	"A String",
				950	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	951	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	952	"A String",
				953	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	954	"pythonFileUris": [ # Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	955	"A String",
				956	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	957	"properties": { # Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	958	"a_key": "A String",
				959	},
				960	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	961	"driverControlFilesUri": "A String", # Output-only. If present, the location of miscellaneous control files which may be used as part of job setup and handling. If not present, control files may be placed in the same location as driver_output_uri.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	962	"sparkJob": { # A Cloud Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN. # Job is a Spark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	963	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	964	"A String",
				965	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	966	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	967	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				968	"a_key": "A String",
				969	},
				970	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	971	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	972	"A String",
				973	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	974	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Spark drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	975	"A String",
				976	],
				977	"mainClass": "A String", # The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	978	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Spark drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	979	"A String",
				980	],
				981	"mainJarFileUri": "A String", # The HCFS URI of the jar file that contains the main class.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	982	"properties": { # Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	983	"a_key": "A String",
				984	},
				985	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	986	"yarnApplications": [ # Output-only. The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	987	{ # A YARN application created by a job. Application information is a subset of <code>org.apache.hadoop.yarn.proto.YarnProtos.ApplicationReportProto</code>.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	988	"progress": 3.14, # Required. The numerical progress of the application, from 1 to 100.
				989	"state": "A String", # Required. The application state.
				990	"name": "A String", # Required. The application name.
				991	"trackingUrl": "A String", # Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	992	},
				993	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	994	"scheduling": { # Job scheduling options.Beta Feature: These options are available for testing purposes only. They may be changed before final release. # Optional. Job scheduling configuration.
				995	"maxFailuresPerHour": 42, # Optional. Maximum number of times per hour a driver may be restarted as a result of driver terminating with non-zero code before job is reported failed.A job may be reported as thrashing if driver exits with non-zero code 4 times within 10 minute window.Maximum value is 10.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	996	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	997	"statusHistory": [ # Output-only. The previous job status.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	998	{ # Cloud Dataproc job status.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	999	"state": "A String", # Output-only. A state message specifying the overall job state.
				1000	"stateStartTime": "A String", # Output-only. The time when this state was entered.
				1001	"details": "A String", # Output-only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				1002	"substate": "A String", # Output-only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1003	},
				1004	],
				1005	"hiveJob": { # A Cloud Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN. # Job is a Hive job.
				1006	"queryFileUri": "A String", # The HCFS URI of the script that contains Hive queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1007	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1008	"a_key": "A String",
				1009	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1010	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1011	"A String",
				1012	],
				1013	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1014	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1015	# "hiveJob": {
				1016	# "queryList": {
				1017	# "queries": [
				1018	# "query1",
				1019	# "query2",
				1020	# "query3;query4",
				1021	# ]
				1022	# }
				1023	# }
				1024	"A String",
				1025	],
				1026	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1027	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				1028	"properties": { # Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1029	"a_key": "A String",
				1030	},
				1031	},
				1032	}
				1033
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1034	updateMask: string, Required. Specifies the path, relative to <code>Job</code>, of the field to update. For example, to update the labels of a Job the <code>update_mask</code> parameter would be specified as <code>labels</code>, and the PATCH request body would specify the new value. <strong>Note:</strong> Currently, <code>labels</code> is the only field that can be updated.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1035	x__xgafv: string, V1 error format.
				1036	Allowed values
				1037	1 - v1 error format
				1038	2 - v2 error format
				1039
				1040	Returns:
				1041	An object of the form:
				1042
				1043	{ # A Cloud Dataproc job resource.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1044	"status": { # Cloud Dataproc job status. # Output-only. The job status. Additional application-specific status information may be contained in the <code>type_job</code> and <code>yarn_applications</code> fields.
				1045	"state": "A String", # Output-only. A state message specifying the overall job state.
				1046	"stateStartTime": "A String", # Output-only. The time when this state was entered.
				1047	"details": "A String", # Output-only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				1048	"substate": "A String", # Output-only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1049	},
				1050	"sparkSqlJob": { # A Cloud Dataproc job for running Apache Spark SQL (http://spark.apache.org/sql/) queries. # Job is a SparkSql job.
				1051	"queryFileUri": "A String", # The HCFS URI of the script that contains SQL queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1052	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1053	"a_key": "A String",
				1054	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1055	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1056	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1057	"a_key": "A String",
				1058	},
				1059	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1060	"jarFileUris": [ # Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1061	"A String",
				1062	],
				1063	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1064	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1065	# "hiveJob": {
				1066	# "queryList": {
				1067	# "queries": [
				1068	# "query1",
				1069	# "query2",
				1070	# "query3;query4",
				1071	# ]
				1072	# }
				1073	# }
				1074	"A String",
				1075	],
				1076	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1077	"properties": { # Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Cloud Dataproc API may be overwritten.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1078	"a_key": "A String",
				1079	},
				1080	},
				1081	"pigJob": { # A Cloud Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN. # Job is a Pig job.
				1082	"queryFileUri": "A String", # The HCFS URI of the script that contains the Pig queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1083	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1084	"a_key": "A String",
				1085	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1086	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1087	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1088	"a_key": "A String",
				1089	},
				1090	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1091	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1092	"A String",
				1093	],
				1094	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1095	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1096	# "hiveJob": {
				1097	# "queryList": {
				1098	# "queries": [
				1099	# "query1",
				1100	# "query2",
				1101	# "query3;query4",
				1102	# ]
				1103	# }
				1104	# }
				1105	"A String",
				1106	],
				1107	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1108	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				1109	"properties": { # Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1110	"a_key": "A String",
				1111	},
				1112	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1113	"placement": { # Cloud Dataproc job config. # Required. Job information, including how, when, and where to run the job.
				1114	"clusterName": "A String", # Required. The name of the cluster where the job will be submitted.
				1115	"clusterUuid": "A String", # Output-only. A cluster UUID generated by the Cloud Dataproc service when the job is submitted.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1116	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1117	"reference": { # Encapsulates the full scoping used to reference a job. # Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a <code>job_id</code>.
				1118	"projectId": "A String", # Required. The ID of the Google Cloud Platform project that the job belongs to.
				1119	"jobId": "A String", # Optional. The job ID, which must be unique within the project. The job ID is generated by the server upon job submission or provided by the user as a means to perform retries without creating duplicate jobs. The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1120	},
				1121	"hadoopJob": { # A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). # Job is a Hadoop job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1122	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1123	"A String",
				1124	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1125	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1126	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1127	"a_key": "A String",
				1128	},
				1129	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1130	"jarFileUris": [ # Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1131	"A String",
				1132	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1133	"fileUris": [ # Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1134	"A String",
				1135	],
				1136	"mainClass": "A String", # The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1137	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1138	"A String",
				1139	],
				1140	"mainJarFileUri": "A String", # The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1141	"properties": { # Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1142	"a_key": "A String",
				1143	},
				1144	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1145	"labels": { # Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values may be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1146	"a_key": "A String",
				1147	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1148	"driverOutputResourceUri": "A String", # Output-only. A URI pointing to the location of the stdout of the job's driver program.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1149	"pysparkJob": { # A Cloud Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/python-programming-guide.html) applications on YARN. # Job is a Pyspark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1150	"mainPythonFileUri": "A String", # Required. The HCFS URI of the main Python file to use as the driver. Must be a .py file.
				1151	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1152	"A String",
				1153	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1154	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1155	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1156	"a_key": "A String",
				1157	},
				1158	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1159	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1160	"A String",
				1161	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1162	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1163	"A String",
				1164	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1165	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1166	"A String",
				1167	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1168	"pythonFileUris": [ # Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1169	"A String",
				1170	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1171	"properties": { # Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1172	"a_key": "A String",
				1173	},
				1174	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1175	"driverControlFilesUri": "A String", # Output-only. If present, the location of miscellaneous control files which may be used as part of job setup and handling. If not present, control files may be placed in the same location as driver_output_uri.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1176	"sparkJob": { # A Cloud Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN. # Job is a Spark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1177	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1178	"A String",
				1179	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1180	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1181	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1182	"a_key": "A String",
				1183	},
				1184	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1185	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1186	"A String",
				1187	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1188	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Spark drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1189	"A String",
				1190	],
				1191	"mainClass": "A String", # The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1192	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Spark drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1193	"A String",
				1194	],
				1195	"mainJarFileUri": "A String", # The HCFS URI of the jar file that contains the main class.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1196	"properties": { # Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1197	"a_key": "A String",
				1198	},
				1199	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1200	"yarnApplications": [ # Output-only. The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1201	{ # A YARN application created by a job. Application information is a subset of <code>org.apache.hadoop.yarn.proto.YarnProtos.ApplicationReportProto</code>.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1202	"progress": 3.14, # Required. The numerical progress of the application, from 1 to 100.
				1203	"state": "A String", # Required. The application state.
				1204	"name": "A String", # Required. The application name.
				1205	"trackingUrl": "A String", # Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1206	},
				1207	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1208	"scheduling": { # Job scheduling options.Beta Feature: These options are available for testing purposes only. They may be changed before final release. # Optional. Job scheduling configuration.
				1209	"maxFailuresPerHour": 42, # Optional. Maximum number of times per hour a driver may be restarted as a result of driver terminating with non-zero code before job is reported failed.A job may be reported as thrashing if driver exits with non-zero code 4 times within 10 minute window.Maximum value is 10.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1210	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1211	"statusHistory": [ # Output-only. The previous job status.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1212	{ # Cloud Dataproc job status.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1213	"state": "A String", # Output-only. A state message specifying the overall job state.
				1214	"stateStartTime": "A String", # Output-only. The time when this state was entered.
				1215	"details": "A String", # Output-only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				1216	"substate": "A String", # Output-only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1217	},
				1218	],
				1219	"hiveJob": { # A Cloud Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN. # Job is a Hive job.
				1220	"queryFileUri": "A String", # The HCFS URI of the script that contains Hive queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1221	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1222	"a_key": "A String",
				1223	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1224	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1225	"A String",
				1226	],
				1227	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1228	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1229	# "hiveJob": {
				1230	# "queryList": {
				1231	# "queries": [
				1232	# "query1",
				1233	# "query2",
				1234	# "query3;query4",
				1235	# ]
				1236	# }
				1237	# }
				1238	"A String",
				1239	],
				1240	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1241	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				1242	"properties": { # Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1243	"a_key": "A String",
				1244	},
				1245	},
				1246	}</pre>
				1247	</div>
				1248
				1249	<div class="method">
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1250	<code class="details" id="submit">submit(projectId, region, body, x__xgafv=None)</code>
				1251	<pre>Submits a job to a cluster.
				1252
				1253	Args:
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1254	projectId: string, Required. The ID of the Google Cloud Platform project that the job belongs to. (required)
				1255	region: string, Required. The Cloud Dataproc region in which to handle the request. (required)
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1256	body: object, The request body. (required)
				1257	The object takes the form of:
				1258
				1259	{ # A request to submit a job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1260	"job": { # A Cloud Dataproc job resource. # Required. The job resource.
				1261	"status": { # Cloud Dataproc job status. # Output-only. The job status. Additional application-specific status information may be contained in the <code>type_job</code> and <code>yarn_applications</code> fields.
				1262	"state": "A String", # Output-only. A state message specifying the overall job state.
				1263	"stateStartTime": "A String", # Output-only. The time when this state was entered.
				1264	"details": "A String", # Output-only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				1265	"substate": "A String", # Output-only. Additional state information, which includes status reported by the agent.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1266	},
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1267	"sparkSqlJob": { # A Cloud Dataproc job for running Apache Spark SQL (http://spark.apache.org/sql/) queries. # Job is a SparkSql job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1268	"queryFileUri": "A String", # The HCFS URI of the script that contains SQL queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1269	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1270	"a_key": "A String",
				1271	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1272	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1273	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1274	"a_key": "A String",
				1275	},
				1276	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1277	"jarFileUris": [ # Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1278	"A String",
				1279	],
				1280	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1281	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1282	# "hiveJob": {
				1283	# "queryList": {
				1284	# "queries": [
				1285	# "query1",
				1286	# "query2",
				1287	# "query3;query4",
				1288	# ]
				1289	# }
				1290	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1291	"A String",
				1292	],
				1293	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1294	"properties": { # Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Cloud Dataproc API may be overwritten.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1295	"a_key": "A String",
				1296	},
				1297	},
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1298	"pigJob": { # A Cloud Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN. # Job is a Pig job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1299	"queryFileUri": "A String", # The HCFS URI of the script that contains the Pig queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1300	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1301	"a_key": "A String",
				1302	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1303	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1304	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1305	"a_key": "A String",
				1306	},
				1307	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1308	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1309	"A String",
				1310	],
				1311	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1312	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1313	# "hiveJob": {
				1314	# "queryList": {
				1315	# "queries": [
				1316	# "query1",
				1317	# "query2",
				1318	# "query3;query4",
				1319	# ]
				1320	# }
				1321	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1322	"A String",
				1323	],
				1324	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1325	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				1326	"properties": { # Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1327	"a_key": "A String",
				1328	},
				1329	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1330	"placement": { # Cloud Dataproc job config. # Required. Job information, including how, when, and where to run the job.
				1331	"clusterName": "A String", # Required. The name of the cluster where the job will be submitted.
				1332	"clusterUuid": "A String", # Output-only. A cluster UUID generated by the Cloud Dataproc service when the job is submitted.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1333	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1334	"reference": { # Encapsulates the full scoping used to reference a job. # Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a <code>job_id</code>.
				1335	"projectId": "A String", # Required. The ID of the Google Cloud Platform project that the job belongs to.
				1336	"jobId": "A String", # Optional. The job ID, which must be unique within the project. The job ID is generated by the server upon job submission or provided by the user as a means to perform retries without creating duplicate jobs. The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1337	},
				1338	"hadoopJob": { # A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). # Job is a Hadoop job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1339	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1340	"A String",
				1341	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1342	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1343	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1344	"a_key": "A String",
				1345	},
				1346	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1347	"jarFileUris": [ # Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1348	"A String",
				1349	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1350	"fileUris": [ # Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1351	"A String",
				1352	],
				1353	"mainClass": "A String", # The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1354	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1355	"A String",
				1356	],
				1357	"mainJarFileUri": "A String", # The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1358	"properties": { # Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1359	"a_key": "A String",
				1360	},
				1361	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1362	"labels": { # Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values may be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1363	"a_key": "A String",
				1364	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1365	"driverOutputResourceUri": "A String", # Output-only. A URI pointing to the location of the stdout of the job's driver program.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1366	"pysparkJob": { # A Cloud Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/python-programming-guide.html) applications on YARN. # Job is a Pyspark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1367	"mainPythonFileUri": "A String", # Required. The HCFS URI of the main Python file to use as the driver. Must be a .py file.
				1368	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1369	"A String",
				1370	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1371	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1372	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1373	"a_key": "A String",
				1374	},
				1375	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1376	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1377	"A String",
				1378	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1379	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1380	"A String",
				1381	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1382	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1383	"A String",
				1384	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1385	"pythonFileUris": [ # Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1386	"A String",
				1387	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1388	"properties": { # Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1389	"a_key": "A String",
				1390	},
				1391	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1392	"driverControlFilesUri": "A String", # Output-only. If present, the location of miscellaneous control files which may be used as part of job setup and handling. If not present, control files may be placed in the same location as driver_output_uri.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1393	"sparkJob": { # A Cloud Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN. # Job is a Spark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1394	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1395	"A String",
				1396	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1397	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1398	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1399	"a_key": "A String",
				1400	},
				1401	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1402	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1403	"A String",
				1404	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1405	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Spark drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1406	"A String",
				1407	],
				1408	"mainClass": "A String", # The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1409	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Spark drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1410	"A String",
				1411	],
Jon Wayne Parrott	0a471d3	2016-05-19 10:54:38 -0700	[diff] [blame]	1412	"mainJarFileUri": "A String", # The HCFS URI of the jar file that contains the main class.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1413	"properties": { # Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1414	"a_key": "A String",
				1415	},
				1416	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1417	"yarnApplications": [ # Output-only. The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1418	{ # A YARN application created by a job. Application information is a subset of <code>org.apache.hadoop.yarn.proto.YarnProtos.ApplicationReportProto</code>.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1419	"progress": 3.14, # Required. The numerical progress of the application, from 1 to 100.
				1420	"state": "A String", # Required. The application state.
				1421	"name": "A String", # Required. The application name.
				1422	"trackingUrl": "A String", # Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
Jon Wayne Parrott	692617a	2017-01-06 09:58:29 -0800	[diff] [blame]	1423	},
				1424	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1425	"scheduling": { # Job scheduling options.Beta Feature: These options are available for testing purposes only. They may be changed before final release. # Optional. Job scheduling configuration.
				1426	"maxFailuresPerHour": 42, # Optional. Maximum number of times per hour a driver may be restarted as a result of driver terminating with non-zero code before job is reported failed.A job may be reported as thrashing if driver exits with non-zero code 4 times within 10 minute window.Maximum value is 10.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1427	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1428	"statusHistory": [ # Output-only. The previous job status.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1429	{ # Cloud Dataproc job status.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1430	"state": "A String", # Output-only. A state message specifying the overall job state.
				1431	"stateStartTime": "A String", # Output-only. The time when this state was entered.
				1432	"details": "A String", # Output-only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				1433	"substate": "A String", # Output-only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1434	},
				1435	],
				1436	"hiveJob": { # A Cloud Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN. # Job is a Hive job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1437	"queryFileUri": "A String", # The HCFS URI of the script that contains Hive queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1438	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1439	"a_key": "A String",
				1440	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1441	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1442	"A String",
				1443	],
				1444	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1445	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1446	# "hiveJob": {
				1447	# "queryList": {
				1448	# "queries": [
				1449	# "query1",
				1450	# "query2",
				1451	# "query3;query4",
				1452	# ]
				1453	# }
				1454	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1455	"A String",
				1456	],
				1457	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1458	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				1459	"properties": { # Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1460	"a_key": "A String",
				1461	},
				1462	},
				1463	},
				1464	}
				1465
				1466	x__xgafv: string, V1 error format.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1467	Allowed values
				1468	1 - v1 error format
				1469	2 - v2 error format
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1470
				1471	Returns:
				1472	An object of the form:
				1473
				1474	{ # A Cloud Dataproc job resource.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1475	"status": { # Cloud Dataproc job status. # Output-only. The job status. Additional application-specific status information may be contained in the <code>type_job</code> and <code>yarn_applications</code> fields.
				1476	"state": "A String", # Output-only. A state message specifying the overall job state.
				1477	"stateStartTime": "A String", # Output-only. The time when this state was entered.
				1478	"details": "A String", # Output-only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				1479	"substate": "A String", # Output-only. Additional state information, which includes status reported by the agent.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1480	},
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1481	"sparkSqlJob": { # A Cloud Dataproc job for running Apache Spark SQL (http://spark.apache.org/sql/) queries. # Job is a SparkSql job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1482	"queryFileUri": "A String", # The HCFS URI of the script that contains SQL queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1483	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1484	"a_key": "A String",
				1485	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1486	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1487	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1488	"a_key": "A String",
				1489	},
				1490	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1491	"jarFileUris": [ # Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1492	"A String",
				1493	],
				1494	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1495	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1496	# "hiveJob": {
				1497	# "queryList": {
				1498	# "queries": [
				1499	# "query1",
				1500	# "query2",
				1501	# "query3;query4",
				1502	# ]
				1503	# }
				1504	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1505	"A String",
				1506	],
				1507	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1508	"properties": { # Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Cloud Dataproc API may be overwritten.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1509	"a_key": "A String",
				1510	},
				1511	},
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1512	"pigJob": { # A Cloud Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN. # Job is a Pig job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1513	"queryFileUri": "A String", # The HCFS URI of the script that contains the Pig queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1514	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1515	"a_key": "A String",
				1516	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1517	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1518	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1519	"a_key": "A String",
				1520	},
				1521	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1522	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1523	"A String",
				1524	],
				1525	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1526	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1527	# "hiveJob": {
				1528	# "queryList": {
				1529	# "queries": [
				1530	# "query1",
				1531	# "query2",
				1532	# "query3;query4",
				1533	# ]
				1534	# }
				1535	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1536	"A String",
				1537	],
				1538	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1539	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				1540	"properties": { # Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1541	"a_key": "A String",
				1542	},
				1543	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1544	"placement": { # Cloud Dataproc job config. # Required. Job information, including how, when, and where to run the job.
				1545	"clusterName": "A String", # Required. The name of the cluster where the job will be submitted.
				1546	"clusterUuid": "A String", # Output-only. A cluster UUID generated by the Cloud Dataproc service when the job is submitted.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1547	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1548	"reference": { # Encapsulates the full scoping used to reference a job. # Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a <code>job_id</code>.
				1549	"projectId": "A String", # Required. The ID of the Google Cloud Platform project that the job belongs to.
				1550	"jobId": "A String", # Optional. The job ID, which must be unique within the project. The job ID is generated by the server upon job submission or provided by the user as a means to perform retries without creating duplicate jobs. The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1551	},
				1552	"hadoopJob": { # A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). # Job is a Hadoop job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1553	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1554	"A String",
				1555	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1556	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1557	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1558	"a_key": "A String",
				1559	},
				1560	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1561	"jarFileUris": [ # Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1562	"A String",
				1563	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1564	"fileUris": [ # Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1565	"A String",
				1566	],
				1567	"mainClass": "A String", # The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1568	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1569	"A String",
				1570	],
				1571	"mainJarFileUri": "A String", # The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1572	"properties": { # Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1573	"a_key": "A String",
				1574	},
				1575	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1576	"labels": { # Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values may be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1577	"a_key": "A String",
				1578	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1579	"driverOutputResourceUri": "A String", # Output-only. A URI pointing to the location of the stdout of the job's driver program.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1580	"pysparkJob": { # A Cloud Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/python-programming-guide.html) applications on YARN. # Job is a Pyspark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1581	"mainPythonFileUri": "A String", # Required. The HCFS URI of the main Python file to use as the driver. Must be a .py file.
				1582	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1583	"A String",
				1584	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1585	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1586	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1587	"a_key": "A String",
				1588	},
				1589	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1590	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1591	"A String",
				1592	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1593	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1594	"A String",
				1595	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1596	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1597	"A String",
				1598	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1599	"pythonFileUris": [ # Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1600	"A String",
				1601	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1602	"properties": { # Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1603	"a_key": "A String",
				1604	},
				1605	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1606	"driverControlFilesUri": "A String", # Output-only. If present, the location of miscellaneous control files which may be used as part of job setup and handling. If not present, control files may be placed in the same location as driver_output_uri.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1607	"sparkJob": { # A Cloud Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN. # Job is a Spark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1608	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1609	"A String",
				1610	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1611	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1612	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1613	"a_key": "A String",
				1614	},
				1615	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1616	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1617	"A String",
				1618	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1619	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Spark drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1620	"A String",
				1621	],
				1622	"mainClass": "A String", # The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1623	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Spark drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1624	"A String",
				1625	],
Jon Wayne Parrott	0a471d3	2016-05-19 10:54:38 -0700	[diff] [blame]	1626	"mainJarFileUri": "A String", # The HCFS URI of the jar file that contains the main class.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1627	"properties": { # Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1628	"a_key": "A String",
				1629	},
				1630	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1631	"yarnApplications": [ # Output-only. The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1632	{ # A YARN application created by a job. Application information is a subset of <code>org.apache.hadoop.yarn.proto.YarnProtos.ApplicationReportProto</code>.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1633	"progress": 3.14, # Required. The numerical progress of the application, from 1 to 100.
				1634	"state": "A String", # Required. The application state.
				1635	"name": "A String", # Required. The application name.
				1636	"trackingUrl": "A String", # Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
Jon Wayne Parrott	692617a	2017-01-06 09:58:29 -0800	[diff] [blame]	1637	},
				1638	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1639	"scheduling": { # Job scheduling options.Beta Feature: These options are available for testing purposes only. They may be changed before final release. # Optional. Job scheduling configuration.
				1640	"maxFailuresPerHour": 42, # Optional. Maximum number of times per hour a driver may be restarted as a result of driver terminating with non-zero code before job is reported failed.A job may be reported as thrashing if driver exits with non-zero code 4 times within 10 minute window.Maximum value is 10.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1641	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1642	"statusHistory": [ # Output-only. The previous job status.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1643	{ # Cloud Dataproc job status.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1644	"state": "A String", # Output-only. A state message specifying the overall job state.
				1645	"stateStartTime": "A String", # Output-only. The time when this state was entered.
				1646	"details": "A String", # Output-only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				1647	"substate": "A String", # Output-only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1648	},
				1649	],
				1650	"hiveJob": { # A Cloud Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN. # Job is a Hive job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1651	"queryFileUri": "A String", # The HCFS URI of the script that contains Hive queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1652	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1653	"a_key": "A String",
				1654	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1655	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1656	"A String",
				1657	],
				1658	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1659	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1660	# "hiveJob": {
				1661	# "queryList": {
				1662	# "queries": [
				1663	# "query1",
				1664	# "query2",
				1665	# "query3;query4",
				1666	# ]
				1667	# }
				1668	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1669	"A String",
				1670	],
				1671	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame^]	1672	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				1673	"properties": { # Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1674	"a_key": "A String",
				1675	},
				1676	},
				1677	}</pre>
				1678	</div>
				1679
				1680	</body></html>