Blame - docs/dyn/dataproc_v1.projects.regions.jobs.html - platform/external/python/google-api-python-client

blob: c9a93c5b37e8adc1694df743477d5b0026b88af7 [file] [log] [blame]

Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1	<html><body>
				2	<style>
				3
				4	body, h1, h2, h3, div, span, p, pre, a {
				5	margin: 0;
				6	padding: 0;
				7	border: 0;
				8	font-weight: inherit;
				9	font-style: inherit;
				10	font-size: 100%;
				11	font-family: inherit;
				12	vertical-align: baseline;
				13	}
				14
				15	body {
				16	font-size: 13px;
				17	padding: 1em;
				18	}
				19
				20	h1 {
				21	font-size: 26px;
				22	margin-bottom: 1em;
				23	}
				24
				25	h2 {
				26	font-size: 24px;
				27	margin-bottom: 1em;
				28	}
				29
				30	h3 {
				31	font-size: 20px;
				32	margin-bottom: 1em;
				33	margin-top: 1em;
				34	}
				35
				36	pre, code {
				37	line-height: 1.5;
				38	font-family: Monaco, 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Lucida Console', monospace;
				39	}
				40
				41	pre {
				42	margin-top: 0.5em;
				43	}
				44
				45	h1, h2, h3, p {
				46	font-family: Arial, sans serif;
				47	}
				48
				49	h1, h2, h3 {
				50	border-bottom: solid #CCC 1px;
				51	}
				52
				53	.toc_element {
				54	margin-top: 0.5em;
				55	}
				56
				57	.firstline {
				58	margin-left: 2 em;
				59	}
				60
				61	.method {
				62	margin-top: 1em;
				63	border: solid 1px #CCC;
				64	padding: 1em;
				65	background: #EEE;
				66	}
				67
				68	.details {
				69	font-weight: bold;
				70	font-size: 14px;
				71	}
				72
				73	</style>
				74
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	75	<h1><a href="dataproc_v1.html">Cloud Dataproc API</a> . <a href="dataproc_v1.projects.html">projects</a> . <a href="dataproc_v1.projects.regions.html">regions</a> . <a href="dataproc_v1.projects.regions.jobs.html">jobs</a></h1>
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	76	<h2>Instance Methods</h2>
				77	<p class="toc_element">
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	78	<code><a href="#cancel">cancel(projectId, region, jobId, body=None, x__xgafv=None)</a></code></p>
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	79	<p class="firstline">Starts a job cancellation request. To access the job resource after cancellation, call regions/{region}/jobs.list or regions/{region}/jobs.get.</p>
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	80	<p class="toc_element">
				81	<code><a href="#delete">delete(projectId, region, jobId, x__xgafv=None)</a></code></p>
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	82	<p class="firstline">Deletes the job from the project. If the job is active, the delete fails, and the response returns FAILED_PRECONDITION.</p>
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	83	<p class="toc_element">
				84	<code><a href="#get">get(projectId, region, jobId, x__xgafv=None)</a></code></p>
				85	<p class="firstline">Gets the resource representation for a job in a project.</p>
				86	<p class="toc_element">
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	87	<code><a href="#getIamPolicy">getIamPolicy(resource, body=None, x__xgafv=None)</a></code></p>
				88	<p class="firstline">Gets the access control policy for a resource. Returns an empty policy if the resource exists and does not have a policy set.</p>
				89	<p class="toc_element">
				90	<code><a href="#list">list(projectId, region, pageSize=None, x__xgafv=None, jobStateMatcher=None, pageToken=None, clusterName=None, filter=None)</a></code></p>
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	91	<p class="firstline">Lists regions/{region}/jobs in a project.</p>
				92	<p class="toc_element">
				93	<code><a href="#list_next">list_next(previous_request, previous_response)</a></code></p>
				94	<p class="firstline">Retrieves the next page of results.</p>
				95	<p class="toc_element">
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	96	<code><a href="#patch">patch(projectId, region, jobId, body, updateMask=None, x__xgafv=None)</a></code></p>
				97	<p class="firstline">Updates a job in a project.</p>
				98	<p class="toc_element">
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	99	<code><a href="#setIamPolicy">setIamPolicy(resource, body, x__xgafv=None)</a></code></p>
				100	<p class="firstline">Sets the access control policy on the specified resource. Replaces any existing policy.</p>
				101	<p class="toc_element">
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	102	<code><a href="#submit">submit(projectId, region, body, x__xgafv=None)</a></code></p>
				103	<p class="firstline">Submits a job to a cluster.</p>
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	104	<p class="toc_element">
				105	<code><a href="#testIamPermissions">testIamPermissions(resource, body, x__xgafv=None)</a></code></p>
				106	<p class="firstline">Returns permissions that a caller has on the specified resource. If the resource does not exist, this will return an empty set of permissions, not a NOT_FOUND error.Note: This operation is designed to be used for building permission-aware UIs and command-line tools, not for authorization checking. This operation may "fail open" without warning.</p>
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	107	<h3>Method Details</h3>
				108	<div class="method">
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	109	<code class="details" id="cancel">cancel(projectId, region, jobId, body=None, x__xgafv=None)</code>
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	110	<pre>Starts a job cancellation request. To access the job resource after cancellation, call regions/{region}/jobs.list or regions/{region}/jobs.get.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	111
				112	Args:
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	113	projectId: string, Required. The ID of the Google Cloud Platform project that the job belongs to. (required)
				114	region: string, Required. The Cloud Dataproc region in which to handle the request. (required)
				115	jobId: string, Required. The job ID. (required)
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	116	body: object, The request body.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	117	The object takes the form of:
				118
				119	{ # A request to cancel a job.
				120	}
				121
				122	x__xgafv: string, V1 error format.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	123	Allowed values
				124	1 - v1 error format
				125	2 - v2 error format
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	126
				127	Returns:
				128	An object of the form:
				129
				130	{ # A Cloud Dataproc job resource.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	131	"status": { # Cloud Dataproc job status. # Output only. The job status. Additional application-specific status information may be contained in the <code>type_job</code> and <code>yarn_applications</code> fields.
				132	"state": "A String", # Output only. A state message specifying the overall job state.
				133	"stateStartTime": "A String", # Output only. The time when this state was entered.
				134	"details": "A String", # Output only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				135	"substate": "A String", # Output only. Additional state information, which includes status reported by the agent.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	136	},
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	137	"sparkSqlJob": { # A Cloud Dataproc job for running Apache Spark SQL (http://spark.apache.org/sql/) queries. # Job is a SparkSql job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	138	"queryFileUri": "A String", # The HCFS URI of the script that contains SQL queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	139	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	140	"a_key": "A String",
				141	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	142	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	143	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	144	"a_key": "A String",
				145	},
				146	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	147	"jarFileUris": [ # Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	148	"A String",
				149	],
				150	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	151	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	152	# "hiveJob": {
				153	# "queryList": {
				154	# "queries": [
				155	# "query1",
				156	# "query2",
				157	# "query3;query4",
				158	# ]
				159	# }
				160	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	161	"A String",
				162	],
				163	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	164	"properties": { # Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Cloud Dataproc API may be overwritten.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	165	"a_key": "A String",
				166	},
				167	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	168	"labels": { # Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values may be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
				169	"a_key": "A String",
				170	},
				171	"placement": { # Cloud Dataproc job config. # Required. Job information, including how, when, and where to run the job.
				172	"clusterName": "A String", # Required. The name of the cluster where the job will be submitted.
				173	"clusterUuid": "A String", # Output only. A cluster UUID generated by the Cloud Dataproc service when the job is submitted.
				174	},
				175	"reference": { # Encapsulates the full scoping used to reference a job. # Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a <code>job_id</code>.
				176	"projectId": "A String", # Required. The ID of the Google Cloud Platform project that the job belongs to.
				177	"jobId": "A String", # Optional. The job ID, which must be unique within the project.The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.If not specified by the caller, the job ID will be provided by the server.
				178	},
				179	"hadoopJob": { # A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). # Job is a Hadoop job.
				180	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
				181	"A String",
				182	],
				183	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
				184	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				185	"a_key": "A String",
				186	},
				187	},
				188	"jarFileUris": [ # Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
				189	"A String",
				190	],
				191	"fileUris": [ # Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
				192	"A String",
				193	],
				194	"mainClass": "A String", # The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
				195	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
				196	"A String",
				197	],
				198	"mainJarFileUri": "A String", # The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
				199	"properties": { # Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
				200	"a_key": "A String",
				201	},
				202	},
				203	"yarnApplications": [ # Output only. The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
				204	{ # A YARN application created by a job. Application information is a subset of <code>org.apache.hadoop.yarn.proto.YarnProtos.ApplicationReportProto</code>.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
				205	"progress": 3.14, # Required. The numerical progress of the application, from 1 to 100.
				206	"state": "A String", # Required. The application state.
				207	"name": "A String", # Required. The application name.
				208	"trackingUrl": "A String", # Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
				209	},
				210	],
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	211	"pigJob": { # A Cloud Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN. # Job is a Pig job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	212	"queryFileUri": "A String", # The HCFS URI of the script that contains the Pig queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	213	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	214	"a_key": "A String",
				215	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	216	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	217	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	218	"a_key": "A String",
				219	},
				220	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	221	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	222	"A String",
				223	],
				224	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	225	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	226	# "hiveJob": {
				227	# "queryList": {
				228	# "queries": [
				229	# "query1",
				230	# "query2",
				231	# "query3;query4",
				232	# ]
				233	# }
				234	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	235	"A String",
				236	],
				237	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	238	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				239	"properties": { # Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	240	"a_key": "A String",
				241	},
				242	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	243	"driverOutputResourceUri": "A String", # Output only. A URI pointing to the location of the stdout of the job's driver program.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	244	"pysparkJob": { # A Cloud Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/python-programming-guide.html) applications on YARN. # Job is a Pyspark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	245	"mainPythonFileUri": "A String", # Required. The HCFS URI of the main Python file to use as the driver. Must be a .py file.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	246	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	247	"A String",
				248	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	249	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	250	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				251	"a_key": "A String",
				252	},
				253	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	254	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	255	"A String",
				256	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	257	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	258	"A String",
				259	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	260	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	261	"A String",
				262	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	263	"pythonFileUris": [ # Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	264	"A String",
				265	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	266	"properties": { # Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	267	"a_key": "A String",
				268	},
				269	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	270	"driverControlFilesUri": "A String", # Output only. If present, the location of miscellaneous control files which may be used as part of job setup and handling. If not present, control files may be placed in the same location as driver_output_uri.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	271	"sparkJob": { # A Cloud Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN. # Job is a Spark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	272	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	273	"A String",
				274	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	275	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	276	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				277	"a_key": "A String",
				278	},
				279	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	280	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	281	"A String",
				282	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	283	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Spark drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	284	"A String",
				285	],
				286	"mainClass": "A String", # The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	287	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Spark drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	288	"A String",
				289	],
Jon Wayne Parrott	0a471d3	2016-05-19 10:54:38 -0700	[diff] [blame]	290	"mainJarFileUri": "A String", # The HCFS URI of the jar file that contains the main class.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	291	"properties": { # Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	292	"a_key": "A String",
				293	},
				294	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	295	"jobUuid": "A String", # Output only. A UUID that uniquely identifies a job within the project over time. This is in contrast to a user-settable reference.job_id that may be reused over time.
				296	"scheduling": { # Job scheduling options. # Optional. Job scheduling configuration.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	297	"maxFailuresPerHour": 42, # Optional. Maximum number of times per hour a driver may be restarted as a result of driver terminating with non-zero code before job is reported failed.A job may be reported as thrashing if driver exits with non-zero code 4 times within 10 minute window.Maximum value is 10.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	298	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	299	"statusHistory": [ # Output only. The previous job status.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	300	{ # Cloud Dataproc job status.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	301	"state": "A String", # Output only. A state message specifying the overall job state.
				302	"stateStartTime": "A String", # Output only. The time when this state was entered.
				303	"details": "A String", # Output only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				304	"substate": "A String", # Output only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	305	},
				306	],
				307	"hiveJob": { # A Cloud Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN. # Job is a Hive job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	308	"queryFileUri": "A String", # The HCFS URI of the script that contains Hive queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	309	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	310	"a_key": "A String",
				311	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	312	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	313	"A String",
				314	],
				315	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	316	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	317	# "hiveJob": {
				318	# "queryList": {
				319	# "queries": [
				320	# "query1",
				321	# "query2",
				322	# "query3;query4",
				323	# ]
				324	# }
				325	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	326	"A String",
				327	],
				328	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	329	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				330	"properties": { # Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	331	"a_key": "A String",
				332	},
				333	},
				334	}</pre>
				335	</div>
				336
				337	<div class="method">
				338	<code class="details" id="delete">delete(projectId, region, jobId, x__xgafv=None)</code>
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	339	<pre>Deletes the job from the project. If the job is active, the delete fails, and the response returns FAILED_PRECONDITION.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	340
				341	Args:
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	342	projectId: string, Required. The ID of the Google Cloud Platform project that the job belongs to. (required)
				343	region: string, Required. The Cloud Dataproc region in which to handle the request. (required)
				344	jobId: string, Required. The job ID. (required)
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	345	x__xgafv: string, V1 error format.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	346	Allowed values
				347	1 - v1 error format
				348	2 - v2 error format
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	349
				350	Returns:
				351	An object of the form:
				352
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	353	{ # A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance:
				354	# service Foo {
				355	# rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty);
				356	# }
				357	# The JSON representation for Empty is empty JSON object {}.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	358	}</pre>
				359	</div>
				360
				361	<div class="method">
				362	<code class="details" id="get">get(projectId, region, jobId, x__xgafv=None)</code>
				363	<pre>Gets the resource representation for a job in a project.
				364
				365	Args:
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	366	projectId: string, Required. The ID of the Google Cloud Platform project that the job belongs to. (required)
				367	region: string, Required. The Cloud Dataproc region in which to handle the request. (required)
				368	jobId: string, Required. The job ID. (required)
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	369	x__xgafv: string, V1 error format.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	370	Allowed values
				371	1 - v1 error format
				372	2 - v2 error format
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	373
				374	Returns:
				375	An object of the form:
				376
				377	{ # A Cloud Dataproc job resource.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	378	"status": { # Cloud Dataproc job status. # Output only. The job status. Additional application-specific status information may be contained in the <code>type_job</code> and <code>yarn_applications</code> fields.
				379	"state": "A String", # Output only. A state message specifying the overall job state.
				380	"stateStartTime": "A String", # Output only. The time when this state was entered.
				381	"details": "A String", # Output only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				382	"substate": "A String", # Output only. Additional state information, which includes status reported by the agent.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	383	},
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	384	"sparkSqlJob": { # A Cloud Dataproc job for running Apache Spark SQL (http://spark.apache.org/sql/) queries. # Job is a SparkSql job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	385	"queryFileUri": "A String", # The HCFS URI of the script that contains SQL queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	386	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	387	"a_key": "A String",
				388	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	389	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	390	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	391	"a_key": "A String",
				392	},
				393	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	394	"jarFileUris": [ # Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	395	"A String",
				396	],
				397	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	398	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	399	# "hiveJob": {
				400	# "queryList": {
				401	# "queries": [
				402	# "query1",
				403	# "query2",
				404	# "query3;query4",
				405	# ]
				406	# }
				407	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	408	"A String",
				409	],
				410	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	411	"properties": { # Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Cloud Dataproc API may be overwritten.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	412	"a_key": "A String",
				413	},
				414	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	415	"labels": { # Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values may be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
				416	"a_key": "A String",
				417	},
				418	"placement": { # Cloud Dataproc job config. # Required. Job information, including how, when, and where to run the job.
				419	"clusterName": "A String", # Required. The name of the cluster where the job will be submitted.
				420	"clusterUuid": "A String", # Output only. A cluster UUID generated by the Cloud Dataproc service when the job is submitted.
				421	},
				422	"reference": { # Encapsulates the full scoping used to reference a job. # Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a <code>job_id</code>.
				423	"projectId": "A String", # Required. The ID of the Google Cloud Platform project that the job belongs to.
				424	"jobId": "A String", # Optional. The job ID, which must be unique within the project.The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.If not specified by the caller, the job ID will be provided by the server.
				425	},
				426	"hadoopJob": { # A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). # Job is a Hadoop job.
				427	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
				428	"A String",
				429	],
				430	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
				431	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				432	"a_key": "A String",
				433	},
				434	},
				435	"jarFileUris": [ # Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
				436	"A String",
				437	],
				438	"fileUris": [ # Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
				439	"A String",
				440	],
				441	"mainClass": "A String", # The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
				442	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
				443	"A String",
				444	],
				445	"mainJarFileUri": "A String", # The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
				446	"properties": { # Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
				447	"a_key": "A String",
				448	},
				449	},
				450	"yarnApplications": [ # Output only. The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
				451	{ # A YARN application created by a job. Application information is a subset of <code>org.apache.hadoop.yarn.proto.YarnProtos.ApplicationReportProto</code>.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
				452	"progress": 3.14, # Required. The numerical progress of the application, from 1 to 100.
				453	"state": "A String", # Required. The application state.
				454	"name": "A String", # Required. The application name.
				455	"trackingUrl": "A String", # Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
				456	},
				457	],
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	458	"pigJob": { # A Cloud Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN. # Job is a Pig job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	459	"queryFileUri": "A String", # The HCFS URI of the script that contains the Pig queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	460	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	461	"a_key": "A String",
				462	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	463	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	464	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	465	"a_key": "A String",
				466	},
				467	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	468	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	469	"A String",
				470	],
				471	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	472	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	473	# "hiveJob": {
				474	# "queryList": {
				475	# "queries": [
				476	# "query1",
				477	# "query2",
				478	# "query3;query4",
				479	# ]
				480	# }
				481	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	482	"A String",
				483	],
				484	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	485	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				486	"properties": { # Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	487	"a_key": "A String",
				488	},
				489	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	490	"driverOutputResourceUri": "A String", # Output only. A URI pointing to the location of the stdout of the job's driver program.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	491	"pysparkJob": { # A Cloud Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/python-programming-guide.html) applications on YARN. # Job is a Pyspark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	492	"mainPythonFileUri": "A String", # Required. The HCFS URI of the main Python file to use as the driver. Must be a .py file.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	493	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	494	"A String",
				495	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	496	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	497	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				498	"a_key": "A String",
				499	},
				500	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	501	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	502	"A String",
				503	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	504	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	505	"A String",
				506	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	507	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	508	"A String",
				509	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	510	"pythonFileUris": [ # Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	511	"A String",
				512	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	513	"properties": { # Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	514	"a_key": "A String",
				515	},
				516	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	517	"driverControlFilesUri": "A String", # Output only. If present, the location of miscellaneous control files which may be used as part of job setup and handling. If not present, control files may be placed in the same location as driver_output_uri.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	518	"sparkJob": { # A Cloud Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN. # Job is a Spark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	519	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	520	"A String",
				521	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	522	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	523	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				524	"a_key": "A String",
				525	},
				526	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	527	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	528	"A String",
				529	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	530	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Spark drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	531	"A String",
				532	],
				533	"mainClass": "A String", # The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	534	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Spark drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	535	"A String",
				536	],
Jon Wayne Parrott	0a471d3	2016-05-19 10:54:38 -0700	[diff] [blame]	537	"mainJarFileUri": "A String", # The HCFS URI of the jar file that contains the main class.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	538	"properties": { # Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	539	"a_key": "A String",
				540	},
				541	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	542	"jobUuid": "A String", # Output only. A UUID that uniquely identifies a job within the project over time. This is in contrast to a user-settable reference.job_id that may be reused over time.
				543	"scheduling": { # Job scheduling options. # Optional. Job scheduling configuration.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	544	"maxFailuresPerHour": 42, # Optional. Maximum number of times per hour a driver may be restarted as a result of driver terminating with non-zero code before job is reported failed.A job may be reported as thrashing if driver exits with non-zero code 4 times within 10 minute window.Maximum value is 10.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	545	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	546	"statusHistory": [ # Output only. The previous job status.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	547	{ # Cloud Dataproc job status.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	548	"state": "A String", # Output only. A state message specifying the overall job state.
				549	"stateStartTime": "A String", # Output only. The time when this state was entered.
				550	"details": "A String", # Output only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				551	"substate": "A String", # Output only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	552	},
				553	],
				554	"hiveJob": { # A Cloud Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN. # Job is a Hive job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	555	"queryFileUri": "A String", # The HCFS URI of the script that contains Hive queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	556	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	557	"a_key": "A String",
				558	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	559	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	560	"A String",
				561	],
				562	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	563	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	564	# "hiveJob": {
				565	# "queryList": {
				566	# "queries": [
				567	# "query1",
				568	# "query2",
				569	# "query3;query4",
				570	# ]
				571	# }
				572	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	573	"A String",
				574	],
				575	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	576	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				577	"properties": { # Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	578	"a_key": "A String",
				579	},
				580	},
				581	}</pre>
				582	</div>
				583
				584	<div class="method">
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	585	<code class="details" id="getIamPolicy">getIamPolicy(resource, body=None, x__xgafv=None)</code>
				586	<pre>Gets the access control policy for a resource. Returns an empty policy if the resource exists and does not have a policy set.
				587
				588	Args:
				589	resource: string, REQUIRED: The resource for which the policy is being requested. See the operation documentation for the appropriate value for this field. (required)
				590	body: object, The request body.
				591	The object takes the form of:
				592
				593	{ # Request message for GetIamPolicy method.
				594	}
				595
				596	x__xgafv: string, V1 error format.
				597	Allowed values
				598	1 - v1 error format
				599	2 - v2 error format
				600
				601	Returns:
				602	An object of the form:
				603
				604	{ # Defines an Identity and Access Management (IAM) policy. It is used to specify access control policies for Cloud Platform resources.A Policy consists of a list of bindings. A binding binds a list of members to a role, where the members can be user accounts, Google groups, Google domains, and service accounts. A role is a named list of permissions defined by IAM.JSON Example
				605	# {
				606	# "bindings": [
				607	# {
				608	# "role": "roles/owner",
				609	# "members": [
				610	# "user:mike@example.com",
				611	# "group:admins@example.com",
				612	# "domain:google.com",
				613	# "serviceAccount:my-other-app@appspot.gserviceaccount.com"
				614	# ]
				615	# },
				616	# {
				617	# "role": "roles/viewer",
				618	# "members": ["user:sean@example.com"]
				619	# }
				620	# ]
				621	# }
				622	# YAML Example
				623	# bindings:
				624	# - members:
				625	# - user:mike@example.com
				626	# - group:admins@example.com
				627	# - domain:google.com
				628	# - serviceAccount:my-other-app@appspot.gserviceaccount.com
				629	# role: roles/owner
				630	# - members:
				631	# - user:sean@example.com
				632	# role: roles/viewer
				633	# For a description of IAM and its features, see the IAM developer's guide (https://cloud.google.com/iam/docs).
				634	"bindings": [ # Associates a list of members to a role. bindings with no members will result in an error.
				635	{ # Associates members with a role.
				636	"role": "A String", # Role that is assigned to members. For example, roles/viewer, roles/editor, or roles/owner.
				637	"members": [ # Specifies the identities requesting access for a Cloud Platform resource. members can have the following values:
				638	# allUsers: A special identifier that represents anyone who is on the internet; with or without a Google account.
				639	# allAuthenticatedUsers: A special identifier that represents anyone who is authenticated with a Google account or a service account.
				640	# user:{emailid}: An email address that represents a specific Google account. For example, alice@gmail.com .
				641	# serviceAccount:{emailid}: An email address that represents a service account. For example, my-other-app@appspot.gserviceaccount.com.
				642	# group:{emailid}: An email address that represents a Google group. For example, admins@example.com.
				643	# domain:{domain}: The G Suite domain (primary) that represents all the users of that domain. For example, google.com or example.com.
				644	"A String",
				645	],
				646	"condition": { # Represents an expression text. Example: # The condition that is associated with this binding. NOTE: An unsatisfied condition will not allow user access via current binding. Different bindings, including their conditions, are examined independently.
				647	# title: "User account presence"
				648	# description: "Determines whether the request has a user account"
				649	# expression: "size(request.user) > 0"
				650	"location": "A String", # An optional string indicating the location of the expression for error reporting, e.g. a file name and a position in the file.
				651	"expression": "A String", # Textual representation of an expression in Common Expression Language syntax.The application context of the containing message determines which well-known feature set of CEL is supported.
				652	"description": "A String", # An optional description of the expression. This is a longer text which describes the expression, e.g. when hovered over it in a UI.
				653	"title": "A String", # An optional title for the expression, i.e. a short string describing its purpose. This can be used e.g. in UIs which allow to enter the expression.
				654	},
				655	},
				656	],
				657	"etag": "A String", # etag is used for optimistic concurrency control as a way to help prevent simultaneous updates of a policy from overwriting each other. It is strongly suggested that systems make use of the etag in the read-modify-write cycle to perform policy updates in order to avoid race conditions: An etag is returned in the response to getIamPolicy, and systems are expected to put that etag in the request to setIamPolicy to ensure that their change will be applied to the same version of the policy.If no etag is provided in the call to setIamPolicy, then the existing policy is overwritten blindly.
				658	"version": 42, # Deprecated.
				659	}</pre>
				660	</div>
				661
				662	<div class="method">
				663	<code class="details" id="list">list(projectId, region, pageSize=None, x__xgafv=None, jobStateMatcher=None, pageToken=None, clusterName=None, filter=None)</code>
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	664	<pre>Lists regions/{region}/jobs in a project.
				665
				666	Args:
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	667	projectId: string, Required. The ID of the Google Cloud Platform project that the job belongs to. (required)
				668	region: string, Required. The Cloud Dataproc region in which to handle the request. (required)
				669	pageSize: integer, Optional. The number of results to return in each response.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	670	x__xgafv: string, V1 error format.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	671	Allowed values
				672	1 - v1 error format
				673	2 - v2 error format
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	674	jobStateMatcher: string, Optional. Specifies enumerated categories of jobs to list. (default = match ALL jobs).If filter is provided, jobStateMatcher will be ignored.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	675	pageToken: string, Optional. The page token, returned by a previous call, to request the next page of results.
				676	clusterName: string, Optional. If set, the returned jobs list includes only jobs that were submitted to the named cluster.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	677	filter: string, Optional. A filter constraining the jobs to list. Filters are case-sensitive and have the following syntax:field = value AND field = value ...where field is status.state or labels.[KEY], and [KEY] is a label key. value can be * to match all values. status.state can be either ACTIVE or NON_ACTIVE. Only the logical AND operator is supported; space-separated items are treated as having an implicit AND operator.Example filter:status.state = ACTIVE AND labels.env = staging AND labels.starred = *
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	678
				679	Returns:
				680	An object of the form:
				681
				682	{ # A list of jobs in a project.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	683	"nextPageToken": "A String", # Optional. This token is included in the response if there are more results to fetch. To fetch additional results, provide this value as the page_token in a subsequent <code>ListJobsRequest</code>.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	684	"jobs": [ # Output only. Jobs list.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	685	{ # A Cloud Dataproc job resource.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	686	"status": { # Cloud Dataproc job status. # Output only. The job status. Additional application-specific status information may be contained in the <code>type_job</code> and <code>yarn_applications</code> fields.
				687	"state": "A String", # Output only. A state message specifying the overall job state.
				688	"stateStartTime": "A String", # Output only. The time when this state was entered.
				689	"details": "A String", # Output only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				690	"substate": "A String", # Output only. Additional state information, which includes status reported by the agent.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	691	},
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	692	"sparkSqlJob": { # A Cloud Dataproc job for running Apache Spark SQL (http://spark.apache.org/sql/) queries. # Job is a SparkSql job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	693	"queryFileUri": "A String", # The HCFS URI of the script that contains SQL queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	694	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	695	"a_key": "A String",
				696	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	697	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	698	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	699	"a_key": "A String",
				700	},
				701	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	702	"jarFileUris": [ # Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	703	"A String",
				704	],
				705	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	706	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	707	# "hiveJob": {
				708	# "queryList": {
				709	# "queries": [
				710	# "query1",
				711	# "query2",
				712	# "query3;query4",
				713	# ]
				714	# }
				715	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	716	"A String",
				717	],
				718	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	719	"properties": { # Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Cloud Dataproc API may be overwritten.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	720	"a_key": "A String",
				721	},
				722	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	723	"labels": { # Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values may be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
				724	"a_key": "A String",
				725	},
				726	"placement": { # Cloud Dataproc job config. # Required. Job information, including how, when, and where to run the job.
				727	"clusterName": "A String", # Required. The name of the cluster where the job will be submitted.
				728	"clusterUuid": "A String", # Output only. A cluster UUID generated by the Cloud Dataproc service when the job is submitted.
				729	},
				730	"reference": { # Encapsulates the full scoping used to reference a job. # Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a <code>job_id</code>.
				731	"projectId": "A String", # Required. The ID of the Google Cloud Platform project that the job belongs to.
				732	"jobId": "A String", # Optional. The job ID, which must be unique within the project.The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.If not specified by the caller, the job ID will be provided by the server.
				733	},
				734	"hadoopJob": { # A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). # Job is a Hadoop job.
				735	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
				736	"A String",
				737	],
				738	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
				739	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				740	"a_key": "A String",
				741	},
				742	},
				743	"jarFileUris": [ # Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
				744	"A String",
				745	],
				746	"fileUris": [ # Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
				747	"A String",
				748	],
				749	"mainClass": "A String", # The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
				750	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
				751	"A String",
				752	],
				753	"mainJarFileUri": "A String", # The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
				754	"properties": { # Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
				755	"a_key": "A String",
				756	},
				757	},
				758	"yarnApplications": [ # Output only. The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
				759	{ # A YARN application created by a job. Application information is a subset of <code>org.apache.hadoop.yarn.proto.YarnProtos.ApplicationReportProto</code>.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
				760	"progress": 3.14, # Required. The numerical progress of the application, from 1 to 100.
				761	"state": "A String", # Required. The application state.
				762	"name": "A String", # Required. The application name.
				763	"trackingUrl": "A String", # Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
				764	},
				765	],
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	766	"pigJob": { # A Cloud Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN. # Job is a Pig job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	767	"queryFileUri": "A String", # The HCFS URI of the script that contains the Pig queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	768	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	769	"a_key": "A String",
				770	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	771	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	772	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	773	"a_key": "A String",
				774	},
				775	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	776	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	777	"A String",
				778	],
				779	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	780	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	781	# "hiveJob": {
				782	# "queryList": {
				783	# "queries": [
				784	# "query1",
				785	# "query2",
				786	# "query3;query4",
				787	# ]
				788	# }
				789	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	790	"A String",
				791	],
				792	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	793	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				794	"properties": { # Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	795	"a_key": "A String",
				796	},
				797	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	798	"driverOutputResourceUri": "A String", # Output only. A URI pointing to the location of the stdout of the job's driver program.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	799	"pysparkJob": { # A Cloud Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/python-programming-guide.html) applications on YARN. # Job is a Pyspark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	800	"mainPythonFileUri": "A String", # Required. The HCFS URI of the main Python file to use as the driver. Must be a .py file.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	801	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	802	"A String",
				803	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	804	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	805	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				806	"a_key": "A String",
				807	},
				808	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	809	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	810	"A String",
				811	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	812	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	813	"A String",
				814	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	815	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	816	"A String",
				817	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	818	"pythonFileUris": [ # Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	819	"A String",
				820	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	821	"properties": { # Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	822	"a_key": "A String",
				823	},
				824	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	825	"driverControlFilesUri": "A String", # Output only. If present, the location of miscellaneous control files which may be used as part of job setup and handling. If not present, control files may be placed in the same location as driver_output_uri.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	826	"sparkJob": { # A Cloud Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN. # Job is a Spark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	827	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	828	"A String",
				829	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	830	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	831	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				832	"a_key": "A String",
				833	},
				834	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	835	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	836	"A String",
				837	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	838	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Spark drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	839	"A String",
				840	],
				841	"mainClass": "A String", # The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	842	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Spark drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	843	"A String",
				844	],
Jon Wayne Parrott	0a471d3	2016-05-19 10:54:38 -0700	[diff] [blame]	845	"mainJarFileUri": "A String", # The HCFS URI of the jar file that contains the main class.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	846	"properties": { # Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	847	"a_key": "A String",
				848	},
				849	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	850	"jobUuid": "A String", # Output only. A UUID that uniquely identifies a job within the project over time. This is in contrast to a user-settable reference.job_id that may be reused over time.
				851	"scheduling": { # Job scheduling options. # Optional. Job scheduling configuration.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	852	"maxFailuresPerHour": 42, # Optional. Maximum number of times per hour a driver may be restarted as a result of driver terminating with non-zero code before job is reported failed.A job may be reported as thrashing if driver exits with non-zero code 4 times within 10 minute window.Maximum value is 10.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	853	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	854	"statusHistory": [ # Output only. The previous job status.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	855	{ # Cloud Dataproc job status.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	856	"state": "A String", # Output only. A state message specifying the overall job state.
				857	"stateStartTime": "A String", # Output only. The time when this state was entered.
				858	"details": "A String", # Output only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				859	"substate": "A String", # Output only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	860	},
				861	],
				862	"hiveJob": { # A Cloud Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN. # Job is a Hive job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	863	"queryFileUri": "A String", # The HCFS URI of the script that contains Hive queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	864	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	865	"a_key": "A String",
				866	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	867	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	868	"A String",
				869	],
				870	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	871	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	872	# "hiveJob": {
				873	# "queryList": {
				874	# "queries": [
				875	# "query1",
				876	# "query2",
				877	# "query3;query4",
				878	# ]
				879	# }
				880	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	881	"A String",
				882	],
				883	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	884	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				885	"properties": { # Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	886	"a_key": "A String",
				887	},
				888	},
				889	},
				890	],
				891	}</pre>
				892	</div>
				893
				894	<div class="method">
				895	<code class="details" id="list_next">list_next(previous_request, previous_response)</code>
				896	<pre>Retrieves the next page of results.
				897
				898	Args:
				899	previous_request: The request for the previous page. (required)
				900	previous_response: The response from the request for the previous page. (required)
				901
				902	Returns:
				903	A request object that you can call 'execute()' on to request the next
				904	page. Returns None if there are no more items in the collection.
				905	</pre>
				906	</div>
				907
				908	<div class="method">
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	909	<code class="details" id="patch">patch(projectId, region, jobId, body, updateMask=None, x__xgafv=None)</code>
				910	<pre>Updates a job in a project.
				911
				912	Args:
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	913	projectId: string, Required. The ID of the Google Cloud Platform project that the job belongs to. (required)
				914	region: string, Required. The Cloud Dataproc region in which to handle the request. (required)
				915	jobId: string, Required. The job ID. (required)
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	916	body: object, The request body. (required)
				917	The object takes the form of:
				918
				919	{ # A Cloud Dataproc job resource.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	920	"status": { # Cloud Dataproc job status. # Output only. The job status. Additional application-specific status information may be contained in the <code>type_job</code> and <code>yarn_applications</code> fields.
				921	"state": "A String", # Output only. A state message specifying the overall job state.
				922	"stateStartTime": "A String", # Output only. The time when this state was entered.
				923	"details": "A String", # Output only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				924	"substate": "A String", # Output only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	925	},
				926	"sparkSqlJob": { # A Cloud Dataproc job for running Apache Spark SQL (http://spark.apache.org/sql/) queries. # Job is a SparkSql job.
				927	"queryFileUri": "A String", # The HCFS URI of the script that contains SQL queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	928	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	929	"a_key": "A String",
				930	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	931	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	932	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				933	"a_key": "A String",
				934	},
				935	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	936	"jarFileUris": [ # Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	937	"A String",
				938	],
				939	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	940	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	941	# "hiveJob": {
				942	# "queryList": {
				943	# "queries": [
				944	# "query1",
				945	# "query2",
				946	# "query3;query4",
				947	# ]
				948	# }
				949	# }
				950	"A String",
				951	],
				952	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	953	"properties": { # Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Cloud Dataproc API may be overwritten.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	954	"a_key": "A String",
				955	},
				956	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	957	"labels": { # Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values may be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
				958	"a_key": "A String",
				959	},
				960	"placement": { # Cloud Dataproc job config. # Required. Job information, including how, when, and where to run the job.
				961	"clusterName": "A String", # Required. The name of the cluster where the job will be submitted.
				962	"clusterUuid": "A String", # Output only. A cluster UUID generated by the Cloud Dataproc service when the job is submitted.
				963	},
				964	"reference": { # Encapsulates the full scoping used to reference a job. # Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a <code>job_id</code>.
				965	"projectId": "A String", # Required. The ID of the Google Cloud Platform project that the job belongs to.
				966	"jobId": "A String", # Optional. The job ID, which must be unique within the project.The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.If not specified by the caller, the job ID will be provided by the server.
				967	},
				968	"hadoopJob": { # A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). # Job is a Hadoop job.
				969	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
				970	"A String",
				971	],
				972	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
				973	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				974	"a_key": "A String",
				975	},
				976	},
				977	"jarFileUris": [ # Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
				978	"A String",
				979	],
				980	"fileUris": [ # Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
				981	"A String",
				982	],
				983	"mainClass": "A String", # The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
				984	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
				985	"A String",
				986	],
				987	"mainJarFileUri": "A String", # The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
				988	"properties": { # Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
				989	"a_key": "A String",
				990	},
				991	},
				992	"yarnApplications": [ # Output only. The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
				993	{ # A YARN application created by a job. Application information is a subset of <code>org.apache.hadoop.yarn.proto.YarnProtos.ApplicationReportProto</code>.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
				994	"progress": 3.14, # Required. The numerical progress of the application, from 1 to 100.
				995	"state": "A String", # Required. The application state.
				996	"name": "A String", # Required. The application name.
				997	"trackingUrl": "A String", # Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
				998	},
				999	],
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1000	"pigJob": { # A Cloud Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN. # Job is a Pig job.
				1001	"queryFileUri": "A String", # The HCFS URI of the script that contains the Pig queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1002	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1003	"a_key": "A String",
				1004	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1005	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1006	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1007	"a_key": "A String",
				1008	},
				1009	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1010	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1011	"A String",
				1012	],
				1013	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1014	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1015	# "hiveJob": {
				1016	# "queryList": {
				1017	# "queries": [
				1018	# "query1",
				1019	# "query2",
				1020	# "query3;query4",
				1021	# ]
				1022	# }
				1023	# }
				1024	"A String",
				1025	],
				1026	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1027	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				1028	"properties": { # Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1029	"a_key": "A String",
				1030	},
				1031	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1032	"driverOutputResourceUri": "A String", # Output only. A URI pointing to the location of the stdout of the job's driver program.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1033	"pysparkJob": { # A Cloud Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/python-programming-guide.html) applications on YARN. # Job is a Pyspark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1034	"mainPythonFileUri": "A String", # Required. The HCFS URI of the main Python file to use as the driver. Must be a .py file.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1035	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1036	"A String",
				1037	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1038	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1039	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1040	"a_key": "A String",
				1041	},
				1042	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1043	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1044	"A String",
				1045	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1046	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1047	"A String",
				1048	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1049	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1050	"A String",
				1051	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1052	"pythonFileUris": [ # Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1053	"A String",
				1054	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1055	"properties": { # Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1056	"a_key": "A String",
				1057	},
				1058	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1059	"driverControlFilesUri": "A String", # Output only. If present, the location of miscellaneous control files which may be used as part of job setup and handling. If not present, control files may be placed in the same location as driver_output_uri.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1060	"sparkJob": { # A Cloud Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN. # Job is a Spark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1061	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1062	"A String",
				1063	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1064	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1065	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1066	"a_key": "A String",
				1067	},
				1068	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1069	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1070	"A String",
				1071	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1072	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Spark drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1073	"A String",
				1074	],
				1075	"mainClass": "A String", # The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1076	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Spark drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1077	"A String",
				1078	],
				1079	"mainJarFileUri": "A String", # The HCFS URI of the jar file that contains the main class.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1080	"properties": { # Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1081	"a_key": "A String",
				1082	},
				1083	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1084	"jobUuid": "A String", # Output only. A UUID that uniquely identifies a job within the project over time. This is in contrast to a user-settable reference.job_id that may be reused over time.
				1085	"scheduling": { # Job scheduling options. # Optional. Job scheduling configuration.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1086	"maxFailuresPerHour": 42, # Optional. Maximum number of times per hour a driver may be restarted as a result of driver terminating with non-zero code before job is reported failed.A job may be reported as thrashing if driver exits with non-zero code 4 times within 10 minute window.Maximum value is 10.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1087	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1088	"statusHistory": [ # Output only. The previous job status.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1089	{ # Cloud Dataproc job status.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1090	"state": "A String", # Output only. A state message specifying the overall job state.
				1091	"stateStartTime": "A String", # Output only. The time when this state was entered.
				1092	"details": "A String", # Output only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				1093	"substate": "A String", # Output only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1094	},
				1095	],
				1096	"hiveJob": { # A Cloud Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN. # Job is a Hive job.
				1097	"queryFileUri": "A String", # The HCFS URI of the script that contains Hive queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1098	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1099	"a_key": "A String",
				1100	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1101	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1102	"A String",
				1103	],
				1104	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1105	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1106	# "hiveJob": {
				1107	# "queryList": {
				1108	# "queries": [
				1109	# "query1",
				1110	# "query2",
				1111	# "query3;query4",
				1112	# ]
				1113	# }
				1114	# }
				1115	"A String",
				1116	],
				1117	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1118	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				1119	"properties": { # Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1120	"a_key": "A String",
				1121	},
				1122	},
				1123	}
				1124
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1125	updateMask: string, Required. Specifies the path, relative to <code>Job</code>, of the field to update. For example, to update the labels of a Job the <code>update_mask</code> parameter would be specified as <code>labels</code>, and the PATCH request body would specify the new value. <strong>Note:</strong> Currently, <code>labels</code> is the only field that can be updated.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1126	x__xgafv: string, V1 error format.
				1127	Allowed values
				1128	1 - v1 error format
				1129	2 - v2 error format
				1130
				1131	Returns:
				1132	An object of the form:
				1133
				1134	{ # A Cloud Dataproc job resource.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1135	"status": { # Cloud Dataproc job status. # Output only. The job status. Additional application-specific status information may be contained in the <code>type_job</code> and <code>yarn_applications</code> fields.
				1136	"state": "A String", # Output only. A state message specifying the overall job state.
				1137	"stateStartTime": "A String", # Output only. The time when this state was entered.
				1138	"details": "A String", # Output only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				1139	"substate": "A String", # Output only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1140	},
				1141	"sparkSqlJob": { # A Cloud Dataproc job for running Apache Spark SQL (http://spark.apache.org/sql/) queries. # Job is a SparkSql job.
				1142	"queryFileUri": "A String", # The HCFS URI of the script that contains SQL queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1143	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1144	"a_key": "A String",
				1145	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1146	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1147	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1148	"a_key": "A String",
				1149	},
				1150	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1151	"jarFileUris": [ # Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1152	"A String",
				1153	],
				1154	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1155	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1156	# "hiveJob": {
				1157	# "queryList": {
				1158	# "queries": [
				1159	# "query1",
				1160	# "query2",
				1161	# "query3;query4",
				1162	# ]
				1163	# }
				1164	# }
				1165	"A String",
				1166	],
				1167	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1168	"properties": { # Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Cloud Dataproc API may be overwritten.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1169	"a_key": "A String",
				1170	},
				1171	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1172	"labels": { # Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values may be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
				1173	"a_key": "A String",
				1174	},
				1175	"placement": { # Cloud Dataproc job config. # Required. Job information, including how, when, and where to run the job.
				1176	"clusterName": "A String", # Required. The name of the cluster where the job will be submitted.
				1177	"clusterUuid": "A String", # Output only. A cluster UUID generated by the Cloud Dataproc service when the job is submitted.
				1178	},
				1179	"reference": { # Encapsulates the full scoping used to reference a job. # Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a <code>job_id</code>.
				1180	"projectId": "A String", # Required. The ID of the Google Cloud Platform project that the job belongs to.
				1181	"jobId": "A String", # Optional. The job ID, which must be unique within the project.The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.If not specified by the caller, the job ID will be provided by the server.
				1182	},
				1183	"hadoopJob": { # A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). # Job is a Hadoop job.
				1184	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
				1185	"A String",
				1186	],
				1187	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
				1188	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1189	"a_key": "A String",
				1190	},
				1191	},
				1192	"jarFileUris": [ # Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
				1193	"A String",
				1194	],
				1195	"fileUris": [ # Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
				1196	"A String",
				1197	],
				1198	"mainClass": "A String", # The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
				1199	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
				1200	"A String",
				1201	],
				1202	"mainJarFileUri": "A String", # The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
				1203	"properties": { # Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
				1204	"a_key": "A String",
				1205	},
				1206	},
				1207	"yarnApplications": [ # Output only. The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
				1208	{ # A YARN application created by a job. Application information is a subset of <code>org.apache.hadoop.yarn.proto.YarnProtos.ApplicationReportProto</code>.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
				1209	"progress": 3.14, # Required. The numerical progress of the application, from 1 to 100.
				1210	"state": "A String", # Required. The application state.
				1211	"name": "A String", # Required. The application name.
				1212	"trackingUrl": "A String", # Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
				1213	},
				1214	],
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1215	"pigJob": { # A Cloud Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN. # Job is a Pig job.
				1216	"queryFileUri": "A String", # The HCFS URI of the script that contains the Pig queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1217	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1218	"a_key": "A String",
				1219	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1220	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1221	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1222	"a_key": "A String",
				1223	},
				1224	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1225	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1226	"A String",
				1227	],
				1228	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1229	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1230	# "hiveJob": {
				1231	# "queryList": {
				1232	# "queries": [
				1233	# "query1",
				1234	# "query2",
				1235	# "query3;query4",
				1236	# ]
				1237	# }
				1238	# }
				1239	"A String",
				1240	],
				1241	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1242	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				1243	"properties": { # Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1244	"a_key": "A String",
				1245	},
				1246	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1247	"driverOutputResourceUri": "A String", # Output only. A URI pointing to the location of the stdout of the job's driver program.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1248	"pysparkJob": { # A Cloud Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/python-programming-guide.html) applications on YARN. # Job is a Pyspark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1249	"mainPythonFileUri": "A String", # Required. The HCFS URI of the main Python file to use as the driver. Must be a .py file.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1250	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1251	"A String",
				1252	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1253	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1254	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1255	"a_key": "A String",
				1256	},
				1257	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1258	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1259	"A String",
				1260	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1261	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1262	"A String",
				1263	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1264	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1265	"A String",
				1266	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1267	"pythonFileUris": [ # Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1268	"A String",
				1269	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1270	"properties": { # Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1271	"a_key": "A String",
				1272	},
				1273	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1274	"driverControlFilesUri": "A String", # Output only. If present, the location of miscellaneous control files which may be used as part of job setup and handling. If not present, control files may be placed in the same location as driver_output_uri.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1275	"sparkJob": { # A Cloud Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN. # Job is a Spark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1276	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1277	"A String",
				1278	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1279	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1280	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1281	"a_key": "A String",
				1282	},
				1283	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1284	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1285	"A String",
				1286	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1287	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Spark drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1288	"A String",
				1289	],
				1290	"mainClass": "A String", # The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1291	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Spark drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1292	"A String",
				1293	],
				1294	"mainJarFileUri": "A String", # The HCFS URI of the jar file that contains the main class.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1295	"properties": { # Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1296	"a_key": "A String",
				1297	},
				1298	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1299	"jobUuid": "A String", # Output only. A UUID that uniquely identifies a job within the project over time. This is in contrast to a user-settable reference.job_id that may be reused over time.
				1300	"scheduling": { # Job scheduling options. # Optional. Job scheduling configuration.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1301	"maxFailuresPerHour": 42, # Optional. Maximum number of times per hour a driver may be restarted as a result of driver terminating with non-zero code before job is reported failed.A job may be reported as thrashing if driver exits with non-zero code 4 times within 10 minute window.Maximum value is 10.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1302	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1303	"statusHistory": [ # Output only. The previous job status.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1304	{ # Cloud Dataproc job status.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1305	"state": "A String", # Output only. A state message specifying the overall job state.
				1306	"stateStartTime": "A String", # Output only. The time when this state was entered.
				1307	"details": "A String", # Output only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				1308	"substate": "A String", # Output only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1309	},
				1310	],
				1311	"hiveJob": { # A Cloud Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN. # Job is a Hive job.
				1312	"queryFileUri": "A String", # The HCFS URI of the script that contains Hive queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1313	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1314	"a_key": "A String",
				1315	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1316	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1317	"A String",
				1318	],
				1319	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1320	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1321	# "hiveJob": {
				1322	# "queryList": {
				1323	# "queries": [
				1324	# "query1",
				1325	# "query2",
				1326	# "query3;query4",
				1327	# ]
				1328	# }
				1329	# }
				1330	"A String",
				1331	],
				1332	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1333	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				1334	"properties": { # Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1335	"a_key": "A String",
				1336	},
				1337	},
				1338	}</pre>
				1339	</div>
				1340
				1341	<div class="method">
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1342	<code class="details" id="setIamPolicy">setIamPolicy(resource, body, x__xgafv=None)</code>
				1343	<pre>Sets the access control policy on the specified resource. Replaces any existing policy.
				1344
				1345	Args:
				1346	resource: string, REQUIRED: The resource for which the policy is being specified. See the operation documentation for the appropriate value for this field. (required)
				1347	body: object, The request body. (required)
				1348	The object takes the form of:
				1349
				1350	{ # Request message for SetIamPolicy method.
				1351	"policy": { # Defines an Identity and Access Management (IAM) policy. It is used to specify access control policies for Cloud Platform resources.A Policy consists of a list of bindings. A binding binds a list of members to a role, where the members can be user accounts, Google groups, Google domains, and service accounts. A role is a named list of permissions defined by IAM.JSON Example # REQUIRED: The complete policy to be applied to the resource. The size of the policy is limited to a few 10s of KB. An empty policy is a valid policy but certain Cloud Platform services (such as Projects) might reject them.
				1352	# {
				1353	# "bindings": [
				1354	# {
				1355	# "role": "roles/owner",
				1356	# "members": [
				1357	# "user:mike@example.com",
				1358	# "group:admins@example.com",
				1359	# "domain:google.com",
				1360	# "serviceAccount:my-other-app@appspot.gserviceaccount.com"
				1361	# ]
				1362	# },
				1363	# {
				1364	# "role": "roles/viewer",
				1365	# "members": ["user:sean@example.com"]
				1366	# }
				1367	# ]
				1368	# }
				1369	# YAML Example
				1370	# bindings:
				1371	# - members:
				1372	# - user:mike@example.com
				1373	# - group:admins@example.com
				1374	# - domain:google.com
				1375	# - serviceAccount:my-other-app@appspot.gserviceaccount.com
				1376	# role: roles/owner
				1377	# - members:
				1378	# - user:sean@example.com
				1379	# role: roles/viewer
				1380	# For a description of IAM and its features, see the IAM developer's guide (https://cloud.google.com/iam/docs).
				1381	"bindings": [ # Associates a list of members to a role. bindings with no members will result in an error.
				1382	{ # Associates members with a role.
				1383	"role": "A String", # Role that is assigned to members. For example, roles/viewer, roles/editor, or roles/owner.
				1384	"members": [ # Specifies the identities requesting access for a Cloud Platform resource. members can have the following values:
				1385	# allUsers: A special identifier that represents anyone who is on the internet; with or without a Google account.
				1386	# allAuthenticatedUsers: A special identifier that represents anyone who is authenticated with a Google account or a service account.
				1387	# user:{emailid}: An email address that represents a specific Google account. For example, alice@gmail.com .
				1388	# serviceAccount:{emailid}: An email address that represents a service account. For example, my-other-app@appspot.gserviceaccount.com.
				1389	# group:{emailid}: An email address that represents a Google group. For example, admins@example.com.
				1390	# domain:{domain}: The G Suite domain (primary) that represents all the users of that domain. For example, google.com or example.com.
				1391	"A String",
				1392	],
				1393	"condition": { # Represents an expression text. Example: # The condition that is associated with this binding. NOTE: An unsatisfied condition will not allow user access via current binding. Different bindings, including their conditions, are examined independently.
				1394	# title: "User account presence"
				1395	# description: "Determines whether the request has a user account"
				1396	# expression: "size(request.user) > 0"
				1397	"location": "A String", # An optional string indicating the location of the expression for error reporting, e.g. a file name and a position in the file.
				1398	"expression": "A String", # Textual representation of an expression in Common Expression Language syntax.The application context of the containing message determines which well-known feature set of CEL is supported.
				1399	"description": "A String", # An optional description of the expression. This is a longer text which describes the expression, e.g. when hovered over it in a UI.
				1400	"title": "A String", # An optional title for the expression, i.e. a short string describing its purpose. This can be used e.g. in UIs which allow to enter the expression.
				1401	},
				1402	},
				1403	],
				1404	"etag": "A String", # etag is used for optimistic concurrency control as a way to help prevent simultaneous updates of a policy from overwriting each other. It is strongly suggested that systems make use of the etag in the read-modify-write cycle to perform policy updates in order to avoid race conditions: An etag is returned in the response to getIamPolicy, and systems are expected to put that etag in the request to setIamPolicy to ensure that their change will be applied to the same version of the policy.If no etag is provided in the call to setIamPolicy, then the existing policy is overwritten blindly.
				1405	"version": 42, # Deprecated.
				1406	},
				1407	}
				1408
				1409	x__xgafv: string, V1 error format.
				1410	Allowed values
				1411	1 - v1 error format
				1412	2 - v2 error format
				1413
				1414	Returns:
				1415	An object of the form:
				1416
				1417	{ # Defines an Identity and Access Management (IAM) policy. It is used to specify access control policies for Cloud Platform resources.A Policy consists of a list of bindings. A binding binds a list of members to a role, where the members can be user accounts, Google groups, Google domains, and service accounts. A role is a named list of permissions defined by IAM.JSON Example
				1418	# {
				1419	# "bindings": [
				1420	# {
				1421	# "role": "roles/owner",
				1422	# "members": [
				1423	# "user:mike@example.com",
				1424	# "group:admins@example.com",
				1425	# "domain:google.com",
				1426	# "serviceAccount:my-other-app@appspot.gserviceaccount.com"
				1427	# ]
				1428	# },
				1429	# {
				1430	# "role": "roles/viewer",
				1431	# "members": ["user:sean@example.com"]
				1432	# }
				1433	# ]
				1434	# }
				1435	# YAML Example
				1436	# bindings:
				1437	# - members:
				1438	# - user:mike@example.com
				1439	# - group:admins@example.com
				1440	# - domain:google.com
				1441	# - serviceAccount:my-other-app@appspot.gserviceaccount.com
				1442	# role: roles/owner
				1443	# - members:
				1444	# - user:sean@example.com
				1445	# role: roles/viewer
				1446	# For a description of IAM and its features, see the IAM developer's guide (https://cloud.google.com/iam/docs).
				1447	"bindings": [ # Associates a list of members to a role. bindings with no members will result in an error.
				1448	{ # Associates members with a role.
				1449	"role": "A String", # Role that is assigned to members. For example, roles/viewer, roles/editor, or roles/owner.
				1450	"members": [ # Specifies the identities requesting access for a Cloud Platform resource. members can have the following values:
				1451	# allUsers: A special identifier that represents anyone who is on the internet; with or without a Google account.
				1452	# allAuthenticatedUsers: A special identifier that represents anyone who is authenticated with a Google account or a service account.
				1453	# user:{emailid}: An email address that represents a specific Google account. For example, alice@gmail.com .
				1454	# serviceAccount:{emailid}: An email address that represents a service account. For example, my-other-app@appspot.gserviceaccount.com.
				1455	# group:{emailid}: An email address that represents a Google group. For example, admins@example.com.
				1456	# domain:{domain}: The G Suite domain (primary) that represents all the users of that domain. For example, google.com or example.com.
				1457	"A String",
				1458	],
				1459	"condition": { # Represents an expression text. Example: # The condition that is associated with this binding. NOTE: An unsatisfied condition will not allow user access via current binding. Different bindings, including their conditions, are examined independently.
				1460	# title: "User account presence"
				1461	# description: "Determines whether the request has a user account"
				1462	# expression: "size(request.user) > 0"
				1463	"location": "A String", # An optional string indicating the location of the expression for error reporting, e.g. a file name and a position in the file.
				1464	"expression": "A String", # Textual representation of an expression in Common Expression Language syntax.The application context of the containing message determines which well-known feature set of CEL is supported.
				1465	"description": "A String", # An optional description of the expression. This is a longer text which describes the expression, e.g. when hovered over it in a UI.
				1466	"title": "A String", # An optional title for the expression, i.e. a short string describing its purpose. This can be used e.g. in UIs which allow to enter the expression.
				1467	},
				1468	},
				1469	],
				1470	"etag": "A String", # etag is used for optimistic concurrency control as a way to help prevent simultaneous updates of a policy from overwriting each other. It is strongly suggested that systems make use of the etag in the read-modify-write cycle to perform policy updates in order to avoid race conditions: An etag is returned in the response to getIamPolicy, and systems are expected to put that etag in the request to setIamPolicy to ensure that their change will be applied to the same version of the policy.If no etag is provided in the call to setIamPolicy, then the existing policy is overwritten blindly.
				1471	"version": 42, # Deprecated.
				1472	}</pre>
				1473	</div>
				1474
				1475	<div class="method">
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1476	<code class="details" id="submit">submit(projectId, region, body, x__xgafv=None)</code>
				1477	<pre>Submits a job to a cluster.
				1478
				1479	Args:
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1480	projectId: string, Required. The ID of the Google Cloud Platform project that the job belongs to. (required)
				1481	region: string, Required. The Cloud Dataproc region in which to handle the request. (required)
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1482	body: object, The request body. (required)
				1483	The object takes the form of:
				1484
				1485	{ # A request to submit a job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1486	"job": { # A Cloud Dataproc job resource. # Required. The job resource.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1487	"status": { # Cloud Dataproc job status. # Output only. The job status. Additional application-specific status information may be contained in the <code>type_job</code> and <code>yarn_applications</code> fields.
				1488	"state": "A String", # Output only. A state message specifying the overall job state.
				1489	"stateStartTime": "A String", # Output only. The time when this state was entered.
				1490	"details": "A String", # Output only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				1491	"substate": "A String", # Output only. Additional state information, which includes status reported by the agent.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1492	},
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1493	"sparkSqlJob": { # A Cloud Dataproc job for running Apache Spark SQL (http://spark.apache.org/sql/) queries. # Job is a SparkSql job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1494	"queryFileUri": "A String", # The HCFS URI of the script that contains SQL queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1495	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1496	"a_key": "A String",
				1497	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1498	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1499	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1500	"a_key": "A String",
				1501	},
				1502	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1503	"jarFileUris": [ # Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1504	"A String",
				1505	],
				1506	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1507	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1508	# "hiveJob": {
				1509	# "queryList": {
				1510	# "queries": [
				1511	# "query1",
				1512	# "query2",
				1513	# "query3;query4",
				1514	# ]
				1515	# }
				1516	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1517	"A String",
				1518	],
				1519	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1520	"properties": { # Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Cloud Dataproc API may be overwritten.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1521	"a_key": "A String",
				1522	},
				1523	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1524	"labels": { # Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values may be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
				1525	"a_key": "A String",
				1526	},
				1527	"placement": { # Cloud Dataproc job config. # Required. Job information, including how, when, and where to run the job.
				1528	"clusterName": "A String", # Required. The name of the cluster where the job will be submitted.
				1529	"clusterUuid": "A String", # Output only. A cluster UUID generated by the Cloud Dataproc service when the job is submitted.
				1530	},
				1531	"reference": { # Encapsulates the full scoping used to reference a job. # Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a <code>job_id</code>.
				1532	"projectId": "A String", # Required. The ID of the Google Cloud Platform project that the job belongs to.
				1533	"jobId": "A String", # Optional. The job ID, which must be unique within the project.The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.If not specified by the caller, the job ID will be provided by the server.
				1534	},
				1535	"hadoopJob": { # A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). # Job is a Hadoop job.
				1536	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
				1537	"A String",
				1538	],
				1539	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
				1540	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1541	"a_key": "A String",
				1542	},
				1543	},
				1544	"jarFileUris": [ # Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
				1545	"A String",
				1546	],
				1547	"fileUris": [ # Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
				1548	"A String",
				1549	],
				1550	"mainClass": "A String", # The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
				1551	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
				1552	"A String",
				1553	],
				1554	"mainJarFileUri": "A String", # The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
				1555	"properties": { # Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
				1556	"a_key": "A String",
				1557	},
				1558	},
				1559	"yarnApplications": [ # Output only. The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
				1560	{ # A YARN application created by a job. Application information is a subset of <code>org.apache.hadoop.yarn.proto.YarnProtos.ApplicationReportProto</code>.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
				1561	"progress": 3.14, # Required. The numerical progress of the application, from 1 to 100.
				1562	"state": "A String", # Required. The application state.
				1563	"name": "A String", # Required. The application name.
				1564	"trackingUrl": "A String", # Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
				1565	},
				1566	],
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1567	"pigJob": { # A Cloud Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN. # Job is a Pig job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1568	"queryFileUri": "A String", # The HCFS URI of the script that contains the Pig queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1569	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1570	"a_key": "A String",
				1571	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1572	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1573	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1574	"a_key": "A String",
				1575	},
				1576	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1577	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1578	"A String",
				1579	],
				1580	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1581	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1582	# "hiveJob": {
				1583	# "queryList": {
				1584	# "queries": [
				1585	# "query1",
				1586	# "query2",
				1587	# "query3;query4",
				1588	# ]
				1589	# }
				1590	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1591	"A String",
				1592	],
				1593	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1594	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				1595	"properties": { # Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1596	"a_key": "A String",
				1597	},
				1598	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1599	"driverOutputResourceUri": "A String", # Output only. A URI pointing to the location of the stdout of the job's driver program.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1600	"pysparkJob": { # A Cloud Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/python-programming-guide.html) applications on YARN. # Job is a Pyspark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1601	"mainPythonFileUri": "A String", # Required. The HCFS URI of the main Python file to use as the driver. Must be a .py file.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1602	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1603	"A String",
				1604	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1605	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1606	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1607	"a_key": "A String",
				1608	},
				1609	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1610	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1611	"A String",
				1612	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1613	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1614	"A String",
				1615	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1616	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1617	"A String",
				1618	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1619	"pythonFileUris": [ # Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1620	"A String",
				1621	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1622	"properties": { # Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1623	"a_key": "A String",
				1624	},
				1625	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1626	"driverControlFilesUri": "A String", # Output only. If present, the location of miscellaneous control files which may be used as part of job setup and handling. If not present, control files may be placed in the same location as driver_output_uri.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1627	"sparkJob": { # A Cloud Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN. # Job is a Spark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1628	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1629	"A String",
				1630	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1631	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1632	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1633	"a_key": "A String",
				1634	},
				1635	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1636	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1637	"A String",
				1638	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1639	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Spark drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1640	"A String",
				1641	],
				1642	"mainClass": "A String", # The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1643	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Spark drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1644	"A String",
				1645	],
Jon Wayne Parrott	0a471d3	2016-05-19 10:54:38 -0700	[diff] [blame]	1646	"mainJarFileUri": "A String", # The HCFS URI of the jar file that contains the main class.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1647	"properties": { # Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1648	"a_key": "A String",
				1649	},
				1650	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1651	"jobUuid": "A String", # Output only. A UUID that uniquely identifies a job within the project over time. This is in contrast to a user-settable reference.job_id that may be reused over time.
				1652	"scheduling": { # Job scheduling options. # Optional. Job scheduling configuration.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1653	"maxFailuresPerHour": 42, # Optional. Maximum number of times per hour a driver may be restarted as a result of driver terminating with non-zero code before job is reported failed.A job may be reported as thrashing if driver exits with non-zero code 4 times within 10 minute window.Maximum value is 10.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1654	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1655	"statusHistory": [ # Output only. The previous job status.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1656	{ # Cloud Dataproc job status.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1657	"state": "A String", # Output only. A state message specifying the overall job state.
				1658	"stateStartTime": "A String", # Output only. The time when this state was entered.
				1659	"details": "A String", # Output only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				1660	"substate": "A String", # Output only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1661	},
				1662	],
				1663	"hiveJob": { # A Cloud Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN. # Job is a Hive job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1664	"queryFileUri": "A String", # The HCFS URI of the script that contains Hive queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1665	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1666	"a_key": "A String",
				1667	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1668	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1669	"A String",
				1670	],
				1671	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1672	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1673	# "hiveJob": {
				1674	# "queryList": {
				1675	# "queries": [
				1676	# "query1",
				1677	# "query2",
				1678	# "query3;query4",
				1679	# ]
				1680	# }
				1681	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1682	"A String",
				1683	],
				1684	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1685	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				1686	"properties": { # Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1687	"a_key": "A String",
				1688	},
				1689	},
				1690	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1691	"requestId": "A String", # Optional. A unique id used to identify the request. If the server receives two SubmitJobRequest requests with the same id, then the second request will be ignored and the first Job created and stored in the backend is returned.It is recommended to always set this value to a UUID (https://en.wikipedia.org/wiki/Universally_unique_identifier).The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). The maximum length is 40 characters.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1692	}
				1693
				1694	x__xgafv: string, V1 error format.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1695	Allowed values
				1696	1 - v1 error format
				1697	2 - v2 error format
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1698
				1699	Returns:
				1700	An object of the form:
				1701
				1702	{ # A Cloud Dataproc job resource.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1703	"status": { # Cloud Dataproc job status. # Output only. The job status. Additional application-specific status information may be contained in the <code>type_job</code> and <code>yarn_applications</code> fields.
				1704	"state": "A String", # Output only. A state message specifying the overall job state.
				1705	"stateStartTime": "A String", # Output only. The time when this state was entered.
				1706	"details": "A String", # Output only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				1707	"substate": "A String", # Output only. Additional state information, which includes status reported by the agent.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1708	},
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1709	"sparkSqlJob": { # A Cloud Dataproc job for running Apache Spark SQL (http://spark.apache.org/sql/) queries. # Job is a SparkSql job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1710	"queryFileUri": "A String", # The HCFS URI of the script that contains SQL queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1711	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1712	"a_key": "A String",
				1713	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1714	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1715	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1716	"a_key": "A String",
				1717	},
				1718	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1719	"jarFileUris": [ # Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1720	"A String",
				1721	],
				1722	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1723	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1724	# "hiveJob": {
				1725	# "queryList": {
				1726	# "queries": [
				1727	# "query1",
				1728	# "query2",
				1729	# "query3;query4",
				1730	# ]
				1731	# }
				1732	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1733	"A String",
				1734	],
				1735	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1736	"properties": { # Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Cloud Dataproc API may be overwritten.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1737	"a_key": "A String",
				1738	},
				1739	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1740	"labels": { # Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values may be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
				1741	"a_key": "A String",
				1742	},
				1743	"placement": { # Cloud Dataproc job config. # Required. Job information, including how, when, and where to run the job.
				1744	"clusterName": "A String", # Required. The name of the cluster where the job will be submitted.
				1745	"clusterUuid": "A String", # Output only. A cluster UUID generated by the Cloud Dataproc service when the job is submitted.
				1746	},
				1747	"reference": { # Encapsulates the full scoping used to reference a job. # Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a <code>job_id</code>.
				1748	"projectId": "A String", # Required. The ID of the Google Cloud Platform project that the job belongs to.
				1749	"jobId": "A String", # Optional. The job ID, which must be unique within the project.The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.If not specified by the caller, the job ID will be provided by the server.
				1750	},
				1751	"hadoopJob": { # A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). # Job is a Hadoop job.
				1752	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
				1753	"A String",
				1754	],
				1755	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
				1756	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1757	"a_key": "A String",
				1758	},
				1759	},
				1760	"jarFileUris": [ # Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
				1761	"A String",
				1762	],
				1763	"fileUris": [ # Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
				1764	"A String",
				1765	],
				1766	"mainClass": "A String", # The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
				1767	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
				1768	"A String",
				1769	],
				1770	"mainJarFileUri": "A String", # The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
				1771	"properties": { # Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
				1772	"a_key": "A String",
				1773	},
				1774	},
				1775	"yarnApplications": [ # Output only. The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
				1776	{ # A YARN application created by a job. Application information is a subset of <code>org.apache.hadoop.yarn.proto.YarnProtos.ApplicationReportProto</code>.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
				1777	"progress": 3.14, # Required. The numerical progress of the application, from 1 to 100.
				1778	"state": "A String", # Required. The application state.
				1779	"name": "A String", # Required. The application name.
				1780	"trackingUrl": "A String", # Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
				1781	},
				1782	],
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1783	"pigJob": { # A Cloud Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN. # Job is a Pig job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1784	"queryFileUri": "A String", # The HCFS URI of the script that contains the Pig queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1785	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1786	"a_key": "A String",
				1787	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1788	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1789	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1790	"a_key": "A String",
				1791	},
				1792	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1793	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1794	"A String",
				1795	],
				1796	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1797	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1798	# "hiveJob": {
				1799	# "queryList": {
				1800	# "queries": [
				1801	# "query1",
				1802	# "query2",
				1803	# "query3;query4",
				1804	# ]
				1805	# }
				1806	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1807	"A String",
				1808	],
				1809	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1810	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				1811	"properties": { # Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1812	"a_key": "A String",
				1813	},
				1814	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1815	"driverOutputResourceUri": "A String", # Output only. A URI pointing to the location of the stdout of the job's driver program.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1816	"pysparkJob": { # A Cloud Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/python-programming-guide.html) applications on YARN. # Job is a Pyspark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1817	"mainPythonFileUri": "A String", # Required. The HCFS URI of the main Python file to use as the driver. Must be a .py file.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1818	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1819	"A String",
				1820	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1821	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1822	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1823	"a_key": "A String",
				1824	},
				1825	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1826	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Thomas Coffee	2f24537	2017-03-27 10:39:26 -0700	[diff] [blame]	1827	"A String",
				1828	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1829	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1830	"A String",
				1831	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1832	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1833	"A String",
				1834	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1835	"pythonFileUris": [ # Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1836	"A String",
				1837	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1838	"properties": { # Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1839	"a_key": "A String",
				1840	},
				1841	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1842	"driverControlFilesUri": "A String", # Output only. If present, the location of miscellaneous control files which may be used as part of job setup and handling. If not present, control files may be placed in the same location as driver_output_uri.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1843	"sparkJob": { # A Cloud Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN. # Job is a Spark job.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1844	"args": [ # Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1845	"A String",
				1846	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1847	"loggingConfig": { # The runtime logging config of the job. # Optional. The runtime log config for job execution.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1848	"driverLogLevels": { # The per-package log levels for the driver. This may include "root" package name to configure rootLogger. Examples: 'com.google = FATAL', 'root = INFO', 'org.apache = DEBUG'
				1849	"a_key": "A String",
				1850	},
				1851	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1852	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1853	"A String",
				1854	],
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1855	"fileUris": [ # Optional. HCFS URIs of files to be copied to the working directory of Spark drivers and distributed tasks. Useful for naively parallel tasks.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1856	"A String",
				1857	],
				1858	"mainClass": "A String", # The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1859	"archiveUris": [ # Optional. HCFS URIs of archives to be extracted in the working directory of Spark drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1860	"A String",
				1861	],
Jon Wayne Parrott	0a471d3	2016-05-19 10:54:38 -0700	[diff] [blame]	1862	"mainJarFileUri": "A String", # The HCFS URI of the jar file that contains the main class.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1863	"properties": { # Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1864	"a_key": "A String",
				1865	},
				1866	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1867	"jobUuid": "A String", # Output only. A UUID that uniquely identifies a job within the project over time. This is in contrast to a user-settable reference.job_id that may be reused over time.
				1868	"scheduling": { # Job scheduling options. # Optional. Job scheduling configuration.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1869	"maxFailuresPerHour": 42, # Optional. Maximum number of times per hour a driver may be restarted as a result of driver terminating with non-zero code before job is reported failed.A job may be reported as thrashing if driver exits with non-zero code 4 times within 10 minute window.Maximum value is 10.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1870	},
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1871	"statusHistory": [ # Output only. The previous job status.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1872	{ # Cloud Dataproc job status.
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1873	"state": "A String", # Output only. A state message specifying the overall job state.
				1874	"stateStartTime": "A String", # Output only. The time when this state was entered.
				1875	"details": "A String", # Output only. Optional job state details, such as an error description if the state is <code>ERROR</code>.
				1876	"substate": "A String", # Output only. Additional state information, which includes status reported by the agent.
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1877	},
				1878	],
				1879	"hiveJob": { # A Cloud Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN. # Job is a Hive job.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1880	"queryFileUri": "A String", # The HCFS URI of the script that contains Hive queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1881	"scriptVariables": { # Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1882	"a_key": "A String",
				1883	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1884	"jarFileUris": [ # Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1885	"A String",
				1886	],
				1887	"queryList": { # A list of queries to run on a cluster. # A list of queries.
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1888	"queries": [ # Required. The queries to execute. You do not need to terminate a query with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of an Cloud Dataproc API snippet that uses a QueryList to specify a HiveJob:
Sai Cheemalapati	c30d2b5	2017-03-13 12:12:03 -0400	[diff] [blame]	1889	# "hiveJob": {
				1890	# "queryList": {
				1891	# "queries": [
				1892	# "query1",
				1893	# "query2",
				1894	# "query3;query4",
				1895	# ]
				1896	# }
				1897	# }
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1898	"A String",
				1899	],
				1900	},
Sai Cheemalapati	4ba8c23	2017-06-06 18:46:08 -0400	[diff] [blame]	1901	"continueOnFailure": True or False, # Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
				1902	"properties": { # Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1903	"a_key": "A String",
				1904	},
				1905	},
				1906	}</pre>
				1907	</div>
				1908
Bu Sun Kim	715bd7f	2019-06-14 16:50:42 -0700	[diff] [blame]	1909	<div class="method">
				1910	<code class="details" id="testIamPermissions">testIamPermissions(resource, body, x__xgafv=None)</code>
				1911	<pre>Returns permissions that a caller has on the specified resource. If the resource does not exist, this will return an empty set of permissions, not a NOT_FOUND error.Note: This operation is designed to be used for building permission-aware UIs and command-line tools, not for authorization checking. This operation may "fail open" without warning.
				1912
				1913	Args:
				1914	resource: string, REQUIRED: The resource for which the policy detail is being requested. See the operation documentation for the appropriate value for this field. (required)
				1915	body: object, The request body. (required)
				1916	The object takes the form of:
				1917
				1918	{ # Request message for TestIamPermissions method.
				1919	"permissions": [ # The set of permissions to check for the resource. Permissions with wildcards (such as '' or 'storage.') are not allowed. For more information see IAM Overview (https://cloud.google.com/iam/docs/overview#permissions).
				1920	"A String",
				1921	],
				1922	}
				1923
				1924	x__xgafv: string, V1 error format.
				1925	Allowed values
				1926	1 - v1 error format
				1927	2 - v2 error format
				1928
				1929	Returns:
				1930	An object of the form:
				1931
				1932	{ # Response message for TestIamPermissions method.
				1933	"permissions": [ # A subset of TestPermissionsRequest.permissions that the caller is allowed.
				1934	"A String",
				1935	],
				1936	}</pre>
				1937	</div>
				1938
Jon Wayne Parrott	36e41bc	2016-02-19 16:02:29 -0800	[diff] [blame]	1939	</body></html>