chore: Update discovery artifacts (#1486)
## Deleted keys were detected in the following stable discovery artifacts:
displayvideo v1 https://github.com/googleapis/google-api-python-client/commit/21f81c824089d0060dbd624f9755eea554464af2
sts v1 https://github.com/googleapis/google-api-python-client/commit/a8ebc2bef5d0ff6c449d65fcc9c0554c1f388985
## Deleted keys were detected in the following pre-stable discovery artifacts:
networkconnectivity v1alpha1 https://github.com/googleapis/google-api-python-client/commit/68588ad4744aa3c7a3a0afa12a1566f54ab3be94
## Discovery Artifact Change Summary:
feat(apigee): update the api https://github.com/googleapis/google-api-python-client/commit/41ae79b3554ffc9f1622e44757751009eeed76f8
feat(bigqueryreservation): update the api https://github.com/googleapis/google-api-python-client/commit/b0483aba20b17580b400283fb0073f07715bf254
feat(chat): update the api https://github.com/googleapis/google-api-python-client/commit/270312294f1cf89041825b0f31bfcf872fda9854
feat(cloudbuild): update the api https://github.com/googleapis/google-api-python-client/commit/30c122278e75a99b5b2a6ed330f317015134a23d
feat(cloudscheduler): update the api https://github.com/googleapis/google-api-python-client/commit/0405b1db79fe87305be25ba4f9389e94490040a7
feat(contactcenterinsights): update the api https://github.com/googleapis/google-api-python-client/commit/3abcf5334053f5612fd79163f2c433e5793ee782
feat(dialogflow): update the api https://github.com/googleapis/google-api-python-client/commit/c4935840495b1487d4be47b63f6c416113b9413f
feat(displayvideo): update the api https://github.com/googleapis/google-api-python-client/commit/21f81c824089d0060dbd624f9755eea554464af2
feat(monitoring): update the api https://github.com/googleapis/google-api-python-client/commit/d997b2c0d13203ac29b55c19858ef375380a5b7b
feat(networkconnectivity): update the api https://github.com/googleapis/google-api-python-client/commit/68588ad4744aa3c7a3a0afa12a1566f54ab3be94
feat(notebooks): update the api https://github.com/googleapis/google-api-python-client/commit/b975c34568f62bbf19f67f4731fe091aeeda8a75
feat(retail): update the api https://github.com/googleapis/google-api-python-client/commit/c2bcb7081a4f656b9e136ea38637cf262b60f387
feat(servicenetworking): update the api https://github.com/googleapis/google-api-python-client/commit/3bdc7b9d0854918ef65ccb4a3798d2f3f3b19d70
feat(sts): update the api https://github.com/googleapis/google-api-python-client/commit/a8ebc2bef5d0ff6c449d65fcc9c0554c1f388985
diff --git a/docs/dyn/dataproc_v1.projects.regions.workflowTemplates.html b/docs/dyn/dataproc_v1.projects.regions.workflowTemplates.html
index 870306e..2cacd9f 100644
--- a/docs/dyn/dataproc_v1.projects.regions.workflowTemplates.html
+++ b/docs/dyn/dataproc_v1.projects.regions.workflowTemplates.html
@@ -360,7 +360,7 @@
"autoscalingConfig": { # Autoscaling Policy config associated with the cluster. # Optional. Autoscaling config for the policy associated with the cluster. Cluster does not autoscale if this field is unset.
"policyUri": "A String", # Optional. The autoscaling policy used by the cluster.Only resource names including projectid and location (region) are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/locations/[dataproc_region]/autoscalingPolicies/[policy_id] projects/[project_id]/locations/[dataproc_region]/autoscalingPolicies/[policy_id]Note that the policy must be in the same project and Dataproc region.
},
- "configBucket": "A String", # Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging bucket (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a URI to a Cloud Storage bucket.
+ "configBucket": "A String", # Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
"encryptionConfig": { # Encryption settings for the cluster. # Optional. Encryption settings for the cluster.
"gcePdKmsKeyName": "A String", # Optional. The Cloud KMS key name to use for PD disk encryption for all instances in the cluster.
},
@@ -405,7 +405,7 @@
],
"zoneUri": "A String", # Optional. The zone where the Compute Engine cluster will be located. On a create request, it is required in the "global" region. If omitted in a non-global Dataproc region, the service will pick a zone in the corresponding Compute Engine region. On a get request, zone will always be present.A full URL, partial URI, or short name are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/zones/[zone] projects/[project_id]/zones/[zone] us-central1-f
},
- "gkeClusterConfig": { # The GKE config for this cluster. # Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.
+ "gkeClusterConfig": { # The cluster's GKE config. # Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.
"namespacedGkeDeploymentTarget": { # A full, namespace-isolated deployment target for an existing GKE cluster. # Optional. A target for the deployment.
"clusterNamespace": "A String", # Optional. A namespace within the GKE cluster to deploy into.
"targetGkeCluster": "A String", # Optional. The target GKE cluster to deploy to. Format: 'projects/{project}/locations/{location}/clusters/{cluster_id}'
@@ -423,7 +423,7 @@
"idleDeleteTtl": "A String", # Optional. The duration to keep the cluster alive while idling (when no jobs are running). Passing this threshold will cause the cluster to be deleted. Minimum value is 5 minutes; maximum value is 14 days (see JSON representation of Duration (https://developers.google.com/protocol-buffers/docs/proto3#json)).
"idleStartTime": "A String", # Output only. The time when cluster became idle (most recent job finished) and became eligible for deletion due to idleness (see JSON representation of Timestamp (https://developers.google.com/protocol-buffers/docs/proto3#json)).
},
- "masterConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the master instance in a cluster.
+ "masterConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the cluster's master instance.
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -460,7 +460,7 @@
"metastoreConfig": { # Specifies a Metastore configuration. # Optional. Metastore configuration.
"dataprocMetastoreService": "A String", # Required. Resource name of an existing Dataproc Metastore service.Example: projects/[project_id]/locations/[dataproc_region]/services/[service-name]
},
- "secondaryWorkerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for additional worker instances in a cluster.
+ "secondaryWorkerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for a cluster's secondary worker instances
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -518,7 +518,7 @@
"truststoreUri": "A String", # Optional. The Cloud Storage URI of the truststore file used for SSL encryption. If not provided, Dataproc will provide a self-signed certificate.
},
},
- "softwareConfig": { # Specifies the selection and config of software inside the cluster. # Optional. The config settings for software inside the cluster.
+ "softwareConfig": { # Specifies the selection and config of software inside the cluster. # Optional. The config settings for cluster software.
"imageVersion": "A String", # Optional. The version of software inside the cluster. It must be one of the supported Dataproc Versions (https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions#supported_dataproc_versions), such as "1.2" (including a subminor version, such as "1.2.29"), or the "preview" version (https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions#other_versions). If unspecified, it defaults to the latest Debian version.
"optionalComponents": [ # Optional. The set of components to activate on the cluster.
"A String",
@@ -527,8 +527,8 @@
"a_key": "A String",
},
},
- "tempBucket": "A String", # Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket. This field requires a Cloud Storage bucket name, not a URI to a Cloud Storage bucket.
- "workerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for worker instances in a cluster.
+ "tempBucket": "A String", # Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
+ "workerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the cluster's worker instances.
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -815,7 +815,7 @@
"autoscalingConfig": { # Autoscaling Policy config associated with the cluster. # Optional. Autoscaling config for the policy associated with the cluster. Cluster does not autoscale if this field is unset.
"policyUri": "A String", # Optional. The autoscaling policy used by the cluster.Only resource names including projectid and location (region) are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/locations/[dataproc_region]/autoscalingPolicies/[policy_id] projects/[project_id]/locations/[dataproc_region]/autoscalingPolicies/[policy_id]Note that the policy must be in the same project and Dataproc region.
},
- "configBucket": "A String", # Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging bucket (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a URI to a Cloud Storage bucket.
+ "configBucket": "A String", # Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
"encryptionConfig": { # Encryption settings for the cluster. # Optional. Encryption settings for the cluster.
"gcePdKmsKeyName": "A String", # Optional. The Cloud KMS key name to use for PD disk encryption for all instances in the cluster.
},
@@ -860,7 +860,7 @@
],
"zoneUri": "A String", # Optional. The zone where the Compute Engine cluster will be located. On a create request, it is required in the "global" region. If omitted in a non-global Dataproc region, the service will pick a zone in the corresponding Compute Engine region. On a get request, zone will always be present.A full URL, partial URI, or short name are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/zones/[zone] projects/[project_id]/zones/[zone] us-central1-f
},
- "gkeClusterConfig": { # The GKE config for this cluster. # Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.
+ "gkeClusterConfig": { # The cluster's GKE config. # Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.
"namespacedGkeDeploymentTarget": { # A full, namespace-isolated deployment target for an existing GKE cluster. # Optional. A target for the deployment.
"clusterNamespace": "A String", # Optional. A namespace within the GKE cluster to deploy into.
"targetGkeCluster": "A String", # Optional. The target GKE cluster to deploy to. Format: 'projects/{project}/locations/{location}/clusters/{cluster_id}'
@@ -878,7 +878,7 @@
"idleDeleteTtl": "A String", # Optional. The duration to keep the cluster alive while idling (when no jobs are running). Passing this threshold will cause the cluster to be deleted. Minimum value is 5 minutes; maximum value is 14 days (see JSON representation of Duration (https://developers.google.com/protocol-buffers/docs/proto3#json)).
"idleStartTime": "A String", # Output only. The time when cluster became idle (most recent job finished) and became eligible for deletion due to idleness (see JSON representation of Timestamp (https://developers.google.com/protocol-buffers/docs/proto3#json)).
},
- "masterConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the master instance in a cluster.
+ "masterConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the cluster's master instance.
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -915,7 +915,7 @@
"metastoreConfig": { # Specifies a Metastore configuration. # Optional. Metastore configuration.
"dataprocMetastoreService": "A String", # Required. Resource name of an existing Dataproc Metastore service.Example: projects/[project_id]/locations/[dataproc_region]/services/[service-name]
},
- "secondaryWorkerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for additional worker instances in a cluster.
+ "secondaryWorkerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for a cluster's secondary worker instances
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -973,7 +973,7 @@
"truststoreUri": "A String", # Optional. The Cloud Storage URI of the truststore file used for SSL encryption. If not provided, Dataproc will provide a self-signed certificate.
},
},
- "softwareConfig": { # Specifies the selection and config of software inside the cluster. # Optional. The config settings for software inside the cluster.
+ "softwareConfig": { # Specifies the selection and config of software inside the cluster. # Optional. The config settings for cluster software.
"imageVersion": "A String", # Optional. The version of software inside the cluster. It must be one of the supported Dataproc Versions (https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions#supported_dataproc_versions), such as "1.2" (including a subminor version, such as "1.2.29"), or the "preview" version (https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions#other_versions). If unspecified, it defaults to the latest Debian version.
"optionalComponents": [ # Optional. The set of components to activate on the cluster.
"A String",
@@ -982,8 +982,8 @@
"a_key": "A String",
},
},
- "tempBucket": "A String", # Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket. This field requires a Cloud Storage bucket name, not a URI to a Cloud Storage bucket.
- "workerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for worker instances in a cluster.
+ "tempBucket": "A String", # Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
+ "workerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the cluster's worker instances.
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -1297,7 +1297,7 @@
"autoscalingConfig": { # Autoscaling Policy config associated with the cluster. # Optional. Autoscaling config for the policy associated with the cluster. Cluster does not autoscale if this field is unset.
"policyUri": "A String", # Optional. The autoscaling policy used by the cluster.Only resource names including projectid and location (region) are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/locations/[dataproc_region]/autoscalingPolicies/[policy_id] projects/[project_id]/locations/[dataproc_region]/autoscalingPolicies/[policy_id]Note that the policy must be in the same project and Dataproc region.
},
- "configBucket": "A String", # Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging bucket (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a URI to a Cloud Storage bucket.
+ "configBucket": "A String", # Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
"encryptionConfig": { # Encryption settings for the cluster. # Optional. Encryption settings for the cluster.
"gcePdKmsKeyName": "A String", # Optional. The Cloud KMS key name to use for PD disk encryption for all instances in the cluster.
},
@@ -1342,7 +1342,7 @@
],
"zoneUri": "A String", # Optional. The zone where the Compute Engine cluster will be located. On a create request, it is required in the "global" region. If omitted in a non-global Dataproc region, the service will pick a zone in the corresponding Compute Engine region. On a get request, zone will always be present.A full URL, partial URI, or short name are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/zones/[zone] projects/[project_id]/zones/[zone] us-central1-f
},
- "gkeClusterConfig": { # The GKE config for this cluster. # Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.
+ "gkeClusterConfig": { # The cluster's GKE config. # Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.
"namespacedGkeDeploymentTarget": { # A full, namespace-isolated deployment target for an existing GKE cluster. # Optional. A target for the deployment.
"clusterNamespace": "A String", # Optional. A namespace within the GKE cluster to deploy into.
"targetGkeCluster": "A String", # Optional. The target GKE cluster to deploy to. Format: 'projects/{project}/locations/{location}/clusters/{cluster_id}'
@@ -1360,7 +1360,7 @@
"idleDeleteTtl": "A String", # Optional. The duration to keep the cluster alive while idling (when no jobs are running). Passing this threshold will cause the cluster to be deleted. Minimum value is 5 minutes; maximum value is 14 days (see JSON representation of Duration (https://developers.google.com/protocol-buffers/docs/proto3#json)).
"idleStartTime": "A String", # Output only. The time when cluster became idle (most recent job finished) and became eligible for deletion due to idleness (see JSON representation of Timestamp (https://developers.google.com/protocol-buffers/docs/proto3#json)).
},
- "masterConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the master instance in a cluster.
+ "masterConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the cluster's master instance.
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -1397,7 +1397,7 @@
"metastoreConfig": { # Specifies a Metastore configuration. # Optional. Metastore configuration.
"dataprocMetastoreService": "A String", # Required. Resource name of an existing Dataproc Metastore service.Example: projects/[project_id]/locations/[dataproc_region]/services/[service-name]
},
- "secondaryWorkerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for additional worker instances in a cluster.
+ "secondaryWorkerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for a cluster's secondary worker instances
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -1455,7 +1455,7 @@
"truststoreUri": "A String", # Optional. The Cloud Storage URI of the truststore file used for SSL encryption. If not provided, Dataproc will provide a self-signed certificate.
},
},
- "softwareConfig": { # Specifies the selection and config of software inside the cluster. # Optional. The config settings for software inside the cluster.
+ "softwareConfig": { # Specifies the selection and config of software inside the cluster. # Optional. The config settings for cluster software.
"imageVersion": "A String", # Optional. The version of software inside the cluster. It must be one of the supported Dataproc Versions (https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions#supported_dataproc_versions), such as "1.2" (including a subminor version, such as "1.2.29"), or the "preview" version (https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions#other_versions). If unspecified, it defaults to the latest Debian version.
"optionalComponents": [ # Optional. The set of components to activate on the cluster.
"A String",
@@ -1464,8 +1464,8 @@
"a_key": "A String",
},
},
- "tempBucket": "A String", # Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket. This field requires a Cloud Storage bucket name, not a URI to a Cloud Storage bucket.
- "workerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for worker instances in a cluster.
+ "tempBucket": "A String", # Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
+ "workerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the cluster's worker instances.
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -1533,7 +1533,7 @@
Returns:
An object of the form:
- { # An Identity and Access Management (IAM) policy, which specifies access controls for Google Cloud resources.A Policy is a collection of bindings. A binding binds one or more members to a single role. Members can be user accounts, service accounts, Google groups, and domains (such as G Suite). A role is a named list of permissions; each role can be an IAM predefined role or a user-created custom role.For some types of Google Cloud resources, a binding can also specify a condition, which is a logical expression that allows access to a resource only if the expression evaluates to true. A condition can add constraints based on attributes of the request, the resource, or both. To learn which resources support conditions in their IAM policies, see the IAM documentation (https://cloud.google.com/iam/help/conditions/resource-policies).JSON example: { "bindings": [ { "role": "roles/resourcemanager.organizationAdmin", "members": [ "user:mike@example.com", "group:admins@example.com", "domain:google.com", "serviceAccount:my-project-id@appspot.gserviceaccount.com" ] }, { "role": "roles/resourcemanager.organizationViewer", "members": [ "user:eve@example.com" ], "condition": { "title": "expirable access", "description": "Does not grant access after Sep 2020", "expression": "request.time < timestamp('2020-10-01T00:00:00.000Z')", } } ], "etag": "BwWWja0YfJA=", "version": 3 } YAML example: bindings: - members: - user:mike@example.com - group:admins@example.com - domain:google.com - serviceAccount:my-project-id@appspot.gserviceaccount.com role: roles/resourcemanager.organizationAdmin - members: - user:eve@example.com role: roles/resourcemanager.organizationViewer condition: title: expirable access description: Does not grant access after Sep 2020 expression: request.time < timestamp('2020-10-01T00:00:00.000Z') - etag: BwWWja0YfJA= - version: 3 For a description of IAM and its features, see the IAM documentation (https://cloud.google.com/iam/docs/).
+ { # An Identity and Access Management (IAM) policy, which specifies access controls for Google Cloud resources.A Policy is a collection of bindings. A binding binds one or more members to a single role. Members can be user accounts, service accounts, Google groups, and domains (such as G Suite). A role is a named list of permissions; each role can be an IAM predefined role or a user-created custom role.For some types of Google Cloud resources, a binding can also specify a condition, which is a logical expression that allows access to a resource only if the expression evaluates to true. A condition can add constraints based on attributes of the request, the resource, or both. To learn which resources support conditions in their IAM policies, see the IAM documentation (https://cloud.google.com/iam/help/conditions/resource-policies).JSON example: { "bindings": [ { "role": "roles/resourcemanager.organizationAdmin", "members": [ "user:mike@example.com", "group:admins@example.com", "domain:google.com", "serviceAccount:my-project-id@appspot.gserviceaccount.com" ] }, { "role": "roles/resourcemanager.organizationViewer", "members": [ "user:eve@example.com" ], "condition": { "title": "expirable access", "description": "Does not grant access after Sep 2020", "expression": "request.time < timestamp('2020-10-01T00:00:00.000Z')", } } ], "etag": "BwWWja0YfJA=", "version": 3 } YAML example: bindings: - members: - user:mike@example.com - group:admins@example.com - domain:google.com - serviceAccount:my-project-id@appspot.gserviceaccount.com role: roles/resourcemanager.organizationAdmin - members: - user:eve@example.com role: roles/resourcemanager.organizationViewer condition: title: expirable access description: Does not grant access after Sep 2020 expression: request.time < timestamp('2020-10-01T00:00:00.000Z') etag: BwWWja0YfJA= version: 3 For a description of IAM and its features, see the IAM documentation (https://cloud.google.com/iam/docs/).
"bindings": [ # Associates a list of members to a role. Optionally, may specify a condition that determines how and when the bindings are applied. Each of the bindings must contain at least one member.
{ # Associates members with a role.
"condition": { # Represents a textual expression in the Common Expression Language (CEL) syntax. CEL is a C-like expression language. The syntax and semantics of CEL are documented at https://github.com/google/cel-spec.Example (Comparison): title: "Summary size limit" description: "Determines if a summary is less than 100 chars" expression: "document.summary.size() < 100" Example (Equality): title: "Requestor is owner" description: "Determines if requestor is the document owner" expression: "document.owner == request.auth.claims.email" Example (Logic): title: "Public documents" description: "Determine whether the document should be publicly visible" expression: "document.type != 'private' && document.type != 'internal'" Example (Data Manipulation): title: "Notification string" description: "Create a notification string with a timestamp." expression: "'New message received at ' + string(document.create_time)" The exact variables and functions that may be referenced within an expression are determined by the service that evaluates it. See the service documentation for additional information. # The condition that is associated with this binding.If the condition evaluates to true, then this binding applies to the current request.If the condition evaluates to false, then this binding does not apply to the current request. However, a different role binding might grant the same role to one or more of the members in this binding.To learn which resources support conditions in their IAM policies, see the IAM documentation (https://cloud.google.com/iam/help/conditions/resource-policies).
@@ -1843,7 +1843,7 @@
"autoscalingConfig": { # Autoscaling Policy config associated with the cluster. # Optional. Autoscaling config for the policy associated with the cluster. Cluster does not autoscale if this field is unset.
"policyUri": "A String", # Optional. The autoscaling policy used by the cluster.Only resource names including projectid and location (region) are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/locations/[dataproc_region]/autoscalingPolicies/[policy_id] projects/[project_id]/locations/[dataproc_region]/autoscalingPolicies/[policy_id]Note that the policy must be in the same project and Dataproc region.
},
- "configBucket": "A String", # Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging bucket (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a URI to a Cloud Storage bucket.
+ "configBucket": "A String", # Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
"encryptionConfig": { # Encryption settings for the cluster. # Optional. Encryption settings for the cluster.
"gcePdKmsKeyName": "A String", # Optional. The Cloud KMS key name to use for PD disk encryption for all instances in the cluster.
},
@@ -1888,7 +1888,7 @@
],
"zoneUri": "A String", # Optional. The zone where the Compute Engine cluster will be located. On a create request, it is required in the "global" region. If omitted in a non-global Dataproc region, the service will pick a zone in the corresponding Compute Engine region. On a get request, zone will always be present.A full URL, partial URI, or short name are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/zones/[zone] projects/[project_id]/zones/[zone] us-central1-f
},
- "gkeClusterConfig": { # The GKE config for this cluster. # Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.
+ "gkeClusterConfig": { # The cluster's GKE config. # Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.
"namespacedGkeDeploymentTarget": { # A full, namespace-isolated deployment target for an existing GKE cluster. # Optional. A target for the deployment.
"clusterNamespace": "A String", # Optional. A namespace within the GKE cluster to deploy into.
"targetGkeCluster": "A String", # Optional. The target GKE cluster to deploy to. Format: 'projects/{project}/locations/{location}/clusters/{cluster_id}'
@@ -1906,7 +1906,7 @@
"idleDeleteTtl": "A String", # Optional. The duration to keep the cluster alive while idling (when no jobs are running). Passing this threshold will cause the cluster to be deleted. Minimum value is 5 minutes; maximum value is 14 days (see JSON representation of Duration (https://developers.google.com/protocol-buffers/docs/proto3#json)).
"idleStartTime": "A String", # Output only. The time when cluster became idle (most recent job finished) and became eligible for deletion due to idleness (see JSON representation of Timestamp (https://developers.google.com/protocol-buffers/docs/proto3#json)).
},
- "masterConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the master instance in a cluster.
+ "masterConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the cluster's master instance.
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -1943,7 +1943,7 @@
"metastoreConfig": { # Specifies a Metastore configuration. # Optional. Metastore configuration.
"dataprocMetastoreService": "A String", # Required. Resource name of an existing Dataproc Metastore service.Example: projects/[project_id]/locations/[dataproc_region]/services/[service-name]
},
- "secondaryWorkerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for additional worker instances in a cluster.
+ "secondaryWorkerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for a cluster's secondary worker instances
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -2001,7 +2001,7 @@
"truststoreUri": "A String", # Optional. The Cloud Storage URI of the truststore file used for SSL encryption. If not provided, Dataproc will provide a self-signed certificate.
},
},
- "softwareConfig": { # Specifies the selection and config of software inside the cluster. # Optional. The config settings for software inside the cluster.
+ "softwareConfig": { # Specifies the selection and config of software inside the cluster. # Optional. The config settings for cluster software.
"imageVersion": "A String", # Optional. The version of software inside the cluster. It must be one of the supported Dataproc Versions (https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions#supported_dataproc_versions), such as "1.2" (including a subminor version, such as "1.2.29"), or the "preview" version (https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions#other_versions). If unspecified, it defaults to the latest Debian version.
"optionalComponents": [ # Optional. The set of components to activate on the cluster.
"A String",
@@ -2010,8 +2010,8 @@
"a_key": "A String",
},
},
- "tempBucket": "A String", # Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket. This field requires a Cloud Storage bucket name, not a URI to a Cloud Storage bucket.
- "workerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for worker instances in a cluster.
+ "tempBucket": "A String", # Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
+ "workerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the cluster's worker instances.
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -2339,7 +2339,7 @@
"autoscalingConfig": { # Autoscaling Policy config associated with the cluster. # Optional. Autoscaling config for the policy associated with the cluster. Cluster does not autoscale if this field is unset.
"policyUri": "A String", # Optional. The autoscaling policy used by the cluster.Only resource names including projectid and location (region) are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/locations/[dataproc_region]/autoscalingPolicies/[policy_id] projects/[project_id]/locations/[dataproc_region]/autoscalingPolicies/[policy_id]Note that the policy must be in the same project and Dataproc region.
},
- "configBucket": "A String", # Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging bucket (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a URI to a Cloud Storage bucket.
+ "configBucket": "A String", # Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
"encryptionConfig": { # Encryption settings for the cluster. # Optional. Encryption settings for the cluster.
"gcePdKmsKeyName": "A String", # Optional. The Cloud KMS key name to use for PD disk encryption for all instances in the cluster.
},
@@ -2384,7 +2384,7 @@
],
"zoneUri": "A String", # Optional. The zone where the Compute Engine cluster will be located. On a create request, it is required in the "global" region. If omitted in a non-global Dataproc region, the service will pick a zone in the corresponding Compute Engine region. On a get request, zone will always be present.A full URL, partial URI, or short name are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/zones/[zone] projects/[project_id]/zones/[zone] us-central1-f
},
- "gkeClusterConfig": { # The GKE config for this cluster. # Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.
+ "gkeClusterConfig": { # The cluster's GKE config. # Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.
"namespacedGkeDeploymentTarget": { # A full, namespace-isolated deployment target for an existing GKE cluster. # Optional. A target for the deployment.
"clusterNamespace": "A String", # Optional. A namespace within the GKE cluster to deploy into.
"targetGkeCluster": "A String", # Optional. The target GKE cluster to deploy to. Format: 'projects/{project}/locations/{location}/clusters/{cluster_id}'
@@ -2402,7 +2402,7 @@
"idleDeleteTtl": "A String", # Optional. The duration to keep the cluster alive while idling (when no jobs are running). Passing this threshold will cause the cluster to be deleted. Minimum value is 5 minutes; maximum value is 14 days (see JSON representation of Duration (https://developers.google.com/protocol-buffers/docs/proto3#json)).
"idleStartTime": "A String", # Output only. The time when cluster became idle (most recent job finished) and became eligible for deletion due to idleness (see JSON representation of Timestamp (https://developers.google.com/protocol-buffers/docs/proto3#json)).
},
- "masterConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the master instance in a cluster.
+ "masterConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the cluster's master instance.
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -2439,7 +2439,7 @@
"metastoreConfig": { # Specifies a Metastore configuration. # Optional. Metastore configuration.
"dataprocMetastoreService": "A String", # Required. Resource name of an existing Dataproc Metastore service.Example: projects/[project_id]/locations/[dataproc_region]/services/[service-name]
},
- "secondaryWorkerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for additional worker instances in a cluster.
+ "secondaryWorkerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for a cluster's secondary worker instances
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -2497,7 +2497,7 @@
"truststoreUri": "A String", # Optional. The Cloud Storage URI of the truststore file used for SSL encryption. If not provided, Dataproc will provide a self-signed certificate.
},
},
- "softwareConfig": { # Specifies the selection and config of software inside the cluster. # Optional. The config settings for software inside the cluster.
+ "softwareConfig": { # Specifies the selection and config of software inside the cluster. # Optional. The config settings for cluster software.
"imageVersion": "A String", # Optional. The version of software inside the cluster. It must be one of the supported Dataproc Versions (https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions#supported_dataproc_versions), such as "1.2" (including a subminor version, such as "1.2.29"), or the "preview" version (https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions#other_versions). If unspecified, it defaults to the latest Debian version.
"optionalComponents": [ # Optional. The set of components to activate on the cluster.
"A String",
@@ -2506,8 +2506,8 @@
"a_key": "A String",
},
},
- "tempBucket": "A String", # Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket. This field requires a Cloud Storage bucket name, not a URI to a Cloud Storage bucket.
- "workerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for worker instances in a cluster.
+ "tempBucket": "A String", # Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
+ "workerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the cluster's worker instances.
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -2578,7 +2578,7 @@
The object takes the form of:
{ # Request message for SetIamPolicy method.
- "policy": { # An Identity and Access Management (IAM) policy, which specifies access controls for Google Cloud resources.A Policy is a collection of bindings. A binding binds one or more members to a single role. Members can be user accounts, service accounts, Google groups, and domains (such as G Suite). A role is a named list of permissions; each role can be an IAM predefined role or a user-created custom role.For some types of Google Cloud resources, a binding can also specify a condition, which is a logical expression that allows access to a resource only if the expression evaluates to true. A condition can add constraints based on attributes of the request, the resource, or both. To learn which resources support conditions in their IAM policies, see the IAM documentation (https://cloud.google.com/iam/help/conditions/resource-policies).JSON example: { "bindings": [ { "role": "roles/resourcemanager.organizationAdmin", "members": [ "user:mike@example.com", "group:admins@example.com", "domain:google.com", "serviceAccount:my-project-id@appspot.gserviceaccount.com" ] }, { "role": "roles/resourcemanager.organizationViewer", "members": [ "user:eve@example.com" ], "condition": { "title": "expirable access", "description": "Does not grant access after Sep 2020", "expression": "request.time < timestamp('2020-10-01T00:00:00.000Z')", } } ], "etag": "BwWWja0YfJA=", "version": 3 } YAML example: bindings: - members: - user:mike@example.com - group:admins@example.com - domain:google.com - serviceAccount:my-project-id@appspot.gserviceaccount.com role: roles/resourcemanager.organizationAdmin - members: - user:eve@example.com role: roles/resourcemanager.organizationViewer condition: title: expirable access description: Does not grant access after Sep 2020 expression: request.time < timestamp('2020-10-01T00:00:00.000Z') - etag: BwWWja0YfJA= - version: 3 For a description of IAM and its features, see the IAM documentation (https://cloud.google.com/iam/docs/). # REQUIRED: The complete policy to be applied to the resource. The size of the policy is limited to a few 10s of KB. An empty policy is a valid policy but certain Cloud Platform services (such as Projects) might reject them.
+ "policy": { # An Identity and Access Management (IAM) policy, which specifies access controls for Google Cloud resources.A Policy is a collection of bindings. A binding binds one or more members to a single role. Members can be user accounts, service accounts, Google groups, and domains (such as G Suite). A role is a named list of permissions; each role can be an IAM predefined role or a user-created custom role.For some types of Google Cloud resources, a binding can also specify a condition, which is a logical expression that allows access to a resource only if the expression evaluates to true. A condition can add constraints based on attributes of the request, the resource, or both. To learn which resources support conditions in their IAM policies, see the IAM documentation (https://cloud.google.com/iam/help/conditions/resource-policies).JSON example: { "bindings": [ { "role": "roles/resourcemanager.organizationAdmin", "members": [ "user:mike@example.com", "group:admins@example.com", "domain:google.com", "serviceAccount:my-project-id@appspot.gserviceaccount.com" ] }, { "role": "roles/resourcemanager.organizationViewer", "members": [ "user:eve@example.com" ], "condition": { "title": "expirable access", "description": "Does not grant access after Sep 2020", "expression": "request.time < timestamp('2020-10-01T00:00:00.000Z')", } } ], "etag": "BwWWja0YfJA=", "version": 3 } YAML example: bindings: - members: - user:mike@example.com - group:admins@example.com - domain:google.com - serviceAccount:my-project-id@appspot.gserviceaccount.com role: roles/resourcemanager.organizationAdmin - members: - user:eve@example.com role: roles/resourcemanager.organizationViewer condition: title: expirable access description: Does not grant access after Sep 2020 expression: request.time < timestamp('2020-10-01T00:00:00.000Z') etag: BwWWja0YfJA= version: 3 For a description of IAM and its features, see the IAM documentation (https://cloud.google.com/iam/docs/). # REQUIRED: The complete policy to be applied to the resource. The size of the policy is limited to a few 10s of KB. An empty policy is a valid policy but certain Cloud Platform services (such as Projects) might reject them.
"bindings": [ # Associates a list of members to a role. Optionally, may specify a condition that determines how and when the bindings are applied. Each of the bindings must contain at least one member.
{ # Associates members with a role.
"condition": { # Represents a textual expression in the Common Expression Language (CEL) syntax. CEL is a C-like expression language. The syntax and semantics of CEL are documented at https://github.com/google/cel-spec.Example (Comparison): title: "Summary size limit" description: "Determines if a summary is less than 100 chars" expression: "document.summary.size() < 100" Example (Equality): title: "Requestor is owner" description: "Determines if requestor is the document owner" expression: "document.owner == request.auth.claims.email" Example (Logic): title: "Public documents" description: "Determine whether the document should be publicly visible" expression: "document.type != 'private' && document.type != 'internal'" Example (Data Manipulation): title: "Notification string" description: "Create a notification string with a timestamp." expression: "'New message received at ' + string(document.create_time)" The exact variables and functions that may be referenced within an expression are determined by the service that evaluates it. See the service documentation for additional information. # The condition that is associated with this binding.If the condition evaluates to true, then this binding applies to the current request.If the condition evaluates to false, then this binding does not apply to the current request. However, a different role binding might grant the same role to one or more of the members in this binding.To learn which resources support conditions in their IAM policies, see the IAM documentation (https://cloud.google.com/iam/help/conditions/resource-policies).
@@ -2606,7 +2606,7 @@
Returns:
An object of the form:
- { # An Identity and Access Management (IAM) policy, which specifies access controls for Google Cloud resources.A Policy is a collection of bindings. A binding binds one or more members to a single role. Members can be user accounts, service accounts, Google groups, and domains (such as G Suite). A role is a named list of permissions; each role can be an IAM predefined role or a user-created custom role.For some types of Google Cloud resources, a binding can also specify a condition, which is a logical expression that allows access to a resource only if the expression evaluates to true. A condition can add constraints based on attributes of the request, the resource, or both. To learn which resources support conditions in their IAM policies, see the IAM documentation (https://cloud.google.com/iam/help/conditions/resource-policies).JSON example: { "bindings": [ { "role": "roles/resourcemanager.organizationAdmin", "members": [ "user:mike@example.com", "group:admins@example.com", "domain:google.com", "serviceAccount:my-project-id@appspot.gserviceaccount.com" ] }, { "role": "roles/resourcemanager.organizationViewer", "members": [ "user:eve@example.com" ], "condition": { "title": "expirable access", "description": "Does not grant access after Sep 2020", "expression": "request.time < timestamp('2020-10-01T00:00:00.000Z')", } } ], "etag": "BwWWja0YfJA=", "version": 3 } YAML example: bindings: - members: - user:mike@example.com - group:admins@example.com - domain:google.com - serviceAccount:my-project-id@appspot.gserviceaccount.com role: roles/resourcemanager.organizationAdmin - members: - user:eve@example.com role: roles/resourcemanager.organizationViewer condition: title: expirable access description: Does not grant access after Sep 2020 expression: request.time < timestamp('2020-10-01T00:00:00.000Z') - etag: BwWWja0YfJA= - version: 3 For a description of IAM and its features, see the IAM documentation (https://cloud.google.com/iam/docs/).
+ { # An Identity and Access Management (IAM) policy, which specifies access controls for Google Cloud resources.A Policy is a collection of bindings. A binding binds one or more members to a single role. Members can be user accounts, service accounts, Google groups, and domains (such as G Suite). A role is a named list of permissions; each role can be an IAM predefined role or a user-created custom role.For some types of Google Cloud resources, a binding can also specify a condition, which is a logical expression that allows access to a resource only if the expression evaluates to true. A condition can add constraints based on attributes of the request, the resource, or both. To learn which resources support conditions in their IAM policies, see the IAM documentation (https://cloud.google.com/iam/help/conditions/resource-policies).JSON example: { "bindings": [ { "role": "roles/resourcemanager.organizationAdmin", "members": [ "user:mike@example.com", "group:admins@example.com", "domain:google.com", "serviceAccount:my-project-id@appspot.gserviceaccount.com" ] }, { "role": "roles/resourcemanager.organizationViewer", "members": [ "user:eve@example.com" ], "condition": { "title": "expirable access", "description": "Does not grant access after Sep 2020", "expression": "request.time < timestamp('2020-10-01T00:00:00.000Z')", } } ], "etag": "BwWWja0YfJA=", "version": 3 } YAML example: bindings: - members: - user:mike@example.com - group:admins@example.com - domain:google.com - serviceAccount:my-project-id@appspot.gserviceaccount.com role: roles/resourcemanager.organizationAdmin - members: - user:eve@example.com role: roles/resourcemanager.organizationViewer condition: title: expirable access description: Does not grant access after Sep 2020 expression: request.time < timestamp('2020-10-01T00:00:00.000Z') etag: BwWWja0YfJA= version: 3 For a description of IAM and its features, see the IAM documentation (https://cloud.google.com/iam/docs/).
"bindings": [ # Associates a list of members to a role. Optionally, may specify a condition that determines how and when the bindings are applied. Each of the bindings must contain at least one member.
{ # Associates members with a role.
"condition": { # Represents a textual expression in the Common Expression Language (CEL) syntax. CEL is a C-like expression language. The syntax and semantics of CEL are documented at https://github.com/google/cel-spec.Example (Comparison): title: "Summary size limit" description: "Determines if a summary is less than 100 chars" expression: "document.summary.size() < 100" Example (Equality): title: "Requestor is owner" description: "Determines if requestor is the document owner" expression: "document.owner == request.auth.claims.email" Example (Logic): title: "Public documents" description: "Determine whether the document should be publicly visible" expression: "document.type != 'private' && document.type != 'internal'" Example (Data Manipulation): title: "Notification string" description: "Create a notification string with a timestamp." expression: "'New message received at ' + string(document.create_time)" The exact variables and functions that may be referenced within an expression are determined by the service that evaluates it. See the service documentation for additional information. # The condition that is associated with this binding.If the condition evaluates to true, then this binding applies to the current request.If the condition evaluates to false, then this binding does not apply to the current request. However, a different role binding might grant the same role to one or more of the members in this binding.To learn which resources support conditions in their IAM policies, see the IAM documentation (https://cloud.google.com/iam/help/conditions/resource-policies).
@@ -2900,7 +2900,7 @@
"autoscalingConfig": { # Autoscaling Policy config associated with the cluster. # Optional. Autoscaling config for the policy associated with the cluster. Cluster does not autoscale if this field is unset.
"policyUri": "A String", # Optional. The autoscaling policy used by the cluster.Only resource names including projectid and location (region) are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/locations/[dataproc_region]/autoscalingPolicies/[policy_id] projects/[project_id]/locations/[dataproc_region]/autoscalingPolicies/[policy_id]Note that the policy must be in the same project and Dataproc region.
},
- "configBucket": "A String", # Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging bucket (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a URI to a Cloud Storage bucket.
+ "configBucket": "A String", # Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
"encryptionConfig": { # Encryption settings for the cluster. # Optional. Encryption settings for the cluster.
"gcePdKmsKeyName": "A String", # Optional. The Cloud KMS key name to use for PD disk encryption for all instances in the cluster.
},
@@ -2945,7 +2945,7 @@
],
"zoneUri": "A String", # Optional. The zone where the Compute Engine cluster will be located. On a create request, it is required in the "global" region. If omitted in a non-global Dataproc region, the service will pick a zone in the corresponding Compute Engine region. On a get request, zone will always be present.A full URL, partial URI, or short name are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/zones/[zone] projects/[project_id]/zones/[zone] us-central1-f
},
- "gkeClusterConfig": { # The GKE config for this cluster. # Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.
+ "gkeClusterConfig": { # The cluster's GKE config. # Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.
"namespacedGkeDeploymentTarget": { # A full, namespace-isolated deployment target for an existing GKE cluster. # Optional. A target for the deployment.
"clusterNamespace": "A String", # Optional. A namespace within the GKE cluster to deploy into.
"targetGkeCluster": "A String", # Optional. The target GKE cluster to deploy to. Format: 'projects/{project}/locations/{location}/clusters/{cluster_id}'
@@ -2963,7 +2963,7 @@
"idleDeleteTtl": "A String", # Optional. The duration to keep the cluster alive while idling (when no jobs are running). Passing this threshold will cause the cluster to be deleted. Minimum value is 5 minutes; maximum value is 14 days (see JSON representation of Duration (https://developers.google.com/protocol-buffers/docs/proto3#json)).
"idleStartTime": "A String", # Output only. The time when cluster became idle (most recent job finished) and became eligible for deletion due to idleness (see JSON representation of Timestamp (https://developers.google.com/protocol-buffers/docs/proto3#json)).
},
- "masterConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the master instance in a cluster.
+ "masterConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the cluster's master instance.
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -3000,7 +3000,7 @@
"metastoreConfig": { # Specifies a Metastore configuration. # Optional. Metastore configuration.
"dataprocMetastoreService": "A String", # Required. Resource name of an existing Dataproc Metastore service.Example: projects/[project_id]/locations/[dataproc_region]/services/[service-name]
},
- "secondaryWorkerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for additional worker instances in a cluster.
+ "secondaryWorkerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for a cluster's secondary worker instances
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -3058,7 +3058,7 @@
"truststoreUri": "A String", # Optional. The Cloud Storage URI of the truststore file used for SSL encryption. If not provided, Dataproc will provide a self-signed certificate.
},
},
- "softwareConfig": { # Specifies the selection and config of software inside the cluster. # Optional. The config settings for software inside the cluster.
+ "softwareConfig": { # Specifies the selection and config of software inside the cluster. # Optional. The config settings for cluster software.
"imageVersion": "A String", # Optional. The version of software inside the cluster. It must be one of the supported Dataproc Versions (https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions#supported_dataproc_versions), such as "1.2" (including a subminor version, such as "1.2.29"), or the "preview" version (https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions#other_versions). If unspecified, it defaults to the latest Debian version.
"optionalComponents": [ # Optional. The set of components to activate on the cluster.
"A String",
@@ -3067,8 +3067,8 @@
"a_key": "A String",
},
},
- "tempBucket": "A String", # Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket. This field requires a Cloud Storage bucket name, not a URI to a Cloud Storage bucket.
- "workerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for worker instances in a cluster.
+ "tempBucket": "A String", # Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
+ "workerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the cluster's worker instances.
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -3355,7 +3355,7 @@
"autoscalingConfig": { # Autoscaling Policy config associated with the cluster. # Optional. Autoscaling config for the policy associated with the cluster. Cluster does not autoscale if this field is unset.
"policyUri": "A String", # Optional. The autoscaling policy used by the cluster.Only resource names including projectid and location (region) are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/locations/[dataproc_region]/autoscalingPolicies/[policy_id] projects/[project_id]/locations/[dataproc_region]/autoscalingPolicies/[policy_id]Note that the policy must be in the same project and Dataproc region.
},
- "configBucket": "A String", # Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging bucket (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a URI to a Cloud Storage bucket.
+ "configBucket": "A String", # Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
"encryptionConfig": { # Encryption settings for the cluster. # Optional. Encryption settings for the cluster.
"gcePdKmsKeyName": "A String", # Optional. The Cloud KMS key name to use for PD disk encryption for all instances in the cluster.
},
@@ -3400,7 +3400,7 @@
],
"zoneUri": "A String", # Optional. The zone where the Compute Engine cluster will be located. On a create request, it is required in the "global" region. If omitted in a non-global Dataproc region, the service will pick a zone in the corresponding Compute Engine region. On a get request, zone will always be present.A full URL, partial URI, or short name are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/zones/[zone] projects/[project_id]/zones/[zone] us-central1-f
},
- "gkeClusterConfig": { # The GKE config for this cluster. # Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.
+ "gkeClusterConfig": { # The cluster's GKE config. # Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.
"namespacedGkeDeploymentTarget": { # A full, namespace-isolated deployment target for an existing GKE cluster. # Optional. A target for the deployment.
"clusterNamespace": "A String", # Optional. A namespace within the GKE cluster to deploy into.
"targetGkeCluster": "A String", # Optional. The target GKE cluster to deploy to. Format: 'projects/{project}/locations/{location}/clusters/{cluster_id}'
@@ -3418,7 +3418,7 @@
"idleDeleteTtl": "A String", # Optional. The duration to keep the cluster alive while idling (when no jobs are running). Passing this threshold will cause the cluster to be deleted. Minimum value is 5 minutes; maximum value is 14 days (see JSON representation of Duration (https://developers.google.com/protocol-buffers/docs/proto3#json)).
"idleStartTime": "A String", # Output only. The time when cluster became idle (most recent job finished) and became eligible for deletion due to idleness (see JSON representation of Timestamp (https://developers.google.com/protocol-buffers/docs/proto3#json)).
},
- "masterConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the master instance in a cluster.
+ "masterConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the cluster's master instance.
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -3455,7 +3455,7 @@
"metastoreConfig": { # Specifies a Metastore configuration. # Optional. Metastore configuration.
"dataprocMetastoreService": "A String", # Required. Resource name of an existing Dataproc Metastore service.Example: projects/[project_id]/locations/[dataproc_region]/services/[service-name]
},
- "secondaryWorkerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for additional worker instances in a cluster.
+ "secondaryWorkerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for a cluster's secondary worker instances
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.
@@ -3513,7 +3513,7 @@
"truststoreUri": "A String", # Optional. The Cloud Storage URI of the truststore file used for SSL encryption. If not provided, Dataproc will provide a self-signed certificate.
},
},
- "softwareConfig": { # Specifies the selection and config of software inside the cluster. # Optional. The config settings for software inside the cluster.
+ "softwareConfig": { # Specifies the selection and config of software inside the cluster. # Optional. The config settings for cluster software.
"imageVersion": "A String", # Optional. The version of software inside the cluster. It must be one of the supported Dataproc Versions (https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions#supported_dataproc_versions), such as "1.2" (including a subminor version, such as "1.2.29"), or the "preview" version (https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions#other_versions). If unspecified, it defaults to the latest Debian version.
"optionalComponents": [ # Optional. The set of components to activate on the cluster.
"A String",
@@ -3522,8 +3522,8 @@
"a_key": "A String",
},
},
- "tempBucket": "A String", # Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket. This field requires a Cloud Storage bucket name, not a URI to a Cloud Storage bucket.
- "workerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for worker instances in a cluster.
+ "tempBucket": "A String", # Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
+ "workerConfig": { # The config settings for Compute Engine resources in an instance group, such as a master or worker group. # Optional. The Compute Engine config settings for the cluster's worker instances.
"accelerators": [ # Optional. The Compute Engine accelerator configuration for these instances.
{ # Specifies the type and number of accelerator cards attached to the instances of an instance. See GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/).
"acceleratorCount": 42, # The number of the accelerator cards of this type exposed to this instance.