Blame - docs/dyn/ml_v1.projects.jobs.html - platform/external/python/google-api-python-client

2017-03-13 12:12:03 -0400

[diff] [blame]

76

<h2>Instance Methods</h2>

77

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

78

<code><a href="#cancel">cancel(name, body=None, x__xgafv=None)</a></code></p>

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

79

<p class="firstline">Cancels a running job.</p>

80

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

81

<code><a href="#create">create(parent, body=None, x__xgafv=None)</a></code></p>

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

82

<p class="firstline">Creates a training or a batch prediction job.</p>

83

Thomas Coffee

2f24537

2017-03-27 10:39:26 -0700

[diff] [blame]

84

<code><a href="#get">get(name, x__xgafv=None)</a></code></p>

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

85

<p class="firstline">Describes a job.</p>

86

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

87

<code><a href="#getIamPolicy">getIamPolicy(resource, options_requestedPolicyVersion=None, x__xgafv=None)</a></code></p>

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

88

<p class="firstline">Gets the access control policy for a resource.</p>

89

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

90

<code><a href="#list">list(parent, filter=None, pageToken=None, pageSize=None, x__xgafv=None)</a></code></p>

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

91

<p class="firstline">Lists the jobs in the project.</p>

92

93

<code><a href="#list_next">list_next(previous_request, previous_response)</a></code></p>

94

<p class="firstline">Retrieves the next page of results.</p>

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

95

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

96

<code><a href="#patch">patch(name, body=None, updateMask=None, x__xgafv=None)</a></code></p>

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

97

<p class="firstline">Updates a specific job resource.</p>

98

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

99

<code><a href="#setIamPolicy">setIamPolicy(resource, body=None, x__xgafv=None)</a></code></p>

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

100

<p class="firstline">Sets the access control policy on the specified resource. Replaces any</p>

101

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

102

<code><a href="#testIamPermissions">testIamPermissions(resource, body=None, x__xgafv=None)</a></code></p>

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

103

<p class="firstline">Returns permissions that a caller has on the specified resource.</p>

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

104

<h3>Method Details</h3>

105

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

106

<code class="details" id="cancel">cancel(name, body=None, x__xgafv=None)</code>

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

107

<pre>Cancels a running job.

108

109

Args:

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

110

name: string, Required. The name of the job to cancel. (required)

111

body: object, The request body.

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

112

The object takes the form of:

113

114

{ # Request message for the CancelJob method.

115

}

116

117

x__xgafv: string, V1 error format.

Allowed values

1 - v1 error format

2 - v2 error format

Returns:

An object of the form:

124

125

{ # A generic empty message that you can re-use to avoid defining duplicated

126

# empty messages in your APIs. A typical example is to use it as the request

127

# or the response type of an API method. For instance:

128

#

129

# service Foo {

130

# rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty);

131

# }

132

#

133

# The JSON representation for `Empty` is empty JSON object `{}`.

}</pre>

</div>

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

138

<code class="details" id="create">create(parent, body=None, x__xgafv=None)</code>

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

139

<pre>Creates a training or a batch prediction job.

140

141

Args:

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

142

parent: string, Required. The project name. (required)

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

143

body: object, The request body.

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

144

The object takes the form of:

145

146

{ # Represents a training or prediction job.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

147

"predictionInput": { # Represents input parameters for a prediction job. # Input parameters to create a prediction job.

148

"versionName": "A String", # Use this field if you want to specify a version of the model to use. The

149

# string is formatted the same way as `model_version`, with the addition

150

# of the version information:

151

#

152

# `"projects/YOUR_PROJECT/models/YOUR_MODEL/versions/YOUR_VERSION"`

153

"modelName": "A String", # Use this field if you want to use the default version for the specified

154

# model. The string must use the following format:

155

#

156

# `"projects/YOUR_PROJECT/models/YOUR_MODEL"`

157

"uri": "A String", # Use this field if you want to specify a Google Cloud Storage path for

158

# the model to use.

159

"maxWorkerCount": "A String", # Optional. The maximum number of workers to be used for parallel processing.

160

# Defaults to 10 if not specified.

161

"signatureName": "A String", # Optional. The name of the signature defined in the SavedModel to use for

162

# this job. Please refer to

163

# [SavedModel](https://tensorflow.github.io/serving/serving_basic.html)

164

# for information about how to use signatures.

165

#

166

# Defaults to

167

# [DEFAULT_SERVING_SIGNATURE_DEF_KEY](https://www.tensorflow.org/api_docs/python/tf/saved_model/signature_constants)

168

# , which is "serving_default".

169

"outputPath": "A String", # Required. The output Google Cloud Storage location.

170

"outputDataFormat": "A String", # Optional. Format of the output data files, defaults to JSON.

171

"dataFormat": "A String", # Required. The format of the input data files.

172

"batchSize": "A String", # Optional. Number of records per batch, defaults to 64.

173

# The service will buffer batch_size number of records in memory before

174

# invoking one Tensorflow prediction call internally. So take the record

175

# size and memory available into consideration when setting this parameter.

176

"runtimeVersion": "A String", # Optional. The AI Platform runtime version to use for this batch

177

# prediction. If not set, AI Platform will pick the runtime version used

178

# during the CreateVersion request for this model version, or choose the

179

# latest stable version when model version information is not available

180

# such as when the model is specified by uri.

181

"inputPaths": [ # Required. The Cloud Storage location of the input data files. May contain

182

# <a href="/storage/docs/gsutil/addlhelp/WildcardNames">wildcards</a>.

183

"A String",

184

],

185

"region": "A String", # Required. The Google Compute Engine region to run the prediction job in.

186

# See the <a href="/ml-engine/docs/tensorflow/regions">available regions</a>

187

# for AI Platform services.

188

},

189

"errorMessage": "A String", # Output only. The details of a failure or a cancellation.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

190

"etag": "A String", # `etag` is used for optimistic concurrency control as a way to help

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

191

# prevent simultaneous updates of a job from overwriting each other.

192

# It is strongly suggested that systems make use of the `etag` in the

193

# read-modify-write cycle to perform job updates in order to avoid race

194

# conditions: An `etag` is returned in the response to `GetJob`, and

195

# systems are expected to put that etag in the request to `UpdateJob` to

196

# ensure that their change will be applied to the same version of the job.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

197

"trainingInput": { # Represents input parameters for a training job. When using the gcloud command # Input parameters to create a training job.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

198

# to submit your training job, you can specify the input parameters as

199

# command-line arguments and/or in a YAML configuration file referenced from

200

# the --config command-line argument. For details, see the guide to [submitting

201

# a training job](/ai-platform/training/docs/training-jobs).

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

202

"workerCount": "A String", # Optional. The number of worker replicas to use for the training job. Each

203

# replica in the cluster will be of the type specified in `worker_type`.

204

#

205

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

206

# set this value, you must also set `worker_type`.

207

#

208

# The default value is zero.

209

"pythonModule": "A String", # Required. The Python module name to run after installing the packages.

210

"args": [ # Optional. Command-line arguments passed to the training application when it

211

# starts. If your job uses a custom container, then the arguments are passed

212

# to the container's <a class="external" target="_blank"

213

# href="https://docs.docker.com/engine/reference/builder/#entrypoint">

214

# `ENTRYPOINT`</a> command.

215

"A String",

216

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

217

"parameterServerConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for parameter servers.

218

#

219

# You should only set `parameterServerConfig.acceleratorConfig` if

220

# `parameterServerType` is set to a Compute Engine machine type. [Learn

221

# about restrictions on accelerator configurations for

222

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

223

#

224

# Set `parameterServerConfig.imageUri` only if you build a custom image for

225

# your parameter server. If `parameterServerConfig.imageUri` has not been

226

# set, AI Platform uses the value of `masterConfig.imageUri`. Learn more about [configuring custom

227

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

228

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

229

# Registry. Learn more about [configuring custom

230

# containers](/ai-platform/training/docs/distributed-training-containers).

231

"containerArgs": [ # Arguments to the entrypoint command.

232

# The following rules apply for container_command and container_args:

233

# - If you do not supply command or args:

234

# The defaults defined in the Docker image are used.

235

# - If you supply a command but no args:

236

# The default EntryPoint and the default Cmd defined in the Docker image

237

# are ignored. Your command is run without any arguments.

238

# - If you supply only args:

239

# The default Entrypoint defined in the Docker image is run with the args

240

# that you supplied.

241

# - If you supply a command and args:

242

# The default Entrypoint and the default Cmd defined in the Docker image

243

# are ignored. Your command is run with your args.

244

# It cannot be set if custom container image is

245

# not provided.

246

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

247

# both cannot be set at the same time.

248

"A String",

249

],

250

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

251

# [Learn about restrictions on accelerator configurations for

252

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

253

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

254

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

255

# [accelerators for online

256

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

257

"count": "A String", # The number of accelerators to attach to each machine running the job.

258

"type": "A String", # The type of accelerator to use.

259

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

260

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

261

# the one used in the custom container. This field is required if the replica

262

# is a TPU worker that uses a custom container. Otherwise, do not specify

263

# this field. This must be a [runtime version that currently supports

264

# training with

265

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

266

#

267

# Note that the version of TensorFlow included in a runtime version may

268

# differ from the numbering of the runtime version itself, because it may

269

# have a different [patch

270

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

271

# In this field, you must specify the runtime version (TensorFlow minor

272

# version). For example, if your custom container runs TensorFlow `1.x.y`,

273

# specify `1.x`.

274

"containerCommand": [ # The command with which the replica's custom container is run.

275

# If provided, it will override default ENTRYPOINT of the docker image.

276

# If not provided, the docker image's ENTRYPOINT is used.

277

# It cannot be set if custom container image is

278

# not provided.

279

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

280

# both cannot be set at the same time.

281

"A String",

282

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

283

},

284

"encryptionConfig": { # Represents a custom encryption key configuration that can be applied to # Optional. Options for using customer-managed encryption keys (CMEK) to

285

# protect resources created by a training job, instead of using Google's

286

# default encryption. If this is set, then all resources created by the

287

# training job will be encrypted with the customer-managed encryption key

288

# that you specify.

289

#

290

# [Learn how and when to use CMEK with AI Platform

291

# Training](/ai-platform/training/docs/cmek).

292

# a resource.

293

"kmsKeyName": "A String", # The Cloud KMS resource identifier of the customer-managed encryption key

294

# used to protect a resource, such as a training job. It has the following

295

# format:

296

# `projects/{PROJECT_ID}/locations/{REGION}/keyRings/{KEY_RING_NAME}/cryptoKeys/{KEY_NAME}`

297

},

298

"hyperparameters": { # Represents a set of hyperparameters to optimize. # Optional. The set of Hyperparameters to tune.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

299

"params": [ # Required. The set of parameters to tune.

300

{ # Represents a single hyperparameter to optimize.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

301

"categoricalValues": [ # Required if type is `CATEGORICAL`. The list of possible categories.

302

"A String",

303

],

304

"parameterName": "A String", # Required. The parameter name must be unique amongst all ParameterConfigs in

305

# a HyperparameterSpec message. E.g., "learning_rate".

306

"minValue": 3.14, # Required if type is `DOUBLE` or `INTEGER`. This field

307

# should be unset if type is `CATEGORICAL`. This value should be integers if

308

# type is INTEGER.

309

"discreteValues": [ # Required if type is `DISCRETE`.

310

# A list of feasible points.

311

# The list should be in strictly increasing order. For instance, this

312

# parameter might have possible settings of 1.5, 2.5, and 4.0. This list

313

# should not contain more than 1,000 values.

314

3.14,

315

],

316

"scaleType": "A String", # Optional. How the parameter should be scaled to the hypercube.

317

# Leave unset for categorical parameters.

318

# Some kind of scaling is strongly recommended for real or integral

319

# parameters (e.g., `UNIT_LINEAR_SCALE`).

320

"maxValue": 3.14, # Required if type is `DOUBLE` or `INTEGER`. This field

321

# should be unset if type is `CATEGORICAL`. This value should be integers if

322

# type is `INTEGER`.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

323

"type": "A String", # Required. The type of the parameter.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

324

},

325

],

326

"enableTrialEarlyStopping": True or False, # Optional. Indicates if the hyperparameter tuning job enables auto trial

327

# early stopping.

328

"resumePreviousJobId": "A String", # Optional. The prior hyperparameter tuning job id that users hope to

329

# continue with. The job id will be used to find the corresponding vizier

330

# study guid and resume the study.

331

"maxParallelTrials": 42, # Optional. The number of training trials to run concurrently.

332

# You can reduce the time it takes to perform hyperparameter tuning by adding

333

# trials in parallel. However, each trail only benefits from the information

334

# gained in completed trials. That means that a trial does not get access to

335

# the results of trials running at the same time, which could reduce the

336

# quality of the overall optimization.

337

#

338

# Each trial will use the same scale tier and machine types.

339

#

340

# Defaults to one.

341

"maxFailedTrials": 42, # Optional. The number of failed trials that need to be seen before failing

342

# the hyperparameter tuning job. You can specify this field to override the

343

# default failing criteria for AI Platform hyperparameter tuning jobs.

344

#

345

# Defaults to zero, which means the service decides when a hyperparameter

346

# job should fail.

347

"goal": "A String", # Required. The type of goal to use for tuning. Available types are

348

# `MAXIMIZE` and `MINIMIZE`.

349

#

350

# Defaults to `MAXIMIZE`.

351

"maxTrials": 42, # Optional. How many training trials should be attempted to optimize

352

# the specified hyperparameters.

353

#

354

# Defaults to one.

355

"algorithm": "A String", # Optional. The search algorithm specified for the hyperparameter

356

# tuning job.

357

# Uses the default AI Platform hyperparameter tuning

358

# algorithm if unspecified.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

359

"hyperparameterMetricTag": "A String", # Optional. The TensorFlow summary tag name to use for optimizing trials. For

360

# current versions of TensorFlow, this tag name should exactly match what is

361

# shown in TensorBoard, including all scopes. For versions of TensorFlow

362

# prior to 0.12, this should be only the tag passed to tf.Summary.

363

# By default, "training/hptuning/metric" will be used.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

364

},

365

"workerConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for workers.

366

#

367

# You should only set `workerConfig.acceleratorConfig` if `workerType` is set

368

# to a Compute Engine machine type. [Learn about restrictions on accelerator

369

# configurations for

370

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

371

#

372

# Set `workerConfig.imageUri` only if you build a custom image for your

373

# worker. If `workerConfig.imageUri` has not been set, AI Platform uses

374

# the value of `masterConfig.imageUri`. Learn more about [configuring custom

375

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

376

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

377

# Registry. Learn more about [configuring custom

378

# containers](/ai-platform/training/docs/distributed-training-containers).

379

"containerArgs": [ # Arguments to the entrypoint command.

380

# The following rules apply for container_command and container_args:

381

# - If you do not supply command or args:

382

# The defaults defined in the Docker image are used.

383

# - If you supply a command but no args:

384

# The default EntryPoint and the default Cmd defined in the Docker image

385

# are ignored. Your command is run without any arguments.

386

# - If you supply only args:

387

# The default Entrypoint defined in the Docker image is run with the args

388

# that you supplied.

389

# - If you supply a command and args:

390

# The default Entrypoint and the default Cmd defined in the Docker image

391

# are ignored. Your command is run with your args.

392

# It cannot be set if custom container image is

393

# not provided.

394

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

395

# both cannot be set at the same time.

396

"A String",

397

],

398

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

399

# [Learn about restrictions on accelerator configurations for

400

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

401

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

402

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

403

# [accelerators for online

404

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

405

"count": "A String", # The number of accelerators to attach to each machine running the job.

406

"type": "A String", # The type of accelerator to use.

407

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

408

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

409

# the one used in the custom container. This field is required if the replica

410

# is a TPU worker that uses a custom container. Otherwise, do not specify

411

# this field. This must be a [runtime version that currently supports

412

# training with

413

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

414

#

415

# Note that the version of TensorFlow included in a runtime version may

416

# differ from the numbering of the runtime version itself, because it may

417

# have a different [patch

418

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

419

# In this field, you must specify the runtime version (TensorFlow minor

420

# version). For example, if your custom container runs TensorFlow `1.x.y`,

421

# specify `1.x`.

422

"containerCommand": [ # The command with which the replica's custom container is run.

423

# If provided, it will override default ENTRYPOINT of the docker image.

424

# If not provided, the docker image's ENTRYPOINT is used.

425

# It cannot be set if custom container image is

426

# not provided.

427

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

428

# both cannot be set at the same time.

429

"A String",

430

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

431

},

432

"parameterServerCount": "A String", # Optional. The number of parameter server replicas to use for the training

433

# job. Each replica in the cluster will be of the type specified in

434

# `parameter_server_type`.

435

#

436

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

437

# set this value, you must also set `parameter_server_type`.

438

#

439

# The default value is zero.

440

"packageUris": [ # Required. The Google Cloud Storage location of the packages with

441

# the training program and any additional dependencies.

442

# The maximum number of package URIs is 100.

443

"A String",

444

],

445

"evaluatorCount": "A String", # Optional. The number of evaluator replicas to use for the training job.

446

# Each replica in the cluster will be of the type specified in

447

# `evaluator_type`.

448

#

449

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

450

# set this value, you must also set `evaluator_type`.

451

#

452

# The default value is zero.

453

"masterType": "A String", # Optional. Specifies the type of virtual machine to use for your training

454

# job's master worker. You must specify this field when `scaleTier` is set to

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

455

# `CUSTOM`.

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

456

#

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

457

# You can use certain Compute Engine machine types directly in this field.

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

458

# The following types are supported:

459

#

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

# - `n1-standard-4`

# - `n1-standard-8`

# - `n1-standard-16`

# - `n1-standard-32`

# - `n1-standard-64`

# - `n1-standard-96`

# - `n1-highmem-2`

# - `n1-highmem-4`

# - `n1-highmem-8`

# - `n1-highmem-16`

# - `n1-highmem-32`

# - `n1-highmem-64`

# - `n1-highmem-96`

# - `n1-highcpu-16`

# - `n1-highcpu-32`

# - `n1-highcpu-64`

# - `n1-highcpu-96`

#

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

478

# Learn more about [using Compute Engine machine

479

# types](/ml-engine/docs/machine-types#compute-engine-machine-types).

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

480

#

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

481

# Alternatively, you can use the following legacy machine types:

#

# - `standard`

# - `large_model`

# - `complex_model_s`

486

# - `complex_model_m`

487

# - `complex_model_l`

488

# - `standard_gpu`

489

# - `complex_model_m_gpu`

490

# - `complex_model_l_gpu`

491

# - `standard_p100`

492

# - `complex_model_m_p100`

493

# - `standard_v100`

494

# - `large_model_v100`

495

# - `complex_model_m_v100`

496

# - `complex_model_l_v100`

497

#

498

# Learn more about [using legacy machine

499

# types](/ml-engine/docs/machine-types#legacy-machine-types).

500

#

501

# Finally, if you want to use a TPU for training, specify `cloud_tpu` in this

502

# field. Learn more about the [special configuration options for training

503

# with

504

# TPUs](/ml-engine/docs/tensorflow/using-tpus#configuring_a_custom_tpu_machine).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

505

"runtimeVersion": "A String", # Optional. The AI Platform runtime version to use for training. You must

506

# either specify this field or specify `masterConfig.imageUri`.

507

#

508

# For more information, see the [runtime version

509

# list](/ai-platform/training/docs/runtime-version-list) and learn [how to

510

# manage runtime versions](/ai-platform/training/docs/versioning).

511

"evaluatorType": "A String", # Optional. Specifies the type of virtual machine to use for your training

512

# job's evaluator nodes.

513

#

514

# The supported values are the same as those described in the entry for

515

# `masterType`.

516

#

517

# This value must be consistent with the category of machine type that

518

# `masterType` uses. In other words, both must be Compute Engine machine

519

# types or both must be legacy machine types.

520

#

521

# This value must be present when `scaleTier` is set to `CUSTOM` and

522

# `evaluatorCount` is greater than zero.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

523

"workerType": "A String", # Optional. Specifies the type of virtual machine to use for your training

524

# job's worker nodes.

525

#

526

# The supported values are the same as those described in the entry for

527

# `masterType`.

528

#

529

# This value must be consistent with the category of machine type that

530

# `masterType` uses. In other words, both must be Compute Engine machine

531

# types or both must be legacy machine types.

532

#

533

# If you use `cloud_tpu` for this value, see special instructions for

534

# [configuring a custom TPU

535

# machine](/ml-engine/docs/tensorflow/using-tpus#configuring_a_custom_tpu_machine).

536

#

537

# This value must be present when `scaleTier` is set to `CUSTOM` and

538

# `workerCount` is greater than zero.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

539

"region": "A String", # Required. The region to run the training job in. See the [available

540

# regions](/ai-platform/training/docs/regions) for AI Platform Training.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

541

"parameterServerType": "A String", # Optional. Specifies the type of virtual machine to use for your training

542

# job's parameter server.

543

#

544

# The supported values are the same as those described in the entry for

545

# `master_type`.

546

#

547

# This value must be consistent with the category of machine type that

548

# `masterType` uses. In other words, both must be Compute Engine machine

549

# types or both must be legacy machine types.

550

#

551

# This value must be present when `scaleTier` is set to `CUSTOM` and

552

# `parameter_server_count` is greater than zero.

553

"masterConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for your master worker.

554

#

555

# You should only set `masterConfig.acceleratorConfig` if `masterType` is set

556

# to a Compute Engine machine type. Learn about [restrictions on accelerator

557

# configurations for

558

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

559

#

560

# Set `masterConfig.imageUri` only if you build a custom image. Only one of

561

# `masterConfig.imageUri` and `runtimeVersion` should be set. Learn more

562

# about [configuring custom

563

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

564

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

565

# Registry. Learn more about [configuring custom

566

# containers](/ai-platform/training/docs/distributed-training-containers).

567

"containerArgs": [ # Arguments to the entrypoint command.

568

# The following rules apply for container_command and container_args:

569

# - If you do not supply command or args:

570

# The defaults defined in the Docker image are used.

571

# - If you supply a command but no args:

572

# The default EntryPoint and the default Cmd defined in the Docker image

573

# are ignored. Your command is run without any arguments.

574

# - If you supply only args:

575

# The default Entrypoint defined in the Docker image is run with the args

576

# that you supplied.

577

# - If you supply a command and args:

578

# The default Entrypoint and the default Cmd defined in the Docker image

579

# are ignored. Your command is run with your args.

580

# It cannot be set if custom container image is

581

# not provided.

582

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

583

# both cannot be set at the same time.

584

"A String",

585

],

586

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

587

# [Learn about restrictions on accelerator configurations for

588

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

589

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

590

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

591

# [accelerators for online

592

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

593

"count": "A String", # The number of accelerators to attach to each machine running the job.

594

"type": "A String", # The type of accelerator to use.

595

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

596

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

597

# the one used in the custom container. This field is required if the replica

598

# is a TPU worker that uses a custom container. Otherwise, do not specify

599

# this field. This must be a [runtime version that currently supports

600

# training with

601

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

602

#

603

# Note that the version of TensorFlow included in a runtime version may

604

# differ from the numbering of the runtime version itself, because it may

605

# have a different [patch

606

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

607

# In this field, you must specify the runtime version (TensorFlow minor

608

# version). For example, if your custom container runs TensorFlow `1.x.y`,

609

# specify `1.x`.

610

"containerCommand": [ # The command with which the replica's custom container is run.

611

# If provided, it will override default ENTRYPOINT of the docker image.

612

# If not provided, the docker image's ENTRYPOINT is used.

613

# It cannot be set if custom container image is

614

# not provided.

615

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

616

# both cannot be set at the same time.

617

"A String",

618

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

619

},

620

"scaleTier": "A String", # Required. Specifies the machine types, the number of replicas for workers

621

# and parameter servers.

622

"jobDir": "A String", # Optional. A Google Cloud Storage path in which to store training outputs

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

623

# and other data needed for training. This path is passed to your TensorFlow

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

624

# program as the '--job-dir' command-line argument. The benefit of specifying

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

625

# this field is that Cloud ML validates the path for use in training.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

626

"pythonVersion": "A String", # Optional. The version of Python used in training. You must either specify

627

# this field or specify `masterConfig.imageUri`.

628

#

629

# The following Python versions are available:

630

#

631

# * Python '3.7' is available when `runtime_version` is set to '1.15' or

632

# later.

633

# * Python '3.5' is available when `runtime_version` is set to a version

634

# from '1.4' to '1.14'.

635

# * Python '2.7' is available when `runtime_version` is set to '1.15' or

636

# earlier.

637

#

638

# Read more about the Python versions available for [each runtime

639

# version](/ml-engine/docs/runtime-version-list).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

640

"scheduling": { # All parameters related to scheduling of training jobs. # Optional. Scheduling options for a training job.

641

"maxWaitTime": "A String",

642

"maxRunningTime": "A String", # Optional. The maximum job running time, expressed in seconds. The field can

643

# contain up to nine fractional digits, terminated by `s`. If not specified,

644

# this field defaults to `604800s` (seven days).

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

645

#

646

# If the training job is still running after this duration, AI Platform

647

# Training cancels it.

648

#

649

# For example, if you want to ensure your job runs for no more than 2 hours,

650

# set this field to `7200s` (2 hours * 60 minutes / hour * 60 seconds /

651

# minute).

652

#

653

# If you submit your training job using the `gcloud` tool, you can [provide

654

# this field in a `config.yaml`

655

# file](/ai-platform/training/docs/training-jobs#formatting_your_configuration_parameters).

# For example:

#

# ```yaml

# trainingInput:

# ...

# scheduling:

# maxRunningTime: 7200s

663

# ...

664

# ```

665

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

666

"network": "A String", # Optional. The full name of the Google Compute Engine

667

# [network](/compute/docs/networks-and-firewalls#networks) to which the Job

668

# is peered. For example, projects/12345/global/networks/myVPC. Format is of

669

# the form projects/{project}/global/networks/{network}. Where {project} is a

670

# project number, as in '12345', and {network} is network name.".

671

#

672

# Private services access must already be configured for the network. If left

673

# unspecified, the Job is not peered with any network. Learn more -

674

# Connecting Job to user network over private

675

# IP.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

676

"evaluatorConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for evaluators.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

677

#

678

# You should only set `evaluatorConfig.acceleratorConfig` if

679

# `evaluatorType` is set to a Compute Engine machine type. [Learn

680

# about restrictions on accelerator configurations for

681

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

682

#

683

# Set `evaluatorConfig.imageUri` only if you build a custom image for

684

# your evaluator. If `evaluatorConfig.imageUri` has not been

685

# set, AI Platform uses the value of `masterConfig.imageUri`. Learn more about [configuring custom

686

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

687

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

688

# Registry. Learn more about [configuring custom

689

# containers](/ai-platform/training/docs/distributed-training-containers).

690

"containerArgs": [ # Arguments to the entrypoint command.

691

# The following rules apply for container_command and container_args:

692

# - If you do not supply command or args:

693

# The defaults defined in the Docker image are used.

694

# - If you supply a command but no args:

695

# The default EntryPoint and the default Cmd defined in the Docker image

696

# are ignored. Your command is run without any arguments.

697

# - If you supply only args:

698

# The default Entrypoint defined in the Docker image is run with the args

699

# that you supplied.

700

# - If you supply a command and args:

701

# The default Entrypoint and the default Cmd defined in the Docker image

702

# are ignored. Your command is run with your args.

703

# It cannot be set if custom container image is

704

# not provided.

705

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

706

# both cannot be set at the same time.

707

"A String",

708

],

709

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

710

# [Learn about restrictions on accelerator configurations for

711

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

712

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

713

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

714

# [accelerators for online

715

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

716

"count": "A String", # The number of accelerators to attach to each machine running the job.

717

"type": "A String", # The type of accelerator to use.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

718

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

719

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

720

# the one used in the custom container. This field is required if the replica

721

# is a TPU worker that uses a custom container. Otherwise, do not specify

722

# this field. This must be a [runtime version that currently supports

723

# training with

724

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

725

#

726

# Note that the version of TensorFlow included in a runtime version may

727

# differ from the numbering of the runtime version itself, because it may

728

# have a different [patch

729

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

730

# In this field, you must specify the runtime version (TensorFlow minor

731

# version). For example, if your custom container runs TensorFlow `1.x.y`,

732

# specify `1.x`.

733

"containerCommand": [ # The command with which the replica's custom container is run.

734

# If provided, it will override default ENTRYPOINT of the docker image.

735

# If not provided, the docker image's ENTRYPOINT is used.

736

# It cannot be set if custom container image is

737

# not provided.

738

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

739

# both cannot be set at the same time.

740

"A String",

741

],

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

742

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

743

"useChiefInTfConfig": True or False, # Optional. Use `chief` instead of `master` in the `TF_CONFIG` environment

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

744

# variable when training with a custom container. Defaults to `false`. [Learn

745

# more about this

746

# field.](/ai-platform/training/docs/distributed-training-details#chief-versus-master)

747

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

748

# This field has no effect for training jobs that don't use a custom

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

749

# container.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

750

},

751

"state": "A String", # Output only. The detailed state of a job.

752

"jobId": "A String", # Required. The user-specified id of the job.

753

"endTime": "A String", # Output only. When the job processing was completed.

754

"startTime": "A String", # Output only. When the job processing was started.

755

"predictionOutput": { # Represents results of a prediction job. # The current prediction job result.

756

"errorCount": "A String", # The number of data instances which resulted in errors.

757

"outputPath": "A String", # The output Google Cloud Storage location provided at the job creation time.

758

"nodeHours": 3.14, # Node hours used by the batch prediction job.

759

"predictionCount": "A String", # The number of generated predictions.

760

},

761

"trainingOutput": { # Represents results of a training job. Output only. # The current training job result.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

762

"isBuiltInAlgorithmJob": True or False, # Whether this job is a built-in Algorithm job.

763

"builtInAlgorithmOutput": { # Represents output related to a built-in algorithm Job. # Details related to built-in algorithms jobs.

764

# Only set for built-in algorithms jobs.

765

"modelPath": "A String", # The Cloud Storage path to the `model/` directory where the training job

766

# saves the trained model. Only set for successful jobs that don't use

767

# hyperparameter tuning.

768

"pythonVersion": "A String", # Python version on which the built-in algorithm was trained.

769

"runtimeVersion": "A String", # AI Platform runtime version on which the built-in algorithm was

770

# trained.

771

"framework": "A String", # Framework on which the built-in algorithm was trained.

772

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

773

"trials": [ # Results for individual Hyperparameter trials.

774

# Only set for hyperparameter tuning jobs.

775

{ # Represents the result of a single hyperparameter tuning trial from a

776

# training job. The TrainingOutput object that is returned on successful

777

# completion of a training job with hyperparameter tuning includes a list

778

# of HyperparameterOutput objects, one for each successful trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

779

"trialId": "A String", # The trial id for these results.

780

"endTime": "A String", # Output only. End time for the trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

781

"startTime": "A String", # Output only. Start time for the trial.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

782

"isTrialStoppedEarly": True or False, # True if the trial is stopped early.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

783

"finalMetric": { # An observed value of a metric. # The final objective metric seen for this trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

784

"trainingStep": "A String", # The global training step for this metric.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

785

"objectiveValue": 3.14, # The objective value at this training step.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

786

},

787

"builtInAlgorithmOutput": { # Represents output related to a built-in algorithm Job. # Details related to built-in algorithms jobs.

788

# Only set for trials of built-in algorithms jobs that have succeeded.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

789

"modelPath": "A String", # The Cloud Storage path to the `model/` directory where the training job

790

# saves the trained model. Only set for successful jobs that don't use

791

# hyperparameter tuning.

792

"pythonVersion": "A String", # Python version on which the built-in algorithm was trained.

793

"runtimeVersion": "A String", # AI Platform runtime version on which the built-in algorithm was

794

# trained.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

795

"framework": "A String", # Framework on which the built-in algorithm was trained.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

796

},

797

"state": "A String", # Output only. The detailed state of the trial.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

798

"allMetrics": [ # All recorded object metrics for this trial. This field is not currently

799

# populated.

800

{ # An observed value of a metric.

801

"trainingStep": "A String", # The global training step for this metric.

802

"objectiveValue": 3.14, # The objective value at this training step.

803

},

804

],

805

"hyperparameters": { # The hyperparameters given to this trial.

806

"a_key": "A String",

807

},

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

808

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

809

],

810

"hyperparameterMetricTag": "A String", # The TensorFlow summary tag name used for optimizing hyperparameter tuning

811

# trials. See

812

# [`HyperparameterSpec.hyperparameterMetricTag`](#HyperparameterSpec.FIELDS.hyperparameter_metric_tag)

813

# for more information. Only set for hyperparameter tuning jobs.

814

"completedTrialCount": "A String", # The number of hyperparameter tuning trials that completed successfully.

815

# Only set for hyperparameter tuning jobs.

816

"isHyperparameterTuningJob": True or False, # Whether this job is a hyperparameter tuning job.

817

"consumedMLUnits": 3.14, # The amount of ML units consumed by the job.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

818

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

819

"createTime": "A String", # Output only. When the job was created.

820

"labels": { # Optional. One or more labels that you can add, to organize your jobs.

821

# Each label is a key-value pair, where both the key and the value are

822

# arbitrary strings that you supply.

823

# For more information, see the documentation on

824

# <a href="/ml-engine/docs/tensorflow/resource-labels">using labels</a>.

825

"a_key": "A String",

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

826

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

827

}

828

829

x__xgafv: string, V1 error format.

Allowed values

1 - v1 error format

2 - v2 error format

Returns:

An object of the form:

836

837

{ # Represents a training or prediction job.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

838

"predictionInput": { # Represents input parameters for a prediction job. # Input parameters to create a prediction job.

839

"versionName": "A String", # Use this field if you want to specify a version of the model to use. The

840

# string is formatted the same way as `model_version`, with the addition

841

# of the version information:

842

#

843

# `"projects/YOUR_PROJECT/models/YOUR_MODEL/versions/YOUR_VERSION"`

844

"modelName": "A String", # Use this field if you want to use the default version for the specified

845

# model. The string must use the following format:

846

#

847

# `"projects/YOUR_PROJECT/models/YOUR_MODEL"`

848

"uri": "A String", # Use this field if you want to specify a Google Cloud Storage path for

849

# the model to use.

850

"maxWorkerCount": "A String", # Optional. The maximum number of workers to be used for parallel processing.

851

# Defaults to 10 if not specified.

852

"signatureName": "A String", # Optional. The name of the signature defined in the SavedModel to use for

853

# this job. Please refer to

854

# [SavedModel](https://tensorflow.github.io/serving/serving_basic.html)

855

# for information about how to use signatures.

856

#

857

# Defaults to

858

# [DEFAULT_SERVING_SIGNATURE_DEF_KEY](https://www.tensorflow.org/api_docs/python/tf/saved_model/signature_constants)

859

# , which is "serving_default".

860

"outputPath": "A String", # Required. The output Google Cloud Storage location.

861

"outputDataFormat": "A String", # Optional. Format of the output data files, defaults to JSON.

862

"dataFormat": "A String", # Required. The format of the input data files.

863

"batchSize": "A String", # Optional. Number of records per batch, defaults to 64.

864

# The service will buffer batch_size number of records in memory before

865

# invoking one Tensorflow prediction call internally. So take the record

866

# size and memory available into consideration when setting this parameter.

867

"runtimeVersion": "A String", # Optional. The AI Platform runtime version to use for this batch

868

# prediction. If not set, AI Platform will pick the runtime version used

869

# during the CreateVersion request for this model version, or choose the

870

# latest stable version when model version information is not available

871

# such as when the model is specified by uri.

872

"inputPaths": [ # Required. The Cloud Storage location of the input data files. May contain

873

# <a href="/storage/docs/gsutil/addlhelp/WildcardNames">wildcards</a>.

874

"A String",

875

],

876

"region": "A String", # Required. The Google Compute Engine region to run the prediction job in.

877

# See the <a href="/ml-engine/docs/tensorflow/regions">available regions</a>

878

# for AI Platform services.

879

},

880

"errorMessage": "A String", # Output only. The details of a failure or a cancellation.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

881

"etag": "A String", # `etag` is used for optimistic concurrency control as a way to help

882

# prevent simultaneous updates of a job from overwriting each other.

883

# It is strongly suggested that systems make use of the `etag` in the

884

# read-modify-write cycle to perform job updates in order to avoid race

885

# conditions: An `etag` is returned in the response to `GetJob`, and

886

# systems are expected to put that etag in the request to `UpdateJob` to

887

# ensure that their change will be applied to the same version of the job.

888

"trainingInput": { # Represents input parameters for a training job. When using the gcloud command # Input parameters to create a training job.

889

# to submit your training job, you can specify the input parameters as

890

# command-line arguments and/or in a YAML configuration file referenced from

891

# the --config command-line argument. For details, see the guide to [submitting

892

# a training job](/ai-platform/training/docs/training-jobs).

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

893

"workerCount": "A String", # Optional. The number of worker replicas to use for the training job. Each

894

# replica in the cluster will be of the type specified in `worker_type`.

895

#

896

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

897

# set this value, you must also set `worker_type`.

898

#

899

# The default value is zero.

900

"pythonModule": "A String", # Required. The Python module name to run after installing the packages.

901

"args": [ # Optional. Command-line arguments passed to the training application when it

902

# starts. If your job uses a custom container, then the arguments are passed

903

# to the container's <a class="external" target="_blank"

904

# href="https://docs.docker.com/engine/reference/builder/#entrypoint">

905

# `ENTRYPOINT`</a> command.

906

"A String",

907

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

908

"parameterServerConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for parameter servers.

909

#

910

# You should only set `parameterServerConfig.acceleratorConfig` if

911

# `parameterServerType` is set to a Compute Engine machine type. [Learn

912

# about restrictions on accelerator configurations for

913

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

914

#

915

# Set `parameterServerConfig.imageUri` only if you build a custom image for

916

# your parameter server. If `parameterServerConfig.imageUri` has not been

917

# set, AI Platform uses the value of `masterConfig.imageUri`. Learn more about [configuring custom

918

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

919

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

920

# Registry. Learn more about [configuring custom

921

# containers](/ai-platform/training/docs/distributed-training-containers).

922

"containerArgs": [ # Arguments to the entrypoint command.

923

# The following rules apply for container_command and container_args:

924

# - If you do not supply command or args:

925

# The defaults defined in the Docker image are used.

926

# - If you supply a command but no args:

927

# The default EntryPoint and the default Cmd defined in the Docker image

928

# are ignored. Your command is run without any arguments.

929

# - If you supply only args:

930

# The default Entrypoint defined in the Docker image is run with the args

931

# that you supplied.

932

# - If you supply a command and args:

933

# The default Entrypoint and the default Cmd defined in the Docker image

934

# are ignored. Your command is run with your args.

935

# It cannot be set if custom container image is

936

# not provided.

937

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

938

# both cannot be set at the same time.

939

"A String",

940

],

941

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

942

# [Learn about restrictions on accelerator configurations for

943

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

944

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

945

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

946

# [accelerators for online

947

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

948

"count": "A String", # The number of accelerators to attach to each machine running the job.

949

"type": "A String", # The type of accelerator to use.

950

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

951

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

952

# the one used in the custom container. This field is required if the replica

953

# is a TPU worker that uses a custom container. Otherwise, do not specify

954

# this field. This must be a [runtime version that currently supports

955

# training with

956

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

957

#

958

# Note that the version of TensorFlow included in a runtime version may

959

# differ from the numbering of the runtime version itself, because it may

960

# have a different [patch

961

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

962

# In this field, you must specify the runtime version (TensorFlow minor

963

# version). For example, if your custom container runs TensorFlow `1.x.y`,

964

# specify `1.x`.

965

"containerCommand": [ # The command with which the replica's custom container is run.

966

# If provided, it will override default ENTRYPOINT of the docker image.

967

# If not provided, the docker image's ENTRYPOINT is used.

968

# It cannot be set if custom container image is

969

# not provided.

970

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

971

# both cannot be set at the same time.

972

"A String",

973

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

974

},

975

"encryptionConfig": { # Represents a custom encryption key configuration that can be applied to # Optional. Options for using customer-managed encryption keys (CMEK) to

976

# protect resources created by a training job, instead of using Google's

977

# default encryption. If this is set, then all resources created by the

978

# training job will be encrypted with the customer-managed encryption key

979

# that you specify.

980

#

981

# [Learn how and when to use CMEK with AI Platform

982

# Training](/ai-platform/training/docs/cmek).

983

# a resource.

984

"kmsKeyName": "A String", # The Cloud KMS resource identifier of the customer-managed encryption key

985

# used to protect a resource, such as a training job. It has the following

986

# format:

987

# `projects/{PROJECT_ID}/locations/{REGION}/keyRings/{KEY_RING_NAME}/cryptoKeys/{KEY_NAME}`

988

},

989

"hyperparameters": { # Represents a set of hyperparameters to optimize. # Optional. The set of Hyperparameters to tune.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

990

"params": [ # Required. The set of parameters to tune.

991

{ # Represents a single hyperparameter to optimize.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

992

"categoricalValues": [ # Required if type is `CATEGORICAL`. The list of possible categories.

993

"A String",

994

],

995

"parameterName": "A String", # Required. The parameter name must be unique amongst all ParameterConfigs in

996

# a HyperparameterSpec message. E.g., "learning_rate".

997

"minValue": 3.14, # Required if type is `DOUBLE` or `INTEGER`. This field

998

# should be unset if type is `CATEGORICAL`. This value should be integers if

999

# type is INTEGER.

1000

"discreteValues": [ # Required if type is `DISCRETE`.

1001

# A list of feasible points.

1002

# The list should be in strictly increasing order. For instance, this

1003

# parameter might have possible settings of 1.5, 2.5, and 4.0. This list

1004

# should not contain more than 1,000 values.

1005

3.14,

1006

],

1007

"scaleType": "A String", # Optional. How the parameter should be scaled to the hypercube.

1008

# Leave unset for categorical parameters.

1009

# Some kind of scaling is strongly recommended for real or integral

1010

# parameters (e.g., `UNIT_LINEAR_SCALE`).

1011

"maxValue": 3.14, # Required if type is `DOUBLE` or `INTEGER`. This field

1012

# should be unset if type is `CATEGORICAL`. This value should be integers if

1013

# type is `INTEGER`.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1014

"type": "A String", # Required. The type of the parameter.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1015

},

1016

],

1017

"enableTrialEarlyStopping": True or False, # Optional. Indicates if the hyperparameter tuning job enables auto trial

1018

# early stopping.

1019

"resumePreviousJobId": "A String", # Optional. The prior hyperparameter tuning job id that users hope to

1020

# continue with. The job id will be used to find the corresponding vizier

1021

# study guid and resume the study.

1022

"maxParallelTrials": 42, # Optional. The number of training trials to run concurrently.

1023

# You can reduce the time it takes to perform hyperparameter tuning by adding

1024

# trials in parallel. However, each trail only benefits from the information

1025

# gained in completed trials. That means that a trial does not get access to

1026

# the results of trials running at the same time, which could reduce the

1027

# quality of the overall optimization.

1028

#

1029

# Each trial will use the same scale tier and machine types.

1030

#

1031

# Defaults to one.

1032

"maxFailedTrials": 42, # Optional. The number of failed trials that need to be seen before failing

1033

# the hyperparameter tuning job. You can specify this field to override the

1034

# default failing criteria for AI Platform hyperparameter tuning jobs.

1035

#

1036

# Defaults to zero, which means the service decides when a hyperparameter

1037

# job should fail.

1038

"goal": "A String", # Required. The type of goal to use for tuning. Available types are

1039

# `MAXIMIZE` and `MINIMIZE`.

1040

#

1041

# Defaults to `MAXIMIZE`.

1042

"maxTrials": 42, # Optional. How many training trials should be attempted to optimize

1043

# the specified hyperparameters.

1044

#

1045

# Defaults to one.

1046

"algorithm": "A String", # Optional. The search algorithm specified for the hyperparameter

1047

# tuning job.

1048

# Uses the default AI Platform hyperparameter tuning

1049

# algorithm if unspecified.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1050

"hyperparameterMetricTag": "A String", # Optional. The TensorFlow summary tag name to use for optimizing trials. For

1051

# current versions of TensorFlow, this tag name should exactly match what is

1052

# shown in TensorBoard, including all scopes. For versions of TensorFlow

1053

# prior to 0.12, this should be only the tag passed to tf.Summary.

1054

# By default, "training/hptuning/metric" will be used.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1055

},

1056

"workerConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for workers.

1057

#

1058

# You should only set `workerConfig.acceleratorConfig` if `workerType` is set

1059

# to a Compute Engine machine type. [Learn about restrictions on accelerator

1060

# configurations for

1061

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

1062

#

1063

# Set `workerConfig.imageUri` only if you build a custom image for your

1064

# worker. If `workerConfig.imageUri` has not been set, AI Platform uses

1065

# the value of `masterConfig.imageUri`. Learn more about [configuring custom

1066

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1067

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

1068

# Registry. Learn more about [configuring custom

1069

# containers](/ai-platform/training/docs/distributed-training-containers).

1070

"containerArgs": [ # Arguments to the entrypoint command.

1071

# The following rules apply for container_command and container_args:

1072

# - If you do not supply command or args:

1073

# The defaults defined in the Docker image are used.

1074

# - If you supply a command but no args:

1075

# The default EntryPoint and the default Cmd defined in the Docker image

1076

# are ignored. Your command is run without any arguments.

1077

# - If you supply only args:

1078

# The default Entrypoint defined in the Docker image is run with the args

1079

# that you supplied.

1080

# - If you supply a command and args:

1081

# The default Entrypoint and the default Cmd defined in the Docker image

1082

# are ignored. Your command is run with your args.

1083

# It cannot be set if custom container image is

1084

# not provided.

1085

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

1086

# both cannot be set at the same time.

1087

"A String",

1088

],

1089

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

1090

# [Learn about restrictions on accelerator configurations for

1091

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

1092

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

1093

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

1094

# [accelerators for online

1095

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

1096

"count": "A String", # The number of accelerators to attach to each machine running the job.

1097

"type": "A String", # The type of accelerator to use.

1098

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1099

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

1100

# the one used in the custom container. This field is required if the replica

1101

# is a TPU worker that uses a custom container. Otherwise, do not specify

1102

# this field. This must be a [runtime version that currently supports

1103

# training with

1104

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

1105

#

1106

# Note that the version of TensorFlow included in a runtime version may

1107

# differ from the numbering of the runtime version itself, because it may

1108

# have a different [patch

1109

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

1110

# In this field, you must specify the runtime version (TensorFlow minor

1111

# version). For example, if your custom container runs TensorFlow `1.x.y`,

1112

# specify `1.x`.

1113

"containerCommand": [ # The command with which the replica's custom container is run.

1114

# If provided, it will override default ENTRYPOINT of the docker image.

1115

# If not provided, the docker image's ENTRYPOINT is used.

1116

# It cannot be set if custom container image is

1117

# not provided.

1118

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

1119

# both cannot be set at the same time.

1120

"A String",

1121

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1122

},

1123

"parameterServerCount": "A String", # Optional. The number of parameter server replicas to use for the training

1124

# job. Each replica in the cluster will be of the type specified in

1125

# `parameter_server_type`.

1126

#

1127

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

1128

# set this value, you must also set `parameter_server_type`.

1129

#

1130

# The default value is zero.

1131

"packageUris": [ # Required. The Google Cloud Storage location of the packages with

1132

# the training program and any additional dependencies.

1133

# The maximum number of package URIs is 100.

1134

"A String",

1135

],

1136

"evaluatorCount": "A String", # Optional. The number of evaluator replicas to use for the training job.

1137

# Each replica in the cluster will be of the type specified in

1138

# `evaluator_type`.

1139

#

1140

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

1141

# set this value, you must also set `evaluator_type`.

1142

#

1143

# The default value is zero.

1144

"masterType": "A String", # Optional. Specifies the type of virtual machine to use for your training

1145

# job's master worker. You must specify this field when `scaleTier` is set to

1146

# `CUSTOM`.

1147

#

1148

# You can use certain Compute Engine machine types directly in this field.

1149

# The following types are supported:

#

# - `n1-standard-4`

# - `n1-standard-8`

# - `n1-standard-16`

# - `n1-standard-32`

# - `n1-standard-64`

# - `n1-standard-96`

# - `n1-highmem-2`

# - `n1-highmem-4`

# - `n1-highmem-8`

# - `n1-highmem-16`

# - `n1-highmem-32`

# - `n1-highmem-64`

# - `n1-highmem-96`

# - `n1-highcpu-16`

# - `n1-highcpu-32`

# - `n1-highcpu-64`

# - `n1-highcpu-96`

#

# Learn more about [using Compute Engine machine

1170

# types](/ml-engine/docs/machine-types#compute-engine-machine-types).

1171

#

1172

# Alternatively, you can use the following legacy machine types:

#

# - `standard`

# - `large_model`

# - `complex_model_s`

1177

# - `complex_model_m`

1178

# - `complex_model_l`

1179

# - `standard_gpu`

1180

# - `complex_model_m_gpu`

1181

# - `complex_model_l_gpu`

1182

# - `standard_p100`

1183

# - `complex_model_m_p100`

1184

# - `standard_v100`

1185

# - `large_model_v100`

1186

# - `complex_model_m_v100`

1187

# - `complex_model_l_v100`

1188

#

1189

# Learn more about [using legacy machine

1190

# types](/ml-engine/docs/machine-types#legacy-machine-types).

1191

#

1192

# Finally, if you want to use a TPU for training, specify `cloud_tpu` in this

1193

# field. Learn more about the [special configuration options for training

1194

# with

1195

# TPUs](/ml-engine/docs/tensorflow/using-tpus#configuring_a_custom_tpu_machine).

1196

"runtimeVersion": "A String", # Optional. The AI Platform runtime version to use for training. You must

1197

# either specify this field or specify `masterConfig.imageUri`.

1198

#

1199

# For more information, see the [runtime version

1200

# list](/ai-platform/training/docs/runtime-version-list) and learn [how to

1201

# manage runtime versions](/ai-platform/training/docs/versioning).

1202

"evaluatorType": "A String", # Optional. Specifies the type of virtual machine to use for your training

1203

# job's evaluator nodes.

1204

#

1205

# The supported values are the same as those described in the entry for

1206

# `masterType`.

1207

#

1208

# This value must be consistent with the category of machine type that

1209

# `masterType` uses. In other words, both must be Compute Engine machine

1210

# types or both must be legacy machine types.

1211

#

1212

# This value must be present when `scaleTier` is set to `CUSTOM` and

1213

# `evaluatorCount` is greater than zero.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1214

"workerType": "A String", # Optional. Specifies the type of virtual machine to use for your training

1215

# job's worker nodes.

1216

#

1217

# The supported values are the same as those described in the entry for

1218

# `masterType`.

1219

#

1220

# This value must be consistent with the category of machine type that

1221

# `masterType` uses. In other words, both must be Compute Engine machine

1222

# types or both must be legacy machine types.

1223

#

1224

# If you use `cloud_tpu` for this value, see special instructions for

1225

# [configuring a custom TPU

1226

# machine](/ml-engine/docs/tensorflow/using-tpus#configuring_a_custom_tpu_machine).

1227

#

1228

# This value must be present when `scaleTier` is set to `CUSTOM` and

1229

# `workerCount` is greater than zero.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1230

"region": "A String", # Required. The region to run the training job in. See the [available

1231

# regions](/ai-platform/training/docs/regions) for AI Platform Training.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1232

"parameterServerType": "A String", # Optional. Specifies the type of virtual machine to use for your training

1233

# job's parameter server.

1234

#

1235

# The supported values are the same as those described in the entry for

1236

# `master_type`.

1237

#

1238

# This value must be consistent with the category of machine type that

1239

# `masterType` uses. In other words, both must be Compute Engine machine

1240

# types or both must be legacy machine types.

1241

#

1242

# This value must be present when `scaleTier` is set to `CUSTOM` and

1243

# `parameter_server_count` is greater than zero.

1244

"masterConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for your master worker.

1245

#

1246

# You should only set `masterConfig.acceleratorConfig` if `masterType` is set

1247

# to a Compute Engine machine type. Learn about [restrictions on accelerator

1248

# configurations for

1249

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

1250

#

1251

# Set `masterConfig.imageUri` only if you build a custom image. Only one of

1252

# `masterConfig.imageUri` and `runtimeVersion` should be set. Learn more

1253

# about [configuring custom

1254

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1255

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

1256

# Registry. Learn more about [configuring custom

1257

# containers](/ai-platform/training/docs/distributed-training-containers).

1258

"containerArgs": [ # Arguments to the entrypoint command.

1259

# The following rules apply for container_command and container_args:

1260

# - If you do not supply command or args:

1261

# The defaults defined in the Docker image are used.

1262

# - If you supply a command but no args:

1263

# The default EntryPoint and the default Cmd defined in the Docker image

1264

# are ignored. Your command is run without any arguments.

1265

# - If you supply only args:

1266

# The default Entrypoint defined in the Docker image is run with the args

1267

# that you supplied.

1268

# - If you supply a command and args:

1269

# The default Entrypoint and the default Cmd defined in the Docker image

1270

# are ignored. Your command is run with your args.

1271

# It cannot be set if custom container image is

1272

# not provided.

1273

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

1274

# both cannot be set at the same time.

1275

"A String",

1276

],

1277

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

1278

# [Learn about restrictions on accelerator configurations for

1279

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

1280

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

1281

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

1282

# [accelerators for online

1283

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

1284

"count": "A String", # The number of accelerators to attach to each machine running the job.

1285

"type": "A String", # The type of accelerator to use.

1286

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1287

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

1288

# the one used in the custom container. This field is required if the replica

1289

# is a TPU worker that uses a custom container. Otherwise, do not specify

1290

# this field. This must be a [runtime version that currently supports

1291

# training with

1292

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

1293

#

1294

# Note that the version of TensorFlow included in a runtime version may

1295

# differ from the numbering of the runtime version itself, because it may

1296

# have a different [patch

1297

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

1298

# In this field, you must specify the runtime version (TensorFlow minor

1299

# version). For example, if your custom container runs TensorFlow `1.x.y`,

1300

# specify `1.x`.

1301

"containerCommand": [ # The command with which the replica's custom container is run.

1302

# If provided, it will override default ENTRYPOINT of the docker image.

1303

# If not provided, the docker image's ENTRYPOINT is used.

1304

# It cannot be set if custom container image is

1305

# not provided.

1306

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

1307

# both cannot be set at the same time.

1308

"A String",

1309

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1310

},

1311

"scaleTier": "A String", # Required. Specifies the machine types, the number of replicas for workers

1312

# and parameter servers.

1313

"jobDir": "A String", # Optional. A Google Cloud Storage path in which to store training outputs

1314

# and other data needed for training. This path is passed to your TensorFlow

1315

# program as the '--job-dir' command-line argument. The benefit of specifying

1316

# this field is that Cloud ML validates the path for use in training.

1317

"pythonVersion": "A String", # Optional. The version of Python used in training. You must either specify

1318

# this field or specify `masterConfig.imageUri`.

1319

#

1320

# The following Python versions are available:

1321

#

1322

# * Python '3.7' is available when `runtime_version` is set to '1.15' or

1323

# later.

1324

# * Python '3.5' is available when `runtime_version` is set to a version

1325

# from '1.4' to '1.14'.

1326

# * Python '2.7' is available when `runtime_version` is set to '1.15' or

1327

# earlier.

1328

#

1329

# Read more about the Python versions available for [each runtime

1330

# version](/ml-engine/docs/runtime-version-list).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1331

"scheduling": { # All parameters related to scheduling of training jobs. # Optional. Scheduling options for a training job.

1332

"maxWaitTime": "A String",

1333

"maxRunningTime": "A String", # Optional. The maximum job running time, expressed in seconds. The field can

1334

# contain up to nine fractional digits, terminated by `s`. If not specified,

1335

# this field defaults to `604800s` (seven days).

1336

#

1337

# If the training job is still running after this duration, AI Platform

1338

# Training cancels it.

1339

#

1340

# For example, if you want to ensure your job runs for no more than 2 hours,

1341

# set this field to `7200s` (2 hours * 60 minutes / hour * 60 seconds /

1342

# minute).

1343

#

1344

# If you submit your training job using the `gcloud` tool, you can [provide

1345

# this field in a `config.yaml`

1346

# file](/ai-platform/training/docs/training-jobs#formatting_your_configuration_parameters).

# For example:

#

# ```yaml

# trainingInput:

# ...

# scheduling:

# maxRunningTime: 7200s

1354

# ...

1355

# ```

1356

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1357

"network": "A String", # Optional. The full name of the Google Compute Engine

1358

# [network](/compute/docs/networks-and-firewalls#networks) to which the Job

1359

# is peered. For example, projects/12345/global/networks/myVPC. Format is of

1360

# the form projects/{project}/global/networks/{network}. Where {project} is a

1361

# project number, as in '12345', and {network} is network name.".

1362

#

1363

# Private services access must already be configured for the network. If left

1364

# unspecified, the Job is not peered with any network. Learn more -

1365

# Connecting Job to user network over private

1366

# IP.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1367

"evaluatorConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for evaluators.

1368

#

1369

# You should only set `evaluatorConfig.acceleratorConfig` if

1370

# `evaluatorType` is set to a Compute Engine machine type. [Learn

1371

# about restrictions on accelerator configurations for

1372

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

1373

#

1374

# Set `evaluatorConfig.imageUri` only if you build a custom image for

1375

# your evaluator. If `evaluatorConfig.imageUri` has not been

1376

# set, AI Platform uses the value of `masterConfig.imageUri`. Learn more about [configuring custom

1377

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1378

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

1379

# Registry. Learn more about [configuring custom

1380

# containers](/ai-platform/training/docs/distributed-training-containers).

1381

"containerArgs": [ # Arguments to the entrypoint command.

1382

# The following rules apply for container_command and container_args:

1383

# - If you do not supply command or args:

1384

# The defaults defined in the Docker image are used.

1385

# - If you supply a command but no args:

1386

# The default EntryPoint and the default Cmd defined in the Docker image

1387

# are ignored. Your command is run without any arguments.

1388

# - If you supply only args:

1389

# The default Entrypoint defined in the Docker image is run with the args

1390

# that you supplied.

1391

# - If you supply a command and args:

1392

# The default Entrypoint and the default Cmd defined in the Docker image

1393

# are ignored. Your command is run with your args.

1394

# It cannot be set if custom container image is

1395

# not provided.

1396

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

1397

# both cannot be set at the same time.

1398

"A String",

1399

],

1400

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

1401

# [Learn about restrictions on accelerator configurations for

1402

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

1403

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

1404

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

1405

# [accelerators for online

1406

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

1407

"count": "A String", # The number of accelerators to attach to each machine running the job.

1408

"type": "A String", # The type of accelerator to use.

1409

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1410

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

1411

# the one used in the custom container. This field is required if the replica

1412

# is a TPU worker that uses a custom container. Otherwise, do not specify

1413

# this field. This must be a [runtime version that currently supports

1414

# training with

1415

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

1416

#

1417

# Note that the version of TensorFlow included in a runtime version may

1418

# differ from the numbering of the runtime version itself, because it may

1419

# have a different [patch

1420

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

1421

# In this field, you must specify the runtime version (TensorFlow minor

1422

# version). For example, if your custom container runs TensorFlow `1.x.y`,

1423

# specify `1.x`.

1424

"containerCommand": [ # The command with which the replica's custom container is run.

1425

# If provided, it will override default ENTRYPOINT of the docker image.

1426

# If not provided, the docker image's ENTRYPOINT is used.

1427

# It cannot be set if custom container image is

1428

# not provided.

1429

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

1430

# both cannot be set at the same time.

1431

"A String",

1432

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1433

},

1434

"useChiefInTfConfig": True or False, # Optional. Use `chief` instead of `master` in the `TF_CONFIG` environment

1435

# variable when training with a custom container. Defaults to `false`. [Learn

1436

# more about this

1437

# field.](/ai-platform/training/docs/distributed-training-details#chief-versus-master)

1438

#

1439

# This field has no effect for training jobs that don't use a custom

1440

# container.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1441

},

1442

"state": "A String", # Output only. The detailed state of a job.

1443

"jobId": "A String", # Required. The user-specified id of the job.

1444

"endTime": "A String", # Output only. When the job processing was completed.

1445

"startTime": "A String", # Output only. When the job processing was started.

1446

"predictionOutput": { # Represents results of a prediction job. # The current prediction job result.

1447

"errorCount": "A String", # The number of data instances which resulted in errors.

1448

"outputPath": "A String", # The output Google Cloud Storage location provided at the job creation time.

1449

"nodeHours": 3.14, # Node hours used by the batch prediction job.

1450

"predictionCount": "A String", # The number of generated predictions.

1451

},

1452

"trainingOutput": { # Represents results of a training job. Output only. # The current training job result.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1453

"isBuiltInAlgorithmJob": True or False, # Whether this job is a built-in Algorithm job.

1454

"builtInAlgorithmOutput": { # Represents output related to a built-in algorithm Job. # Details related to built-in algorithms jobs.

1455

# Only set for built-in algorithms jobs.

1456

"modelPath": "A String", # The Cloud Storage path to the `model/` directory where the training job

1457

# saves the trained model. Only set for successful jobs that don't use

1458

# hyperparameter tuning.

1459

"pythonVersion": "A String", # Python version on which the built-in algorithm was trained.

1460

"runtimeVersion": "A String", # AI Platform runtime version on which the built-in algorithm was

1461

# trained.

1462

"framework": "A String", # Framework on which the built-in algorithm was trained.

1463

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1464

"trials": [ # Results for individual Hyperparameter trials.

1465

# Only set for hyperparameter tuning jobs.

1466

{ # Represents the result of a single hyperparameter tuning trial from a

1467

# training job. The TrainingOutput object that is returned on successful

1468

# completion of a training job with hyperparameter tuning includes a list

1469

# of HyperparameterOutput objects, one for each successful trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1470

"trialId": "A String", # The trial id for these results.

1471

"endTime": "A String", # Output only. End time for the trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1472

"startTime": "A String", # Output only. Start time for the trial.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1473

"isTrialStoppedEarly": True or False, # True if the trial is stopped early.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1474

"finalMetric": { # An observed value of a metric. # The final objective metric seen for this trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1475

"trainingStep": "A String", # The global training step for this metric.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1476

"objectiveValue": 3.14, # The objective value at this training step.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1477

},

1478

"builtInAlgorithmOutput": { # Represents output related to a built-in algorithm Job. # Details related to built-in algorithms jobs.

1479

# Only set for trials of built-in algorithms jobs that have succeeded.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1480

"modelPath": "A String", # The Cloud Storage path to the `model/` directory where the training job

1481

# saves the trained model. Only set for successful jobs that don't use

1482

# hyperparameter tuning.

1483

"pythonVersion": "A String", # Python version on which the built-in algorithm was trained.

1484

"runtimeVersion": "A String", # AI Platform runtime version on which the built-in algorithm was

1485

# trained.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1486

"framework": "A String", # Framework on which the built-in algorithm was trained.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1487

},

1488

"state": "A String", # Output only. The detailed state of the trial.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1489

"allMetrics": [ # All recorded object metrics for this trial. This field is not currently

1490

# populated.

1491

{ # An observed value of a metric.

1492

"trainingStep": "A String", # The global training step for this metric.

1493

"objectiveValue": 3.14, # The objective value at this training step.

1494

},

1495

],

1496

"hyperparameters": { # The hyperparameters given to this trial.

1497

"a_key": "A String",

1498

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1499

},

1500

],

1501

"hyperparameterMetricTag": "A String", # The TensorFlow summary tag name used for optimizing hyperparameter tuning

1502

# trials. See

1503

# [`HyperparameterSpec.hyperparameterMetricTag`](#HyperparameterSpec.FIELDS.hyperparameter_metric_tag)

1504

# for more information. Only set for hyperparameter tuning jobs.

1505

"completedTrialCount": "A String", # The number of hyperparameter tuning trials that completed successfully.

1506

# Only set for hyperparameter tuning jobs.

1507

"isHyperparameterTuningJob": True or False, # Whether this job is a hyperparameter tuning job.

1508

"consumedMLUnits": 3.14, # The amount of ML units consumed by the job.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1509

},

1510

"createTime": "A String", # Output only. When the job was created.

1511

"labels": { # Optional. One or more labels that you can add, to organize your jobs.

1512

# Each label is a key-value pair, where both the key and the value are

1513

# arbitrary strings that you supply.

1514

# For more information, see the documentation on

1515

# <a href="/ml-engine/docs/tensorflow/resource-labels">using labels</a>.

1516

"a_key": "A String",

1517

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1518

}</pre>

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

</div>

<code class="details" id="get">get(name, x__xgafv=None)</code>

1523

<pre>Describes a job.

1524

1525

Args:

1526

name: string, Required. The name of the job to get the description of. (required)

1527

x__xgafv: string, V1 error format.

Allowed values

1 - v1 error format

2 - v2 error format

Returns:

An object of the form:

1534

1535

{ # Represents a training or prediction job.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1536

"predictionInput": { # Represents input parameters for a prediction job. # Input parameters to create a prediction job.

1537

"versionName": "A String", # Use this field if you want to specify a version of the model to use. The

1538

# string is formatted the same way as `model_version`, with the addition

1539

# of the version information:

1540

#

1541

# `"projects/YOUR_PROJECT/models/YOUR_MODEL/versions/YOUR_VERSION"`

1542

"modelName": "A String", # Use this field if you want to use the default version for the specified

1543

# model. The string must use the following format:

1544

#

1545

# `"projects/YOUR_PROJECT/models/YOUR_MODEL"`

1546

"uri": "A String", # Use this field if you want to specify a Google Cloud Storage path for

1547

# the model to use.

1548

"maxWorkerCount": "A String", # Optional. The maximum number of workers to be used for parallel processing.

1549

# Defaults to 10 if not specified.

1550

"signatureName": "A String", # Optional. The name of the signature defined in the SavedModel to use for

1551

# this job. Please refer to

1552

# [SavedModel](https://tensorflow.github.io/serving/serving_basic.html)

1553

# for information about how to use signatures.

1554

#

1555

# Defaults to

1556

# [DEFAULT_SERVING_SIGNATURE_DEF_KEY](https://www.tensorflow.org/api_docs/python/tf/saved_model/signature_constants)

1557

# , which is "serving_default".

1558

"outputPath": "A String", # Required. The output Google Cloud Storage location.

1559

"outputDataFormat": "A String", # Optional. Format of the output data files, defaults to JSON.

1560

"dataFormat": "A String", # Required. The format of the input data files.

1561

"batchSize": "A String", # Optional. Number of records per batch, defaults to 64.

1562

# The service will buffer batch_size number of records in memory before

1563

# invoking one Tensorflow prediction call internally. So take the record

1564

# size and memory available into consideration when setting this parameter.

1565

"runtimeVersion": "A String", # Optional. The AI Platform runtime version to use for this batch

1566

# prediction. If not set, AI Platform will pick the runtime version used

1567

# during the CreateVersion request for this model version, or choose the

1568

# latest stable version when model version information is not available

1569

# such as when the model is specified by uri.

1570

"inputPaths": [ # Required. The Cloud Storage location of the input data files. May contain

1571

# <a href="/storage/docs/gsutil/addlhelp/WildcardNames">wildcards</a>.

1572

"A String",

1573

],

1574

"region": "A String", # Required. The Google Compute Engine region to run the prediction job in.

1575

# See the <a href="/ml-engine/docs/tensorflow/regions">available regions</a>

1576

# for AI Platform services.

1577

},

1578

"errorMessage": "A String", # Output only. The details of a failure or a cancellation.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1579

"etag": "A String", # `etag` is used for optimistic concurrency control as a way to help

1580

# prevent simultaneous updates of a job from overwriting each other.

1581

# It is strongly suggested that systems make use of the `etag` in the

1582

# read-modify-write cycle to perform job updates in order to avoid race

1583

# conditions: An `etag` is returned in the response to `GetJob`, and

1584

# systems are expected to put that etag in the request to `UpdateJob` to

1585

# ensure that their change will be applied to the same version of the job.

1586

"trainingInput": { # Represents input parameters for a training job. When using the gcloud command # Input parameters to create a training job.

1587

# to submit your training job, you can specify the input parameters as

1588

# command-line arguments and/or in a YAML configuration file referenced from

1589

# the --config command-line argument. For details, see the guide to [submitting

1590

# a training job](/ai-platform/training/docs/training-jobs).

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1591

"workerCount": "A String", # Optional. The number of worker replicas to use for the training job. Each

1592

# replica in the cluster will be of the type specified in `worker_type`.

1593

#

1594

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

1595

# set this value, you must also set `worker_type`.

1596

#

1597

# The default value is zero.

1598

"pythonModule": "A String", # Required. The Python module name to run after installing the packages.

1599

"args": [ # Optional. Command-line arguments passed to the training application when it

1600

# starts. If your job uses a custom container, then the arguments are passed

1601

# to the container's <a class="external" target="_blank"

1602

# href="https://docs.docker.com/engine/reference/builder/#entrypoint">

1603

# `ENTRYPOINT`</a> command.

1604

"A String",

1605

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1606

"parameterServerConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for parameter servers.

1607

#

1608

# You should only set `parameterServerConfig.acceleratorConfig` if

1609

# `parameterServerType` is set to a Compute Engine machine type. [Learn

1610

# about restrictions on accelerator configurations for

1611

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

1612

#

1613

# Set `parameterServerConfig.imageUri` only if you build a custom image for

1614

# your parameter server. If `parameterServerConfig.imageUri` has not been

1615

# set, AI Platform uses the value of `masterConfig.imageUri`. Learn more about [configuring custom

1616

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1617

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

1618

# Registry. Learn more about [configuring custom

1619

# containers](/ai-platform/training/docs/distributed-training-containers).

1620

"containerArgs": [ # Arguments to the entrypoint command.

1621

# The following rules apply for container_command and container_args:

1622

# - If you do not supply command or args:

1623

# The defaults defined in the Docker image are used.

1624

# - If you supply a command but no args:

1625

# The default EntryPoint and the default Cmd defined in the Docker image

1626

# are ignored. Your command is run without any arguments.

1627

# - If you supply only args:

1628

# The default Entrypoint defined in the Docker image is run with the args

1629

# that you supplied.

1630

# - If you supply a command and args:

1631

# The default Entrypoint and the default Cmd defined in the Docker image

1632

# are ignored. Your command is run with your args.

1633

# It cannot be set if custom container image is

1634

# not provided.

1635

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

1636

# both cannot be set at the same time.

1637

"A String",

1638

],

1639

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

1640

# [Learn about restrictions on accelerator configurations for

1641

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

1642

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

1643

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

1644

# [accelerators for online

1645

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

1646

"count": "A String", # The number of accelerators to attach to each machine running the job.

1647

"type": "A String", # The type of accelerator to use.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

1648

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1649

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

1650

# the one used in the custom container. This field is required if the replica

1651

# is a TPU worker that uses a custom container. Otherwise, do not specify

1652

# this field. This must be a [runtime version that currently supports

1653

# training with

1654

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

1655

#

1656

# Note that the version of TensorFlow included in a runtime version may

1657

# differ from the numbering of the runtime version itself, because it may

1658

# have a different [patch

1659

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

1660

# In this field, you must specify the runtime version (TensorFlow minor

1661

# version). For example, if your custom container runs TensorFlow `1.x.y`,

1662

# specify `1.x`.

1663

"containerCommand": [ # The command with which the replica's custom container is run.

1664

# If provided, it will override default ENTRYPOINT of the docker image.

1665

# If not provided, the docker image's ENTRYPOINT is used.

1666

# It cannot be set if custom container image is

1667

# not provided.

1668

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

1669

# both cannot be set at the same time.

1670

"A String",

1671

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1672

},

1673

"encryptionConfig": { # Represents a custom encryption key configuration that can be applied to # Optional. Options for using customer-managed encryption keys (CMEK) to

1674

# protect resources created by a training job, instead of using Google's

1675

# default encryption. If this is set, then all resources created by the

1676

# training job will be encrypted with the customer-managed encryption key

1677

# that you specify.

1678

#

1679

# [Learn how and when to use CMEK with AI Platform

1680

# Training](/ai-platform/training/docs/cmek).

1681

# a resource.

1682

"kmsKeyName": "A String", # The Cloud KMS resource identifier of the customer-managed encryption key

1683

# used to protect a resource, such as a training job. It has the following

1684

# format:

1685

# `projects/{PROJECT_ID}/locations/{REGION}/keyRings/{KEY_RING_NAME}/cryptoKeys/{KEY_NAME}`

1686

},

1687

"hyperparameters": { # Represents a set of hyperparameters to optimize. # Optional. The set of Hyperparameters to tune.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1688

"params": [ # Required. The set of parameters to tune.

1689

{ # Represents a single hyperparameter to optimize.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1690

"categoricalValues": [ # Required if type is `CATEGORICAL`. The list of possible categories.

1691

"A String",

1692

],

1693

"parameterName": "A String", # Required. The parameter name must be unique amongst all ParameterConfigs in

1694

# a HyperparameterSpec message. E.g., "learning_rate".

1695

"minValue": 3.14, # Required if type is `DOUBLE` or `INTEGER`. This field

1696

# should be unset if type is `CATEGORICAL`. This value should be integers if

1697

# type is INTEGER.

1698

"discreteValues": [ # Required if type is `DISCRETE`.

1699

# A list of feasible points.

1700

# The list should be in strictly increasing order. For instance, this

1701

# parameter might have possible settings of 1.5, 2.5, and 4.0. This list

1702

# should not contain more than 1,000 values.

1703

3.14,

1704

],

1705

"scaleType": "A String", # Optional. How the parameter should be scaled to the hypercube.

1706

# Leave unset for categorical parameters.

1707

# Some kind of scaling is strongly recommended for real or integral

1708

# parameters (e.g., `UNIT_LINEAR_SCALE`).

1709

"maxValue": 3.14, # Required if type is `DOUBLE` or `INTEGER`. This field

1710

# should be unset if type is `CATEGORICAL`. This value should be integers if

1711

# type is `INTEGER`.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1712

"type": "A String", # Required. The type of the parameter.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

1713

},

1714

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1715

"enableTrialEarlyStopping": True or False, # Optional. Indicates if the hyperparameter tuning job enables auto trial

1716

# early stopping.

1717

"resumePreviousJobId": "A String", # Optional. The prior hyperparameter tuning job id that users hope to

1718

# continue with. The job id will be used to find the corresponding vizier

1719

# study guid and resume the study.

1720

"maxParallelTrials": 42, # Optional. The number of training trials to run concurrently.

1721

# You can reduce the time it takes to perform hyperparameter tuning by adding

1722

# trials in parallel. However, each trail only benefits from the information

1723

# gained in completed trials. That means that a trial does not get access to

1724

# the results of trials running at the same time, which could reduce the

1725

# quality of the overall optimization.

1726

#

1727

# Each trial will use the same scale tier and machine types.

1728

#

1729

# Defaults to one.

1730

"maxFailedTrials": 42, # Optional. The number of failed trials that need to be seen before failing

1731

# the hyperparameter tuning job. You can specify this field to override the

1732

# default failing criteria for AI Platform hyperparameter tuning jobs.

1733

#

1734

# Defaults to zero, which means the service decides when a hyperparameter

1735

# job should fail.

1736

"goal": "A String", # Required. The type of goal to use for tuning. Available types are

1737

# `MAXIMIZE` and `MINIMIZE`.

1738

#

1739

# Defaults to `MAXIMIZE`.

1740

"maxTrials": 42, # Optional. How many training trials should be attempted to optimize

1741

# the specified hyperparameters.

1742

#

1743

# Defaults to one.

1744

"algorithm": "A String", # Optional. The search algorithm specified for the hyperparameter

1745

# tuning job.

1746

# Uses the default AI Platform hyperparameter tuning

1747

# algorithm if unspecified.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1748

"hyperparameterMetricTag": "A String", # Optional. The TensorFlow summary tag name to use for optimizing trials. For

1749

# current versions of TensorFlow, this tag name should exactly match what is

1750

# shown in TensorBoard, including all scopes. For versions of TensorFlow

1751

# prior to 0.12, this should be only the tag passed to tf.Summary.

1752

# By default, "training/hptuning/metric" will be used.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1753

},

1754

"workerConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for workers.

1755

#

1756

# You should only set `workerConfig.acceleratorConfig` if `workerType` is set

1757

# to a Compute Engine machine type. [Learn about restrictions on accelerator

1758

# configurations for

1759

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

1760

#

1761

# Set `workerConfig.imageUri` only if you build a custom image for your

1762

# worker. If `workerConfig.imageUri` has not been set, AI Platform uses

1763

# the value of `masterConfig.imageUri`. Learn more about [configuring custom

1764

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1765

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

1766

# Registry. Learn more about [configuring custom

1767

# containers](/ai-platform/training/docs/distributed-training-containers).

1768

"containerArgs": [ # Arguments to the entrypoint command.

1769

# The following rules apply for container_command and container_args:

1770

# - If you do not supply command or args:

1771

# The defaults defined in the Docker image are used.

1772

# - If you supply a command but no args:

1773

# The default EntryPoint and the default Cmd defined in the Docker image

1774

# are ignored. Your command is run without any arguments.

1775

# - If you supply only args:

1776

# The default Entrypoint defined in the Docker image is run with the args

1777

# that you supplied.

1778

# - If you supply a command and args:

1779

# The default Entrypoint and the default Cmd defined in the Docker image

1780

# are ignored. Your command is run with your args.

1781

# It cannot be set if custom container image is

1782

# not provided.

1783

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

1784

# both cannot be set at the same time.

1785

"A String",

1786

],

1787

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

1788

# [Learn about restrictions on accelerator configurations for

1789

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

1790

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

1791

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

1792

# [accelerators for online

1793

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

1794

"count": "A String", # The number of accelerators to attach to each machine running the job.

1795

"type": "A String", # The type of accelerator to use.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

1796

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1797

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

1798

# the one used in the custom container. This field is required if the replica

1799

# is a TPU worker that uses a custom container. Otherwise, do not specify

1800

# this field. This must be a [runtime version that currently supports

1801

# training with

1802

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

1803

#

1804

# Note that the version of TensorFlow included in a runtime version may

1805

# differ from the numbering of the runtime version itself, because it may

1806

# have a different [patch

1807

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

1808

# In this field, you must specify the runtime version (TensorFlow minor

1809

# version). For example, if your custom container runs TensorFlow `1.x.y`,

1810

# specify `1.x`.

1811

"containerCommand": [ # The command with which the replica's custom container is run.

1812

# If provided, it will override default ENTRYPOINT of the docker image.

1813

# If not provided, the docker image's ENTRYPOINT is used.

1814

# It cannot be set if custom container image is

1815

# not provided.

1816

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

1817

# both cannot be set at the same time.

1818

"A String",

1819

],

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

1820

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1821

"parameterServerCount": "A String", # Optional. The number of parameter server replicas to use for the training

1822

# job. Each replica in the cluster will be of the type specified in

1823

# `parameter_server_type`.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

1824

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1825

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

1826

# set this value, you must also set `parameter_server_type`.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

1827

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1828

# The default value is zero.

1829

"packageUris": [ # Required. The Google Cloud Storage location of the packages with

1830

# the training program and any additional dependencies.

1831

# The maximum number of package URIs is 100.

1832

"A String",

1833

],

1834

"evaluatorCount": "A String", # Optional. The number of evaluator replicas to use for the training job.

1835

# Each replica in the cluster will be of the type specified in

1836

# `evaluator_type`.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

1837

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1838

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

1839

# set this value, you must also set `evaluator_type`.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

1840

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1841

# The default value is zero.

1842

"masterType": "A String", # Optional. Specifies the type of virtual machine to use for your training

1843

# job's master worker. You must specify this field when `scaleTier` is set to

1844

# `CUSTOM`.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

1845

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1846

# You can use certain Compute Engine machine types directly in this field.

1847

# The following types are supported:

#

# - `n1-standard-4`

# - `n1-standard-8`

# - `n1-standard-16`

# - `n1-standard-32`

# - `n1-standard-64`

# - `n1-standard-96`

# - `n1-highmem-2`

# - `n1-highmem-4`

# - `n1-highmem-8`

# - `n1-highmem-16`

# - `n1-highmem-32`

# - `n1-highmem-64`

# - `n1-highmem-96`

# - `n1-highcpu-16`

# - `n1-highcpu-32`

# - `n1-highcpu-64`

# - `n1-highcpu-96`

#

# Learn more about [using Compute Engine machine

1868

# types](/ml-engine/docs/machine-types#compute-engine-machine-types).

1869

#

1870

# Alternatively, you can use the following legacy machine types:

#

# - `standard`

# - `large_model`

# - `complex_model_s`

1875

# - `complex_model_m`

1876

# - `complex_model_l`

1877

# - `standard_gpu`

1878

# - `complex_model_m_gpu`

1879

# - `complex_model_l_gpu`

1880

# - `standard_p100`

1881

# - `complex_model_m_p100`

1882

# - `standard_v100`

1883

# - `large_model_v100`

1884

# - `complex_model_m_v100`

1885

# - `complex_model_l_v100`

1886

#

1887

# Learn more about [using legacy machine

1888

# types](/ml-engine/docs/machine-types#legacy-machine-types).

1889

#

1890

# Finally, if you want to use a TPU for training, specify `cloud_tpu` in this

1891

# field. Learn more about the [special configuration options for training

1892

# with

1893

# TPUs](/ml-engine/docs/tensorflow/using-tpus#configuring_a_custom_tpu_machine).

1894

"runtimeVersion": "A String", # Optional. The AI Platform runtime version to use for training. You must

1895

# either specify this field or specify `masterConfig.imageUri`.

1896

#

1897

# For more information, see the [runtime version

1898

# list](/ai-platform/training/docs/runtime-version-list) and learn [how to

1899

# manage runtime versions](/ai-platform/training/docs/versioning).

1900

"evaluatorType": "A String", # Optional. Specifies the type of virtual machine to use for your training

1901

# job's evaluator nodes.

1902

#

1903

# The supported values are the same as those described in the entry for

1904

# `masterType`.

1905

#

1906

# This value must be consistent with the category of machine type that

1907

# `masterType` uses. In other words, both must be Compute Engine machine

1908

# types or both must be legacy machine types.

1909

#

1910

# This value must be present when `scaleTier` is set to `CUSTOM` and

1911

# `evaluatorCount` is greater than zero.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1912

"workerType": "A String", # Optional. Specifies the type of virtual machine to use for your training

1913

# job's worker nodes.

1914

#

1915

# The supported values are the same as those described in the entry for

1916

# `masterType`.

1917

#

1918

# This value must be consistent with the category of machine type that

1919

# `masterType` uses. In other words, both must be Compute Engine machine

1920

# types or both must be legacy machine types.

1921

#

1922

# If you use `cloud_tpu` for this value, see special instructions for

1923

# [configuring a custom TPU

1924

# machine](/ml-engine/docs/tensorflow/using-tpus#configuring_a_custom_tpu_machine).

1925

#

1926

# This value must be present when `scaleTier` is set to `CUSTOM` and

1927

# `workerCount` is greater than zero.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1928

"region": "A String", # Required. The region to run the training job in. See the [available

1929

# regions](/ai-platform/training/docs/regions) for AI Platform Training.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1930

"parameterServerType": "A String", # Optional. Specifies the type of virtual machine to use for your training

1931

# job's parameter server.

1932

#

1933

# The supported values are the same as those described in the entry for

1934

# `master_type`.

1935

#

1936

# This value must be consistent with the category of machine type that

1937

# `masterType` uses. In other words, both must be Compute Engine machine

1938

# types or both must be legacy machine types.

1939

#

1940

# This value must be present when `scaleTier` is set to `CUSTOM` and

1941

# `parameter_server_count` is greater than zero.

1942

"masterConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for your master worker.

1943

#

1944

# You should only set `masterConfig.acceleratorConfig` if `masterType` is set

1945

# to a Compute Engine machine type. Learn about [restrictions on accelerator

1946

# configurations for

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

1947

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

1948

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1949

# Set `masterConfig.imageUri` only if you build a custom image. Only one of

1950

# `masterConfig.imageUri` and `runtimeVersion` should be set. Learn more

1951

# about [configuring custom

1952

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

1953

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

1954

# Registry. Learn more about [configuring custom

1955

# containers](/ai-platform/training/docs/distributed-training-containers).

1956

"containerArgs": [ # Arguments to the entrypoint command.

1957

# The following rules apply for container_command and container_args:

1958

# - If you do not supply command or args:

1959

# The defaults defined in the Docker image are used.

1960

# - If you supply a command but no args:

1961

# The default EntryPoint and the default Cmd defined in the Docker image

1962

# are ignored. Your command is run without any arguments.

1963

# - If you supply only args:

1964

# The default Entrypoint defined in the Docker image is run with the args

1965

# that you supplied.

1966

# - If you supply a command and args:

1967

# The default Entrypoint and the default Cmd defined in the Docker image

1968

# are ignored. Your command is run with your args.

1969

# It cannot be set if custom container image is

1970

# not provided.

1971

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

1972

# both cannot be set at the same time.

1973

"A String",

1974

],

1975

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

1976

# [Learn about restrictions on accelerator configurations for

1977

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

1978

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

1979

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

1980

# [accelerators for online

1981

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

1982

"count": "A String", # The number of accelerators to attach to each machine running the job.

1983

"type": "A String", # The type of accelerator to use.

1984

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

1985

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

1986

# the one used in the custom container. This field is required if the replica

1987

# is a TPU worker that uses a custom container. Otherwise, do not specify

1988

# this field. This must be a [runtime version that currently supports

1989

# training with

1990

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

1991

#

1992

# Note that the version of TensorFlow included in a runtime version may

1993

# differ from the numbering of the runtime version itself, because it may

1994

# have a different [patch

1995

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

1996

# In this field, you must specify the runtime version (TensorFlow minor

1997

# version). For example, if your custom container runs TensorFlow `1.x.y`,

1998

# specify `1.x`.

1999

"containerCommand": [ # The command with which the replica's custom container is run.

2000

# If provided, it will override default ENTRYPOINT of the docker image.

2001

# If not provided, the docker image's ENTRYPOINT is used.

2002

# It cannot be set if custom container image is

2003

# not provided.

2004

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

2005

# both cannot be set at the same time.

2006

"A String",

2007

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2008

},

2009

"scaleTier": "A String", # Required. Specifies the machine types, the number of replicas for workers

2010

# and parameter servers.

2011

"jobDir": "A String", # Optional. A Google Cloud Storage path in which to store training outputs

2012

# and other data needed for training. This path is passed to your TensorFlow

2013

# program as the '--job-dir' command-line argument. The benefit of specifying

2014

# this field is that Cloud ML validates the path for use in training.

2015

"pythonVersion": "A String", # Optional. The version of Python used in training. You must either specify

2016

# this field or specify `masterConfig.imageUri`.

2017

#

2018

# The following Python versions are available:

2019

#

2020

# * Python '3.7' is available when `runtime_version` is set to '1.15' or

2021

# later.

2022

# * Python '3.5' is available when `runtime_version` is set to a version

2023

# from '1.4' to '1.14'.

2024

# * Python '2.7' is available when `runtime_version` is set to '1.15' or

2025

# earlier.

2026

#

2027

# Read more about the Python versions available for [each runtime

2028

# version](/ml-engine/docs/runtime-version-list).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2029

"scheduling": { # All parameters related to scheduling of training jobs. # Optional. Scheduling options for a training job.

2030

"maxWaitTime": "A String",

2031

"maxRunningTime": "A String", # Optional. The maximum job running time, expressed in seconds. The field can

2032

# contain up to nine fractional digits, terminated by `s`. If not specified,

2033

# this field defaults to `604800s` (seven days).

2034

#

2035

# If the training job is still running after this duration, AI Platform

2036

# Training cancels it.

2037

#

2038

# For example, if you want to ensure your job runs for no more than 2 hours,

2039

# set this field to `7200s` (2 hours * 60 minutes / hour * 60 seconds /

2040

# minute).

2041

#

2042

# If you submit your training job using the `gcloud` tool, you can [provide

2043

# this field in a `config.yaml`

2044

# file](/ai-platform/training/docs/training-jobs#formatting_your_configuration_parameters).

# For example:

#

# ```yaml

# trainingInput:

# ...

# scheduling:

# maxRunningTime: 7200s

2052

# ...

2053

# ```

2054

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2055

"network": "A String", # Optional. The full name of the Google Compute Engine

2056

# [network](/compute/docs/networks-and-firewalls#networks) to which the Job

2057

# is peered. For example, projects/12345/global/networks/myVPC. Format is of

2058

# the form projects/{project}/global/networks/{network}. Where {project} is a

2059

# project number, as in '12345', and {network} is network name.".

2060

#

2061

# Private services access must already be configured for the network. If left

2062

# unspecified, the Job is not peered with any network. Learn more -

2063

# Connecting Job to user network over private

2064

# IP.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2065

"evaluatorConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for evaluators.

2066

#

2067

# You should only set `evaluatorConfig.acceleratorConfig` if

2068

# `evaluatorType` is set to a Compute Engine machine type. [Learn

2069

# about restrictions on accelerator configurations for

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2070

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2071

#

2072

# Set `evaluatorConfig.imageUri` only if you build a custom image for

2073

# your evaluator. If `evaluatorConfig.imageUri` has not been

2074

# set, AI Platform uses the value of `masterConfig.imageUri`. Learn more about [configuring custom

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2075

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2076

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

2077

# Registry. Learn more about [configuring custom

2078

# containers](/ai-platform/training/docs/distributed-training-containers).

2079

"containerArgs": [ # Arguments to the entrypoint command.

2080

# The following rules apply for container_command and container_args:

2081

# - If you do not supply command or args:

2082

# The defaults defined in the Docker image are used.

2083

# - If you supply a command but no args:

2084

# The default EntryPoint and the default Cmd defined in the Docker image

2085

# are ignored. Your command is run without any arguments.

2086

# - If you supply only args:

2087

# The default Entrypoint defined in the Docker image is run with the args

2088

# that you supplied.

2089

# - If you supply a command and args:

2090

# The default Entrypoint and the default Cmd defined in the Docker image

2091

# are ignored. Your command is run with your args.

2092

# It cannot be set if custom container image is

2093

# not provided.

2094

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

2095

# both cannot be set at the same time.

2096

"A String",

2097

],

2098

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

2099

# [Learn about restrictions on accelerator configurations for

2100

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

2101

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

2102

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

2103

# [accelerators for online

2104

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

2105

"count": "A String", # The number of accelerators to attach to each machine running the job.

2106

"type": "A String", # The type of accelerator to use.

2107

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2108

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

2109

# the one used in the custom container. This field is required if the replica

2110

# is a TPU worker that uses a custom container. Otherwise, do not specify

2111

# this field. This must be a [runtime version that currently supports

2112

# training with

2113

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

2114

#

2115

# Note that the version of TensorFlow included in a runtime version may

2116

# differ from the numbering of the runtime version itself, because it may

2117

# have a different [patch

2118

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

2119

# In this field, you must specify the runtime version (TensorFlow minor

2120

# version). For example, if your custom container runs TensorFlow `1.x.y`,

2121

# specify `1.x`.

2122

"containerCommand": [ # The command with which the replica's custom container is run.

2123

# If provided, it will override default ENTRYPOINT of the docker image.

2124

# If not provided, the docker image's ENTRYPOINT is used.

2125

# It cannot be set if custom container image is

2126

# not provided.

2127

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

2128

# both cannot be set at the same time.

2129

"A String",

2130

],

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2131

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2132

"useChiefInTfConfig": True or False, # Optional. Use `chief` instead of `master` in the `TF_CONFIG` environment

2133

# variable when training with a custom container. Defaults to `false`. [Learn

2134

# more about this

2135

# field.](/ai-platform/training/docs/distributed-training-details#chief-versus-master)

2136

#

2137

# This field has no effect for training jobs that don't use a custom

2138

# container.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2139

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2140

"state": "A String", # Output only. The detailed state of a job.

2141

"jobId": "A String", # Required. The user-specified id of the job.

2142

"endTime": "A String", # Output only. When the job processing was completed.

2143

"startTime": "A String", # Output only. When the job processing was started.

2144

"predictionOutput": { # Represents results of a prediction job. # The current prediction job result.

2145

"errorCount": "A String", # The number of data instances which resulted in errors.

2146

"outputPath": "A String", # The output Google Cloud Storage location provided at the job creation time.

2147

"nodeHours": 3.14, # Node hours used by the batch prediction job.

2148

"predictionCount": "A String", # The number of generated predictions.

2149

},

2150

"trainingOutput": { # Represents results of a training job. Output only. # The current training job result.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2151

"isBuiltInAlgorithmJob": True or False, # Whether this job is a built-in Algorithm job.

2152

"builtInAlgorithmOutput": { # Represents output related to a built-in algorithm Job. # Details related to built-in algorithms jobs.

2153

# Only set for built-in algorithms jobs.

2154

"modelPath": "A String", # The Cloud Storage path to the `model/` directory where the training job

2155

# saves the trained model. Only set for successful jobs that don't use

2156

# hyperparameter tuning.

2157

"pythonVersion": "A String", # Python version on which the built-in algorithm was trained.

2158

"runtimeVersion": "A String", # AI Platform runtime version on which the built-in algorithm was

2159

# trained.

2160

"framework": "A String", # Framework on which the built-in algorithm was trained.

2161

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2162

"trials": [ # Results for individual Hyperparameter trials.

2163

# Only set for hyperparameter tuning jobs.

2164

{ # Represents the result of a single hyperparameter tuning trial from a

2165

# training job. The TrainingOutput object that is returned on successful

2166

# completion of a training job with hyperparameter tuning includes a list

2167

# of HyperparameterOutput objects, one for each successful trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2168

"trialId": "A String", # The trial id for these results.

2169

"endTime": "A String", # Output only. End time for the trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2170

"startTime": "A String", # Output only. Start time for the trial.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2171

"isTrialStoppedEarly": True or False, # True if the trial is stopped early.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2172

"finalMetric": { # An observed value of a metric. # The final objective metric seen for this trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2173

"trainingStep": "A String", # The global training step for this metric.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2174

"objectiveValue": 3.14, # The objective value at this training step.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2175

},

2176

"builtInAlgorithmOutput": { # Represents output related to a built-in algorithm Job. # Details related to built-in algorithms jobs.

2177

# Only set for trials of built-in algorithms jobs that have succeeded.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2178

"modelPath": "A String", # The Cloud Storage path to the `model/` directory where the training job

2179

# saves the trained model. Only set for successful jobs that don't use

2180

# hyperparameter tuning.

2181

"pythonVersion": "A String", # Python version on which the built-in algorithm was trained.

2182

"runtimeVersion": "A String", # AI Platform runtime version on which the built-in algorithm was

2183

# trained.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2184

"framework": "A String", # Framework on which the built-in algorithm was trained.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2185

},

2186

"state": "A String", # Output only. The detailed state of the trial.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2187

"allMetrics": [ # All recorded object metrics for this trial. This field is not currently

2188

# populated.

2189

{ # An observed value of a metric.

2190

"trainingStep": "A String", # The global training step for this metric.

2191

"objectiveValue": 3.14, # The objective value at this training step.

2192

},

2193

],

2194

"hyperparameters": { # The hyperparameters given to this trial.

2195

"a_key": "A String",

2196

},

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2197

},

2198

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2199

"hyperparameterMetricTag": "A String", # The TensorFlow summary tag name used for optimizing hyperparameter tuning

2200

# trials. See

2201

# [`HyperparameterSpec.hyperparameterMetricTag`](#HyperparameterSpec.FIELDS.hyperparameter_metric_tag)

2202

# for more information. Only set for hyperparameter tuning jobs.

2203

"completedTrialCount": "A String", # The number of hyperparameter tuning trials that completed successfully.

2204

# Only set for hyperparameter tuning jobs.

2205

"isHyperparameterTuningJob": True or False, # Whether this job is a hyperparameter tuning job.

2206

"consumedMLUnits": 3.14, # The amount of ML units consumed by the job.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2207

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2208

"createTime": "A String", # Output only. When the job was created.

2209

"labels": { # Optional. One or more labels that you can add, to organize your jobs.

2210

# Each label is a key-value pair, where both the key and the value are

2211

# arbitrary strings that you supply.

2212

# For more information, see the documentation on

2213

# <a href="/ml-engine/docs/tensorflow/resource-labels">using labels</a>.

2214

"a_key": "A String",

2215

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2216

}</pre>

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

</div>

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2220

<code class="details" id="getIamPolicy">getIamPolicy(resource, options_requestedPolicyVersion=None, x__xgafv=None)</code>

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2221

<pre>Gets the access control policy for a resource.

2222

Returns an empty policy if the resource exists and does not have a policy

2223

set.

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

2224

2225

Args:

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2226

resource: string, REQUIRED: The resource for which the policy is being requested.

2227

See the operation documentation for the appropriate value for this field. (required)

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2228

options_requestedPolicyVersion: integer, Optional. The policy format version to be returned.

2229

2230

Valid values are 0, 1, and 3. Requests specifying an invalid value will be

2231

rejected.

2232

2233

Requests for policies with any conditional bindings must specify version 3.

2234

Policies without any conditional bindings may specify any valid value or

2235

leave the field unset.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2236

2237

To learn which resources support conditions in their IAM policies, see the

2238

[IAM

2239

documentation](https://cloud.google.com/iam/help/conditions/resource-policies).

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

2240

x__xgafv: string, V1 error format.

Allowed values

1 - v1 error format

2 - v2 error format

Returns:

An object of the form:

2247

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2248

{ # An Identity and Access Management (IAM) policy, which specifies access

2249

# controls for Google Cloud resources.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2250

#

2251

#

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2252

# A `Policy` is a collection of `bindings`. A `binding` binds one or more

2253

# `members` to a single `role`. Members can be user accounts, service accounts,

2254

# Google groups, and domains (such as G Suite). A `role` is a named list of

2255

# permissions; each `role` can be an IAM predefined role or a user-created

2256

# custom role.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2257

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2258

# For some types of Google Cloud resources, a `binding` can also specify a

2259

# `condition`, which is a logical expression that allows access to a resource

2260

# only if the expression evaluates to `true`. A condition can add constraints

2261

# based on attributes of the request, the resource, or both. To learn which

2262

# resources support conditions in their IAM policies, see the

2263

# [IAM documentation](https://cloud.google.com/iam/help/conditions/resource-policies).

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2264

#

2265

# **JSON example:**

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2266

#

2267

# {

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2268

# "bindings": [

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2269

# {

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2270

# "role": "roles/resourcemanager.organizationAdmin",

2271

# "members": [

2272

# "user:mike@example.com",

2273

# "group:admins@example.com",

2274

# "domain:google.com",

2275

# "serviceAccount:my-project-id@appspot.gserviceaccount.com"

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2276

# ]

2277

# },

2278

# {

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2279

# "role": "roles/resourcemanager.organizationViewer",

2280

# "members": [

2281

# "user:eve@example.com"

2282

# ],

2283

# "condition": {

2284

# "title": "expirable access",

2285

# "description": "Does not grant access after Sep 2020",

2286

# "expression": "request.time < timestamp('2020-10-01T00:00:00.000Z')",

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2287

# }

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2288

# }

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2289

# ],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2290

# "etag": "BwWWja0YfJA=",

2291

# "version": 3

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2292

# }

2293

#

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2294

# **YAML example:**

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

#

# bindings:

# - members:

# - user:mike@example.com

2299

# - group:admins@example.com

2300

# - domain:google.com

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2301

# - serviceAccount:my-project-id@appspot.gserviceaccount.com

2302

# role: roles/resourcemanager.organizationAdmin

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2303

# - members:

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2304

# - user:eve@example.com

2305

# role: roles/resourcemanager.organizationViewer

2306

# condition:

2307

# title: expirable access

2308

# description: Does not grant access after Sep 2020

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2309

# expression: request.time < timestamp('2020-10-01T00:00:00.000Z')

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2310

# - etag: BwWWja0YfJA=

2311

# - version: 3

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2312

#

2313

# For a description of IAM and its features, see the

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2314

# [IAM documentation](https://cloud.google.com/iam/docs/).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2315

"version": 42, # Specifies the format of the policy.

2316

#

2317

# Valid values are `0`, `1`, and `3`. Requests that specify an invalid value

2318

# are rejected.

2319

#

2320

# Any operation that affects conditional role bindings must specify version

2321

# `3`. This requirement applies to the following operations:

2322

#

2323

# * Getting a policy that includes a conditional role binding

2324

# * Adding a conditional role binding to a policy

2325

# * Changing a conditional role binding in a policy

2326

# * Removing any role binding, with or without a condition, from a policy

2327

# that includes conditions

2328

#

2329

# **Important:** If you use IAM Conditions, you must include the `etag` field

2330

# whenever you call `setIamPolicy`. If you omit this field, then IAM allows

2331

# you to overwrite a version `3` policy with a version `1` policy, and all of

2332

# the conditions in the version `3` policy are lost.

2333

#

2334

# If a policy does not include any conditions, operations on that policy may

2335

# specify any valid version or leave the field unset.

2336

#

2337

# To learn which resources support conditions in their IAM policies, see the

2338

# [IAM documentation](https://cloud.google.com/iam/help/conditions/resource-policies).

2339

"auditConfigs": [ # Specifies cloud audit logging configuration for this policy.

2340

{ # Specifies the audit configuration for a service.

2341

# The configuration determines which permission types are logged, and what

2342

# identities, if any, are exempted from logging.

2343

# An AuditConfig must have one or more AuditLogConfigs.

2344

#

2345

# If there are AuditConfigs for both `allServices` and a specific service,

2346

# the union of the two AuditConfigs is used for that service: the log_types

2347

# specified in each AuditConfig are enabled, and the exempted_members in each

2348

# AuditLogConfig are exempted.

2349

#

2350

# Example Policy with multiple AuditConfigs:

#

# {

# "audit_configs": [

# {

# "service": "allServices"

2356

# "audit_log_configs": [

2357

# {

2358

# "log_type": "DATA_READ",

2359

# "exempted_members": [

2360

# "user:jose@example.com"

# ]

# },

# {

# "log_type": "DATA_WRITE",

2365

# },

2366

# {

2367

# "log_type": "ADMIN_READ",

# }

# ]

# },

# {

# "service": "sampleservice.googleapis.com"

2373

# "audit_log_configs": [

2374

# {

2375

# "log_type": "DATA_READ",

2376

# },

2377

# {

2378

# "log_type": "DATA_WRITE",

2379

# "exempted_members": [

2380

# "user:aliya@example.com"

# ]

# }

# ]

# }

# ]

# }

#

# For sampleservice, this policy enables DATA_READ, DATA_WRITE and ADMIN_READ

2389

# logging. It also exempts jose@example.com from DATA_READ logging, and

2390

# aliya@example.com from DATA_WRITE logging.

2391

"service": "A String", # Specifies a service that will be enabled for audit logging.

2392

# For example, `storage.googleapis.com`, `cloudsql.googleapis.com`.

2393

# `allServices` is a special value that covers all services.

2394

"auditLogConfigs": [ # The configuration for logging of each type of permission.

2395

{ # Provides the configuration for logging a type of permissions.

# Example:

#

# {

# "audit_log_configs": [

2400

# {

2401

# "log_type": "DATA_READ",

2402

# "exempted_members": [

2403

# "user:jose@example.com"

# ]

# },

# {

# "log_type": "DATA_WRITE",

# }

# ]

# }

#

# This enables 'DATA_READ' and 'DATA_WRITE' logging, while exempting

2413

# jose@example.com from DATA_READ logging.

2414

"exemptedMembers": [ # Specifies the identities that do not cause logging for this type of

2415

# permission.

2416

# Follows the same format of Binding.members.

2417

"A String",

2418

],

2419

"logType": "A String", # The log type that this config enables.

},

],

},

],

"bindings": [ # Associates a list of `members` to a `role`. Optionally, may specify a

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2425

# `condition` that determines how and when the `bindings` are applied. Each

2426

# of the `bindings` must contain at least one member.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2427

{ # Associates `members` with a `role`.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2428

"condition": { # Represents a textual expression in the Common Expression Language (CEL) # The condition that is associated with this binding.

2429

#

2430

# If the condition evaluates to `true`, then this binding applies to the

2431

# current request.

2432

#

2433

# If the condition evaluates to `false`, then this binding does not apply to

2434

# the current request. However, a different role binding might grant the same

2435

# role to one or more of the members in this binding.

2436

#

2437

# To learn which resources support conditions in their IAM policies, see the

2438

# [IAM

2439

# documentation](https://cloud.google.com/iam/help/conditions/resource-policies).

2440

# syntax. CEL is a C-like expression language. The syntax and semantics of CEL

2441

# are documented at https://github.com/google/cel-spec.

2442

#

2443

# Example (Comparison):

2444

#

2445

# title: "Summary size limit"

2446

# description: "Determines if a summary is less than 100 chars"

2447

# expression: "document.summary.size() < 100"

2448

#

2449

# Example (Equality):

2450

#

2451

# title: "Requestor is owner"

2452

# description: "Determines if requestor is the document owner"

2453

# expression: "document.owner == request.auth.claims.email"

#

# Example (Logic):

#

# title: "Public documents"

2458

# description: "Determine whether the document should be publicly visible"

2459

# expression: "document.type != 'private' && document.type != 'internal'"

2460

#

2461

# Example (Data Manipulation):

2462

#

2463

# title: "Notification string"

2464

# description: "Create a notification string with a timestamp."

2465

# expression: "'New message received at ' + string(document.create_time)"

2466

#

2467

# The exact variables and functions that may be referenced within an expression

2468

# are determined by the service that evaluates it. See the service

2469

# documentation for additional information.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2470

"description": "A String", # Optional. Description of the expression. This is a longer text which

2471

# describes the expression, e.g. when hovered over it in a UI.

2472

"expression": "A String", # Textual representation of an expression in Common Expression Language

2473

# syntax.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2474

"title": "A String", # Optional. Title for the expression, i.e. a short string describing

2475

# its purpose. This can be used e.g. in UIs which allow to enter the

2476

# expression.

2477

"location": "A String", # Optional. String indicating the location of the expression for error

2478

# reporting, e.g. a file name and a position in the file.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2479

},

2480

"members": [ # Specifies the identities requesting access for a Cloud Platform resource.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2481

# `members` can have the following values:

2482

#

2483

# * `allUsers`: A special identifier that represents anyone who is

2484

# on the internet; with or without a Google account.

2485

#

2486

# * `allAuthenticatedUsers`: A special identifier that represents anyone

2487

# who is authenticated with a Google account or a service account.

2488

#

2489

# * `user:{emailid}`: An email address that represents a specific Google

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2490

# account. For example, `alice@example.com` .

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2491

#

2492

#

2493

# * `serviceAccount:{emailid}`: An email address that represents a service

2494

# account. For example, `my-other-app@appspot.gserviceaccount.com`.

2495

#

2496

# * `group:{emailid}`: An email address that represents a Google group.

2497

# For example, `admins@example.com`.

2498

#

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2499

# * `deleted:user:{emailid}?uid={uniqueid}`: An email address (plus unique

2500

# identifier) representing a user that has been recently deleted. For

2501

# example, `alice@example.com?uid=123456789012345678901`. If the user is

2502

# recovered, this value reverts to `user:{emailid}` and the recovered user

2503

# retains the role in the binding.

2504

#

2505

# * `deleted:serviceAccount:{emailid}?uid={uniqueid}`: An email address (plus

2506

# unique identifier) representing a service account that has been recently

2507

# deleted. For example,

2508

# `my-other-app@appspot.gserviceaccount.com?uid=123456789012345678901`.

2509

# If the service account is undeleted, this value reverts to

2510

# `serviceAccount:{emailid}` and the undeleted service account retains the

2511

# role in the binding.

2512

#

2513

# * `deleted:group:{emailid}?uid={uniqueid}`: An email address (plus unique

2514

# identifier) representing a Google group that has been recently

2515

# deleted. For example, `admins@example.com?uid=123456789012345678901`. If

2516

# the group is recovered, this value reverts to `group:{emailid}` and the

2517

# recovered group retains the role in the binding.

2518

#

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2519

#

2520

# * `domain:{domain}`: The G Suite domain (primary) that represents all the

2521

# users of that domain. For example, `google.com` or `example.com`.

2522

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2523

"A String",

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2524

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2525

"role": "A String", # Role that is assigned to `members`.

2526

# For example, `roles/viewer`, `roles/editor`, or `roles/owner`.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2527

},

2528

],

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2529

"etag": "A String", # `etag` is used for optimistic concurrency control as a way to help

2530

# prevent simultaneous updates of a policy from overwriting each other.

2531

# It is strongly suggested that systems make use of the `etag` in the

2532

# read-modify-write cycle to perform policy updates in order to avoid race

2533

# conditions: An `etag` is returned in the response to `getIamPolicy`, and

2534

# systems are expected to put that etag in the request to `setIamPolicy` to

2535

# ensure that their change will be applied to the same version of the policy.

2536

#

2537

# **Important:** If you use IAM Conditions, you must include the `etag` field

2538

# whenever you call `setIamPolicy`. If you omit this field, then IAM allows

2539

# you to overwrite a version `3` policy with a version `1` policy, and all of

2540

# the conditions in the version `3` policy are lost.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2541

}</pre>

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

</div>

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2545

<code class="details" id="list">list(parent, filter=None, pageToken=None, pageSize=None, x__xgafv=None)</code>

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

2546

<pre>Lists the jobs in the project.

2547

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2548

If there are no jobs that match the request parameters, the list

2549

request returns an empty response body: {}.

2550

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

2551

Args:

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2552

parent: string, Required. The name of the project for which to list jobs. (required)

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2553

filter: string, Optional. Specifies the subset of jobs to retrieve.

2554

You can filter on the value of one or more attributes of the job object.

2555

For example, retrieve jobs with a job identifier that starts with 'census':

2556

<p><code>gcloud ai-platform jobs list --filter='jobId:census*'</code>

2557

<p>List all failed jobs with names that start with 'rnn':

2558

<p><code>gcloud ai-platform jobs list --filter='jobId:rnn*

2559

AND state:FAILED'</code>

2560

<p>For more examples, see the guide to

2561

<a href="/ml-engine/docs/tensorflow/monitor-training">monitoring jobs</a>.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2562

pageToken: string, Optional. A page token to request the next page of results.

2563

2564

You get the token from the `next_page_token` field of the response from

2565

the previous call.

2566

pageSize: integer, Optional. The number of jobs to retrieve per "page" of results. If there

2567

are more remaining results than this number, the response message will

2568

contain a valid value in the `next_page_token` field.

2569

2570

The default value is 20, and the maximum page size is 100.

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

2571

x__xgafv: string, V1 error format.

Allowed values

1 - v1 error format

2 - v2 error format

Returns:

An object of the form:

2578

2579

{ # Response message for the ListJobs method.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2580

"jobs": [ # The list of jobs.

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

2581

{ # Represents a training or prediction job.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2582

"predictionInput": { # Represents input parameters for a prediction job. # Input parameters to create a prediction job.

2583

"versionName": "A String", # Use this field if you want to specify a version of the model to use. The

2584

# string is formatted the same way as `model_version`, with the addition

2585

# of the version information:

2586

#

2587

# `"projects/YOUR_PROJECT/models/YOUR_MODEL/versions/YOUR_VERSION"`

2588

"modelName": "A String", # Use this field if you want to use the default version for the specified

2589

# model. The string must use the following format:

2590

#

2591

# `"projects/YOUR_PROJECT/models/YOUR_MODEL"`

2592

"uri": "A String", # Use this field if you want to specify a Google Cloud Storage path for

2593

# the model to use.

2594

"maxWorkerCount": "A String", # Optional. The maximum number of workers to be used for parallel processing.

2595

# Defaults to 10 if not specified.

2596

"signatureName": "A String", # Optional. The name of the signature defined in the SavedModel to use for

2597

# this job. Please refer to

2598

# [SavedModel](https://tensorflow.github.io/serving/serving_basic.html)

2599

# for information about how to use signatures.

2600

#

2601

# Defaults to

2602

# [DEFAULT_SERVING_SIGNATURE_DEF_KEY](https://www.tensorflow.org/api_docs/python/tf/saved_model/signature_constants)

2603

# , which is "serving_default".

2604

"outputPath": "A String", # Required. The output Google Cloud Storage location.

2605

"outputDataFormat": "A String", # Optional. Format of the output data files, defaults to JSON.

2606

"dataFormat": "A String", # Required. The format of the input data files.

2607

"batchSize": "A String", # Optional. Number of records per batch, defaults to 64.

2608

# The service will buffer batch_size number of records in memory before

2609

# invoking one Tensorflow prediction call internally. So take the record

2610

# size and memory available into consideration when setting this parameter.

2611

"runtimeVersion": "A String", # Optional. The AI Platform runtime version to use for this batch

2612

# prediction. If not set, AI Platform will pick the runtime version used

2613

# during the CreateVersion request for this model version, or choose the

2614

# latest stable version when model version information is not available

2615

# such as when the model is specified by uri.

2616

"inputPaths": [ # Required. The Cloud Storage location of the input data files. May contain

2617

# <a href="/storage/docs/gsutil/addlhelp/WildcardNames">wildcards</a>.

2618

"A String",

2619

],

2620

"region": "A String", # Required. The Google Compute Engine region to run the prediction job in.

2621

# See the <a href="/ml-engine/docs/tensorflow/regions">available regions</a>

2622

# for AI Platform services.

2623

},

2624

"errorMessage": "A String", # Output only. The details of a failure or a cancellation.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2625

"etag": "A String", # `etag` is used for optimistic concurrency control as a way to help

2626

# prevent simultaneous updates of a job from overwriting each other.

2627

# It is strongly suggested that systems make use of the `etag` in the

2628

# read-modify-write cycle to perform job updates in order to avoid race

2629

# conditions: An `etag` is returned in the response to `GetJob`, and

2630

# systems are expected to put that etag in the request to `UpdateJob` to

2631

# ensure that their change will be applied to the same version of the job.

2632

"trainingInput": { # Represents input parameters for a training job. When using the gcloud command # Input parameters to create a training job.

2633

# to submit your training job, you can specify the input parameters as

2634

# command-line arguments and/or in a YAML configuration file referenced from

2635

# the --config command-line argument. For details, see the guide to [submitting

2636

# a training job](/ai-platform/training/docs/training-jobs).

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2637

"workerCount": "A String", # Optional. The number of worker replicas to use for the training job. Each

2638

# replica in the cluster will be of the type specified in `worker_type`.

2639

#

2640

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

2641

# set this value, you must also set `worker_type`.

2642

#

2643

# The default value is zero.

2644

"pythonModule": "A String", # Required. The Python module name to run after installing the packages.

2645

"args": [ # Optional. Command-line arguments passed to the training application when it

2646

# starts. If your job uses a custom container, then the arguments are passed

2647

# to the container's <a class="external" target="_blank"

2648

# href="https://docs.docker.com/engine/reference/builder/#entrypoint">

2649

# `ENTRYPOINT`</a> command.

2650

"A String",

2651

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2652

"parameterServerConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for parameter servers.

2653

#

2654

# You should only set `parameterServerConfig.acceleratorConfig` if

2655

# `parameterServerType` is set to a Compute Engine machine type. [Learn

2656

# about restrictions on accelerator configurations for

2657

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

2658

#

2659

# Set `parameterServerConfig.imageUri` only if you build a custom image for

2660

# your parameter server. If `parameterServerConfig.imageUri` has not been

2661

# set, AI Platform uses the value of `masterConfig.imageUri`. Learn more about [configuring custom

2662

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2663

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

2664

# Registry. Learn more about [configuring custom

2665

# containers](/ai-platform/training/docs/distributed-training-containers).

2666

"containerArgs": [ # Arguments to the entrypoint command.

2667

# The following rules apply for container_command and container_args:

2668

# - If you do not supply command or args:

2669

# The defaults defined in the Docker image are used.

2670

# - If you supply a command but no args:

2671

# The default EntryPoint and the default Cmd defined in the Docker image

2672

# are ignored. Your command is run without any arguments.

2673

# - If you supply only args:

2674

# The default Entrypoint defined in the Docker image is run with the args

2675

# that you supplied.

2676

# - If you supply a command and args:

2677

# The default Entrypoint and the default Cmd defined in the Docker image

2678

# are ignored. Your command is run with your args.

2679

# It cannot be set if custom container image is

2680

# not provided.

2681

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

2682

# both cannot be set at the same time.

2683

"A String",

2684

],

2685

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

2686

# [Learn about restrictions on accelerator configurations for

2687

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

2688

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

2689

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

2690

# [accelerators for online

2691

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

2692

"count": "A String", # The number of accelerators to attach to each machine running the job.

2693

"type": "A String", # The type of accelerator to use.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2694

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2695

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

2696

# the one used in the custom container. This field is required if the replica

2697

# is a TPU worker that uses a custom container. Otherwise, do not specify

2698

# this field. This must be a [runtime version that currently supports

2699

# training with

2700

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

2701

#

2702

# Note that the version of TensorFlow included in a runtime version may

2703

# differ from the numbering of the runtime version itself, because it may

2704

# have a different [patch

2705

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

2706

# In this field, you must specify the runtime version (TensorFlow minor

2707

# version). For example, if your custom container runs TensorFlow `1.x.y`,

2708

# specify `1.x`.

2709

"containerCommand": [ # The command with which the replica's custom container is run.

2710

# If provided, it will override default ENTRYPOINT of the docker image.

2711

# If not provided, the docker image's ENTRYPOINT is used.

2712

# It cannot be set if custom container image is

2713

# not provided.

2714

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

2715

# both cannot be set at the same time.

2716

"A String",

2717

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2718

},

2719

"encryptionConfig": { # Represents a custom encryption key configuration that can be applied to # Optional. Options for using customer-managed encryption keys (CMEK) to

2720

# protect resources created by a training job, instead of using Google's

2721

# default encryption. If this is set, then all resources created by the

2722

# training job will be encrypted with the customer-managed encryption key

2723

# that you specify.

2724

#

2725

# [Learn how and when to use CMEK with AI Platform

2726

# Training](/ai-platform/training/docs/cmek).

2727

# a resource.

2728

"kmsKeyName": "A String", # The Cloud KMS resource identifier of the customer-managed encryption key

2729

# used to protect a resource, such as a training job. It has the following

2730

# format:

2731

# `projects/{PROJECT_ID}/locations/{REGION}/keyRings/{KEY_RING_NAME}/cryptoKeys/{KEY_NAME}`

2732

},

2733

"hyperparameters": { # Represents a set of hyperparameters to optimize. # Optional. The set of Hyperparameters to tune.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2734

"params": [ # Required. The set of parameters to tune.

2735

{ # Represents a single hyperparameter to optimize.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2736

"categoricalValues": [ # Required if type is `CATEGORICAL`. The list of possible categories.

2737

"A String",

2738

],

2739

"parameterName": "A String", # Required. The parameter name must be unique amongst all ParameterConfigs in

2740

# a HyperparameterSpec message. E.g., "learning_rate".

2741

"minValue": 3.14, # Required if type is `DOUBLE` or `INTEGER`. This field

2742

# should be unset if type is `CATEGORICAL`. This value should be integers if

2743

# type is INTEGER.

2744

"discreteValues": [ # Required if type is `DISCRETE`.

2745

# A list of feasible points.

2746

# The list should be in strictly increasing order. For instance, this

2747

# parameter might have possible settings of 1.5, 2.5, and 4.0. This list

2748

# should not contain more than 1,000 values.

2749

3.14,

2750

],

2751

"scaleType": "A String", # Optional. How the parameter should be scaled to the hypercube.

2752

# Leave unset for categorical parameters.

2753

# Some kind of scaling is strongly recommended for real or integral

2754

# parameters (e.g., `UNIT_LINEAR_SCALE`).

2755

"maxValue": 3.14, # Required if type is `DOUBLE` or `INTEGER`. This field

2756

# should be unset if type is `CATEGORICAL`. This value should be integers if

2757

# type is `INTEGER`.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2758

"type": "A String", # Required. The type of the parameter.

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

2759

},

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2760

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2761

"enableTrialEarlyStopping": True or False, # Optional. Indicates if the hyperparameter tuning job enables auto trial

2762

# early stopping.

2763

"resumePreviousJobId": "A String", # Optional. The prior hyperparameter tuning job id that users hope to

2764

# continue with. The job id will be used to find the corresponding vizier

2765

# study guid and resume the study.

2766

"maxParallelTrials": 42, # Optional. The number of training trials to run concurrently.

2767

# You can reduce the time it takes to perform hyperparameter tuning by adding

2768

# trials in parallel. However, each trail only benefits from the information

2769

# gained in completed trials. That means that a trial does not get access to

2770

# the results of trials running at the same time, which could reduce the

2771

# quality of the overall optimization.

2772

#

2773

# Each trial will use the same scale tier and machine types.

2774

#

2775

# Defaults to one.

2776

"maxFailedTrials": 42, # Optional. The number of failed trials that need to be seen before failing

2777

# the hyperparameter tuning job. You can specify this field to override the

2778

# default failing criteria for AI Platform hyperparameter tuning jobs.

2779

#

2780

# Defaults to zero, which means the service decides when a hyperparameter

2781

# job should fail.

2782

"goal": "A String", # Required. The type of goal to use for tuning. Available types are

2783

# `MAXIMIZE` and `MINIMIZE`.

2784

#

2785

# Defaults to `MAXIMIZE`.

2786

"maxTrials": 42, # Optional. How many training trials should be attempted to optimize

2787

# the specified hyperparameters.

2788

#

2789

# Defaults to one.

2790

"algorithm": "A String", # Optional. The search algorithm specified for the hyperparameter

2791

# tuning job.

2792

# Uses the default AI Platform hyperparameter tuning

2793

# algorithm if unspecified.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2794

"hyperparameterMetricTag": "A String", # Optional. The TensorFlow summary tag name to use for optimizing trials. For

2795

# current versions of TensorFlow, this tag name should exactly match what is

2796

# shown in TensorBoard, including all scopes. For versions of TensorFlow

2797

# prior to 0.12, this should be only the tag passed to tf.Summary.

2798

# By default, "training/hptuning/metric" will be used.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2799

},

2800

"workerConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for workers.

2801

#

2802

# You should only set `workerConfig.acceleratorConfig` if `workerType` is set

2803

# to a Compute Engine machine type. [Learn about restrictions on accelerator

2804

# configurations for

2805

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

2806

#

2807

# Set `workerConfig.imageUri` only if you build a custom image for your

2808

# worker. If `workerConfig.imageUri` has not been set, AI Platform uses

2809

# the value of `masterConfig.imageUri`. Learn more about [configuring custom

2810

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2811

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

2812

# Registry. Learn more about [configuring custom

2813

# containers](/ai-platform/training/docs/distributed-training-containers).

2814

"containerArgs": [ # Arguments to the entrypoint command.

2815

# The following rules apply for container_command and container_args:

2816

# - If you do not supply command or args:

2817

# The defaults defined in the Docker image are used.

2818

# - If you supply a command but no args:

2819

# The default EntryPoint and the default Cmd defined in the Docker image

2820

# are ignored. Your command is run without any arguments.

2821

# - If you supply only args:

2822

# The default Entrypoint defined in the Docker image is run with the args

2823

# that you supplied.

2824

# - If you supply a command and args:

2825

# The default Entrypoint and the default Cmd defined in the Docker image

2826

# are ignored. Your command is run with your args.

2827

# It cannot be set if custom container image is

2828

# not provided.

2829

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

2830

# both cannot be set at the same time.

2831

"A String",

2832

],

2833

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

2834

# [Learn about restrictions on accelerator configurations for

2835

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

2836

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

2837

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

2838

# [accelerators for online

2839

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

2840

"count": "A String", # The number of accelerators to attach to each machine running the job.

2841

"type": "A String", # The type of accelerator to use.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2842

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2843

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

2844

# the one used in the custom container. This field is required if the replica

2845

# is a TPU worker that uses a custom container. Otherwise, do not specify

2846

# this field. This must be a [runtime version that currently supports

2847

# training with

2848

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

2849

#

2850

# Note that the version of TensorFlow included in a runtime version may

2851

# differ from the numbering of the runtime version itself, because it may

2852

# have a different [patch

2853

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

2854

# In this field, you must specify the runtime version (TensorFlow minor

2855

# version). For example, if your custom container runs TensorFlow `1.x.y`,

2856

# specify `1.x`.

2857

"containerCommand": [ # The command with which the replica's custom container is run.

2858

# If provided, it will override default ENTRYPOINT of the docker image.

2859

# If not provided, the docker image's ENTRYPOINT is used.

2860

# It cannot be set if custom container image is

2861

# not provided.

2862

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

2863

# both cannot be set at the same time.

2864

"A String",

2865

],

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

2866

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2867

"parameterServerCount": "A String", # Optional. The number of parameter server replicas to use for the training

2868

# job. Each replica in the cluster will be of the type specified in

2869

# `parameter_server_type`.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2870

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2871

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

2872

# set this value, you must also set `parameter_server_type`.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2873

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2874

# The default value is zero.

2875

"packageUris": [ # Required. The Google Cloud Storage location of the packages with

2876

# the training program and any additional dependencies.

2877

# The maximum number of package URIs is 100.

2878

"A String",

2879

],

2880

"evaluatorCount": "A String", # Optional. The number of evaluator replicas to use for the training job.

2881

# Each replica in the cluster will be of the type specified in

2882

# `evaluator_type`.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2883

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2884

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

2885

# set this value, you must also set `evaluator_type`.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2886

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2887

# The default value is zero.

2888

"masterType": "A String", # Optional. Specifies the type of virtual machine to use for your training

2889

# job's master worker. You must specify this field when `scaleTier` is set to

2890

# `CUSTOM`.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2891

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2892

# You can use certain Compute Engine machine types directly in this field.

2893

# The following types are supported:

#

# - `n1-standard-4`

# - `n1-standard-8`

# - `n1-standard-16`

# - `n1-standard-32`

# - `n1-standard-64`

# - `n1-standard-96`

# - `n1-highmem-2`

# - `n1-highmem-4`

# - `n1-highmem-8`

# - `n1-highmem-16`

# - `n1-highmem-32`

# - `n1-highmem-64`

# - `n1-highmem-96`

# - `n1-highcpu-16`

# - `n1-highcpu-32`

# - `n1-highcpu-64`

# - `n1-highcpu-96`

#

# Learn more about [using Compute Engine machine

2914

# types](/ml-engine/docs/machine-types#compute-engine-machine-types).

2915

#

2916

# Alternatively, you can use the following legacy machine types:

#

# - `standard`

# - `large_model`

# - `complex_model_s`

2921

# - `complex_model_m`

2922

# - `complex_model_l`

2923

# - `standard_gpu`

2924

# - `complex_model_m_gpu`

2925

# - `complex_model_l_gpu`

2926

# - `standard_p100`

2927

# - `complex_model_m_p100`

2928

# - `standard_v100`

2929

# - `large_model_v100`

2930

# - `complex_model_m_v100`

2931

# - `complex_model_l_v100`

2932

#

2933

# Learn more about [using legacy machine

2934

# types](/ml-engine/docs/machine-types#legacy-machine-types).

2935

#

2936

# Finally, if you want to use a TPU for training, specify `cloud_tpu` in this

2937

# field. Learn more about the [special configuration options for training

2938

# with

2939

# TPUs](/ml-engine/docs/tensorflow/using-tpus#configuring_a_custom_tpu_machine).

2940

"runtimeVersion": "A String", # Optional. The AI Platform runtime version to use for training. You must

2941

# either specify this field or specify `masterConfig.imageUri`.

2942

#

2943

# For more information, see the [runtime version

2944

# list](/ai-platform/training/docs/runtime-version-list) and learn [how to

2945

# manage runtime versions](/ai-platform/training/docs/versioning).

2946

"evaluatorType": "A String", # Optional. Specifies the type of virtual machine to use for your training

2947

# job's evaluator nodes.

2948

#

2949

# The supported values are the same as those described in the entry for

2950

# `masterType`.

2951

#

2952

# This value must be consistent with the category of machine type that

2953

# `masterType` uses. In other words, both must be Compute Engine machine

2954

# types or both must be legacy machine types.

2955

#

2956

# This value must be present when `scaleTier` is set to `CUSTOM` and

2957

# `evaluatorCount` is greater than zero.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2958

"workerType": "A String", # Optional. Specifies the type of virtual machine to use for your training

2959

# job's worker nodes.

2960

#

2961

# The supported values are the same as those described in the entry for

2962

# `masterType`.

2963

#

2964

# This value must be consistent with the category of machine type that

2965

# `masterType` uses. In other words, both must be Compute Engine machine

2966

# types or both must be legacy machine types.

2967

#

2968

# If you use `cloud_tpu` for this value, see special instructions for

2969

# [configuring a custom TPU

2970

# machine](/ml-engine/docs/tensorflow/using-tpus#configuring_a_custom_tpu_machine).

2971

#

2972

# This value must be present when `scaleTier` is set to `CUSTOM` and

2973

# `workerCount` is greater than zero.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

2974

"region": "A String", # Required. The region to run the training job in. See the [available

2975

# regions](/ai-platform/training/docs/regions) for AI Platform Training.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2976

"parameterServerType": "A String", # Optional. Specifies the type of virtual machine to use for your training

2977

# job's parameter server.

2978

#

2979

# The supported values are the same as those described in the entry for

2980

# `master_type`.

2981

#

2982

# This value must be consistent with the category of machine type that

2983

# `masterType` uses. In other words, both must be Compute Engine machine

2984

# types or both must be legacy machine types.

2985

#

2986

# This value must be present when `scaleTier` is set to `CUSTOM` and

2987

# `parameter_server_count` is greater than zero.

2988

"masterConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for your master worker.

2989

#

2990

# You should only set `masterConfig.acceleratorConfig` if `masterType` is set

2991

# to a Compute Engine machine type. Learn about [restrictions on accelerator

2992

# configurations for

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2993

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

2994

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2995

# Set `masterConfig.imageUri` only if you build a custom image. Only one of

2996

# `masterConfig.imageUri` and `runtimeVersion` should be set. Learn more

2997

# about [configuring custom

2998

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

2999

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

3000

# Registry. Learn more about [configuring custom

3001

# containers](/ai-platform/training/docs/distributed-training-containers).

3002

"containerArgs": [ # Arguments to the entrypoint command.

3003

# The following rules apply for container_command and container_args:

3004

# - If you do not supply command or args:

3005

# The defaults defined in the Docker image are used.

3006

# - If you supply a command but no args:

3007

# The default EntryPoint and the default Cmd defined in the Docker image

3008

# are ignored. Your command is run without any arguments.

3009

# - If you supply only args:

3010

# The default Entrypoint defined in the Docker image is run with the args

3011

# that you supplied.

3012

# - If you supply a command and args:

3013

# The default Entrypoint and the default Cmd defined in the Docker image

3014

# are ignored. Your command is run with your args.

3015

# It cannot be set if custom container image is

3016

# not provided.

3017

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

3018

# both cannot be set at the same time.

3019

"A String",

3020

],

3021

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

3022

# [Learn about restrictions on accelerator configurations for

3023

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

3024

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

3025

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

3026

# [accelerators for online

3027

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

3028

"count": "A String", # The number of accelerators to attach to each machine running the job.

3029

"type": "A String", # The type of accelerator to use.

3030

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3031

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

3032

# the one used in the custom container. This field is required if the replica

3033

# is a TPU worker that uses a custom container. Otherwise, do not specify

3034

# this field. This must be a [runtime version that currently supports

3035

# training with

3036

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

3037

#

3038

# Note that the version of TensorFlow included in a runtime version may

3039

# differ from the numbering of the runtime version itself, because it may

3040

# have a different [patch

3041

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

3042

# In this field, you must specify the runtime version (TensorFlow minor

3043

# version). For example, if your custom container runs TensorFlow `1.x.y`,

3044

# specify `1.x`.

3045

"containerCommand": [ # The command with which the replica's custom container is run.

3046

# If provided, it will override default ENTRYPOINT of the docker image.

3047

# If not provided, the docker image's ENTRYPOINT is used.

3048

# It cannot be set if custom container image is

3049

# not provided.

3050

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

3051

# both cannot be set at the same time.

3052

"A String",

3053

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3054

},

3055

"scaleTier": "A String", # Required. Specifies the machine types, the number of replicas for workers

3056

# and parameter servers.

3057

"jobDir": "A String", # Optional. A Google Cloud Storage path in which to store training outputs

3058

# and other data needed for training. This path is passed to your TensorFlow

3059

# program as the '--job-dir' command-line argument. The benefit of specifying

3060

# this field is that Cloud ML validates the path for use in training.

3061

"pythonVersion": "A String", # Optional. The version of Python used in training. You must either specify

3062

# this field or specify `masterConfig.imageUri`.

3063

#

3064

# The following Python versions are available:

3065

#

3066

# * Python '3.7' is available when `runtime_version` is set to '1.15' or

3067

# later.

3068

# * Python '3.5' is available when `runtime_version` is set to a version

3069

# from '1.4' to '1.14'.

3070

# * Python '2.7' is available when `runtime_version` is set to '1.15' or

3071

# earlier.

3072

#

3073

# Read more about the Python versions available for [each runtime

3074

# version](/ml-engine/docs/runtime-version-list).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3075

"scheduling": { # All parameters related to scheduling of training jobs. # Optional. Scheduling options for a training job.

3076

"maxWaitTime": "A String",

3077

"maxRunningTime": "A String", # Optional. The maximum job running time, expressed in seconds. The field can

3078

# contain up to nine fractional digits, terminated by `s`. If not specified,

3079

# this field defaults to `604800s` (seven days).

3080

#

3081

# If the training job is still running after this duration, AI Platform

3082

# Training cancels it.

3083

#

3084

# For example, if you want to ensure your job runs for no more than 2 hours,

3085

# set this field to `7200s` (2 hours * 60 minutes / hour * 60 seconds /

3086

# minute).

3087

#

3088

# If you submit your training job using the `gcloud` tool, you can [provide

3089

# this field in a `config.yaml`

3090

# file](/ai-platform/training/docs/training-jobs#formatting_your_configuration_parameters).

# For example:

#

# ```yaml

# trainingInput:

# ...

# scheduling:

# maxRunningTime: 7200s

3098

# ...

3099

# ```

3100

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3101

"network": "A String", # Optional. The full name of the Google Compute Engine

3102

# [network](/compute/docs/networks-and-firewalls#networks) to which the Job

3103

# is peered. For example, projects/12345/global/networks/myVPC. Format is of

3104

# the form projects/{project}/global/networks/{network}. Where {project} is a

3105

# project number, as in '12345', and {network} is network name.".

3106

#

3107

# Private services access must already be configured for the network. If left

3108

# unspecified, the Job is not peered with any network. Learn more -

3109

# Connecting Job to user network over private

3110

# IP.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3111

"evaluatorConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for evaluators.

3112

#

3113

# You should only set `evaluatorConfig.acceleratorConfig` if

3114

# `evaluatorType` is set to a Compute Engine machine type. [Learn

3115

# about restrictions on accelerator configurations for

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3116

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3117

#

3118

# Set `evaluatorConfig.imageUri` only if you build a custom image for

3119

# your evaluator. If `evaluatorConfig.imageUri` has not been

3120

# set, AI Platform uses the value of `masterConfig.imageUri`. Learn more about [configuring custom

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3121

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3122

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

3123

# Registry. Learn more about [configuring custom

3124

# containers](/ai-platform/training/docs/distributed-training-containers).

3125

"containerArgs": [ # Arguments to the entrypoint command.

3126

# The following rules apply for container_command and container_args:

3127

# - If you do not supply command or args:

3128

# The defaults defined in the Docker image are used.

3129

# - If you supply a command but no args:

3130

# The default EntryPoint and the default Cmd defined in the Docker image

3131

# are ignored. Your command is run without any arguments.

3132

# - If you supply only args:

3133

# The default Entrypoint defined in the Docker image is run with the args

3134

# that you supplied.

3135

# - If you supply a command and args:

3136

# The default Entrypoint and the default Cmd defined in the Docker image

3137

# are ignored. Your command is run with your args.

3138

# It cannot be set if custom container image is

3139

# not provided.

3140

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

3141

# both cannot be set at the same time.

3142

"A String",

3143

],

3144

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

3145

# [Learn about restrictions on accelerator configurations for

3146

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

3147

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

3148

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

3149

# [accelerators for online

3150

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

3151

"count": "A String", # The number of accelerators to attach to each machine running the job.

3152

"type": "A String", # The type of accelerator to use.

3153

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3154

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

3155

# the one used in the custom container. This field is required if the replica

3156

# is a TPU worker that uses a custom container. Otherwise, do not specify

3157

# this field. This must be a [runtime version that currently supports

3158

# training with

3159

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

3160

#

3161

# Note that the version of TensorFlow included in a runtime version may

3162

# differ from the numbering of the runtime version itself, because it may

3163

# have a different [patch

3164

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

3165

# In this field, you must specify the runtime version (TensorFlow minor

3166

# version). For example, if your custom container runs TensorFlow `1.x.y`,

3167

# specify `1.x`.

3168

"containerCommand": [ # The command with which the replica's custom container is run.

3169

# If provided, it will override default ENTRYPOINT of the docker image.

3170

# If not provided, the docker image's ENTRYPOINT is used.

3171

# It cannot be set if custom container image is

3172

# not provided.

3173

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

3174

# both cannot be set at the same time.

3175

"A String",

3176

],

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3177

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3178

"useChiefInTfConfig": True or False, # Optional. Use `chief` instead of `master` in the `TF_CONFIG` environment

3179

# variable when training with a custom container. Defaults to `false`. [Learn

3180

# more about this

3181

# field.](/ai-platform/training/docs/distributed-training-details#chief-versus-master)

3182

#

3183

# This field has no effect for training jobs that don't use a custom

3184

# container.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3185

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3186

"state": "A String", # Output only. The detailed state of a job.

3187

"jobId": "A String", # Required. The user-specified id of the job.

3188

"endTime": "A String", # Output only. When the job processing was completed.

3189

"startTime": "A String", # Output only. When the job processing was started.

3190

"predictionOutput": { # Represents results of a prediction job. # The current prediction job result.

3191

"errorCount": "A String", # The number of data instances which resulted in errors.

3192

"outputPath": "A String", # The output Google Cloud Storage location provided at the job creation time.

3193

"nodeHours": 3.14, # Node hours used by the batch prediction job.

3194

"predictionCount": "A String", # The number of generated predictions.

3195

},

3196

"trainingOutput": { # Represents results of a training job. Output only. # The current training job result.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3197

"isBuiltInAlgorithmJob": True or False, # Whether this job is a built-in Algorithm job.

3198

"builtInAlgorithmOutput": { # Represents output related to a built-in algorithm Job. # Details related to built-in algorithms jobs.

3199

# Only set for built-in algorithms jobs.

3200

"modelPath": "A String", # The Cloud Storage path to the `model/` directory where the training job

3201

# saves the trained model. Only set for successful jobs that don't use

3202

# hyperparameter tuning.

3203

"pythonVersion": "A String", # Python version on which the built-in algorithm was trained.

3204

"runtimeVersion": "A String", # AI Platform runtime version on which the built-in algorithm was

3205

# trained.

3206

"framework": "A String", # Framework on which the built-in algorithm was trained.

3207

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3208

"trials": [ # Results for individual Hyperparameter trials.

3209

# Only set for hyperparameter tuning jobs.

3210

{ # Represents the result of a single hyperparameter tuning trial from a

3211

# training job. The TrainingOutput object that is returned on successful

3212

# completion of a training job with hyperparameter tuning includes a list

3213

# of HyperparameterOutput objects, one for each successful trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3214

"trialId": "A String", # The trial id for these results.

3215

"endTime": "A String", # Output only. End time for the trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3216

"startTime": "A String", # Output only. Start time for the trial.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3217

"isTrialStoppedEarly": True or False, # True if the trial is stopped early.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3218

"finalMetric": { # An observed value of a metric. # The final objective metric seen for this trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3219

"trainingStep": "A String", # The global training step for this metric.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3220

"objectiveValue": 3.14, # The objective value at this training step.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3221

},

3222

"builtInAlgorithmOutput": { # Represents output related to a built-in algorithm Job. # Details related to built-in algorithms jobs.

3223

# Only set for trials of built-in algorithms jobs that have succeeded.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3224

"modelPath": "A String", # The Cloud Storage path to the `model/` directory where the training job

3225

# saves the trained model. Only set for successful jobs that don't use

3226

# hyperparameter tuning.

3227

"pythonVersion": "A String", # Python version on which the built-in algorithm was trained.

3228

"runtimeVersion": "A String", # AI Platform runtime version on which the built-in algorithm was

3229

# trained.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3230

"framework": "A String", # Framework on which the built-in algorithm was trained.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3231

},

3232

"state": "A String", # Output only. The detailed state of the trial.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3233

"allMetrics": [ # All recorded object metrics for this trial. This field is not currently

3234

# populated.

3235

{ # An observed value of a metric.

3236

"trainingStep": "A String", # The global training step for this metric.

3237

"objectiveValue": 3.14, # The objective value at this training step.

3238

},

3239

],

3240

"hyperparameters": { # The hyperparameters given to this trial.

3241

"a_key": "A String",

3242

},

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

3243

},

3244

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3245

"hyperparameterMetricTag": "A String", # The TensorFlow summary tag name used for optimizing hyperparameter tuning

3246

# trials. See

3247

# [`HyperparameterSpec.hyperparameterMetricTag`](#HyperparameterSpec.FIELDS.hyperparameter_metric_tag)

3248

# for more information. Only set for hyperparameter tuning jobs.

3249

"completedTrialCount": "A String", # The number of hyperparameter tuning trials that completed successfully.

3250

# Only set for hyperparameter tuning jobs.

3251

"isHyperparameterTuningJob": True or False, # Whether this job is a hyperparameter tuning job.

3252

"consumedMLUnits": 3.14, # The amount of ML units consumed by the job.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3253

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3254

"createTime": "A String", # Output only. When the job was created.

3255

"labels": { # Optional. One or more labels that you can add, to organize your jobs.

3256

# Each label is a key-value pair, where both the key and the value are

3257

# arbitrary strings that you supply.

3258

# For more information, see the documentation on

3259

# <a href="/ml-engine/docs/tensorflow/resource-labels">using labels</a>.

3260

"a_key": "A String",

3261

},

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

3262

},

3263

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3264

"nextPageToken": "A String", # Optional. Pass this token as the `page_token` field of the request for a

3265

# subsequent call.

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

}</pre>

</div>

<code class="details" id="list_next">list_next(previous_request, previous_response)</code>

3271

<pre>Retrieves the next page of results.

3272

3273

Args:

3274

previous_request: The request for the previous page. (required)

3275

previous_response: The response from the request for the previous page. (required)

3276

3277

Returns:

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3278

A request object that you can call 'execute()' on to request the next

Sai Cheemalapati

2017-03-13 12:12:03 -0400

[diff] [blame]

3279

page. Returns None if there are no more items in the collection.

</pre>

</div>

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

3283

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3284

<code class="details" id="patch">patch(name, body=None, updateMask=None, x__xgafv=None)</code>

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

3285

<pre>Updates a specific job resource.

3286

3287

Currently the only supported fields to update are `labels`.

3288

3289

Args:

3290

name: string, Required. The job name. (required)

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3291

body: object, The request body.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

3292

The object takes the form of:

3293

3294

{ # Represents a training or prediction job.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3295

"predictionInput": { # Represents input parameters for a prediction job. # Input parameters to create a prediction job.

3296

"versionName": "A String", # Use this field if you want to specify a version of the model to use. The

3297

# string is formatted the same way as `model_version`, with the addition

3298

# of the version information:

3299

#

3300

# `"projects/YOUR_PROJECT/models/YOUR_MODEL/versions/YOUR_VERSION"`

3301

"modelName": "A String", # Use this field if you want to use the default version for the specified

3302

# model. The string must use the following format:

3303

#

3304

# `"projects/YOUR_PROJECT/models/YOUR_MODEL"`

3305

"uri": "A String", # Use this field if you want to specify a Google Cloud Storage path for

3306

# the model to use.

3307

"maxWorkerCount": "A String", # Optional. The maximum number of workers to be used for parallel processing.

3308

# Defaults to 10 if not specified.

3309

"signatureName": "A String", # Optional. The name of the signature defined in the SavedModel to use for

3310

# this job. Please refer to

3311

# [SavedModel](https://tensorflow.github.io/serving/serving_basic.html)

3312

# for information about how to use signatures.

3313

#

3314

# Defaults to

3315

# [DEFAULT_SERVING_SIGNATURE_DEF_KEY](https://www.tensorflow.org/api_docs/python/tf/saved_model/signature_constants)

3316

# , which is "serving_default".

3317

"outputPath": "A String", # Required. The output Google Cloud Storage location.

3318

"outputDataFormat": "A String", # Optional. Format of the output data files, defaults to JSON.

3319

"dataFormat": "A String", # Required. The format of the input data files.

3320

"batchSize": "A String", # Optional. Number of records per batch, defaults to 64.

3321

# The service will buffer batch_size number of records in memory before

3322

# invoking one Tensorflow prediction call internally. So take the record

3323

# size and memory available into consideration when setting this parameter.

3324

"runtimeVersion": "A String", # Optional. The AI Platform runtime version to use for this batch

3325

# prediction. If not set, AI Platform will pick the runtime version used

3326

# during the CreateVersion request for this model version, or choose the

3327

# latest stable version when model version information is not available

3328

# such as when the model is specified by uri.

3329

"inputPaths": [ # Required. The Cloud Storage location of the input data files. May contain

3330

# <a href="/storage/docs/gsutil/addlhelp/WildcardNames">wildcards</a>.

3331

"A String",

3332

],

3333

"region": "A String", # Required. The Google Compute Engine region to run the prediction job in.

3334

# See the <a href="/ml-engine/docs/tensorflow/regions">available regions</a>

3335

# for AI Platform services.

3336

},

3337

"errorMessage": "A String", # Output only. The details of a failure or a cancellation.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3338

"etag": "A String", # `etag` is used for optimistic concurrency control as a way to help

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3339

# prevent simultaneous updates of a job from overwriting each other.

3340

# It is strongly suggested that systems make use of the `etag` in the

3341

# read-modify-write cycle to perform job updates in order to avoid race

3342

# conditions: An `etag` is returned in the response to `GetJob`, and

3343

# systems are expected to put that etag in the request to `UpdateJob` to

3344

# ensure that their change will be applied to the same version of the job.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3345

"trainingInput": { # Represents input parameters for a training job. When using the gcloud command # Input parameters to create a training job.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3346

# to submit your training job, you can specify the input parameters as

3347

# command-line arguments and/or in a YAML configuration file referenced from

3348

# the --config command-line argument. For details, see the guide to [submitting

3349

# a training job](/ai-platform/training/docs/training-jobs).

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3350

"workerCount": "A String", # Optional. The number of worker replicas to use for the training job. Each

3351

# replica in the cluster will be of the type specified in `worker_type`.

3352

#

3353

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

3354

# set this value, you must also set `worker_type`.

3355

#

3356

# The default value is zero.

3357

"pythonModule": "A String", # Required. The Python module name to run after installing the packages.

3358

"args": [ # Optional. Command-line arguments passed to the training application when it

3359

# starts. If your job uses a custom container, then the arguments are passed

3360

# to the container's <a class="external" target="_blank"

3361

# href="https://docs.docker.com/engine/reference/builder/#entrypoint">

3362

# `ENTRYPOINT`</a> command.

3363

"A String",

3364

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3365

"parameterServerConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for parameter servers.

3366

#

3367

# You should only set `parameterServerConfig.acceleratorConfig` if

3368

# `parameterServerType` is set to a Compute Engine machine type. [Learn

3369

# about restrictions on accelerator configurations for

3370

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

3371

#

3372

# Set `parameterServerConfig.imageUri` only if you build a custom image for

3373

# your parameter server. If `parameterServerConfig.imageUri` has not been

3374

# set, AI Platform uses the value of `masterConfig.imageUri`. Learn more about [configuring custom

3375

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3376

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

3377

# Registry. Learn more about [configuring custom

3378

# containers](/ai-platform/training/docs/distributed-training-containers).

3379

"containerArgs": [ # Arguments to the entrypoint command.

3380

# The following rules apply for container_command and container_args:

3381

# - If you do not supply command or args:

3382

# The defaults defined in the Docker image are used.

3383

# - If you supply a command but no args:

3384

# The default EntryPoint and the default Cmd defined in the Docker image

3385

# are ignored. Your command is run without any arguments.

3386

# - If you supply only args:

3387

# The default Entrypoint defined in the Docker image is run with the args

3388

# that you supplied.

3389

# - If you supply a command and args:

3390

# The default Entrypoint and the default Cmd defined in the Docker image

3391

# are ignored. Your command is run with your args.

3392

# It cannot be set if custom container image is

3393

# not provided.

3394

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

3395

# both cannot be set at the same time.

3396

"A String",

3397

],

3398

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

3399

# [Learn about restrictions on accelerator configurations for

3400

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

3401

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

3402

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

3403

# [accelerators for online

3404

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

3405

"count": "A String", # The number of accelerators to attach to each machine running the job.

3406

"type": "A String", # The type of accelerator to use.

3407

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3408

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

3409

# the one used in the custom container. This field is required if the replica

3410

# is a TPU worker that uses a custom container. Otherwise, do not specify

3411

# this field. This must be a [runtime version that currently supports

3412

# training with

3413

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

3414

#

3415

# Note that the version of TensorFlow included in a runtime version may

3416

# differ from the numbering of the runtime version itself, because it may

3417

# have a different [patch

3418

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

3419

# In this field, you must specify the runtime version (TensorFlow minor

3420

# version). For example, if your custom container runs TensorFlow `1.x.y`,

3421

# specify `1.x`.

3422

"containerCommand": [ # The command with which the replica's custom container is run.

3423

# If provided, it will override default ENTRYPOINT of the docker image.

3424

# If not provided, the docker image's ENTRYPOINT is used.

3425

# It cannot be set if custom container image is

3426

# not provided.

3427

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

3428

# both cannot be set at the same time.

3429

"A String",

3430

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3431

},

3432

"encryptionConfig": { # Represents a custom encryption key configuration that can be applied to # Optional. Options for using customer-managed encryption keys (CMEK) to

3433

# protect resources created by a training job, instead of using Google's

3434

# default encryption. If this is set, then all resources created by the

3435

# training job will be encrypted with the customer-managed encryption key

3436

# that you specify.

3437

#

3438

# [Learn how and when to use CMEK with AI Platform

3439

# Training](/ai-platform/training/docs/cmek).

3440

# a resource.

3441

"kmsKeyName": "A String", # The Cloud KMS resource identifier of the customer-managed encryption key

3442

# used to protect a resource, such as a training job. It has the following

3443

# format:

3444

# `projects/{PROJECT_ID}/locations/{REGION}/keyRings/{KEY_RING_NAME}/cryptoKeys/{KEY_NAME}`

3445

},

3446

"hyperparameters": { # Represents a set of hyperparameters to optimize. # Optional. The set of Hyperparameters to tune.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3447

"params": [ # Required. The set of parameters to tune.

3448

{ # Represents a single hyperparameter to optimize.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3449

"categoricalValues": [ # Required if type is `CATEGORICAL`. The list of possible categories.

3450

"A String",

3451

],

3452

"parameterName": "A String", # Required. The parameter name must be unique amongst all ParameterConfigs in

3453

# a HyperparameterSpec message. E.g., "learning_rate".

3454

"minValue": 3.14, # Required if type is `DOUBLE` or `INTEGER`. This field

3455

# should be unset if type is `CATEGORICAL`. This value should be integers if

3456

# type is INTEGER.

3457

"discreteValues": [ # Required if type is `DISCRETE`.

3458

# A list of feasible points.

3459

# The list should be in strictly increasing order. For instance, this

3460

# parameter might have possible settings of 1.5, 2.5, and 4.0. This list

3461

# should not contain more than 1,000 values.

3462

3.14,

3463

],

3464

"scaleType": "A String", # Optional. How the parameter should be scaled to the hypercube.

3465

# Leave unset for categorical parameters.

3466

# Some kind of scaling is strongly recommended for real or integral

3467

# parameters (e.g., `UNIT_LINEAR_SCALE`).

3468

"maxValue": 3.14, # Required if type is `DOUBLE` or `INTEGER`. This field

3469

# should be unset if type is `CATEGORICAL`. This value should be integers if

3470

# type is `INTEGER`.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3471

"type": "A String", # Required. The type of the parameter.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3472

},

3473

],

3474

"enableTrialEarlyStopping": True or False, # Optional. Indicates if the hyperparameter tuning job enables auto trial

3475

# early stopping.

3476

"resumePreviousJobId": "A String", # Optional. The prior hyperparameter tuning job id that users hope to

3477

# continue with. The job id will be used to find the corresponding vizier

3478

# study guid and resume the study.

3479

"maxParallelTrials": 42, # Optional. The number of training trials to run concurrently.

3480

# You can reduce the time it takes to perform hyperparameter tuning by adding

3481

# trials in parallel. However, each trail only benefits from the information

3482

# gained in completed trials. That means that a trial does not get access to

3483

# the results of trials running at the same time, which could reduce the

3484

# quality of the overall optimization.

3485

#

3486

# Each trial will use the same scale tier and machine types.

3487

#

3488

# Defaults to one.

3489

"maxFailedTrials": 42, # Optional. The number of failed trials that need to be seen before failing

3490

# the hyperparameter tuning job. You can specify this field to override the

3491

# default failing criteria for AI Platform hyperparameter tuning jobs.

3492

#

3493

# Defaults to zero, which means the service decides when a hyperparameter

3494

# job should fail.

3495

"goal": "A String", # Required. The type of goal to use for tuning. Available types are

3496

# `MAXIMIZE` and `MINIMIZE`.

3497

#

3498

# Defaults to `MAXIMIZE`.

3499

"maxTrials": 42, # Optional. How many training trials should be attempted to optimize

3500

# the specified hyperparameters.

3501

#

3502

# Defaults to one.

3503

"algorithm": "A String", # Optional. The search algorithm specified for the hyperparameter

3504

# tuning job.

3505

# Uses the default AI Platform hyperparameter tuning

3506

# algorithm if unspecified.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3507

"hyperparameterMetricTag": "A String", # Optional. The TensorFlow summary tag name to use for optimizing trials. For

3508

# current versions of TensorFlow, this tag name should exactly match what is

3509

# shown in TensorBoard, including all scopes. For versions of TensorFlow

3510

# prior to 0.12, this should be only the tag passed to tf.Summary.

3511

# By default, "training/hptuning/metric" will be used.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3512

},

3513

"workerConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for workers.

3514

#

3515

# You should only set `workerConfig.acceleratorConfig` if `workerType` is set

3516

# to a Compute Engine machine type. [Learn about restrictions on accelerator

3517

# configurations for

3518

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

3519

#

3520

# Set `workerConfig.imageUri` only if you build a custom image for your

3521

# worker. If `workerConfig.imageUri` has not been set, AI Platform uses

3522

# the value of `masterConfig.imageUri`. Learn more about [configuring custom

3523

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3524

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

3525

# Registry. Learn more about [configuring custom

3526

# containers](/ai-platform/training/docs/distributed-training-containers).

3527

"containerArgs": [ # Arguments to the entrypoint command.

3528

# The following rules apply for container_command and container_args:

3529

# - If you do not supply command or args:

3530

# The defaults defined in the Docker image are used.

3531

# - If you supply a command but no args:

3532

# The default EntryPoint and the default Cmd defined in the Docker image

3533

# are ignored. Your command is run without any arguments.

3534

# - If you supply only args:

3535

# The default Entrypoint defined in the Docker image is run with the args

3536

# that you supplied.

3537

# - If you supply a command and args:

3538

# The default Entrypoint and the default Cmd defined in the Docker image

3539

# are ignored. Your command is run with your args.

3540

# It cannot be set if custom container image is

3541

# not provided.

3542

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

3543

# both cannot be set at the same time.

3544

"A String",

3545

],

3546

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

3547

# [Learn about restrictions on accelerator configurations for

3548

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

3549

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

3550

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

3551

# [accelerators for online

3552

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

3553

"count": "A String", # The number of accelerators to attach to each machine running the job.

3554

"type": "A String", # The type of accelerator to use.

3555

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3556

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

3557

# the one used in the custom container. This field is required if the replica

3558

# is a TPU worker that uses a custom container. Otherwise, do not specify

3559

# this field. This must be a [runtime version that currently supports

3560

# training with

3561

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

3562

#

3563

# Note that the version of TensorFlow included in a runtime version may

3564

# differ from the numbering of the runtime version itself, because it may

3565

# have a different [patch

3566

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

3567

# In this field, you must specify the runtime version (TensorFlow minor

3568

# version). For example, if your custom container runs TensorFlow `1.x.y`,

3569

# specify `1.x`.

3570

"containerCommand": [ # The command with which the replica's custom container is run.

3571

# If provided, it will override default ENTRYPOINT of the docker image.

3572

# If not provided, the docker image's ENTRYPOINT is used.

3573

# It cannot be set if custom container image is

3574

# not provided.

3575

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

3576

# both cannot be set at the same time.

3577

"A String",

3578

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3579

},

3580

"parameterServerCount": "A String", # Optional. The number of parameter server replicas to use for the training

3581

# job. Each replica in the cluster will be of the type specified in

3582

# `parameter_server_type`.

3583

#

3584

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

3585

# set this value, you must also set `parameter_server_type`.

3586

#

3587

# The default value is zero.

3588

"packageUris": [ # Required. The Google Cloud Storage location of the packages with

3589

# the training program and any additional dependencies.

3590

# The maximum number of package URIs is 100.

3591

"A String",

3592

],

3593

"evaluatorCount": "A String", # Optional. The number of evaluator replicas to use for the training job.

3594

# Each replica in the cluster will be of the type specified in

3595

# `evaluator_type`.

3596

#

3597

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

3598

# set this value, you must also set `evaluator_type`.

3599

#

3600

# The default value is zero.

3601

"masterType": "A String", # Optional. Specifies the type of virtual machine to use for your training

3602

# job's master worker. You must specify this field when `scaleTier` is set to

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3603

# `CUSTOM`.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

3604

#

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3605

# You can use certain Compute Engine machine types directly in this field.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

3606

# The following types are supported:

3607

#

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

# - `n1-standard-4`

# - `n1-standard-8`

# - `n1-standard-16`

# - `n1-standard-32`

# - `n1-standard-64`

# - `n1-standard-96`

# - `n1-highmem-2`

# - `n1-highmem-4`

# - `n1-highmem-8`

# - `n1-highmem-16`

# - `n1-highmem-32`

# - `n1-highmem-64`

# - `n1-highmem-96`

# - `n1-highcpu-16`

# - `n1-highcpu-32`

# - `n1-highcpu-64`

# - `n1-highcpu-96`

#

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3626

# Learn more about [using Compute Engine machine

3627

# types](/ml-engine/docs/machine-types#compute-engine-machine-types).

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

3628

#

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3629

# Alternatively, you can use the following legacy machine types:

#

# - `standard`

# - `large_model`

# - `complex_model_s`

3634

# - `complex_model_m`

3635

# - `complex_model_l`

3636

# - `standard_gpu`

3637

# - `complex_model_m_gpu`

3638

# - `complex_model_l_gpu`

3639

# - `standard_p100`

3640

# - `complex_model_m_p100`

3641

# - `standard_v100`

3642

# - `large_model_v100`

3643

# - `complex_model_m_v100`

3644

# - `complex_model_l_v100`

3645

#

3646

# Learn more about [using legacy machine

3647

# types](/ml-engine/docs/machine-types#legacy-machine-types).

3648

#

3649

# Finally, if you want to use a TPU for training, specify `cloud_tpu` in this

3650

# field. Learn more about the [special configuration options for training

3651

# with

3652

# TPUs](/ml-engine/docs/tensorflow/using-tpus#configuring_a_custom_tpu_machine).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3653

"runtimeVersion": "A String", # Optional. The AI Platform runtime version to use for training. You must

3654

# either specify this field or specify `masterConfig.imageUri`.

3655

#

3656

# For more information, see the [runtime version

3657

# list](/ai-platform/training/docs/runtime-version-list) and learn [how to

3658

# manage runtime versions](/ai-platform/training/docs/versioning).

3659

"evaluatorType": "A String", # Optional. Specifies the type of virtual machine to use for your training

3660

# job's evaluator nodes.

3661

#

3662

# The supported values are the same as those described in the entry for

3663

# `masterType`.

3664

#

3665

# This value must be consistent with the category of machine type that

3666

# `masterType` uses. In other words, both must be Compute Engine machine

3667

# types or both must be legacy machine types.

3668

#

3669

# This value must be present when `scaleTier` is set to `CUSTOM` and

3670

# `evaluatorCount` is greater than zero.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3671

"workerType": "A String", # Optional. Specifies the type of virtual machine to use for your training

3672

# job's worker nodes.

3673

#

3674

# The supported values are the same as those described in the entry for

3675

# `masterType`.

3676

#

3677

# This value must be consistent with the category of machine type that

3678

# `masterType` uses. In other words, both must be Compute Engine machine

3679

# types or both must be legacy machine types.

3680

#

3681

# If you use `cloud_tpu` for this value, see special instructions for

3682

# [configuring a custom TPU

3683

# machine](/ml-engine/docs/tensorflow/using-tpus#configuring_a_custom_tpu_machine).

3684

#

3685

# This value must be present when `scaleTier` is set to `CUSTOM` and

3686

# `workerCount` is greater than zero.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3687

"region": "A String", # Required. The region to run the training job in. See the [available

3688

# regions](/ai-platform/training/docs/regions) for AI Platform Training.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3689

"parameterServerType": "A String", # Optional. Specifies the type of virtual machine to use for your training

3690

# job's parameter server.

3691

#

3692

# The supported values are the same as those described in the entry for

3693

# `master_type`.

3694

#

3695

# This value must be consistent with the category of machine type that

3696

# `masterType` uses. In other words, both must be Compute Engine machine

3697

# types or both must be legacy machine types.

3698

#

3699

# This value must be present when `scaleTier` is set to `CUSTOM` and

3700

# `parameter_server_count` is greater than zero.

3701

"masterConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for your master worker.

3702

#

3703

# You should only set `masterConfig.acceleratorConfig` if `masterType` is set

3704

# to a Compute Engine machine type. Learn about [restrictions on accelerator

3705

# configurations for

3706

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

3707

#

3708

# Set `masterConfig.imageUri` only if you build a custom image. Only one of

3709

# `masterConfig.imageUri` and `runtimeVersion` should be set. Learn more

3710

# about [configuring custom

3711

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3712

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

3713

# Registry. Learn more about [configuring custom

3714

# containers](/ai-platform/training/docs/distributed-training-containers).

3715

"containerArgs": [ # Arguments to the entrypoint command.

3716

# The following rules apply for container_command and container_args:

3717

# - If you do not supply command or args:

3718

# The defaults defined in the Docker image are used.

3719

# - If you supply a command but no args:

3720

# The default EntryPoint and the default Cmd defined in the Docker image

3721

# are ignored. Your command is run without any arguments.

3722

# - If you supply only args:

3723

# The default Entrypoint defined in the Docker image is run with the args

3724

# that you supplied.

3725

# - If you supply a command and args:

3726

# The default Entrypoint and the default Cmd defined in the Docker image

3727

# are ignored. Your command is run with your args.

3728

# It cannot be set if custom container image is

3729

# not provided.

3730

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

3731

# both cannot be set at the same time.

3732

"A String",

3733

],

3734

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

3735

# [Learn about restrictions on accelerator configurations for

3736

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

3737

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

3738

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

3739

# [accelerators for online

3740

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

3741

"count": "A String", # The number of accelerators to attach to each machine running the job.

3742

"type": "A String", # The type of accelerator to use.

3743

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3744

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

3745

# the one used in the custom container. This field is required if the replica

3746

# is a TPU worker that uses a custom container. Otherwise, do not specify

3747

# this field. This must be a [runtime version that currently supports

3748

# training with

3749

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

3750

#

3751

# Note that the version of TensorFlow included in a runtime version may

3752

# differ from the numbering of the runtime version itself, because it may

3753

# have a different [patch

3754

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

3755

# In this field, you must specify the runtime version (TensorFlow minor

3756

# version). For example, if your custom container runs TensorFlow `1.x.y`,

3757

# specify `1.x`.

3758

"containerCommand": [ # The command with which the replica's custom container is run.

3759

# If provided, it will override default ENTRYPOINT of the docker image.

3760

# If not provided, the docker image's ENTRYPOINT is used.

3761

# It cannot be set if custom container image is

3762

# not provided.

3763

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

3764

# both cannot be set at the same time.

3765

"A String",

3766

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3767

},

3768

"scaleTier": "A String", # Required. Specifies the machine types, the number of replicas for workers

3769

# and parameter servers.

3770

"jobDir": "A String", # Optional. A Google Cloud Storage path in which to store training outputs

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3771

# and other data needed for training. This path is passed to your TensorFlow

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3772

# program as the '--job-dir' command-line argument. The benefit of specifying

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3773

# this field is that Cloud ML validates the path for use in training.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3774

"pythonVersion": "A String", # Optional. The version of Python used in training. You must either specify

3775

# this field or specify `masterConfig.imageUri`.

3776

#

3777

# The following Python versions are available:

3778

#

3779

# * Python '3.7' is available when `runtime_version` is set to '1.15' or

3780

# later.

3781

# * Python '3.5' is available when `runtime_version` is set to a version

3782

# from '1.4' to '1.14'.

3783

# * Python '2.7' is available when `runtime_version` is set to '1.15' or

3784

# earlier.

3785

#

3786

# Read more about the Python versions available for [each runtime

3787

# version](/ml-engine/docs/runtime-version-list).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3788

"scheduling": { # All parameters related to scheduling of training jobs. # Optional. Scheduling options for a training job.

3789

"maxWaitTime": "A String",

3790

"maxRunningTime": "A String", # Optional. The maximum job running time, expressed in seconds. The field can

3791

# contain up to nine fractional digits, terminated by `s`. If not specified,

3792

# this field defaults to `604800s` (seven days).

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3793

#

3794

# If the training job is still running after this duration, AI Platform

3795

# Training cancels it.

3796

#

3797

# For example, if you want to ensure your job runs for no more than 2 hours,

3798

# set this field to `7200s` (2 hours * 60 minutes / hour * 60 seconds /

3799

# minute).

3800

#

3801

# If you submit your training job using the `gcloud` tool, you can [provide

3802

# this field in a `config.yaml`

3803

# file](/ai-platform/training/docs/training-jobs#formatting_your_configuration_parameters).

# For example:

#

# ```yaml

# trainingInput:

# ...

# scheduling:

# maxRunningTime: 7200s

3811

# ...

3812

# ```

3813

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3814

"network": "A String", # Optional. The full name of the Google Compute Engine

3815

# [network](/compute/docs/networks-and-firewalls#networks) to which the Job

3816

# is peered. For example, projects/12345/global/networks/myVPC. Format is of

3817

# the form projects/{project}/global/networks/{network}. Where {project} is a

3818

# project number, as in '12345', and {network} is network name.".

3819

#

3820

# Private services access must already be configured for the network. If left

3821

# unspecified, the Job is not peered with any network. Learn more -

3822

# Connecting Job to user network over private

3823

# IP.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3824

"evaluatorConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for evaluators.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3825

#

3826

# You should only set `evaluatorConfig.acceleratorConfig` if

3827

# `evaluatorType` is set to a Compute Engine machine type. [Learn

3828

# about restrictions on accelerator configurations for

3829

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

3830

#

3831

# Set `evaluatorConfig.imageUri` only if you build a custom image for

3832

# your evaluator. If `evaluatorConfig.imageUri` has not been

3833

# set, AI Platform uses the value of `masterConfig.imageUri`. Learn more about [configuring custom

3834

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3835

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

3836

# Registry. Learn more about [configuring custom

3837

# containers](/ai-platform/training/docs/distributed-training-containers).

3838

"containerArgs": [ # Arguments to the entrypoint command.

3839

# The following rules apply for container_command and container_args:

3840

# - If you do not supply command or args:

3841

# The defaults defined in the Docker image are used.

3842

# - If you supply a command but no args:

3843

# The default EntryPoint and the default Cmd defined in the Docker image

3844

# are ignored. Your command is run without any arguments.

3845

# - If you supply only args:

3846

# The default Entrypoint defined in the Docker image is run with the args

3847

# that you supplied.

3848

# - If you supply a command and args:

3849

# The default Entrypoint and the default Cmd defined in the Docker image

3850

# are ignored. Your command is run with your args.

3851

# It cannot be set if custom container image is

3852

# not provided.

3853

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

3854

# both cannot be set at the same time.

3855

"A String",

3856

],

3857

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3858

# [Learn about restrictions on accelerator configurations for

3859

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

3860

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

3861

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

3862

# [accelerators for online

3863

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3864

"count": "A String", # The number of accelerators to attach to each machine running the job.

3865

"type": "A String", # The type of accelerator to use.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3866

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3867

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

3868

# the one used in the custom container. This field is required if the replica

3869

# is a TPU worker that uses a custom container. Otherwise, do not specify

3870

# this field. This must be a [runtime version that currently supports

3871

# training with

3872

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

3873

#

3874

# Note that the version of TensorFlow included in a runtime version may

3875

# differ from the numbering of the runtime version itself, because it may

3876

# have a different [patch

3877

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

3878

# In this field, you must specify the runtime version (TensorFlow minor

3879

# version). For example, if your custom container runs TensorFlow `1.x.y`,

3880

# specify `1.x`.

3881

"containerCommand": [ # The command with which the replica's custom container is run.

3882

# If provided, it will override default ENTRYPOINT of the docker image.

3883

# If not provided, the docker image's ENTRYPOINT is used.

3884

# It cannot be set if custom container image is

3885

# not provided.

3886

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

3887

# both cannot be set at the same time.

3888

"A String",

3889

],

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3890

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3891

"useChiefInTfConfig": True or False, # Optional. Use `chief` instead of `master` in the `TF_CONFIG` environment

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3892

# variable when training with a custom container. Defaults to `false`. [Learn

3893

# more about this

3894

# field.](/ai-platform/training/docs/distributed-training-details#chief-versus-master)

3895

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3896

# This field has no effect for training jobs that don't use a custom

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3897

# container.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3898

},

3899

"state": "A String", # Output only. The detailed state of a job.

3900

"jobId": "A String", # Required. The user-specified id of the job.

3901

"endTime": "A String", # Output only. When the job processing was completed.

3902

"startTime": "A String", # Output only. When the job processing was started.

3903

"predictionOutput": { # Represents results of a prediction job. # The current prediction job result.

3904

"errorCount": "A String", # The number of data instances which resulted in errors.

3905

"outputPath": "A String", # The output Google Cloud Storage location provided at the job creation time.

3906

"nodeHours": 3.14, # Node hours used by the batch prediction job.

3907

"predictionCount": "A String", # The number of generated predictions.

3908

},

3909

"trainingOutput": { # Represents results of a training job. Output only. # The current training job result.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3910

"isBuiltInAlgorithmJob": True or False, # Whether this job is a built-in Algorithm job.

3911

"builtInAlgorithmOutput": { # Represents output related to a built-in algorithm Job. # Details related to built-in algorithms jobs.

3912

# Only set for built-in algorithms jobs.

3913

"modelPath": "A String", # The Cloud Storage path to the `model/` directory where the training job

3914

# saves the trained model. Only set for successful jobs that don't use

3915

# hyperparameter tuning.

3916

"pythonVersion": "A String", # Python version on which the built-in algorithm was trained.

3917

"runtimeVersion": "A String", # AI Platform runtime version on which the built-in algorithm was

3918

# trained.

3919

"framework": "A String", # Framework on which the built-in algorithm was trained.

3920

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3921

"trials": [ # Results for individual Hyperparameter trials.

3922

# Only set for hyperparameter tuning jobs.

3923

{ # Represents the result of a single hyperparameter tuning trial from a

3924

# training job. The TrainingOutput object that is returned on successful

3925

# completion of a training job with hyperparameter tuning includes a list

3926

# of HyperparameterOutput objects, one for each successful trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3927

"trialId": "A String", # The trial id for these results.

3928

"endTime": "A String", # Output only. End time for the trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3929

"startTime": "A String", # Output only. Start time for the trial.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3930

"isTrialStoppedEarly": True or False, # True if the trial is stopped early.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3931

"finalMetric": { # An observed value of a metric. # The final objective metric seen for this trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3932

"trainingStep": "A String", # The global training step for this metric.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3933

"objectiveValue": 3.14, # The objective value at this training step.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3934

},

3935

"builtInAlgorithmOutput": { # Represents output related to a built-in algorithm Job. # Details related to built-in algorithms jobs.

3936

# Only set for trials of built-in algorithms jobs that have succeeded.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3937

"modelPath": "A String", # The Cloud Storage path to the `model/` directory where the training job

3938

# saves the trained model. Only set for successful jobs that don't use

3939

# hyperparameter tuning.

3940

"pythonVersion": "A String", # Python version on which the built-in algorithm was trained.

3941

"runtimeVersion": "A String", # AI Platform runtime version on which the built-in algorithm was

3942

# trained.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3943

"framework": "A String", # Framework on which the built-in algorithm was trained.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3944

},

3945

"state": "A String", # Output only. The detailed state of the trial.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

3946

"allMetrics": [ # All recorded object metrics for this trial. This field is not currently

3947

# populated.

3948

{ # An observed value of a metric.

3949

"trainingStep": "A String", # The global training step for this metric.

3950

"objectiveValue": 3.14, # The objective value at this training step.

3951

},

3952

],

3953

"hyperparameters": { # The hyperparameters given to this trial.

3954

"a_key": "A String",

3955

},

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

3956

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3957

],

3958

"hyperparameterMetricTag": "A String", # The TensorFlow summary tag name used for optimizing hyperparameter tuning

3959

# trials. See

3960

# [`HyperparameterSpec.hyperparameterMetricTag`](#HyperparameterSpec.FIELDS.hyperparameter_metric_tag)

3961

# for more information. Only set for hyperparameter tuning jobs.

3962

"completedTrialCount": "A String", # The number of hyperparameter tuning trials that completed successfully.

3963

# Only set for hyperparameter tuning jobs.

3964

"isHyperparameterTuningJob": True or False, # Whether this job is a hyperparameter tuning job.

3965

"consumedMLUnits": 3.14, # The amount of ML units consumed by the job.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

3966

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3967

"createTime": "A String", # Output only. When the job was created.

3968

"labels": { # Optional. One or more labels that you can add, to organize your jobs.

3969

# Each label is a key-value pair, where both the key and the value are

3970

# arbitrary strings that you supply.

3971

# For more information, see the documentation on

3972

# <a href="/ml-engine/docs/tensorflow/resource-labels">using labels</a>.

3973

"a_key": "A String",

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

3974

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

3975

}

3976

3977

updateMask: string, Required. Specifies the path, relative to `Job`, of the field to update.

3978

To adopt etag mechanism, include `etag` field in the mask, and include the

3979

`etag` value in your job resource.

3980

3981

For example, to change the labels of a job, the `update_mask` parameter

3982

would be specified as `labels`, `etag`, and the

3983

`PATCH` request body would specify the new value, as follows:

{

"labels": {

"owner": "Google",

"color": "Blue"

}

"etag": "33a64df551425fcc55e4d42a148795d9f25f89d4"

3990

}

3991

If `etag` matches the one on the server, the labels of the job will be

3992

replaced with the given ones, and the server end `etag` will be

3993

recalculated.

3994

3995

Currently the only supported update masks are `labels` and `etag`.

3996

x__xgafv: string, V1 error format.

Allowed values

1 - v1 error format

2 - v2 error format

Returns:

An object of the form:

4003

4004

{ # Represents a training or prediction job.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4005

"predictionInput": { # Represents input parameters for a prediction job. # Input parameters to create a prediction job.

4006

"versionName": "A String", # Use this field if you want to specify a version of the model to use. The

4007

# string is formatted the same way as `model_version`, with the addition

4008

# of the version information:

4009

#

4010

# `"projects/YOUR_PROJECT/models/YOUR_MODEL/versions/YOUR_VERSION"`

4011

"modelName": "A String", # Use this field if you want to use the default version for the specified

4012

# model. The string must use the following format:

4013

#

4014

# `"projects/YOUR_PROJECT/models/YOUR_MODEL"`

4015

"uri": "A String", # Use this field if you want to specify a Google Cloud Storage path for

4016

# the model to use.

4017

"maxWorkerCount": "A String", # Optional. The maximum number of workers to be used for parallel processing.

4018

# Defaults to 10 if not specified.

4019

"signatureName": "A String", # Optional. The name of the signature defined in the SavedModel to use for

4020

# this job. Please refer to

4021

# [SavedModel](https://tensorflow.github.io/serving/serving_basic.html)

4022

# for information about how to use signatures.

4023

#

4024

# Defaults to

4025

# [DEFAULT_SERVING_SIGNATURE_DEF_KEY](https://www.tensorflow.org/api_docs/python/tf/saved_model/signature_constants)

4026

# , which is "serving_default".

4027

"outputPath": "A String", # Required. The output Google Cloud Storage location.

4028

"outputDataFormat": "A String", # Optional. Format of the output data files, defaults to JSON.

4029

"dataFormat": "A String", # Required. The format of the input data files.

4030

"batchSize": "A String", # Optional. Number of records per batch, defaults to 64.

4031

# The service will buffer batch_size number of records in memory before

4032

# invoking one Tensorflow prediction call internally. So take the record

4033

# size and memory available into consideration when setting this parameter.

4034

"runtimeVersion": "A String", # Optional. The AI Platform runtime version to use for this batch

4035

# prediction. If not set, AI Platform will pick the runtime version used

4036

# during the CreateVersion request for this model version, or choose the

4037

# latest stable version when model version information is not available

4038

# such as when the model is specified by uri.

4039

"inputPaths": [ # Required. The Cloud Storage location of the input data files. May contain

4040

# <a href="/storage/docs/gsutil/addlhelp/WildcardNames">wildcards</a>.

4041

"A String",

4042

],

4043

"region": "A String", # Required. The Google Compute Engine region to run the prediction job in.

4044

# See the <a href="/ml-engine/docs/tensorflow/regions">available regions</a>

4045

# for AI Platform services.

4046

},

4047

"errorMessage": "A String", # Output only. The details of a failure or a cancellation.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4048

"etag": "A String", # `etag` is used for optimistic concurrency control as a way to help

4049

# prevent simultaneous updates of a job from overwriting each other.

4050

# It is strongly suggested that systems make use of the `etag` in the

4051

# read-modify-write cycle to perform job updates in order to avoid race

4052

# conditions: An `etag` is returned in the response to `GetJob`, and

4053

# systems are expected to put that etag in the request to `UpdateJob` to

4054

# ensure that their change will be applied to the same version of the job.

4055

"trainingInput": { # Represents input parameters for a training job. When using the gcloud command # Input parameters to create a training job.

4056

# to submit your training job, you can specify the input parameters as

4057

# command-line arguments and/or in a YAML configuration file referenced from

4058

# the --config command-line argument. For details, see the guide to [submitting

4059

# a training job](/ai-platform/training/docs/training-jobs).

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4060

"workerCount": "A String", # Optional. The number of worker replicas to use for the training job. Each

4061

# replica in the cluster will be of the type specified in `worker_type`.

4062

#

4063

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

4064

# set this value, you must also set `worker_type`.

4065

#

4066

# The default value is zero.

4067

"pythonModule": "A String", # Required. The Python module name to run after installing the packages.

4068

"args": [ # Optional. Command-line arguments passed to the training application when it

4069

# starts. If your job uses a custom container, then the arguments are passed

4070

# to the container's <a class="external" target="_blank"

4071

# href="https://docs.docker.com/engine/reference/builder/#entrypoint">

4072

# `ENTRYPOINT`</a> command.

4073

"A String",

4074

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4075

"parameterServerConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for parameter servers.

4076

#

4077

# You should only set `parameterServerConfig.acceleratorConfig` if

4078

# `parameterServerType` is set to a Compute Engine machine type. [Learn

4079

# about restrictions on accelerator configurations for

4080

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

4081

#

4082

# Set `parameterServerConfig.imageUri` only if you build a custom image for

4083

# your parameter server. If `parameterServerConfig.imageUri` has not been

4084

# set, AI Platform uses the value of `masterConfig.imageUri`. Learn more about [configuring custom

4085

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4086

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

4087

# Registry. Learn more about [configuring custom

4088

# containers](/ai-platform/training/docs/distributed-training-containers).

4089

"containerArgs": [ # Arguments to the entrypoint command.

4090

# The following rules apply for container_command and container_args:

4091

# - If you do not supply command or args:

4092

# The defaults defined in the Docker image are used.

4093

# - If you supply a command but no args:

4094

# The default EntryPoint and the default Cmd defined in the Docker image

4095

# are ignored. Your command is run without any arguments.

4096

# - If you supply only args:

4097

# The default Entrypoint defined in the Docker image is run with the args

4098

# that you supplied.

4099

# - If you supply a command and args:

4100

# The default Entrypoint and the default Cmd defined in the Docker image

4101

# are ignored. Your command is run with your args.

4102

# It cannot be set if custom container image is

4103

# not provided.

4104

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

4105

# both cannot be set at the same time.

4106

"A String",

4107

],

4108

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

4109

# [Learn about restrictions on accelerator configurations for

4110

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

4111

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

4112

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

4113

# [accelerators for online

4114

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

4115

"count": "A String", # The number of accelerators to attach to each machine running the job.

4116

"type": "A String", # The type of accelerator to use.

4117

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4118

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

4119

# the one used in the custom container. This field is required if the replica

4120

# is a TPU worker that uses a custom container. Otherwise, do not specify

4121

# this field. This must be a [runtime version that currently supports

4122

# training with

4123

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

4124

#

4125

# Note that the version of TensorFlow included in a runtime version may

4126

# differ from the numbering of the runtime version itself, because it may

4127

# have a different [patch

4128

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

4129

# In this field, you must specify the runtime version (TensorFlow minor

4130

# version). For example, if your custom container runs TensorFlow `1.x.y`,

4131

# specify `1.x`.

4132

"containerCommand": [ # The command with which the replica's custom container is run.

4133

# If provided, it will override default ENTRYPOINT of the docker image.

4134

# If not provided, the docker image's ENTRYPOINT is used.

4135

# It cannot be set if custom container image is

4136

# not provided.

4137

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

4138

# both cannot be set at the same time.

4139

"A String",

4140

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4141

},

4142

"encryptionConfig": { # Represents a custom encryption key configuration that can be applied to # Optional. Options for using customer-managed encryption keys (CMEK) to

4143

# protect resources created by a training job, instead of using Google's

4144

# default encryption. If this is set, then all resources created by the

4145

# training job will be encrypted with the customer-managed encryption key

4146

# that you specify.

4147

#

4148

# [Learn how and when to use CMEK with AI Platform

4149

# Training](/ai-platform/training/docs/cmek).

4150

# a resource.

4151

"kmsKeyName": "A String", # The Cloud KMS resource identifier of the customer-managed encryption key

4152

# used to protect a resource, such as a training job. It has the following

4153

# format:

4154

# `projects/{PROJECT_ID}/locations/{REGION}/keyRings/{KEY_RING_NAME}/cryptoKeys/{KEY_NAME}`

4155

},

4156

"hyperparameters": { # Represents a set of hyperparameters to optimize. # Optional. The set of Hyperparameters to tune.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4157

"params": [ # Required. The set of parameters to tune.

4158

{ # Represents a single hyperparameter to optimize.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4159

"categoricalValues": [ # Required if type is `CATEGORICAL`. The list of possible categories.

4160

"A String",

4161

],

4162

"parameterName": "A String", # Required. The parameter name must be unique amongst all ParameterConfigs in

4163

# a HyperparameterSpec message. E.g., "learning_rate".

4164

"minValue": 3.14, # Required if type is `DOUBLE` or `INTEGER`. This field

4165

# should be unset if type is `CATEGORICAL`. This value should be integers if

4166

# type is INTEGER.

4167

"discreteValues": [ # Required if type is `DISCRETE`.

4168

# A list of feasible points.

4169

# The list should be in strictly increasing order. For instance, this

4170

# parameter might have possible settings of 1.5, 2.5, and 4.0. This list

4171

# should not contain more than 1,000 values.

4172

3.14,

4173

],

4174

"scaleType": "A String", # Optional. How the parameter should be scaled to the hypercube.

4175

# Leave unset for categorical parameters.

4176

# Some kind of scaling is strongly recommended for real or integral

4177

# parameters (e.g., `UNIT_LINEAR_SCALE`).

4178

"maxValue": 3.14, # Required if type is `DOUBLE` or `INTEGER`. This field

4179

# should be unset if type is `CATEGORICAL`. This value should be integers if

4180

# type is `INTEGER`.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4181

"type": "A String", # Required. The type of the parameter.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4182

},

4183

],

4184

"enableTrialEarlyStopping": True or False, # Optional. Indicates if the hyperparameter tuning job enables auto trial

4185

# early stopping.

4186

"resumePreviousJobId": "A String", # Optional. The prior hyperparameter tuning job id that users hope to

4187

# continue with. The job id will be used to find the corresponding vizier

4188

# study guid and resume the study.

4189

"maxParallelTrials": 42, # Optional. The number of training trials to run concurrently.

4190

# You can reduce the time it takes to perform hyperparameter tuning by adding

4191

# trials in parallel. However, each trail only benefits from the information

4192

# gained in completed trials. That means that a trial does not get access to

4193

# the results of trials running at the same time, which could reduce the

4194

# quality of the overall optimization.

4195

#

4196

# Each trial will use the same scale tier and machine types.

4197

#

4198

# Defaults to one.

4199

"maxFailedTrials": 42, # Optional. The number of failed trials that need to be seen before failing

4200

# the hyperparameter tuning job. You can specify this field to override the

4201

# default failing criteria for AI Platform hyperparameter tuning jobs.

4202

#

4203

# Defaults to zero, which means the service decides when a hyperparameter

4204

# job should fail.

4205

"goal": "A String", # Required. The type of goal to use for tuning. Available types are

4206

# `MAXIMIZE` and `MINIMIZE`.

4207

#

4208

# Defaults to `MAXIMIZE`.

4209

"maxTrials": 42, # Optional. How many training trials should be attempted to optimize

4210

# the specified hyperparameters.

4211

#

4212

# Defaults to one.

4213

"algorithm": "A String", # Optional. The search algorithm specified for the hyperparameter

4214

# tuning job.

4215

# Uses the default AI Platform hyperparameter tuning

4216

# algorithm if unspecified.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4217

"hyperparameterMetricTag": "A String", # Optional. The TensorFlow summary tag name to use for optimizing trials. For

4218

# current versions of TensorFlow, this tag name should exactly match what is

4219

# shown in TensorBoard, including all scopes. For versions of TensorFlow

4220

# prior to 0.12, this should be only the tag passed to tf.Summary.

4221

# By default, "training/hptuning/metric" will be used.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4222

},

4223

"workerConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for workers.

4224

#

4225

# You should only set `workerConfig.acceleratorConfig` if `workerType` is set

4226

# to a Compute Engine machine type. [Learn about restrictions on accelerator

4227

# configurations for

4228

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

4229

#

4230

# Set `workerConfig.imageUri` only if you build a custom image for your

4231

# worker. If `workerConfig.imageUri` has not been set, AI Platform uses

4232

# the value of `masterConfig.imageUri`. Learn more about [configuring custom

4233

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4234

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

4235

# Registry. Learn more about [configuring custom

4236

# containers](/ai-platform/training/docs/distributed-training-containers).

4237

"containerArgs": [ # Arguments to the entrypoint command.

4238

# The following rules apply for container_command and container_args:

4239

# - If you do not supply command or args:

4240

# The defaults defined in the Docker image are used.

4241

# - If you supply a command but no args:

4242

# The default EntryPoint and the default Cmd defined in the Docker image

4243

# are ignored. Your command is run without any arguments.

4244

# - If you supply only args:

4245

# The default Entrypoint defined in the Docker image is run with the args

4246

# that you supplied.

4247

# - If you supply a command and args:

4248

# The default Entrypoint and the default Cmd defined in the Docker image

4249

# are ignored. Your command is run with your args.

4250

# It cannot be set if custom container image is

4251

# not provided.

4252

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

4253

# both cannot be set at the same time.

4254

"A String",

4255

],

4256

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

4257

# [Learn about restrictions on accelerator configurations for

4258

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

4259

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

4260

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

4261

# [accelerators for online

4262

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

4263

"count": "A String", # The number of accelerators to attach to each machine running the job.

4264

"type": "A String", # The type of accelerator to use.

4265

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4266

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

4267

# the one used in the custom container. This field is required if the replica

4268

# is a TPU worker that uses a custom container. Otherwise, do not specify

4269

# this field. This must be a [runtime version that currently supports

4270

# training with

4271

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

4272

#

4273

# Note that the version of TensorFlow included in a runtime version may

4274

# differ from the numbering of the runtime version itself, because it may

4275

# have a different [patch

4276

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

4277

# In this field, you must specify the runtime version (TensorFlow minor

4278

# version). For example, if your custom container runs TensorFlow `1.x.y`,

4279

# specify `1.x`.

4280

"containerCommand": [ # The command with which the replica's custom container is run.

4281

# If provided, it will override default ENTRYPOINT of the docker image.

4282

# If not provided, the docker image's ENTRYPOINT is used.

4283

# It cannot be set if custom container image is

4284

# not provided.

4285

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

4286

# both cannot be set at the same time.

4287

"A String",

4288

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4289

},

4290

"parameterServerCount": "A String", # Optional. The number of parameter server replicas to use for the training

4291

# job. Each replica in the cluster will be of the type specified in

4292

# `parameter_server_type`.

4293

#

4294

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

4295

# set this value, you must also set `parameter_server_type`.

4296

#

4297

# The default value is zero.

4298

"packageUris": [ # Required. The Google Cloud Storage location of the packages with

4299

# the training program and any additional dependencies.

4300

# The maximum number of package URIs is 100.

4301

"A String",

4302

],

4303

"evaluatorCount": "A String", # Optional. The number of evaluator replicas to use for the training job.

4304

# Each replica in the cluster will be of the type specified in

4305

# `evaluator_type`.

4306

#

4307

# This value can only be used when `scale_tier` is set to `CUSTOM`. If you

4308

# set this value, you must also set `evaluator_type`.

4309

#

4310

# The default value is zero.

4311

"masterType": "A String", # Optional. Specifies the type of virtual machine to use for your training

4312

# job's master worker. You must specify this field when `scaleTier` is set to

4313

# `CUSTOM`.

4314

#

4315

# You can use certain Compute Engine machine types directly in this field.

4316

# The following types are supported:

#

# - `n1-standard-4`

# - `n1-standard-8`

# - `n1-standard-16`

# - `n1-standard-32`

# - `n1-standard-64`

# - `n1-standard-96`

# - `n1-highmem-2`

# - `n1-highmem-4`

# - `n1-highmem-8`

# - `n1-highmem-16`

# - `n1-highmem-32`

# - `n1-highmem-64`

# - `n1-highmem-96`

# - `n1-highcpu-16`

# - `n1-highcpu-32`

# - `n1-highcpu-64`

# - `n1-highcpu-96`

#

# Learn more about [using Compute Engine machine

4337

# types](/ml-engine/docs/machine-types#compute-engine-machine-types).

4338

#

4339

# Alternatively, you can use the following legacy machine types:

#

# - `standard`

# - `large_model`

# - `complex_model_s`

4344

# - `complex_model_m`

4345

# - `complex_model_l`

4346

# - `standard_gpu`

4347

# - `complex_model_m_gpu`

4348

# - `complex_model_l_gpu`

4349

# - `standard_p100`

4350

# - `complex_model_m_p100`

4351

# - `standard_v100`

4352

# - `large_model_v100`

4353

# - `complex_model_m_v100`

4354

# - `complex_model_l_v100`

4355

#

4356

# Learn more about [using legacy machine

4357

# types](/ml-engine/docs/machine-types#legacy-machine-types).

4358

#

4359

# Finally, if you want to use a TPU for training, specify `cloud_tpu` in this

4360

# field. Learn more about the [special configuration options for training

4361

# with

4362

# TPUs](/ml-engine/docs/tensorflow/using-tpus#configuring_a_custom_tpu_machine).

4363

"runtimeVersion": "A String", # Optional. The AI Platform runtime version to use for training. You must

4364

# either specify this field or specify `masterConfig.imageUri`.

4365

#

4366

# For more information, see the [runtime version

4367

# list](/ai-platform/training/docs/runtime-version-list) and learn [how to

4368

# manage runtime versions](/ai-platform/training/docs/versioning).

4369

"evaluatorType": "A String", # Optional. Specifies the type of virtual machine to use for your training

4370

# job's evaluator nodes.

4371

#

4372

# The supported values are the same as those described in the entry for

4373

# `masterType`.

4374

#

4375

# This value must be consistent with the category of machine type that

4376

# `masterType` uses. In other words, both must be Compute Engine machine

4377

# types or both must be legacy machine types.

4378

#

4379

# This value must be present when `scaleTier` is set to `CUSTOM` and

4380

# `evaluatorCount` is greater than zero.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4381

"workerType": "A String", # Optional. Specifies the type of virtual machine to use for your training

4382

# job's worker nodes.

4383

#

4384

# The supported values are the same as those described in the entry for

4385

# `masterType`.

4386

#

4387

# This value must be consistent with the category of machine type that

4388

# `masterType` uses. In other words, both must be Compute Engine machine

4389

# types or both must be legacy machine types.

4390

#

4391

# If you use `cloud_tpu` for this value, see special instructions for

4392

# [configuring a custom TPU

4393

# machine](/ml-engine/docs/tensorflow/using-tpus#configuring_a_custom_tpu_machine).

4394

#

4395

# This value must be present when `scaleTier` is set to `CUSTOM` and

4396

# `workerCount` is greater than zero.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4397

"region": "A String", # Required. The region to run the training job in. See the [available

4398

# regions](/ai-platform/training/docs/regions) for AI Platform Training.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4399

"parameterServerType": "A String", # Optional. Specifies the type of virtual machine to use for your training

4400

# job's parameter server.

4401

#

4402

# The supported values are the same as those described in the entry for

4403

# `master_type`.

4404

#

4405

# This value must be consistent with the category of machine type that

4406

# `masterType` uses. In other words, both must be Compute Engine machine

4407

# types or both must be legacy machine types.

4408

#

4409

# This value must be present when `scaleTier` is set to `CUSTOM` and

4410

# `parameter_server_count` is greater than zero.

4411

"masterConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for your master worker.

4412

#

4413

# You should only set `masterConfig.acceleratorConfig` if `masterType` is set

4414

# to a Compute Engine machine type. Learn about [restrictions on accelerator

4415

# configurations for

4416

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

4417

#

4418

# Set `masterConfig.imageUri` only if you build a custom image. Only one of

4419

# `masterConfig.imageUri` and `runtimeVersion` should be set. Learn more

4420

# about [configuring custom

4421

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4422

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

4423

# Registry. Learn more about [configuring custom

4424

# containers](/ai-platform/training/docs/distributed-training-containers).

4425

"containerArgs": [ # Arguments to the entrypoint command.

4426

# The following rules apply for container_command and container_args:

4427

# - If you do not supply command or args:

4428

# The defaults defined in the Docker image are used.

4429

# - If you supply a command but no args:

4430

# The default EntryPoint and the default Cmd defined in the Docker image

4431

# are ignored. Your command is run without any arguments.

4432

# - If you supply only args:

4433

# The default Entrypoint defined in the Docker image is run with the args

4434

# that you supplied.

4435

# - If you supply a command and args:

4436

# The default Entrypoint and the default Cmd defined in the Docker image

4437

# are ignored. Your command is run with your args.

4438

# It cannot be set if custom container image is

4439

# not provided.

4440

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

4441

# both cannot be set at the same time.

4442

"A String",

4443

],

4444

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

4445

# [Learn about restrictions on accelerator configurations for

4446

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

4447

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

4448

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

4449

# [accelerators for online

4450

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

4451

"count": "A String", # The number of accelerators to attach to each machine running the job.

4452

"type": "A String", # The type of accelerator to use.

4453

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4454

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

4455

# the one used in the custom container. This field is required if the replica

4456

# is a TPU worker that uses a custom container. Otherwise, do not specify

4457

# this field. This must be a [runtime version that currently supports

4458

# training with

4459

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

4460

#

4461

# Note that the version of TensorFlow included in a runtime version may

4462

# differ from the numbering of the runtime version itself, because it may

4463

# have a different [patch

4464

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

4465

# In this field, you must specify the runtime version (TensorFlow minor

4466

# version). For example, if your custom container runs TensorFlow `1.x.y`,

4467

# specify `1.x`.

4468

"containerCommand": [ # The command with which the replica's custom container is run.

4469

# If provided, it will override default ENTRYPOINT of the docker image.

4470

# If not provided, the docker image's ENTRYPOINT is used.

4471

# It cannot be set if custom container image is

4472

# not provided.

4473

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

4474

# both cannot be set at the same time.

4475

"A String",

4476

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4477

},

4478

"scaleTier": "A String", # Required. Specifies the machine types, the number of replicas for workers

4479

# and parameter servers.

4480

"jobDir": "A String", # Optional. A Google Cloud Storage path in which to store training outputs

4481

# and other data needed for training. This path is passed to your TensorFlow

4482

# program as the '--job-dir' command-line argument. The benefit of specifying

4483

# this field is that Cloud ML validates the path for use in training.

4484

"pythonVersion": "A String", # Optional. The version of Python used in training. You must either specify

4485

# this field or specify `masterConfig.imageUri`.

4486

#

4487

# The following Python versions are available:

4488

#

4489

# * Python '3.7' is available when `runtime_version` is set to '1.15' or

4490

# later.

4491

# * Python '3.5' is available when `runtime_version` is set to a version

4492

# from '1.4' to '1.14'.

4493

# * Python '2.7' is available when `runtime_version` is set to '1.15' or

4494

# earlier.

4495

#

4496

# Read more about the Python versions available for [each runtime

4497

# version](/ml-engine/docs/runtime-version-list).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4498

"scheduling": { # All parameters related to scheduling of training jobs. # Optional. Scheduling options for a training job.

4499

"maxWaitTime": "A String",

4500

"maxRunningTime": "A String", # Optional. The maximum job running time, expressed in seconds. The field can

4501

# contain up to nine fractional digits, terminated by `s`. If not specified,

4502

# this field defaults to `604800s` (seven days).

4503

#

4504

# If the training job is still running after this duration, AI Platform

4505

# Training cancels it.

4506

#

4507

# For example, if you want to ensure your job runs for no more than 2 hours,

4508

# set this field to `7200s` (2 hours * 60 minutes / hour * 60 seconds /

4509

# minute).

4510

#

4511

# If you submit your training job using the `gcloud` tool, you can [provide

4512

# this field in a `config.yaml`

4513

# file](/ai-platform/training/docs/training-jobs#formatting_your_configuration_parameters).

# For example:

#

# ```yaml

# trainingInput:

# ...

# scheduling:

# maxRunningTime: 7200s

4521

# ...

4522

# ```

4523

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4524

"network": "A String", # Optional. The full name of the Google Compute Engine

4525

# [network](/compute/docs/networks-and-firewalls#networks) to which the Job

4526

# is peered. For example, projects/12345/global/networks/myVPC. Format is of

4527

# the form projects/{project}/global/networks/{network}. Where {project} is a

4528

# project number, as in '12345', and {network} is network name.".

4529

#

4530

# Private services access must already be configured for the network. If left

4531

# unspecified, the Job is not peered with any network. Learn more -

4532

# Connecting Job to user network over private

4533

# IP.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4534

"evaluatorConfig": { # Represents the configuration for a replica in a cluster. # Optional. The configuration for evaluators.

4535

#

4536

# You should only set `evaluatorConfig.acceleratorConfig` if

4537

# `evaluatorType` is set to a Compute Engine machine type. [Learn

4538

# about restrictions on accelerator configurations for

4539

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

4540

#

4541

# Set `evaluatorConfig.imageUri` only if you build a custom image for

4542

# your evaluator. If `evaluatorConfig.imageUri` has not been

4543

# set, AI Platform uses the value of `masterConfig.imageUri`. Learn more about [configuring custom

4544

# containers](/ai-platform/training/docs/distributed-training-containers).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4545

"imageUri": "A String", # The Docker image to run on the replica. This image must be in Container

4546

# Registry. Learn more about [configuring custom

4547

# containers](/ai-platform/training/docs/distributed-training-containers).

4548

"containerArgs": [ # Arguments to the entrypoint command.

4549

# The following rules apply for container_command and container_args:

4550

# - If you do not supply command or args:

4551

# The defaults defined in the Docker image are used.

4552

# - If you supply a command but no args:

4553

# The default EntryPoint and the default Cmd defined in the Docker image

4554

# are ignored. Your command is run without any arguments.

4555

# - If you supply only args:

4556

# The default Entrypoint defined in the Docker image is run with the args

4557

# that you supplied.

4558

# - If you supply a command and args:

4559

# The default Entrypoint and the default Cmd defined in the Docker image

4560

# are ignored. Your command is run with your args.

4561

# It cannot be set if custom container image is

4562

# not provided.

4563

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

4564

# both cannot be set at the same time.

4565

"A String",

4566

],

4567

"acceleratorConfig": { # Represents a hardware accelerator request config. # Represents the type and number of accelerators used by the replica.

4568

# [Learn about restrictions on accelerator configurations for

4569

# training.](/ai-platform/training/docs/using-gpus#compute-engine-machine-types-with-gpu)

4570

# Note that the AcceleratorConfig can be used in both Jobs and Versions.

4571

# Learn more about [accelerators for training](/ml-engine/docs/using-gpus) and

4572

# [accelerators for online

4573

# prediction](/ml-engine/docs/machine-types-online-prediction#gpus).

4574

"count": "A String", # The number of accelerators to attach to each machine running the job.

4575

"type": "A String", # The type of accelerator to use.

4576

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4577

"tpuTfVersion": "A String", # The AI Platform runtime version that includes a TensorFlow version matching

4578

# the one used in the custom container. This field is required if the replica

4579

# is a TPU worker that uses a custom container. Otherwise, do not specify

4580

# this field. This must be a [runtime version that currently supports

4581

# training with

4582

# TPUs](/ml-engine/docs/tensorflow/runtime-version-list#tpu-support).

4583

#

4584

# Note that the version of TensorFlow included in a runtime version may

4585

# differ from the numbering of the runtime version itself, because it may

4586

# have a different [patch

4587

# version](https://www.tensorflow.org/guide/version_compat#semantic_versioning_20).

4588

# In this field, you must specify the runtime version (TensorFlow minor

4589

# version). For example, if your custom container runs TensorFlow `1.x.y`,

4590

# specify `1.x`.

4591

"containerCommand": [ # The command with which the replica's custom container is run.

4592

# If provided, it will override default ENTRYPOINT of the docker image.

4593

# If not provided, the docker image's ENTRYPOINT is used.

4594

# It cannot be set if custom container image is

4595

# not provided.

4596

# Note that this field and [TrainingInput.args] are mutually exclusive, i.e.,

4597

# both cannot be set at the same time.

4598

"A String",

4599

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4600

},

4601

"useChiefInTfConfig": True or False, # Optional. Use `chief` instead of `master` in the `TF_CONFIG` environment

4602

# variable when training with a custom container. Defaults to `false`. [Learn

4603

# more about this

4604

# field.](/ai-platform/training/docs/distributed-training-details#chief-versus-master)

4605

#

4606

# This field has no effect for training jobs that don't use a custom

4607

# container.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4608

},

4609

"state": "A String", # Output only. The detailed state of a job.

4610

"jobId": "A String", # Required. The user-specified id of the job.

4611

"endTime": "A String", # Output only. When the job processing was completed.

4612

"startTime": "A String", # Output only. When the job processing was started.

4613

"predictionOutput": { # Represents results of a prediction job. # The current prediction job result.

4614

"errorCount": "A String", # The number of data instances which resulted in errors.

4615

"outputPath": "A String", # The output Google Cloud Storage location provided at the job creation time.

4616

"nodeHours": 3.14, # Node hours used by the batch prediction job.

4617

"predictionCount": "A String", # The number of generated predictions.

4618

},

4619

"trainingOutput": { # Represents results of a training job. Output only. # The current training job result.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4620

"isBuiltInAlgorithmJob": True or False, # Whether this job is a built-in Algorithm job.

4621

"builtInAlgorithmOutput": { # Represents output related to a built-in algorithm Job. # Details related to built-in algorithms jobs.

4622

# Only set for built-in algorithms jobs.

4623

"modelPath": "A String", # The Cloud Storage path to the `model/` directory where the training job

4624

# saves the trained model. Only set for successful jobs that don't use

4625

# hyperparameter tuning.

4626

"pythonVersion": "A String", # Python version on which the built-in algorithm was trained.

4627

"runtimeVersion": "A String", # AI Platform runtime version on which the built-in algorithm was

4628

# trained.

4629

"framework": "A String", # Framework on which the built-in algorithm was trained.

4630

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4631

"trials": [ # Results for individual Hyperparameter trials.

4632

# Only set for hyperparameter tuning jobs.

4633

{ # Represents the result of a single hyperparameter tuning trial from a

4634

# training job. The TrainingOutput object that is returned on successful

4635

# completion of a training job with hyperparameter tuning includes a list

4636

# of HyperparameterOutput objects, one for each successful trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4637

"trialId": "A String", # The trial id for these results.

4638

"endTime": "A String", # Output only. End time for the trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4639

"startTime": "A String", # Output only. Start time for the trial.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4640

"isTrialStoppedEarly": True or False, # True if the trial is stopped early.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4641

"finalMetric": { # An observed value of a metric. # The final objective metric seen for this trial.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4642

"trainingStep": "A String", # The global training step for this metric.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4643

"objectiveValue": 3.14, # The objective value at this training step.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4644

},

4645

"builtInAlgorithmOutput": { # Represents output related to a built-in algorithm Job. # Details related to built-in algorithms jobs.

4646

# Only set for trials of built-in algorithms jobs that have succeeded.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4647

"modelPath": "A String", # The Cloud Storage path to the `model/` directory where the training job

4648

# saves the trained model. Only set for successful jobs that don't use

4649

# hyperparameter tuning.

4650

"pythonVersion": "A String", # Python version on which the built-in algorithm was trained.

4651

"runtimeVersion": "A String", # AI Platform runtime version on which the built-in algorithm was

4652

# trained.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4653

"framework": "A String", # Framework on which the built-in algorithm was trained.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4654

},

4655

"state": "A String", # Output only. The detailed state of the trial.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4656

"allMetrics": [ # All recorded object metrics for this trial. This field is not currently

4657

# populated.

4658

{ # An observed value of a metric.

4659

"trainingStep": "A String", # The global training step for this metric.

4660

"objectiveValue": 3.14, # The objective value at this training step.

4661

},

4662

],

4663

"hyperparameters": { # The hyperparameters given to this trial.

4664

"a_key": "A String",

4665

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4666

},

4667

],

4668

"hyperparameterMetricTag": "A String", # The TensorFlow summary tag name used for optimizing hyperparameter tuning

4669

# trials. See

4670

# [`HyperparameterSpec.hyperparameterMetricTag`](#HyperparameterSpec.FIELDS.hyperparameter_metric_tag)

4671

# for more information. Only set for hyperparameter tuning jobs.

4672

"completedTrialCount": "A String", # The number of hyperparameter tuning trials that completed successfully.

4673

# Only set for hyperparameter tuning jobs.

4674

"isHyperparameterTuningJob": True or False, # Whether this job is a hyperparameter tuning job.

4675

"consumedMLUnits": 3.14, # The amount of ML units consumed by the job.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4676

},

4677

"createTime": "A String", # Output only. When the job was created.

4678

"labels": { # Optional. One or more labels that you can add, to organize your jobs.

4679

# Each label is a key-value pair, where both the key and the value are

4680

# arbitrary strings that you supply.

4681

# For more information, see the documentation on

4682

# <a href="/ml-engine/docs/tensorflow/resource-labels">using labels</a>.

4683

"a_key": "A String",

4684

},

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4685

}</pre>

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

</div>

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

4689

<code class="details" id="setIamPolicy">setIamPolicy(resource, body=None, x__xgafv=None)</code>

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4690

<pre>Sets the access control policy on the specified resource. Replaces any

4691

existing policy.

4692

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4693

Can return `NOT_FOUND`, `INVALID_ARGUMENT`, and `PERMISSION_DENIED` errors.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

4694

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4695

Args:

4696

resource: string, REQUIRED: The resource for which the policy is being specified.

4697

See the operation documentation for the appropriate value for this field. (required)

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

4698

body: object, The request body.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4699

The object takes the form of:

4700

4701

{ # Request message for `SetIamPolicy` method.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4702

"updateMask": "A String", # OPTIONAL: A FieldMask specifying which fields of the policy to modify. Only

4703

# the fields in the mask will be modified. If no mask is provided, the

4704

# following default mask is used:

4705

#

4706

# `paths: "bindings, etag"`

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4707

"policy": { # An Identity and Access Management (IAM) policy, which specifies access # REQUIRED: The complete policy to be applied to the `resource`. The size of

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4708

# the policy is limited to a few 10s of KB. An empty policy is a

4709

# valid policy but certain Cloud Platform services (such as Projects)

4710

# might reject them.

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

4711

# controls for Google Cloud resources.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4712

#

4713

#

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

4714

# A `Policy` is a collection of `bindings`. A `binding` binds one or more

4715

# `members` to a single `role`. Members can be user accounts, service accounts,

4716

# Google groups, and domains (such as G Suite). A `role` is a named list of

4717

# permissions; each `role` can be an IAM predefined role or a user-created

4718

# custom role.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4719

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4720

# For some types of Google Cloud resources, a `binding` can also specify a

4721

# `condition`, which is a logical expression that allows access to a resource

4722

# only if the expression evaluates to `true`. A condition can add constraints

4723

# based on attributes of the request, the resource, or both. To learn which

4724

# resources support conditions in their IAM policies, see the

4725

# [IAM documentation](https://cloud.google.com/iam/help/conditions/resource-policies).

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

4726

#

4727

# **JSON example:**

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4728

#

4729

# {

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4730

# "bindings": [

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4731

# {

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4732

# "role": "roles/resourcemanager.organizationAdmin",

4733

# "members": [

4734

# "user:mike@example.com",

4735

# "group:admins@example.com",

4736

# "domain:google.com",

4737

# "serviceAccount:my-project-id@appspot.gserviceaccount.com"

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4738

# ]

4739

# },

4740

# {

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4741

# "role": "roles/resourcemanager.organizationViewer",

4742

# "members": [

4743

# "user:eve@example.com"

4744

# ],

4745

# "condition": {

4746

# "title": "expirable access",

4747

# "description": "Does not grant access after Sep 2020",

4748

# "expression": "request.time < timestamp('2020-10-01T00:00:00.000Z')",

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

4749

# }

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4750

# }

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

4751

# ],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4752

# "etag": "BwWWja0YfJA=",

4753

# "version": 3

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4754

# }

4755

#

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

4756

# **YAML example:**

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

#

# bindings:

# - members:

# - user:mike@example.com

4761

# - group:admins@example.com

4762

# - domain:google.com

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

4763

# - serviceAccount:my-project-id@appspot.gserviceaccount.com

4764

# role: roles/resourcemanager.organizationAdmin

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4765

# - members:

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

4766

# - user:eve@example.com

4767

# role: roles/resourcemanager.organizationViewer

4768

# condition:

4769

# title: expirable access

4770

# description: Does not grant access after Sep 2020

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4771

# expression: request.time < timestamp('2020-10-01T00:00:00.000Z')

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

4772

# - etag: BwWWja0YfJA=

4773

# - version: 3

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4774

#

4775

# For a description of IAM and its features, see the

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

4776

# [IAM documentation](https://cloud.google.com/iam/docs/).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4777

"version": 42, # Specifies the format of the policy.

4778

#

4779

# Valid values are `0`, `1`, and `3`. Requests that specify an invalid value

4780

# are rejected.

4781

#

4782

# Any operation that affects conditional role bindings must specify version

4783

# `3`. This requirement applies to the following operations:

4784

#

4785

# * Getting a policy that includes a conditional role binding

4786

# * Adding a conditional role binding to a policy

4787

# * Changing a conditional role binding in a policy

4788

# * Removing any role binding, with or without a condition, from a policy

4789

# that includes conditions

4790

#

4791

# **Important:** If you use IAM Conditions, you must include the `etag` field

4792

# whenever you call `setIamPolicy`. If you omit this field, then IAM allows

4793

# you to overwrite a version `3` policy with a version `1` policy, and all of

4794

# the conditions in the version `3` policy are lost.

4795

#

4796

# If a policy does not include any conditions, operations on that policy may

4797

# specify any valid version or leave the field unset.

4798

#

4799

# To learn which resources support conditions in their IAM policies, see the

4800

# [IAM documentation](https://cloud.google.com/iam/help/conditions/resource-policies).

4801

"auditConfigs": [ # Specifies cloud audit logging configuration for this policy.

4802

{ # Specifies the audit configuration for a service.

4803

# The configuration determines which permission types are logged, and what

4804

# identities, if any, are exempted from logging.

4805

# An AuditConfig must have one or more AuditLogConfigs.

4806

#

4807

# If there are AuditConfigs for both `allServices` and a specific service,

4808

# the union of the two AuditConfigs is used for that service: the log_types

4809

# specified in each AuditConfig are enabled, and the exempted_members in each

4810

# AuditLogConfig are exempted.

4811

#

4812

# Example Policy with multiple AuditConfigs:

#

# {

# "audit_configs": [

# {

# "service": "allServices"

4818

# "audit_log_configs": [

4819

# {

4820

# "log_type": "DATA_READ",

4821

# "exempted_members": [

4822

# "user:jose@example.com"

# ]

# },

# {

# "log_type": "DATA_WRITE",

4827

# },

4828

# {

4829

# "log_type": "ADMIN_READ",

# }

# ]

# },

# {

# "service": "sampleservice.googleapis.com"

4835

# "audit_log_configs": [

4836

# {

4837

# "log_type": "DATA_READ",

4838

# },

4839

# {

4840

# "log_type": "DATA_WRITE",

4841

# "exempted_members": [

4842

# "user:aliya@example.com"

# ]

# }

# ]

# }

# ]

# }

#

# For sampleservice, this policy enables DATA_READ, DATA_WRITE and ADMIN_READ

4851

# logging. It also exempts jose@example.com from DATA_READ logging, and

4852

# aliya@example.com from DATA_WRITE logging.

4853

"service": "A String", # Specifies a service that will be enabled for audit logging.

4854

# For example, `storage.googleapis.com`, `cloudsql.googleapis.com`.

4855

# `allServices` is a special value that covers all services.

4856

"auditLogConfigs": [ # The configuration for logging of each type of permission.

4857

{ # Provides the configuration for logging a type of permissions.

# Example:

#

# {

# "audit_log_configs": [

4862

# {

4863

# "log_type": "DATA_READ",

4864

# "exempted_members": [

4865

# "user:jose@example.com"

# ]

# },

# {

# "log_type": "DATA_WRITE",

# }

# ]

# }

#

# This enables 'DATA_READ' and 'DATA_WRITE' logging, while exempting

4875

# jose@example.com from DATA_READ logging.

4876

"exemptedMembers": [ # Specifies the identities that do not cause logging for this type of

4877

# permission.

4878

# Follows the same format of Binding.members.

4879

"A String",

4880

],

4881

"logType": "A String", # The log type that this config enables.

},

],

},

],

"bindings": [ # Associates a list of `members` to a `role`. Optionally, may specify a

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

4887

# `condition` that determines how and when the `bindings` are applied. Each

4888

# of the `bindings` must contain at least one member.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4889

{ # Associates `members` with a `role`.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4890

"condition": { # Represents a textual expression in the Common Expression Language (CEL) # The condition that is associated with this binding.

4891

#

4892

# If the condition evaluates to `true`, then this binding applies to the

4893

# current request.

4894

#

4895

# If the condition evaluates to `false`, then this binding does not apply to

4896

# the current request. However, a different role binding might grant the same

4897

# role to one or more of the members in this binding.

4898

#

4899

# To learn which resources support conditions in their IAM policies, see the

4900

# [IAM

4901

# documentation](https://cloud.google.com/iam/help/conditions/resource-policies).

4902

# syntax. CEL is a C-like expression language. The syntax and semantics of CEL

4903

# are documented at https://github.com/google/cel-spec.

4904

#

4905

# Example (Comparison):

4906

#

4907

# title: "Summary size limit"

4908

# description: "Determines if a summary is less than 100 chars"

4909

# expression: "document.summary.size() < 100"

4910

#

4911

# Example (Equality):

4912

#

4913

# title: "Requestor is owner"

4914

# description: "Determines if requestor is the document owner"

4915

# expression: "document.owner == request.auth.claims.email"

#

# Example (Logic):

#

# title: "Public documents"

4920

# description: "Determine whether the document should be publicly visible"

4921

# expression: "document.type != 'private' && document.type != 'internal'"

4922

#

4923

# Example (Data Manipulation):

4924

#

4925

# title: "Notification string"

4926

# description: "Create a notification string with a timestamp."

4927

# expression: "'New message received at ' + string(document.create_time)"

4928

#

4929

# The exact variables and functions that may be referenced within an expression

4930

# are determined by the service that evaluates it. See the service

4931

# documentation for additional information.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4932

"description": "A String", # Optional. Description of the expression. This is a longer text which

4933

# describes the expression, e.g. when hovered over it in a UI.

4934

"expression": "A String", # Textual representation of an expression in Common Expression Language

4935

# syntax.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4936

"title": "A String", # Optional. Title for the expression, i.e. a short string describing

4937

# its purpose. This can be used e.g. in UIs which allow to enter the

4938

# expression.

4939

"location": "A String", # Optional. String indicating the location of the expression for error

4940

# reporting, e.g. a file name and a position in the file.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4941

},

4942

"members": [ # Specifies the identities requesting access for a Cloud Platform resource.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4943

# `members` can have the following values:

4944

#

4945

# * `allUsers`: A special identifier that represents anyone who is

4946

# on the internet; with or without a Google account.

4947

#

4948

# * `allAuthenticatedUsers`: A special identifier that represents anyone

4949

# who is authenticated with a Google account or a service account.

4950

#

4951

# * `user:{emailid}`: An email address that represents a specific Google

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

4952

# account. For example, `alice@example.com` .

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4953

#

4954

#

4955

# * `serviceAccount:{emailid}`: An email address that represents a service

4956

# account. For example, `my-other-app@appspot.gserviceaccount.com`.

4957

#

4958

# * `group:{emailid}`: An email address that represents a Google group.

4959

# For example, `admins@example.com`.

4960

#

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

4961

# * `deleted:user:{emailid}?uid={uniqueid}`: An email address (plus unique

4962

# identifier) representing a user that has been recently deleted. For

4963

# example, `alice@example.com?uid=123456789012345678901`. If the user is

4964

# recovered, this value reverts to `user:{emailid}` and the recovered user

4965

# retains the role in the binding.

4966

#

4967

# * `deleted:serviceAccount:{emailid}?uid={uniqueid}`: An email address (plus

4968

# unique identifier) representing a service account that has been recently

4969

# deleted. For example,

4970

# `my-other-app@appspot.gserviceaccount.com?uid=123456789012345678901`.

4971

# If the service account is undeleted, this value reverts to

4972

# `serviceAccount:{emailid}` and the undeleted service account retains the

4973

# role in the binding.

4974

#

4975

# * `deleted:group:{emailid}?uid={uniqueid}`: An email address (plus unique

4976

# identifier) representing a Google group that has been recently

4977

# deleted. For example, `admins@example.com?uid=123456789012345678901`. If

4978

# the group is recovered, this value reverts to `group:{emailid}` and the

4979

# recovered group retains the role in the binding.

4980

#

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4981

#

4982

# * `domain:{domain}`: The G Suite domain (primary) that represents all the

4983

# users of that domain. For example, `google.com` or `example.com`.

4984

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4985

"A String",

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4986

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

4987

"role": "A String", # Role that is assigned to `members`.

4988

# For example, `roles/viewer`, `roles/editor`, or `roles/owner`.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

4989

},

4990

],

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

4991

"etag": "A String", # `etag` is used for optimistic concurrency control as a way to help

4992

# prevent simultaneous updates of a policy from overwriting each other.

4993

# It is strongly suggested that systems make use of the `etag` in the

4994

# read-modify-write cycle to perform policy updates in order to avoid race

4995

# conditions: An `etag` is returned in the response to `getIamPolicy`, and

4996

# systems are expected to put that etag in the request to `setIamPolicy` to

4997

# ensure that their change will be applied to the same version of the policy.

4998

#

4999

# **Important:** If you use IAM Conditions, you must include the `etag` field

5000

# whenever you call `setIamPolicy`. If you omit this field, then IAM allows

5001

# you to overwrite a version `3` policy with a version `1` policy, and all of

5002

# the conditions in the version `3` policy are lost.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5003

},

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5004

}

5005

5006

x__xgafv: string, V1 error format.

Allowed values

1 - v1 error format

2 - v2 error format

Returns:

An object of the form:

5013

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

5014

{ # An Identity and Access Management (IAM) policy, which specifies access

5015

# controls for Google Cloud resources.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5016

#

5017

#

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

5018

# A `Policy` is a collection of `bindings`. A `binding` binds one or more

5019

# `members` to a single `role`. Members can be user accounts, service accounts,

5020

# Google groups, and domains (such as G Suite). A `role` is a named list of

5021

# permissions; each `role` can be an IAM predefined role or a user-created

5022

# custom role.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5023

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5024

# For some types of Google Cloud resources, a `binding` can also specify a

5025

# `condition`, which is a logical expression that allows access to a resource

5026

# only if the expression evaluates to `true`. A condition can add constraints

5027

# based on attributes of the request, the resource, or both. To learn which

5028

# resources support conditions in their IAM policies, see the

5029

# [IAM documentation](https://cloud.google.com/iam/help/conditions/resource-policies).

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

5030

#

5031

# **JSON example:**

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5032

#

5033

# {

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5034

# "bindings": [

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5035

# {

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5036

# "role": "roles/resourcemanager.organizationAdmin",

5037

# "members": [

5038

# "user:mike@example.com",

5039

# "group:admins@example.com",

5040

# "domain:google.com",

5041

# "serviceAccount:my-project-id@appspot.gserviceaccount.com"

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5042

# ]

5043

# },

5044

# {

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5045

# "role": "roles/resourcemanager.organizationViewer",

5046

# "members": [

5047

# "user:eve@example.com"

5048

# ],

5049

# "condition": {

5050

# "title": "expirable access",

5051

# "description": "Does not grant access after Sep 2020",

5052

# "expression": "request.time < timestamp('2020-10-01T00:00:00.000Z')",

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

5053

# }

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5054

# }

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

5055

# ],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5056

# "etag": "BwWWja0YfJA=",

5057

# "version": 3

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5058

# }

5059

#

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

5060

# **YAML example:**

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

#

# bindings:

# - members:

# - user:mike@example.com

5065

# - group:admins@example.com

5066

# - domain:google.com

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

5067

# - serviceAccount:my-project-id@appspot.gserviceaccount.com

5068

# role: roles/resourcemanager.organizationAdmin

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5069

# - members:

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

5070

# - user:eve@example.com

5071

# role: roles/resourcemanager.organizationViewer

5072

# condition:

5073

# title: expirable access

5074

# description: Does not grant access after Sep 2020

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5075

# expression: request.time < timestamp('2020-10-01T00:00:00.000Z')

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

5076

# - etag: BwWWja0YfJA=

5077

# - version: 3

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5078

#

5079

# For a description of IAM and its features, see the

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

5080

# [IAM documentation](https://cloud.google.com/iam/docs/).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5081

"version": 42, # Specifies the format of the policy.

5082

#

5083

# Valid values are `0`, `1`, and `3`. Requests that specify an invalid value

5084

# are rejected.

5085

#

5086

# Any operation that affects conditional role bindings must specify version

5087

# `3`. This requirement applies to the following operations:

5088

#

5089

# * Getting a policy that includes a conditional role binding

5090

# * Adding a conditional role binding to a policy

5091

# * Changing a conditional role binding in a policy

5092

# * Removing any role binding, with or without a condition, from a policy

5093

# that includes conditions

5094

#

5095

# **Important:** If you use IAM Conditions, you must include the `etag` field

5096

# whenever you call `setIamPolicy`. If you omit this field, then IAM allows

5097

# you to overwrite a version `3` policy with a version `1` policy, and all of

5098

# the conditions in the version `3` policy are lost.

5099

#

5100

# If a policy does not include any conditions, operations on that policy may

5101

# specify any valid version or leave the field unset.

5102

#

5103

# To learn which resources support conditions in their IAM policies, see the

5104

# [IAM documentation](https://cloud.google.com/iam/help/conditions/resource-policies).

5105

"auditConfigs": [ # Specifies cloud audit logging configuration for this policy.

5106

{ # Specifies the audit configuration for a service.

5107

# The configuration determines which permission types are logged, and what

5108

# identities, if any, are exempted from logging.

5109

# An AuditConfig must have one or more AuditLogConfigs.

5110

#

5111

# If there are AuditConfigs for both `allServices` and a specific service,

5112

# the union of the two AuditConfigs is used for that service: the log_types

5113

# specified in each AuditConfig are enabled, and the exempted_members in each

5114

# AuditLogConfig are exempted.

5115

#

5116

# Example Policy with multiple AuditConfigs:

#

# {

# "audit_configs": [

# {

# "service": "allServices"

5122

# "audit_log_configs": [

5123

# {

5124

# "log_type": "DATA_READ",

5125

# "exempted_members": [

5126

# "user:jose@example.com"

# ]

# },

# {

# "log_type": "DATA_WRITE",

5131

# },

5132

# {

5133

# "log_type": "ADMIN_READ",

# }

# ]

# },

# {

# "service": "sampleservice.googleapis.com"

5139

# "audit_log_configs": [

5140

# {

5141

# "log_type": "DATA_READ",

5142

# },

5143

# {

5144

# "log_type": "DATA_WRITE",

5145

# "exempted_members": [

5146

# "user:aliya@example.com"

# ]

# }

# ]

# }

# ]

# }

#

# For sampleservice, this policy enables DATA_READ, DATA_WRITE and ADMIN_READ

5155

# logging. It also exempts jose@example.com from DATA_READ logging, and

5156

# aliya@example.com from DATA_WRITE logging.

5157

"service": "A String", # Specifies a service that will be enabled for audit logging.

5158

# For example, `storage.googleapis.com`, `cloudsql.googleapis.com`.

5159

# `allServices` is a special value that covers all services.

5160

"auditLogConfigs": [ # The configuration for logging of each type of permission.

5161

{ # Provides the configuration for logging a type of permissions.

# Example:

#

# {

# "audit_log_configs": [

5166

# {

5167

# "log_type": "DATA_READ",

5168

# "exempted_members": [

5169

# "user:jose@example.com"

# ]

# },

# {

# "log_type": "DATA_WRITE",

# }

# ]

# }

#

# This enables 'DATA_READ' and 'DATA_WRITE' logging, while exempting

5179

# jose@example.com from DATA_READ logging.

5180

"exemptedMembers": [ # Specifies the identities that do not cause logging for this type of

5181

# permission.

5182

# Follows the same format of Binding.members.

5183

"A String",

5184

],

5185

"logType": "A String", # The log type that this config enables.

},

],

},

],

"bindings": [ # Associates a list of `members` to a `role`. Optionally, may specify a

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

5191

# `condition` that determines how and when the `bindings` are applied. Each

5192

# of the `bindings` must contain at least one member.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5193

{ # Associates `members` with a `role`.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5194

"condition": { # Represents a textual expression in the Common Expression Language (CEL) # The condition that is associated with this binding.

5195

#

5196

# If the condition evaluates to `true`, then this binding applies to the

5197

# current request.

5198

#

5199

# If the condition evaluates to `false`, then this binding does not apply to

5200

# the current request. However, a different role binding might grant the same

5201

# role to one or more of the members in this binding.

5202

#

5203

# To learn which resources support conditions in their IAM policies, see the

5204

# [IAM

5205

# documentation](https://cloud.google.com/iam/help/conditions/resource-policies).

5206

# syntax. CEL is a C-like expression language. The syntax and semantics of CEL

5207

# are documented at https://github.com/google/cel-spec.

5208

#

5209

# Example (Comparison):

5210

#

5211

# title: "Summary size limit"

5212

# description: "Determines if a summary is less than 100 chars"

5213

# expression: "document.summary.size() < 100"

5214

#

5215

# Example (Equality):

5216

#

5217

# title: "Requestor is owner"

5218

# description: "Determines if requestor is the document owner"

5219

# expression: "document.owner == request.auth.claims.email"

#

# Example (Logic):

#

# title: "Public documents"

5224

# description: "Determine whether the document should be publicly visible"

5225

# expression: "document.type != 'private' && document.type != 'internal'"

5226

#

5227

# Example (Data Manipulation):

5228

#

5229

# title: "Notification string"

5230

# description: "Create a notification string with a timestamp."

5231

# expression: "'New message received at ' + string(document.create_time)"

5232

#

5233

# The exact variables and functions that may be referenced within an expression

5234

# are determined by the service that evaluates it. See the service

5235

# documentation for additional information.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

5236

"description": "A String", # Optional. Description of the expression. This is a longer text which

5237

# describes the expression, e.g. when hovered over it in a UI.

5238

"expression": "A String", # Textual representation of an expression in Common Expression Language

5239

# syntax.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5240

"title": "A String", # Optional. Title for the expression, i.e. a short string describing

5241

# its purpose. This can be used e.g. in UIs which allow to enter the

5242

# expression.

5243

"location": "A String", # Optional. String indicating the location of the expression for error

5244

# reporting, e.g. a file name and a position in the file.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5245

},

5246

"members": [ # Specifies the identities requesting access for a Cloud Platform resource.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5247

# `members` can have the following values:

5248

#

5249

# * `allUsers`: A special identifier that represents anyone who is

5250

# on the internet; with or without a Google account.

5251

#

5252

# * `allAuthenticatedUsers`: A special identifier that represents anyone

5253

# who is authenticated with a Google account or a service account.

5254

#

5255

# * `user:{emailid}`: An email address that represents a specific Google

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

5256

# account. For example, `alice@example.com` .

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5257

#

5258

#

5259

# * `serviceAccount:{emailid}`: An email address that represents a service

5260

# account. For example, `my-other-app@appspot.gserviceaccount.com`.

5261

#

5262

# * `group:{emailid}`: An email address that represents a Google group.

5263

# For example, `admins@example.com`.

5264

#

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

5265

# * `deleted:user:{emailid}?uid={uniqueid}`: An email address (plus unique

5266

# identifier) representing a user that has been recently deleted. For

5267

# example, `alice@example.com?uid=123456789012345678901`. If the user is

5268

# recovered, this value reverts to `user:{emailid}` and the recovered user

5269

# retains the role in the binding.

5270

#

5271

# * `deleted:serviceAccount:{emailid}?uid={uniqueid}`: An email address (plus

5272

# unique identifier) representing a service account that has been recently

5273

# deleted. For example,

5274

# `my-other-app@appspot.gserviceaccount.com?uid=123456789012345678901`.

5275

# If the service account is undeleted, this value reverts to

5276

# `serviceAccount:{emailid}` and the undeleted service account retains the

5277

# role in the binding.

5278

#

5279

# * `deleted:group:{emailid}?uid={uniqueid}`: An email address (plus unique

5280

# identifier) representing a Google group that has been recently

5281

# deleted. For example, `admins@example.com?uid=123456789012345678901`. If

5282

# the group is recovered, this value reverts to `group:{emailid}` and the

5283

# recovered group retains the role in the binding.

5284

#

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5285

#

5286

# * `domain:{domain}`: The G Suite domain (primary) that represents all the

5287

# users of that domain. For example, `google.com` or `example.com`.

5288

#

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5289

"A String",

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5290

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5291

"role": "A String", # Role that is assigned to `members`.

5292

# For example, `roles/viewer`, `roles/editor`, or `roles/owner`.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5293

},

5294

],

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

5295

"etag": "A String", # `etag` is used for optimistic concurrency control as a way to help

5296

# prevent simultaneous updates of a policy from overwriting each other.

5297

# It is strongly suggested that systems make use of the `etag` in the

5298

# read-modify-write cycle to perform policy updates in order to avoid race

5299

# conditions: An `etag` is returned in the response to `getIamPolicy`, and

5300

# systems are expected to put that etag in the request to `setIamPolicy` to

5301

# ensure that their change will be applied to the same version of the policy.

5302

#

5303

# **Important:** If you use IAM Conditions, you must include the `etag` field

5304

# whenever you call `setIamPolicy`. If you omit this field, then IAM allows

5305

# you to overwrite a version `3` policy with a version `1` policy, and all of

5306

# the conditions in the version `3` policy are lost.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

}</pre>

</div>

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

5311

<code class="details" id="testIamPermissions">testIamPermissions(resource, body=None, x__xgafv=None)</code>

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5312

<pre>Returns permissions that a caller has on the specified resource.

5313

If the resource does not exist, this will return an empty set of

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5314

permissions, not a `NOT_FOUND` error.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5315

5316

Note: This operation is designed to be used for building permission-aware

5317

UIs and command-line tools, not for authorization checking. This operation

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5318

may "fail open" without warning.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5319

5320

Args:

5321

resource: string, REQUIRED: The resource for which the policy detail is being requested.

5322

See the operation documentation for the appropriate value for this field. (required)

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

5323

body: object, The request body.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5324

The object takes the form of:

5325

5326

{ # Request message for `TestIamPermissions` method.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5327

"permissions": [ # The set of permissions to check for the `resource`. Permissions with

5328

# wildcards (such as '*' or 'storage.*') are not allowed. For more

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5329

# information see

5330

# [IAM Overview](https://cloud.google.com/iam/docs/overview#permissions).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5331

"A String",

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

],

}

x__xgafv: string, V1 error format.

Allowed values

1 - v1 error format

2 - v2 error format

Returns:

An object of the form:

5342

5343

{ # Response message for `TestIamPermissions` method.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5344

"permissions": [ # A subset of `TestPermissionsRequest.permissions` that the caller is

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

5345

# allowed.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

5346

"A String",

Bu Sun Kim