blob: 6fe6f8c85a9384504aea44b0224550c4a6610375 [file] [log] [blame]
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001<html><body>
2<style>
3
4body, h1, h2, h3, div, span, p, pre, a {
5 margin: 0;
6 padding: 0;
7 border: 0;
8 font-weight: inherit;
9 font-style: inherit;
10 font-size: 100%;
11 font-family: inherit;
12 vertical-align: baseline;
13}
14
15body {
16 font-size: 13px;
17 padding: 1em;
18}
19
20h1 {
21 font-size: 26px;
22 margin-bottom: 1em;
23}
24
25h2 {
26 font-size: 24px;
27 margin-bottom: 1em;
28}
29
30h3 {
31 font-size: 20px;
32 margin-bottom: 1em;
33 margin-top: 1em;
34}
35
36pre, code {
37 line-height: 1.5;
38 font-family: Monaco, 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Lucida Console', monospace;
39}
40
41pre {
42 margin-top: 0.5em;
43}
44
45h1, h2, h3, p {
46 font-family: Arial, sans serif;
47}
48
49h1, h2, h3 {
50 border-bottom: solid #CCC 1px;
51}
52
53.toc_element {
54 margin-top: 0.5em;
55}
56
57.firstline {
58 margin-left: 2 em;
59}
60
61.method {
62 margin-top: 1em;
63 border: solid 1px #CCC;
64 padding: 1em;
65 background: #EEE;
66}
67
68.details {
69 font-weight: bold;
70 font-size: 14px;
71}
72
73</style>
74
75<h1><a href="genomics_v1alpha2.html">Genomics API</a> . <a href="genomics_v1alpha2.pipelines.html">pipelines</a></h1>
76<h2>Instance Methods</h2>
77<p class="toc_element">
78 <code><a href="#create">create(body, x__xgafv=None)</a></code></p>
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -040079<p class="firstline">Creates a pipeline that can be run later. Create takes a Pipeline that</p>
Sai Cheemalapatidf613972016-10-21 13:59:49 -070080<p class="toc_element">
Thomas Coffee2f245372017-03-27 10:39:26 -070081 <code><a href="#delete">delete(pipelineId, x__xgafv=None)</a></code></p>
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -040082<p class="firstline">Deletes a pipeline based on ID.</p>
Sai Cheemalapatidf613972016-10-21 13:59:49 -070083<p class="toc_element">
Thomas Coffee2f245372017-03-27 10:39:26 -070084 <code><a href="#get">get(pipelineId, x__xgafv=None)</a></code></p>
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -040085<p class="firstline">Retrieves a pipeline based on ID.</p>
Sai Cheemalapatidf613972016-10-21 13:59:49 -070086<p class="toc_element">
87 <code><a href="#getControllerConfig">getControllerConfig(validationToken=None, x__xgafv=None, operationId=None)</a></code></p>
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -040088<p class="firstline">Gets controller configuration information. Should only be called</p>
Sai Cheemalapatidf613972016-10-21 13:59:49 -070089<p class="toc_element">
90 <code><a href="#list">list(pageSize=None, projectId=None, namePrefix=None, pageToken=None, x__xgafv=None)</a></code></p>
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -040091<p class="firstline">Lists pipelines.</p>
Sai Cheemalapatidf613972016-10-21 13:59:49 -070092<p class="toc_element">
93 <code><a href="#list_next">list_next(previous_request, previous_response)</a></code></p>
94<p class="firstline">Retrieves the next page of results.</p>
95<p class="toc_element">
96 <code><a href="#run">run(body, x__xgafv=None)</a></code></p>
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -040097<p class="firstline">Runs a pipeline. If `pipelineId` is specified in the request, then</p>
Sai Cheemalapatidf613972016-10-21 13:59:49 -070098<p class="toc_element">
99 <code><a href="#setOperationStatus">setOperationStatus(body, x__xgafv=None)</a></code></p>
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400100<p class="firstline">Sets status of a given operation. Any new timestamps (as determined by</p>
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700101<h3>Method Details</h3>
102<div class="method">
103 <code class="details" id="create">create(body, x__xgafv=None)</code>
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400104 <pre>Creates a pipeline that can be run later. Create takes a Pipeline that
105has all fields other than `pipelineId` populated, and then returns
106the same pipeline with `pipelineId` populated. This id can be used
107to run the pipeline.
108
109Caller must have WRITE permission to the project.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700110
111Args:
112 body: object, The request body. (required)
113 The object takes the form of:
114
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400115{ # The pipeline object. Represents a transformation from a set of input
116 # parameters to a set of output parameters. The transformation is defined
117 # as a docker image and command to run within that image. Each pipeline
118 # is run on a Google Compute Engine VM. A pipeline can be created with the
119 # `create` method and then later run with the `run` method, or a pipeline can
120 # be defined and run all at once with the `run` method.
121 "pipelineId": "A String", # Unique pipeline id that is generated by the service when CreatePipeline
122 # is called. Cannot be specified in the Pipeline used in the
123 # CreatePipelineRequest, and will be populated in the response to
124 # CreatePipeline and all subsequent Get and List calls. Indicates that the
125 # service has registered this pipeline.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700126 "description": "A String", # User-specified description.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400127 "projectId": "A String", # Required. The project in which to create the pipeline. The caller must have
128 # WRITE access.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700129 "outputParameters": [ # Output parameters of the pipeline.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400130 { # Parameters facilitate setting and delivering data into the
131 # pipeline's execution environment. They are defined at create time,
132 # with optional defaults, and can be overridden at run time.
133 #
134 # If `localCopy` is unset, then the parameter specifies a string that
135 # is passed as-is into the pipeline, as the value of the environment
136 # variable with the given name. A default value can be optionally
137 # specified at create time. The default can be overridden at run time
138 # using the inputs map. If no default is given, a value must be
139 # supplied at runtime.
140 #
141 # If `localCopy` is defined, then the parameter specifies a data
142 # source or sink, both in Google Cloud Storage and on the Docker container
143 # where the pipeline computation is run. The service account associated with
144 # the Pipeline (by
145 # default the project's Compute Engine service account) must have access to the
146 # Google Cloud Storage paths.
147 #
148 # At run time, the Google Cloud Storage paths can be overridden if a default
149 # was provided at create time, or must be set otherwise. The pipeline runner
150 # should add a key/value pair to either the inputs or outputs map. The
151 # indicated data copies will be carried out before/after pipeline execution,
152 # just as if the corresponding arguments were provided to `gsutil cp`.
153 #
154 # For example: Given the following `PipelineParameter`, specified
155 # in the `inputParameters` list:
156 #
157 # ```
158 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}}
159 # ```
160 #
161 # where `disk` is defined in the `PipelineResources` object as:
162 #
163 # ```
164 # {name: "pd1", mountPoint: "/mnt/disk/"}
165 # ```
166 #
167 # We create a disk named `pd1`, mount it on the host VM, and map
168 # `/mnt/pd1` to `/mnt/disk` in the docker container. At
169 # runtime, an entry for `input_file` would be required in the inputs
170 # map, such as:
171 #
172 # ```
173 # inputs["input_file"] = "gs://my-bucket/bar.txt"
174 # ```
175 #
176 # This would generate the following gsutil call:
177 #
178 # ```
179 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt
180 # ```
181 #
182 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the
183 # Docker container. Acceptable paths are:
184 #
185 # <table>
186 # <thead>
187 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr>
188 # </thead>
189 # <tbody>
190 # <tr><td>file</td><td>file</td></tr>
191 # <tr><td>glob</td><td>directory</td></tr>
192 # </tbody>
193 # </table>
194 #
195 # For outputs, the direction of the copy is reversed:
196 #
197 # ```
198 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt
199 # ```
200 #
201 # Acceptable paths are:
202 #
203 # <table>
204 # <thead>
205 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr>
206 # </thead>
207 # <tbody>
208 # <tr><td>file</td><td>file</td></tr>
209 # <tr>
210 # <td>file</td>
211 # <td>directory - directory must already exist</td>
212 # </tr>
213 # <tr>
214 # <td>glob</td>
215 # <td>directory - directory will be created if it doesn't exist</td></tr>
216 # </tbody>
217 # </table>
218 #
219 # One restriction due to docker limitations, is that for outputs that are found
220 # on the boot disk, the local path cannot be a glob and must be a file.
221 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime.
222 # If `localCopy` is present, then this must be a Google Cloud Storage path
223 # beginning with `gs://`.
224 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM.
225 # `LocalCopy` indicates where on the VM the file should be. The value
226 # given to this parameter (either at runtime or using `defaultValue`)
227 # must be the remote path where the file should be.
228 "path": "A String", # Required. The path within the user's docker container where
229 # this input should be localized to and from, relative to the specified
230 # disk's mount point. For example: file.txt,
231 "disk": "A String", # Required. The name of the disk where this parameter is
232 # located. Can be the name of one of the disks specified in the
233 # Resources field, or "boot", which represents the Docker
234 # instance's boot disk and has a mount point of `/`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700235 },
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700236 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string
237 # as the key to the input and output maps in RunPipeline.
238 "description": "A String", # Human-readable description.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700239 },
240 ],
241 "inputParameters": [ # Input parameters of the pipeline.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400242 { # Parameters facilitate setting and delivering data into the
243 # pipeline's execution environment. They are defined at create time,
244 # with optional defaults, and can be overridden at run time.
245 #
246 # If `localCopy` is unset, then the parameter specifies a string that
247 # is passed as-is into the pipeline, as the value of the environment
248 # variable with the given name. A default value can be optionally
249 # specified at create time. The default can be overridden at run time
250 # using the inputs map. If no default is given, a value must be
251 # supplied at runtime.
252 #
253 # If `localCopy` is defined, then the parameter specifies a data
254 # source or sink, both in Google Cloud Storage and on the Docker container
255 # where the pipeline computation is run. The service account associated with
256 # the Pipeline (by
257 # default the project's Compute Engine service account) must have access to the
258 # Google Cloud Storage paths.
259 #
260 # At run time, the Google Cloud Storage paths can be overridden if a default
261 # was provided at create time, or must be set otherwise. The pipeline runner
262 # should add a key/value pair to either the inputs or outputs map. The
263 # indicated data copies will be carried out before/after pipeline execution,
264 # just as if the corresponding arguments were provided to `gsutil cp`.
265 #
266 # For example: Given the following `PipelineParameter`, specified
267 # in the `inputParameters` list:
268 #
269 # ```
270 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}}
271 # ```
272 #
273 # where `disk` is defined in the `PipelineResources` object as:
274 #
275 # ```
276 # {name: "pd1", mountPoint: "/mnt/disk/"}
277 # ```
278 #
279 # We create a disk named `pd1`, mount it on the host VM, and map
280 # `/mnt/pd1` to `/mnt/disk` in the docker container. At
281 # runtime, an entry for `input_file` would be required in the inputs
282 # map, such as:
283 #
284 # ```
285 # inputs["input_file"] = "gs://my-bucket/bar.txt"
286 # ```
287 #
288 # This would generate the following gsutil call:
289 #
290 # ```
291 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt
292 # ```
293 #
294 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the
295 # Docker container. Acceptable paths are:
296 #
297 # <table>
298 # <thead>
299 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr>
300 # </thead>
301 # <tbody>
302 # <tr><td>file</td><td>file</td></tr>
303 # <tr><td>glob</td><td>directory</td></tr>
304 # </tbody>
305 # </table>
306 #
307 # For outputs, the direction of the copy is reversed:
308 #
309 # ```
310 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt
311 # ```
312 #
313 # Acceptable paths are:
314 #
315 # <table>
316 # <thead>
317 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr>
318 # </thead>
319 # <tbody>
320 # <tr><td>file</td><td>file</td></tr>
321 # <tr>
322 # <td>file</td>
323 # <td>directory - directory must already exist</td>
324 # </tr>
325 # <tr>
326 # <td>glob</td>
327 # <td>directory - directory will be created if it doesn't exist</td></tr>
328 # </tbody>
329 # </table>
330 #
331 # One restriction due to docker limitations, is that for outputs that are found
332 # on the boot disk, the local path cannot be a glob and must be a file.
333 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime.
334 # If `localCopy` is present, then this must be a Google Cloud Storage path
335 # beginning with `gs://`.
336 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM.
337 # `LocalCopy` indicates where on the VM the file should be. The value
338 # given to this parameter (either at runtime or using `defaultValue`)
339 # must be the remote path where the file should be.
340 "path": "A String", # Required. The path within the user's docker container where
341 # this input should be localized to and from, relative to the specified
342 # disk's mount point. For example: file.txt,
343 "disk": "A String", # Required. The name of the disk where this parameter is
344 # located. Can be the name of one of the disks specified in the
345 # Resources field, or "boot", which represents the Docker
346 # instance's boot disk and has a mount point of `/`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700347 },
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700348 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string
349 # as the key to the input and output maps in RunPipeline.
350 "description": "A String", # Human-readable description.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700351 },
352 ],
353 "docker": { # The Docker execuctor specification. # Specifies the docker run information.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400354 "cmd": "A String", # Required. The command or newline delimited script to run. The command
355 # string will be executed within a bash shell.
356 #
357 # If the command exits with a non-zero exit code, output parameter
358 # de-localization will be skipped and the pipeline operation's
359 # `error` field will be populated.
360 #
361 # Maximum command string length is 16384.
362 "imageName": "A String", # Required. Image name from either Docker Hub or Google Container Registry.
363 # Users that run pipelines must have READ access to the image.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700364 },
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400365 "resources": { # The system resources for the pipeline run. # Required. Specifies resource requirements for the pipeline run.
366 # Required fields:
367 #
368 # *
369 # minimumCpuCores
370 #
371 # *
372 # minimumRamGb
373 "noAddress": True or False, # Whether to assign an external IP to the instance. This is an experimental
374 # feature that may go away. Defaults to false.
375 # Corresponds to `--no_address` flag for [gcloud compute instances create]
376 # (https://cloud.google.com/sdk/gcloud/reference/compute/instances/create).
377 # In order to use this, must be true for both create time and run time.
378 # Cannot be true at run time if false at create time. If you need to ssh into
379 # a private IP VM for debugging, you can ssh to a public VM and then ssh into
380 # the private VM's Internal IP. If noAddress is set, this pipeline run may
381 # only load docker images from Google Container Registry and not Docker Hub.
382 # Before using this, you must
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700383 # [configure access to Google services from internal
384 # IPs](https://cloud.google.com/compute/docs/configure-private-google-access#configuring_access_to_google_services_from_internal_ips).
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700385 "minimumRamGb": 3.14, # The minimum amount of RAM to use. Defaults to 3.75 (GB)
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400386 "preemptible": True or False, # Whether to use preemptible VMs. Defaults to `false`. In order to use this,
387 # must be true for both create time and run time. Cannot be true at run time
388 # if false at create time.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700389 "disks": [ # Disks to attach.
390 { # A Google Compute Engine disk resource specification.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700391 "name": "A String", # Required. The name of the disk that can be used in the pipeline
392 # parameters. Must be 1 - 63 characters.
393 # The name "boot" is reserved for system use.
Sai Cheemalapati4ba8c232017-06-06 18:46:08 -0400394 "sizeGb": 42, # The size of the disk. Defaults to 500 (GB).
395 # This field is not applicable for local SSD.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700396 "autoDelete": True or False, # Deprecated. Disks created by the Pipelines API will be deleted at the end
397 # of the pipeline run, regardless of what this field is set to.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400398 "source": "A String", # The full or partial URL of the persistent disk to attach. See
399 # https://cloud.google.com/compute/docs/reference/latest/instances#resource
400 # and
401 # https://cloud.google.com/compute/docs/disks/persistent-disks#snapshots
402 # for more details.
Sai Cheemalapati4ba8c232017-06-06 18:46:08 -0400403 "readOnly": True or False, # Specifies how a sourced-base persistent disk will be mounted. See
404 # https://cloud.google.com/compute/docs/disks/persistent-disks#use_multi_instances
405 # for more details.
406 # Can only be set at create time.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400407 "mountPoint": "A String", # Required at create time and cannot be overridden at run time.
408 # Specifies the path in the docker container where files on
409 # this disk should be located. For example, if `mountPoint`
410 # is `/mnt/disk`, and the parameter has `localPath`
411 # `inputs/file.txt`, the docker container can access the data at
412 # `/mnt/disk/inputs/file.txt`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700413 "type": "A String", # Required. The type of the disk to create.
414 },
415 ],
416 "bootDiskSizeGb": 42, # The size of the boot disk. Defaults to 10 (GB).
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400417 "zones": [ # List of Google Compute Engine availability zones to which resource
418 # creation will restricted. If empty, any zone may be chosen.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700419 "A String",
420 ],
421 "minimumCpuCores": 42, # The minimum number of cores to use. Defaults to 1.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700422 "acceleratorType": "A String", # Optional. The Compute Engine defined accelerator type.
423 # By specifying this parameter, you will download and install the following
424 # third-party software onto your managed Compute Engine instances: NVIDIA®
425 # Tesla® drivers and NVIDIA® CUDA toolkit.
426 # Please see https://cloud.google.com/compute/docs/gpus/ for a list of
427 # available accelerator types.
428 "acceleratorCount": "A String", # Optional. The number of accelerators of the specified type to attach.
429 # By specifying this parameter, you will download and install the following
430 # third-party software onto your managed Compute Engine instances:
431 # NVIDIA® Tesla® drivers and NVIDIA® CUDA toolkit.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700432 },
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700433 "name": "A String", # Required. A user specified pipeline name that does not have to be unique.
434 # This name can be used for filtering Pipelines in ListPipelines.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700435}
436
437 x__xgafv: string, V1 error format.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400438 Allowed values
439 1 - v1 error format
440 2 - v2 error format
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700441
442Returns:
443 An object of the form:
444
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400445 { # The pipeline object. Represents a transformation from a set of input
446 # parameters to a set of output parameters. The transformation is defined
447 # as a docker image and command to run within that image. Each pipeline
448 # is run on a Google Compute Engine VM. A pipeline can be created with the
449 # `create` method and then later run with the `run` method, or a pipeline can
450 # be defined and run all at once with the `run` method.
451 "pipelineId": "A String", # Unique pipeline id that is generated by the service when CreatePipeline
452 # is called. Cannot be specified in the Pipeline used in the
453 # CreatePipelineRequest, and will be populated in the response to
454 # CreatePipeline and all subsequent Get and List calls. Indicates that the
455 # service has registered this pipeline.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700456 "description": "A String", # User-specified description.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400457 "projectId": "A String", # Required. The project in which to create the pipeline. The caller must have
458 # WRITE access.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700459 "outputParameters": [ # Output parameters of the pipeline.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400460 { # Parameters facilitate setting and delivering data into the
461 # pipeline's execution environment. They are defined at create time,
462 # with optional defaults, and can be overridden at run time.
463 #
464 # If `localCopy` is unset, then the parameter specifies a string that
465 # is passed as-is into the pipeline, as the value of the environment
466 # variable with the given name. A default value can be optionally
467 # specified at create time. The default can be overridden at run time
468 # using the inputs map. If no default is given, a value must be
469 # supplied at runtime.
470 #
471 # If `localCopy` is defined, then the parameter specifies a data
472 # source or sink, both in Google Cloud Storage and on the Docker container
473 # where the pipeline computation is run. The service account associated with
474 # the Pipeline (by
475 # default the project's Compute Engine service account) must have access to the
476 # Google Cloud Storage paths.
477 #
478 # At run time, the Google Cloud Storage paths can be overridden if a default
479 # was provided at create time, or must be set otherwise. The pipeline runner
480 # should add a key/value pair to either the inputs or outputs map. The
481 # indicated data copies will be carried out before/after pipeline execution,
482 # just as if the corresponding arguments were provided to `gsutil cp`.
483 #
484 # For example: Given the following `PipelineParameter`, specified
485 # in the `inputParameters` list:
486 #
487 # ```
488 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}}
489 # ```
490 #
491 # where `disk` is defined in the `PipelineResources` object as:
492 #
493 # ```
494 # {name: "pd1", mountPoint: "/mnt/disk/"}
495 # ```
496 #
497 # We create a disk named `pd1`, mount it on the host VM, and map
498 # `/mnt/pd1` to `/mnt/disk` in the docker container. At
499 # runtime, an entry for `input_file` would be required in the inputs
500 # map, such as:
501 #
502 # ```
503 # inputs["input_file"] = "gs://my-bucket/bar.txt"
504 # ```
505 #
506 # This would generate the following gsutil call:
507 #
508 # ```
509 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt
510 # ```
511 #
512 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the
513 # Docker container. Acceptable paths are:
514 #
515 # <table>
516 # <thead>
517 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr>
518 # </thead>
519 # <tbody>
520 # <tr><td>file</td><td>file</td></tr>
521 # <tr><td>glob</td><td>directory</td></tr>
522 # </tbody>
523 # </table>
524 #
525 # For outputs, the direction of the copy is reversed:
526 #
527 # ```
528 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt
529 # ```
530 #
531 # Acceptable paths are:
532 #
533 # <table>
534 # <thead>
535 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr>
536 # </thead>
537 # <tbody>
538 # <tr><td>file</td><td>file</td></tr>
539 # <tr>
540 # <td>file</td>
541 # <td>directory - directory must already exist</td>
542 # </tr>
543 # <tr>
544 # <td>glob</td>
545 # <td>directory - directory will be created if it doesn't exist</td></tr>
546 # </tbody>
547 # </table>
548 #
549 # One restriction due to docker limitations, is that for outputs that are found
550 # on the boot disk, the local path cannot be a glob and must be a file.
551 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime.
552 # If `localCopy` is present, then this must be a Google Cloud Storage path
553 # beginning with `gs://`.
554 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM.
555 # `LocalCopy` indicates where on the VM the file should be. The value
556 # given to this parameter (either at runtime or using `defaultValue`)
557 # must be the remote path where the file should be.
558 "path": "A String", # Required. The path within the user's docker container where
559 # this input should be localized to and from, relative to the specified
560 # disk's mount point. For example: file.txt,
561 "disk": "A String", # Required. The name of the disk where this parameter is
562 # located. Can be the name of one of the disks specified in the
563 # Resources field, or "boot", which represents the Docker
564 # instance's boot disk and has a mount point of `/`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700565 },
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700566 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string
567 # as the key to the input and output maps in RunPipeline.
568 "description": "A String", # Human-readable description.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700569 },
570 ],
571 "inputParameters": [ # Input parameters of the pipeline.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400572 { # Parameters facilitate setting and delivering data into the
573 # pipeline's execution environment. They are defined at create time,
574 # with optional defaults, and can be overridden at run time.
575 #
576 # If `localCopy` is unset, then the parameter specifies a string that
577 # is passed as-is into the pipeline, as the value of the environment
578 # variable with the given name. A default value can be optionally
579 # specified at create time. The default can be overridden at run time
580 # using the inputs map. If no default is given, a value must be
581 # supplied at runtime.
582 #
583 # If `localCopy` is defined, then the parameter specifies a data
584 # source or sink, both in Google Cloud Storage and on the Docker container
585 # where the pipeline computation is run. The service account associated with
586 # the Pipeline (by
587 # default the project's Compute Engine service account) must have access to the
588 # Google Cloud Storage paths.
589 #
590 # At run time, the Google Cloud Storage paths can be overridden if a default
591 # was provided at create time, or must be set otherwise. The pipeline runner
592 # should add a key/value pair to either the inputs or outputs map. The
593 # indicated data copies will be carried out before/after pipeline execution,
594 # just as if the corresponding arguments were provided to `gsutil cp`.
595 #
596 # For example: Given the following `PipelineParameter`, specified
597 # in the `inputParameters` list:
598 #
599 # ```
600 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}}
601 # ```
602 #
603 # where `disk` is defined in the `PipelineResources` object as:
604 #
605 # ```
606 # {name: "pd1", mountPoint: "/mnt/disk/"}
607 # ```
608 #
609 # We create a disk named `pd1`, mount it on the host VM, and map
610 # `/mnt/pd1` to `/mnt/disk` in the docker container. At
611 # runtime, an entry for `input_file` would be required in the inputs
612 # map, such as:
613 #
614 # ```
615 # inputs["input_file"] = "gs://my-bucket/bar.txt"
616 # ```
617 #
618 # This would generate the following gsutil call:
619 #
620 # ```
621 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt
622 # ```
623 #
624 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the
625 # Docker container. Acceptable paths are:
626 #
627 # <table>
628 # <thead>
629 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr>
630 # </thead>
631 # <tbody>
632 # <tr><td>file</td><td>file</td></tr>
633 # <tr><td>glob</td><td>directory</td></tr>
634 # </tbody>
635 # </table>
636 #
637 # For outputs, the direction of the copy is reversed:
638 #
639 # ```
640 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt
641 # ```
642 #
643 # Acceptable paths are:
644 #
645 # <table>
646 # <thead>
647 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr>
648 # </thead>
649 # <tbody>
650 # <tr><td>file</td><td>file</td></tr>
651 # <tr>
652 # <td>file</td>
653 # <td>directory - directory must already exist</td>
654 # </tr>
655 # <tr>
656 # <td>glob</td>
657 # <td>directory - directory will be created if it doesn't exist</td></tr>
658 # </tbody>
659 # </table>
660 #
661 # One restriction due to docker limitations, is that for outputs that are found
662 # on the boot disk, the local path cannot be a glob and must be a file.
663 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime.
664 # If `localCopy` is present, then this must be a Google Cloud Storage path
665 # beginning with `gs://`.
666 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM.
667 # `LocalCopy` indicates where on the VM the file should be. The value
668 # given to this parameter (either at runtime or using `defaultValue`)
669 # must be the remote path where the file should be.
670 "path": "A String", # Required. The path within the user's docker container where
671 # this input should be localized to and from, relative to the specified
672 # disk's mount point. For example: file.txt,
673 "disk": "A String", # Required. The name of the disk where this parameter is
674 # located. Can be the name of one of the disks specified in the
675 # Resources field, or "boot", which represents the Docker
676 # instance's boot disk and has a mount point of `/`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700677 },
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700678 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string
679 # as the key to the input and output maps in RunPipeline.
680 "description": "A String", # Human-readable description.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700681 },
682 ],
683 "docker": { # The Docker execuctor specification. # Specifies the docker run information.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400684 "cmd": "A String", # Required. The command or newline delimited script to run. The command
685 # string will be executed within a bash shell.
686 #
687 # If the command exits with a non-zero exit code, output parameter
688 # de-localization will be skipped and the pipeline operation's
689 # `error` field will be populated.
690 #
691 # Maximum command string length is 16384.
692 "imageName": "A String", # Required. Image name from either Docker Hub or Google Container Registry.
693 # Users that run pipelines must have READ access to the image.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700694 },
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400695 "resources": { # The system resources for the pipeline run. # Required. Specifies resource requirements for the pipeline run.
696 # Required fields:
697 #
698 # *
699 # minimumCpuCores
700 #
701 # *
702 # minimumRamGb
703 "noAddress": True or False, # Whether to assign an external IP to the instance. This is an experimental
704 # feature that may go away. Defaults to false.
705 # Corresponds to `--no_address` flag for [gcloud compute instances create]
706 # (https://cloud.google.com/sdk/gcloud/reference/compute/instances/create).
707 # In order to use this, must be true for both create time and run time.
708 # Cannot be true at run time if false at create time. If you need to ssh into
709 # a private IP VM for debugging, you can ssh to a public VM and then ssh into
710 # the private VM's Internal IP. If noAddress is set, this pipeline run may
711 # only load docker images from Google Container Registry and not Docker Hub.
712 # Before using this, you must
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700713 # [configure access to Google services from internal
714 # IPs](https://cloud.google.com/compute/docs/configure-private-google-access#configuring_access_to_google_services_from_internal_ips).
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700715 "minimumRamGb": 3.14, # The minimum amount of RAM to use. Defaults to 3.75 (GB)
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400716 "preemptible": True or False, # Whether to use preemptible VMs. Defaults to `false`. In order to use this,
717 # must be true for both create time and run time. Cannot be true at run time
718 # if false at create time.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700719 "disks": [ # Disks to attach.
720 { # A Google Compute Engine disk resource specification.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700721 "name": "A String", # Required. The name of the disk that can be used in the pipeline
722 # parameters. Must be 1 - 63 characters.
723 # The name "boot" is reserved for system use.
Sai Cheemalapati4ba8c232017-06-06 18:46:08 -0400724 "sizeGb": 42, # The size of the disk. Defaults to 500 (GB).
725 # This field is not applicable for local SSD.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700726 "autoDelete": True or False, # Deprecated. Disks created by the Pipelines API will be deleted at the end
727 # of the pipeline run, regardless of what this field is set to.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400728 "source": "A String", # The full or partial URL of the persistent disk to attach. See
729 # https://cloud.google.com/compute/docs/reference/latest/instances#resource
730 # and
731 # https://cloud.google.com/compute/docs/disks/persistent-disks#snapshots
732 # for more details.
Sai Cheemalapati4ba8c232017-06-06 18:46:08 -0400733 "readOnly": True or False, # Specifies how a sourced-base persistent disk will be mounted. See
734 # https://cloud.google.com/compute/docs/disks/persistent-disks#use_multi_instances
735 # for more details.
736 # Can only be set at create time.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400737 "mountPoint": "A String", # Required at create time and cannot be overridden at run time.
738 # Specifies the path in the docker container where files on
739 # this disk should be located. For example, if `mountPoint`
740 # is `/mnt/disk`, and the parameter has `localPath`
741 # `inputs/file.txt`, the docker container can access the data at
742 # `/mnt/disk/inputs/file.txt`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700743 "type": "A String", # Required. The type of the disk to create.
744 },
745 ],
746 "bootDiskSizeGb": 42, # The size of the boot disk. Defaults to 10 (GB).
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400747 "zones": [ # List of Google Compute Engine availability zones to which resource
748 # creation will restricted. If empty, any zone may be chosen.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700749 "A String",
750 ],
751 "minimumCpuCores": 42, # The minimum number of cores to use. Defaults to 1.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700752 "acceleratorType": "A String", # Optional. The Compute Engine defined accelerator type.
753 # By specifying this parameter, you will download and install the following
754 # third-party software onto your managed Compute Engine instances: NVIDIA®
755 # Tesla® drivers and NVIDIA® CUDA toolkit.
756 # Please see https://cloud.google.com/compute/docs/gpus/ for a list of
757 # available accelerator types.
758 "acceleratorCount": "A String", # Optional. The number of accelerators of the specified type to attach.
759 # By specifying this parameter, you will download and install the following
760 # third-party software onto your managed Compute Engine instances:
761 # NVIDIA® Tesla® drivers and NVIDIA® CUDA toolkit.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700762 },
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700763 "name": "A String", # Required. A user specified pipeline name that does not have to be unique.
764 # This name can be used for filtering Pipelines in ListPipelines.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700765 }</pre>
766</div>
767
768<div class="method">
Thomas Coffee2f245372017-03-27 10:39:26 -0700769 <code class="details" id="delete">delete(pipelineId, x__xgafv=None)</code>
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400770 <pre>Deletes a pipeline based on ID.
771
772Caller must have WRITE permission to the project.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700773
774Args:
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400775 pipelineId: string, Caller must have WRITE access to the project in which this pipeline
776is defined. (required)
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700777 x__xgafv: string, V1 error format.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400778 Allowed values
779 1 - v1 error format
780 2 - v2 error format
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700781
782Returns:
783 An object of the form:
784
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400785 { # A generic empty message that you can re-use to avoid defining duplicated
786 # empty messages in your APIs. A typical example is to use it as the request
787 # or the response type of an API method. For instance:
788 #
789 # service Foo {
790 # rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty);
791 # }
792 #
793 # The JSON representation for `Empty` is empty JSON object `{}`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700794 }</pre>
795</div>
796
797<div class="method">
Thomas Coffee2f245372017-03-27 10:39:26 -0700798 <code class="details" id="get">get(pipelineId, x__xgafv=None)</code>
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400799 <pre>Retrieves a pipeline based on ID.
800
801Caller must have READ permission to the project.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700802
803Args:
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400804 pipelineId: string, Caller must have READ access to the project in which this pipeline
805is defined. (required)
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700806 x__xgafv: string, V1 error format.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400807 Allowed values
808 1 - v1 error format
809 2 - v2 error format
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700810
811Returns:
812 An object of the form:
813
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400814 { # The pipeline object. Represents a transformation from a set of input
815 # parameters to a set of output parameters. The transformation is defined
816 # as a docker image and command to run within that image. Each pipeline
817 # is run on a Google Compute Engine VM. A pipeline can be created with the
818 # `create` method and then later run with the `run` method, or a pipeline can
819 # be defined and run all at once with the `run` method.
820 "pipelineId": "A String", # Unique pipeline id that is generated by the service when CreatePipeline
821 # is called. Cannot be specified in the Pipeline used in the
822 # CreatePipelineRequest, and will be populated in the response to
823 # CreatePipeline and all subsequent Get and List calls. Indicates that the
824 # service has registered this pipeline.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700825 "description": "A String", # User-specified description.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400826 "projectId": "A String", # Required. The project in which to create the pipeline. The caller must have
827 # WRITE access.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700828 "outputParameters": [ # Output parameters of the pipeline.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400829 { # Parameters facilitate setting and delivering data into the
830 # pipeline's execution environment. They are defined at create time,
831 # with optional defaults, and can be overridden at run time.
832 #
833 # If `localCopy` is unset, then the parameter specifies a string that
834 # is passed as-is into the pipeline, as the value of the environment
835 # variable with the given name. A default value can be optionally
836 # specified at create time. The default can be overridden at run time
837 # using the inputs map. If no default is given, a value must be
838 # supplied at runtime.
839 #
840 # If `localCopy` is defined, then the parameter specifies a data
841 # source or sink, both in Google Cloud Storage and on the Docker container
842 # where the pipeline computation is run. The service account associated with
843 # the Pipeline (by
844 # default the project's Compute Engine service account) must have access to the
845 # Google Cloud Storage paths.
846 #
847 # At run time, the Google Cloud Storage paths can be overridden if a default
848 # was provided at create time, or must be set otherwise. The pipeline runner
849 # should add a key/value pair to either the inputs or outputs map. The
850 # indicated data copies will be carried out before/after pipeline execution,
851 # just as if the corresponding arguments were provided to `gsutil cp`.
852 #
853 # For example: Given the following `PipelineParameter`, specified
854 # in the `inputParameters` list:
855 #
856 # ```
857 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}}
858 # ```
859 #
860 # where `disk` is defined in the `PipelineResources` object as:
861 #
862 # ```
863 # {name: "pd1", mountPoint: "/mnt/disk/"}
864 # ```
865 #
866 # We create a disk named `pd1`, mount it on the host VM, and map
867 # `/mnt/pd1` to `/mnt/disk` in the docker container. At
868 # runtime, an entry for `input_file` would be required in the inputs
869 # map, such as:
870 #
871 # ```
872 # inputs["input_file"] = "gs://my-bucket/bar.txt"
873 # ```
874 #
875 # This would generate the following gsutil call:
876 #
877 # ```
878 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt
879 # ```
880 #
881 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the
882 # Docker container. Acceptable paths are:
883 #
884 # <table>
885 # <thead>
886 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr>
887 # </thead>
888 # <tbody>
889 # <tr><td>file</td><td>file</td></tr>
890 # <tr><td>glob</td><td>directory</td></tr>
891 # </tbody>
892 # </table>
893 #
894 # For outputs, the direction of the copy is reversed:
895 #
896 # ```
897 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt
898 # ```
899 #
900 # Acceptable paths are:
901 #
902 # <table>
903 # <thead>
904 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr>
905 # </thead>
906 # <tbody>
907 # <tr><td>file</td><td>file</td></tr>
908 # <tr>
909 # <td>file</td>
910 # <td>directory - directory must already exist</td>
911 # </tr>
912 # <tr>
913 # <td>glob</td>
914 # <td>directory - directory will be created if it doesn't exist</td></tr>
915 # </tbody>
916 # </table>
917 #
918 # One restriction due to docker limitations, is that for outputs that are found
919 # on the boot disk, the local path cannot be a glob and must be a file.
920 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime.
921 # If `localCopy` is present, then this must be a Google Cloud Storage path
922 # beginning with `gs://`.
923 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM.
924 # `LocalCopy` indicates where on the VM the file should be. The value
925 # given to this parameter (either at runtime or using `defaultValue`)
926 # must be the remote path where the file should be.
927 "path": "A String", # Required. The path within the user's docker container where
928 # this input should be localized to and from, relative to the specified
929 # disk's mount point. For example: file.txt,
930 "disk": "A String", # Required. The name of the disk where this parameter is
931 # located. Can be the name of one of the disks specified in the
932 # Resources field, or "boot", which represents the Docker
933 # instance's boot disk and has a mount point of `/`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700934 },
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700935 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string
936 # as the key to the input and output maps in RunPipeline.
937 "description": "A String", # Human-readable description.
Sai Cheemalapatidf613972016-10-21 13:59:49 -0700938 },
939 ],
940 "inputParameters": [ # Input parameters of the pipeline.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400941 { # Parameters facilitate setting and delivering data into the
942 # pipeline's execution environment. They are defined at create time,
943 # with optional defaults, and can be overridden at run time.
944 #
945 # If `localCopy` is unset, then the parameter specifies a string that
946 # is passed as-is into the pipeline, as the value of the environment
947 # variable with the given name. A default value can be optionally
948 # specified at create time. The default can be overridden at run time
949 # using the inputs map. If no default is given, a value must be
950 # supplied at runtime.
951 #
952 # If `localCopy` is defined, then the parameter specifies a data
953 # source or sink, both in Google Cloud Storage and on the Docker container
954 # where the pipeline computation is run. The service account associated with
955 # the Pipeline (by
956 # default the project's Compute Engine service account) must have access to the
957 # Google Cloud Storage paths.
958 #
959 # At run time, the Google Cloud Storage paths can be overridden if a default
960 # was provided at create time, or must be set otherwise. The pipeline runner
961 # should add a key/value pair to either the inputs or outputs map. The
962 # indicated data copies will be carried out before/after pipeline execution,
963 # just as if the corresponding arguments were provided to `gsutil cp`.
964 #
965 # For example: Given the following `PipelineParameter`, specified
966 # in the `inputParameters` list:
967 #
968 # ```
969 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}}
970 # ```
971 #
972 # where `disk` is defined in the `PipelineResources` object as:
973 #
974 # ```
975 # {name: "pd1", mountPoint: "/mnt/disk/"}
976 # ```
977 #
978 # We create a disk named `pd1`, mount it on the host VM, and map
979 # `/mnt/pd1` to `/mnt/disk` in the docker container. At
980 # runtime, an entry for `input_file` would be required in the inputs
981 # map, such as:
982 #
983 # ```
984 # inputs["input_file"] = "gs://my-bucket/bar.txt"
985 # ```
986 #
987 # This would generate the following gsutil call:
988 #
989 # ```
990 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt
991 # ```
992 #
993 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the
994 # Docker container. Acceptable paths are:
995 #
996 # <table>
997 # <thead>
998 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr>
999 # </thead>
1000 # <tbody>
1001 # <tr><td>file</td><td>file</td></tr>
1002 # <tr><td>glob</td><td>directory</td></tr>
1003 # </tbody>
1004 # </table>
1005 #
1006 # For outputs, the direction of the copy is reversed:
1007 #
1008 # ```
1009 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt
1010 # ```
1011 #
1012 # Acceptable paths are:
1013 #
1014 # <table>
1015 # <thead>
1016 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr>
1017 # </thead>
1018 # <tbody>
1019 # <tr><td>file</td><td>file</td></tr>
1020 # <tr>
1021 # <td>file</td>
1022 # <td>directory - directory must already exist</td>
1023 # </tr>
1024 # <tr>
1025 # <td>glob</td>
1026 # <td>directory - directory will be created if it doesn't exist</td></tr>
1027 # </tbody>
1028 # </table>
1029 #
1030 # One restriction due to docker limitations, is that for outputs that are found
1031 # on the boot disk, the local path cannot be a glob and must be a file.
1032 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime.
1033 # If `localCopy` is present, then this must be a Google Cloud Storage path
1034 # beginning with `gs://`.
1035 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM.
1036 # `LocalCopy` indicates where on the VM the file should be. The value
1037 # given to this parameter (either at runtime or using `defaultValue`)
1038 # must be the remote path where the file should be.
1039 "path": "A String", # Required. The path within the user's docker container where
1040 # this input should be localized to and from, relative to the specified
1041 # disk's mount point. For example: file.txt,
1042 "disk": "A String", # Required. The name of the disk where this parameter is
1043 # located. Can be the name of one of the disks specified in the
1044 # Resources field, or "boot", which represents the Docker
1045 # instance's boot disk and has a mount point of `/`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001046 },
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001047 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string
1048 # as the key to the input and output maps in RunPipeline.
1049 "description": "A String", # Human-readable description.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001050 },
1051 ],
1052 "docker": { # The Docker execuctor specification. # Specifies the docker run information.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001053 "cmd": "A String", # Required. The command or newline delimited script to run. The command
1054 # string will be executed within a bash shell.
1055 #
1056 # If the command exits with a non-zero exit code, output parameter
1057 # de-localization will be skipped and the pipeline operation's
1058 # `error` field will be populated.
1059 #
1060 # Maximum command string length is 16384.
1061 "imageName": "A String", # Required. Image name from either Docker Hub or Google Container Registry.
1062 # Users that run pipelines must have READ access to the image.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001063 },
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001064 "resources": { # The system resources for the pipeline run. # Required. Specifies resource requirements for the pipeline run.
1065 # Required fields:
1066 #
1067 # *
1068 # minimumCpuCores
1069 #
1070 # *
1071 # minimumRamGb
1072 "noAddress": True or False, # Whether to assign an external IP to the instance. This is an experimental
1073 # feature that may go away. Defaults to false.
1074 # Corresponds to `--no_address` flag for [gcloud compute instances create]
1075 # (https://cloud.google.com/sdk/gcloud/reference/compute/instances/create).
1076 # In order to use this, must be true for both create time and run time.
1077 # Cannot be true at run time if false at create time. If you need to ssh into
1078 # a private IP VM for debugging, you can ssh to a public VM and then ssh into
1079 # the private VM's Internal IP. If noAddress is set, this pipeline run may
1080 # only load docker images from Google Container Registry and not Docker Hub.
1081 # Before using this, you must
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001082 # [configure access to Google services from internal
1083 # IPs](https://cloud.google.com/compute/docs/configure-private-google-access#configuring_access_to_google_services_from_internal_ips).
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001084 "minimumRamGb": 3.14, # The minimum amount of RAM to use. Defaults to 3.75 (GB)
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001085 "preemptible": True or False, # Whether to use preemptible VMs. Defaults to `false`. In order to use this,
1086 # must be true for both create time and run time. Cannot be true at run time
1087 # if false at create time.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001088 "disks": [ # Disks to attach.
1089 { # A Google Compute Engine disk resource specification.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001090 "name": "A String", # Required. The name of the disk that can be used in the pipeline
1091 # parameters. Must be 1 - 63 characters.
1092 # The name "boot" is reserved for system use.
Sai Cheemalapati4ba8c232017-06-06 18:46:08 -04001093 "sizeGb": 42, # The size of the disk. Defaults to 500 (GB).
1094 # This field is not applicable for local SSD.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001095 "autoDelete": True or False, # Deprecated. Disks created by the Pipelines API will be deleted at the end
1096 # of the pipeline run, regardless of what this field is set to.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001097 "source": "A String", # The full or partial URL of the persistent disk to attach. See
1098 # https://cloud.google.com/compute/docs/reference/latest/instances#resource
1099 # and
1100 # https://cloud.google.com/compute/docs/disks/persistent-disks#snapshots
1101 # for more details.
Sai Cheemalapati4ba8c232017-06-06 18:46:08 -04001102 "readOnly": True or False, # Specifies how a sourced-base persistent disk will be mounted. See
1103 # https://cloud.google.com/compute/docs/disks/persistent-disks#use_multi_instances
1104 # for more details.
1105 # Can only be set at create time.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001106 "mountPoint": "A String", # Required at create time and cannot be overridden at run time.
1107 # Specifies the path in the docker container where files on
1108 # this disk should be located. For example, if `mountPoint`
1109 # is `/mnt/disk`, and the parameter has `localPath`
1110 # `inputs/file.txt`, the docker container can access the data at
1111 # `/mnt/disk/inputs/file.txt`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001112 "type": "A String", # Required. The type of the disk to create.
1113 },
1114 ],
1115 "bootDiskSizeGb": 42, # The size of the boot disk. Defaults to 10 (GB).
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001116 "zones": [ # List of Google Compute Engine availability zones to which resource
1117 # creation will restricted. If empty, any zone may be chosen.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001118 "A String",
1119 ],
1120 "minimumCpuCores": 42, # The minimum number of cores to use. Defaults to 1.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001121 "acceleratorType": "A String", # Optional. The Compute Engine defined accelerator type.
1122 # By specifying this parameter, you will download and install the following
1123 # third-party software onto your managed Compute Engine instances: NVIDIA®
1124 # Tesla® drivers and NVIDIA® CUDA toolkit.
1125 # Please see https://cloud.google.com/compute/docs/gpus/ for a list of
1126 # available accelerator types.
1127 "acceleratorCount": "A String", # Optional. The number of accelerators of the specified type to attach.
1128 # By specifying this parameter, you will download and install the following
1129 # third-party software onto your managed Compute Engine instances:
1130 # NVIDIA® Tesla® drivers and NVIDIA® CUDA toolkit.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001131 },
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001132 "name": "A String", # Required. A user specified pipeline name that does not have to be unique.
1133 # This name can be used for filtering Pipelines in ListPipelines.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001134 }</pre>
1135</div>
1136
1137<div class="method">
1138 <code class="details" id="getControllerConfig">getControllerConfig(validationToken=None, x__xgafv=None, operationId=None)</code>
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001139 <pre>Gets controller configuration information. Should only be called
1140by VMs created by the Pipelines Service and not by end users.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001141
1142Args:
1143 validationToken: string, A parameter
1144 x__xgafv: string, V1 error format.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001145 Allowed values
1146 1 - v1 error format
1147 2 - v2 error format
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001148 operationId: string, The operation to retrieve controller configuration for.
1149
1150Returns:
1151 An object of the form:
1152
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001153 { # Stores the information that the controller will fetch from the
1154 # server in order to run. Should only be used by VMs created by the
1155 # Pipelines Service and not by end users.
Thomas Coffee2f245372017-03-27 10:39:26 -07001156 "cmd": "A String",
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001157 "machineType": "A String",
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001158 "vars": {
1159 "a_key": "A String",
1160 },
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001161 "gcsLogPath": "A String",
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001162 "gcsSinks": {
1163 "a_key": {
1164 "values": [
1165 "A String",
1166 ],
1167 },
1168 },
1169 "image": "A String",
Thomas Coffee2f245372017-03-27 10:39:26 -07001170 "disks": {
1171 "a_key": "A String",
1172 },
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001173 "gcsSources": {
1174 "a_key": {
1175 "values": [
1176 "A String",
1177 ],
1178 },
1179 },
1180 }</pre>
1181</div>
1182
1183<div class="method">
1184 <code class="details" id="list">list(pageSize=None, projectId=None, namePrefix=None, pageToken=None, x__xgafv=None)</code>
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001185 <pre>Lists pipelines.
1186
1187Caller must have READ permission to the project.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001188
1189Args:
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001190 pageSize: integer, Number of pipelines to return at once. Defaults to 256, and max
1191is 2048.
1192 projectId: string, Required. The name of the project to search for pipelines. Caller
1193must have READ access to this project.
1194 namePrefix: string, Pipelines with names that match this prefix should be
1195returned. If unspecified, all pipelines in the project, up to
1196`pageSize`, will be returned.
1197 pageToken: string, Token to use to indicate where to start getting results.
1198If unspecified, returns the first page of results.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001199 x__xgafv: string, V1 error format.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001200 Allowed values
1201 1 - v1 error format
1202 2 - v2 error format
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001203
1204Returns:
1205 An object of the form:
1206
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001207 { # The response of ListPipelines. Contains at most `pageSize`
1208 # pipelines. If it contains `pageSize` pipelines, and more pipelines
1209 # exist, then `nextPageToken` will be populated and should be
1210 # used as the `pageToken` argument to a subsequent ListPipelines
1211 # request.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001212 "nextPageToken": "A String", # The token to use to get the next page of results.
1213 "pipelines": [ # The matched pipelines.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001214 { # The pipeline object. Represents a transformation from a set of input
1215 # parameters to a set of output parameters. The transformation is defined
1216 # as a docker image and command to run within that image. Each pipeline
1217 # is run on a Google Compute Engine VM. A pipeline can be created with the
1218 # `create` method and then later run with the `run` method, or a pipeline can
1219 # be defined and run all at once with the `run` method.
1220 "pipelineId": "A String", # Unique pipeline id that is generated by the service when CreatePipeline
1221 # is called. Cannot be specified in the Pipeline used in the
1222 # CreatePipelineRequest, and will be populated in the response to
1223 # CreatePipeline and all subsequent Get and List calls. Indicates that the
1224 # service has registered this pipeline.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001225 "description": "A String", # User-specified description.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001226 "projectId": "A String", # Required. The project in which to create the pipeline. The caller must have
1227 # WRITE access.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001228 "outputParameters": [ # Output parameters of the pipeline.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001229 { # Parameters facilitate setting and delivering data into the
1230 # pipeline's execution environment. They are defined at create time,
1231 # with optional defaults, and can be overridden at run time.
1232 #
1233 # If `localCopy` is unset, then the parameter specifies a string that
1234 # is passed as-is into the pipeline, as the value of the environment
1235 # variable with the given name. A default value can be optionally
1236 # specified at create time. The default can be overridden at run time
1237 # using the inputs map. If no default is given, a value must be
1238 # supplied at runtime.
1239 #
1240 # If `localCopy` is defined, then the parameter specifies a data
1241 # source or sink, both in Google Cloud Storage and on the Docker container
1242 # where the pipeline computation is run. The service account associated with
1243 # the Pipeline (by
1244 # default the project's Compute Engine service account) must have access to the
1245 # Google Cloud Storage paths.
1246 #
1247 # At run time, the Google Cloud Storage paths can be overridden if a default
1248 # was provided at create time, or must be set otherwise. The pipeline runner
1249 # should add a key/value pair to either the inputs or outputs map. The
1250 # indicated data copies will be carried out before/after pipeline execution,
1251 # just as if the corresponding arguments were provided to `gsutil cp`.
1252 #
1253 # For example: Given the following `PipelineParameter`, specified
1254 # in the `inputParameters` list:
1255 #
1256 # ```
1257 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}}
1258 # ```
1259 #
1260 # where `disk` is defined in the `PipelineResources` object as:
1261 #
1262 # ```
1263 # {name: "pd1", mountPoint: "/mnt/disk/"}
1264 # ```
1265 #
1266 # We create a disk named `pd1`, mount it on the host VM, and map
1267 # `/mnt/pd1` to `/mnt/disk` in the docker container. At
1268 # runtime, an entry for `input_file` would be required in the inputs
1269 # map, such as:
1270 #
1271 # ```
1272 # inputs["input_file"] = "gs://my-bucket/bar.txt"
1273 # ```
1274 #
1275 # This would generate the following gsutil call:
1276 #
1277 # ```
1278 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt
1279 # ```
1280 #
1281 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the
1282 # Docker container. Acceptable paths are:
1283 #
1284 # <table>
1285 # <thead>
1286 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr>
1287 # </thead>
1288 # <tbody>
1289 # <tr><td>file</td><td>file</td></tr>
1290 # <tr><td>glob</td><td>directory</td></tr>
1291 # </tbody>
1292 # </table>
1293 #
1294 # For outputs, the direction of the copy is reversed:
1295 #
1296 # ```
1297 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt
1298 # ```
1299 #
1300 # Acceptable paths are:
1301 #
1302 # <table>
1303 # <thead>
1304 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr>
1305 # </thead>
1306 # <tbody>
1307 # <tr><td>file</td><td>file</td></tr>
1308 # <tr>
1309 # <td>file</td>
1310 # <td>directory - directory must already exist</td>
1311 # </tr>
1312 # <tr>
1313 # <td>glob</td>
1314 # <td>directory - directory will be created if it doesn't exist</td></tr>
1315 # </tbody>
1316 # </table>
1317 #
1318 # One restriction due to docker limitations, is that for outputs that are found
1319 # on the boot disk, the local path cannot be a glob and must be a file.
1320 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime.
1321 # If `localCopy` is present, then this must be a Google Cloud Storage path
1322 # beginning with `gs://`.
1323 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM.
1324 # `LocalCopy` indicates where on the VM the file should be. The value
1325 # given to this parameter (either at runtime or using `defaultValue`)
1326 # must be the remote path where the file should be.
1327 "path": "A String", # Required. The path within the user's docker container where
1328 # this input should be localized to and from, relative to the specified
1329 # disk's mount point. For example: file.txt,
1330 "disk": "A String", # Required. The name of the disk where this parameter is
1331 # located. Can be the name of one of the disks specified in the
1332 # Resources field, or "boot", which represents the Docker
1333 # instance's boot disk and has a mount point of `/`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001334 },
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001335 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string
1336 # as the key to the input and output maps in RunPipeline.
1337 "description": "A String", # Human-readable description.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001338 },
1339 ],
1340 "inputParameters": [ # Input parameters of the pipeline.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001341 { # Parameters facilitate setting and delivering data into the
1342 # pipeline's execution environment. They are defined at create time,
1343 # with optional defaults, and can be overridden at run time.
1344 #
1345 # If `localCopy` is unset, then the parameter specifies a string that
1346 # is passed as-is into the pipeline, as the value of the environment
1347 # variable with the given name. A default value can be optionally
1348 # specified at create time. The default can be overridden at run time
1349 # using the inputs map. If no default is given, a value must be
1350 # supplied at runtime.
1351 #
1352 # If `localCopy` is defined, then the parameter specifies a data
1353 # source or sink, both in Google Cloud Storage and on the Docker container
1354 # where the pipeline computation is run. The service account associated with
1355 # the Pipeline (by
1356 # default the project's Compute Engine service account) must have access to the
1357 # Google Cloud Storage paths.
1358 #
1359 # At run time, the Google Cloud Storage paths can be overridden if a default
1360 # was provided at create time, or must be set otherwise. The pipeline runner
1361 # should add a key/value pair to either the inputs or outputs map. The
1362 # indicated data copies will be carried out before/after pipeline execution,
1363 # just as if the corresponding arguments were provided to `gsutil cp`.
1364 #
1365 # For example: Given the following `PipelineParameter`, specified
1366 # in the `inputParameters` list:
1367 #
1368 # ```
1369 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}}
1370 # ```
1371 #
1372 # where `disk` is defined in the `PipelineResources` object as:
1373 #
1374 # ```
1375 # {name: "pd1", mountPoint: "/mnt/disk/"}
1376 # ```
1377 #
1378 # We create a disk named `pd1`, mount it on the host VM, and map
1379 # `/mnt/pd1` to `/mnt/disk` in the docker container. At
1380 # runtime, an entry for `input_file` would be required in the inputs
1381 # map, such as:
1382 #
1383 # ```
1384 # inputs["input_file"] = "gs://my-bucket/bar.txt"
1385 # ```
1386 #
1387 # This would generate the following gsutil call:
1388 #
1389 # ```
1390 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt
1391 # ```
1392 #
1393 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the
1394 # Docker container. Acceptable paths are:
1395 #
1396 # <table>
1397 # <thead>
1398 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr>
1399 # </thead>
1400 # <tbody>
1401 # <tr><td>file</td><td>file</td></tr>
1402 # <tr><td>glob</td><td>directory</td></tr>
1403 # </tbody>
1404 # </table>
1405 #
1406 # For outputs, the direction of the copy is reversed:
1407 #
1408 # ```
1409 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt
1410 # ```
1411 #
1412 # Acceptable paths are:
1413 #
1414 # <table>
1415 # <thead>
1416 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr>
1417 # </thead>
1418 # <tbody>
1419 # <tr><td>file</td><td>file</td></tr>
1420 # <tr>
1421 # <td>file</td>
1422 # <td>directory - directory must already exist</td>
1423 # </tr>
1424 # <tr>
1425 # <td>glob</td>
1426 # <td>directory - directory will be created if it doesn't exist</td></tr>
1427 # </tbody>
1428 # </table>
1429 #
1430 # One restriction due to docker limitations, is that for outputs that are found
1431 # on the boot disk, the local path cannot be a glob and must be a file.
1432 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime.
1433 # If `localCopy` is present, then this must be a Google Cloud Storage path
1434 # beginning with `gs://`.
1435 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM.
1436 # `LocalCopy` indicates where on the VM the file should be. The value
1437 # given to this parameter (either at runtime or using `defaultValue`)
1438 # must be the remote path where the file should be.
1439 "path": "A String", # Required. The path within the user's docker container where
1440 # this input should be localized to and from, relative to the specified
1441 # disk's mount point. For example: file.txt,
1442 "disk": "A String", # Required. The name of the disk where this parameter is
1443 # located. Can be the name of one of the disks specified in the
1444 # Resources field, or "boot", which represents the Docker
1445 # instance's boot disk and has a mount point of `/`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001446 },
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001447 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string
1448 # as the key to the input and output maps in RunPipeline.
1449 "description": "A String", # Human-readable description.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001450 },
1451 ],
1452 "docker": { # The Docker execuctor specification. # Specifies the docker run information.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001453 "cmd": "A String", # Required. The command or newline delimited script to run. The command
1454 # string will be executed within a bash shell.
1455 #
1456 # If the command exits with a non-zero exit code, output parameter
1457 # de-localization will be skipped and the pipeline operation's
1458 # `error` field will be populated.
1459 #
1460 # Maximum command string length is 16384.
1461 "imageName": "A String", # Required. Image name from either Docker Hub or Google Container Registry.
1462 # Users that run pipelines must have READ access to the image.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001463 },
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001464 "resources": { # The system resources for the pipeline run. # Required. Specifies resource requirements for the pipeline run.
1465 # Required fields:
1466 #
1467 # *
1468 # minimumCpuCores
1469 #
1470 # *
1471 # minimumRamGb
1472 "noAddress": True or False, # Whether to assign an external IP to the instance. This is an experimental
1473 # feature that may go away. Defaults to false.
1474 # Corresponds to `--no_address` flag for [gcloud compute instances create]
1475 # (https://cloud.google.com/sdk/gcloud/reference/compute/instances/create).
1476 # In order to use this, must be true for both create time and run time.
1477 # Cannot be true at run time if false at create time. If you need to ssh into
1478 # a private IP VM for debugging, you can ssh to a public VM and then ssh into
1479 # the private VM's Internal IP. If noAddress is set, this pipeline run may
1480 # only load docker images from Google Container Registry and not Docker Hub.
1481 # Before using this, you must
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001482 # [configure access to Google services from internal
1483 # IPs](https://cloud.google.com/compute/docs/configure-private-google-access#configuring_access_to_google_services_from_internal_ips).
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001484 "minimumRamGb": 3.14, # The minimum amount of RAM to use. Defaults to 3.75 (GB)
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001485 "preemptible": True or False, # Whether to use preemptible VMs. Defaults to `false`. In order to use this,
1486 # must be true for both create time and run time. Cannot be true at run time
1487 # if false at create time.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001488 "disks": [ # Disks to attach.
1489 { # A Google Compute Engine disk resource specification.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001490 "name": "A String", # Required. The name of the disk that can be used in the pipeline
1491 # parameters. Must be 1 - 63 characters.
1492 # The name "boot" is reserved for system use.
Sai Cheemalapati4ba8c232017-06-06 18:46:08 -04001493 "sizeGb": 42, # The size of the disk. Defaults to 500 (GB).
1494 # This field is not applicable for local SSD.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001495 "autoDelete": True or False, # Deprecated. Disks created by the Pipelines API will be deleted at the end
1496 # of the pipeline run, regardless of what this field is set to.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001497 "source": "A String", # The full or partial URL of the persistent disk to attach. See
1498 # https://cloud.google.com/compute/docs/reference/latest/instances#resource
1499 # and
1500 # https://cloud.google.com/compute/docs/disks/persistent-disks#snapshots
1501 # for more details.
Sai Cheemalapati4ba8c232017-06-06 18:46:08 -04001502 "readOnly": True or False, # Specifies how a sourced-base persistent disk will be mounted. See
1503 # https://cloud.google.com/compute/docs/disks/persistent-disks#use_multi_instances
1504 # for more details.
1505 # Can only be set at create time.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001506 "mountPoint": "A String", # Required at create time and cannot be overridden at run time.
1507 # Specifies the path in the docker container where files on
1508 # this disk should be located. For example, if `mountPoint`
1509 # is `/mnt/disk`, and the parameter has `localPath`
1510 # `inputs/file.txt`, the docker container can access the data at
1511 # `/mnt/disk/inputs/file.txt`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001512 "type": "A String", # Required. The type of the disk to create.
1513 },
1514 ],
1515 "bootDiskSizeGb": 42, # The size of the boot disk. Defaults to 10 (GB).
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001516 "zones": [ # List of Google Compute Engine availability zones to which resource
1517 # creation will restricted. If empty, any zone may be chosen.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001518 "A String",
1519 ],
1520 "minimumCpuCores": 42, # The minimum number of cores to use. Defaults to 1.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001521 "acceleratorType": "A String", # Optional. The Compute Engine defined accelerator type.
1522 # By specifying this parameter, you will download and install the following
1523 # third-party software onto your managed Compute Engine instances: NVIDIA®
1524 # Tesla® drivers and NVIDIA® CUDA toolkit.
1525 # Please see https://cloud.google.com/compute/docs/gpus/ for a list of
1526 # available accelerator types.
1527 "acceleratorCount": "A String", # Optional. The number of accelerators of the specified type to attach.
1528 # By specifying this parameter, you will download and install the following
1529 # third-party software onto your managed Compute Engine instances:
1530 # NVIDIA® Tesla® drivers and NVIDIA® CUDA toolkit.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001531 },
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001532 "name": "A String", # Required. A user specified pipeline name that does not have to be unique.
1533 # This name can be used for filtering Pipelines in ListPipelines.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001534 },
1535 ],
1536 }</pre>
1537</div>
1538
1539<div class="method">
1540 <code class="details" id="list_next">list_next(previous_request, previous_response)</code>
1541 <pre>Retrieves the next page of results.
1542
1543Args:
1544 previous_request: The request for the previous page. (required)
1545 previous_response: The response from the request for the previous page. (required)
1546
1547Returns:
1548 A request object that you can call 'execute()' on to request the next
1549 page. Returns None if there are no more items in the collection.
1550 </pre>
1551</div>
1552
1553<div class="method">
1554 <code class="details" id="run">run(body, x__xgafv=None)</code>
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001555 <pre>Runs a pipeline. If `pipelineId` is specified in the request, then
1556run a saved pipeline. If `ephemeralPipeline` is specified, then run
1557that pipeline once without saving a copy.
1558
1559The caller must have READ permission to the project where the pipeline
1560is stored and WRITE permission to the project where the pipeline will be
1561run, as VMs will be created and storage will be used.
1562
1563If a pipeline operation is still running after 6 days, it will be canceled.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001564
1565Args:
1566 body: object, The request body. (required)
1567 The object takes the form of:
1568
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001569{ # The request to run a pipeline. If `pipelineId` is specified, it
1570 # refers to a saved pipeline created with CreatePipeline and set as
1571 # the `pipelineId` of the returned Pipeline object. If
1572 # `ephemeralPipeline` is specified, that pipeline is run once
1573 # with the given args and not saved. It is an error to specify both
1574 # `pipelineId` and `ephemeralPipeline`. `pipelineArgs`
1575 # must be specified.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001576 "pipelineArgs": { # The pipeline run arguments. # The arguments to use when running this pipeline.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001577 "inputs": { # Pipeline input arguments; keys are defined in the pipeline documentation.
1578 # All input parameters that do not have default values must be specified.
1579 # If parameters with defaults are specified here, the defaults will be
1580 # overridden.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001581 "a_key": "A String",
1582 },
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001583 "logging": { # The logging options for the pipeline run. # Required. Logging options. Used by the service to communicate results
1584 # to the user.
1585 "gcsPath": "A String", # The location in Google Cloud Storage to which the pipeline logs
1586 # will be copied. Can be specified as a fully qualified directory
1587 # path, in which case logs will be output with a unique identifier
1588 # as the filename in that directory, or as a fully specified path,
1589 # which must end in `.log`, in which case that path will be
1590 # used, and the user must ensure that logs are not
1591 # overwritten. Stdout and stderr logs from the run are also
1592 # generated and output as `-stdout.log` and `-stderr.log`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001593 },
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001594 "outputs": { # Pipeline output arguments; keys are defined in the pipeline
1595 # documentation. All output parameters of without default values
1596 # must be specified. If parameters with defaults are specified
1597 # here, the defaults will be overridden.
Jon Wayne Parrott692617a2017-01-06 09:58:29 -08001598 "a_key": "A String",
1599 },
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001600 "labels": { # Labels to apply to this pipeline run. Labels will also be applied to
1601 # compute resources (VM, disks) created by this pipeline run. When listing
1602 # operations, operations can filtered by labels.
1603 # Label keys may not be empty; label values may be empty. Non-empty labels
1604 # must be 1-63 characters long, and comply with [RFC1035]
1605 # (https://www.ietf.org/rfc/rfc1035.txt).
1606 # Specifically, the name must be 1-63 characters long and match the regular
1607 # expression `[a-z]([-a-z0-9]*[a-z0-9])?` which means the first
1608 # character must be a lowercase letter, and all following characters must be
1609 # a dash, lowercase letter, or digit, except the last character, which cannot
1610 # be a dash.
1611 "a_key": "A String",
1612 },
1613 "serviceAccount": { # A Google Cloud Service Account. # The Google Cloud Service Account that will be used to access data and
1614 # services. By default, the compute service account associated with
1615 # `projectId` is used.
1616 "scopes": [ # List of scopes to be enabled for this service account on the VM.
1617 # The following scopes are automatically included:
1618 #
1619 # * https://www.googleapis.com/auth/compute
1620 # * https://www.googleapis.com/auth/devstorage.full_control
1621 # * https://www.googleapis.com/auth/genomics
1622 # * https://www.googleapis.com/auth/logging.write
1623 # * https://www.googleapis.com/auth/monitoring.write
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001624 "A String",
1625 ],
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001626 "email": "A String", # Email address of the service account. Defaults to `default`,
1627 # which uses the compute service account associated with the project.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001628 },
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001629 "clientId": "A String", # This field is deprecated. Use `labels` instead. Client-specified pipeline
1630 # operation identifier.
1631 "projectId": "A String", # Required. The project in which to run the pipeline. The caller must have
1632 # WRITER access to all Google Cloud services and resources (e.g. Google
1633 # Compute Engine) will be used.
1634 "keepVmAliveOnFailureDuration": "A String", # How long to keep the VM up after a failure (for example docker command
1635 # failed, copying input or output files failed, etc). While the VM is up, one
1636 # can ssh into the VM to debug. Default is 0; maximum allowed value is 1 day.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001637 "resources": { # The system resources for the pipeline run. # Specifies resource requirements/overrides for the pipeline run.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001638 "noAddress": True or False, # Whether to assign an external IP to the instance. This is an experimental
1639 # feature that may go away. Defaults to false.
1640 # Corresponds to `--no_address` flag for [gcloud compute instances create]
1641 # (https://cloud.google.com/sdk/gcloud/reference/compute/instances/create).
1642 # In order to use this, must be true for both create time and run time.
1643 # Cannot be true at run time if false at create time. If you need to ssh into
1644 # a private IP VM for debugging, you can ssh to a public VM and then ssh into
1645 # the private VM's Internal IP. If noAddress is set, this pipeline run may
1646 # only load docker images from Google Container Registry and not Docker Hub.
1647 # Before using this, you must
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001648 # [configure access to Google services from internal
1649 # IPs](https://cloud.google.com/compute/docs/configure-private-google-access#configuring_access_to_google_services_from_internal_ips).
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001650 "minimumRamGb": 3.14, # The minimum amount of RAM to use. Defaults to 3.75 (GB)
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001651 "preemptible": True or False, # Whether to use preemptible VMs. Defaults to `false`. In order to use this,
1652 # must be true for both create time and run time. Cannot be true at run time
1653 # if false at create time.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001654 "disks": [ # Disks to attach.
1655 { # A Google Compute Engine disk resource specification.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001656 "name": "A String", # Required. The name of the disk that can be used in the pipeline
1657 # parameters. Must be 1 - 63 characters.
1658 # The name "boot" is reserved for system use.
Sai Cheemalapati4ba8c232017-06-06 18:46:08 -04001659 "sizeGb": 42, # The size of the disk. Defaults to 500 (GB).
1660 # This field is not applicable for local SSD.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001661 "autoDelete": True or False, # Deprecated. Disks created by the Pipelines API will be deleted at the end
1662 # of the pipeline run, regardless of what this field is set to.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001663 "source": "A String", # The full or partial URL of the persistent disk to attach. See
1664 # https://cloud.google.com/compute/docs/reference/latest/instances#resource
1665 # and
1666 # https://cloud.google.com/compute/docs/disks/persistent-disks#snapshots
1667 # for more details.
Sai Cheemalapati4ba8c232017-06-06 18:46:08 -04001668 "readOnly": True or False, # Specifies how a sourced-base persistent disk will be mounted. See
1669 # https://cloud.google.com/compute/docs/disks/persistent-disks#use_multi_instances
1670 # for more details.
1671 # Can only be set at create time.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001672 "mountPoint": "A String", # Required at create time and cannot be overridden at run time.
1673 # Specifies the path in the docker container where files on
1674 # this disk should be located. For example, if `mountPoint`
1675 # is `/mnt/disk`, and the parameter has `localPath`
1676 # `inputs/file.txt`, the docker container can access the data at
1677 # `/mnt/disk/inputs/file.txt`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001678 "type": "A String", # Required. The type of the disk to create.
1679 },
1680 ],
1681 "bootDiskSizeGb": 42, # The size of the boot disk. Defaults to 10 (GB).
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001682 "zones": [ # List of Google Compute Engine availability zones to which resource
1683 # creation will restricted. If empty, any zone may be chosen.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001684 "A String",
1685 ],
1686 "minimumCpuCores": 42, # The minimum number of cores to use. Defaults to 1.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001687 "acceleratorType": "A String", # Optional. The Compute Engine defined accelerator type.
1688 # By specifying this parameter, you will download and install the following
1689 # third-party software onto your managed Compute Engine instances: NVIDIA®
1690 # Tesla® drivers and NVIDIA® CUDA toolkit.
1691 # Please see https://cloud.google.com/compute/docs/gpus/ for a list of
1692 # available accelerator types.
1693 "acceleratorCount": "A String", # Optional. The number of accelerators of the specified type to attach.
1694 # By specifying this parameter, you will download and install the following
1695 # third-party software onto your managed Compute Engine instances:
1696 # NVIDIA® Tesla® drivers and NVIDIA® CUDA toolkit.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001697 },
1698 },
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001699 "ephemeralPipeline": { # The pipeline object. Represents a transformation from a set of input # A new pipeline object to run once and then delete.
1700 # parameters to a set of output parameters. The transformation is defined
1701 # as a docker image and command to run within that image. Each pipeline
1702 # is run on a Google Compute Engine VM. A pipeline can be created with the
1703 # `create` method and then later run with the `run` method, or a pipeline can
1704 # be defined and run all at once with the `run` method.
1705 "pipelineId": "A String", # Unique pipeline id that is generated by the service when CreatePipeline
1706 # is called. Cannot be specified in the Pipeline used in the
1707 # CreatePipelineRequest, and will be populated in the response to
1708 # CreatePipeline and all subsequent Get and List calls. Indicates that the
1709 # service has registered this pipeline.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001710 "description": "A String", # User-specified description.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001711 "projectId": "A String", # Required. The project in which to create the pipeline. The caller must have
1712 # WRITE access.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001713 "outputParameters": [ # Output parameters of the pipeline.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001714 { # Parameters facilitate setting and delivering data into the
1715 # pipeline's execution environment. They are defined at create time,
1716 # with optional defaults, and can be overridden at run time.
1717 #
1718 # If `localCopy` is unset, then the parameter specifies a string that
1719 # is passed as-is into the pipeline, as the value of the environment
1720 # variable with the given name. A default value can be optionally
1721 # specified at create time. The default can be overridden at run time
1722 # using the inputs map. If no default is given, a value must be
1723 # supplied at runtime.
1724 #
1725 # If `localCopy` is defined, then the parameter specifies a data
1726 # source or sink, both in Google Cloud Storage and on the Docker container
1727 # where the pipeline computation is run. The service account associated with
1728 # the Pipeline (by
1729 # default the project's Compute Engine service account) must have access to the
1730 # Google Cloud Storage paths.
1731 #
1732 # At run time, the Google Cloud Storage paths can be overridden if a default
1733 # was provided at create time, or must be set otherwise. The pipeline runner
1734 # should add a key/value pair to either the inputs or outputs map. The
1735 # indicated data copies will be carried out before/after pipeline execution,
1736 # just as if the corresponding arguments were provided to `gsutil cp`.
1737 #
1738 # For example: Given the following `PipelineParameter`, specified
1739 # in the `inputParameters` list:
1740 #
1741 # ```
1742 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}}
1743 # ```
1744 #
1745 # where `disk` is defined in the `PipelineResources` object as:
1746 #
1747 # ```
1748 # {name: "pd1", mountPoint: "/mnt/disk/"}
1749 # ```
1750 #
1751 # We create a disk named `pd1`, mount it on the host VM, and map
1752 # `/mnt/pd1` to `/mnt/disk` in the docker container. At
1753 # runtime, an entry for `input_file` would be required in the inputs
1754 # map, such as:
1755 #
1756 # ```
1757 # inputs["input_file"] = "gs://my-bucket/bar.txt"
1758 # ```
1759 #
1760 # This would generate the following gsutil call:
1761 #
1762 # ```
1763 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt
1764 # ```
1765 #
1766 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the
1767 # Docker container. Acceptable paths are:
1768 #
1769 # <table>
1770 # <thead>
1771 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr>
1772 # </thead>
1773 # <tbody>
1774 # <tr><td>file</td><td>file</td></tr>
1775 # <tr><td>glob</td><td>directory</td></tr>
1776 # </tbody>
1777 # </table>
1778 #
1779 # For outputs, the direction of the copy is reversed:
1780 #
1781 # ```
1782 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt
1783 # ```
1784 #
1785 # Acceptable paths are:
1786 #
1787 # <table>
1788 # <thead>
1789 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr>
1790 # </thead>
1791 # <tbody>
1792 # <tr><td>file</td><td>file</td></tr>
1793 # <tr>
1794 # <td>file</td>
1795 # <td>directory - directory must already exist</td>
1796 # </tr>
1797 # <tr>
1798 # <td>glob</td>
1799 # <td>directory - directory will be created if it doesn't exist</td></tr>
1800 # </tbody>
1801 # </table>
1802 #
1803 # One restriction due to docker limitations, is that for outputs that are found
1804 # on the boot disk, the local path cannot be a glob and must be a file.
1805 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime.
1806 # If `localCopy` is present, then this must be a Google Cloud Storage path
1807 # beginning with `gs://`.
1808 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM.
1809 # `LocalCopy` indicates where on the VM the file should be. The value
1810 # given to this parameter (either at runtime or using `defaultValue`)
1811 # must be the remote path where the file should be.
1812 "path": "A String", # Required. The path within the user's docker container where
1813 # this input should be localized to and from, relative to the specified
1814 # disk's mount point. For example: file.txt,
1815 "disk": "A String", # Required. The name of the disk where this parameter is
1816 # located. Can be the name of one of the disks specified in the
1817 # Resources field, or "boot", which represents the Docker
1818 # instance's boot disk and has a mount point of `/`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001819 },
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001820 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string
1821 # as the key to the input and output maps in RunPipeline.
1822 "description": "A String", # Human-readable description.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001823 },
1824 ],
1825 "inputParameters": [ # Input parameters of the pipeline.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001826 { # Parameters facilitate setting and delivering data into the
1827 # pipeline's execution environment. They are defined at create time,
1828 # with optional defaults, and can be overridden at run time.
1829 #
1830 # If `localCopy` is unset, then the parameter specifies a string that
1831 # is passed as-is into the pipeline, as the value of the environment
1832 # variable with the given name. A default value can be optionally
1833 # specified at create time. The default can be overridden at run time
1834 # using the inputs map. If no default is given, a value must be
1835 # supplied at runtime.
1836 #
1837 # If `localCopy` is defined, then the parameter specifies a data
1838 # source or sink, both in Google Cloud Storage and on the Docker container
1839 # where the pipeline computation is run. The service account associated with
1840 # the Pipeline (by
1841 # default the project's Compute Engine service account) must have access to the
1842 # Google Cloud Storage paths.
1843 #
1844 # At run time, the Google Cloud Storage paths can be overridden if a default
1845 # was provided at create time, or must be set otherwise. The pipeline runner
1846 # should add a key/value pair to either the inputs or outputs map. The
1847 # indicated data copies will be carried out before/after pipeline execution,
1848 # just as if the corresponding arguments were provided to `gsutil cp`.
1849 #
1850 # For example: Given the following `PipelineParameter`, specified
1851 # in the `inputParameters` list:
1852 #
1853 # ```
1854 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}}
1855 # ```
1856 #
1857 # where `disk` is defined in the `PipelineResources` object as:
1858 #
1859 # ```
1860 # {name: "pd1", mountPoint: "/mnt/disk/"}
1861 # ```
1862 #
1863 # We create a disk named `pd1`, mount it on the host VM, and map
1864 # `/mnt/pd1` to `/mnt/disk` in the docker container. At
1865 # runtime, an entry for `input_file` would be required in the inputs
1866 # map, such as:
1867 #
1868 # ```
1869 # inputs["input_file"] = "gs://my-bucket/bar.txt"
1870 # ```
1871 #
1872 # This would generate the following gsutil call:
1873 #
1874 # ```
1875 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt
1876 # ```
1877 #
1878 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the
1879 # Docker container. Acceptable paths are:
1880 #
1881 # <table>
1882 # <thead>
1883 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr>
1884 # </thead>
1885 # <tbody>
1886 # <tr><td>file</td><td>file</td></tr>
1887 # <tr><td>glob</td><td>directory</td></tr>
1888 # </tbody>
1889 # </table>
1890 #
1891 # For outputs, the direction of the copy is reversed:
1892 #
1893 # ```
1894 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt
1895 # ```
1896 #
1897 # Acceptable paths are:
1898 #
1899 # <table>
1900 # <thead>
1901 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr>
1902 # </thead>
1903 # <tbody>
1904 # <tr><td>file</td><td>file</td></tr>
1905 # <tr>
1906 # <td>file</td>
1907 # <td>directory - directory must already exist</td>
1908 # </tr>
1909 # <tr>
1910 # <td>glob</td>
1911 # <td>directory - directory will be created if it doesn't exist</td></tr>
1912 # </tbody>
1913 # </table>
1914 #
1915 # One restriction due to docker limitations, is that for outputs that are found
1916 # on the boot disk, the local path cannot be a glob and must be a file.
1917 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime.
1918 # If `localCopy` is present, then this must be a Google Cloud Storage path
1919 # beginning with `gs://`.
1920 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM.
1921 # `LocalCopy` indicates where on the VM the file should be. The value
1922 # given to this parameter (either at runtime or using `defaultValue`)
1923 # must be the remote path where the file should be.
1924 "path": "A String", # Required. The path within the user's docker container where
1925 # this input should be localized to and from, relative to the specified
1926 # disk's mount point. For example: file.txt,
1927 "disk": "A String", # Required. The name of the disk where this parameter is
1928 # located. Can be the name of one of the disks specified in the
1929 # Resources field, or "boot", which represents the Docker
1930 # instance's boot disk and has a mount point of `/`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001931 },
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001932 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string
1933 # as the key to the input and output maps in RunPipeline.
1934 "description": "A String", # Human-readable description.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001935 },
1936 ],
1937 "docker": { # The Docker execuctor specification. # Specifies the docker run information.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001938 "cmd": "A String", # Required. The command or newline delimited script to run. The command
1939 # string will be executed within a bash shell.
1940 #
1941 # If the command exits with a non-zero exit code, output parameter
1942 # de-localization will be skipped and the pipeline operation's
1943 # `error` field will be populated.
1944 #
1945 # Maximum command string length is 16384.
1946 "imageName": "A String", # Required. Image name from either Docker Hub or Google Container Registry.
1947 # Users that run pipelines must have READ access to the image.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001948 },
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001949 "resources": { # The system resources for the pipeline run. # Required. Specifies resource requirements for the pipeline run.
1950 # Required fields:
1951 #
1952 # *
1953 # minimumCpuCores
1954 #
1955 # *
1956 # minimumRamGb
1957 "noAddress": True or False, # Whether to assign an external IP to the instance. This is an experimental
1958 # feature that may go away. Defaults to false.
1959 # Corresponds to `--no_address` flag for [gcloud compute instances create]
1960 # (https://cloud.google.com/sdk/gcloud/reference/compute/instances/create).
1961 # In order to use this, must be true for both create time and run time.
1962 # Cannot be true at run time if false at create time. If you need to ssh into
1963 # a private IP VM for debugging, you can ssh to a public VM and then ssh into
1964 # the private VM's Internal IP. If noAddress is set, this pipeline run may
1965 # only load docker images from Google Container Registry and not Docker Hub.
1966 # Before using this, you must
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001967 # [configure access to Google services from internal
1968 # IPs](https://cloud.google.com/compute/docs/configure-private-google-access#configuring_access_to_google_services_from_internal_ips).
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001969 "minimumRamGb": 3.14, # The minimum amount of RAM to use. Defaults to 3.75 (GB)
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001970 "preemptible": True or False, # Whether to use preemptible VMs. Defaults to `false`. In order to use this,
1971 # must be true for both create time and run time. Cannot be true at run time
1972 # if false at create time.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001973 "disks": [ # Disks to attach.
1974 { # A Google Compute Engine disk resource specification.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001975 "name": "A String", # Required. The name of the disk that can be used in the pipeline
1976 # parameters. Must be 1 - 63 characters.
1977 # The name "boot" is reserved for system use.
Sai Cheemalapati4ba8c232017-06-06 18:46:08 -04001978 "sizeGb": 42, # The size of the disk. Defaults to 500 (GB).
1979 # This field is not applicable for local SSD.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07001980 "autoDelete": True or False, # Deprecated. Disks created by the Pipelines API will be deleted at the end
1981 # of the pipeline run, regardless of what this field is set to.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001982 "source": "A String", # The full or partial URL of the persistent disk to attach. See
1983 # https://cloud.google.com/compute/docs/reference/latest/instances#resource
1984 # and
1985 # https://cloud.google.com/compute/docs/disks/persistent-disks#snapshots
1986 # for more details.
Sai Cheemalapati4ba8c232017-06-06 18:46:08 -04001987 "readOnly": True or False, # Specifies how a sourced-base persistent disk will be mounted. See
1988 # https://cloud.google.com/compute/docs/disks/persistent-disks#use_multi_instances
1989 # for more details.
1990 # Can only be set at create time.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04001991 "mountPoint": "A String", # Required at create time and cannot be overridden at run time.
1992 # Specifies the path in the docker container where files on
1993 # this disk should be located. For example, if `mountPoint`
1994 # is `/mnt/disk`, and the parameter has `localPath`
1995 # `inputs/file.txt`, the docker container can access the data at
1996 # `/mnt/disk/inputs/file.txt`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07001997 "type": "A String", # Required. The type of the disk to create.
1998 },
1999 ],
2000 "bootDiskSizeGb": 42, # The size of the boot disk. Defaults to 10 (GB).
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04002001 "zones": [ # List of Google Compute Engine availability zones to which resource
2002 # creation will restricted. If empty, any zone may be chosen.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07002003 "A String",
2004 ],
2005 "minimumCpuCores": 42, # The minimum number of cores to use. Defaults to 1.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07002006 "acceleratorType": "A String", # Optional. The Compute Engine defined accelerator type.
2007 # By specifying this parameter, you will download and install the following
2008 # third-party software onto your managed Compute Engine instances: NVIDIA®
2009 # Tesla® drivers and NVIDIA® CUDA toolkit.
2010 # Please see https://cloud.google.com/compute/docs/gpus/ for a list of
2011 # available accelerator types.
2012 "acceleratorCount": "A String", # Optional. The number of accelerators of the specified type to attach.
2013 # By specifying this parameter, you will download and install the following
2014 # third-party software onto your managed Compute Engine instances:
2015 # NVIDIA® Tesla® drivers and NVIDIA® CUDA toolkit.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07002016 },
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07002017 "name": "A String", # Required. A user specified pipeline name that does not have to be unique.
2018 # This name can be used for filtering Pipelines in ListPipelines.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07002019 },
2020 "pipelineId": "A String", # The already created pipeline to run.
2021 }
2022
2023 x__xgafv: string, V1 error format.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04002024 Allowed values
2025 1 - v1 error format
2026 2 - v2 error format
Sai Cheemalapatidf613972016-10-21 13:59:49 -07002027
2028Returns:
2029 An object of the form:
2030
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04002031 { # This resource represents a long-running operation that is the result of a
2032 # network API call.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07002033 "metadata": { # An OperationMetadata or Metadata object. This will always be returned with the Operation.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07002034 "a_key": "", # Properties of the object. Contains field @type with type URL.
2035 },
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07002036 "done": True or False, # If the value is `false`, it means the operation is still in progress.
2037 # If `true`, the operation is completed, and either `error` or `response` is
2038 # available.
2039 "response": { # An Empty object.
2040 "a_key": "", # Properties of the object. Contains field @type with type URL.
2041 },
2042 "name": "A String", # The server-assigned name, which is only unique within the same service that originally returns it. For example&#58; `operations/CJHU7Oi_ChDrveSpBRjfuL-qzoWAgEw`
2043 "error": { # The `Status` type defines a logical error model that is suitable for # The error result of the operation in case of failure or cancellation.
2044 # different programming environments, including REST APIs and RPC APIs. It is
2045 # used by [gRPC](https://github.com/grpc). Each `Status` message contains
2046 # three pieces of data: error code, error message, and error details.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04002047 #
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07002048 # You can find out more about this error model and how to work with it in the
2049 # [API Design Guide](https://cloud.google.com/apis/design/errors).
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04002050 "message": "A String", # A developer-facing error message, which should be in English. Any
2051 # user-facing error message should be localized and sent in the
2052 # google.rpc.Status.details field, or localized by the client.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07002053 "code": 42, # The status code, which should be an enum value of google.rpc.Code.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -07002054 "details": [ # A list of messages that carry the error details. There is a common set of
2055 # message types for APIs to use.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07002056 {
2057 "a_key": "", # Properties of the object. Contains field @type with type URL.
2058 },
2059 ],
2060 },
Sai Cheemalapatidf613972016-10-21 13:59:49 -07002061 }</pre>
2062</div>
2063
2064<div class="method">
2065 <code class="details" id="setOperationStatus">setOperationStatus(body, x__xgafv=None)</code>
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04002066 <pre>Sets status of a given operation. Any new timestamps (as determined by
2067description) are appended to TimestampEvents. Should only be called by VMs
2068created by the Pipelines Service and not by end users.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07002069
2070Args:
2071 body: object, The request body. (required)
2072 The object takes the form of:
2073
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04002074{ # Request to set operation status. Should only be used by VMs
2075 # created by the Pipelines Service and not by end users.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07002076 "errorCode": "A String",
Thomas Coffee2f245372017-03-27 10:39:26 -07002077 "errorMessage": "A String",
Sai Cheemalapatidf613972016-10-21 13:59:49 -07002078 "timestampEvents": [
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04002079 { # Stores the list of events and times they occured for major events in job
2080 # execution.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07002081 "timestamp": "A String", # The time this event occured.
2082 "description": "A String", # String indicating the type of event
2083 },
2084 ],
Sai Cheemalapatidf613972016-10-21 13:59:49 -07002085 "operationId": "A String",
Thomas Coffee2f245372017-03-27 10:39:26 -07002086 "validationToken": "A String",
Sai Cheemalapatidf613972016-10-21 13:59:49 -07002087 }
2088
2089 x__xgafv: string, V1 error format.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04002090 Allowed values
2091 1 - v1 error format
2092 2 - v2 error format
Sai Cheemalapatidf613972016-10-21 13:59:49 -07002093
2094Returns:
2095 An object of the form:
2096
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -04002097 { # A generic empty message that you can re-use to avoid defining duplicated
2098 # empty messages in your APIs. A typical example is to use it as the request
2099 # or the response type of an API method. For instance:
2100 #
2101 # service Foo {
2102 # rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty);
2103 # }
2104 #
2105 # The JSON representation for `Empty` is empty JSON object `{}`.
Sai Cheemalapatidf613972016-10-21 13:59:49 -07002106 }</pre>
2107</div>
2108
2109</body></html>