blob: 275876c359070855bb18f1a9ffd3fd2d32f32801 [file] [log] [blame]
Nathaniel Manista4f877e52015-06-15 16:44:50 +00001<html><body>
2<style>
3
4body, h1, h2, h3, div, span, p, pre, a {
5 margin: 0;
6 padding: 0;
7 border: 0;
8 font-weight: inherit;
9 font-style: inherit;
10 font-size: 100%;
11 font-family: inherit;
12 vertical-align: baseline;
13}
14
15body {
16 font-size: 13px;
17 padding: 1em;
18}
19
20h1 {
21 font-size: 26px;
22 margin-bottom: 1em;
23}
24
25h2 {
26 font-size: 24px;
27 margin-bottom: 1em;
28}
29
30h3 {
31 font-size: 20px;
32 margin-bottom: 1em;
33 margin-top: 1em;
34}
35
36pre, code {
37 line-height: 1.5;
38 font-family: Monaco, 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Lucida Console', monospace;
39}
40
41pre {
42 margin-top: 0.5em;
43}
44
45h1, h2, h3, p {
46 font-family: Arial, sans serif;
47}
48
49h1, h2, h3 {
50 border-bottom: solid #CCC 1px;
51}
52
53.toc_element {
54 margin-top: 0.5em;
55}
56
57.firstline {
58 margin-left: 2 em;
59}
60
61.method {
62 margin-top: 1em;
63 border: solid 1px #CCC;
64 padding: 1em;
65 background: #EEE;
66}
67
68.details {
69 font-weight: bold;
70 font-size: 14px;
71}
72
73</style>
74
Bu Sun Kim715bd7f2019-06-14 16:50:42 -070075<h1><a href="dataflow_v1b3.html">Dataflow API</a> . <a href="dataflow_v1b3.projects.html">projects</a></h1>
Nathaniel Manista4f877e52015-06-15 16:44:50 +000076<h2>Instance Methods</h2>
77<p class="toc_element">
78 <code><a href="dataflow_v1b3.projects.jobs.html">jobs()</a></code>
79</p>
80<p class="firstline">Returns the jobs Resource.</p>
81
Jon Wayne Parrott36e41bc2016-02-19 16:02:29 -080082<p class="toc_element">
Jon Wayne Parrott692617a2017-01-06 09:58:29 -080083 <code><a href="dataflow_v1b3.projects.locations.html">locations()</a></code>
84</p>
85<p class="firstline">Returns the locations Resource.</p>
86
87<p class="toc_element">
Bu Sun Kim715bd7f2019-06-14 16:50:42 -070088 <code><a href="dataflow_v1b3.projects.snapshots.html">snapshots()</a></code>
89</p>
90<p class="firstline">Returns the snapshots Resource.</p>
91
92<p class="toc_element">
Jon Wayne Parrott7d5badb2016-08-16 12:44:29 -070093 <code><a href="dataflow_v1b3.projects.templates.html">templates()</a></code>
94</p>
95<p class="firstline">Returns the templates Resource.</p>
96
97<p class="toc_element">
Bu Sun Kim65020912020-05-20 12:08:20 -070098 <code><a href="#deleteSnapshots">deleteSnapshots(projectId, snapshotId=None, location=None, x__xgafv=None)</a></code></p>
Bu Sun Kim715bd7f2019-06-14 16:50:42 -070099<p class="firstline">Deletes a snapshot.</p>
100<p class="toc_element">
Dan O'Mearadd494642020-05-01 07:42:23 -0700101 <code><a href="#workerMessages">workerMessages(projectId, body=None, x__xgafv=None)</a></code></p>
Jon Wayne Parrott36e41bc2016-02-19 16:02:29 -0800102<p class="firstline">Send a worker_message to the service.</p>
103<h3>Method Details</h3>
104<div class="method">
Bu Sun Kim65020912020-05-20 12:08:20 -0700105 <code class="details" id="deleteSnapshots">deleteSnapshots(projectId, snapshotId=None, location=None, x__xgafv=None)</code>
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700106 <pre>Deletes a snapshot.
107
108Args:
109 projectId: string, The ID of the Cloud Platform project that the snapshot belongs to. (required)
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700110 snapshotId: string, The ID of the snapshot.
Bu Sun Kim65020912020-05-20 12:08:20 -0700111 location: string, The location that contains this snapshot.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700112 x__xgafv: string, V1 error format.
113 Allowed values
114 1 - v1 error format
115 2 - v2 error format
116
117Returns:
118 An object of the form:
119
120 { # Response from deleting a snapshot.
121 }</pre>
122</div>
123
124<div class="method">
Dan O'Mearadd494642020-05-01 07:42:23 -0700125 <code class="details" id="workerMessages">workerMessages(projectId, body=None, x__xgafv=None)</code>
Jon Wayne Parrott36e41bc2016-02-19 16:02:29 -0800126 <pre>Send a worker_message to the service.
127
128Args:
129 projectId: string, The project to send the WorkerMessages to. (required)
Dan O'Mearadd494642020-05-01 07:42:23 -0700130 body: object, The request body.
Jon Wayne Parrott36e41bc2016-02-19 16:02:29 -0800131 The object takes the form of:
132
133{ # A request for sending worker messages to the service.
Bu Sun Kim65020912020-05-20 12:08:20 -0700134 &quot;location&quot;: &quot;A String&quot;, # The [regional endpoint]
135 # (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that
136 # contains the job.
137 &quot;workerMessages&quot;: [ # The WorkerMessages to send.
Jon Wayne Parrott36e41bc2016-02-19 16:02:29 -0800138 { # WorkerMessage provides information to the backend about a worker.
Bu Sun Kim65020912020-05-20 12:08:20 -0700139 &quot;labels&quot;: { # Labels are used to group WorkerMessages.
140 # For example, a worker_message about a particular container
141 # might have the labels:
142 # { &quot;JOB_ID&quot;: &quot;2015-04-22&quot;,
143 # &quot;WORKER_ID&quot;: &quot;wordcount-vm-2015…&quot;
144 # &quot;CONTAINER_TYPE&quot;: &quot;worker&quot;,
145 # &quot;CONTAINER_ID&quot;: &quot;ac1234def&quot;}
146 # Label tags typically correspond to Label enum values. However, for ease
147 # of development other strings can be used as tags. LABEL_UNSPECIFIED should
148 # not be used here.
149 &quot;a_key&quot;: &quot;A String&quot;,
150 },
151 &quot;time&quot;: &quot;A String&quot;, # The timestamp of the worker_message.
152 &quot;workerLifecycleEvent&quot;: { # A report of an event in a worker&#x27;s lifecycle. # Record of worker lifecycle events.
153 # The proto contains one event, because the worker is expected to
154 # asynchronously send each message immediately after the event.
155 # Due to this asynchrony, messages may arrive out of order (or missing), and it
156 # is up to the consumer to interpret.
157 # The timestamp of the event is in the enclosing WorkerMessage proto.
158 &quot;event&quot;: &quot;A String&quot;, # The event being reported.
159 &quot;containerStartTime&quot;: &quot;A String&quot;, # The start time of this container. All events will report this so that
160 # events can be grouped together across container/VM restarts.
161 &quot;metadata&quot;: { # Other stats that can accompany an event. E.g.
162 # { &quot;downloaded_bytes&quot; : &quot;123456&quot; }
163 &quot;a_key&quot;: &quot;A String&quot;,
164 },
165 },
166 &quot;workerShutdownNotice&quot;: { # Shutdown notification from workers. This is to be sent by the shutdown # Shutdown notice by workers.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700167 # script of the worker VM so that the backend knows that the VM is being
168 # shut down.
Bu Sun Kim65020912020-05-20 12:08:20 -0700169 &quot;reason&quot;: &quot;A String&quot;, # The reason for the worker shutdown.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700170 # Current possible values are:
Bu Sun Kim65020912020-05-20 12:08:20 -0700171 # &quot;UNKNOWN&quot;: shutdown reason is unknown.
172 # &quot;PREEMPTION&quot;: shutdown reason is preemption.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700173 # Other possible reasons may be added in the future.
Jon Wayne Parrott36e41bc2016-02-19 16:02:29 -0800174 },
Bu Sun Kim65020912020-05-20 12:08:20 -0700175 &quot;workerHealthReport&quot;: { # WorkerHealthReport contains information about the health of a worker. # The health of a worker.
176 #
177 # The VM should be identified by the labels attached to the WorkerMessage that
178 # this health ping belongs to.
179 &quot;vmIsHealthy&quot;: True or False, # Whether the VM is currently healthy.
180 &quot;reportInterval&quot;: &quot;A String&quot;, # The interval at which the worker is sending health reports.
181 # The default value of 0 should be interpreted as the field is not being
182 # explicitly set by the worker.
183 &quot;vmStartupTime&quot;: &quot;A String&quot;, # The time the VM was booted.
184 &quot;vmIsBroken&quot;: True or False, # Whether the VM is in a permanently broken state.
185 # Broken VMs should be abandoned or deleted ASAP to avoid assigning or
186 # completing any work.
187 &quot;msg&quot;: &quot;A String&quot;, # A message describing any unusual health reports.
188 &quot;pods&quot;: [ # The pods running on the worker. See:
189 # http://kubernetes.io/v1.1/docs/api-reference/v1/definitions.html#_v1_pod
190 #
191 # This field is used by the worker to send the status of the indvidual
192 # containers running on each worker.
193 {
194 &quot;a_key&quot;: &quot;&quot;, # Properties of the object.
Dan O'Mearadd494642020-05-01 07:42:23 -0700195 },
196 ],
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400197 },
Bu Sun Kim65020912020-05-20 12:08:20 -0700198 &quot;workerMetrics&quot;: { # Worker metrics exported from workers. This contains resource utilization # Resource metrics reported by workers.
199 # metrics accumulated from a variety of sources. For more information, see
200 # go/df-resource-signals.
201 &quot;memoryInfo&quot;: [ # Memory utilization samples.
202 { # Information about the memory usage of a worker or a container within a
203 # worker.
204 &quot;currentRssBytes&quot;: &quot;A String&quot;, # Instantenous memory (RSS) size in bytes.
205 &quot;currentLimitBytes&quot;: &quot;A String&quot;, # Instantenous memory limit in bytes.
206 &quot;totalGbMs&quot;: &quot;A String&quot;, # Total memory (RSS) usage since start up in GB * ms.
207 &quot;timestamp&quot;: &quot;A String&quot;, # Timestamp of the measurement.
208 },
209 ],
210 &quot;containers&quot;: { # Per container information.
211 # Key: container name.
212 &quot;a_key&quot;: # Object with schema name: ResourceUtilizationReport
213 },
214 &quot;cpuTime&quot;: [ # CPU utilization samples.
215 { # Modeled after information exposed by /proc/stat.
216 &quot;totalMs&quot;: &quot;A String&quot;, # Total active CPU time across all cores (ie., non-idle) in milliseconds
217 # since start-up.
218 &quot;rate&quot;: 3.14, # Average CPU utilization rate (% non-idle cpu / second) since previous
219 # sample.
220 &quot;timestamp&quot;: &quot;A String&quot;, # Timestamp of the measurement.
221 },
222 ],
223 },
224 &quot;workerMessageCode&quot;: { # A message code is used to report status and error messages to the service. # A worker message code.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400225 # The message codes are intended to be machine readable. The service will
226 # take care of translating these into user understandable messages if
227 # necessary.
228 #
229 # Example use cases:
230 # 1. Worker processes reporting successful startup.
231 # 2. Worker processes reporting specific errors (e.g. package staging
232 # failure).
Bu Sun Kim65020912020-05-20 12:08:20 -0700233 &quot;parameters&quot;: { # Parameters contains specific information about the code.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400234 #
235 # This is a struct to allow parameters of different types.
236 #
237 # Examples:
Bu Sun Kim65020912020-05-20 12:08:20 -0700238 # 1. For a &quot;HARNESS_STARTED&quot; message parameters might provide the name
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400239 # of the worker and additional data like timing information.
Bu Sun Kim65020912020-05-20 12:08:20 -0700240 # 2. For a &quot;GCS_DOWNLOAD_ERROR&quot; parameters might contain fields listing
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400241 # the GCS objects being downloaded and fields containing errors.
242 #
243 # In general complex data structures should be avoided. If a worker
244 # needs to send a specific and complicated data structure then please
245 # consider defining a new proto and adding it to the data oneof in
246 # WorkerMessageResponse.
247 #
248 # Conventions:
Bu Sun Kim65020912020-05-20 12:08:20 -0700249 # Parameters should only be used for information that isn&#x27;t typically passed
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400250 # as a label.
251 # hostname and other worker identifiers should almost always be passed
252 # as labels since they will be included on most messages.
Bu Sun Kim65020912020-05-20 12:08:20 -0700253 &quot;a_key&quot;: &quot;&quot;, # Properties of the object.
Jon Wayne Parrott36e41bc2016-02-19 16:02:29 -0800254 },
Bu Sun Kim65020912020-05-20 12:08:20 -0700255 &quot;code&quot;: &quot;A String&quot;, # The code is a string intended for consumption by a machine that identifies
256 # the type of message being sent.
257 # Examples:
258 # 1. &quot;HARNESS_STARTED&quot; might be used to indicate the worker harness has
259 # started.
260 # 2. &quot;GCS_DOWNLOAD_ERROR&quot; might be used to indicate an error downloading
261 # a GCS file as part of the boot process of one of the worker containers.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700262 #
Bu Sun Kim65020912020-05-20 12:08:20 -0700263 # This is a string and not an enum to make it easy to add new codes without
264 # waiting for an API change.
Bu Sun Kim715bd7f2019-06-14 16:50:42 -0700265 },
Jon Wayne Parrott36e41bc2016-02-19 16:02:29 -0800266 },
267 ],
268 }
269
270 x__xgafv: string, V1 error format.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400271 Allowed values
272 1 - v1 error format
273 2 - v2 error format
Jon Wayne Parrott36e41bc2016-02-19 16:02:29 -0800274
275Returns:
276 An object of the form:
277
278 { # The response to the worker messages.
Bu Sun Kim65020912020-05-20 12:08:20 -0700279 &quot;workerMessageResponses&quot;: [ # The servers response to the worker messages.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400280 { # A worker_message response allows the server to pass information to the
281 # sender.
Bu Sun Kim65020912020-05-20 12:08:20 -0700282 &quot;workerMetricsResponse&quot;: { # Service-side response to WorkerMessage reporting resource utilization. # Service&#x27;s response to reporting worker metrics (currently empty).
283 },
284 &quot;workerShutdownNoticeResponse&quot;: { # Service-side response to WorkerMessage issuing shutdown notice. # Service&#x27;s response to shutdown notice (currently empty).
285 },
286 &quot;workerHealthReportResponse&quot;: { # WorkerHealthReportResponse contains information returned to the worker # The service&#x27;s response to a worker&#x27;s health report.
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400287 # in response to a health ping.
Bu Sun Kim65020912020-05-20 12:08:20 -0700288 &quot;reportInterval&quot;: &quot;A String&quot;, # A positive value indicates the worker should change its reporting interval
Sai Cheemalapatic30d2b52017-03-13 12:12:03 -0400289 # to the specified value.
290 #
291 # The default value of zero means no change in report rate is requested by
292 # the server.
293 },
Jon Wayne Parrott36e41bc2016-02-19 16:02:29 -0800294 },
295 ],
296 }</pre>
297</div>
298
Nathaniel Manista4f877e52015-06-15 16:44:50 +0000299</body></html>