Blame - docs/dyn/dataflow_v1b3.projects.locations.html - platform/external/python/google-api-python-client

2020-05-01 07:42:23 -0700

[diff] [blame]

78

<code><a href="dataflow_v1b3.projects.locations.flexTemplates.html">flexTemplates()</a></code>

79

</p>

80

<p class="firstline">Returns the flexTemplates Resource.</p>

81

82

Jon Wayne Parrott

692617a

2017-01-06 09:58:29 -0800

[diff] [blame]

83

84

</p>

85

<p class="firstline">Returns the jobs Resource.</p>

86

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

87

Bu Sun Kim

715bd7f

2019-06-14 16:50:42 -0700

[diff] [blame]

88

<code><a href="dataflow_v1b3.projects.locations.snapshots.html">snapshots()</a></code>

89

</p>

90

<p class="firstline">Returns the snapshots Resource.</p>

</p>

<p class="firstline">Returns the sql Resource.</p>

96

97

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

98

<code><a href="dataflow_v1b3.projects.locations.templates.html">templates()</a></code>

99

</p>

100

<p class="firstline">Returns the templates Resource.</p>

101

102

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

103

<code><a href="#workerMessages">workerMessages(projectId, location, body=None, x__xgafv=None)</a></code></p>

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

104

<p class="firstline">Send a worker_message to the service.</p>

105

<h3>Method Details</h3>

106

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

107

<code class="details" id="workerMessages">workerMessages(projectId, location, body=None, x__xgafv=None)</code>

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

108

<pre>Send a worker_message to the service.

109

110

Args:

111

projectId: string, The project to send the WorkerMessages to. (required)

Bu Sun Kim

715bd7f

2019-06-14 16:50:42 -0700

[diff] [blame]

112

location: string, The [regional endpoint]

113

(https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that

114

contains the job. (required)

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

115

body: object, The request body.

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

116

The object takes the form of:

117

118

{ # A request for sending worker messages to the service.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

119

"workerMessages": [ # The WorkerMessages to send.

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

120

{ # WorkerMessage provides information to the backend about a worker.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

121

"workerMessageCode": { # A message code is used to report status and error messages to the service. # A worker message code.

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

122

# The message codes are intended to be machine readable. The service will

123

# take care of translating these into user understandable messages if

# necessary.

#

# Example use cases:

# 1. Worker processes reporting successful startup.

128

# 2. Worker processes reporting specific errors (e.g. package staging

129

# failure).

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

130

"parameters": { # Parameters contains specific information about the code.

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

131

#

132

# This is a struct to allow parameters of different types.

133

#

134

# Examples:

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

135

# 1. For a "HARNESS_STARTED" message parameters might provide the name

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

136

# of the worker and additional data like timing information.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

137

# 2. For a "GCS_DOWNLOAD_ERROR" parameters might contain fields listing

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

138

# the GCS objects being downloaded and fields containing errors.

139

#

140

# In general complex data structures should be avoided. If a worker

141

# needs to send a specific and complicated data structure then please

142

# consider defining a new proto and adding it to the data oneof in

143

# WorkerMessageResponse.

144

#

145

# Conventions:

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

146

# Parameters should only be used for information that isn't typically passed

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

147

# as a label.

148

# hostname and other worker identifiers should almost always be passed

149

# as labels since they will be included on most messages.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

150

"a_key": "", # Properties of the object.

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

151

},

Bu Sun Kim

d059ad8

2020-07-22 17:02:09 -0700

[diff] [blame]

152

"code": "A String", # The code is a string intended for consumption by a machine that identifies

153

# the type of message being sent.

154

# Examples:

155

# 1. "HARNESS_STARTED" might be used to indicate the worker harness has

156

# started.

157

# 2. "GCS_DOWNLOAD_ERROR" might be used to indicate an error downloading

158

# a GCS file as part of the boot process of one of the worker containers.

159

#

160

# This is a string and not an enum to make it easy to add new codes without

161

# waiting for an API change.

162

},

163

"time": "A String", # The timestamp of the worker_message.

164

"workerLifecycleEvent": { # A report of an event in a worker's lifecycle. # Record of worker lifecycle events.

165

# The proto contains one event, because the worker is expected to

166

# asynchronously send each message immediately after the event.

167

# Due to this asynchrony, messages may arrive out of order (or missing), and it

168

# is up to the consumer to interpret.

169

# The timestamp of the event is in the enclosing WorkerMessage proto.

170

"event": "A String", # The event being reported.

171

"metadata": { # Other stats that can accompany an event. E.g.

172

# { "downloaded_bytes" : "123456" }

173

"a_key": "A String",

174

},

175

"containerStartTime": "A String", # The start time of this container. All events will report this so that

176

# events can be grouped together across container/VM restarts.

177

},

178

"workerMetrics": { # Worker metrics exported from workers. This contains resource utilization # Resource metrics reported by workers.

179

# metrics accumulated from a variety of sources. For more information, see

180

# go/df-resource-signals.

181

"containers": { # Per container information.

182

# Key: container name.

183

"a_key": # Object with schema name: ResourceUtilizationReport

184

},

185

"cpuTime": [ # CPU utilization samples.

186

{ # Modeled after information exposed by /proc/stat.

187

"rate": 3.14, # Average CPU utilization rate (% non-idle cpu / second) since previous

188

# sample.

189

"totalMs": "A String", # Total active CPU time across all cores (ie., non-idle) in milliseconds

190

# since start-up.

191

"timestamp": "A String", # Timestamp of the measurement.

192

},

193

],

194

"memoryInfo": [ # Memory utilization samples.

195

{ # Information about the memory usage of a worker or a container within a

196

# worker.

197

"totalGbMs": "A String", # Total memory (RSS) usage since start up in GB * ms.

198

"timestamp": "A String", # Timestamp of the measurement.

199

"currentLimitBytes": "A String", # Instantenous memory limit in bytes.

200

"currentRssBytes": "A String", # Instantenous memory (RSS) size in bytes.

},

],

},

"workerHealthReport": { # WorkerHealthReport contains information about the health of a worker. # The health of a worker.

205

#

206

# The VM should be identified by the labels attached to the WorkerMessage that

207

# this health ping belongs to.

208

"vmIsHealthy": True or False, # Whether the VM is currently healthy.

209

"pods": [ # The pods running on the worker. See:

210

# http://kubernetes.io/v1.1/docs/api-reference/v1/definitions.html#_v1_pod

211

#

212

# This field is used by the worker to send the status of the indvidual

213

# containers running on each worker.

214

{

215

"a_key": "", # Properties of the object.

216

},

217

],

218

"msg": "A String", # A message describing any unusual health reports.

219

"vmStartupTime": "A String", # The time the VM was booted.

220

"reportInterval": "A String", # The interval at which the worker is sending health reports.

221

# The default value of 0 should be interpreted as the field is not being

222

# explicitly set by the worker.

223

"vmIsBroken": True or False, # Whether the VM is in a permanently broken state.

224

# Broken VMs should be abandoned or deleted ASAP to avoid assigning or

225

# completing any work.

Bu Sun Kim

4ed7d3f

2020-05-27 12:20:54 -0700

[diff] [blame]

226

},

227

"labels": { # Labels are used to group WorkerMessages.

228

# For example, a worker_message about a particular container

229

# might have the labels:

230

# { "JOB_ID": "2015-04-22",

231

# "WORKER_ID": "wordcount-vm-2015…"

232

# "CONTAINER_TYPE": "worker",

233

# "CONTAINER_ID": "ac1234def"}

234

# Label tags typically correspond to Label enum values. However, for ease

235

# of development other strings can be used as tags. LABEL_UNSPECIFIED should

236

# not be used here.

237

"a_key": "A String",

238

},

Bu Sun Kim

4ed7d3f

2020-05-27 12:20:54 -0700

[diff] [blame]

239

"workerShutdownNotice": { # Shutdown notification from workers. This is to be sent by the shutdown # Shutdown notice by workers.

240

# script of the worker VM so that the backend knows that the VM is being

241

# shut down.

242

"reason": "A String", # The reason for the worker shutdown.

243

# Current possible values are:

244

# "UNKNOWN": shutdown reason is unknown.

245

# "PREEMPTION": shutdown reason is preemption.

246

# Other possible reasons may be added in the future.

247

},

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

248

},

249

],

Bu Sun Kim

d059ad8

2020-07-22 17:02:09 -0700

[diff] [blame]

250

"location": "A String", # The [regional endpoint]

251

# (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that

252

# contains the job.

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

253

}

254

255

x__xgafv: string, V1 error format.

Allowed values

1 - v1 error format

2 - v2 error format

Returns:

An object of the form:

262

263

{ # The response to the worker messages.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

264

"workerMessageResponses": [ # The servers response to the worker messages.

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

265

{ # A worker_message response allows the server to pass information to the

266

# sender.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

267

"workerHealthReportResponse": { # WorkerHealthReportResponse contains information returned to the worker # The service's response to a worker's health report.

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

268

# in response to a health ping.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

269

"reportInterval": "A String", # A positive value indicates the worker should change its reporting interval

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

270

# to the specified value.

271

#

272

# The default value of zero means no change in report rate is requested by

273

# the server.

274

},

Bu Sun Kim

4ed7d3f

2020-05-27 12:20:54 -0700

[diff] [blame]

275

"workerMetricsResponse": { # Service-side response to WorkerMessage reporting resource utilization. # Service's response to reporting worker metrics (currently empty).

276

},

Bu Sun Kim

d059ad8

2020-07-22 17:02:09 -0700

[diff] [blame]

277

"workerShutdownNoticeResponse": { # Service-side response to WorkerMessage issuing shutdown notice. # Service's response to shutdown notice (currently empty).

278

},

Sai Cheemalapati