Blame - docs/dyn/dataflow_v1b3.projects.locations.html - platform/external/python/google-api-python-client

2020-05-01 07:42:23 -0700

[diff] [blame]

78

<code><a href="dataflow_v1b3.projects.locations.flexTemplates.html">flexTemplates()</a></code>

79

</p>

80

<p class="firstline">Returns the flexTemplates Resource.</p>

81

82

Jon Wayne Parrott

692617a

2017-01-06 09:58:29 -0800

[diff] [blame]

83

84

</p>

85

<p class="firstline">Returns the jobs Resource.</p>

86

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

87

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

88

<code><a href="dataflow_v1b3.projects.locations.snapshots.html">snapshots()</a></code>

89

</p>

90

<p class="firstline">Returns the snapshots Resource.</p>

</p>

<p class="firstline">Returns the sql Resource.</p>

96

97

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

98

<code><a href="dataflow_v1b3.projects.locations.templates.html">templates()</a></code>

99

</p>

100

<p class="firstline">Returns the templates Resource.</p>

101

102

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

103

<code><a href="#workerMessages">workerMessages(projectId, location, body=None, x__xgafv=None)</a></code></p>

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

104

<p class="firstline">Send a worker_message to the service.</p>

105

<h3>Method Details</h3>

106

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

107

<code class="details" id="workerMessages">workerMessages(projectId, location, body=None, x__xgafv=None)</code>

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

108

<pre>Send a worker_message to the service.

109

110

Args:

111

projectId: string, The project to send the WorkerMessages to. (required)

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

112

location: string, The [regional endpoint]

113

(https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that

114

contains the job. (required)

Dan O'Meara

2020-05-01 07:42:23 -0700

[diff] [blame]

115

body: object, The request body.

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

116

The object takes the form of:

117

118

{ # A request for sending worker messages to the service.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

119

"location": "A String", # The [regional endpoint]

120

# (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that

121

# contains the job.

122

"workerMessages": [ # The WorkerMessages to send.

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

123

{ # WorkerMessage provides information to the backend about a worker.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

124

"workerMetrics": { # Worker metrics exported from workers. This contains resource utilization # Resource metrics reported by workers.

125

# metrics accumulated from a variety of sources. For more information, see

126

# go/df-resource-signals.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

127

"containers": { # Per container information.

128

# Key: container name.

129

"a_key": # Object with schema name: ResourceUtilizationReport

130

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

131

"memoryInfo": [ # Memory utilization samples.

132

{ # Information about the memory usage of a worker or a container within a

133

# worker.

134

"timestamp": "A String", # Timestamp of the measurement.

135

"currentLimitBytes": "A String", # Instantenous memory limit in bytes.

136

"currentRssBytes": "A String", # Instantenous memory (RSS) size in bytes.

137

"totalGbMs": "A String", # Total memory (RSS) usage since start up in GB * ms.

138

},

139

],

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

140

"cpuTime": [ # CPU utilization samples.

141

{ # Modeled after information exposed by /proc/stat.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

142

"rate": 3.14, # Average CPU utilization rate (% non-idle cpu / second) since previous

143

# sample.

144

"timestamp": "A String", # Timestamp of the measurement.

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

145

"totalMs": "A String", # Total active CPU time across all cores (ie., non-idle) in milliseconds

146

# since start-up.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

},

],

},

"workerMessageCode": { # A message code is used to report status and error messages to the service. # A worker message code.

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

151

# The message codes are intended to be machine readable. The service will

152

# take care of translating these into user understandable messages if

# necessary.

#

# Example use cases:

# 1. Worker processes reporting successful startup.

157

# 2. Worker processes reporting specific errors (e.g. package staging

158

# failure).

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

159

"code": "A String", # The code is a string intended for consumption by a machine that identifies

160

# the type of message being sent.

161

# Examples:

162

# 1. "HARNESS_STARTED" might be used to indicate the worker harness has

163

# started.

164

# 2. "GCS_DOWNLOAD_ERROR" might be used to indicate an error downloading

165

# a GCS file as part of the boot process of one of the worker containers.

166

#

167

# This is a string and not an enum to make it easy to add new codes without

168

# waiting for an API change.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

169

"parameters": { # Parameters contains specific information about the code.

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

170

#

171

# This is a struct to allow parameters of different types.

172

#

173

# Examples:

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

174

# 1. For a "HARNESS_STARTED" message parameters might provide the name

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

175

# of the worker and additional data like timing information.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

176

# 2. For a "GCS_DOWNLOAD_ERROR" parameters might contain fields listing

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

177

# the GCS objects being downloaded and fields containing errors.

178

#

179

# In general complex data structures should be avoided. If a worker

180

# needs to send a specific and complicated data structure then please

181

# consider defining a new proto and adding it to the data oneof in

182

# WorkerMessageResponse.

183

#

184

# Conventions:

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

185

# Parameters should only be used for information that isn't typically passed

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

186

# as a label.

187

# hostname and other worker identifiers should almost always be passed

188

# as labels since they will be included on most messages.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

189

"a_key": "", # Properties of the object.

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

190

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

191

},

192

"labels": { # Labels are used to group WorkerMessages.

193

# For example, a worker_message about a particular container

194

# might have the labels:

195

# { "JOB_ID": "2015-04-22",

196

# "WORKER_ID": "wordcount-vm-2015…"

197

# "CONTAINER_TYPE": "worker",

198

# "CONTAINER_ID": "ac1234def"}

199

# Label tags typically correspond to Label enum values. However, for ease

200

# of development other strings can be used as tags. LABEL_UNSPECIFIED should

# not be used here.

"a_key": "A String",

},

"time": "A String", # The timestamp of the worker_message.

205

"workerLifecycleEvent": { # A report of an event in a worker's lifecycle. # Record of worker lifecycle events.

206

# The proto contains one event, because the worker is expected to

207

# asynchronously send each message immediately after the event.

208

# Due to this asynchrony, messages may arrive out of order (or missing), and it

209

# is up to the consumer to interpret.

210

# The timestamp of the event is in the enclosing WorkerMessage proto.

211

"containerStartTime": "A String", # The start time of this container. All events will report this so that

212

# events can be grouped together across container/VM restarts.

213

"event": "A String", # The event being reported.

214

"metadata": { # Other stats that can accompany an event. E.g.

215

# { "downloaded_bytes" : "123456" }

"a_key": "A String",

},

},

"workerShutdownNotice": { # Shutdown notification from workers. This is to be sent by the shutdown # Shutdown notice by workers.

220

# script of the worker VM so that the backend knows that the VM is being

221

# shut down.

222

"reason": "A String", # The reason for the worker shutdown.

223

# Current possible values are:

224

# "UNKNOWN": shutdown reason is unknown.

225

# "PREEMPTION": shutdown reason is preemption.

226

# Other possible reasons may be added in the future.

227

},

228

"workerHealthReport": { # WorkerHealthReport contains information about the health of a worker. # The health of a worker.

229

#

230

# The VM should be identified by the labels attached to the WorkerMessage that

231

# this health ping belongs to.

232

"pods": [ # The pods running on the worker. See:

233

# http://kubernetes.io/v1.1/docs/api-reference/v1/definitions.html#_v1_pod

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

234

#

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

235

# This field is used by the worker to send the status of the indvidual

236

# containers running on each worker.

237

{

238

"a_key": "", # Properties of the object.

239

},

240

],

241

"reportInterval": "A String", # The interval at which the worker is sending health reports.

242

# The default value of 0 should be interpreted as the field is not being

243

# explicitly set by the worker.

244

"vmIsHealthy": True or False, # Whether the VM is currently healthy.

245

"vmIsBroken": True or False, # Whether the VM is in a permanently broken state.

246

# Broken VMs should be abandoned or deleted ASAP to avoid assigning or

247

# completing any work.

248

"vmStartupTime": "A String", # The time the VM was booted.

249

"msg": "A String", # A message describing any unusual health reports.

Bu Sun Kim

2019-06-14 16:50:42 -0700

[diff] [blame]

250

},

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

251

},

252

],

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

253

}

254

255

x__xgafv: string, V1 error format.

Allowed values

1 - v1 error format

2 - v2 error format

Returns:

An object of the form:

262

263

{ # The response to the worker messages.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

264

"workerMessageResponses": [ # The servers response to the worker messages.

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

265

{ # A worker_message response allows the server to pass information to the

266

# sender.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

267

"workerHealthReportResponse": { # WorkerHealthReportResponse contains information returned to the worker # The service's response to a worker's health report.

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

268

# in response to a health ping.

Bu Sun Kim

2020-05-20 12:08:20 -0700

[diff] [blame]

269

"reportInterval": "A String", # A positive value indicates the worker should change its reporting interval

Sai Cheemalapati

2017-06-06 18:46:08 -0400

[diff] [blame]

270

# to the specified value.

271

#

272

# The default value of zero means no change in report rate is requested by

273

# the server.

274

},

Bu Sun Kim

2020-05-27 12:20:54 -0700

[diff] [blame]

275

"workerShutdownNoticeResponse": { # Service-side response to WorkerMessage issuing shutdown notice. # Service's response to shutdown notice (currently empty).

276

},

277

"workerMetricsResponse": { # Service-side response to WorkerMessage reporting resource utilization. # Service's response to reporting worker metrics (currently empty).

278

},

Sai Cheemalapati