监控REST API
Flink有一个监控API,可用于查询正在运行的作业的状态和统计信息,以及最近完成的作业。此监视API由Flink自己的仪表板使用,但也可用于自定义监视工具。
监控API是一个REST-ful API,它接受HTTP请求并使用JSON数据进行响应。
概览
监视API由作为_Dispatcher_一部分运行的Web服务器提供支持。默认情况下,此服务器在post处侦听8081
,可以在flink-conf.yaml
via中配置rest.port
。请注意,监视API Web服务器和Web仪表板Web服务器当前是相同的,因此在同一端口上一起运行。但是,它们会响应不同的HTTP URL。
对于多个Dispatchers(用于高可用性),每个Dispatcher将运行其自己的监视API实例,该实例提供有关已完成和正在运行的作业的信息,同时Dispatcher被选为集群Leader。
开发
REST API后台位于flink-runtime
项目中。核心类是org.apache.flink.runtime.webmonitor.WebMonitorEndpoint
设置服务器和请求路由。
我们使用_Netty_和_Netty Router_库来处理REST请求和转换URL。之所以做出这样的选择是因为这种组合具有轻量级依赖性,并且Netty HTTP的性能非常好。
要添加新请求,Required
- 添加一个新
MessageHeaders
类,作为新请求的接口, - 添加一个新
AbstractRestHandler
类,根据添加的类处理请求MessageHeaders
, - 添加处理程序
org.apache.flink.runtime.webmonitor.WebMonitorEndpoint#initializeHandlers()
。
一个很好的例子就是org.apache.flink.runtime.rest.handler.job.JobExceptionsHandler
使用了org.apache.flink.runtime.rest.messages.JobExceptionsHeaders
。
API
REST API是版本化的,通过在url前加上版本前缀,可以查询特定版本。前缀始终是形式v[version_number]
。例如,要访问/foo/bar
一个版本1 将查询/v1/foo/bar
。
如果未指定版本,Flink将默认为支持请求的_最旧_版本。
查询不受支持/不存在的版本将返回404错误。
注意如果群集在传统模式下运行,则REST API版本控制_无效_。对于这种情况,请参阅下面的遗留API。
调度员
/cluster
动作: DELETE
响应代码: 200 OK
关闭群集
请求:
{}
响应:
{}
/config
动作: GET
响应代码: 200 OK
返回WebUI的配置。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:DashboardConfiguration",
"properties" : {
"refresh-interval" : {
"type" : "integer"
},
"timezone-name" : {
"type" : "string"
},
"timezone-offset" : {
"type" : "integer"
},
"flink-version" : {
"type" : "string"
},
"flink-revision" : {
"type" : "string"
}
}
}
/jars
动作: GET
响应代码: 200 OK
返回先前通过'/ jars / upload'上传的所有jar的列表。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:webmonitor:handlers:JarListInfo",
"properties" : {
"address" : {
"type" : "string"
},
"files" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:webmonitor:handlers:JarListInfo:JarFileInfo",
"properties" : {
"id" : {
"type" : "string"
},
"name" : {
"type" : "string"
},
"uploaded" : {
"type" : "integer"
},
"entry" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:webmonitor:handlers:JarListInfo:JarEntryInfo",
"properties" : {
"name" : {
"type" : "string"
},
"description" : {
"type" : "string"
}
}
}
}
}
}
}
}
}
/jars/upload
动作: POST
响应代码: 200 OK
将jar上传到集群。jar必须作为多部分数据发送。确保“Content-Type”标头设置为“application / x-java-archive”,因为某些http库默认情况下不添加标头。使用'curl'你可以通过'curl -X POST -H“上传jar。期望:” - F“jarfile =#path / to / flink-job.jar”http:// hostname:port / jars / upload'。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:webmonitor:handlers:JarUploadResponseBody",
"properties" : {
"filename" : {
"type" : "string"
},
"status" : {
"type" : "string",
"enum" : [ "success" ]
}
}
}
/jars/:jarid
动作: DELETE
响应代码: 200 OK
删除之前通过'/ jars / upload'上传的jar。
路径参数:
jarid
- 标识jar的字符串值。上传jar时会返回一个路径,其中文件名是ID。此值等同于上载的jar(/ jars)列表中的id
字段。
请求:
{}
响应:
{}
/jars/:jarid /plan
动作: GET
响应代码: 200 OK
返回先前通过'/ jars / upload'上传的jar中包含的作业的数据流计划。
路径参数:
jarid
- 标识jar的字符串值。上传jar时会返回一个路径,其中文件名是ID。此值等同于上载的jar(/ jars)列表中的id
字段。
查询参数:
entry-class
(可选):字符串值,指定入口点类的完全限定名称。覆盖jar文件清单中定义的类。parallelism
(可选):正整数值,指定作业所需的并行度。program-args
(可选):字符串值,指定程序或计划的参数。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobPlanInfo",
"properties" : {
"plan" : {
"type" : "any"
}
}
}
/jars/:jarid /run
动作: POST
响应代码: 200 OK
通过运行先前通过'/ jars / upload'上传的jar来提交作业。
路径参数:
jarid
- 标识jar的字符串值。上传jar时会返回一个路径,其中文件名是ID。此值等同于上载的jar(/ jars)列表中的id
字段。
查询参数:
program-args
(可选):字符串值,指定程序或计划的参数。entry-class
(可选):字符串值,指定入口点类的完全限定名称。覆盖jar文件清单中定义的类。parallelism
(可选):正整数值,指定作业所需的并行度。allowNonRestoredState
(可选):布尔值,指定如果保存点包含无法映射回作业的状态,是否应拒绝作业提交。savepointPath
(可选):字符串值,指定要从中还原作业的保存点的路径。
请求:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:webmonitor:handlers:JarRunRequestBody",
"properties" : {
"entryClass" : {
"type" : "string"
},
"programArgs" : {
"type" : "string"
},
"parallelism" : {
"type" : "integer"
},
"allowNonRestoredState" : {
"type" : "boolean"
},
"savepointPath" : {
"type" : "string"
}
}
}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:webmonitor:handlers:JarRunResponseBody",
"properties" : {
"jobid" : {
"type" : "any"
}
}
}
/jobmanager/config
动作: GET
响应代码: 200 OK
返回群集配置。
请求:
{}
响应:
{
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:ClusterConfigurationInfoEntry",
"properties" : {
"key" : {
"type" : "string"
},
"value" : {
"type" : "string"
}
}
}
}
/jobmanager/metrics
动作: GET
响应代码: 200 OK
提供对JobManager指标的访问。
查询参数
get
(可选):以逗号分隔的字符串值列表,用于选择特定指标。
请求:
{}
响应:
{
"type" : "any"
}
/jobs
动作: GET
响应代码: 200 OK
返回所有作业及其当前状态的概述。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:messages:webmonitor:JobIdsWithStatusOverview",
"properties" : {
"jobs" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:messages:webmonitor:JobIdsWithStatusOverview:JobIdWithStatus",
"properties" : {
"id" : {
"type" : "any"
},
"status" : {
"type" : "string",
"enum" : [ "CREATED", "RUNNING", "FAILING", "FAILED", "CANCELLING", "CANCELED", "FINISHED", "RESTARTING", "SUSPENDING", "SUSPENDED", "RECONCILING" ]
}
}
}
}
}
}
/jobs
动作: POST
响应代码: 202 Accepted
提交工作。此调用主要供Flink客户端使用。此调用需要一个multipart / form-data请求,其中包含序列化JobGraph,jar和分布式缓存工件的文件上载以及JSON有效负载的名为“request”的属性。
请求:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:JobSubmitRequestBody",
"properties" : {
"jobGraphFileName" : {
"type" : "string"
},
"jobJarFileNames" : {
"type" : "array",
"items" : {
"type" : "string"
}
},
"jobArtifactFileNames" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:JobSubmitRequestBody:DistributedCacheFile",
"properties" : {
"entryName" : {
"type" : "string"
},
"fileName" : {
"type" : "string"
}
}
}
}
}
}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:JobSubmitResponseBody",
"properties" : {
"jobUrl" : {
"type" : "string"
}
}
}
/jobs/metrics
动作: GET
响应代码: 200 OK
提供对聚合作业指标的访问。
查询参数
get
(可选):以逗号分隔的字符串值列表,用于选择特定指标。agg
(可选):应该计算的以逗号分隔的聚合模式列表。可用的聚合是:“min,max,sum,avg”。jobs
(可选):以逗号分隔的32个字符的十六进制字符串列表,用于选择特定的作业。
请求:
{}
响应:
{
"type" : "any"
}
/jobs/overview
动作: GET
响应代码: 200 OK
返回所有作业的概述。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:messages:webmonitor:MultipleJobsDetails",
"properties" : {
"jobs" : {
"type" : "array",
"items" : {
"type" : "any"
}
}
}
}
/jobs/:jobid
动作: GET
响应代码: 200 OK
返回作业的详细信息。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:JobDetailsInfo",
"properties" : {
"jid" : {
"type" : "any"
},
"name" : {
"type" : "string"
},
"isStoppable" : {
"type" : "boolean"
},
"state" : {
"type" : "string",
"enum" : [ "CREATED", "RUNNING", "FAILING", "FAILED", "CANCELLING", "CANCELED", "FINISHED", "RESTARTING", "SUSPENDING", "SUSPENDED", "RECONCILING" ]
},
"start-time" : {
"type" : "integer"
},
"end-time" : {
"type" : "integer"
},
"duration" : {
"type" : "integer"
},
"now" : {
"type" : "integer"
},
"timestamps" : {
"type" : "object",
"additionalProperties" : {
"type" : "integer"
}
},
"vertices" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:JobDetailsInfo:JobVertexDetailsInfo",
"properties" : {
"id" : {
"type" : "any"
},
"name" : {
"type" : "string"
},
"parallelism" : {
"type" : "integer"
},
"status" : {
"type" : "string",
"enum" : [ "CREATED", "SCHEDULED", "DEPLOYING", "RUNNING", "FINISHED", "CANCELING", "CANCELED", "FAILED", "RECONCILING" ]
},
"start-time" : {
"type" : "integer"
},
"end-time" : {
"type" : "integer"
},
"duration" : {
"type" : "integer"
},
"tasks" : {
"type" : "object",
"additionalProperties" : {
"type" : "integer"
}
},
"metrics" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:metrics:IOMetricsInfo",
"properties" : {
"read-bytes" : {
"type" : "integer"
},
"read-bytes-complete" : {
"type" : "boolean"
},
"write-bytes" : {
"type" : "integer"
},
"write-bytes-complete" : {
"type" : "boolean"
},
"read-records" : {
"type" : "integer"
},
"read-records-complete" : {
"type" : "boolean"
},
"write-records" : {
"type" : "integer"
},
"write-records-complete" : {
"type" : "boolean"
}
}
}
}
}
},
"status-counts" : {
"type" : "object",
"additionalProperties" : {
"type" : "integer"
}
},
"plan" : {
"type" : "string"
}
}
}
/jobs/:jobid
动作: PATCH
响应代码: 202 Accepted
终止工作。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。
查询参数:
mode
(可选):指定终止模式的字符串值。支持的值包括:“取消,停止”。
请求:
{}
响应:
{}
/jobs/:jobid /accumulators
动作: GET
响应代码: 200 OK
返回作业的所有任务的累加器,汇总在各个子任务中。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。
查询参数:
includeSerializedValue
(可选):布尔值,指定序列化用户任务累加器是否应包含在响应中。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobAccumulatorsInfo",
"properties" : {
"job-accumulators" : {
"type" : "array",
"items" : {
"type" : "any"
}
},
"user-task-accumulators" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobAccumulatorsInfo:UserTaskAccumulator",
"properties" : {
"name" : {
"type" : "string"
},
"type" : {
"type" : "string"
},
"value" : {
"type" : "string"
}
}
}
},
"serialized-user-task-accumulators" : {
"type" : "object",
"additionalProperties" : {
"type" : "any"
}
}
}
}
/jobs/:jobid /checkpoints
动作: GET
响应代码: 200 OK
返回作业的检查点统计信息。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointingStatistics",
"properties" : {
"counts" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointingStatistics:Counts",
"properties" : {
"restored" : {
"type" : "integer"
},
"total" : {
"type" : "integer"
},
"in_progress" : {
"type" : "integer"
},
"completed" : {
"type" : "integer"
},
"failed" : {
"type" : "integer"
}
}
},
"summary" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointingStatistics:Summary",
"properties" : {
"state_size" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics",
"properties" : {
"min" : {
"type" : "integer"
},
"max" : {
"type" : "integer"
},
"avg" : {
"type" : "integer"
}
}
},
"end_to_end_duration" : {
"type" : "object",
"$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics"
},
"alignment_buffered" : {
"type" : "object",
"$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics"
}
}
},
"latest" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointingStatistics:LatestCheckpoints",
"properties" : {
"completed" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointStatistics:CompletedCheckpointStatistics",
"properties" : {
"id" : {
"type" : "integer"
},
"status" : {
"type" : "string",
"enum" : [ "IN_PROGRESS", "COMPLETED", "FAILED" ]
},
"is_savepoint" : {
"type" : "boolean"
},
"trigger_timestamp" : {
"type" : "integer"
},
"latest_ack_timestamp" : {
"type" : "integer"
},
"state_size" : {
"type" : "integer"
},
"end_to_end_duration" : {
"type" : "integer"
},
"alignment_buffered" : {
"type" : "integer"
},
"num_subtasks" : {
"type" : "integer"
},
"num_acknowledged_subtasks" : {
"type" : "integer"
},
"tasks" : {
"type" : "object",
"additionalProperties" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:TaskCheckpointStatistics",
"properties" : {
"id" : {
"type" : "integer"
},
"status" : {
"type" : "string",
"enum" : [ "IN_PROGRESS", "COMPLETED", "FAILED" ]
},
"latest_ack_timestamp" : {
"type" : "integer"
},
"state_size" : {
"type" : "integer"
},
"end_to_end_duration" : {
"type" : "integer"
},
"alignment_buffered" : {
"type" : "integer"
},
"num_subtasks" : {
"type" : "integer"
},
"num_acknowledged_subtasks" : {
"type" : "integer"
}
}
}
},
"external_path" : {
"type" : "string"
},
"discarded" : {
"type" : "boolean"
}
}
},
"savepoint" : {
"type" : "object",
"$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointStatistics:CompletedCheckpointStatistics"
},
"failed" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointStatistics:FailedCheckpointStatistics",
"properties" : {
"id" : {
"type" : "integer"
},
"status" : {
"type" : "string",
"enum" : [ "IN_PROGRESS", "COMPLETED", "FAILED" ]
},
"is_savepoint" : {
"type" : "boolean"
},
"trigger_timestamp" : {
"type" : "integer"
},
"latest_ack_timestamp" : {
"type" : "integer"
},
"state_size" : {
"type" : "integer"
},
"end_to_end_duration" : {
"type" : "integer"
},
"alignment_buffered" : {
"type" : "integer"
},
"num_subtasks" : {
"type" : "integer"
},
"num_acknowledged_subtasks" : {
"type" : "integer"
},
"tasks" : {
"type" : "object",
"additionalProperties" : {
"type" : "object",
"$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:TaskCheckpointStatistics"
}
},
"failure_timestamp" : {
"type" : "integer"
},
"failure_message" : {
"type" : "string"
}
}
},
"restored" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointingStatistics:RestoredCheckpointStatistics",
"properties" : {
"id" : {
"type" : "integer"
},
"restore_timestamp" : {
"type" : "integer"
},
"is_savepoint" : {
"type" : "boolean"
},
"external_path" : {
"type" : "string"
}
}
}
}
},
"history" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointStatistics",
"properties" : {
"id" : {
"type" : "integer"
},
"status" : {
"type" : "string",
"enum" : [ "IN_PROGRESS", "COMPLETED", "FAILED" ]
},
"is_savepoint" : {
"type" : "boolean"
},
"trigger_timestamp" : {
"type" : "integer"
},
"latest_ack_timestamp" : {
"type" : "integer"
},
"state_size" : {
"type" : "integer"
},
"end_to_end_duration" : {
"type" : "integer"
},
"alignment_buffered" : {
"type" : "integer"
},
"num_subtasks" : {
"type" : "integer"
},
"num_acknowledged_subtasks" : {
"type" : "integer"
},
"tasks" : {
"type" : "object",
"additionalProperties" : {
"type" : "object",
"$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:TaskCheckpointStatistics"
}
}
}
}
}
}
}
/jobs/:jobid /checkpoints/config
动作: GET
响应代码: 200 OK
返回检查点配置。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointConfigInfo",
"properties" : {
"mode" : {
"type" : "any"
},
"interval" : {
"type" : "integer"
},
"timeout" : {
"type" : "integer"
},
"min_pause" : {
"type" : "integer"
},
"max_concurrent" : {
"type" : "integer"
},
"externalization" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointConfigInfo:ExternalizedCheckpointInfo",
"properties" : {
"enabled" : {
"type" : "boolean"
},
"delete_on_cancellation" : {
"type" : "boolean"
}
}
}
}
}
/jobs/:jobid /checkpoints/details/:checkpointid
动作: GET
响应代码: 200 OK
返回检查点的详细信息。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。checkpointid
- 标识检查点的长值。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointStatistics",
"properties" : {
"id" : {
"type" : "integer"
},
"status" : {
"type" : "string",
"enum" : [ "IN_PROGRESS", "COMPLETED", "FAILED" ]
},
"is_savepoint" : {
"type" : "boolean"
},
"trigger_timestamp" : {
"type" : "integer"
},
"latest_ack_timestamp" : {
"type" : "integer"
},
"state_size" : {
"type" : "integer"
},
"end_to_end_duration" : {
"type" : "integer"
},
"alignment_buffered" : {
"type" : "integer"
},
"num_subtasks" : {
"type" : "integer"
},
"num_acknowledged_subtasks" : {
"type" : "integer"
},
"tasks" : {
"type" : "object",
"additionalProperties" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:TaskCheckpointStatistics",
"properties" : {
"id" : {
"type" : "integer"
},
"status" : {
"type" : "string",
"enum" : [ "IN_PROGRESS", "COMPLETED", "FAILED" ]
},
"latest_ack_timestamp" : {
"type" : "integer"
},
"state_size" : {
"type" : "integer"
},
"end_to_end_duration" : {
"type" : "integer"
},
"alignment_buffered" : {
"type" : "integer"
},
"num_subtasks" : {
"type" : "integer"
},
"num_acknowledged_subtasks" : {
"type" : "integer"
}
}
}
}
}
}
/jobs/:jobid /checkpoints/details/:checkpointid /subtasks/:vertexid
动作: GET
响应代码: 200 OK
返回任务及其子任务的检查点统计信息。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。checkpointid
- 标识检查点的长值。vertexid
- 标识作业顶点的32个字符的十六进制字符串值。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:TaskCheckpointStatisticsWithSubtaskDetails",
"properties" : {
"id" : {
"type" : "integer"
},
"status" : {
"type" : "string",
"enum" : [ "IN_PROGRESS", "COMPLETED", "FAILED" ]
},
"latest_ack_timestamp" : {
"type" : "integer"
},
"state_size" : {
"type" : "integer"
},
"end_to_end_duration" : {
"type" : "integer"
},
"alignment_buffered" : {
"type" : "integer"
},
"num_subtasks" : {
"type" : "integer"
},
"num_acknowledged_subtasks" : {
"type" : "integer"
},
"summary" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:TaskCheckpointStatisticsWithSubtaskDetails:Summary",
"properties" : {
"state_size" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics",
"properties" : {
"min" : {
"type" : "integer"
},
"max" : {
"type" : "integer"
},
"avg" : {
"type" : "integer"
}
}
},
"end_to_end_duration" : {
"type" : "object",
"$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics"
},
"checkpoint_duration" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:TaskCheckpointStatisticsWithSubtaskDetails:CheckpointDuration",
"properties" : {
"sync" : {
"type" : "object",
"$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics"
},
"async" : {
"type" : "object",
"$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics"
}
}
},
"alignment" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:TaskCheckpointStatisticsWithSubtaskDetails:CheckpointAlignment",
"properties" : {
"buffered" : {
"type" : "object",
"$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics"
},
"duration" : {
"type" : "object",
"$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics"
}
}
}
}
},
"subtasks" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:SubtaskCheckpointStatistics",
"properties" : {
"index" : {
"type" : "integer"
},
"status" : {
"type" : "string"
}
}
}
}
}
}
/jobs/:jobid /config
动作: GET
响应代码: 200 OK
返回作业的配置。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。
请求:
{}
响应:
{
"type" : "any"
}
/jobs/:jobid /exceptions
动作: GET
响应代码: 200 OK
返回作业已观察到的不可恢复的异常。截断标志定义是否发生了更多异常,但未列出,因为否则响应会变得太大。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobExceptionsInfo",
"properties" : {
"root-exception" : {
"type" : "string"
},
"timestamp" : {
"type" : "integer"
},
"all-exceptions" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobExceptionsInfo:ExecutionExceptionInfo",
"properties" : {
"exception" : {
"type" : "string"
},
"task" : {
"type" : "string"
},
"location" : {
"type" : "string"
},
"timestamp" : {
"type" : "integer"
}
}
}
},
"truncated" : {
"type" : "boolean"
}
}
}
/jobs/:jobid /execution-result
动作: GET
响应代码: 200 OK
返回作业执行的结果。允许访问作业的执行时间以及此作业创建的所有累加器。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:JobExecutionResultResponseBody",
"properties" : {
"status" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:queue:QueueStatus",
"required" : true,
"properties" : {
"id" : {
"type" : "string",
"required" : true,
"enum" : [ "IN_PROGRESS", "COMPLETED" ]
}
}
},
"job-execution-result" : {
"type" : "any"
}
}
}
/jobs/:jobid /metrics
动作: GET
响应代码: 200 OK
提供对作业指标的访问。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。
查询参数:
get
(可选):以逗号分隔的字符串值列表,用于选择特定指标。
请求:
{}
响应:
{
"type" : "any"
}
/jobs/:jobid /plan
动作: GET
响应代码: 200 OK
返回作业的数据流计划。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobPlanInfo",
"properties" : {
"plan" : {
"type" : "any"
}
}
}
/jobs/:jobid /rescaling
动作: PATCH
响应代码: 200 OK
触发重新调整作业。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。
查询参数:
parallelism
(强制项):正整数值,指定所需的并行度。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:handler:async:TriggerResponse",
"properties" : {
"request-id" : {
"type" : "any"
}
}
}
/jobs/:jobid /rescaling/:triggerid
动作: GET
响应代码: 200 OK
返回重新缩放 算子操作的状态。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。triggerid
- 32个字符的十六进制字符串,用于标识异步 算子操作触发器ID。返回ID然后触发 算子操作。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:handler:async:AsynchronousOperationResult",
"properties" : {
"status" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:queue:QueueStatus",
"properties" : {
"id" : {
"type" : "string",
"required" : true,
"enum" : [ "IN_PROGRESS", "COMPLETED" ]
}
}
},
"operation" : {
"type" : "any"
}
}
}
/jobs/:jobid /savepoints
动作: POST
响应代码: 202 Accepted
触发保存点,然后可选择取消作业。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。
请求:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:savepoints:SavepointTriggerRequestBody",
"properties" : {
"target-directory" : {
"type" : "string"
},
"cancel-job" : {
"type" : "boolean"
}
}
}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:handler:async:TriggerResponse",
"properties" : {
"request-id" : {
"type" : "any"
}
}
}
/jobs/:jobid /savepoints/:triggerid
动作: GET
响应代码: 200 OK
返回保存点 算子操作的状态。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。triggerid
- 32个字符的十六进制字符串,用于标识异步 算子操作触发器ID。返回ID然后触发 算子操作。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:handler:async:AsynchronousOperationResult",
"properties" : {
"status" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:queue:QueueStatus",
"properties" : {
"id" : {
"type" : "string",
"required" : true,
"enum" : [ "IN_PROGRESS", "COMPLETED" ]
}
}
},
"operation" : {
"type" : "any"
}
}
}
/jobs/:jobid /vertices/:vertexid
动作: GET
响应代码: 200 OK
返回任务的详细信息,并为每个子任务提供摘要。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。vertexid
- 标识作业顶点的32个字符的十六进制字符串值。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobVertexDetailsInfo",
"properties" : {
"id" : {
"type" : "any"
},
"name" : {
"type" : "string"
},
"parallelism" : {
"type" : "integer"
},
"now" : {
"type" : "integer"
},
"subtasks" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobVertexDetailsInfo:VertexTaskDetail",
"properties" : {
"subtask" : {
"type" : "integer"
},
"status" : {
"type" : "string",
"enum" : [ "CREATED", "SCHEDULED", "DEPLOYING", "RUNNING", "FINISHED", "CANCELING", "CANCELED", "FAILED", "RECONCILING" ]
},
"attempt" : {
"type" : "integer"
},
"host" : {
"type" : "string"
},
"start_time" : {
"type" : "integer"
},
"end-time" : {
"type" : "integer"
},
"duration" : {
"type" : "integer"
},
"metrics" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:metrics:IOMetricsInfo",
"properties" : {
"read-bytes" : {
"type" : "integer"
},
"read-bytes-complete" : {
"type" : "boolean"
},
"write-bytes" : {
"type" : "integer"
},
"write-bytes-complete" : {
"type" : "boolean"
},
"read-records" : {
"type" : "integer"
},
"read-records-complete" : {
"type" : "boolean"
},
"write-records" : {
"type" : "integer"
},
"write-records-complete" : {
"type" : "boolean"
}
}
}
}
}
}
}
}
/jobs/:jobid /vertices/:vertexid /accumulators
动作: GET
响应代码: 200 OK
返回在所有子任务中聚合的任务的用户定义累加器。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。vertexid
- 标识作业顶点的32个字符的十六进制字符串值。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobVertexAccumulatorsInfo",
"properties" : {
"id" : {
"type" : "string"
},
"user-accumulators" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:UserAccumulator",
"properties" : {
"name" : {
"type" : "string"
},
"type" : {
"type" : "string"
},
"value" : {
"type" : "string"
}
}
}
}
}
}
/jobs/:jobid /vertices/:vertexid /backpressure
动作: GET
响应代码: 200 OK
返回作业的背压信息,并在必要时启动背压采样。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。vertexid
- 标识作业顶点的32个字符的十六进制字符串值。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobVertexBackPressureInfo",
"properties" : {
"status" : {
"type" : "string",
"enum" : [ "deprecated", "ok" ]
},
"backpressure-level" : {
"type" : "string",
"enum" : [ "ok", "low", "high" ]
},
"end-timestamp" : {
"type" : "integer"
},
"subtasks" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobVertexBackPressureInfo:SubtaskBackPressureInfo",
"properties" : {
"subtask" : {
"type" : "integer"
},
"backpressure-level" : {
"type" : "string",
"enum" : [ "ok", "low", "high" ]
},
"ratio" : {
"type" : "number"
}
}
}
}
}
}
/jobs/:jobid /vertices/:vertexid /metrics
动作: GET
响应代码: 200 OK
提供对任务指标的访问。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。vertexid
- 标识作业顶点的32个字符的十六进制字符串值。
查询参数:
get
(可选):以逗号分隔的字符串值列表,用于选择特定指标。
请求:
{}
响应:
{
"type" : "any"
}
/jobs/:jobid /vertices/:vertexid /subtasks/accumulators
动作: GET
响应代码: 200 OK
返回任务的所有子任务的所有用户定义的累加器。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。vertexid
- 标识作业顶点的32个字符的十六进制字符串值。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:SubtasksAllAccumulatorsInfo",
"properties" : {
"id" : {
"type" : "any"
},
"parallelism" : {
"type" : "integer"
},
"subtasks" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:SubtasksAllAccumulatorsInfo:SubtaskAccumulatorsInfo",
"properties" : {
"subtask" : {
"type" : "integer"
},
"attempt" : {
"type" : "integer"
},
"host" : {
"type" : "string"
},
"user-accumulators" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:UserAccumulator",
"properties" : {
"name" : {
"type" : "string"
},
"type" : {
"type" : "string"
},
"value" : {
"type" : "string"
}
}
}
}
}
}
}
}
}
/jobs/:jobid /vertices/:vertexid /subtasks/metrics
动作: GET
响应代码: 200 OK
提供对聚合子任务度量标准的访问。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。vertexid
- 标识作业顶点的32个字符的十六进制字符串值。
查询参数:
get
(可选):以逗号分隔的字符串值列表,用于选择特定指标。agg
(可选):应该计算的以逗号分隔的聚合模式列表。可用的聚合是:“min,max,sum,avg”。subtasks
(可选):以逗号分隔的整数范围列表(例如“1,3,5-9”)以选择特定的子任务。
请求:
{}
响应:
{
"type" : "any"
}
/jobs/:jobid /vertices/:vertexid /subtasks/:subtaskindex
动作: GET
响应代码: 200 OK
返回子任务的当前或最新执行尝试的详细信息。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。vertexid
- 标识作业顶点的32个字符的十六进制字符串值。subtaskindex
- 标识子任务的正整数值。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:SubtaskExecutionAttemptDetailsInfo",
"properties" : {
"subtask" : {
"type" : "integer"
},
"status" : {
"type" : "string",
"enum" : [ "CREATED", "SCHEDULED", "DEPLOYING", "RUNNING", "FINISHED", "CANCELING", "CANCELED", "FAILED", "RECONCILING" ]
},
"attempt" : {
"type" : "integer"
},
"host" : {
"type" : "string"
},
"start-time" : {
"type" : "integer"
},
"end-time" : {
"type" : "integer"
},
"duration" : {
"type" : "integer"
},
"metrics" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:metrics:IOMetricsInfo",
"properties" : {
"read-bytes" : {
"type" : "integer"
},
"read-bytes-complete" : {
"type" : "boolean"
},
"write-bytes" : {
"type" : "integer"
},
"write-bytes-complete" : {
"type" : "boolean"
},
"read-records" : {
"type" : "integer"
},
"read-records-complete" : {
"type" : "boolean"
},
"write-records" : {
"type" : "integer"
},
"write-records-complete" : {
"type" : "boolean"
}
}
}
}
}
/jobs/:jobid /vertices/:vertexid /subtasks/:subtaskindex /attempts/:attempt
动作: GET
响应代码: 200 OK
返回子任务执行尝试的详细信息。在发生故障/恢复时会发生多次执行尝试。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。vertexid
- 标识作业顶点的32个字符的十六进制字符串值。subtaskindex
- 标识子任务的正整数值。attempt
- 标识执行尝试的正整数值。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:SubtaskExecutionAttemptDetailsInfo",
"properties" : {
"subtask" : {
"type" : "integer"
},
"status" : {
"type" : "string",
"enum" : [ "CREATED", "SCHEDULED", "DEPLOYING", "RUNNING", "FINISHED", "CANCELING", "CANCELED", "FAILED", "RECONCILING" ]
},
"attempt" : {
"type" : "integer"
},
"host" : {
"type" : "string"
},
"start-time" : {
"type" : "integer"
},
"end-time" : {
"type" : "integer"
},
"duration" : {
"type" : "integer"
},
"metrics" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:metrics:IOMetricsInfo",
"properties" : {
"read-bytes" : {
"type" : "integer"
},
"read-bytes-complete" : {
"type" : "boolean"
},
"write-bytes" : {
"type" : "integer"
},
"write-bytes-complete" : {
"type" : "boolean"
},
"read-records" : {
"type" : "integer"
},
"read-records-complete" : {
"type" : "boolean"
},
"write-records" : {
"type" : "integer"
},
"write-records-complete" : {
"type" : "boolean"
}
}
}
}
}
/jobs/:jobid /vertices/:vertexid /subtasks/:subtaskindex /attempts/:attempt/accumulators
动作: GET
响应代码: 200 OK
返回子任务执行尝试的累加器。在发生故障/恢复时会发生多次执行尝试。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。vertexid
- 标识作业顶点的32个字符的十六进制字符串值。subtaskindex
- 标识子任务的正整数值。attempt
- 标识执行尝试的正整数值。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:SubtaskExecutionAttemptAccumulatorsInfo",
"properties" : {
"subtask" : {
"type" : "integer"
},
"attempt" : {
"type" : "integer"
},
"id" : {
"type" : "string"
},
"user-accumulators" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:UserAccumulator",
"properties" : {
"name" : {
"type" : "string"
},
"type" : {
"type" : "string"
},
"value" : {
"type" : "string"
}
}
}
}
}
}
/jobs/:jobid /vertices/:vertexid /subtasks/:subtaskindex /metrics
动作: GET
响应代码: 200 OK
提供对子任务度量标准的访问。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。vertexid
- 标识作业顶点的32个字符的十六进制字符串值。subtaskindex
- 标识子任务的正整数值。
查询参数:
get
(可选):以逗号分隔的字符串值列表,用于选择特定指标。
请求:
{}
响应:
{
"type" : "any"
}
/jobs/:jobid /vertices/:vertexid / subtasktimes
动作: GET
响应代码: 200 OK
返回任务的所有子任务的时间相关信息。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。vertexid
- 标识作业顶点的32个字符的十六进制字符串值。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:SubtasksTimesInfo",
"properties" : {
"id" : {
"type" : "string"
},
"name" : {
"type" : "string"
},
"now" : {
"type" : "integer"
},
"subtasks" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:SubtasksTimesInfo:SubtaskTimeInfo",
"properties" : {
"subtask" : {
"type" : "integer"
},
"host" : {
"type" : "string"
},
"duration" : {
"type" : "integer"
},
"timestamps" : {
"type" : "object",
"additionalProperties" : {
"type" : "integer"
}
}
}
}
}
}
}
/jobs/:jobid /vertices/:vertexid / taskmanagers
动作: GET
响应代码: 200 OK
返回TaskManager聚合的任务信息。
路径参数:
jobid
- 标识作业的32个字符的十六进制字符串值。vertexid
- 标识作业顶点的32个字符的十六进制字符串值。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobVertexTaskManagersInfo",
"properties" : {
"id" : {
"type" : "any"
},
"name" : {
"type" : "string"
},
"now" : {
"type" : "integer"
},
"taskmanagers" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobVertexTaskManagersInfo:TaskManagersInfo",
"properties" : {
"host" : {
"type" : "string"
},
"status" : {
"type" : "string",
"enum" : [ "CREATED", "SCHEDULED", "DEPLOYING", "RUNNING", "FINISHED", "CANCELING", "CANCELED", "FAILED", "RECONCILING" ]
},
"start-time" : {
"type" : "integer"
},
"end-time" : {
"type" : "integer"
},
"duration" : {
"type" : "integer"
},
"metrics" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:metrics:IOMetricsInfo",
"properties" : {
"read-bytes" : {
"type" : "integer"
},
"read-bytes-complete" : {
"type" : "boolean"
},
"write-bytes" : {
"type" : "integer"
},
"write-bytes-complete" : {
"type" : "boolean"
},
"read-records" : {
"type" : "integer"
},
"read-records-complete" : {
"type" : "boolean"
},
"write-records" : {
"type" : "integer"
},
"write-records-complete" : {
"type" : "boolean"
}
}
},
"status-counts" : {
"type" : "object",
"additionalProperties" : {
"type" : "integer"
}
}
}
}
}
}
}
/overview
动作: GET
响应代码: 200 OK
返回Flink集群的概述。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:handler:legacy:messages:ClusterOverviewWithVersion",
"properties" : {
"taskmanagers" : {
"type" : "integer"
},
"slots-total" : {
"type" : "integer"
},
"slots-available" : {
"type" : "integer"
},
"jobs-running" : {
"type" : "integer"
},
"jobs-finished" : {
"type" : "integer"
},
"jobs-cancelled" : {
"type" : "integer"
},
"jobs-failed" : {
"type" : "integer"
},
"flink-version" : {
"type" : "string"
},
"flink-commit" : {
"type" : "string"
}
}
}
/savepoint-disposal
动作: POST
响应代码: 200 OK
触发保存点的废弃处理。
请求:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:savepoints:SavepointDisposalRequest",
"properties" : {
"savepoint-path" : {
"type" : "string"
}
}
}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:handler:async:TriggerResponse",
"properties" : {
"request-id" : {
"type" : "any"
}
}
}
/savepoint-disposal/:triggerid
动作: GET
响应代码: 200 OK
返回保存点处理 算子操作的状态。
路径参数:
triggerid
- 32个字符的十六进制字符串,用于标识异步 算子操作触发器ID。返回ID然后触发 算子操作。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:handler:async:AsynchronousOperationResult",
"properties" : {
"status" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:queue:QueueStatus",
"properties" : {
"id" : {
"type" : "string",
"required" : true,
"enum" : [ "IN_PROGRESS", "COMPLETED" ]
}
}
},
"operation" : {
"type" : "any"
}
}
}
/ taskmanagers
动作: GET
响应代码: 200 OK
返回所有TaskManager的概述。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:taskmanager:TaskManagersInfo",
"properties" : {
"taskmanagers" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:taskmanager:TaskManagerInfo",
"properties" : {
"id" : {
"type" : "any"
},
"path" : {
"type" : "string"
},
"dataPort" : {
"type" : "integer"
},
"timeSinceLastHeartbeat" : {
"type" : "integer"
},
"slotsNumber" : {
"type" : "integer"
},
"freeSlots" : {
"type" : "integer"
},
"hardware" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:instance:HardwareDescription",
"properties" : {
"cpuCores" : {
"type" : "integer"
},
"physicalMemory" : {
"type" : "integer"
},
"freeMemory" : {
"type" : "integer"
},
"managedMemory" : {
"type" : "integer"
}
}
}
}
}
}
}
}
/ taskmanagers /metrics
动作: GET
响应代码: 200 OK
提供对聚合TaskManager指标的访问。
查询参数
get
(可选):以逗号分隔的字符串值列表,用于选择特定指标。agg
(可选):应该计算的以逗号分隔的聚合模式列表。可用的聚合是:“min,max,sum,avg”。taskmanagers
(可选):以逗号分隔的32个字符的十六进制字符串列表,用于选择特定的TaskManager。
请求:
{}
响应:
{
"type" : "any"
}
/ taskmanagers /:taskmanagerid
动作: GET
响应代码: 200 OK
返回TaskManager的详细信息。
路径参数:
taskmanagerid
- 标识TaskManager的32个字符的十六进制字符串。
请求:
{}
响应:
{
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:taskmanager:TaskManagerDetailsInfo",
"properties" : {
"id" : {
"type" : "any"
},
"path" : {
"type" : "string"
},
"dataPort" : {
"type" : "integer"
},
"timeSinceLastHeartbeat" : {
"type" : "integer"
},
"slotsNumber" : {
"type" : "integer"
},
"freeSlots" : {
"type" : "integer"
},
"hardware" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:instance:HardwareDescription",
"properties" : {
"cpuCores" : {
"type" : "integer"
},
"physicalMemory" : {
"type" : "integer"
},
"freeMemory" : {
"type" : "integer"
},
"managedMemory" : {
"type" : "integer"
}
}
},
"metrics" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:taskmanager:TaskManagerMetricsInfo",
"properties" : {
"heapUsed" : {
"type" : "integer"
},
"heapCommitted" : {
"type" : "integer"
},
"heapMax" : {
"type" : "integer"
},
"nonHeapUsed" : {
"type" : "integer"
},
"nonHeapCommitted" : {
"type" : "integer"
},
"nonHeapMax" : {
"type" : "integer"
},
"directCount" : {
"type" : "integer"
},
"directUsed" : {
"type" : "integer"
},
"directMax" : {
"type" : "integer"
},
"mappedCount" : {
"type" : "integer"
},
"mappedUsed" : {
"type" : "integer"
},
"mappedMax" : {
"type" : "integer"
},
"memorySegmentsAvailable" : {
"type" : "integer"
},
"memorySegmentsTotal" : {
"type" : "integer"
},
"garbageCollectors" : {
"type" : "array",
"items" : {
"type" : "object",
"id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:taskmanager:TaskManagerMetricsInfo:GarbageCollectorInfo",
"properties" : {
"name" : {
"type" : "string"
},
"count" : {
"type" : "integer"
},
"time" : {
"type" : "integer"
}
}
}
}
}
}
}
}
/ taskmanagers /:taskmanagerid /metrics
动作: GET
响应代码: 200 OK
提供对TaskManager指标的访问。
路径参数:
taskmanagerid
- 标识TaskManager的32个字符的十六进制字符串。
查询参数:
get
(可选):以逗号分隔的字符串值列表,用于选择特定指标。
请求:
{}
响应:
{
"type" : "any"
}
仅当群集在传统模式下运行时,此部分才相关。
以下是可用请求列表,其中包含示例JSON响应。所有请求都是样本表单http://hostname:8081/jobs
,下面我们仅列出URL 的_路径_部分。
尖括号中的值是变量,例如http://hostname:8081/jobs/<jobid>/exceptions
必须请求例如http://hostname:8081/jobs/7684be6004e4e955c2a558a9bc463f65/exceptions
。
/config
/overview
/jobs/overview
/jobs/<jobid>
/jobs/<jobid>/vertices
/jobs/<jobid>/config
/jobs/<jobid>/exceptions
/jobs/<jobid>/accumulators
/jobs/<jobid>/vertices/<vertexid>
/jobs/<jobid>/vertices/<vertexid>/subtasktimes
/jobs/<jobid>/vertices/<vertexid>/taskmanagers
/jobs/<jobid>/vertices/<vertexid>/accumulators
/jobs/<jobid>/vertices/<vertexid>/subtasks/accumulators
/jobs/<jobid>/vertices/<vertexid>/subtasks/<subtasknum>
/jobs/<jobid>/vertices/<vertexid>/subtasks/<subtasknum>/attempts/<attempt>
/jobs/<jobid>/vertices/<vertexid>/subtasks/<subtasknum>/attempts/<attempt>/accumulators
/jobs/<jobid>/plan
/jars/upload
/jars
/jars/:jarid
/jars/:jarid/plan
/jars/:jarid/run
一般
/config
有关监视API和服务器设置的一些信息。
样本结果:
{ "refresh-interval": 3000, "timezone-offset": 3600000, "timezone-name": "Central European Time", "flink-version": "1.7-SNAPSHOT", "flink-revision": "8124545 @ 16.09.2015 @ 15:38:42 CEST" }
/overview
Flink集群状态的简单摘要。
样本结果:
{ "taskmanagers": 17, "slots-total": 68, "slots-available": 68, "jobs-running": 0, "jobs-finished": 3, "jobs-cancelled": 1, "jobs-failed": 0 }
工作概览
/jobs/overview
使用小摘要概述所有作业。
样本结果:
{ "jobs":[ { "jid": "7684be6004e4e955c2a558a9bc463f65", "name": "Flink Java Job at Wed Sep 16 18:08:21 CEST 2015", "state": "FINISHED", "start-time": 1442419702857, "end-time": 1442419975312, "duration":272455, "last-modification": 1442419975312, "tasks": { "total": 6, "pending": 0, "running": 0, "finished": 6, "canceling": 0, "canceled": 0, "failed": 0 } }, { "jid": "49306f94d0920216b636e8dd503a6409", "name": "Flink Java Job at Wed Sep 16 18:16:39 CEST 2015", ... }] }
正在运行或已完成的工作的详细信息
/jobs/<jobid>
一个作业的摘要,列出数据流计划,状态,状态转换的时间戳,每个顶点( 算子)的聚合信息。
样本结果:
{ "jid": "ab78dcdbb1db025539e30217ec54ee16", "name": "WordCount Example", "state":"FINISHED", "start-time":1442421277536, "end-time":1442421299791, "duration":22255, "now":1442421991768, "timestamps": { "CREATED": 1442421277536, "RUNNING": 1442421277609, "FAILING": 0, "FAILED": 0, "CANCELLING": 0, "CANCELED": 0, "FINISHED": 1442421299791, "RESTARTING": 0 }, "vertices": [ { "id": "19b5b24062c48a06e4eac65422ac3317", "name": "CHAIN DataSource (at getTextDataSet(WordCount.java:142) ...", "parallelism": 2, "status": "FINISHED", "start-time": 1442421277609, "end-time": 1442421299469, "duration": 21860, "tasks": { "CREATED": 0, "SCHEDULED": 0, "DEPLOYING": 0, "RUNNING": 0, "FINISHED": 2, "CANCELING": 0, "CANCELED": 0, "FAILED": 0 }, "metrics": { "read-bytes": 0, "write-bytes": 37098, "read-records": 0, "write-records": 3312 } }, { "id": "f00c89b349b5c998cfd9fe2a06e50fd0", "name":"Reduce (SUM(1), at main(WordCount.java:67)", "parallelism": 2, .... }, { "id": "0a36cbc29102d7bc993d0a9bf23afa12", "name": "DataSink (CsvOutputFormat (path: /tmp/abzs, delimiter: ))", ... } ], "status-counts": { "CREATED": 0, "SCHEDULED": 0, "DEPLOYING": 0, "RUNNING": 0, "FINISHED": 3, "CANCELING": 0, "CANCELED": 0, "FAILED": 0 }, "plan": { // see plan details below } }
/jobs/<jobid>/vertices
目前一样 /jobs/<jobid>
/jobs/<jobid>/config
作业使用的用户定义的执行配置。
样本结果:
{ "jid": "ab78dcdbb1db025539e30217ec54ee16", "name": "WordCount Example", "execution-config": { "execution-mode": "PIPELINED", "restart-strategy": "Restart deactivated", "job-parallelism": -1, "object-reuse-mode": false } }
/jobs/<jobid>/exceptions
工作中观察到的不可恢复的异常。该truncated
标志定义是否发生了更多异常,但未列出,因为否则响应会变得太大。
样本结果:
{ "root-exception": "java.io.IOException: File already exists:/tmp/abzs/2\n\tat org.apache.flink.core.fs.local.LocalFileSystem. ...", "all-exceptions": [ { "exception": "java.io.IOException: File already exists:/tmp/abzs/1\n\tat org.apache.flink...", "task": "DataSink (CsvOutputFormat (path: /tmp/abzs, delimiter: )) (1/2)", "location": "localhost:49220" }, { "exception": "java.io.IOException: File already exists:/tmp/abzs/2\n\tat org.apache.flink...", "task": "DataSink (CsvOutputFormat (path: /tmp/abzs, delimiter: )) (2/2)", "location": "localhost:49220" } ], "truncated":false }
/jobs/<jobid>/accumulators
聚合的用户累加器加上作业累加器。
样本结果:
{ "job-accumulators":[], "user-task-accumulators": [ { "name": "avglen", "type": "DoubleCounter", "value": "DoubleCounter 61.5162972" }, { "name": "genwords", "type": "LongCounter", "value": "LongCounter 37500000" } ] }
/jobs/<jobid>/vertices/<vertexid>
有关一个特定顶点的信息,以及每个子任务的摘要。
样本结果:
{ "id": "dceafe2df1f57a1206fcb907cb38ad97", "name": "CHAIN DataSource -> Map -> FlatMap -> Combine(SUM(1))", "parallelism": 2, "now": 1442424002154, "subtasks": [ { "subtask":0, "status": "FINISHED", "attempt": 0, "host": "localhost", "start-time": 1442421093762, "end-time": 1442421386680, "duration": 292918, "metrics": { "read-bytes": 0, "write-bytes": 12684375, "read-records": 0, "write-records": 1153125 } }, { "subtask": 1, "status": "FINISHED", "attempt": 0, "host": "localhost", "start-time": 1442421093774, "end-time": 1442421386267, "duration": 292493, "metrics": { "read-bytes": 0, "write-bytes": 12684375, "read-records": 0, "write-records": 1153125 } } ] }
/jobs/<jobid>/vertices/<vertexid>/subtasktimes
此请求返回给定顶点的所有子任务的状态转换的时间戳。例如,这些可用于在子任务之间创建时间线比较。
样本结果:
{ "id": "dceafe2df1f57a1206fcb907cb38ad97", "name": "CHAIN DataSource -> Map -> Combine(SUM(1))", "now":1442423745088, "subtasks": [ { "subtask": 0, "host": "localhost", "duration": 292924, "timestamps": { "CREATED": 1442421093741, "SCHEDULED": 1442421093756, "DEPLOYING": 1442421093762, "RUNNING": 1442421094026, "FINISHED": 1442421386680, "CANCELING": 0, "CANCELED": 0, "FAILED": 0 } }, { "subtask": 1, "host": "localhost", "duration": 292494, "timestamps": { "CREATED": 1442421093741, "SCHEDULED": 1442421093773, "DEPLOYING": 1442421093774, "RUNNING": 1442421094013, "FINISHED": 1442421386267, "CANCELING": 0, "CANCELED": 0, "FAILED": 0 } } ] }
/jobs/<jobid>/vertices/<vertexid>/taskmanagers
一个特定顶点的TaskManager统计信息。这是由返回的子任务统计信息的聚合/jobs/<jobid>/vertices/<vertexid>
。
样本结果:
{ "id": "fe20bcc29b87cdc76589ca42114c2499", "name": "Reduce (SUM(1), at main(WordCount.java:72)", "now": 1454348282653, "taskmanagers": [ { "host": "ip-10-0-43-227:35413", "status": "FINISHED", "start-time": 1454347870991, "end-time": 1454347872111, "duration": 1120, "metrics": { "read-bytes": 32503056, "write-bytes": 9637041, "read-records": 2906087, "write-records": 849467 }, "status-counts": { "CREATED": 0, "SCHEDULED": 0, "DEPLOYING": 0, "RUNNING": 0, "FINISHED": 18, "CANCELING": 0, "CANCELED": 0, "FAILED": 0 } },{ "host": "ip-10-0-43-227:41486", "status": "FINISHED", "start-time": 1454347871001, "end-time": 1454347872395, "duration": 1394, "metrics": { "read-bytes": 32389499, "write-bytes": 9608829, "read-records": 2895999, "write-records": 846948 }, "status-counts": { "CREATED": 0, "SCHEDULED": 0, "DEPLOYING": 0, "RUNNING": 0, "FINISHED": 18, "CANCELING": 0, "CANCELED": 0, "FAILED": 0 } } ] }
/jobs/<jobid>/vertices/<vertexid>/accumulators
用于特定顶点的聚合的用户定义累加器。
样本结果:
{ "id": "dceafe2df1f57a1206fcb907cb38ad97", "user-accumulators": [ { "name": "avglen", "type": "DoubleCounter", "value": "DoubleCounter 123.03259440000001" }, { "name": "genwords", "type": "LongCounter", "value": "LongCounter 75000000" } ] }
/jobs/<jobid>/vertices/<vertexid>/subtasks/accumulators
获取给定顶点的所有子任务的所有用户定义的累加器。这些是由请求以聚合形式返回的各个累加器/jobs/<jobid>/vertices/<vertexid>/accumulators
。
样本结果:
{ "id": "dceafe2df1f57a1206fcb907cb38ad97", "parallelism": 2, "subtasks": [ { "subtask": 0, "attempt": 0, "host": "localhost", "user-accumulators": [ { "name": "genwords", "type": "LongCounter", "value": "LongCounter 62500000" }, { "name": "genletters", "type": "LongCounter", "value": "LongCounter 1281589525" } ] }, { "subtask": 1, "attempt": 0, "host": "localhost", "user-accumulators": [ { "name": "genwords", "type": "LongCounter", "value": "LongCounter 12500000" }, { "name": "genletters", "type": "LongCounter", "value": "LongCounter 256317905" } ] } ] }
/jobs/<jobid>/vertices/<vertexid>/subtasks/<subtasknum>
特定子任务的当前或最新执行尝试的摘要。请参阅下面的示例。
/jobs/<jobid>/vertices/<vertexid>/subtasks/<subtasknum>/attempts/<attempt>
特定子任务的特定执行尝试的摘要。在发生故障/恢复时会发生多次执行尝试。
样本结果:
{ "subtask": 0, "status": "FINISHED", "attempt": 0, "host": "localhost", "start-time": 1442421093762, "end-time": 1442421386680, "duration": 292918, "metrics": { "read-bytes": 0, "write-bytes": 12684375, "read-records": 0, "write-records": 1153125 } }
/jobs/<jobid>/vertices/<vertexid>/subtasks/<subtasknum>/attempts/<attempt>/accumulators
在一次特定执行尝试期间为一个特定子任务收集累加器(在故障/恢复的情况下发生多次尝试)。
样本结果:
{ "subtask": 0, "attempt": 0, "id": "b22f94d91bf41ddb", "user-accumulators": [ { "name": "genwords", "type":"LongCounter", "value":"LongCounter 62500000" }, { "name": "genletters", "type": "LongCounter", "value": "LongCounter 1281589525" }, { "name": "avglen", "type": "DoubleCounter", "value": "DoubleCounter 102.527162" } ] }
/jobs/<jobid>/plan
作业的数据流计划。该计划也包含在工作总结(/jobs/<jobid>
)中。
样本结果:
{ "jid":"ab78dcdbb1db025539e30217ec54ee16", "name":"WordCount Example", "nodes": [ { "id": "f00c89b349b5c998cfd9fe2a06e50fd0", "parallelism": 2, "operator": "GroupReduce", "operator_strategy": "Sorted Group Reduce", "description": "Reduce (SUM(1), at main(WordCount.java:67)", "inputs": [ { "num": 0, "id":"19b5b24062c48a06e4eac65422ac3317", "ship_strategy": "Hash Partition on [0]", "local_strategy":"Sort (combining) on [0:ASC]", "exchange":"pipelined" } ], "optimizer_properties": { "global_properties": [ { "name":"Partitioning", "value":"HASH_PARTITIONED" }, { "name":"Partitioned on", "value":"[0]" }, { "name":"Partitioning Order", "value":"(none)" }, { "name":"Uniqueness", "value":"not unique" } ], "local_properties": [ { "name":"Order", "value":"[0:ASC]" }, { "name":"Grouped on", "value":"[0]" }, { "name":"Uniqueness", "value":"not unique" } ], "estimates": [ { "name":"Est. Output Size", "value":"(unknown)" }, { "name":"Est. Cardinality", "value":"(unknown)" } ], "costs": [ { "name":"Network", "value":"(unknown)" }, { "name":"Disk I/O", "value":"(unknown)" }, { "name":"CPU", "value":"(unknown)" }, { "name":"Cumulative Network", "value":"(unknown)" }, { "name":"Cumulative Disk I/O", "value":"(unknown)" }, { "name":"Cumulative CPU","value":"(unknown)" } ], "compiler_hints": [ { "name":"Output Size (bytes)", "value":"(none)" }, { "name":"Output Cardinality", "value":"(none)" }, { "name":"Avg. Output Record Size (bytes)", "value":"(none)" }, { "name":"Filter Factor", "value":"(none)" } ] } }, { "id": "19b5b24062c48a06e4eac65422ac3317", "parallelism": 2, "operator": "Data Source -> FlatMap -> GroupCombine", "operator_strategy":" (none) -> FlatMap -> Sorted Combine", "description":"DataSource (at getTextDataSet(WordCount.java:142) (org.apache.flink.api.java.io.TextInputFormat)) -> FlatMap (FlatMap at main(WordCount.java:67)) -> Combine(SUM(1), at main(WordCount.java:67)", "optimizer_properties": { ... } }, { "id": "0a36cbc29102d7bc993d0a9bf23afa12", "parallelism": 2, "operator": "Data Sink", "operator_strategy": "(none)", "description": "DataSink (CsvOutputFormat (path: /tmp/abzs, delimiter: ))", "inputs":[ { "num": 0, "id": "f00c89b349b5c998cfd9fe2a06e50fd0", "ship_strategy": "Forward", "exchange": "pipelined" } ], "optimizer_properties": { ... } } ] }
取消工作
取消工作
DELETE
要求/jobs/:jobid/cancel
。
触发取消工作,成功的结果是{}
。
使用Savepoint取消作业
保存点成功后触发保存点并取消作业。
GET
请求/jobs/:jobid/cancel-with-savepoint/
触发保存点到默认保存点目录并取消作业。
GET
请求/jobs/:jobid/cancel-with-savepoint/target-directory/:targetDirectory
触发保存点到给定目标目录并取消作业。
由于保存点可能需要一些时间才能完成,因此异步执行此 算子操作。此请求的结果是正在进行的取消的位置。
样本触发结果:
{ "status": "accepted", "request-id": 1, "location": "/jobs/:jobid/cancel-with-savepoint/in-progress/1" }
监测进展
取消的进度必须由用户监控
/jobs/:jobid/cancel-with-savepoint/in-progress/:requestId
触发结果返回请求ID。
进行中
{ "status": "in-progress", "request-id": 1 }
成功
{ "status": "success", "request-id": 1, "savepoint-path": "<savepointPath>" }
的savepointPath
点到保存点的外部路径,其可用于恢复保存点。
失败
{ "status": "failed", "request-id": 1, "cause": "<error message>" }
提交程序
可以通过REST API和Web前端上传,运行和列出Flink程序。
上传新的JAR文件
发送POST
请求,/jars/upload
将您的jar文件作为多部分数据发送到该jarfile
文件下。还要确保多部分数据包含Content-Type
文件本身,某些http库默认情况下不添加标头。
多部分有效负载应该开始
------BoundaryXXXX
Content-Disposition: form-data; name="jarfile"; filename="YourFileName.jar"
Content-Type: application/x-java-archive
运行程序(POST)
发送POST
请求/jars/:jarid/run
。该jarid
参数是配置的Web前端上载目录(配置Keysweb.upload.dir
)中程序JAR的文件名。
您可以指定以下查询参数(全部可选):
- 程序参数:
program-args=arg1 arg2 arg3
- 要执行的主要类:
entry-class=EntryClassName.class
- 默认并行度:
parallelism=4
- 要从中恢复的保存点路径:
savepointPath=hdfs://path/to/savepoint
- 允许非恢复状态:
allowNonRestoredState=true
如果呼叫成功,您将收到包含已提交作业的ID的响应。
示例:使用保存点运行程序
请求:
POST: /jars/MyProgram.jar/run?savepointPath=/my-savepoints/savepoint-1bae02a80464&allowNonRestoredState=true
响应:
{"jobid": "869a9868d49c679e7355700e0857af85"}