blob: bc7f7bf359cba9f4815d8ce85d283886ce564b42 [file] [log] [blame]
Lusheng Ji72c4fba2017-10-10 18:12:22 +00001===============
2CDAP Broker API
3===============
4
5:Date: 2017-10-09
6
7.. contents::
8 :depth: 3
9..
10
11Overview
12========
13
14Version information
15-------------------
16
17*Version* : 4.0.10
18
19Paths
20=====
21
22GET /
23-----
24
25Description
26~~~~~~~~~~~
27
28shows some information about this service
29
30Responses
31~~~~~~~~~
32
33+---------+----------------------------------------------------+----------------+
34| HTTP | Description | Schema |
35| Code | | |
36+=========+====================================================+================+
37| **200** | successful response | `info <#_info> |
38| | | `__ |
39+---------+----------------------------------------------------+----------------+
40
41GET /application
42----------------
43
44Description
45~~~~~~~~~~~
46
47get all applications registered with this broker
48
49Responses
50~~~~~~~~~
51
52+---------+----------------------------------------------------+----------------+
53| HTTP | Description | Schema |
54| Code | | |
55+=========+====================================================+================+
56| **200** | successful response | < |
57| | | `appname <#_ap |
58| | | pname>`__ |
59| | | > array |
60+---------+----------------------------------------------------+----------------+
61
62PUT /application\*/{appname}
63----------------------------
64
65Description
66~~~~~~~~~~~
67
68(This is a hacky way of supporting "oneOf" because Swagger does not
69support oneOf https://github.com/OAI/OpenAPI-Specification/issues/333.
70This is the same endpoint as PUT /application/appname, except the PUT
71body is different.)
72
73Register a hydrator app for service and configuration discovery. This
74will light up a metrics and health endpoint for this app. ``appname`` is
75assumed to also be the key in consul.
76
77Parameters
78~~~~~~~~~~
79
80+---------+-------------+--------------------------------------+------------------+
81| Type | Name | Description | Schema |
82+=========+=============+======================================+==================+
83| **Path* | | **appname | Name of the application. | string (text) |
84| * | ** | | |
85| | | *required | | |
86| | * | | |
87+---------+-------------+--------------------------------------+------------------+
88| **Body* | | **putbody | required put body | `hydratorappput |
89| * | ** | | <#_hydratorapppu |
90| | | *required | | t>`__ |
91| | * | | |
92+---------+-------------+--------------------------------------+------------------+
93
94Responses
95~~~~~~~~~
96
97+---------+----------------------------------------------------+----------------+
98| HTTP | Description | Schema |
99| Code | | |
100+=========+====================================================+================+
101| **200** | Successful response | `Application < |
102| | | #_application> |
103| | | `__ |
104+---------+----------------------------------------------------+----------------+
105| **400** | put was performed but the appname was already | No Content |
106| | registered with the broker, or Invalid PUT body | |
107+---------+----------------------------------------------------+----------------+
108
109Consumes
110~~~~~~~~
111
112- ``application/json``
113
114Produces
115~~~~~~~~
116
117- ``application/json``
118
119POST /application/delete
120------------------------
121
122Description
123~~~~~~~~~~~
124
125endpoint to delete multiple applications at once. Returns an array of
126status codes, where statuscode[i] = response returned from
127DELETE(application/i)
128
129Parameters
130~~~~~~~~~~
131
132+---------+-------------+--------------------------------------+------------------+
133| Type | Name | Description | Schema |
134+=========+=============+======================================+==================+
135| **Body* | | **postbod | required post body | `multideleteput |
136| * | y** | | <#_multideletepu |
137| | | *required | | t>`__ |
138| | * | | |
139+---------+-------------+--------------------------------------+------------------+
140
141Responses
142~~~~~~~~~
143
144+---------+----------------------------------------------------+----------------+
145| HTTP | Description | Schema |
146| Code | | |
147+=========+====================================================+================+
148| **200** | successful response | < |
149| | | `returncode <# |
150| | | _returncode>`_ |
151| | | _ |
152| | | > array |
153+---------+----------------------------------------------------+----------------+
154
155GET /application/{appname}
156--------------------------
157
158Description
159~~~~~~~~~~~
160
161Returns the representation of the application resource, including the
162links for healthcheck and metrics.
163
164Parameters
165~~~~~~~~~~
166
167+---------+-------------+--------------------------------------+------------------+
168| Type | Name | Description | Schema |
169+=========+=============+======================================+==================+
170| **Path* | | **appname | Name of the application. | string (text) |
171| * | ** | | |
172| | | *required | | |
173| | * | | |
174+---------+-------------+--------------------------------------+------------------+
175
176Responses
177~~~~~~~~~
178
179+---------+----------------------------------------------------+----------------+
180| HTTP | Description | Schema |
181| Code | | |
182+=========+====================================================+================+
183| **200** | Successful response | `Application < |
184| | | #_application> |
185| | | `__ |
186+---------+----------------------------------------------------+----------------+
187| **404** | no app with name 'appname' registered with this | No Content |
188| | broker. | |
189+---------+----------------------------------------------------+----------------+
190
191PUT /application/{appname}
192--------------------------
193
194Description
195~~~~~~~~~~~
196
197Register an app for service and configuration discovery. This will light
198up a metrics and health endpoint for this app. ``appname`` is assumed to
199also be the key in consul.
200
201Parameters
202~~~~~~~~~~
203
204+---------+-------------+--------------------------------------+------------------+
205| Type | Name | Description | Schema |
206+=========+=============+======================================+==================+
207| **Path* | | **appname | Name of the application. | string (text) |
208| * | ** | | |
209| | | *required | | |
210| | * | | |
211+---------+-------------+--------------------------------------+------------------+
212| **Body* | | **putbody | required put body | `appput <#_apppu |
213| * | ** | | t>`__ |
214| | | *required | | |
215| | * | | |
216+---------+-------------+--------------------------------------+------------------+
217
218Responses
219~~~~~~~~~
220
221+---------+----------------------------------------------------+----------------+
222| HTTP | Description | Schema |
223| Code | | |
224+=========+====================================================+================+
225| **200** | Successful response | `Application < |
226| | | #_application> |
227| | | `__ |
228+---------+----------------------------------------------------+----------------+
229| **400** | put was performed but the appname was already | No Content |
230| | registered with the broker, or Invalid PUT body | |
231+---------+----------------------------------------------------+----------------+
232
233Consumes
234~~~~~~~~
235
236- ``application/json``
237
238Produces
239~~~~~~~~
240
241- ``application/json``
242
243DELETE /application/{appname}
244-----------------------------
245
246Description
247~~~~~~~~~~~
248
249Remove an app for service and configuration discovery. This will remove
250the metrics and health endpoints for this app.
251
252Parameters
253~~~~~~~~~~
254
255+---------+-------------+--------------------------------------+------------------+
256| Type | Name | Description | Schema |
257+=========+=============+======================================+==================+
258| **Path* | | **appname | Name of the application. | string (text) |
259| * | ** | | |
260| | | *required | | |
261| | * | | |
262+---------+-------------+--------------------------------------+------------------+
263
264Responses
265~~~~~~~~~
266
267+---------+----------------------------------------------------+----------------+
268| HTTP | Description | Schema |
269| Code | | |
270+=========+====================================================+================+
271| **200** | Successful response | No Content |
272+---------+----------------------------------------------------+----------------+
273| **404** | no app with name 'appname' registered with this | No Content |
274| | broker. | |
275+---------+----------------------------------------------------+----------------+
276
277GET /application/{appname}/healthcheck
278--------------------------------------
279
280Description
281~~~~~~~~~~~
282
283Perform a healthcheck on the running app appname.
284
285Parameters
286~~~~~~~~~~
287
288+---------+-------------+--------------------------------------+------------------+
289| Type | Name | Description | Schema |
290+=========+=============+======================================+==================+
291| **Path* | | **appname | Name of the application to get the | string (test) |
292| * | ** | healthcheck for. | |
293| | | *required | | |
294| | * | | |
295+---------+-------------+--------------------------------------+------------------+
296
297Responses
298~~~~~~~~~
299
300+---------+----------------------------------------------------+----------------+
301| HTTP | Description | Schema |
302| Code | | |
303+=========+====================================================+================+
304| **200** | Successful response, healthcheck pass | No Content |
305+---------+----------------------------------------------------+----------------+
306| **404** | no app with name 'appname' registered with this | No Content |
307| | broker, or the healthcheck has failed (though I | |
308| | would like to disambiguiate from the first case, | |
309| | CDAP returns a 404 for this). | |
310+---------+----------------------------------------------------+----------------+
311
312GET /application/{appname}/metrics
313----------------------------------
314
315Description
316~~~~~~~~~~~
317
318Get live (real-time) app specific metrics for the running app appname.
319Metrics are customized per each app by the component developer
320
321Parameters
322~~~~~~~~~~
323
324+---------+-------------+--------------------------------------+------------------+
325| Type | Name | Description | Schema |
326+=========+=============+======================================+==================+
327| **Path* | | **appname | Name of the application to get | string (test) |
328| * | ** | metrics for. | |
329| | | *required | | |
330| | * | | |
331+---------+-------------+--------------------------------------+------------------+
332
333Responses
334~~~~~~~~~
335
336+---------+----------------------------------------------------+----------------+
337| HTTP | Description | Schema |
338| Code | | |
339+=========+====================================================+================+
340| **200** | Successful response | `MetricsObject |
341| | | <#_metricsobj |
342| | | ect>`__ |
343+---------+----------------------------------------------------+----------------+
344| **404** | no app with name 'appname' registered with this | No Content |
345| | broker. | |
346+---------+----------------------------------------------------+----------------+
347
348PUT /application/{appname}/reconfigure
349--------------------------------------
350
351Description
352~~~~~~~~~~~
353
354Reconfigures the application.
355
356Parameters
357~~~~~~~~~~
358
359+---------+-------------+--------------------------------------+------------------+
360| Type | Name | Description | Schema |
361+=========+=============+======================================+==================+
362| **Path* | | **appname | Name of the application. | string (text) |
363| * | ** | | |
364| | | *required | | |
365| | * | | |
366+---------+-------------+--------------------------------------+------------------+
367| **Body* | | **putbody | required put body | `reconfigput <#_ |
368| * | ** | | reconfigput>`__ |
369| | | *required | | |
370| | * | | |
371+---------+-------------+--------------------------------------+------------------+
372
373Responses
374~~~~~~~~~
375
376+---------+----------------------------------------------------+----------------+
377| HTTP | Description | Schema |
378| Code | | |
379+=========+====================================================+================+
380| **200** | Successful response | No Content |
381+---------+----------------------------------------------------+----------------+
382| **400** | Bad request. Can happen with 1) {appname} is not | No Content |
383| | registered with the broker, 2) the required PUT | |
384| | body is wrong, or 3) the smart interface was | |
385| | chosen and none of the config keys match anything | |
386| | in app\_config or app\_preferences | |
387+---------+----------------------------------------------------+----------------+
388
389Definitions
390===========
391
392Application
393-----------
394
395+-------------+---------------------------------------------+------------------+
396| Name | Description | Schema |
397+=============+=============================================+==================+
398| | **appname | application name | string |
399| ** | | |
400| | *optional | | |
401| * | | |
402+-------------+---------------------------------------------+------------------+
403| | **connect | input URL that you can POST data into (URL | string |
404| ionurl** | of the CDAP stream) | |
405| | *optional | | |
406| * | | |
407+-------------+---------------------------------------------+------------------+
408| | **healthc | fully qualified url to perform healthcheck | string |
409| heckurl** | | |
410| | *optional | | |
411| * | | |
412+-------------+---------------------------------------------+------------------+
413| | **metrics | fully qualified url to get metrics from | string |
414| url** | | |
415| | *optional | | |
416| * | | |
417+-------------+---------------------------------------------+------------------+
418| | **service | a list of HTTP services exposed by this | < |
419| endpoints** | CDAP application | `service\_method |
420| | *optional | | <#_service_meth |
421| * | | od>`__ |
422| | | > array |
423+-------------+---------------------------------------------+------------------+
424| | **url** | fully qualified url of the resource | string |
425| | *optional | | |
426| * | | |
427+-------------+---------------------------------------------+------------------+
428
429MetricsObject
430-------------
431
432key,value object where the key is 'appmetrics' and the value is an app
433dependent json and specified by the component developer
434
435+--------------------------------+-------------------------------------------+
436| Name | Schema |
437+================================+===========================================+
438| | **appmetrics** | object |
439| | *optional* | |
440+--------------------------------+-------------------------------------------+
441
442appname
443-------
444
445an application name
446
447*Type* : string
448
449appput
450------
451
452+-------------+---------------------------------------------+------------------+
453| Name | Description | Schema |
454+=============+=============================================+==================+
455| | **app\_co | the application config JSON | object |
456| nfig** | | |
457| | *optional | | |
458| * | | |
459+-------------+---------------------------------------------+------------------+
460| | **app\_pr | the application preferences JSON | object |
461| eferences** | | |
462| | *optional | | |
463| * | | |
464+-------------+---------------------------------------------+------------------+
465| | **artifac | the name of the CDAP artifact to be added | string |
466| t\_name** | | |
467| | *optional | | |
468| * | | |
469+-------------+---------------------------------------------+------------------+
470| | **cdap\_a | denotes whether this is a program-flowlet | enum |
471| pplication\ | style application or a hydrator pipeline. | (program-flowlet |
472| _type** | For program-flowlet style apps, this value | ) |
473| | *optional | must be "program-flowlet" | |
474| * | | |
475+-------------+---------------------------------------------+------------------+
476| | **jar\_ur | the URL that the JAR youre deploying | string |
477| l** | resides | |
478| | *optional | | |
479| * | | |
480+-------------+---------------------------------------------+------------------+
481| | **namespa | the cdap namespace this is deployed into | string |
482| ce** | | |
483| | *optional | | |
484| * | | |
485+-------------+---------------------------------------------+------------------+
486| | **program | | < |
487| \_preferenc | | `programpref <#_ |
488| es** | | programpref>`__ |
489| | *optional | | > array |
490| * | | |
491+-------------+---------------------------------------------+------------------+
492| | **program | | < |
493| s** | | `programs <#_pro |
494| | *optional | | grams>`__ |
495| * | | > array |
496+-------------+---------------------------------------------+------------------+
497| | **service | | < |
498| s** | | `service\_endpoi |
499| | *optional | | nt <#_service_en |
500| * | | dpoint>`__ |
501| | | > array |
502+-------------+---------------------------------------------+------------------+
503| | **streamn | name of the CDAP stream to ingest data into | string |
504| ame** | this app. Should come from the developer | |
505| | *optional | and Tosca model. | |
506| * | | |
507+-------------+---------------------------------------------+------------------+
508
509hydratorappput
510--------------
511
512+-------------+---------------------------------------------+------------------+
513| Name | Description | Schema |
514+=============+=============================================+==================+
515| | **cdap\_a | denotes whether this is a program-flowlet | enum |
516| pplication\ | style application or a hydrator pipeline. | (hydrator-pipeli |
517| _type** | For hydrator, this value must be | ne) |
518| | *required | "hydrator-pipeline" | |
519| * | | |
520+-------------+---------------------------------------------+------------------+
521| | **depende | represents a list of dependencies to be | < |
522| ncies** | loaded for this pipeline. Not required. | `hydratordep <#_ |
523| | *optional | | hydratordep>`__ |
524| * | | > array |
525+-------------+---------------------------------------------+------------------+
526| | **namespa | the cdap namespace this is deployed into | string |
527| ce** | | |
528| | *required | | |
529| * | | |
530+-------------+---------------------------------------------+------------------+
531| | **pipelin | the URL of the config.json for this | string |
532| e\_config\_ | pipeline | |
533| json\_url** | | |
534| | *required | | |
535| * | | |
536+-------------+---------------------------------------------+------------------+
537| | **streamn | name of the CDAP stream to ingest data into | string |
538| ame** | this app. Should come from the developer | |
539| | *required | and Tosca model. | |
540| * | | |
541+-------------+---------------------------------------------+------------------+
542
543hydratordep
544-----------
545
546represents a hydrator pipeline dependency. An equivelent to the
547following CURLs are formed with the below four params shown in CAPS
548"curl -v -w"\\n" -X POST
549http://cdapurl:11015/v3/namespaces/setelsewhere/artifacts/ARTIFACT_NAME
550-H "Artifact-Extends:ARTIFACT\_EXTENDS\_HEADER" -H
551“Artifact-Version:ARTIFACT\_VERSION\_HEADER” –data-binary @(DOWNLOADED
552FROM ARTIFACT\_URL)","curl -v -w"\\n" -X PUT
553http://cdapurl:11015/v3/namespaces/setelsewhere/artifacts/ARTIFACT_NAME/versions/ARTIFACT_VERSION_HEADER/properties
554-d (DOWNLOADED FROM UI\_PROPERTIES\_URL)"
555
556+-------------+---------------------------------------------+------------------+
557| Name | Description | Schema |
558+=============+=============================================+==================+
559| | **artifac | the value of the header that gets passed in | string |
560| t\_extends\ | for artifact-extends, e.g., | |
561| _header** | "Artifact-Extends:system:cdap-data-pipeline | |
562| | *required | [4.0.1,5.0.0)" | |
563| * | | |
564+-------------+---------------------------------------------+------------------+
565| | **artifac | the name of the artifact | string |
566| t\_name** | | |
567| | *required | | |
568| * | | |
569+-------------+---------------------------------------------+------------------+
570| | **artifac | the URL of the artifact JAR | string |
571| t\_url** | | |
572| | *required | | |
573| * | | |
574+-------------+---------------------------------------------+------------------+
575| | **artifac | the value of the header that gets passed in | string |
576| t\_version\ | for artifact-version, e.g., | |
577| _header** | "Artifact-Version:1.0.0-SNAPSHOT" | |
578| | *required | | |
579| * | | |
580+-------------+---------------------------------------------+------------------+
581| | **ui\_pro | the URL of the properties.json if the | string |
582| perties\_ur | custom artifact has UI properties. This is | |
583| l** | optional. | |
584| | *optional | | |
585| * | | |
586+-------------+---------------------------------------------+------------------+
587
588info
589----
590
591some broker information
592
593+-------------+---------------------------------------------+------------------+
594| Name | Description | Schema |
595+=============+=============================================+==================+
596| | **broker | the API version of this running broker | string |
597| API | | |
598| version** | | |
599| | *optional | | |
600| * | | |
601+-------------+---------------------------------------------+------------------+
602| | **cdap | The GUI port of the CDAP cluster this | integer |
603| GUI | broker is managing. Mostly to help users of | |
604| port** | this API check their application in cdap. | |
605| | *optional | Note, will return UNKNOWN\_CDAP\_VERSION if | |
606| * | it cannot be determined. | |
607+-------------+---------------------------------------------+------------------+
608| | **cdap | the version of the CDAP cluster this broker | string |
609| cluster | is managing. Note, will return | |
610| version** | UKNOWN\_CDAP\_VERSION if it cannot be | |
611| | *optional | determined. | |
612| * | | |
613+-------------+---------------------------------------------+------------------+
614| | **managed | the url of the CDAP cluster API this broker | string |
615| cdap | is managing | |
616| url** | | |
617| | *optional | | |
618| * | | |
619+-------------+---------------------------------------------+------------------+
620| | **number | | integer |
621| of | | |
622| applicati | | |
623| ons | | |
624| registere | | |
625| d** | | |
626| | *optional | | |
627| * | | |
628+-------------+---------------------------------------------+------------------+
629| | **uptime | | integer |
630| (s)** | | |
631| | *optional | | |
632| * | | |
633+-------------+---------------------------------------------+------------------+
634
635multideleteput
636--------------
637
638+--------------------------------+-------------------------------------------+
639| Name | Schema |
640+================================+===========================================+
641| | **appnames** | < `appname <#_appname>`__ > array |
642| | *optional* | |
643+--------------------------------+-------------------------------------------+
644
645programpref
646-----------
647
648the list of programs in this CDAP app
649
650+-------------+---------------------------------------------+------------------+
651| Name | Description | Schema |
652+=============+=============================================+==================+
653| | **program | the name of the program | string |
654| \_id** | | |
655| | *optional | | |
656| * | | |
657+-------------+---------------------------------------------+------------------+
658| | **program | the preference JSON to set for this program | object |
659| \_pref** | | |
660| | *optional | | |
661| * | | |
662+-------------+---------------------------------------------+------------------+
663| | **program | must be one of flows, mapreduce, schedules, | string |
664| \_type** | spark, workflows, workers, or services | |
665| | *optional | | |
666| * | | |
667+-------------+---------------------------------------------+------------------+
668
669programs
670--------
671
672the list of programs in this CDAP app
673
674+-------------+---------------------------------------------+------------------+
675| Name | Description | Schema |
676+=============+=============================================+==================+
677| | **program | the name of the program | string |
678| \_id** | | |
679| | *optional | | |
680| * | | |
681+-------------+---------------------------------------------+------------------+
682| | **program | must be one of flows, mapreduce, schedules, | string |
683| \_type** | spark, workflows, workers, or services | |
684| | *optional | | |
685| * | | |
686+-------------+---------------------------------------------+------------------+
687
688reconfigput
689-----------
690
691+-------------+---------------------------------------------+------------------+
692| Name | Description | Schema |
693+=============+=============================================+==================+
694| | **config* | the config JSON | object |
695| * | | |
696| | *required | | |
697| * | | |
698+-------------+---------------------------------------------+------------------+
699| | **reconfi | the type of reconfiguration | enum |
700| guration\_t | | (program-flowlet |
701| ype** | | -app-config, |
702| | *required | | program-flowlet- |
703| * | | app-preferences, |
704| | | program-flowlet- |
705| | | smart) |
706+-------------+---------------------------------------------+------------------+
707
708returncode
709----------
710
711an httpreturncode
712
713*Type* : integer
714
715service\_endpoint
716-----------------
717
718descirbes a service endpoint, including the service name, the method
719name, and the method type (GET, PUT, etc, most of the time will be GET)
720
721+-------------+---------------------------------------------+------------------+
722| Name | Description | Schema |
723+=============+=============================================+==================+
724| | **endpoin | GET, POST, PUT, etc | string |
725| t\_method** | | |
726| | *optional | | |
727| * | | |
728+-------------+---------------------------------------------+------------------+
729| | **service | the name of the endpoint on the service | string |
730| \_endpoint* | | |
731| * | | |
732| | *optional | | |
733| * | | |
734+-------------+---------------------------------------------+------------------+
735| | **service | the name of the service | string |
736| \_name** | | |
737| | *optional | | |
738| * | | |
739+-------------+---------------------------------------------+------------------+
740
741service\_method
742---------------
743
744a URL and HTTP method exposed via a CDAP service
745
746+-------------+---------------------------------------------+------------------+
747| Name | Description | Schema |
748+=============+=============================================+==================+
749| | **method* | HTTP method you can perform on the URL, | string |
750| * | e.g., GET, PUT, etc | |
751| | *optional | | |
752| * | | |
753+-------------+---------------------------------------------+------------------+
754| | **url** | the fully qualified URL in CDAP for this | string |
755| | *optional | service | |
756| * | | |
757+-------------+---------------------------------------------+------------------+