Gitiles
Code Review Sign In
gerrit.nordix.org / onap / dcaegen2 / 09f2cc5eb10c6193ac8406555da983479576e672 / . / docs / sections / installation_manual.rst
blob: 070e36ab9b07e1e168b17f7016e8ac7e6f6788c7 [file] [log] [blame]
Lusheng Jia73548f2017-11-05 19:24:05 -0500[diff] [blame]1DCAE mS Installation
2====================
3
4The below steps covers manual setup of DCAE VM’s and DCAE service
5components.
6
7VESCollector
8------------
9
10 
11
12DCAE VES Collector can be configured on VM with ubuntu-16.04 image
13(m1.small should suffice if this is only service) and 20Gb cinder
14storage
15
161. Install docker
17
18  sudo apt-get update
19
20  sudo apt install `docker.io <http://docker.io/>`__
21
222. Pull the latest container from onap nexus
23
24 sudo docker login -u docker -p docker
25 `nexus.onap.org <http://nexus.onap.org/>`__:10001
26
27 sudo docker pull
28 `nexus.onap.org <http://nexus.onap.org/>`__:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1
29
303. Start the VESCollector with below command
31
32 sudo docker run -d --name vescollector -p 8080:8080/tcp -p
33 8443:8443/tcp -P -e DMAAPHOST='<dmaap IP>'
34 `nexus.onap.org <http://nexus.onap.org/>`__:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1
35
36  Note:  Change the dmaaphost to required DMAAP ip. To change the
37 dmaap information for a running container,  stop the active
38 container and rerun above command changing the dmaap IP.
39
404. Verification
41
42i. Check logs under container /opt/app/VESCollector/logs/collector.log
43 for errors
44
45ii. If no active feed, you can simulate an event into collector via curl
46
47 curl -i  -X POST -d @<sampleves> --header "Content-Type:
48 application/json" http://localhost:8080/eventListener/v5 -k
49
50 Note: If DMAAPHOST provided is invalid, you will see exception
51 around publish on the collector.logs (collector queues and attempts
52 to resend the event hence exceptions reported will be periodic). 
53
54i. Below two topic configuration are pre-set into this container.  When
55 valid DMAAP instance ip was provided and VES events are received,
56 the collector will post to below topics.
57
58 Fault -
59  http://<dmaaphost>:3904/events/unauthenticated.SEC\_FAULT\_OUTPUT
60
61 Measurement
62 -http://<dmaaphost>:3904/events/unauthenticated.SEC\_MEASUREMENT\_OUTPUT
63
64VM Init
65~~~~~~
66
67To address windriver server in-stability, the below **init.sh** script
68was used to start the container on VM restart.  
69
70+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
71| #!/bin/sh |
72| |
73| sudo docker ps \| grep "vescollector" |
74| |
75| if [ $? -ne 0 ]; then |
76| |
77|         sudo docker login -u docker -p docker nexus.onap.org:10001 |
78| |
79|         sudo docker pull nexus.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1 |
80| |
81|         sudo docker rm -f vescollector |
82| |
83|         echo "Collector process not running - $(date)" >> /home/ubuntu/startuplog |
84| |
85|         sudo docker run -d --name vescollector -p 8080:8080/tcp -p 8443:8443/tcp -P -e DMAAPHOST='10.12.25.96' nexus.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1 |
86| |
87| else |
88| |
89|         echo "Collector process running - $(date)" >> /home/ubuntu/startuplog |
90| |
91| fi |
92+==============================================================================================================================================================================================+
93+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
94
95This script was invoked via VM init script (rc.d).
96
97ln -s /home/ubuntu/init.sh /etc/init.d/init.sh
98
99sudo  update-rc.d init.sh start 2
100
101 
102
103ThresholdCrossingAnalysis (TCA/CDAP)
104------------------------------------
105
106The platform deploys CDAP as cluster and instantiates TCA. For the
107manual setup, we will leverage the CDAP SDK docker container to deploy
108TCA instances.  To setup TCA, choose VM with ubuntu-16.04 image,
109m1.medium size and 50gb cinder volumes.
110
1111. Install docker
112
113  sudo apt-get update
114
115  sudo apt install `docker.io <http://docker.io/>`__
116
1172. Pull CDAP SDK container
118
119sudo docker pull caskdata/cdap-standalone:4.1.2
120
1213. Deploy and run the CDAP container
122
123 sudo docker run -d --name cdap-sdk-2 -p 11011:11011 -p 11015:11015
124 caskdata/cdap-standalone:4.1.2
125
1264. Create Namespace on CDAP application
127
128curl -X PUT http://localhost:11015/v3/namespaces/cdap_tca_hi_lo
129
1305. Create TCA app config file - "tca\_app\_config.json" under ~ubuntu as
131 below
132
133+------------------------------------------------------------------------------+
134| { |
135| |
136|  "artifact": { |
137| |
138|   "name": "dcae-analytics-cdap-tca", |
139| |
140|   "version": "2.0.0", |
141| |
142|   "scope": "user" |
143| |
144|  }, |
145| |
146|  "config": { |
147| |
148|   "appName": "dcae-tca", |
149| |
150|   "appDescription": "DCAE Analytics Threshold Crossing Alert Application", |
151| |
152|   "tcaVESMessageStatusTableName": "TCAVESMessageStatusTable", |
153| |
154|   "tcaVESMessageStatusTableTTLSeconds": 86400.0, |
155| |
156|   "tcaAlertsAbatementTableName": "TCAAlertsAbatementTable", |
157| |
158|   "tcaAlertsAbatementTableTTLSeconds": 1728000.0, |
159| |
160|   "tcaVESAlertsTableName": "TCAVESAlertsTable", |
161| |
162|   "tcaVESAlertsTableTTLSeconds": 1728000.0, |
163| |
164|   "thresholdCalculatorFlowletInstances": 2.0, |
165| |
166|   "tcaSubscriberOutputStreamName": "TCASubscriberOutputStream" |
167| |
168|  } |
169| |
170| } |
171+==============================================================================+
172+------------------------------------------------------------------------------+
173
1746. Create TCA app preference file under ~ubuntu as below
175
176+--------------------------------------------------------------------------------------------------------------------------------------------+
177| { |
178| |
179|   "publisherContentType" : "application/json", |
180| |
181|   "publisherHostName" : "10.12.25.96", |
182| |
183|   "publisherHostPort" : "3904", |
184| |
185|   "publisherMaxBatchSize" : "1", |
186| |
187|   "publisherMaxRecoveryQueueSize" : "100000", |
188| |
189|   "publisherPollingInterval" : "20000", |
190| |
191|   "publisherProtocol" : "http", |
192| |
193|   "publisherTopicName" : "unauthenticated.DCAE\_CL\_OUTPUT", |
194| |
195|   "subscriberConsumerGroup" : "OpenDCAE-c1", |
196| |
197|   "subscriberConsumerId" : "c1", |
198| |
199|   "subscriberContentType" : "application/json", |
200| |
201|   "subscriberHostName" : "10.12.25.96", |
202| |
203|   "subscriberHostPort" : "3904", |
204| |
205|   "subscriberMessageLimit" : "-1", |
206| |
207|   "subscriberPollingInterval" : "20000", |
208| |
209|   "subscriberProtocol" : "http", |
210| |
211|   "subscriberTimeoutMS" : "-1", |
212| |
213|   "subscriberTopicName" : "unauthenticated.SEC\_MEASUREMENT\_OUTPUT", |
214| |
215|   "enableAAIEnrichment" : false, |
216| |
217|   "aaiEnrichmentHost" : "10.12.25.72", |
218| |
219|   "aaiEnrichmentPortNumber" : 8443, |
220| |
221|   "aaiEnrichmentProtocol" : "https", |
222| |
223|   "aaiEnrichmentUserName" : "DCAE", |
224| |
225|   "aaiEnrichmentUserPassword" : "DCAE", |
226| |
227|   "aaiEnrichmentIgnoreSSLCertificateErrors" : false, |
228| |
229|   "aaiVNFEnrichmentAPIPath" : "/aai/v11/network/generic-vnfs/generic-vnf", |
230| |
231|   "aaiVMEnrichmentAPIPath" :  "/aai/v11/search/nodes-query", |
232| |
233|   "tca\_policy" : "{ |
234| |
235|         \\"domain\\": \\"measurementsForVfScaling\\", |
236| |
237|         \\"metricsPerEventName\\": [{ |
238| |
239|                 \\"eventName\\": \\"vFirewallBroadcastPackets\\", |
240| |
241|                 \\"controlLoopSchemaType\\": \\"VNF\\", |
242| |
243|                 \\"policyScope\\": \\"DCAE\\", |
244| |
245|                 \\"policyName\\": \\"DCAE.Config\_tca-hi-lo\\", |
246| |
247|                 \\"policyVersion\\": \\"v0.0.1\\", |
248| |
249|                 \\"thresholds\\": [{ |
250| |
251|                         \\"closedLoopControlName\\": \\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\\", |
252| |
253|                         \\"version\\": \\"1.0.2\\", |
254| |
255|                         \\"fieldPath\\": \\"$.event.measurementsForVfScalingFields.vNicUsageArray[\*].receivedTotalPacketsDelta\\", |
256| |
257|                         \\"thresholdValue\\": 300, |
258| |
259|                         \\"direction\\": \\"LESS\_OR\_EQUAL\\", |
260| |
261|                         \\"severity\\": \\"MAJOR\\", |
262| |
263|                         \\"closedLoopEventStatus\\": \\"ONSET\\" |
264| |
265|                 }, { |
266| |
267|                         \\"closedLoopControlName\\": \\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\\", |
268| |
269|                         \\"version\\": \\"1.0.2\\", |
270| |
271|                         \\"fieldPath\\": \\"$.event.measurementsForVfScalingFields.vNicUsageArray[\*].receivedTotalPacketsDelta\\", |
272| |
273|                         \\"thresholdValue\\": 700, |
274| |
275|                         \\"direction\\": \\"GREATER\_OR\_EQUAL\\", |
276| |
277|                         \\"severity\\": \\"CRITICAL\\", |
278| |
279|                         \\"closedLoopEventStatus\\": \\"ONSET\\" |
280| |
281|                 }] |
282| |
283|         }, { |
284| |
285|                 \\"eventName\\": \\"vLoadBalancer\\", |
286| |
287|                 \\"controlLoopSchemaType\\": \\"VM\\", |
288| |
289|                 \\"policyScope\\": \\"DCAE\\", |
290| |
291|                 \\"policyName\\": \\"DCAE.Config\_tca-hi-lo\\", |
292| |
293|                 \\"policyVersion\\": \\"v0.0.1\\", |
294| |
295|                 \\"thresholds\\": [{ |
296| |
297|                         \\"closedLoopControlName\\": \\"ControlLoop-vDNS-6f37f56d-a87d-4b85-b6a9-cc953cf779b3\\", |
298| |
299|                         \\"version\\": \\"1.0.2\\", |
300| |
301|                         \\"fieldPath\\": \\"$.event.measurementsForVfScalingFields.vNicUsageArray[\*].receivedTotalPacketsDelta\\", |
302| |
303|                         \\"thresholdValue\\": 300, |
304| |
305|                         \\"direction\\": \\"GREATER\_OR\_EQUAL\\", |
306| |
307|                         \\"severity\\": \\"CRITICAL\\", |
308| |
309|                         \\"closedLoopEventStatus\\": \\"ONSET\\" |
310| |
311|                 }] |
312| |
313|         }, { |
314| |
315|                 \\"eventName\\": \\"Measurement\_vGMUX\\", |
316| |
317|                 \\"controlLoopSchemaType\\": \\"VNF\\", |
318| |
319|                 \\"policyScope\\": \\"DCAE\\", |
320| |
321|                 \\"policyName\\": \\"DCAE.Config\_tca-hi-lo\\", |
322| |
323|                 \\"policyVersion\\": \\"v0.0.1\\", |
324| |
325|                 \\"thresholds\\": [{ |
326| |
327|                         \\"closedLoopControlName\\": \\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\\", |
328| |
329|                         \\"version\\": \\"1.0.2\\", |
330| |
331|                         \\"fieldPath\\": \\"$.event.measurementsForVfScalingFields.additionalMeasurements[\*].arrayOfFields[0].value\\", |
332| |
333|                         \\"thresholdValue\\": 0, |
334| |
335|                         \\"direction\\": \\"EQUAL\\", |
336| |
337|                         \\"severity\\": \\"MAJOR\\", |
338| |
339|                         \\"closedLoopEventStatus\\": \\"ABATED\\" |
340| |
341|                 }, { |
342| |
343|                         \\"closedLoopControlName\\": \\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\\", |
344| |
345|                         \\"version\\": \\"1.0.2\\", |
346| |
347|                         \\"fieldPath\\": \\"$.event.measurementsForVfScalingFields.additionalMeasurements[\*].arrayOfFields[0].value\\", |
348| |
349|                         \\"thresholdValue\\": 0, |
350| |
351|                         \\"direction\\": \\"GREATER\\", |
352| |
353|                         \\"severity\\": \\"CRITICAL\\", |
354| |
355|                         \\"closedLoopEventStatus\\": \\"ONSET\\" |
356| |
357|                 }] |
358| |
359|         }] |
360| |
361| }" |
362| |
363| } |
364+============================================================================================================================================+
365+--------------------------------------------------------------------------------------------------------------------------------------------+
366
367  Note: Dmaap configuration are specified on this file on
368 publisherHostName and subscriberHostName. To be changed as
369 required\*\*
370
3717. Copy below script to CDAP server (this gets latest image from nexus
372 and deploys TCA application) and execute it
373
374+--------------------------------------------------------------------------------------------------------------------------------------------------+
375| #!/bin/sh |
376| |
377| TCA\_JAR=dcae-analytics-cdap-tca-2.0.0.jar |
378| |
379| rm -f /home/ubuntu/$TCA\_JAR |
380| |
381| cd /home/ubuntu/ |
382| |
383| wget https://nexus.onap.org/service/local/repositories/staging/content/org/onap/dcaegen2/analytics/tca/dcae-analytics-cdap-tca/2.0.0/$TCA\_JAR |
384| |
385| if [ $? -eq 0 ]; then |
386| |
387|         if [ -f /home/ubuntu/$TCA\_JAR ]; then |
388| |
389|                 echo "Restarting TCA CDAP application using $TCA\_JAR artifact" |
390| |
391|         else |
392| |
393|                 echo "ERROR: $TCA\_JAR missing" |
394| |
395|                 exit 1 |
396| |
397|         fi |
398| |
399| else |
400| |
401|         echo "ERROR: $TCA\_JAR not found in nexus" |
402| |
403|         exit 1 |
404| |
405| fi |
406| |
407| # stop programs |
408| |
409| curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/stop |
410| |
411| curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/stop |
412| |
413| curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/stop |
414| |
415| # delete application |
416| |
417| curl -X DELETE http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca |
418| |
419| # delete artifact |
420| |
421| curl -X DELETE http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/artifacts/dcae-analytics-cdap-tca/versions/2.0.0 |
422| |
423| # load artifact |
424| |
425| curl -X POST --data-binary @/home/ubuntu/$TCA\_JAR http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/artifacts/dcae-analytics-cdap-tca |
426| |
427| # create app |
428| |
429| curl -X PUT -d @/home/ubuntu/tca\_app\_config.json http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca |
430| |
431| # load preferences |
432| |
433| curl -X PUT -d @/home/ubuntu/tca\_app\_preferences.json http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/preferences |
434| |
435| # start programs |
436| |
437| curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/start |
438| |
439| curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/start |
440| |
441| curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/start |
442| |
443| echo |
444| |
445| # get status of programs |
446| |
447| curl http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/status |
448| |
449| curl http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/status |
450| |
451| curl http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/status |
452| |
453| echo |
454+==================================================================================================================================================+
455+--------------------------------------------------------------------------------------------------------------------------------------------------+
456
4578. Verify TCA application and logs via CDAP GUI processes
458
459 The overall flow can be checked here
460
461TCA Configuration Change
462~~~~~~~~~~~~~~~~~~~~~~~
463
464Typical configuration changes include changing DMAAP host and/or Policy configuration. If necessary, modify the file on step #6 and run the script noted as step #7 to redeploy TCA with updated configuration.
465~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
466
467VM Init
468~~~~~~
469
470To address windriver server in-stability, the below **init.sh** script
471was used to restart the container on VM restart.  This script was
472invoked via VM init script (rc.d).
473
474+------------------------------------------------------------------------------------------------------------------------------+
475| #!/bin/sh |
476| |
477| #docker run -d --name cdap-sdk -p 11011:11011 -p 11015:11015 caskdata/cdap-standalone:4.1.2 |
478| |
479| sudo docker restart cdap-sdk-2 |
480| |
481| sleep 30 |
482| |
483| # start program |
484| |
485| curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/start |
486| |
487| curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/start |
488| |
489| curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/start |
490+==============================================================================================================================+
491+------------------------------------------------------------------------------------------------------------------------------+
492
493 
494
495This script was invoked via VM init script (rc.d).
496
497ln -s /home/ubuntu/init.sh /etc/init.d/init.sh
498
499sudo  update-rc.d init.sh start 2
500