blob: 05c8251019fe3d6ae2cb777bc0199e78834d2df0 [file] [log] [blame]
JakobKriegf47eeee2020-08-17 21:04:35 +02001.. This work is a derivative of https://wiki.onap.org/display/DW/Running+Blueprints+Processor+Microservice+in+an+IDE
2.. This work is licensed under a Creative Commons Attribution 4.0
3.. International License. http://creativecommons.org/licenses/by/4.0
4.. Copyright (C) 2020 Deutsche Telekom AG.
5
6Running Blueprints Processor Microservice in an IDE
7====================================================
8
9Objective
10~~~~~~~~~~~~
11
12Have the processor running locally is to use the IDE to run the code, while having the database running in a container.
13This way, code changes can be conveniently tested and debugged.
14
15
16Check out the code
17~~~~~~~~~~~~~~~~~~~
18
19Check out the code from Gerrit: https://gerrit.onap.org/r/#/admin/projects/ccsdk/cds
20
21Build it locally
22~~~~~~~~~~~~~~~~~~
23
24In the checked out directory, type
25
26.. code-block:: bash
27
28 mvn clean install -DskipTests=true -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -Dadditionalparam=-Xdoclint:none
29
30Spin up a Docker container with the database
31~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
32
33The Blueprints Processor project uses a database to store information about the blueprints
34and therefore it needs to be online before attempting to run it.
35
36One way to create the database is by using the :file:`docker-compose.yaml`
37file present on the distribution module. This database will require a local directory to mount a volume, before running docker-compose remember to create following directory:
38
39.. code-block:: bash
40
41 mkdir -p -m 755 /opt/app/cds/mysql/data
42
43Navigate to the docker-compose file in the distribution module:
44
45.. tabs::
46
47 .. group-tab:: Latest
48
49 .. code-block:: bash
50
51 cd ms/blueprintsprocessor/application/src/main/dc
52
53 .. group-tab:: Frankfurt
54
55 .. code-block:: bash
56
57 cd ms/blueprintsprocessor/application/src/main/dc
58
59 .. group-tab:: El Alto
60
61 .. code-block:: bash
62
63 ms/blueprintsprocessor/distribution/src/main/dc
64
65 .. group-tab:: Dublin
66
67 .. code-block:: bash
68
69 ms/blueprintsprocessor/distribution/src/main/dc
70
71And run docker-composer:
72
73.. code-block:: bash
74
75 docker-compose up -d db
76
77This should spin up a container of the MariaDB image in the background.
78To check if it has worked, this command can be used:
79
80.. code-block:: bash
81
82 docker-compose logs -f
83
84The phrase ``mysqld: ready for connections`` indicates that the database was started correctly.
85
86From now on, the Docker container will be available on the computer; if it ever gets stopped,
87it can be started again by the command:
88
89.. code-block:: bash
90
91 docker start <id of mariadb container>
92
93
94Set permissions on the local file system
95~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
96
97Blueprints processor uses the local file system for some operations and, therefore,
98need some existing and accessible paths to run properly.
99
100Execute the following commands to create the needed directories, and grant access to the current user to modify them:
101
102.. code-block:: bash
103
104 mkdir -p -m 755 /opt/app/onap/blueprints/archive
105 mkdir -p -m 755 /opt/app/onap/blueprints/deploy
106 mkdir -p -m 755 /opt/app/onap/scripts
107 sudo chown -R $(id -u):$(id -g) /opt/app/onap/
108
109Import the project on the IDE
110~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
111
112For this page, the chosen IDE is IntelliJ, but the instructions present here can
113be adapted to be used in other IDEs as well.
114
115Go to *File | Open* and choose the :file:`pom.xml` file of the cds directory:
116
117|imageImportProject|
118
119Sometimes it may be necessary to reimport Maven project:
120
121|imageReimportMaven|
122
123
124Override some application properties
125~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
126
127After the project is compiled, a Run Configuration profile overriding some application properties
128with custom values needs to be created, to reflect the local environment characteristics.
129
130.. tabs::
131
132 .. group-tab:: Latest
133
134 Navigate to the main class of the Blueprints Processor, the BlueprintProcessorApplication class:
135
136 ``ms/blueprintsprocessor/application/src/main/kotlin/org/onap/ccsdk/cds/blueprintsprocessor/BlueprintProcessorApplication.kt``.
137
138 Right-click inside it, at any point, to load the context menu and select create
139 BlueprintProcessorApplication configuration from context:
140
141 |imageCreateRunConfigkt|
142
143 **The following window will open:**
144
145 |imageRunConfigKt|
146
147 **Add the following in the field `VM Options`:**
148
149 .. code-block:: bash
150 :caption: **Custom values for properties**
151
152 -Dspring.profiles.active=dev
153
154 You can override any value from **application-dev.properties** file here. Use the following pattern:
155
156 .. code-block:: java
157
158 -D<application-dev.properties key>=<application-dev.properties value>
159
160 .. group-tab:: Frankfurt
161
162 Navigate to the main class of the Blueprints Processor, the BlueprintProcessorApplication class:
163
164 ``ms/blueprintsprocessor/application/src/main/kotlin/org/onap/ccsdk/cds/blueprintsprocessor/BlueprintProcessorApplication.kt``.
165
166 Right-click inside it, at any point, to load the context menu and select create
167 BlueprintProcessorApplication configuration from context:
168
169 |imageCreateRunConfigkt|
170
171 **The following window will open:**
172
173 |imageRunConfigKt|
174
175 **Add the following in the field `VM Options`:**
176
177 .. code-block:: bash
178 :caption: **Custom values for properties**
179
180 -Dspring.profiles.active=dev
181
182 You can override any value from **application-dev.properties** file here. Use the following pattern:
183
184 .. code-block:: java
185
186 -D<application-dev.properties key>=<application-dev.properties value>
187
188 .. group-tab:: El Alto
189
190 Navigate to the main class of the Blueprints Processor, the BlueprintProcessorApplication class:
191
192 ``ms/blueprintsprocessor/application/src/main/java/org/onap/ccsdk/cds/blueprintsprocessor/BlueprintProcessorApplication.java.``
193
194 Right-click inside it, at any point, to load the context menu and select create
195 BlueprintProcessorApplication configuration from context:
196
197 |imageCreateRunConfigJava|
198
199 **The following window will open:**
200
201 |imageRunConfigJava|
202
203 **Add the following in the field `VM Options`:**
204
205 .. code-block:: bash
206 :caption: **Custom values for properties**
207
208 -Dspring.profiles.active=dev
209
210 You can override any value from **application-dev.properties** file here. Use the following pattern:
211
212 .. code-block:: java
213
214 -D<application-dev.properties key>=<application-dev.properties value>
215
216 .. group-tab:: Dublin
217
218 Navigate to the main class of the Blueprints Processor, the BlueprintProcessorApplication class:
219
220 ``ms/blueprintsprocessor/application/src/main/java/org/onap/ccsdk/cds/blueprintsprocessor/BlueprintProcessorApplication.java``.
221
222 Right-click inside it, at any point, to load the context menu and select create
223 BlueprintProcessorApplication configuration from context:
224
225 |imageCreateRunConfigJava|
226
227 **The following window will open:**
228
229 |imageRunConfigJava|
230
231 **Add the following in that field:**
232
233 .. code-block:: java
234 :caption: **Custom values for properties**
235
236 -DappName=ControllerBluePrints
237 -Dms_name=org.onap.ccsdk.apps.controllerblueprints
238 -DappVersion=1.0.0
239 -Dspring.config.location=opt/app/onap/config/
240 -Dspring.datasource.url=jdbc:mysql://127.0.0.1:3306/sdnctl
241 -Dspring.datasource.username=sdnctl
242 -Dspring.datasource.password=sdnctl
243 -Dcontrollerblueprints.loadInitialData=true
244 -Dblueprintsprocessor.restclient.sdncodl.url=http://localhost:8282/
245 -Dblueprintsprocessor.db.primary.url=jdbc:mysql://localhost:3306/sdnctl
246 -Dblueprintsprocessor.db.primary.username=sdnctl
247 -Dblueprintsprocessor.db.primary.password=sdnctl
248 -Dblueprintsprocessor.db.primary.driverClassName=org.mariadb.jdbc.Driver
249 -Dblueprintsprocessor.db.primary.hibernateHbm2ddlAuto=update
250 -Dblueprintsprocessor.db.primary.hibernateDDLAuto=none
251 -Dblueprintsprocessor.db.primary.hibernateNamingStrategy=org.hibernate.cfg.ImprovedNamingStrategy
252 -Dblueprintsprocessor.db.primary.hibernateDialect=org.hibernate.dialect.MySQL5InnoDBDialect
253 -Dblueprints.processor.functions.python.executor.executionPath=./components/scripts/python/ccsdk_blueprints
254 -Dblueprints.processor.functions.python.executor.modulePaths=./components/scripts/python/ccsdk_blueprints,./components/scripts/python/ccsdk_netconf,./components/scripts/python/ccsdk_restconf
255 -Dblueprintsprocessor.restconfEnabled=true
256 -Dblueprintsprocessor.restclient.sdncodl.type=basic-auth
257 -Dblueprintsprocessor.restclient.sdncodl.url=http://localhost:8282/
258 -Dblueprintsprocessor.restclient.sdncodl.username=admin
259 -Dblueprintsprocessor.restclient.sdncodl.password=Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U
260 -Dblueprintsprocessor.grpcEnable=false
261 -Dblueprintsprocessor.grpcPort=9111
262 -Dblueprintsprocessor.blueprintDeployPath=/opt/app/onap/blueprints/deploy
263 -Dblueprintsprocessor.blueprintArchivePath=/opt/app/onap/blueprints/archive
264 -Dblueprintsprocessor.blueprintWorkingPath=/opt/app/onap/blueprints/work
265 -Dsecurity.user.password={bcrypt}$2a$10$duaUzVUVW0YPQCSIbGEkQOXwafZGwQ/b32/Ys4R1iwSSawFgz7QNu
266 -Dsecurity.user.name=ccsdkapps
267 -Dblueprintsprocessor.messageclient.self-service-api.kafkaEnable=false
268 -Dblueprintsprocessor.messageclient.self-service-api.topic=producer.t
269 -Dblueprintsprocessor.messageclient.self-service-api.type=kafka-basic-auth
270 -Dblueprintsprocessor.messageclient.self-service-api.bootstrapServers=127.0.0.1:9092
271 -Dblueprintsprocessor.messageclient.self-service-api.consumerTopic=receiver.t
272 -Dblueprintsprocessor.messageclient.self-service-api.groupId=receiver-id
273 -Dblueprintsprocessor.messageclient.self-service-api.clientId=default-client-id
274 -Dspring.profiles.active=dev
275 -Dblueprintsprocessor.httpPort=8080
276 -Dserver.port=55555
277
278
279Add/replace the following in Blueprint's application-dev.properties file:
280~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
281
282.. code-block:: java
283
284 blueprintsprocessor.grpcclient.remote-python.type=token-auth
285 blueprintsprocessor.grpcclient.remote-python.host=localhost
286 blueprintsprocessor.grpcclient.remote-python.port=50051
287 blueprintsprocessor.grpcclient.remote-python.token=Basic Y2NzZGthcHBzOmNjc2RrYXBwcw==
288
289 blueprintprocessor.remoteScriptCommand.enabled=true
290
291
292Run the application
293~~~~~~~~~~~~~~~~~~~
294
295Select either run or debug for this Run Configuration to start the Blueprints Processor:
296
297|imageRunDebug|
298
299|imageBuildLogs|
300
301
302Testing the application
303~~~~~~~~~~~~~~~~~~~~~~~~~
304
305There are two main features of the Blueprints Processor that can be of interest of a developer:
306blueprint upload and blueprint process.
307
308To upload custom blueprints, the endpoint ``api/v1/execution-service/upload`` is used.
309
310To process, the endpoint is ``api/v1/execution-service/process``.
311
312Postman is a software that can be used to send these request, and an example of
313them is present on https://www.getpostman.com/collections/b99863b0cde7565a32fc.
314
315
316Possible Fixes
317~~~~~~~~~~~~~~~~~~~
318
319Imported packages or annotiations are not found, Run Config not available?
320*****************************************************************************
321
3221. Rebuild with ``maven install ...`` (see above)
3232. Potentially change Maven home directory in Settings
3243. Maven reimport in IDE
325
326Compilation error?
327********************
328
329* Change Java Version to 11 or 8
330
331.. |imageRunConfigJava| image:: media/run_config_java.png
332 :scale: 100 %
333 :align: middle
334
335.. |imageRunConfigKt| image:: media/run_config_kt.png
336 :scale: 100 %
337 :align: middle
338
339.. |imageCreateRunConfigJava| image:: media/create_run_config_java.png
340 :scale: 100 %
341 :align: middle
342
343.. |imageCreateRunConfigKt| image:: media/create_run_config_kt.png
344 :scale: 100 %
345 :align: middle
346
347.. |imageImportProject| image:: media/import_project.png
348 :scale: 100 %
349 :align: middle
350
351.. |imageReimportMaven| image:: media/reimport_maven.png
352 :scale: 70 %
353 :align: middle
354
355.. |imageRunDebug| image:: media/run_debug.png
356 :scale: 100 %
357 :align: middle
358
359.. |imageRunDebug| image:: media/run_debug.png
360 :align: middle
361 :scale: 100 %
362
363.. |imageBuildLogs| image:: media/build_logs.png
364 :align: middle
365 :scale: 100 %