.. This work is licensed under a Creative Commons Attribution 4.0 International License. | |
.. http://creativecommons.org/licenses/by/4.0 | |
.. _walkthrough: | |
Walk-through | |
============ | |
The goal of this quickstart is to provide an overview of the | |
functionalities of the ``dcae-cli`` and walk you through the | |
capabilities: | |
- `Adding data formats <#adding-data-formats>`__ | |
- `Adding component <#adding-component>`__ | |
- `Setting profile <#setting-profile>`__ | |
- `Development and testing <#development-and-testing>`__ | |
- `Publishing component <#publishing-component>`__ | |
- `Shared catalog <#shared-catalog>`__ | |
.. This walk-through uses example projects: COMMENTED OUT FOR NOW TBD | |
.. | |
.. - `laika <ONAP%20URL%20TBD>`__ | |
.. - `CDAP examples <ONAP%20URL%20TBD>`__ | |
.. _adding-data-formats: | |
Adding data formats | |
------------------- | |
``data_format`` is the sub-command that is used to execute operations | |
that manage `data formats <../data-formats.md>`__. | |
:: | |
$ dcae_cli data_format --help | |
Usage: dcae_cli data_format [OPTIONS] COMMAND [ARGS]... | |
Options: | |
--help Show this message and exit. | |
Commands: | |
add Tracks a data format file SPECIFICATION... | |
generate Generates a data format file from JSON input examples | |
list Lists all your data formats | |
publish Publishes data format to make publicly... | |
show Provides more information about FORMAT | |
Your data format must be in the catalog in order to use in the component | |
specification. Check the catalog using the ``data_format list`` | |
sub-command: | |
:: | |
$ dcae_cli data_format list | |
Data formats for mh677g | |
+------+---------+-------------+--------+----------+ | |
| Name | Version | Description | Status | Modified | | |
+------+---------+-------------+--------+----------+ | |
| | | | | | | |
+------+---------+-------------+--------+----------+ | |
The fields ``name``, ``version``, ``description`` are referenced from | |
the data format JSON from the ``self`` JSON. | |
There are no data formats so you must add the data formats that your | |
component specification references. Use the ``data_format add`` | |
sub-command: | |
Here’s an example command: | |
:: | |
dcae_cli data_format add health.json | |
Verify that it was added: | |
:: | |
$ dcae_cli data_format list | |
Data formats for mh677g | |
+-------------------------------+---------+-------------------------------------------+--------+----------------------------+ | |
| Name | Version | Description | Status | Modified | | |
+-------------------------------+---------+-------------------------------------------+--------+----------------------------+ | |
| sandbox.platform.laika.health | 0.1.0 | Data format used for the /health endpoint | staged | 2017-05-23 04:02:38.952799 | | |
+-------------------------------+---------+-------------------------------------------+--------+----------------------------+ | |
Go ahead and add other referenced data formats. | |
If you have JSON input you can generate a data format like this: | |
:: | |
$ dcae_cli data_format --keywords generate myname:1.0.0 myjsoninputfile | |
where ``myname`` is the name of your data format, ``1.0.0`` is an | |
example version, and ``myjsoninputfile`` is an example JSON input file | |
(a directory of input JSON files can also be provided). The | |
``--keywords`` option adds additional data attributes that can be | |
completed to provide a more detailed data characterization. In any event | |
the output should be reviewed for accuracy. The data format is written | |
to stdout. | |
.. _adding-component: | |
Adding component | |
---------------- | |
``component`` is the sub-command that is used to work with operations | |
for components: | |
:: | |
$ dcae_cli component --help | |
Usage: dcae_cli component [OPTIONS] COMMAND [ARGS]... | |
Options: | |
--help Show this message and exit. | |
Commands: | |
add | |
dev Set up component in development for... | |
list Lists components in the public catalog. | |
publish Pushes COMPONENT to the public catalog | |
run Runs the latest version of COMPONENT. | |
show Provides more information about COMPONENT | |
undeploy Undeploys the latest version of COMPONENT. | |
Your component must be accessible from the catalog in order for it to be | |
used. Check the catalog using the ``component list`` sub-command: | |
:: | |
$ dcae_cli component list | |
Active profile: solutioning | |
+------+---------+------+-------------+--------+----------+-----------+ | |
| Name | Version | Type | Description | Status | Modified | #Deployed | | |
+------+---------+------+-------------+--------+----------+-----------+ | |
| | | | | | | | | |
+------+---------+------+-------------+--------+----------+-----------+ | |
Use the "--deployed" option to see more details on deployments | |
The fields ``name``, ``version``, ``type``, ``description`` are | |
referenced from the component specification’s ``self`` JSON. | |
There are no components so you must add your component. Use the | |
``component add`` sub-command. The command is the same for docker and | |
cdap components: | |
:: | |
$ dcae_cli component add --help | |
Usage: dcae_cli component add [OPTIONS] SPECIFICATION | |
Options: | |
--update Updates a locally added component if it has not been already | |
pushed | |
--help Show this message and exit. | |
*Note* use the ``--update`` flag to replace existing staged instances. | |
The ``component dev`` sub-command can be useful in validating and | |
experimenting when crafting your component specification. See details | |
about ``dev`` under `Development and | |
testing <#development-and-testing>`__. | |
Once we add the components laika and helloworld, let’s verify that they | |
got added ok: | |
:: | |
$ dcae_cli component list | |
Active profile: solutioning | |
+-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ | |
| Name | Version | Type | Description | Status | Modified | #Deployed | | |
+-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ | |
| cdap.helloworld.endnode | 0.8.0 | cdap | cdap test component | staged | 2017-05-23 04:14:35.588075 | 0 | | |
| sandbox.platform.laika | 0.5.0 | docker | Web service used as a stand-alone test DCAE service compone.. | staged | 2017-05-23 04:07:44.065610 | 0 | | |
+-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ | |
Use the "--deployed" option to see more details on deployments | |
.. _setting-profile: | |
Setting profile | |
--------------- | |
``profile`` is the sub-command that is used to manage profiles. These | |
profiles contain environment variables used to connect to different | |
environments. This is used in the running and deployment of your | |
component using the ``dcae_cli component run`` command. The ``dcae-cli`` | |
ships with profiles for ``solutioning`` and ``rework``. | |
:: | |
$ dcae_cli profiles --help | |
Usage: dcae_cli profiles [OPTIONS] COMMAND [ARGS]... | |
Options: | |
--help Show this message and exit. | |
Commands: | |
activate Sets profile NAME as the active profile | |
create Creates a new profile NAME initialized with... | |
delete Deletes profile NAME | |
list Lists available profiles | |
set Updates profile NAME such that KEY=VALUE | |
show Prints the profile dictionary | |
To see what variables a profile contains, you can use the ``show`` | |
command, as in ``dcae_cli profiles show PROFILE_NAME`` | |
Use the ``create`` sub-command to create your own profile and assign new | |
values using the ``set`` command. Afterwards you will need to | |
``activate`` the profile you wish to use. First take a look at which | |
profile is active: | |
:: | |
$ dcae_cli profiles list | |
rework | |
* solutioning | |
The active profile is ``solutioning`` so to activate *rework* to use | |
``rework``: | |
:: | |
$ dcae_cli profiles activate rework | |
Check | |
:: | |
$ dcae_cli profiles list | |
* rework | |
solutioning | |
.. _development-and-testing: | |
Development and testing | |
----------------------- | |
The following operations under the sub-command ``component`` are aimed | |
to help developers with testing: | |
- ``run`` | |
- ``undeploy`` | |
- ``dev`` | |
``run`` | |
~~~~~~~ | |
The ``run`` operation is to be used for running your application in its | |
container remotely on the activated environment. Docker containers have | |
the additional option to run locally on your development machine. | |
In order to run your application, you must have added your data formats | |
and your component to your catalog. | |
Let’s verify that your component is in the catalog: | |
:: | |
$ dcae_cli component list | |
Active profile: solutioning | |
+-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ | |
| Name | Version | Type | Description | Status | Modified | #Deployed | | |
+-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ | |
| cdap.helloworld.endnode | 0.8.0 | cdap | cdap test component | staged | 2017-05-23 04:14:35.588075 | 0 | | |
| sandbox.platform.laika | 0.5.0 | docker | Web service used as a stand-alone test DCAE service compone.. | staged | 2017-05-23 04:07:44.065610 | 0 | | |
+-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ | |
Use the "--deployed" option to see more details on deployments | |
Docker | |
^^^^^^ | |
**NOTE** Make sure your Docker image has been uploaded to the shared | |
registry. | |
For Docker containers, you can run either attached or unattached. | |
Attached means that the dcae-cli tool will launch the container and not | |
terminate. The dcae-cli while attached will stream in the logs of the | |
Docker container. Doing a Ctrl-C will terminate the run session which | |
means undeploy your container and force a clean up automatically. | |
Running unattached means simply deploy your container. You will need to | |
execute ``undeploy`` when you are done testing. #### CDAP | |
**NOTE** Make sure your CDAP jar has been uploaded to Nexus. | |
TODO | |
``undeploy`` | |
~~~~~~~~~~~~ | |
The ``undeploy`` operation is to be used to undeploy any instances of a | |
specified component, version that you have deployed. This includes | |
cleaning up of configuration. | |
Let’s undeploy ``sandbox.platform.laika`` that was deployed from the | |
previous section: | |
:: | |
$ dcae_cli component undeploy sandbox.platform.laika:0.5.0 | |
DCAE.Undeploy | WARNING | Undeploying components: 1 | |
DCAE.Undeploy | WARNING | Undeployed components: 1 | |
.. _walkthrough-dev: | |
``dev`` | |
~~~~~~~ | |
The ``dev`` operation is a convenient operation that can be useful for | |
the development and testing of your component. It can be used to: | |
- Help validate your experimental component specification before | |
uploading to the catalog | |
- Generate the application configuration from the component | |
specification and make it available in a test environment. This | |
allows you to view your resulting configuration for local development | |
and to help debug potential related issues. | |
Let’s say you have a component specification called | |
``component-spec.json``: | |
:: | |
$ dcae_cli component dev component-spec.json | |
Ready for component development | |
Setup these environment varibles. Run "source env_solutioning": | |
export DOCKER_HOST=SOME_DOCKER_HOST:2376 | |
export SERVICE_CHECK_INTERVAL=15s | |
export CONFIG_BINDING_SERVICE=config_binding_service | |
export HOSTNAME=mh677g.95740959-63d2-492a-b964-62a6dce2591d.0-6-0.sandbox-platform-laika | |
export CONSUL_HOST=SOME_CONSUL_HOST | |
export CDAP_BROKER=cdap_broker | |
export SERVICE_NAME=mh677g.95740959-63d2-492a-b964-62a6dce2591d.0-6-0.sandbox-platform-laika | |
export SERVICE_CHECK_TIMEOUT=1s | |
export SERVICE_CHECK_HTTP=/health | |
Press any key to stop and to clean up | |
Your application configuration is now available under the name | |
``mh677g.95740959-63d2-492a-b964-62a6dce2591d.0-6-0.sandbox-platform-laika``. | |
To view the resulting configuration, you can ``curl`` a request to the | |
config binding service or programmatically fetch your configuration | |
within your application. | |
You need to first query Consul to get the ip and port of config binding | |
service: | |
:: | |
curl http://$CONSUL_HOST:8500/v1/catalog/service/$CONFIG_BINDING_SERVICE | |
[ | |
{ | |
"ID": "983d5c94-c508-4a8a-9be3-5912bd09786b", | |
"Node": "realsolcnsl00", | |
"Address": "10.226.1.22", | |
"TaggedAddresses": { | |
"lan": "10.226.1.22", | |
"wan": "10.226.1.22" | |
}, | |
"NodeMeta": {}, | |
"ServiceID": "5f371f295c90:config_binding_service:10000", | |
"ServiceName": "config_binding_service", | |
"ServiceTags": [], | |
"ServiceAddress": "XXXX", | |
"ServicePort": 32770, | |
"ServiceEnableTagOverride": false, | |
"CreateIndex": 487, | |
"ModifyIndex": 487 | |
} | |
] | |
.. _dmaap-testing: | |
DMaaP testing | |
~~~~~~~~~~~~~ | |
Currently, the dcae-cli does not have the capability of provisioning | |
topics. In order to do testing with message router topics or with data | |
router feeds, the developer must provision the topic or the feed | |
manually and provide the connection details in the form of a JSON in a | |
file to the dcae-cli. This file is to be passed in when using the | |
``run`` and ``dev`` commands with the option ``--dmaap-file``. | |
The structure of the DMaaP JSON is an object of config keys to matching | |
topic or feed connection details. Config keys are the ``config_key`` | |
values specified in your component specification streams section where | |
the streams must be type message router or data router. Information | |
about the associated connection details can be found on `this | |
page <dmaap-connection-objects.md>`__. Please check it out. | |
For example, if you have a component specification that has the | |
following streams entry: | |
.. code:: json | |
"streams": { | |
"publishes": [{ | |
"format": "ves", | |
"version": "1.0.0", | |
"type": "message router", | |
"config_key": "ves_connection" | |
}] | |
} | |
Then to deploy and to run your component, you must use the | |
``--dmaap-file`` command and pass in a JSON that looks like: | |
.. code:: json | |
{ | |
"ves_connection": { | |
"type": "message_router", | |
"dmaap_info": { | |
"topic_url": "https://we-are-message-router.us:3905/events/some-topic" | |
} | |
} | |
} | |
The provided DMaaP JSON is used to simulate the output of provisioning | |
and will be used to merge with the generated application configuration | |
at runtime. | |
Your final application config will look like: | |
.. code:: json | |
{ | |
"streams_publishes": { | |
"ves_connection": { | |
"type": "message_router", | |
"dmaap_info": { | |
"topic_url": "https://we-are-message-router.us:3905/events/some-topic" | |
} | |
} | |
} | |
} | |
Data router subscribers | |
^^^^^^^^^^^^^^^^^^^^^^^ | |
Note for data router subscriber testing, you will need the delivery url | |
in order to provision the subscriber to the feed. This is constructed at | |
deployment time and will be provided by the dcae-cli after you deploy | |
your component. The delivery urls will be displayed to the screen: | |
:: | |
DCAE.Run | WARNING | Your component is a data router subscriber. Here are the delivery urls: | |
some-sub-dr: http://SOME_IP:32838/identity | |
*Sourced at deployment* testing | |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
Components may have configuration parameters whose values are to be | |
sourced at deployment time. For example, there are components whose | |
configuration parameters are to come from DTI events which are only | |
available when the component is getting deployed. These configuration | |
parameters must be setup correctly in the `component | |
specification <http://localhost:8000/components/component-specification/docker-specification/#configuration-parameters>`__ | |
by setting the property ``sourced_at_deployment`` to ``true`` for each | |
and every parameter that is expected to come in at deployment time. | |
Once your component specification has been updated correctly, you must | |
use the ``--inputs-file`` command-line argument when running the | |
commands ``dev`` or ``run`` with your component. This is to simulate | |
providing the dynamic, deployment time values for those parameters | |
marked as ``sourced_at_deployment``. | |
For example, if your component specification has the following | |
configuration parameters: | |
:: | |
"parameters": [{ | |
"name": "vnf-ip", | |
"value": "", | |
"sourced_at_deployment": true | |
}, | |
{ | |
"name": "static-param", | |
"value": 5 | |
}] | |
You would have to pass in an inputs file that looks like: | |
:: | |
{ | |
"vnf-ip": "10.100.1.100" | |
} | |
Your application configuration would look like: | |
:: | |
{ | |
"vnf-ip": "10.100.1.100", | |
"static-param": 5 | |
} | |
Publishing component | |
-------------------- | |
Once components have their component specifications crafted and | |
validated and have been tested, components should be published in the | |
shared onboarding catalog using the ``publish`` sub-command for both | |
data formats and components. You must publish all data formats of a | |
component before publishing a component. | |
Publishing will change the status of a component, be made accessible for | |
other developers to use, and will generate the associated TOSCA models | |
for use in designing of compositions. | |
:: | |
dcae_cli component publish sandbox.platform.laika:0.5.0 | |
Shared catalog | |
-------------- | |
``catalog`` is the sub-command used to access and to browse the shared | |
onboarding catalog to view components and data formats that have been | |
published and that are being worked on. Components and data formats have | |
two statuses ``staged`` and ``published``. | |
Staged means that the resource has been simply added and is under | |
development. It is to be used only by the owner. Published means that | |
the resource has been fully developed and tested and is ready to be | |
shared. | |
Published components can be deployed by non-owners and published data | |
formats can be used in component specifications of non-owners. | |
There are two available operations: | |
:: | |
$ dcae_cli catalog --help | |
Usage: dcae_cli catalog [OPTIONS] COMMAND [ARGS]... | |
Options: | |
--help Show this message and exit. | |
Commands: | |
list | |
show | |
Staged components can be viewed under the ``list`` operation using the | |
``--expanded`` flag. |