Gitiles
Code Review Sign In
gerrit.nordix.org / onap / dcaegen2 / 7f24862e7bee1385b5e240f371be0e944f306c3a / . / docs / sections / services / ves-http / stnd-defined-validation.rst
blob: 932b1ec51d1c9fd84e67891fc9c9e7999d8a3feb [file] [log] [blame]
Michal Banka9a7a9312020-08-24 11:01:56 +0200[diff] [blame]1.. This work is licensed under a Creative Commons Attribution 4.0 International License.
2.. http://creativecommons.org/licenses/by/4.0
3
4StndDefined Events Collection Mechanism
5=======================================
6
7Description
8-----------
9
10This mechanism can be used to validate any JSON content incoming as JsonNode using OpenAPI standardized schemas.
11During validation externally located schemas are mapped to local schema files.
12
13Validated JSON must have one field that will refer to an external schema, which will be mapped to local file and then
14validation of any chosen part of JSON is executed using local schema.
15
16StndDefined validation is integrated with the event collecting functionality available under the endpoint
17*/eventListener/v7*. Process of event collecting includes steps in the following order:
18
191. General event validation (1st stage validation)
202. Event transformation
213. **StndDefined event validation** (2nd stage validation)
224. Event routing to DMaaP
23
24Mapping file is cached on stndDefined validator creation, so it's not read every time validation is performed.
25Schemas' content couldn't be cached due to an external library restrictions (OpenAPI4j).
26
27The value of the 'stndDefinedNamespace' field in any incoming stndDefined JSON event is used to match the topic from
28property *collector.dmaap.streamid*.
29
30StndDefined properties
31----------------------
32
33There are 5 additional properties related to stndDefined validation in collector.properties file.
34
Michal Banka7f248622020-09-09 16:38:37 +0200[diff] [blame^]35+----------------------------------------------+--------------------------------------------------------------------------------+--------------------------------------------------------------------------------+
36| Name | Description | Example |
37+==============================================+================================================================================+================================================================================+
38| collector.externalSchema.checkflag | Flag is responsible for turning on/off stndDefined data validation. | -1 or 1 |
39| | By default this flag is set to 1, which means that the validation is enabled. | |
40| | In case flag is set to -1, validation is disabled. | |
41+----------------------------------------------+--------------------------------------------------------------------------------+--------------------------------------------------------------------------------+
42| collector.externalSchema.mappingFileLocation | This should be a local filesystem path to file with mappings of public URLs | etc/externalRepo/schema-map.json |
43| | to local URLs. | |
44+----------------------------------------------+--------------------------------------------------------------------------------+--------------------------------------------------------------------------------+
45| collector.externalSchema.schemasLocation | Schemas location is a directory under which stndDefined validator will search | ./etc/externalRepo/ and first mapping from example mapping file below is taken,|
46| | for local schemas. | validator will look for schema under the path: |
47| | | ./etc/externalRepo/3gpp/rep/sa5/data-models/blob/REL-16/OpenAPI/faultMnS.yaml |
48+----------------------------------------------+--------------------------------------------------------------------------------+--------------------------------------------------------------------------------+
49| event.externalSchema.schemaRefPath | This is an internal path from validated JSON. It should define which field | $.event.stndDefinedFields.schemaReference |
50| | will be taken as public schema reference, which is later mapped. | |
51+----------------------------------------------+--------------------------------------------------------------------------------+--------------------------------------------------------------------------------+
52| event.externalSchema.stndDefinedDataPath | This is internal path from validated JSON. | $.event.stndDefinedFields.data |
53| | It should define which field will be validated. | |
54+----------------------------------------------+--------------------------------------------------------------------------------+--------------------------------------------------------------------------------+
Michal Banka9a7a9312020-08-24 11:01:56 +0200[diff] [blame]55
56Format of the schema mapping file is a JSON file with list of mappings, as shown in the example below.
57
58.. code-block:: json
59
60 [
Michal Banka7f248622020-09-09 16:38:37 +0200[diff] [blame^]61 {
62 "publicURL": "https://forge.3gpp.org/rep/sa5/data-models/tree/SA88-Rel16/OpenAPI/faultMnS.yaml",
63 "localURL": "3gpp/rep/sa5/data-models/blob/REL-16/OpenAPI/faultMnS.yaml"
64 },
65 {
66 "publicURL": "https://forge.3gpp.org/rep/sa5/data-models/tree/SA88-Rel16/OpenAPI/heartbeatNtf.yaml",
67 "localURL": "3gpp/rep/sa5/data-models/blob/REL-16/OpenAPI/heartbeatNtf.yaml"
68 },
69 {
70 "publicURL": "https://forge.3gpp.org/rep/sa5/data-models/tree/SA88-Rel16/OpenAPI/PerDataFileReportMnS.yaml",
71 "localURL": "3gpp/rep/sa5/data-models/blob/REL-16/OpenAPI/PerDataFileReportMnS.yaml"
72 },
73 {
74 "publicURL": "https://forge.3gpp.org/rep/sa5/data-models/tree/SA88-Rel16/OpenAPI/provMnS.yaml",
75 "localURL": "3gpp/rep/sa5/data-models/blob/REL-16/OpenAPI/provMnS.yaml"
76 }
Michal Banka9a7a9312020-08-24 11:01:56 +0200[diff] [blame]77 ]
78
Michal Banka7f248622020-09-09 16:38:37 +0200[diff] [blame^]79Collector.properties content may be overridden when deploying VES Collector via Cloudify. To keep VES settings consistent
80listed above properties has been updated in the VES Collector Cloudify blueprint (in blueprints/k8s-ves.yaml file under
81dcaegen2/platform/blueprints project) and in componentspec file (in dpo/spec/vescollector-componentspec.json file
82in VES project) which may be used for generation of VES Collector Cloudify blueprints in some scenarios.
83
Michal Banka9a7a9312020-08-24 11:01:56 +0200[diff] [blame]84
85Requirements for stndDefined validation
86---------------------------------------
87
88To run stndDefined validation, both *collector.schema.checkflag* and *collector.externalSchema.checkflag* must be set to 1.
89
90Despite the flag set, the validation will not start when:
91
92- Domain of the incoming event is not 'stndDefined'.
93- General event validation (1st stage) failed.
94- Field of event referenced under the property *event.externalSchema.schemaRefPath* (by default */event/stndDefinedFields/schemaReference*):
95 - Has an empty value.
96 - Does not exist in the incoming event.
97
98Validation scenarios
99--------------------
100
101Positive scenario, which returns 202 Accepted HTTP code after successful stndDefined validation:
102
103- *collector.schema.checkflag* and *collector.externalSchema.checkflag* is set to 1
104- Mapping file has valid format
105- Schema file mapped from referenced in the event is valid
106- The incoming event is valid against schema
107
108Below are scenarios when, the stndDefined validation will end with failure and return 400 Bad Request HTTP code:
109
110- One of stndDefined data fields has wrong type or value
111- StndDefined data has an empty body or is missing required field
112- Field of event referenced under the property event.externalSchema.schemaRefPath has publicURL which is not mapped in the schemas mappings
113- Field defining public schema in event (by default */event/stndDefinedFields/schemaReference*) after "#" has non existing reference in schema file
114
115Schemas repository description
116------------------------------
117
118Schemas and mapping file location might be configured to any local directory through properties in collector.properties
119as described in 'StndDefined properties' section.
120
121By default schemas repository is located under *etc/externalSchema* directory, as well as schemas mapping file called
122*schema-map.json*. There are files stored in the project repository which are schemas from 3GPP organisation. Every
123organisation which adds or mounts external schemas should store them in folder named by organisation
124name. Further folders structure may be whatever as long as schemas are correctly referenced in the mapping file.
125
126Sample directory tree of *etc* directory:
127
128.. code-block:: text
129
130 etc
131 ├── ...
132 └── externalRepo
133 ├── schema-map.json
134 └── 3gpp
135 └── rep
136 └── sa5
137 └── data-models
138 └── blob
139 └── REL-16
140 └── OpenAPI
141 ├── faultMnS.yaml
142 ├── heartbeatNtf.yaml
143 ├── PerDataFileReportMnS.yaml
144 └── provMnS.yaml
145
146Routing of stndDefined domain events
147------------------------------------
148
149All events, expect those with 'stndDefined' domain, are routed to DMaaP topics basing on domain value. Events with
150'stndDefined' domain are sent to proper topic basing on field 'stndDefinedNamespace'.
151
152This is the only difference from standard event routing, specific for 'stndDefined' domain. As in every other event
153routing value is being mapped for specific DMaaP stream. Stream ID to DMaaP channels mappings are located in
154*etc/collector.properties* file under property *collector.dmaap.streamid*. Channels descriptions are in
155*etc/DmaapConfig.json*, where destination DMaaP topics are selected.
156
157With stndDefined domain managment 4 new mappings were added. Their routing has been described in the table below:
158
159 +---------------------------+--------------------------------+------------------------------------------------------+
160 | Stream ID | Channel | DMaaP Stream |
161 +===========================+================================+======================================================+
162 | 3GPP-FaultSupervision | ves-3gpp-fault-supervision | unauthenticated.SEC_3GPP_FAULTSUPERVISION_OUTPUT |
163 +---------------------------+--------------------------------+------------------------------------------------------+
164 | 3GPP-Heartbeat | ves-3gpp-heartbeat | unauthenticated.SEC_3GPP_HEARTBEAT_OUTPUT |
165 +---------------------------+--------------------------------+------------------------------------------------------+
166 | 3GPP-Provisioning | ves-3gpp-provisioning | unauthenticated.SEC_3GPP_PROVISIONING_OUTPUT |
167 +---------------------------+--------------------------------+------------------------------------------------------+
168 | 3GPP-PerformanceAssurance | ves-3gpp-performance-assurance | unauthenticated.SEC_3GPP_PERFORMANCEASSURANCE_OUTPUT |
169 +---------------------------+--------------------------------+------------------------------------------------------+
170
171
172Error scenarios behaviour
173-------------------------
174
175There are few error scenarios described in 'Validation scenarios' section. This section will describe user point of view
176of VES Collector behaviour when they happen. Messages returned as HTTP response contain data described below for each
177scenario.
178
1791. StndDefined fields validation related errors
180
1811.1. Schema file referred under the path from property *event.externalSchema.schemaRefPath* (by default */event/stndDefinedFields/schemaReference*) not present in the schema repository.
182
183 +---------------------+------------------------------------------------------------------+
184 | Property Name | Property Description |
185 +=====================+==================================================================+
186 | MessageId | SVC2004 |
187 +---------------------+------------------------------------------------------------------+
188 | Text | "Invalid input value for %1 %2: %3" |
189 +---------------------+------------------------------------------------------------------+
190 | Variables | %1 – “attribute” |
191 | | %2 – "event.stndDefinedFields.schemaReference" |
192 | | %3 – "Referred external schema not present in schema repository" |
193 +---------------------+------------------------------------------------------------------+
194 | HTTP status code(s) | 400 Bad request |
195 +---------------------+------------------------------------------------------------------+
196
1971.2. File referred under the path from property *event.externalSchema.schemaRefPath* (by default */event/stndDefinedFields/schemaReference*) exists, but internal reference (part of URL after #) is incorrect.
198
199 +---------------------+-----------------------------------------------------------------------------------------------------------------------------------+
200 | Property Name | Property Description |
201 +=====================+===================================================================================================================================+
202 | MessageId | SVC2000 |
203 +---------------------+-----------------------------------------------------------------------------------------------------------------------------------+
204 | Text | The following service error occurred: %1. Error code is %2 |
205 +---------------------+-----------------------------------------------------------------------------------------------------------------------------------+
206 | Variables | %1 - "event.stndDefinedFields.schemaReference value does not correspond to any external event schema file in externalSchema repo" |
207 | | %2 - "400" |
208 +---------------------+-----------------------------------------------------------------------------------------------------------------------------------+
209 | HTTP status code(s) | 400 Bad request |
210 +---------------------+-----------------------------------------------------------------------------------------------------------------------------------+
211
2121.3. StndDefined validation executed, but event contents do not validate with referenced schema.
213
214 +---------------------+---------------------------------------------------------------------------------------------+
215 | Property Name | Property Description |
216 +=====================+=============================================================================================+
217 | MessageId | SVC2000 |
218 +---------------------+---------------------------------------------------------------------------------------------+
219 | Text | The following service error occurred: %1. Error code is %2 |
220 +---------------------+---------------------------------------------------------------------------------------------+
221 | Variables | %1 - "event.stndDefinedFields.data invalid against event.stndDefinedFields.schemaReference" |
222 | | %2 - "400" |
223 +---------------------+---------------------------------------------------------------------------------------------+
224 | HTTP status code(s) | 400 Bad request |
225 +---------------------+---------------------------------------------------------------------------------------------+
226
2272. Problems with routing of stndDefined domain.
228
2292.1. StndDefinedNamespace field not received in the incoming event.
230
231 +---------------------+-----------------------------------------------------+
232 | Property Name | Property Description |
233 +=====================+=====================================================+
234 | MessageId | SVC2006 |
235 +---------------------+-----------------------------------------------------+
236 | Text | Mandatory input %1 %2 is missing from request |
237 +---------------------+-----------------------------------------------------+
238 | Variables | %1 – “attribute” |
239 | | %2 – "event.commonEventHeader.stndDefinedNamespace" |
240 +---------------------+-----------------------------------------------------+
241 | HTTP status code(s) | 400 Bad Request |
242 +---------------------+-----------------------------------------------------+
243
2442.2. StndDefinedNamespace field present, but value is empty.
245
246 +---------------------+-----------------------------------------------------+
247 | Property Name | Property Description |
248 +=====================+=====================================================+
249 | MessageId | SVC2006 |
250 +---------------------+-----------------------------------------------------+
251 | Text | Mandatory input %1 %2 is empty in request |
252 +---------------------+-----------------------------------------------------+
253 | Variables | %1 – “attribute” |
254 | | %2 – "event.commonEventHeader.stndDefinedNamespace" |
255 +---------------------+-----------------------------------------------------+
256 | HTTP status code(s) | 400 Bad Request |
257 +---------------------+-----------------------------------------------------+
258
2592.3. StndDefinedNamespace field present, but value doesn't match any stream ID mapping.
260
261 +---------------------+-------------------------------------------------------------------------------------------------------------------------------------------+
262 | Property Name | Property Description |
263 +=====================+===========================================================================================================================================+
264 | MessageId | SVC2004 |
265 +---------------------+-------------------------------------------------------------------------------------------------------------------------------------------+
266 | Text | "Invalid input value for %1 %2: %3" |
267 +---------------------+-------------------------------------------------------------------------------------------------------------------------------------------+
268 | Variables | %1 – “attribute” |
269 | | %2 – "event.commonEventHeader.stndDefinedNamespace" |
270 | | %3 – "stndDefinedNamespace received not present in VES Collector routing configuration. Unable to route event to appropriate DMaaP topic" |
271 +---------------------+-------------------------------------------------------------------------------------------------------------------------------------------+
272 | HTTP status code(s) | 400 Bad request |
273 +---------------------+-------------------------------------------------------------------------------------------------------------------------------------------+
274
275
276