| .. |
| .. Copyright (c) 2019 AT&T Intellectual Property. |
| .. Copyright (c) 2019 Nokia. |
| .. |
| .. Licensed under the Creative Commons Attribution 4.0 International |
| .. Public License (the "License"); you may not use this file except |
| .. in compliance with the License. You may obtain a copy of the License at |
| .. |
| .. https://creativecommons.org/licenses/by/4.0/ |
| .. |
| .. Unless required by applicable law or agreed to in writing, documentation |
| .. distributed under the License is distributed on an "AS IS" BASIS, |
| .. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| .. |
| .. See the License for the specific language governing permissions and |
| .. limitations under the License. |
| .. |
| |
| |
| ########## |
| User Guide |
| ########## |
| |
| .. raw:: pdf |
| |
| PageBreak |
| |
| .. contents:: |
| :depth: 3 |
| :local: |
| |
| .. raw:: pdf |
| |
| PageBreak |
| |
| Introduction |
| ************ |
| |
| This is the user guide of O-RAN SC SDL C++ library. |
| Shared Data Layer (SDL) provides a lightweight, high-speed interface (API) for |
| accessing shared data storage. SDL can be used for storing and sharing any |
| data. Data can be shared at VNF level. One typical use case for SDL is sharing |
| the state data of stateful application processes. Thus enabling stateful |
| application processes to become stateless, conforming with, e.g., the |
| requirements of the fifth generation mobile networks. |
| |
| Figure below illustrates some main points of SDL: |
| |
| .. image:: ./_static/sdl_intro.png |
| :align: center |
| :alt: SDL introduction |
| |
| SDL has been implemented in many languages: |
| |
| * C++ Linux shared library |
| * Golang package |
| * Python package |
| |
| This document focuses on C++ implementation of SDL but general principles are |
| the same in all implementations. |
| |
| .. raw:: pdf |
| |
| PageBreak |
| |
| Key Concepts |
| ************ |
| |
| **Backend Data Storage** |
| |
| Backend data storage refers to data storage technology behind SDL API which |
| handles the actual data storing. SDL API hides the backend data storage |
| implementation from SDL API clients, and therefore backend data storage |
| technology can be changed without affecting SDL API clients. Currently, Redis |
| database is the most commonly used backend data storage implementation. |
| |
| Figure below illustrates how SDL API hides backend data storage technology |
| from application: |
| |
| .. image:: ./_static/backend_data_storage.png |
| :align: center |
| :alt: SDL API hides backend data storage technology from application |
| |
| `SDL Deployment section <#sdl-deployment>`_ provides further information |
| about backend data storage deployment options. |
| |
| **Namespace** |
| |
| Namespaces provide data isolation within SDL data storage. That is, data in |
| certain namespace is isolated from the data in other namespaces. Each SDL |
| client uses one or more namespaces. |
| |
| Namespaces can be used, for example, to isolate data belonging to different |
| use cases. |
| |
| Figure below shows an example of the SDL namespace concept. There are two SDL |
| clients, both accessing SDL backend data storage using an SDL API instance |
| (C++ object). Client 1 uses both namespaces: A and B, while client 2 uses only |
| namespace: B. Therefore, data in the namespace: A is visible only to client 1 |
| and data in namespace: B is shared between clients 1 and 2: |
| |
| .. image:: ./_static/sdl_namespaces.png |
| :align: center |
| :alt: SDL namespace concept example |
| |
| Namespace management is planned to be moved under a managing entity which |
| enforces some control over how the namespaces are created. For now, however, |
| namespace naming needs to be manually coordinated between clients. |
| |
| **Keys and Data** |
| |
| Clients save key-data pairs. Data is passed as byte vectors. SDL stores the |
| data as it is. Any structure that this data may have (e.g. a serialized JSON) |
| is meaningful only to the client itself. Clients are responsible for managing |
| the keys. As namespaces provide data isolation, keys in different namespaces |
| always access different data. |
| |
| .. raw:: pdf |
| |
| PageBreak |
| |
| APIs |
| **** |
| |
| SDL provides currently following APIs: |
| |
| * Asynchronous API for accessing SDL storage *shareddatalayer::AsyncStorage* |
| * Synchronous API for accessing SDL storage shareddatalayer::SyncStorage |
| |
| Same SDL client can use one or more SDL APIs. There should rarely be need to |
| create several instances of the same SDL API though. All individual operations |
| done using SDL API functions are targeted to one namespace (accessing several |
| namespaces requires multiple operations). |
| |
| SDL API functions are not thread-safe, meaning that same SDL instance must |
| not be shared between multiple threads without explicit locking in SDL client. |
| |
| SDL API functions are atomic unless otherwise indicated. Indication of the |
| non-atomic behavior of certain function can be found from one or many of the |
| following: |
| |
| * Function name |
| * Function parameters |
| * Function doxygen documentation (see below) |
| |
| Refer to doxygen generated SDL API documentation below for further information |
| about SDL APIs and the functions they contain. |
| |
| Doxygen Generated SDL API Documentation |
| ======================================= |
| |
| Pre-built online version of SDL API Doxygen documentation is not yet available. |
| |
| Doxygen documentation can be generated manually. Follow instructions found from |
| :ref:`SDL developer guide <building_sdl_api_doc>`. |
| |
| .. raw:: pdf |
| |
| PageBreak |
| |
| Building Clients Using SDL |
| ************************** |
| |
| SDL API functions can be used by including SDL public headers and by linking |
| SDL shared library. |
| |
| The necessary compilation and linker flags can be acquired with the |
| *pkg-config* tool:: |
| |
| pkg-config --cflags libsdl |
| pkg-config --libs libsdl |
| |
| SDL internal implementation uses C++14, thus SDL clients need to be build |
| using a C++ compiler supporting C++14. However, SDL public API header files |
| contain only features which are available in C++11, thus SDL clients do not |
| need to be implemented (and compiled) using C++14 (C++11 is enough). The |
| compiler just needs to have support for C++14. |
| |
| .. raw:: pdf |
| |
| PageBreak |
| |
| Using SDL in Application Pod |
| **************************** |
| |
| SDL binary artifacts including Debian (.deb) and RPM Package Manager (.rpm) |
| packages are available in O-RAN-SC PackageCloud.io repository. |
| |
| In runtime environment SDL needs also a database backend service, currently |
| SDL supports only Redis database. Recommended solution is to use DBaaS |
| component of the official RIC platform deployment. |
| |
| **Deploying SDL database backend with DBaaS service in the RIC** |
| |
| Download RIC deployment artifacts:: |
| |
| git clone "https://gerrit.o-ran-sc.org/r/it/dep" |
| |
| The **ric-platform** directory contains Helm chart and scripts to deploy RIC |
| platform components, including also DBaaS component. |
| |
| RIC DBaaS service must be running before starting application pod which is |
| using SDL API. DBaaS defines environment variables which are used to contact |
| DBaaS service (offering backend for SDL). Those environment variables are |
| exposed inside application container only if DBaaS service is running when |
| application container is started. Refer to |
| `Database Backend Configuration section <#database-backend-configuration>`_, |
| for information about available environment variables. |
| You may test SDL connectivity to its backend with the *sdltool* command inside |
| your application container:: |
| |
| sdltool test-connectivity |
| |
| *sdltool* comes in SDL binary artifacts which are available in O-RAN-SC |
| PackageCloud.io repository. |
| |
| For more information, see also `README <https://gerrit.o-ran-sc.org/r/gitweb?p=ric-plt/dbaas.git;a=blob;f=README.md;h=6391fc45ea762a5b606dcf9f867fac8087b1222f;hb=HEAD>`_ |
| file of the *dbaas* O-RAN-SC gerrit repository. |
| |
| .. raw:: pdf |
| |
| PageBreak |
| |
| Configuration |
| ************* |
| |
| Certain aspects in SDL functionality can be configured by using environment |
| variables. |
| |
| Database Backend Configuration |
| ============================== |
| |
| Database backend configuration can be used to configure, to which database |
| backend SDL instance connects. A list of available environment variables to |
| configure database backend: |
| |
| * DBAAS_SERVICE_HOST |
| * DBAAS_SERVICE_PORT |
| * DBAAS_SERVICE_SENTINEL_PORT |
| * DBAAS_MASTER_NAME |
| * DBAAS_CLUSTER_ADDR_LIST |
| |
| After DBaaS service is installed, environment variables are exposed to |
| application containers. SDL library will automatically use these environment |
| variables. If DBaaS service is not used, above environment variables needs to |
| be set manually so that SDL backend can connect to correct database. |
| |
| **Examples** |
| |
| An example how environment variables can be set in bash shell, when standalone |
| Redis server is running in a Kubernetes Pod with k8s service name of *dbaas* and |
| port *6379*:: |
| |
| export DBAAS_SERVICE_HOST=dbaas |
| export DBAAS_SERVICE_PORT=6379 |
| |
| Besides hostname, IPv4 and IPv6 addresses can be set to *DBAAS_SERVICE_HOST*. |
| |
| An example how environment variables can be set in bash shell, when Redis |
| HA deployment is used. Please note that DBaaS does not support yet HA |
| deployment option. Below environment variables are only in the form of an |
| example to show how HA deployment would be configured:: |
| |
| export DBAAS_MASTER_NAME=my-master-sentinel |
| export DBAAS_SERVICE_HOST=dbaas |
| export DBAAS_SERVICE_SENTINEL_PORT=23550 |
| |
| .. raw:: pdf |
| |
| PageBreak |
| |
| Errors |
| ****** |
| |
| `Doxygen generated SDL API documentation <#doxygen-generated-sdl-api-documentation>`_ |
| describes which error codes are returned and which exceptions are thrown from |
| each SDL API function. Generally, asynchronous SDL APIs return error codes and |
| synchronous SDL APIs throw exceptions in error situations. |
| |
| Handling Error Codes Returned From Asynchronous SDL APIs |
| ======================================================== |
| |
| Asynchronous SDL APIs return *std::error_code* based error codes in error |
| situations. Typically, error code is returned as a parameter in the related |
| callback function. |
| |
| Returned error code contains detailed information about the error which has |
| occurred. This information is valuable for SDL developers in case the issue |
| needs further investigation, but usually this information is too detailed for |
| SDL client error handling logic. For SDL client error handling purposes SDL |
| provides *shareddatalayer::error* constants and the returned *std::error_code* |
| can be compared against these constants. |
| |
| Therefore SDL clients are recommended to store the returned *std::error_code* |
| somewhere (for example to the log) and implement the error handling logic based |
| on *shareddatalayer::error* constants. C++ code example below illustrates this: |
| |
| .. code-block:: c++ |
| |
| if (error) |
| { |
| log.error() << "SDL operation failed, error: " << error |
| << " message: " << error.message() << std::endl; |
| |
| if (error == shareddatalayer::Error::NOT_CONNECTED) |
| // Error handling logic for shareddatalayer::Error::NOT_CONNECTED |
| else if (error == shareddatalayer::Error::OPERATION_INTERRUPTED) |
| // Error handling logic for shareddatalayer::Error::OPERATION_INTERRUPTED |
| else if (error == shareddatalayer::Error::BACKEND_FAILURE) |
| // Error handling logic for shareddatalayer::Error::BACKEND_FAILURE |
| else if (error == shareddatalayer::Error::REJECTED_BY_BACKEND) |
| // Error handling logic for shareddatalayer::Error::REJECTED_BY_BACKEND |
| } |
| |
| *error* in the code block above is *std::error_code* type variable which is |
| returned from some asynchronous SDL API function. *log* is a logging service |
| what an SDL client is using. Note that this is a simple and incomplete example |
| for demonstration purposes and not meant to be used as such in real |
| environment. Complete error handling implementation depends on SDL client and |
| SDL API function which returned the error. For example, in some cases common |
| handling for several *shareddatalayer::error* constants might be sufficient. |
| |
| **Instructions for Error Handling Logic Implementation** |
| |
| Doxygen documentation contains detailed description for all |
| shareddatalayer::Error constants. This information helps to design error |
| handling logic for each shareddatalayer::Error constant. For example, following |
| information can be found from there: |
| |
| * What has happened |
| * Is data modified in the backend data storage |
| * How to recover from error situation |
| |
| |
| Handling Exceptions Thrown by Synchronous SDL APIs |
| ================================================== |
| |
| Synchronous SDL APIs throw exceptions in error situations. There are |
| corresponding exceptions for all *shareddatalayer::error* constants returned |
| by asynchronous APIs (see previous section). All exceptions thrown by SDL are |
| derived from *shareddatalayer::Exception*. |
| Therefore, a client can catch *shareddatalayer::Exception* in case the client |
| wants to implement common handling for some SDL originated exceptions. Note |
| that external services, which SDL uses, can throw exceptions which are not |
| derived from *shareddatalayer::Exception*. |
| |
| Below is a C++ code example of a scenario where SDL client does common error |
| handling for all exceptions thrown from synchronous SDL API: |
| |
| .. code-block:: c++ |
| |
| try |
| { |
| //Code which executes synchronous SDL API function |
| } |
| catch (const shareddatalayer::Exception& e) |
| { |
| log.error() << "SDL operation failed, error: " << e.what() << std::endl; |
| //Common error handling logic for all SDL errors |
| } |
| //Catch also non-SDL exceptions (like std::exception) if needed |
| |
| Below C++ code example has separate handling for shareddatalayer::BackendError |
| exception and common handling for all other exceptions thrown by SDL: |
| |
| .. code-block:: c++ |
| |
| try |
| { |
| //Code which executes synchronous SDL API function |
| } |
| catch (const shareddatalayer::BackendError& e) |
| { |
| log.error() << "SDL operation failed, error: " << e.what() << std::endl; |
| //Error handling logic for BackendError |
| } |
| catch (const shareddatalayer::Exception& e) |
| { |
| log.error() << "SDL operation failed, error: " << e.what() << std::endl; |
| //Common error handling logic for all other SDL errors than BackendError |
| } |
| //Catch also non-SDL exceptions (like std::exception) if needed |
| |
| *log* is a logging service what an SDL client is using. Note that these are |
| simple and incomplete examples for demonstration purposes and they are not |
| meant to be used as such in real environment. |
| |
| **Instructions for Error Handling Logic Implementation** |
| |
| Doxygen documentation contains documentation for all exceptions thrown by SDL. |
| This documentation contains information which helps to design error handling |
| logic for each exception. For exceptions having corresponding error code, |
| exception documentation is usually a reference to corresponding error code |
| documentation. |
| |
| Each SDL API function, which throws exceptions, has a link to the documentation |
| of those exceptions. This link can be found from the Doxygen documentation of |
| given SDL API function. |
| |
| .. raw:: pdf |
| |
| PageBreak |
| |
| SDL Properties |
| ************** |
| |
| This chapter discusses how certain general data storage related aspects work in |
| SDL. Discussed subjects include, for example, concurrency control and data |
| persistency. |
| |
| SDL Deployment |
| ============== |
| |
| Production environments are typically deployed so that SDL backend data storage |
| and SDL clients are in different nodes (e.g. VM, container). |
| |
| There are two different supported deployment modes for SDL backend data |
| storage: |
| |
| * Standalone (single DB node without redundancy) |
| * Redundant (DB node pair working in master/slave redundancy model) |
| |
| SDL supports also Redis sentinel based DB cluster where deployment has one or |
| more DBAAS Redis sentinel group. Different DBAAS Redis sentinel groups |
| can be used to distribute SDL DB operations to different SDL DB instances. When |
| more than one DBAAS Redis sentinel group exits the selection of SDL DB instance |
| is based on the namespace string hash calculation. |
| |
| SDL does not prevent backend data storage to be deployed in the same node with |
| the SDL client. Such deployments are, however, typically used only in |
| development/testing type of environments. |
| |
| Concurrency Control |
| =================== |
| |
| SDL does not support transactions doing one or more units of work in ACID |
| manner (pessimistic concurrency control). |
| |
| SDL supports optimistic concurrency control by providing Check and Set (CAS) |
| type conditional functions. These conditional functions provide possibility |
| to do certain data modification operations only if data value matches the SDL |
| client's last known value. Thus a SDL client can check that someone else has |
| not changed the data after it was read by the SDL client. If the data would |
| have been changed, SDL does not do the modification operation and this is |
| indicated to the SDL client. The SDL client can then decide how to handle the |
| situation (for example read the latest data and retry modification). |
| |
| *AsyncStorage::setIfAsync* is an example of a conditional function discussed |
| above. Other conditional functions exist as well. |
| |
| Data Persistency |
| ================ |
| |
| Currently all data stored to SDL is stored to in-memory backend data storage. |
| Meaning that, data is not preserved over DB node restart. DB node restart does |
| not necessarily cause data loss for SDL client though. Refer to |
| `SDL Deployment section <#sdl-deployment>`_, for information about SDL backend |
| data storage redundancy models. |
| |
| .. raw:: pdf |
| |
| PageBreak |
| |
| Best Practices |
| ************** |
| |
| This chapter gives recommendations on how to use SDL. |
| |
| Building Clients Using SDL |
| ========================== |
| |
| * Use *pkg-config* tool to acquire needed compilation and linking flags, |
| instead of hardcoding them. This ensures that flags are always up-to-date. |
| See more information from `here <#building-clients-using-sdl>`_. |
| * If you want to mock SDL APIs in unit testing, SDL provides helper classes |
| for that. By using these helper classes you need to implement mock |
| implementation only for those SDL API functions which you use in the unit |
| tests. See more information from `doxygen documentation <#doxygen-generated-sdl-api-documentation>`_ |
| of the helper classes: |
| |
| * *include/sdl/tst/mockableasyncstorage.hpp: MockableAsyncStorage* |
| * *include/sdl/tst/mockablesyncstorage.hpp: MockableSyncStorage* |
| |
| Using SDL APIs |
| ============== |
| |
| * SDL APIs are not thread-safe. If same SDL API instance is shared between |
| multiple threads, SDL client has to use explicit locking to ensure that only |
| one thread at time executes SDL API functions. |
| * Each SDL instance establishes own connection to backend data storage, which |
| requires resources (how heavy this exactly is depends on used backend data |
| storage type). Thus, from performance point of view, only one SDL instance |
| per one SDL API should be used if reasonably possible. One SDL instance can |
| access multiple SDL namespaces when using *AsyncStorage* and *SyncStorage* |
| APIs. |
| * Use waitReadyAsync() function before doing first operation via asynchronous |
| APIs to ensure that SDL and backend data storage are ready to handle |
| operations. See waitReadyAsync() function |
| `doxygen documentation <#doxygen-generated-sdl-api-documentation>`_ |
| for corresponding asynchronous API for details. |
| * Avoid using heavy search functions (for example: *AsyncStorage::findKeys()*). |
| Rather define your keys so that you know which keys should be read. |
| |
| Using SDL Namespaces |
| ==================== |
| |
| * As namespace naming is currently on SDL client's responsibility, use enough |
| specific namespace names that same name is surely not used by someone else |
| (unless you want to share given namespace data with that someone else). |
| * Data entities related to each other should be placed under the same |
| namespace (unless there is a good reason not to). For example, accessing |
| multiple data entities with one SDL operation is possible only for data |
| entities belonging to same namespace. |
| * Identically named keys can be used in different namespaces. Creating own |
| namespaces for different use cases and unrelated data provides more freedom |
| into key name selection. |
| |
| Data Management |
| =============== |
| |
| * Writing or reading one big junk of data at once is more efficient than |
| writing/reading the same amount of data in small steps. For example, create a |
| key list and read it once, rather than reading each key in a loop. |
| * If rolling upgrade needs to be supported, consider using Google Protocol |
| Buffers (or something similar) to make it possible to parse data which is |
| written by older or newer application version. |
| |
| .. raw:: pdf |
| |
| PageBreak |