docs: better docs, mv doxygen to sphinx
This patch refactors the VPP sphinx docs
in order to make it easier to consume
for external readers as well as VPP developers.
It also makes sphinx the single source
of documentation, which simplifies maintenance
and operation.
Most important updates are:
- reformat the existing documentation as rst
- split RELEASE.md and move it into separate rst files
- remove section 'events'
- remove section 'archive'
- remove section 'related projects'
- remove section 'feature by release'
- remove section 'Various links'
- make (Configuration reference, CLI docs,
developer docs) top level items in the list
- move 'Use Cases' as part of 'About VPP'
- move 'Troubleshooting' as part of 'Getting Started'
- move test framework docs into 'Developer Documentation'
- add a 'Contributing' section for gerrit,
docs and other contributer related infos
- deprecate doxygen and test-docs targets
- redirect the "make doxygen" target to "make docs"
Type: refactor
Change-Id: I552a5645d5b7964d547f99b1336e2ac24e7c209f
Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
diff --git a/docs/usecases/contiv/NETWORKING.rst b/docs/usecases/contiv/NETWORKING.rst
new file mode 100644
index 0000000..b679996
--- /dev/null
+++ b/docs/usecases/contiv/NETWORKING.rst
@@ -0,0 +1,196 @@
+Contiv/VPP Network Operation
+============================
+
+This document describes the network operation of the Contiv/VPP k8s
+network plugin. It elaborates the operation and config options of the
+Contiv IPAM, as well as details on how the VPP gets programmed by
+Contiv/VPP control plane.
+
+The following picture shows 2-node k8s deployment of Contiv/VPP, with a
+VXLAN tunnel established between the nodes to forward inter-node POD
+traffic. The IPAM options are depicted on the Node 1, whereas the VPP
+programming is depicted on the Node 2.
+
+.. figure:: /_images/contiv-networking.png
+ :alt: contiv-networking.png
+
+ Contiv/VPP Architecture
+
+Contiv/VPP IPAM (IP Address Management)
+---------------------------------------
+
+IPAM in Contiv/VPP is based on the concept of **Node ID**. The Node ID
+is a number that uniquely identifies a node in the k8s cluster. The
+first node is assigned the ID of 1, the second node 2, etc. If a node
+leaves the cluster, its ID is released back to the pool and will be
+re-used by the next node.
+
+The Node ID is used to calculate per-node IP subnets for PODs and other
+internal subnets that need to be unique on each node. Apart from the
+Node ID, the input for IPAM calculations is a set of config knobs, which
+can be specified in the ``IPAMConfig`` section of the [Contiv/VPP
+deployment YAML](../../../k8s/contiv-vpp.yaml):
+
+- **PodSubnetCIDR** (default ``10.1.0.0/16``): each pod gets an IP
+ address assigned from this range. The size of this range (default
+ ``/16``) dictates upper limit of POD count for the entire k8s cluster
+ (default 65536 PODs).
+
+- **PodNetworkPrefixLen** (default ``24``): per-node dedicated
+ podSubnet range. From the allocatable range defined in
+ ``PodSubnetCIDR``, this value will dictate the allocation for each
+ node. With the default value (``24``) this indicates that each node
+ has a ``/24`` slice of the ``PodSubnetCIDR``. The Node ID is used to
+ address the node. In case of ``PodSubnetCIDR = 10.1.0.0/16``,
+ ``PodNetworkPrefixLen = 24`` and ``NodeID = 5``, the resulting POD
+ subnet for the node would be ``10.1.5.0/24``.
+
+- **PodIfIPCIDR** (default ``10.2.1.0/24``): VPP-internal addresses put
+ the VPP interfaces facing towards the PODs into L3 mode. This IP
+ range will be reused on each node, thereby it is never externally
+ addressable outside of the node itself. The only requirement is that
+ this subnet should not collide with any other IPAM subnet.
+
+- **VPPHostSubnetCIDR** (default ``172.30.0.0/16``): used for
+ addressing the interconnect of VPP with the Linux network stack,
+ within the same node. Since this subnet needs to be unique on each
+ node, the Node ID is used to determine the actual subnet used on the
+ node with the combination of ``VPPHostNetworkPrefixLen``,
+ ``PodSubnetCIDR`` and ``PodNetworkPrefixLen``.
+
+- **VPPHostNetworkPrefixLen** (default ``24``): used to calculate the
+ subnet for addressing the interconnect of VPP with the Linux network
+ stack, within the same node. With
+ ``VPPHostSubnetCIDR = 172.30.0.0/16``,
+ ``VPPHostNetworkPrefixLen = 24`` and ``NodeID = 5`` the resulting
+ subnet for the node would be ``172.30.5.0/24``.
+
+- **NodeInterconnectCIDR** (default ``192.168.16.0/24``): range for the
+ addresses assigned to the data plane interfaces managed by VPP.
+ Unless DHCP is used (``NodeInterconnectDHCP = True``), the Contiv/VPP
+ control plane automatically assigns an IP address from this range to
+ the DPDK-managed ethernet interface bound to VPP on each node. The
+ actual IP address will be calculated from the Node ID (e.g., with
+ ``NodeInterconnectCIDR = 192.168.16.0/24`` and ``NodeID = 5``, the
+ resulting IP address assigned to the ethernet interface on VPP will
+ be ``192.168.16.5`` ).
+
+- **NodeInterconnectDHCP** (default ``False``): instead of assigning
+ the IPs for the data plane interfaces, which are managed by VPP from
+ ``NodeInterconnectCIDR`` by the Contiv/VPP control plane, DHCP
+ assigns the IP addresses. The DHCP must be running in the network
+ where the data plane interface is connected, in case
+ ``NodeInterconnectDHCP = True``, ``NodeInterconnectCIDR`` is ignored.
+
+- **VxlanCIDR** (default ``192.168.30.0/24``): in order to provide
+ inter-node POD to POD connectivity via any underlay network (not
+ necessarily an L2 network), Contiv/VPP sets up a VXLAN tunnel overlay
+ between each of the 2 nodes within the cluster. Each node needs its
+ unique IP address of the VXLAN BVI interface. This IP address is
+ automatically calculated from the Node ID, (e.g., with
+ ``VxlanCIDR = 192.168.30.0/24`` and ``NodeID = 5``, the resulting IP
+ address assigned to the VXLAN BVI interface will be
+ ``192.168.30.5``).
+
+VPP Programming
+---------------
+
+This section describes how the Contiv/VPP control plane programs VPP,
+based on the events it receives from k8s. This section is not
+necessarily for understanding basic Contiv/VPP operation, but is very
+useful for debugging purposes.
+
+Contiv/VPP currently uses a single VRF to forward the traffic between
+PODs on a node, PODs on different nodes, host network stack, and
+DPDK-managed dataplane interface. The forwarding between each of them is
+purely L3-based, even for cases of communication between 2 PODs within
+the same node.
+
+DPDK-Managed Data Interface
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In order to allow inter-node communication between PODs on different
+nodes and between PODs and outside world, Contiv/VPP uses data-plane
+interfaces bound to VPP using DPDK. Each node should have one “main” VPP
+interface, which is unbound from the host network stack and bound to
+VPP. The Contiv/VPP control plane automatically configures the interface
+either via DHCP, or with a statically assigned address (see
+``NodeInterconnectCIDR`` and ``NodeInterconnectDHCP`` yaml settings).
+
+PODs on the Same Node
+~~~~~~~~~~~~~~~~~~~~~
+
+PODs are connected to VPP using virtio-based TAP interfaces created by
+VPP, with the POD-end of the interface placed into the POD container
+network namespace. Each POD is assigned an IP address from the
+``PodSubnetCIDR``. The allocated IP is configured with the prefix length
+``/32``. Additionally, a static route pointing towards the VPP is
+configured in the POD network namespace. The prefix length ``/32`` means
+that all IP traffic will be forwarded to the default route - VPP. To get
+rid of unnecessary broadcasts between POD and VPP, a static ARP entry is
+configured for the gateway IP in the POD namespace, as well as for POD
+IP on VPP. Both ends of the TAP interface have a static (non-default)
+MAC address applied.
+
+PODs with hostNetwork=true
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+PODs with a ``hostNetwork=true`` attribute are not placed into a
+separate network namespace, they instead use the main host Linux network
+namespace; therefore, they are not directly connected to the VPP. They
+rely on the interconnection between the VPP and the host Linux network
+stack, which is described in the next paragraph. Note, when these PODs
+access some service IP, their network communication will be NATed in
+Linux (by iptables rules programmed by kube-proxy) as opposed to VPP,
+which is the case for the PODs connected to VPP directly.
+
+Linux Host Network Stack
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+In order to interconnect the Linux host network stack with VPP (to allow
+access to the cluster resources from the host itself, as well as for the
+PODs with ``hostNetwork=true``), VPP creates a TAP interface between VPP
+and the main network namespace. The TAP interface is configured with IP
+addresses from the ``VPPHostSubnetCIDR`` range, with ``.1`` in the
+latest octet on the VPP side, and ``.2`` on the host side. The name of
+the host interface is ``vpp1``. The host has static routes pointing to
+VPP configured with: - A route to the whole ``PodSubnetCIDR`` to route
+traffic targeting PODs towards VPP. - A route to ``ServiceCIDR``
+(default ``10.96.0.0/12``), to route service IP targeted traffic that
+has not been translated by kube-proxy for some reason towards VPP. - The
+host also has a static ARP entry configured for the IP of the VPP-end
+TAP interface, to get rid of unnecessary broadcasts between the main
+network namespace and VPP.
+
+VXLANs to Other Nodes
+~~~~~~~~~~~~~~~~~~~~~
+
+In order to provide inter-node POD to POD connectivity via any underlay
+network (not necessarily an L2 network), Contiv/VPP sets up a VXLAN
+tunnel overlay between each 2 nodes within the cluster (full mesh).
+
+All VXLAN tunnels are terminated in one bridge domain on each VPP. The
+bridge domain has learning and flooding disabled, the l2fib of the
+bridge domain contains a static entry for each VXLAN tunnel. Each bridge
+domain has a BVI interface, which interconnects the bridge domain with
+the main VRF (L3 forwarding). This interface needs a unique IP address,
+which is assigned from the ``VxlanCIDR`` as describe above.
+
+The main VRF contains several static routes that point to the BVI IP
+addresses of other nodes. For each node, it is a route to PODSubnet and
+VppHostSubnet of the remote node, as well as a route to the management
+IP address of the remote node. For each of these routes, the next hop IP
+is the BVI interface IP of the remote node, which goes via the BVI
+interface of the local node.
+
+The VXLAN tunnels and the static routes pointing to them are
+added/deleted on each VPP, whenever a node is added/deleted in the k8s
+cluster.
+
+More Info
+~~~~~~~~~
+
+Please refer to the [Packet Flow Dev
+Guide](../dev-guide/PACKET_FLOW.html) for more detailed description of
+paths traversed by request and response packets inside Contiv/VPP
+Kubernetes cluster under different situations.