extras/selinux/selinux_doc.rst

.. _selinux_doc:

SELinux - VPP Custom SELinux Policy
===================================

Overview
--------

Security-enhanced Linux (SELinux) is a security feature in the Linux
kernel. At a very high level, SELinux implements mandatory access
controls (MAC), as opposed to discretionary access control (DAC)
implemented in standard Linux. MAC defines how processes can interact
with other system components (Files, Directories, Other Processes,
Pipes, Sockets, Network Ports). Each system component is assigned a
label, and then the SELinux Policy defines which labels and which
actions on each label a process is able to perform. The VPP Custom
SELinux Policy defines the actions VPP is allowed to perform on which
labels.

The VPP Custom SELinux Policy is intended to be installed on RPM based
platforms (tested on CentOS 7 and RHEL 7). Though SELinux can run on
Debian platforms, it typically is not and therefore is not currently
being built for Debian.

The VPP Custom SELinux Policy does not enable or disable SELinux, only
allows VPP to run when SELinux is enabled. A fresh install of either
Fedora, CentOS or RHEL will have SELinux enabled by default. To
determine if SELinux is enabled on a given system and enable it if
needed, run:

::

      $ getenforce
      Permissive

      $ sudo setenforce 1

      $ getenforce
      Enforcing

To make the change persistent, modify the following file to set
``SELINUX=enforcing``:

::

      $ sudo vi /etc/selinux/config
      :
      # This file controls the state of SELinux on the system.
      # SELINUX= can take one of these three values:
      #     enforcing - SELinux security policy is enforced.
      #     permissive - SELinux prints warnings instead of enforcing.
      #     disabled - No SELinux policy is loaded.
      SELINUX=enforcing
      :

Installation
------------

To install VPP, see the installation instructions on the VPP Wiki
(https://wiki.fd.io/view/VPP/Installing_VPP_binaries_from_packages). The
VPP Custom SELinux Policy is packaged in its own RPM starting in 18.04,
``vpp-selinux-policy-<VERSION>-<RELEASE>.rpm``. It is packaged and
installed along with the other VPP RPMs.

Fresh Install of VPP
~~~~~~~~~~~~~~~~~~~~

If VPP has never been installed on a system, then starting in 18.04, the
VPP Custom SELinux Policy will be installed with the other RPMs and all
the system components managed by VPP will be labeled properly.

Fix SELinux Labels for VPP
~~~~~~~~~~~~~~~~~~~~~~~~~~

In the case where the VPP Custom Policy is being installed for the first
time, either because VPP has been upgraded or packages were removed and
then reinstalled, several directories and files will not not be properly
labeled. The labels on these files will need to be fixed for VPP to run
properly with SELinux enabled. After the VPP Custom SELinux Policy is
installed, run the following commands to fix the labels. If VPP is
already running, make sure to restart VPP after the labels are fixed.
This change is persistent for the life of the file. Once the VPP Custom
Policy is installed on the system, subsequent files created by VPP will
be labeled properly. This is only to fix files created by VPP prior to
the VPP Custom Policy being installed.

::

     $ sudo restorecon -Rv /etc/vpp/
     $ sudo restorecon -Rv /usr/lib/vpp_api_test_plugins/
     $ sudo restorecon -Rv /usr/lib/vpp_plugins/
     $ sudo restorecon -Rv /usr/share/vpp/
     $ sudo restorecon -Rv /var/run/vpp/

     $ sudo chcon -t vpp_tmp_t /tmp/vpp_*
     $ sudo chcon -t vpp_var_run_t /var/run/.vpp_*

**NOTE:** Because the VPP APIs allow custom filenames in certain
scenarios, the above commands may not handle all files. Inspect your
system and correct any files that are mislabeled. For example, to verify
all VPP files in ``/tmp/`` are labeled properly, run:

::

     $ sudo ls -alZ /tmp/

Any files not properly labeled with ``vpp_tmp_t``, run:

::

     $ sudo chcon -t vpp_tmp_t /tmp/<filename>

VPP Files
---------

Recommended Default File Directories
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Documentation in the VPP Wiki (https://wiki.fd.io/view/VPP/) and doxygen
generated documentation have examples with files located in certain
directories. Some of the recommend file locations have been moved to
satisfy SELinux. Most of the documentation has been updated, but links
to older documentation still exist and there may have been instances
that were missed. Use the file locations described below to allow
SELinux to properly label the given files.

File locations that have changed: \* VPP Debug CLI Script Files \* vHost
Sockets \* VPP Log Files

VPP Debug CLI Script Files
^^^^^^^^^^^^^^^^^^^^^^^^^^

The VPP Debug CLI, ``vppctl``, allows a sequence of CLI commands to be
read from a file and executed. To avoid from having to grant VPP access
to all of ``/tmp/`` and possibly ``/home/`` sub-directories, it is
recommended that any VPP Debug CLI script files be placed in a common
directory such as ``/usr/share/vpp/``.

For example:

::

   $ cat /usr/share/vpp/scripts/gigup.txt
   set interface state GigabitEthernet0/8/0 up
   set interface state GigabitEthernet0/9/0 up

To execute:

::

   $ vppctl exec /usr/share/vpp/scripts/gigup.txt

Or

::

   $ vppctl
       _______    _        _   _____  ___
    __/ __/ _ \  (_)__    | | / / _ \/ _ \
    _/ _// // / / / _ \   | |/ / ___/ ___/
    /_/ /____(_)_/\___/   |___/_/  /_/

   vpp# exec /usr/share/vpp/scripts/gigup.txt
   vpp# quit

If the file is not labeled properly, you will see something similar to:

::

   $ vppctl exec /home/<user>/dev/vpp/scripts/vppctl/gigup.txt
   exec: failed to open `/home/<user>/dev/vpp/scripts/vppctl/gigup.txt': Permission denied

   $ ls -alZ
   drwxrwxr-x. <user> <user> unconfined_u:object_r:user_home_t:s0 .
   drwxrwxr-x. <user> <user> unconfined_u:object_r:user_home_t:s0 ..
   -rw-r--r--. <user> <user> unconfined_u:object_r:user_home_t:s0 gigup.txt

Original Documentation
''''''''''''''''''''''

Some of the original documentation showed script files being executed
out of ``/tmp/``. Convenience also may lead to script files being placed
in ``/home/<user>/`` subdirectories. If a file is generated by the VPP
process in ``/tmp/``, for example a trace file or pcap file, it will get
properly labeled with the SELinux label ``vpp_tmp_t``. When a file is
created, unless a rule is in place for the process that created it, the
file will inherit the SELinux label of the parent directory. So if a
user creates a file themselves in ``/tmp/``, it will get the SELinux
label ``tmp_t``, which VPP does not have permission to access. Therefore
it is recommended that script files are located as described above.

vHost Sockets
^^^^^^^^^^^^^

vHost sockets are created from VPP perspective in either Server or
Client mode. In Server mode, the socket name is provided to VPP and VPP
creates the socket. In Client mode, the socket name is provided to VPP
and the hypervisor creates the socket. In order for VPP and hypervisor
to share the socket resource with SELinux enabled, a rule in the VPP
Custom SELinux Policy has been added. This rules allows processes with
the ``svirt_t`` label (the hypervisor) to access sockets with the
``vpp_var_run_t`` label. As such, when SELinux is enabled, vHost sockets
should be created in the directory ``/var/run/vpp/``.

.. _original-documentation-1:

Original Documentation
''''''''''''''''''''''

Some of the original documentation showed vHost sockets being created in
the directory ``/tmp/``. To work properly with SELinux enabled, vHost
sockets should be created as described above.

VPP Log Files
^^^^^^^^^^^^^

The VPP log file location is set by updating the
``/etc/vpp/startup.conf`` file:

::

   vi /etc/vpp/startup.conf
   unix {
   :
     log /var/log/vpp/vpp.log
   :
   }

By moving the log file to ``/var/log/vpp/``, it will get the label
``vpp_log_t``, which indicates that the files are log files so they
benefit from the associated rules (for example granting rights to
logrotate so that it can manipulate them).

.. _original-documentation-2:

Original Documentation
''''''''''''''''''''''

The default ``startup.conf`` file creates the VPP log file in
``/tmp/vpp.log``. By leaving the log file in ``/tmp/``, it will get the
label ``vpp_tmp_t``. Moving it to ``/var/log/vpp/``, it will get the
label ``vpp_log_t``.

Use of Non-default File Directories
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

VPP installs multiple files on the system. Some files have fixed
directory and file names: - /etc/bash_completion.d/vppctl_completion -
/etc/sysctl.d/80-vpp.conf - /usr/lib/systemd/system/vpp.service

Others files have default directory and file names but the default can
be overwritten: - /etc/vpp/startup.conf - Can be changed via the
``/usr/lib/systemd/system/vpp.service`` file by changing the -c option
on the VPP command line:

::

   ExecStart=/usr/bin/vpp -c /etc/vpp/startup.conf

-  /run/vpp/cli.sock

   -  Can be changed via the ``/etc/vpp/startup.conf`` file by changing
      the cli-listen setting:

::

   unix {
   :
     cli-listen /run/vpp/cli.sock
   :
   }

-  /var/log/vpp/vpp.log

   -  Can be changed via the ``/etc/vpp/startup.conf`` file by changing
      the log setting:

::

   unix {
     :
     log /var/log/vpp/vpp.log
     :
   }

If the directory of any VPP installed files is changed from the default,
ensure that the proper SELiunx label is applied. The SELinux label can
be determined by passing the -Z option to many common Linux commands:

::

   ls -alZ /run/vpp/
   drwxr-xr-x. root vpp  system_u:object_r:vpp_var_run_t:s0 .
   drwxr-xr-x. root root system_u:object_r:var_run_t:s0     ..
   srwxrwxr-x. root vpp  system_u:object_r:vpp_var_run_t:s0 cli.sock

VPP SELinux Types
~~~~~~~~~~~~~~~~~

The following SELinux types are created by the VPP Custom SELinux
Policy: - ``vpp_t`` - Applied to: - VPP process and spawned threads.

-  ``vpp_config_rw_t`` - Applied to:

   -  ``/etc/vpp/*``

-  ``vpp_tmp_t`` - Applied to:

   -  ``/tmp/*``

-  ``vpp_exec_t`` - Applied to:

   -  ``/usr/bin/*``

-  ``vpp_lib_t`` - Applied to:

   -  ``/usr/lib/vpp_api_test_plugins/*``
   -  ``/usr/lib/vpp_plugins/*``

-  ``vpp_unit_file_t`` - Applied to:

   -  ``/usr/lib/systemd/system/vpp.*``

-  ``vpp_log_t`` - Applied to:

   -  ``/var/log/vpp/*``

-  ``vpp_var_run_t`` - Applied to:

   -  ``/var/run/vpp/*``

Debug SELinux Issues
--------------------

If SELinux issues are suspected, there are a few steps that can be taken
to debug the issue. This section provides a few pointers on on those
steps. Any SELinux JIRAs will need this information to properly address
the issue.

Additional SELinux Packages and Setup
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

First, install the SELinux troubleshooting packages:

::

   $ sudo yum -y install setroubleshoot setroubleshoot-server setools-console
   -- OR --
   $ sudo dnf -y install setroubleshoot setroubleshoot-server setools-console

To enable proper logging, restart auditd:

::

   $ sudo service auditd restart

While debugging issues, it is best to set SELinux to ``Permissive``
mode. In ``Permissive`` mode, SELinux will still detect and flag errors,
but will allow processes to continue normal operation. This allows
multiple errors to be collected at once as opposed to breaking on each
individual error. To set SELinux to ``Permissive`` mode (until next
reboot or it is set back), use:

::

   $ sudo setenforce 0

   $ getenforce
   Permissive

After debugging, to set SELinux back to ``Enforcing`` mode, use:

::

   $ sudo setenforce 1

   $ getenforce
   Enforcing

Debugging
~~~~~~~~~

Once the SELinux troubleshooting packages are installed, perform the
actions that are suspected to be blocked by SELinux. Either ``tail`` the
log during these actions or ``grep`` the log for additional SELinux
logs:

::

   sudo tail -f /var/log/messages
   -- OR --
   sudo journalctl -f

Below are some examples of SELinux logs that are generated:

::

   May 14 11:28:34 svr-22 setroubleshoot: SELinux is preventing /usr/bin/vpp from read access on the file hostCreate.txt. For complete SELinux messages run: sealert -l a418f869-f470-4c8a-b8e9-bdd41f2dd60b
   May 14 11:28:34 svr-22 python: SELinux is preventing /usr/bin/vpp from read access on the file hostCreate.txt.#012#012*****  Plugin catchall (100. confidence) suggests   **************************#012#012If you believe that vpp should be allowed read access on the hostCreate.txt file by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp_main' --raw | audit2allow -M my-vppmain#012# semodule -i my-vppmain.pp#012
   May 14 11:28:34 svr-22 setroubleshoot: SELinux is preventing /usr/bin/vpp from read access on the file hostCreate.txt. For complete SELinux messages run: sealert -l a418f869-f470-4c8a-b8e9-bdd41f2dd60b
   May 14 11:28:34 svr-22 python: SELinux is preventing /usr/bin/vpp from read access on the file hostCreate.txt.#012#012*****  Plugin catchall (100. confidence) suggests   **************************#012#012If you believe that vpp should be allowed read access on the hostCreate.txt file by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp_main' --raw | audit2allow -M my-vppmain#012# semodule -i my-vppmain.pp#012
   May 14 11:28:37 svr-22 setroubleshoot: SELinux is preventing vpp_main from map access on the packet_socket packet_socket. For complete SELinux messages run: sealert -l ab6667d9-3f14-4dbd-96a0-7a655f7b4eb1
   May 14 11:28:37 svr-22 python: SELinux is preventing vpp_main from map access on the packet_socket packet_socket.#012#012*****  Plugin catchall (100. confidence) suggests   **************************#012#012If you believe that vpp_main should be allowed map access on the packet_socket packet_socket by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp_main' --raw | audit2allow -M my-vppmain#012# semodule -i my-vppmain.pp#012
   May 14 11:28:51 svr-22 setroubleshoot: SELinux is preventing vpp_main from map access on the packet_socket packet_socket. For complete SELinux messages run: sealert -l ab6667d9-3f14-4dbd-96a0-7a655f7b4eb1
   May 14 11:28:51 svr-22 python: SELinux is preventing vpp_main from map access on the packet_socket packet_socket.#012#012*****  Plugin catchall (100. confidence) suggests   **************************#012#012If you believe that vpp_main should be allowed map access on the packet_socket packet_socket by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp_main' --raw | audit2allow -M my-vppmain#012# semodule -i my-vppmain.pp#012

From the logs above, there are two sets of commands that are recommended
to be run. The first is to run the ``sealert`` command. The second is to
run the ``ausearch | audit2allow`` commands and the ``semodule``
command.

sealert Command
^^^^^^^^^^^^^^^

This ``sealert`` command provides a more detailed output for the given
issue detected.

::

   $ sealert -l a418f869-f470-4c8a-b8e9-bdd41f2dd60b
   SELinux is preventing /usr/bin/vpp from 'read, write' accesses on the chr_file noiommu-0.

   *****  Plugin device (91.4 confidence) suggests   ****************************

   If you want to allow vpp to have read write access on the noiommu-0 chr_file
   Then you need to change the label on noiommu-0 to a type of a similar device.
   Do
   # semanage fcontext -a -t SIMILAR_TYPE 'noiommu-0'
   # restorecon -v 'noiommu-0'

   *****  Plugin catchall (9.59 confidence) suggests   **************************

   If you believe that vpp should be allowed read write access on the noiommu-0 chr_file by default.
   Then you should report this as a bug.
   You can generate a local policy module to allow this access.
   Do
   allow this access for now by executing:
   # ausearch -c 'vpp' --raw | audit2allow -M my-vpp
   # semodule -i my-vpp.pp


   Additional Information:
   Source Context                system_u:system_r:vpp_t:s0
   Target Context                system_u:object_r:device_t:s0
   Target Objects                noiommu-0 [ chr_file ]
   Source                        vpp
   Source Path                   /usr/bin/vpp
   Port                          <Unknown>
   Host                          vpp_centos7_selinux
   Source RPM Packages           vpp-19.01.2-rc0~17_gcfd3086.x86_64
   Target RPM Packages
   Policy RPM                    selinux-policy-3.13.1-229.el7_6.12.noarch
   Selinux Enabled               True
   Policy Type                   targeted
   Enforcing Mode                Permissive
   Host Name                     vpp_centos7_selinux
   Platform                      Linux vpp_centos7_selinux
                                 3.10.0-957.12.1.el7.x86_64 #1 SMP Mon Apr 29
                                 14:59:59 UTC 2019 x86_64 x86_64
   Alert Count                   1
   First Seen                    2019-05-13 18:10:50 EDT
   Last Seen                     2019-05-13 18:10:50 EDT
   Local ID                      a418f869-f470-4c8a-b8e9-bdd41f2dd60b

   Raw Audit Messages
   type=AVC msg=audit(1557785450.964:257): avc:  denied  { read write } for  pid=5273 comm="vpp" name="noiommu-0" dev="devtmpfs" ino=36022 scontext=system_u:system_r:vpp_t:s0 tcontext=system_u:object_r:device_t:s0 tclass=chr_file permissive=1


   type=AVC msg=audit(1557785450.964:257): avc:  denied  { open } for  pid=5273 comm="vpp" path="/dev/vfio/noiommu-0" dev="devtmpfs" ino=36022 scontext=system_u:system_r:vpp_t:s0 tcontext=system_u:object_r:device_t:s0 tclass=chr_file permissive=1


   type=SYSCALL msg=audit(1557785450.964:257): arch=x86_64 syscall=open success=yes exit=ENOTBLK a0=7fb395ffd7f0 a1=2 a2=7fb395ffd803 a3=7fb395ffe2a0 items=0 ppid=1 pid=5273 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=993 sgid=0 fsgid=993 tty=(none) ses=4294967295 comm=vpp exe=/usr/bin/vpp subj=system_u:system_r:vpp_t:s0 key=(null)

   Hash: vpp,vpp_t,device_t,chr_file,read,write

In general, this command pumps out too much info and is only needed for
additional debugging for tougher issues. Also note that once the process
being tested is restarted, this command loses it’s context and will not
provide any information:

::

   $ sealert -l a418f869-f470-4c8a-b8e9-bdd41f2dd60b
   Error
   query_alerts error (1003): id (a418f869-f470-4c8a-b8e9-bdd41f2dd60b) not found

ausearch \| audit2allow and semodule Commands
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

These set of commands are more useful for basic debugging. The
``ausearch | audit2allow`` commands generate a set files. It may be
worthwhile to run the commands in a temporary subdirectory:

::

   $ mkdir test-01/; cd test-01/

   $ sudo ausearch -c 'vpp_main' --raw | audit2allow -M my-vppmain

   $ ls
   my-vpp.pp  my-vpp.te

   $ cat my-vpp.te
   module my-vpp 1.0;

   require {
           type user_home_t;
           type vpp_t;
           class packet_socket map;
           class file { open read };
   }

   #============= vpp_t ==============
   allow vpp_t self:packet_socket map;
   allow vpp_t user_home_t:file { open read };

As shown above, the file ``my-vpp.te`` has been generated. This file
shows possible changes to the SELinux policy that may fix the issue. If
an SELinux policy was being created from scratch, this policy could be
applied using the ``semodule -i my-vpp.pp`` command. HOWEVER, VPP
already has a policy in place. So these changes need to be incorporated
into the existing policy. The VPP SELinux policy is located in the
following files:

::

   $ ls extras/selinux/
   selinux_doc.md  vpp-custom.fc  vpp-custom.if  vpp-custom.te

In this example, ``map`` needs to be added to the ``packet_socket``
class. If the ``vpp-custom.te`` is examined (prior to this fix), then
one would see that the ``packet_socket`` class is already defined and
just needs to be updated:

::

   $ vi extras/selinux/vpp-custom.te
   :
   allow vpp_t self:process { execmem execstack setsched signal }; # too benevolent
   allow vpp_t self:packet_socket { bind create setopt ioctl };  <---
   allow vpp_t self:tun_socket { create relabelto relabelfrom };
   :

Before blindly applying the changes proposed by the
``ausearch | audit2allow`` commands, try to determine what is being
allowed by the policy and determine if this is desired, or if the code
can be reworked to no longer require the suggested permission. In the
``my-vpp.te`` file from above, it is suggested to allow ``vpp_t``
(i.e. the VPP process) access to all files in the home directory
(``allow vpp_t user_home_t:file { open read };``). This was because a
``vppctl exec`` command was executed calling a script located in the
``/home/<user>/`` directory. Once this script was run from the
``/usr/share/vpp/`` directory as described in a section above, these
permissions were no longer needed.