| VNET (VPP Network Stack) |
| ======================== |
| |
| The files associated with the VPP network stack layer are located in the |
| *./src/vnet* folder. The Network Stack Layer is basically an |
| instantiation of the code in the other layers. This layer has a vnet |
| library that provides vectorized layer-2 and 3 networking graph nodes, a |
| packet generator, and a packet tracer. |
| |
| In terms of building a packet processing application, vnet provides a |
| platform-independent subgraph to which one connects a couple of |
| device-driver nodes. |
| |
| Typical RX connections include “ethernet-input” [full software |
| classification, feeds ipv4-input, ipv6-input, arp-input etc.] and |
| “ipv4-input-no-checksum” [if hardware can classify, perform ipv4 header |
| checksum]. |
| |
| Effective graph dispatch function coding |
| ---------------------------------------- |
| |
| Over the 15 years, multiple coding styles have emerged: a |
| single/dual/quad loop coding model (with variations) and a |
| fully-pipelined coding model. |
| |
| Single/dual loops |
| ----------------- |
| |
| The single/dual/quad loop model variations conveniently solve problems |
| where the number of items to process is not known in advance: typical |
| hardware RX-ring processing. This coding style is also very effective |
| when a given node will not need to cover a complex set of dependent |
| reads. |
| |
| Here is an quad/single loop which can leverage up-to-avx512 SIMD vector |
| units to convert buffer indices to buffer pointers: |
| |
| .. code:: c |
| |
| static uword |
| simulated_ethernet_interface_tx (vlib_main_t * vm, |
| vlib_node_runtime_t * |
| node, vlib_frame_t * frame) |
| { |
| u32 n_left_from, *from; |
| u32 next_index = 0; |
| u32 n_bytes; |
| u32 thread_index = vm->thread_index; |
| vnet_main_t *vnm = vnet_get_main (); |
| vnet_interface_main_t *im = &vnm->interface_main; |
| vlib_buffer_t *bufs[VLIB_FRAME_SIZE], **b; |
| u16 nexts[VLIB_FRAME_SIZE], *next; |
| |
| n_left_from = frame->n_vectors; |
| from = vlib_frame_vector_args (frame); |
| |
| /* |
| * Convert up to VLIB_FRAME_SIZE indices in "from" to |
| * buffer pointers in bufs[] |
| */ |
| vlib_get_buffers (vm, from, bufs, n_left_from); |
| b = bufs; |
| next = nexts; |
| |
| /* |
| * While we have at least 4 vector elements (pkts) to process.. |
| */ |
| while (n_left_from >= 4) |
| { |
| /* Prefetch next quad-loop iteration. */ |
| if (PREDICT_TRUE (n_left_from >= 8)) |
| { |
| vlib_prefetch_buffer_header (b[4], STORE); |
| vlib_prefetch_buffer_header (b[5], STORE); |
| vlib_prefetch_buffer_header (b[6], STORE); |
| vlib_prefetch_buffer_header (b[7], STORE); |
| } |
| |
| /* |
| * $$$ Process 4x packets right here... |
| * set next[0..3] to send the packets where they need to go |
| */ |
| |
| do_something_to (b[0]); |
| do_something_to (b[1]); |
| do_something_to (b[2]); |
| do_something_to (b[3]); |
| |
| /* Process the next 0..4 packets */ |
| b += 4; |
| next += 4; |
| n_left_from -= 4; |
| } |
| /* |
| * Clean up 0...3 remaining packets at the end of the incoming frame |
| */ |
| while (n_left_from > 0) |
| { |
| /* |
| * $$$ Process one packet right here... |
| * set next[0..3] to send the packets where they need to go |
| */ |
| do_something_to (b[0]); |
| |
| /* Process the next packet */ |
| b += 1; |
| next += 1; |
| n_left_from -= 1; |
| } |
| |
| /* |
| * Send the packets along their respective next-node graph arcs |
| * Considerable locality of reference is expected, most if not all |
| * packets in the inbound vector will traverse the same next-node |
| * arc |
| */ |
| vlib_buffer_enqueue_to_next (vm, node, from, nexts, frame->n_vectors); |
| |
| return frame->n_vectors; |
| } |
| |
| Given a packet processing task to implement, it pays to scout around |
| looking for similar tasks, and think about using the same coding |
| pattern. It is not uncommon to recode a given graph node dispatch |
| function several times during performance optimization. |
| |
| Creating Packets from Scratch |
| ----------------------------- |
| |
| At times, it’s necessary to create packets from scratch and send them. |
| Tasks like sending keepalives or actively opening connections come to |
| mind. Its not difficult, but accurate buffer metadata setup is required. |
| |
| Allocating Buffers |
| ~~~~~~~~~~~~~~~~~~ |
| |
| Use vlib_buffer_alloc, which allocates a set of buffer indices. For |
| low-performance applications, it’s OK to allocate one buffer at a time. |
| Note that vlib_buffer_alloc(…) does NOT initialize buffer metadata. See |
| below. |
| |
| In high-performance cases, allocate a vector of buffer indices, and hand |
| them out from the end of the vector; decrement \_vec_len(..) as buffer |
| indices are allocated. See tcp_alloc_tx_buffers(…) and |
| tcp_get_free_buffer_index(…) for an example. |
| |
| Buffer Initialization Example |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| The following example shows the **main points**, but is not to be |
| blindly cut-’n-pasted. |
| |
| .. code:: c |
| |
| u32 bi0; |
| vlib_buffer_t *b0; |
| ip4_header_t *ip; |
| udp_header_t *udp; |
| |
| /* Allocate a buffer */ |
| if (vlib_buffer_alloc (vm, &bi0, 1) != 1) |
| return -1; |
| |
| b0 = vlib_get_buffer (vm, bi0); |
| |
| /* At this point b0->current_data = 0, b0->current_length = 0 */ |
| |
| /* |
| * Copy data into the buffer. This example ASSUMES that data will fit |
| * in a single buffer, and is e.g. an ip4 packet. |
| */ |
| if (have_packet_rewrite) |
| { |
| clib_memcpy (b0->data, data, vec_len (data)); |
| b0->current_length = vec_len (data); |
| } |
| else |
| { |
| /* OR, build a udp-ip packet (for example) */ |
| ip = vlib_buffer_get_current (b0); |
| udp = (udp_header_t *) (ip + 1); |
| data_dst = (u8 *) (udp + 1); |
| |
| ip->ip_version_and_header_length = 0x45; |
| ip->ttl = 254; |
| ip->protocol = IP_PROTOCOL_UDP; |
| ip->length = clib_host_to_net_u16 (sizeof (*ip) + sizeof (*udp) + |
| vec_len(udp_data)); |
| ip->src_address.as_u32 = src_address->as_u32; |
| ip->dst_address.as_u32 = dst_address->as_u32; |
| udp->src_port = clib_host_to_net_u16 (src_port); |
| udp->dst_port = clib_host_to_net_u16 (dst_port); |
| udp->length = clib_host_to_net_u16 (vec_len (udp_data)); |
| clib_memcpy (data_dst, udp_data, vec_len(udp_data)); |
| |
| if (compute_udp_checksum) |
| { |
| /* RFC 7011 section 10.3.2. */ |
| udp->checksum = ip4_tcp_udp_compute_checksum (vm, b0, ip); |
| if (udp->checksum == 0) |
| udp->checksum = 0xffff; |
| } |
| b0->current_length = vec_len (sizeof (*ip) + sizeof (*udp) + |
| vec_len (udp_data)); |
| |
| } |
| b0->flags |= VLIB_BUFFER_TOTAL_LENGTH_VALID; |
| |
| /* sw_if_index 0 is the "local" interface, which always exists */ |
| vnet_buffer (b0)->sw_if_index[VLIB_RX] = 0; |
| |
| /* Use the default FIB index for tx lookup. Set non-zero to use another fib */ |
| vnet_buffer (b0)->sw_if_index[VLIB_TX] = 0; |
| |
| If your use-case calls for large packet transmission, use |
| vlib_buffer_chain_append_data_with_alloc(…) to create the requisite |
| buffer chain. |
| |
| Enqueueing packets for lookup and transmission |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| The simplest way to send a set of packets is to use |
| vlib_get_frame_to_node(…) to allocate fresh frame(s) to ip4_lookup_node |
| or ip6_lookup_node, add the constructed buffer indices, and dispatch the |
| frame using vlib_put_frame_to_node(…). |
| |
| .. code:: c |
| |
| vlib_frame_t *f; |
| f = vlib_get_frame_to_node (vm, ip4_lookup_node.index); |
| f->n_vectors = vec_len(buffer_indices_to_send); |
| to_next = vlib_frame_vector_args (f); |
| |
| for (i = 0; i < vec_len (buffer_indices_to_send); i++) |
| to_next[i] = buffer_indices_to_send[i]; |
| |
| vlib_put_frame_to_node (vm, ip4_lookup_node_index, f); |
| |
| It is inefficient to allocate and schedule single packet frames. That’s |
| typical in case you need to send one packet per second, but should |
| **not** occur in a for-loop! |
| |
| Packet tracer |
| ------------- |
| |
| Vlib includes a frame element [packet] trace facility, with a simple |
| debug CLI interface. The cli is straightforward: “trace add |
| input-node-name count” to start capturing packet traces. |
| |
| To trace 100 packets on a typical x86_64 system running the dpdk plugin: |
| “trace add dpdk-input 100”. When using the packet generator: “trace add |
| pg-input 100” |
| |
| To display the packet trace: “show trace” |
| |
| Each graph node has the opportunity to capture its own trace data. It is |
| almost always a good idea to do so. The trace capture APIs are simple. |
| |
| The packet capture APIs snapshoot binary data, to minimize processing at |
| capture time. Each participating graph node initialization provides a |
| vppinfra format-style user function to pretty-print data when required |
| by the VLIB “show trace” command. |
| |
| Set the VLIB node registration “.format_trace” member to the name of the |
| per-graph node format function. |
| |
| Here’s a simple example: |
| |
| .. code:: c |
| |
| u8 * my_node_format_trace (u8 * s, va_list * args) |
| { |
| vlib_main_t * vm = va_arg (*args, vlib_main_t *); |
| vlib_node_t * node = va_arg (*args, vlib_node_t *); |
| my_node_trace_t * t = va_arg (*args, my_trace_t *); |
| |
| s = format (s, "My trace data was: %d", t-><whatever>); |
| |
| return s; |
| } |
| |
| The trace framework hands the per-node format function the data it |
| captured as the packet whizzed by. The format function pretty-prints the |
| data as desired. |
| |
| Graph Dispatcher Pcap Tracing |
| ----------------------------- |
| |
| The vpp graph dispatcher knows how to capture vectors of packets in pcap |
| format as they’re dispatched. The pcap captures are as follows: |
| |
| :: |
| |
| VPP graph dispatch trace record description: |
| |
| 0 1 2 3 |
| 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| | Major Version | Minor Version | NStrings | ProtoHint | |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| | Buffer index (big endian) | |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| + VPP graph node name ... ... | NULL octet | |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| | Buffer Metadata ... ... | NULL octet | |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| | Buffer Opaque ... ... | NULL octet | |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| | Buffer Opaque 2 ... ... | NULL octet | |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| | VPP ASCII packet trace (if NStrings > 4) | NULL octet | |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| | Packet data (up to 16K) | |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| |
| Graph dispatch records comprise a version stamp, an indication of how |
| many NULL-terminated strings will follow the record header and preceed |
| packet data, and a protocol hint. |
| |
| The buffer index is an opaque 32-bit cookie which allows consumers of |
| these data to easily filter/track single packets as they traverse the |
| forwarding graph. |
| |
| Multiple records per packet are normal, and to be expected. Packets will |
| appear multiple times as they traverse the vpp forwarding graph. In this |
| way, vpp graph dispatch traces are significantly different from regular |
| network packet captures from an end-station. This property complicates |
| stateful packet analysis. |
| |
| Restricting stateful analysis to records from a single vpp graph node |
| such as “ethernet-input” seems likely to improve the situation. |
| |
| As of this writing: major version = 1, minor version = 0. Nstrings |
| SHOULD be 4 or 5. Consumers SHOULD be wary values less than 4 or greater |
| than 5. They MAY attempt to display the claimed number of strings, or |
| they MAY treat the condition as an error. |
| |
| Here is the current set of protocol hints: |
| |
| .. code:: c |
| |
| typedef enum |
| { |
| VLIB_NODE_PROTO_HINT_NONE = 0, |
| VLIB_NODE_PROTO_HINT_ETHERNET, |
| VLIB_NODE_PROTO_HINT_IP4, |
| VLIB_NODE_PROTO_HINT_IP6, |
| VLIB_NODE_PROTO_HINT_TCP, |
| VLIB_NODE_PROTO_HINT_UDP, |
| VLIB_NODE_N_PROTO_HINTS, |
| } vlib_node_proto_hint_t; |
| |
| Example: VLIB_NODE_PROTO_HINT_IP6 means that the first octet of packet |
| data SHOULD be 0x60, and should begin an ipv6 packet header. |
| |
| Downstream consumers of these data SHOULD pay attention to the protocol |
| hint. They MUST tolerate inaccurate hints, which MAY occur from time to |
| time. |
| |
| Dispatch Pcap Trace Debug CLI |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| To start a dispatch trace capture of up to 10,000 trace records: |
| |
| :: |
| |
| pcap dispatch trace on max 10000 file dispatch.pcap |
| |
| To start a dispatch trace which will also include standard vpp packet |
| tracing for packets which originate in dpdk-input: |
| |
| :: |
| |
| pcap dispatch trace on max 10000 file dispatch.pcap buffer-trace dpdk-input 1000 |
| |
| To save the pcap trace, e.g. in /tmp/dispatch.pcap: |
| |
| :: |
| |
| pcap dispatch trace off |
| |
| Wireshark dissection of dispatch pcap traces |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| It almost goes without saying that we built a companion wireshark |
| dissector to display these traces. As of this writing, we have |
| upstreamed the wireshark dissector. |
| |
| Since it will be a while before wireshark/master/latest makes it into |
| all of the popular Linux distros, please see the “How to build a vpp |
| dispatch trace aware Wireshark” page for build info. |
| |
| Here is a sample packet dissection, with some fields omitted for |
| clarity. The point is that the wireshark dissector accurately displays |
| **all** of the vpp buffer metadata, and the name of the graph node in |
| question. |
| |
| :: |
| |
| Frame 1: 2216 bytes on wire (17728 bits), 2216 bytes captured (17728 bits) |
| Encapsulation type: USER 13 (58) |
| [Protocols in frame: vpp:vpp-metadata:vpp-opaque:vpp-opaque2:eth:ethertype:ip:tcp:data] |
| VPP Dispatch Trace |
| BufferIndex: 0x00036663 |
| NodeName: ethernet-input |
| VPP Buffer Metadata |
| Metadata: flags: |
| Metadata: current_data: 0, current_length: 102 |
| Metadata: current_config_index: 0, flow_id: 0, next_buffer: 0 |
| Metadata: error: 0, n_add_refs: 0, buffer_pool_index: 0 |
| Metadata: trace_index: 0, recycle_count: 0, len_not_first_buf: 0 |
| Metadata: free_list_index: 0 |
| Metadata: |
| VPP Buffer Opaque |
| Opaque: raw: 00000007 ffffffff 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 |
| Opaque: sw_if_index[VLIB_RX]: 7, sw_if_index[VLIB_TX]: -1 |
| Opaque: L2 offset 0, L3 offset 0, L4 offset 0, feature arc index 0 |
| Opaque: ip.adj_index[VLIB_RX]: 0, ip.adj_index[VLIB_TX]: 0 |
| Opaque: ip.flow_hash: 0x0, ip.save_protocol: 0x0, ip.fib_index: 0 |
| Opaque: ip.save_rewrite_length: 0, ip.rpf_id: 0 |
| Opaque: ip.icmp.type: 0 ip.icmp.code: 0, ip.icmp.data: 0x0 |
| Opaque: ip.reass.next_index: 0, ip.reass.estimated_mtu: 0 |
| Opaque: ip.reass.fragment_first: 0 ip.reass.fragment_last: 0 |
| Opaque: ip.reass.range_first: 0 ip.reass.range_last: 0 |
| Opaque: ip.reass.next_range_bi: 0x0, ip.reass.ip6_frag_hdr_offset: 0 |
| Opaque: mpls.ttl: 0, mpls.exp: 0, mpls.first: 0, mpls.save_rewrite_length: 0, mpls.bier.n_bytes: 0 |
| Opaque: l2.feature_bitmap: 00000000, l2.bd_index: 0, l2.l2_len: 0, l2.shg: 0, l2.l2fib_sn: 0, l2.bd_age: 0 |
| Opaque: l2.feature_bitmap_input: none configured, L2.feature_bitmap_output: none configured |
| Opaque: l2t.next_index: 0, l2t.session_index: 0 |
| Opaque: l2_classify.table_index: 0, l2_classify.opaque_index: 0, l2_classify.hash: 0x0 |
| Opaque: policer.index: 0 |
| Opaque: ipsec.flags: 0x0, ipsec.sad_index: 0 |
| Opaque: map.mtu: 0 |
| Opaque: map_t.v6.saddr: 0x0, map_t.v6.daddr: 0x0, map_t.v6.frag_offset: 0, map_t.v6.l4_offset: 0 |
| Opaque: map_t.v6.l4_protocol: 0, map_t.checksum_offset: 0, map_t.mtu: 0 |
| Opaque: ip_frag.mtu: 0, ip_frag.next_index: 0, ip_frag.flags: 0x0 |
| Opaque: cop.current_config_index: 0 |
| Opaque: lisp.overlay_afi: 0 |
| Opaque: tcp.connection_index: 0, tcp.seq_number: 0, tcp.seq_end: 0, tcp.ack_number: 0, tcp.hdr_offset: 0, tcp.data_offset: 0 |
| Opaque: tcp.data_len: 0, tcp.flags: 0x0 |
| Opaque: sctp.connection_index: 0, sctp.sid: 0, sctp.ssn: 0, sctp.tsn: 0, sctp.hdr_offset: 0 |
| Opaque: sctp.data_offset: 0, sctp.data_len: 0, sctp.subconn_idx: 0, sctp.flags: 0x0 |
| Opaque: snat.flags: 0x0 |
| Opaque: |
| VPP Buffer Opaque2 |
| Opaque2: raw: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 |
| Opaque2: qos.bits: 0, qos.source: 0 |
| Opaque2: loop_counter: 0 |
| Opaque2: gbp.flags: 0, gbp.src_epg: 0 |
| Opaque2: pg_replay_timestamp: 0 |
| Opaque2: |
| Ethernet II, Src: 06:d6:01:41:3b:92 (06:d6:01:41:3b:92), Dst: IntelCor_3d:f6 Transmission Control Protocol, Src Port: 22432, Dst Port: 54084, Seq: 1, Ack: 1, Len: 36 |
| Source Port: 22432 |
| Destination Port: 54084 |
| TCP payload (36 bytes) |
| Data (36 bytes) |
| |
| 0000 cf aa 8b f5 53 14 d4 c7 29 75 3e 56 63 93 9d 11 ....S...)u>Vc... |
| 0010 e5 f2 92 27 86 56 4c 21 ce c5 23 46 d7 eb ec 0d ...'.VL!..#F.... |
| 0020 a8 98 36 5a ..6Z |
| Data: cfaa8bf55314d4c729753e5663939d11e5f2922786564c21… |
| [Length: 36] |
| |
| It’s a matter of a couple of mouse-clicks in Wireshark to filter the |
| trace to a specific buffer index. With that specific kind of filtration, |
| one can watch a packet walk through the forwarding graph; noting any/all |
| metadata changes, header checksum changes, and so forth. |
| |
| This should be of significant value when developing new vpp graph nodes. |
| If new code mispositions b->current_data, it will be completely obvious |
| from looking at the dispatch trace in wireshark. |
| |
| pcap rx, tx, and drop tracing |
| ----------------------------- |
| |
| vpp also supports rx, tx, and drop packet capture in pcap format, |
| through the “pcap trace” debug CLI command. |
| |
| This command is used to start or stop a packet capture, or show the |
| status of packet capture. Each of “pcap trace rx”, “pcap trace tx”, and |
| “pcap trace drop” is implemented. Supply one or more of “rx”, “tx”, and |
| “drop” to enable multiple simultaneous capture types. |
| |
| These commands have the following optional parameters: |
| |
| - rx - trace received packets. |
| |
| - tx - trace transmitted packets. |
| |
| - drop - trace dropped packets. |
| |
| - max *nnnn*\ - file size, number of packet captures. Once packets |
| have been received, the trace buffer buffer is flushed to the |
| indicated file. Defaults to 1000. Can only be updated if packet |
| capture is off. |
| |
| - max-bytes-per-pkt *nnnn*\ - maximum number of bytes to trace on a |
| per-packet basis. Must be >32 and less than 9000. Default value: |
| |
| 512. |
| |
| - filter - Use the pcap rx / tx / drop trace filter, which must be |
| configured. Use classify filter pcap… to configure the filter. The |
| filter will only be executed if the per-interface or any-interface |
| tests fail. |
| |
| - intfc *interface* \| *any*\ - Used to specify a given interface, or |
| use ‘any’ to run packet capture on all interfaces. ‘any’ is the |
| default if not provided. Settings from a previous packet capture are |
| preserved, so ‘any’ can be used to reset the interface setting. |
| |
| - file *filename*\ - Used to specify the output filename. The file |
| will be placed in the ‘/tmp’ directory. If *filename* already exists, |
| file will be overwritten. If no filename is provided, ‘/tmp/rx.pcap |
| or tx.pcap’ will be used, depending on capture direction. Can only be |
| updated when pcap capture is off. |
| |
| - status - Displays the current status and configured attributes |
| associated with a packet capture. If packet capture is in progress, |
| ‘status’ also will return the number of packets currently in the |
| buffer. Any additional attributes entered on command line with a |
| ‘status’ request will be ignored. |
| |
| - filter - Capture packets which match the current packet trace filter |
| set. See next section. Configure the capture filter first. |
| |
| packet trace capture filtering |
| ------------------------------ |
| |
| The “classify filter pcap \| \| trace” debug CLI command constructs an |
| arbitrary set of packet classifier tables for use with “pcap rx \| tx \| |
| drop trace,” and with the vpp packet tracer on a per-interface or |
| system-wide basis. |
| |
| Packets which match a rule in the classifier table chain will be traced. |
| The tables are automatically ordered so that matches in the most |
| specific table are tried first. |
| |
| It’s reasonably likely that folks will configure a single table with one |
| or two matches. As a result, we configure 8 hash buckets and 128K of |
| match rule space by default. One can override the defaults by specifying |
| “buckets ” and “memory-size ” as desired. |
| |
| To build up complex filter chains, repeatedly issue the classify filter |
| debug CLI command. Each command must specify the desired mask and match |
| values. If a classifier table with a suitable mask already exists, the |
| CLI command adds a match rule to the existing table. If not, the CLI |
| command add a new table and the indicated mask rule |
| |
| Configure a simple pcap classify filter |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| :: |
| |
| classify filter pcap mask l3 ip4 src match l3 ip4 src 192.168.1.11 |
| pcap trace rx max 100 filter |
| |
| Configure a simple per-interface capture filter |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| :: |
| |
| classify filter GigabitEthernet3/0/0 mask l3 ip4 src match l3 ip4 src 192.168.1.11" |
| pcap trace rx max 100 intfc GigabitEthernet3/0/0 |
| |
| Note that per-interface capture filters are *always* applied. |
| |
| Clear per-interface capture filters |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| :: |
| |
| classify filter GigabitEthernet3/0/0 del |
| |
| Configure another fairly simple pcap classify filter |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| :: |
| |
| classify filter pcap mask l3 ip4 src dst match l3 ip4 src 192.168.1.10 dst 192.168.2.10 |
| pcap trace tx max 100 filter |
| |
| Configure a vpp packet tracer filter |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| :: |
| |
| classify filter trace mask l3 ip4 src dst match l3 ip4 src 192.168.1.10 dst 192.168.2.10 |
| trace add dpdk-input 100 filter |
| |
| Clear all current classifier filters |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| :: |
| |
| classify filter [pcap | <interface> | trace] del |
| |
| To inspect the classifier tables |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| :: |
| |
| show classify table [verbose] |
| |
| The verbose form displays all of the match rules, with hit-counters. |
| |
| Terse description of the “mask ” syntax: |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| :: |
| |
| l2 src dst proto tag1 tag2 ignore-tag1 ignore-tag2 cos1 cos2 dot1q dot1ad |
| l3 ip4 <ip4-mask> ip6 <ip6-mask> |
| <ip4-mask> version hdr_length src[/width] dst[/width] |
| tos length fragment_id ttl protocol checksum |
| <ip6-mask> version traffic-class flow-label src dst proto |
| payload_length hop_limit protocol |
| l4 tcp <tcp-mask> udp <udp_mask> src_port dst_port |
| <tcp-mask> src dst # ports |
| <udp-mask> src_port dst_port |
| |
| To construct **matches**, add the values to match after the indicated |
| keywords in the mask syntax. For example: “… mask l3 ip4 src” -> “… |
| match l3 ip4 src 192.168.1.11” |
| |
| VPP Packet Generator |
| -------------------- |
| |
| We use the VPP packet generator to inject packets into the forwarding |
| graph. The packet generator can replay pcap traces, and generate packets |
| out of whole cloth at respectably high performance. |
| |
| The VPP pg enables quite a variety of use-cases, ranging from functional |
| testing of new data-plane nodes to regression testing to performance |
| tuning. |
| |
| PG setup scripts |
| ---------------- |
| |
| PG setup scripts describe traffic in detail, and leverage vpp debug CLI |
| mechanisms. It’s reasonably unusual to construct a pg setup script which |
| doesn’t include a certain amount of interface and FIB configuration. |
| |
| For example: |
| |
| :: |
| |
| loop create |
| set int ip address loop0 192.168.1.1/24 |
| set int state loop0 up |
| |
| packet-generator new { |
| name pg0 |
| limit 100 |
| rate 1e6 |
| size 300-300 |
| interface loop0 |
| node ethernet-input |
| data { IP4: 1.2.3 -> 4.5.6 |
| UDP: 192.168.1.10 - 192.168.1.254 -> 192.168.2.10 |
| UDP: 1234 -> 2345 |
| incrementing 286 |
| } |
| } |
| |
| A packet generator stream definition includes two major sections: - |
| Stream Parameter Setup - Packet Data |
| |
| Stream Parameter Setup |
| ~~~~~~~~~~~~~~~~~~~~~~ |
| |
| Given the example above, let’s look at how to set up stream parameters: |
| |
| - **name pg0** - Name of the stream, in this case “pg0” |
| |
| - **limit 1000** - Number of packets to send when the stream is |
| enabled. “limit 0” means send packets continuously. |
| |
| - **maxframe <nnn>** - Maximum frame size. Handy for injecting multiple |
| frames no larger than <nnn>. Useful for checking dual / quad loop |
| codes |
| |
| - **rate 1e6** - Packet injection rate, in this case 1 MPPS. When not |
| specified, the packet generator injects packets as fast as possible |
| |
| - **size 300-300** - Packet size range, in this case send 300-byte |
| packets |
| |
| - **interface loop0** - Packets appear as if they were received on the |
| specified interface. This datum is used in multiple ways: to select |
| graph arc feature configuration, to select IP FIBs. Configure |
| features e.g. on loop0 to exercise those features. |
| |
| - **tx-interface <name>** - Packets will be transmitted on the |
| indicated interface. Typically required only when injecting packets |
| into post-IP-rewrite graph nodes. |
| |
| - **pcap <filename>** - Replay packets from the indicated pcap capture |
| file. “make test” makes extensive use of this feature: generate |
| packets using scapy, save them in a .pcap file, then inject them into |
| the vpp graph via a vpp pg “pcap <filename>” stream definition |
| |
| - **worker <nn>** - Generate packets for the stream using the indicated |
| vpp worker thread. The vpp pg generates and injects O(10 MPPS / |
| core). Use multiple stream definitions and worker threads to generate |
| and inject enough traffic to easily fill a 40 gbit pipe with small |
| packets. |
| |
| Data definition |
| ~~~~~~~~~~~~~~~ |
| |
| Packet generator data definitions make use of a layered implementation |
| strategy. Networking layers are specified in order, and the notation can |
| seem a bit counter-intuitive. In the example above, the data definition |
| stanza constructs a set of L2-L4 headers layers, and uses an |
| incrementing fill pattern to round out the requested 300-byte packets. |
| |
| - **IP4: 1.2.3 -> 4.5.6** - Construct an L2 (MAC) header with the ip4 |
| ethertype (0x800), src MAC address of 00:01:00:02:00:03 and dst MAC |
| address of 00:04:00:05:00:06. Mac addresses may be specified in |
| either *xxxx.xxxx.xxxx* format or *xx:xx:xx:xx:xx:xx* format. |
| |
| - **UDP: 192.168.1.10 - 192.168.1.254 -> 192.168.2.10** - Construct an |
| incrementing set of L3 (IPv4) headers for successive packets with |
| source addresses ranging from .10 to .254. All packets in the stream |
| have a constant dest address of 192.168.2.10. Set the protocol field |
| to 17, UDP. |
| |
| - **UDP: 1234 -> 2345** - Set the UDP source and destination ports to |
| 1234 and 2345, respectively |
| |
| - **incrementing 256** - Insert up to 256 incrementing data bytes. |
| |
| Obvious variations involve “s/IP4/IP6/” in the above, along with |
| changing from IPv4 to IPv6 address notation. |
| |
| The vpp pg can set any / all IPv4 header fields, including tos, packet |
| length, mf / df / fragment id and offset, ttl, protocol, checksum, and |
| src/dst addresses. Take a look at ../src/vnet/ip/ip[46]_pg.c for |
| details. |
| |
| If all else fails, specify the entire packet data in hex: |
| |
| - **hex 0xabcd…** - copy hex data verbatim into the packet |
| |
| When replaying pcap files (“**pcap <filename>**”), do not specify a data |
| stanza. |
| |
| Diagnosing “packet-generator new” parse failures |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| If you want to inject packets into a brand-new graph node, remember to |
| tell the packet generator debug CLI how to parse the packet data stanza. |
| |
| If the node expects L2 Ethernet MAC headers, specify “.unformat_buffer = |
| unformat_ethernet_header”: |
| |
| .. code:: c |
| |
| VLIB_REGISTER_NODE (ethernet_input_node) = |
| { |
| <snip> |
| .unformat_buffer = unformat_ethernet_header, |
| <snip> |
| }; |
| |
| Beyond that, it may be necessary to set breakpoints in |
| …/src/vnet/pg/cli.c. Debug image suggested. |
| |
| When debugging new nodes, it may be far simpler to directly inject |
| ethernet frames - and add a corresponding vlib_buffer_advance in the new |
| node - than to modify the packet generator. |
| |
| Debug CLI |
| --------- |
| |
| The descriptions above describe the “packet-generator new” debug CLI in |
| detail. |
| |
| Additional debug CLI commands include: |
| |
| :: |
| |
| vpp# packet-generator enable [<stream-name>] |
| |
| which enables the named stream, or all streams. |
| |
| :: |
| |
| vpp# packet-generator disable [<stream-name>] |
| |
| disables the named stream, or all streams. |
| |
| :: |
| |
| vpp# packet-generator delete <stream-name> |
| |
| Deletes the named stream. |
| |
| :: |
| |
| vpp# packet-generator configure <stream-name> [limit <nnn>] |
| [rate <f64-pps>] [size <nn>-<nn>] |
| |
| Changes stream parameters without having to recreate the entire stream |
| definition. Note that re-issuing a “packet-generator new” command will |
| correctly recreate the named stream. |