Internet-Draft Juniper's Secure Vector Routing (SVR) June 2024
Menon, et al. Expires 19 December 2024 [Page]
Workgroup:
Network Working Group
Internet-Draft:
draft-menon-svr-06
Published:
Intended Status:
Informational
Expires:
Authors:
A. Menon
Maia Tech
P. MeLampy
Retired
M. Baj
Juniper Networks
P. Timmons
Maia Tech
H. Kaplan
Juniper Networks

Juniper's Secure Vector Routing (SVR)

Abstract

This document describes Juniper's Secure Vector Routing (SVR). SVR is an overlay inter-networking protocol that operates at the session layer. SVR provides end-to-end communication of network requirements not possible or practical using network header layers. SVR uses application layer cookies that eliminate the need to create and maintain non-overlapping address spaces necessary to manage network routing requirements. SVR is an overlay networking protocol that works through middleboxes and address translators including those existing between private networks, the IPv4 public internet, and the IPv6 public internet. SVR enables SD-WAN and multi-cloud use cases and improves security at the networking routing plane. SVR eliminates the need for encapsulation and decapsulation often used to create non-overlapping address spaces.

This draft provides information for the Internet community. It does not specify an Internet standard that has IETF consensus, nor is this part of a standards track of the IETF. This document is being published for the purposes of interoperability. The authors request suggestions for improvement.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 19 December 2024.

▲

Table of Contents

1. Introduction

There exists a need to communicate network requirements between IP routers and networks to provide an end-to-end experience. Selection of specific paths whose attributes meet or exceed the networking requirements are an objective of SVR. There is also a need for applications to communicate their requirements to networks. This need is increasing as workloads move to public clouds and the numbers of cloud locations increase. The standard practice today is to use an overlay network of tunnels to create a virtual network. SVR is proposed as an alternative to using tunnels. SVR simplifies the network by virtue of having only one network layer. SVR securely transports traffic with authentication and adaptive encryption. The absence of tunneling overhead reduces bandwidth. Since SVR specifies requirements abstractly, it also has the capability to interwork policies between different networks and address spaces.

Most WAN networks are deployed with a virtual private network (VPN) across IP backbone facilities. VPNs have the significant disadvantage of carrying additional network layers increasing packet size and leading to IP fragmentation as well as reduced bandwidth.

1.1. Terminology

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

1.2. Overview

An SVR implementation describes a network requirement semantically and shares this as SVR Metadata with a routing peer. The requirement to a peer is conveyed by means of a cookie, often referred to as first packet SVR Metadata, which is placed in the first packet of a session that is targeted towards the SVR Peer. SVR requires session state on every participating SVR router and sets up a bi-flow (matching forward and reverse flows) based on the requirement. Once the session is established bi-directionally, the cookie is not sent in subsequent packets, resulting in elimination of additional overhead.

Benefits from this approach include:

  • Tunnel Compression: The SVR Metadata contains information required to eliminate tunnel header information for established sessions. This can result in anywhere from 12% to 100% bandwidth savings when compared to IPSEC based tunnels depending on the original packet size.
  • Elimination of Tunnel Generated Elephant Flow problems: Tunnels are very long lived and often contain large aggregates of inner flows. Tunnels are often fixed on a specific network ECMP path or "hash" while each SVR session has a unique ECMP path.
  • QoS support is per flow, not per packet: Because each SVR flow has a unique 5-tuple on the wire, standard MPLS routing and QoS techniques work seamlessly. Adding QoS to Tunnels requires QoS on entry to a tunnel, tunnel DSCP markings, and policies to copy/map inner packet DSCP to Tunnel Packet DSCP. In practice many core networks do not look at the DSCP markings once a fast path forwarding rules are established.
  • Avoid Re-encryption: Tunnels often encrypt all traffic. Much of the traffic in the tunnel is already encrypted, thus there is a re-encryption penalty. SVR support adaptive encryption which performs encryption on only those sessions that require it.
  • Firewalls and security proxies can intercept TLS sessions and perform decryption and encryption if they tolerate SVR Metadata. This is not possible with IPSEC tunnels by design.
  • Scaling of software-based encryption is much higher when session state is available. Encryption performance is limited to what is possible in a single processing core for a single session, and at the time of this document being written the limit is currently 1.5GigE for Tunnel termination.

1.3. Document Organization

This document is structured into six major sections. Section 2 provides a high-level theory of how SVR works. Section 3 presents an example of SVR in action. Section 4 describes the SVR message handling in detail. SVR Peer management is described in Section 5. Section 6 details some additional SVR procedures that may be required. Finally, Section 7 describes individual protocol messages.

1.4. Definitions

The following terms are used throughout this document.

Authority:
A textual definition of the owner of an SVR namespace. Each namespace owner can allocate Tenant names (representing collections of network endpoints sharing common network enforcement policy), and Service names (representing accessible destinations and traffic treatment policy). Authority namespaces should be unique if internetworking is desired. Claiming and resolving disputes about Authority naming are outside the scope of this document.
Tenant(s):
This is a textual description defining network endpoints that share common access policy (allow lists or block lists to network destinations). These may be mapped using any known technique including source IP address mask, a VLAN tag, ingress interface, provided by an authentication system, or even client supplied, and this mapping is outside the scope of this document. Often these are location specific definitions, but the Tenant has an Authority wide scope. Tenant names can conform to domain name syntax, and be expressed as hierarchical structures (e.g., location.department.example).
Session:
A session is the entire sequence of packets in both directions that represents a single TCP or UDP communication. The initiator of the sessions is always the client, and the answering side is always the server.
Service(s):
This is a textual description of what server(s) can be accessed with this intent. Examples include Zoom, or Office365/Outlook. Although outside the scope of this document, these could be defined with any known technique, including URLs, IP address(es) protocol(s) and port(s), CIDR block(s), etc. Having a single text name to describe a network destination makes defining network requirements easier. Other Service specific network requirements including Quality Policies and Security Policies can be associated with Services in data models, but are not described in this document.
Context:
This is the original "5-tuple" of an IP packet, including source IP, source port, destination IP, destination port, and protocol. Optionally, Layer 2 information such as MAC Address or VLAN tags may be included for certain use cases, if required.
Signature:
SVR Metadata packets MUST be cryptographically signed using HMAC by the source router, and all packets traversing an SVR peer pathway SHOULD have an HMAC signature so the next hop router can authenticate the sender of the data and verify its integrity. The portion of the packet that is signed must not include the IP header, as it may go through a NAT or IPv4 to IPv6 conversion.
Direction:
This is inferred, and not a specific SVR Metadata field. The Direction represents the intended client to server direction. The initial network packet of a communication session indicates this direction. For example, a TCP SYN packet would travel from client to server, defining the direction of a service. Forward direction is always client to server, and reverse is always server to the client. These directions have nothing to do with a network topology (for example, hub and spoke), and a single network path could have forward sessions going bi-directionally -- traffic going from node A to node B may represent the forward direction for some sessions and the reverse direction for other sessions.
Peer:
An SVR Peer is a client, server, or router that supports the SVR protocol. The SVR Peer could be either directly adjacent, or reachable through an IP network. The SVR Peer should not be confused with BGP Peer. Since SVR Peers must be able to reach each other, and because SVR Peers are often deployed at network edges, SVR Peers can also be BGP Peers. In this document peer will always mean SVR Peer.
Waypoint:
A Waypoint is a reachable IP Address associated with an SVR Router's interface. Some physical interfaces may have multiple IP Addresses, and as such a single physical interface could represent multiple Waypoints. In some cases, routers use dynamically assigned addresses on interfaces. In these cases, a Waypoint address may change dynamically.
Peer Received IP Address:
This is the destination IP address to send packets to reach a Waypoint Address. Normally, this is the same IP Address as a Waypoint Address, unless there is a NAT present between Peers.
SVR Metadata:
SVR Metadata is a block of TLVs that contain SVR attributes described in Section 7. This block of data communicates network information between SVR routers.
BFD Metadata:
BFD Metadata refers to data added to BFD messages to extend standardized support required for Peer relationships. See Section 5.
Peer Pathway:
An SVR Peer Pathway is a unique pair of Waypoint addresses that can reach each other. The path can be defined as either a pair of IP addresses or a pair of domain names that resolve to IP Addresses. Peer Pathways have attributes related to availability, performance (jitter, latency, packet loss) and cost. Techniques such as BFD [RFC5880] can ensure a Peer Pathway's state and readiness for packet transmission.
Router Certificate:
A Certificate Signing Request (CSR) is created by every router that attaches to an SVR network that contains the routers UUID, Authority, and public key. The resulting certificate is used to authenticate SVR routes on Peer Pathways. The certificate (and public key) are fairly long lived, and seldom used. Keying procedures use derived key functions based on the certificate.
Peer Key:
After authentication, every SVR router peer pathway creates a Peer Key that represents the Peer Relationship. This key is solely used to encrypt certain BFD fields between two peers. The Peer Key is rekeyed by calculating a new shared key as often as required See Section 5.1.8.
SVR Metadata Key:
Each peer will broadcast to all of its peers a key to use for metadata decryption. The key is delivered securely as an encrypted field in BFD Metadata. This permits a single key for each router to be used for metadata decryption for all Peer Paths. See Section 5.1.9.
Payload Key:
Each peer will generate a key for a session that requires payload encryption. This key will be used only by this specific session. The key is generated by the originating router and is included in metadata, which avoids race conditions by using the same key for reverse traffic. Re-keying can be performed by generating a new key, and sending metadata in the first packet encrypted with the new key.
Session HMAC Key:
Timed Based HMAC signatures can be used to protect SVR pathways against replay attacks. Upon start, every session creates a Session HMAC Key which is the Peer Key at the time the session was created. Session HMAC Keys must be saved for the life of a session, and do not change. Time based HMAC signatures essentially change the key every 2 seconds.

2. Theory of operation of Secure Vector Routing

Secure Vector Routing is a session stateful routing protocol that operates at edges of networks where stateful NATs are normally performed. It is at these same locations where multi-path routing is being deployed. These locations include edge routers located at branches, data centers, and public clouds. SVR maps local network requirements into administratively defined text strings that have Authority wide meaning. These are communicated or signaled by insertion of a networking cookie called SVR Metadata directly into IP Packets in transit.

                       +----------+
                       | Network2 |
   +-----------+       |          |        +-----------+
   |          SVR<---->+<-L3-IP-->+<----->SVR          |
   |           |       +----------+        |           |
   | Network1  |       +----------+        | Network 4 |
   |          SVR<--->SVR       SVR<----->SVR          |
   +-----------+       |          |        +-----------+
                       | Network3 |
   +-----------+       |          |
   | Client   SVR<--->SVR         |
   +-----------+       +----------+
Figure 1

The above diagram shows typical places where SVR can be deployed. SVR can be deployed between Network 1 and Network 4, traversing Network 3. This is typical of a connection between a corporate site and a data center through the public internet. SVR can be deployed directly between Network 1 and Network 3, Network 3 and Network 4 to build a multi-network connectivity. Clients that support SVR is connected to Network 3, permitting full SVR based access to Networks 1, 3, and 4.

SVR Metadata is inserted into existing packets directly after the L4 header (see Section 4.2). The SVR Metadata in the first packet of a new session (TCP or UDP bidirectional flow) can be used for path selection and security. SVR Metadata can be inserted in any subsequent packet to change/update the networking requirements. The SVR Metadata is inserted into the payload portion of a packet to guarantee it makes it unchanged between SVR routers.

Sessions supported by SVR include TCP, UDP, UDP Unicast, point-to-point ethernet, and ICMP. Sessions are characterized by having an initial first packet that is unique to an SVR router. Often this is described as a unique 5-tuples as seen by the router. Sessions start when the first packet is processed, and end when either the L4 protocol indicates the session is completed (TCP FIN/FIN ACK) or there has been no activity for a length of time (UDP, ICMP, UDP Unicast, point-to-point ethernet).

2.1. Directionality

SVR utilizes the concept of session direction. The direction of the session is what creates a Secure Vector. Routing policies include a Tenant (source) and Service (destination) pair that exactly match the direction of sessions. When describing SVR Metadata in this document, direction is either forward or reverse; it is not tied to network topology, but rather the direction of session establishment. For TCP, the forward direction is always the client side towards the server side. For UDP, the forward direction is from the sender of the first packet. Reverse is the opposite direction. On a given pathway, Secure Vector Routes could be traversing on the same pathways with opposite directions.

SVR Metadata formats described in this document will be labeled as "forward" or "reverse". Forward SVR Metadata is inserted in packets going from client to server. Reverse SVR Metadata is inserted in packets that travel from server to client.

2.2. Order of Operations

The basic order of operations for the first SVR Router encountered includes:

Receive First Packet of a new Flow:
SVR Routers have active flow tables. When a packet arrives, whose addresses are not in the flow table, the router considers this a first packet of a new flow.
Parse and Route:
SVR Routers will parse all of the L2, and L3 information in the network headers. The SVR Router will perform longest prefix matches to find possible next-hop routers to forward the packet to.
If Next Hop Router is an SVR Router:
If the next-hop router supports SVR, then SVR Procedures as described in this document can be used. If not, the packet is forwarded in a traditional way.
Insert SVR Metadata:
When the next-hop router supports SVR, SVR Metadata is inserted into the very first packet. This provides valuable networking information to the next-hop router.
Update Transport Addresses:
If SVR Metadata is inserted, the transport addresses are updated to steer the packet directly to the next-hop SVR router (Think IPv6 Segment Routing) with the from addresses being the initiating router. This new address pair reflects the chosen transport path between two routers bi-directionally.
Forward Packet:
The SVR Packet with SVR metadata included is forwarded to the next-hop SVR router.

The basic order of operations for a subsequent SVR Router includes:

Receive First Packet of a new SVR Flow:
When a first packet arrives that is not in the flow table AND it has as a destination the routers interface address, the packet must be from a known SVR Peer, and must contain SVR metadata.
Parse and Remove Metadata:
SVR Metadata is decrypted and authenticated. The SVR information can be used in the routing of the packet. The SVR metadata is removed, and the packet addresses are restored to their original values.
Parse and Route:
Subsequent SVR Routers will parse all of the L2, and L3 information in the network headers. The SVR Router will perform longest prefix matches to find possible next-hop routers to forward the packet to. The SVR Metadata contents can be used for route selection.
If Next Hop Router is an SVR Router:
If the next-hop router supports SVR, then SVR Procedures as described in this document can be used. If not, the packet is forwarded in a traditional way.
Insert SVR Metadata:
When the next-hop router supports SVR, SVR Metadata is inserted into the very first packet. This provides valuable networking information to the next-hop router.
Update Transport Addresses:
If SVR Metadata is inserted, the transport addresses are updated to steer the packet directly to the next-hop SVR router (Think IPv6 Segment Routing) with the from addresses being the initiating router. This new address pair reflects the chosen transport path between two routers bi-directionally.
Forward Packet:
The SVR Packet with SVR metadata included is forwarded to the next-hop SVR router.

2.3. SVR with Other Traffic

SVR co-exists with traditional routing. In fact, the router interface addresses known as Waypoints in this document MUST be reachable via traditional networking for every peer relationship. When packet routing is being decided in the router, should the route resolve to an SVR capable router (i.e., the next hop address returned in the route equals a known Waypoint address of an SVR Peer) then SVR Metadata MAY be inserted and session stateful SVR is performed. Otherwise, the packet is forwarded like any traditional IP router.

2.4. SVR Metadata Handshake

To ensure the SVR Metadata is received and understood between peers, a handshake is performed for each routed session.

A sender of SVR metadata confirms it's receipt by receiving a subsequent backward SVR metadata from its peer. Senders must include metadata in every packet until they receive this confirmation. Once confirmed, the sender can stop sending metadata in subsequent packets.

A receiver of SVR metadata from a peer always responds by sending reverse SVR metadata in every subsequent packet of a session. This continues until a packet is received from the peer that no longer has metadata, signaling receipt.

These two comprise what is known as the "SVR Metadata handshake" -- that is, the initiating router includes SVR Metadata in all packets it sends to the recipient router until it receives a reverse packet with SVR Metadata from that recipient. Likewise, the recipient continues to send SVR Metadata to the initiating router until it receives a packet without SVR Metadata. This is how two routers acknowledge receipt of SVR Metadata from their counterparts: the absence of SVR Metadata in a packet indicates that it has received SVR Metadata from its counterpart.

2.5. Pathway Obstructions and Changes

Firewalls and middleboxes that sit along a peer pathway may not propagate TCP SYN messages with data in the payload (Despite being valid), or may verify sequence numbers in TCP streams (which are invalidated due to the inclusion of SVR Metadata). The two devices that represent the peer pathway endpoints may determine through testing if there is a firewall, NAT, or other active middlebox between the two routers. The BFD protocol with SVR Metadata can be used to detect the presence of a NAT. See Section 5.1.2. Additional procedures like STUN [RFC8489], TURN [RFC6062], and ICE [RFC8445] are well-known, and not included in this document.

If a NAT is detected on the Peer Pathway, the SVR Router that determines its Waypoint address is being changed saves this as an attribute of the pathway. The NAT will change the port address assignment, and require NAT keep-alives as exemplified in Section 6.3.

If a middlebox is detected, the packets can be UDP-transformed (i.e., the protocol byte can be changed from TCP to UDP) by the transmitting router and restored to TCP by the receiving router for packets flowing in both directions. See Section 4.2.7 and Section 4.6.3 for more information.

When routers use IP addresses that are dynamic, such as DHCP served addresses or PPPoE network attachments, it's possible to be assigned a new address. If this happens all existing sessions using that Waypoint address must be updated to use the new address. For existing sessions this can be performed in real time be reviewing the sending address. If the address is changed, internal references to the old address can be updated. For idle circuits, BFD with SVR Metadata is used to detect address changes. See Section 5.1.3 for details.

2.6. SVR Metadata removal

To prevent breaking any applications, there MUST be a 100% guarantee that SVR Metadata inserted by a participating SVR device is removed prior to the consumption of the data by the application service. If the client and server support SVR Metadata, then SVR Metadata can be sent end-to-end. When a mid-stream packet router wants to insert SVR Metadata, it must guarantee that the packet is directed to a next hop device that will understand and remove the SVR Metadata.

A router can be certain an SVR capable router is on the path when the next-hop address returned from a FIB table exactly matches a known peer Waypoint address. Before sending the packet with SVR Metadata to the Waypoint address, the originating SVR router should determine the Peer reachability as exemplified in Section 3 and Section 5.

If the next-hop is not a known reachable peer, SVR Metadata insertion MUST NOT be performed.

2.7. Modification of transport addresses

To guarantee that the packet will go to a specific router, the destination address for the packet is changed to the Waypoint Address of the chosen peer. The original addresses are stored in the forward context (see Section 7.4.1) and can be recovered when needed. This is similar to IPv6 segment routing (see [RFC8986]) or a LISP (see [RFC9300]) RLOC with the exception that the original addresses are stored in SVR Metadata within the payload portion of the packet, and not the IP Network Header.

Selection of the Waypoint Address to direct sessions is implementation specific. In the general case a standard FIB lookup returns one or more IP Address(es) (Waypoints) of the next SVR peer. When more than one Waypoint address is returned from the FIB, additional logic can be applied to select the best Waypoint based on observed peer pathway quality OR session layer load balancing. See Section 3 for exemplary details.

To provide a return path for the return flow the source SVR peer changes the source address to be its own egress Waypoint address. This provides a guarantee of a symmetric flow. The state of the session MUST be held in both the source SVR router and the destination SVR peer.

The address translation rules for the session become state information that is processed on every packet after the SVR Metadata handshake. All 5 tuples of addressing information are updated bidirectionally for the session. This action replaces tunnel encapsulation and decapsulation (tunnel compression), and is an order of magnitude simpler computationally.

2.8. Optional use of Tenants and Service names for Routing

SVR Metadata contains contextual IP Addresses (sources, destinations, and Waypoints) along with textual service names (i.e., Zoom, Office365, etc.). The SVR routers can apply policies and route sessions based on the textual names if they have a route information base that contains service names. When performing name based routing, a destination NAT is often required when exiting the SVR network. The primary use case for this is networking between public clouds such as AWS and Azure.

With semantic based routing, the use of Dynamic DNS to locate a service can be eliminated if clients support SVR. Clients can simply request the service by name, and the SVR router can resolve the route, and deliver the session to the best location. The last SVR Router on egress performs a destination NAT for the chosen best instance of a service.

A local DNS server resolving service addresses to a nearby SVR router can also provide semantic based routing. This can eliminate the need to use dynamic DNS for locating services inside data centers.

2.9. Unique 5-Tuples for Every Session

To avoid sharing a hash with all traffic, and to make sessions completely independent on peer pathways, the source port and destination port can be assigned any values that are unique by the source router. When there are no NATs between the two router interfaces, this permits 2^32 (4,294,967,296) different unique sessions on a peer pathway. If there are source NATs, this will be reduced to 2^16 (65,536) different unique sessions. Ports can be reassigned if not in active use. It is also possible that middle boxes will limit what destination ports are permissible, reducing the number of possibilities. Due to all these reasons, range of ports that can be used on a peer pathway are provisioned by an administrator.

The ingress SVR peer (client side) assigns both source and destination ports, even ports for local (source port) and odd ports for remote (destination port). This provides total uniqueness between any two peers, with no negotiation or collision possibilities. This reduces the number of sessions originating by a router to half of the total sessions (or 2^30). Think of the two ports as a Session Identification Tag. Even if a session traveling in the opposite direction was allocated the same exact ports, because the source address and destination addresses would be swapped, the 5-tuples on the wire remain unique.

This unique tuple per TCP/UDP session also allows any DSCP or QoS scheme to work properly. Those fields in the original packet were not modified and the underlay network routers will see those fields on a session-by-session basis.

2.10. Session Packets Post SVR Metadata Exchange

After the SVR Metadata handshake has been completed, all subsequent packets are transformed (all 5-tuples, bidirectionally). Compared to IPSec encapsulation, packet transformation is very efficient as it does not require memory copies, new header creation, new packet checksums, and mandatory encryption.

2.11. Session State Requirements

Each participant (peer) in secure vector routing must maintain state for every active session. This includes the full set of original addresses and translations required. This allows participants to stop sending SVR Metadata once it has been received by the peer as well as directing traffic through a network akin to segment routing. There are three possible scenarios for how state could be, lost. Loss of state can result in sessions that become stale.

  • SVR Ingress Router Loses State: The session state at a router that originated the SVR session loses state. This could happen during a redundancy or power failure.
  • SVR Peer Router Loses State: The session state is lost in an intermediate (2nd to nth) router processing an SVR session.
  • One or more middleboxes lose state between two SVR routers, breaking or modifying the session.
Reacquiring State:
In all cases, securely reacquiring and/or reestablishing the state from an SVR peer for a session is necessary. If neither peer has session state, the session can be considered a new session and treated like any first packet. See Section 3.7.1. Detection of potential loss of state is performed on every SVR router for every session at all times. Just prior to transmitting a packet from an SVR router for a session, the elapsed time since a packet was received from an SVR peer is compared to an inactivity timeout. The inactivity timeout is provisioned, and has a recommended value of 5 seconds. If the inactivity timeout is exceeded, then a loss of session state MAY be indicated. Note that this logic has no relationship with session timers guarding session state against no activity.
Unicast Flows:
For unicast flows, or asynchronous sessions, consisting of packets in one direction, detection of potential loss of state will occur frequently. This will result in this inactivity timeout occurring on a routine basis for these types of sessions.
Potential Loss of State:
If a potential loss of session state is indicated, then a Session Health Check SVR Metadata is inserted in the packet being transmitted. When the SVR peer receives Session Health Check SVR Metadata, if the session is valid, and simply idle, a unicast flow, or an asynchronous session, the SVR peer will respond with a generated packet that includes Forced Drop SVR Metadata with a reason of 8 indicating the session health check is good. For unicast and asymmetric flows, this bi-directional exchange of SVR Metadata ensures the session is valid and any middleboxes between the SVR routers have valid session state. This exchange only occurs during normal packet transmittal, and as such does not replace session keep alive. (see Section 2.12).
State not present:
If an SVR peer receives a packet with Session Health Check SVR Metadata and it has no session state for the session, a generated packet that includes Forced Drop SVR Metadata with reason 2 indicating that full session set SVR Metadata should be sent in the next packet. See Section 6.7 for an example. In certain cases, where a middle box has lost state information, or arbitrarily modified some aspect of the session (e.g., CGNAT), packets with SVR Metadata may not transmit successfully. In cases where there is no response to a Session Health Check, the next forward packet is treated as a new session and is processed as such. See Section 3.7.1.

2.12. NATs and Session Keep Alive

Each SVR router (peer) must statefully remember the source address that a session with SVR Metadata was received on. This may not be the same address the router sent a packet from due to a NAT or firewall in the pathway. Routers use both provisioned and learned Waypoint Addresses. Routers MUST store the actual Waypoint Addresses received on the wire from a peer.

When a firewall or middlebox is detected, the SVR router behind such a device must send SVR Metadata packets periodically on idle sessions to keep any firewall pinhole translations from being removed. For every UDP and TCP session that has seen no packets after a programmable length of time (20 seconds is recommended), then the SVR Peer should send an SVR Control Message on the peer path with the source and dest ports from the idle session's saved state. See Section 7.3.6 for more information and see Section 6.3 for an example.

2.13. Use of BFD on Peer Pathways

BFD [RFC5880] is used to verify Peer Pathways. BFD is used to determine reachability, presence of NATs, changes of Waypoint Addresses, determination of MTU size, current quality on idle circuits, authentication of peers, and maintenance of peer cryptographic keys. Alternative methods can be used for each of these if both peers agree. The use of BFD is included in this specification as a preferred technique for Peer Pathway management.

BFD Metadata is defined and required to measure quality, perform authentication, and maintain security keys because standard BFD authentication fields are insufficient. BFD Metadata is different than SVR Metadata because it is inserted at the very end of a BFD control packet, and not at the end of the layer 4 header. Some BFD Metadata is encrypted. To make processing easy, protobufs are used to transmit the BFD Metadata instead of TLV's. The specifics of BFD Metadata can be found in Section 5.

3. SVR Multi-path Routing Example

The example below shows two SVR capable routers, peering with each other over multiple pathways.


 Client
  LAN
10.x.x.x
   |
   |  +--------+                                  +---------+
   |  |        |                                  |         |
   |  |        |                                  |         |
   +->] East  [203.0.113.1<---MPLS---->203.0.113.89] West   |
      | SVR    |                                  |  SVR    |
      | Router[198.51.80.2<--Inet-+--->198.51.80.8]  Router |
      |        |                   |              |         |
      |       [192.0.2.1<-----LTE--+              |         [<--+
      |        |                                  |         |   |
      +--------+                                  +---------+   |
                <========= Peer Pathways ========>              |
                                                                |
                                                          172.15.11.x
                                                              LAN
                                                             Servers
Figure 2

Note: The client, server, and MPLS network can support the private routes in 10.x.x.x and 172.15.11.x address spaces natively, but the internet and LTE networks do not. This is an example of using secure vectors to join networks together.

3.1. Establishing SVR Peering

The first step in peering SVR routers is to apply any locally defined static L3 routes and begin advertising and receiving routes using L3 networking protocols (BGP, OSPF, etc.) in order to build a forward information base or FIB. This is required initially to ensure that the Waypoints are reachable bidirectionally allowing SVR Peer Paths to be established.

The second step is for both the East and West routers to establish the authenticated peer pathways that make up the SVR Peer relationship. It is recommended that each peer pathway must be authenticated bi-directionally before the SVR pathway is used.

3.1.1. Reachability and Peer Authentication

Authentication of peers is recommended. It is technically possible to send SVR Metadata and determine a key for peers without authentication, but this is discouraged. Either peer could require authentication, and declare the peer relationship invalid should authentication fail.

Authentication is based on a certificate signature request created by the router that contains its UUID and Authority that is signed by a trusted CA (The Router Certificate). A UUID is created and assigned to a router by and administrator. The UUID is used for several reasons. UUID's can be permanently assigned to a router, and if the router name changes, the certificate is still valid. There is also a desire to prevent router names and thus network topology from being easily seen in packet traces. The device registration, creation, signing, and the secure installation of this certificate are omitted from this specification. Please refer to [RFC4210].

Elliptical Curve encryption (see [RFC8422]) techniques are used in SVR. The NIST Curve that is to be used is defined by an administrator. It is recommended that NIST Curve P-256 be used for all SVR Metadata cryptography. All participating routers in an SVR network must use the same elliptic curve.

Each peer sends a BFD packet that contains BFD Metadata in clear text that contains an X.509 Router Certificate in PEM format (see [RFC5758]). See Section 5.1.7 for specifications. Upon receipt, this certificate is verified like any other X.509 certificate. The common name in the certificate provides the authenticated UUID of the peer router. The router must verify that the UUID identified in the certificate is a valid peer in its routing configuration. The certificate should have a locally verifiable chain to a trusted certificate authority (CA).

In our example above, there are three pathways. The BFD message with the X.509 certificate is sent by each side (East and West) on each pathway. Each side verifies the certificate, and then determines if the peer pathway is valid and should be used between peers. To communicate that the peer has received the certificate, and to stop sending it in subsequent BFD packets, a BFD packet without a certificate is sent. This defines the handshake for the local and remote peer. If a certificate is ever required (for example when a router reboots) a peer can request it be transmitted by sending its certificate.

The public key of the router is stored and saved to verify signatures used in subsequent keying procedures (see Section 3.1.2). If the routers certificate is updated, this process must be repeated. Any outstanding valid keys remain operational, preventing outages during recertification.

3.1.2. Peer Cryptographic Key/Re-keying

In the above example Figure 2 there is one peer relationship (East and West) that share three Peer Pathways. Assuming that all three pathways have been authenticated, the East West peer relationship has three usable transport pathways.

To securely exchange BFD traffic between East and West peers, a Peer Key is required. Because the Private Key/Public Key pair used by router is to be very long lived, a Key Derivative Function is used (ECDH-ES). The SVR protocol uses Concat KDF. See [NIST_SP_800-56A].

The Concat KDF process uses a Salt value, one from each peer. The Salt values, private key, and the two public keys are inputs to create a key that is derived from the certificate (essentially authenticated by the certificate). This prevents any man in the middle attacks. Encrypted fields in BFD Metadata are used to distribute keys for SVR Metadata encryption/decryption. Please see Section 5.1.9) for detailed description of how this works.

To rekey, at any time, either party can initiate the same sequence, only with new Salt values.

The same Peer Key is used for all pathways between peers. This is efficient when there are many parallel pathways multipath routing use cases.

3.1.3. Metadata Cryptographic Key/Re-keying

SVR Routers can receive packets with SVR Metadata on any interface. Also, many remote Peers may share a single interface on an SVR Router. Because source addresses changes (NAT Updates) and packets will sometimes arrive on different interfaces due to network flaps, and routing changes, each SVR Router must know in advance what Crypto key to use for each SVR Peer.

Each SVR router creates a key (or obtains a quantum safe key) that it shares with its peers. Any SVR metadata sent from any peer must use this SVR Metadata Key. BFD Encrypted Metadata is used to broadcast the Key to all peers and its version. See Section 5.1.9.

The SVR router will share the same Key with all Peers, therefore all SVR Metadata processed locally will have one key until its rotated.

3.1.4. Bring Peer Into Service

When a peer has at least one working authenticated pathway, and has calculated an Elliptical Curve Peer Key (ECPK), the SVR Peer is assumed ready for transport traffic bidirectionally, and the peer is declared operational and in service.

3.1.5. Resulting Peer Relationship

When in service, East and West independently communicate using BFD to each other's interfaces to ensure operational status and measure path characteristics such as jitter, latency, and packet loss. In our example, assuming 100 percent success, the resulting peer pathways would be:


PEER: East -> West Authenticated/In Service
  Name      Description                    Characteristics
  MPLS      203.0.113.1->203.0.113.89      20ms Lat, 0 Loss,  2 Jit
  Internet  198.51.80.2->198.51.80.8       30ms Lat, 0 Loss,  3 Jit
  LTE       192.0.2.1->198.51.80.8         50ms Lat, 0 Loss, 15 Jit

PEER: West -> East Authenticated/In Service
  Name      Description                    Characteristics
  MPLS      203.0.113.89->203.0.113.1      20ms Lat, 0 Loss,  2 Jit
  Internet  198.51.80.8->198.51.80.2       30ms Lat, 0 Loss,  3 Jit
  LTE       198.51.80.8->192.0.2.1         50ms Lat, 0 Loss, 15 Jit
Figure 3

BFD is also used on in service Peer Pathways to determine MTU size and detect address changes, and monitor quality when idle.

3.2. CIDR based SVR Peer FIB Entries

To route packets and sessions of packets onto SVR Peer Pathways, a route lookup must return an indication of either a SVR peer pathway, or a SVR peer.

In the example shown below our assumption is that there are servers that are located inside 172.15.11.0/24 at the West location. West publishes or otherwise advertises this route to East on each path available to it. Subsequently East's FIB will look like this:


  East's Forward Information Base (FIB)
     Route             Next-Hop IP Addr
     ----------------  -----------------
     172.15.11.0/24    203.0.113.89
     172.15.11.0/24    198.51.80.8
     ....
     [FIB Entries to reach Waypoints omitted]
Figure 4

Additionally, we will assume there exists a network policy created by the administrator of an Authority "example" that defines a Tenant "engineering" as 10.0.0.0/25 VLAN2, and "github.example" as 172.15.11.23 using TCP port 22. The provisioning and/or discovery of this policy is outside the scope of this protocol description.

A first packet from an engineering client with github as a destination received at the East SVR Router will result in a search of the FIB and result in two possible next-hop IP Addresses. East will consult its SVR Peer Pathway list and recognize that three of its peer pathways have an exact match of this next-hop IP Address. These represent the three possible pathways that may be used for routing this session. The resulting potential routes are:

  Possible Routes
    MPLS      20ms Latency, 0 Loss,  2 Jitter
    Internet  30ms Latency, 0 Loss,  3 Jitter
    LTE       50ms Latency, 0 Loss, 15 Jitter
Figure 5

The East router can now choose which Peer Pathway is desired for the specific session. If the East router has quality service levels to maintain, it can choose from any of the Peer Pathways based on their current quality metrics. If all things are equal, the East router could load balance using approaches like "least busy" or other techniques. Once a Peer Pathway is chosen, the first packet SVR Metadata is constructed, inserted into the first packet, and sent down the chosen pathway to the West peer router.

For this example, the private address space in the LAN supported by the East Router is different. This is often the case with large networks. This is illustrative of a branch router performing network address translation (NAT) on a source address to solve overlapping address problems.

In this specific case, assuming MPLS was chosen, East would perform first packet processing resulting in the insertion of SVR Metadata in the first packet (see Section 3.7.1) and send it out East's interface with a source address of 203.0.113.1 and a destination address of 203.0.113.89. These are the exact addresses of the MPLS Peer Pathway.

Both the East and West routers would use the same address pairs (only reversed) for the bidirectional session, using the allocated source and destination ports to recognize the specific session. All packets from all sessions on a peer path will have the same exact IP addresses, differentiated solely by their port numbers.

3.3. Optional FIB Containing Service Names

SVR Metadata in the first packet contains text strings that contain service names. SVR routing can route traffic by these names if the FIB contained text entries. There are some use cases where this is preferred over CIDR lookups:

Avoiding Dynamic DNS:
Dynamic DNS is used to augment network routing protocols by answering the question: What best IP Address is available and best for a session now? Dynamic DNS can be plagued by delays in real time updates and additional complexity and cost. In private networks, path service state may not be reflected in Dynamic DNS responses.
Multi-Cloud Networking:
Public clouds run service instances on dynamically allocated private IP addresses. They provide very accurate and responsive DNS updates to help find IP addresses for networking. These DNS services are not available outside of the cloud, making internetworking difficult. SVR Routers can use DNS resolution to find IP Addresses for named services.

Below is an example FIB that contains named services and traditional FIB entries. The FIB is now an SVR FIB containing service names, protocols, and ports, with next-hop addresses changed to Waypoint Addresses.


  East's Extended SVR Forward Information Base (OPTIONAL)

                                                       Egress
  Service Name        Route                Waypoint    Action
  --------------     ------------------  ------------  --------
  github.example     172.15.11.23:TCP:22  203.0.113.89 FWD
  github.example     172.15.11.23:TCP:22  198.51.80.8  FWD
  logsvc.example     172.15.11.20:UDP:514 203.0.113.89 DNS
  logsvc.example     172.15.11.20:UDP:514 198.51.80.8  DNS
  https.example      172.15.11.24:TCP:443 203.0.113.89 DEST NAT
                                                       -196.168.1.1
                                                       -196.168.1.2
                                                       -196.168.1.3
     [FIB Entries to reach Waypoints omitted]
Figure 6

Longest prefix matching (LPM), protocol and port will be used to match Routes for packets intended for github on ingress to SVR. The text string "github.example" will be used by all other SVR routers until egress from SVR. The SVR fib can be used to LPM match on IP addresses and exactly match protocol and ports. In the above illustrative example, only three protocols are supported (SSH, Syslog, and HTTPs). All other packets will be denied by default.

The egress action in the SVR FIB can be used to support three different egress actions:

Forward Packet (Default):
Restore the IP Addresses and forward. If a source NAT is provided in the SVR Metadata, NAT the source address.
DNS:
Use DNS to resolve the service name locally. In this example DNS resolution procedures would be used on egress to resolve "logsvc.example".
DEST NAT:
NAT the destination address to one (or load balance to a pool of addresses). This is identical to load balancers.

These named routes can co-exist with traditional FIB entries shown above. SVR will always match a named route first, and fall through to the generic routes second.

3.4. SVR Security Definitions

For basic SVR functionality to work between peers, there must be a Authority wide provisioned set of rules. These rules include:

HMAC Method:
This describes the method/technique for signing SVR packets. This could be SHA1, SHA256, or SHA256-128.
Use Time Based HMAC:
This is either YES or NO.
HMAC SVR Metadata or ALL:
This is NONE, SVR Metadata Only, ALL
SVR Metadata Block Cipher:
This is either NONE, AES128, AES256.
Elliptical Curve:
This is the curve to use (defaults to Curve P-256).

SVR does not limit the use of ciphers and techniques to those listed above. The requirements for both signatures and encryption are to use a cypher where the results are fixed, well-known, block sizes.

Security Policies are used during session setup to setup payload encryption specifically for individual sessions. These are exchanged in first packet SVR Metadata.

For this example, the following SVR security definitions are used.


      HMAC: (On, time-based, SHA256-128, ALL Packets)
      SVR Metadata Encryption (On, AES256)
      Elliptical Curve: Curve 256P (NIST)
Figure 7

3.5. Time Based HMAC Details

To positively authenticate and provide integrity for SVR session, SVR peers use Time Based HMAC signatures. HMAC signatures are defined in [RFC2104]. Please see Section 4.5.1.

In our example, we are using SHA256-128 with a size of 16 Bytes.

3.6. Security Keying/Rekeying Considerations

Every SVR Metadata transaction includes a security ID header TLV (see Section 7.3.2).

Each SVR Peer will have its initial Peer Key (version 1) established during the peering establishment. The key may be updated at any time, and the key version will be incremented. The security key version is always sent in SVR Metadata to ensure the peer knows which key to use to decrypt the SVR Metadata just sent. If a peer only has version 1 of a key, and SVR Metadata arrives specifying it is now at version 2, the SVR router must obtain the new key before it can process any packets (see Section 3.1.2).

For networks that are large and actively performing key management, there may be multiple versions of a key active for very brief moments in time, and SVR routers MUST be able to utilize any key for a reasonable amount of time.

3.7. New Session Initiation Detailed

The diagram below shows the example github TCP session flowing between a client and server through the East and West routers in our example network.

Ladder Diagram for SSH Example:


    Engineering                                      Github
    Client . . . . . . . . . . . . . . . . . . . . . Server
      |                                                 |
      +         East Router              West Router    |
      |            |                        |           |
      +---SYN----->|                        |           |
      |            |--SYN[w/MD]------------>|           |
      |            |                        |--SYN----->|
      |            |                        |           |
      |            |                        |<--SYN/ACK-|
      |            |<------SYN/ACK[w/RMD]---|           |
      |<--SYN/ACK--|                        |           |
      |            |                        |           |
      |            |                        |           |
      |<== Session Packets Flow with No SVR Metadata ==>|
Figure 8

The East Router MUST construct and insert SVR Metadata (MD) in the first packet of the SSH session, which will be a TCP SYN packet. The West Router must remove the SVR Metadata, and forward the SYN packet, and wait for the server to respond with a SYN/ACK. Upon receipt of the SYN/ACK, the West Router will create reverse SVR Metadata (RMD), and insert it into the SYN/ACK. This will create the SVR Metadata handshake for the SSH session. All forward and reverse SVR Metadata are inserted into existing packets if possible.

When a client or router detects that a new session is being established, the East Router will insert SVR Metadata into the first packet to communicate intent to the West Router. At both East and West Routers, the first packet will require specialized handling. Detecting a first packet for a session is protocol specific. For TCP, it's a new 5-Tuple packet (new flow) with the just the SYN flag set. For UDP, it's simply a new 5-Tuple packet not currently in active use.

3.7.1. East First Packet Processing

Utilizing the same example, assume that the packet shown below arrives on the East Router from the Client LAN. The packet is the result of an engineer attempting to access a github service via SSH.

Arriving Packet at East Router

      Packet received on LAN side East Router
      Engineer using SSH to access Github
      +---------+---------------------+--------------+----------+
      |L2 HDR   | IP Header           | TCP Header   | PAYLOAD  |
      |  VLAN=2 |    SRC=10.0.1.1     |   Sport=6969 |   Data   |
      |         |    DST=172.15.11.23 |   Dport=22   |  (N/A)   |
      +---------+---------------------+--------------+----------+
Figure 9
3.7.1.1. Determine Tenant

The tenant is a text name which describes the routes and policies that are available for a group of source IP addresses. Tenants are like security zones. In our example, the "engineer" is based upon VLAN 2, and the tenant will be "engineer" as named by the Authority "example". The configuration and data models to map physical network attributes to named tenants is implementation specific. Associating a default tenant with every logical interface on a SVR Router is recommended.

3.7.1.2. Determine Service

There are multiple ways to determine the associated service for a new session. Application Identification technology is used that understands all popular SaaS offerings. These techniques use combinations of IP address ranges and ports, SNI fields in TLS, Common Name from Certificates, and extraction of URLs from HTTP requests. Most popular SaaS vendors today publish and update frequently their CIDR blocks and ports used by their services. This is out of scope for this document.

Longest prefix matching algorithms are used to extract the major and key services at a site. If there is traffic that cannot be identified accurately, often it will be placed into a "catch-all" service called "internet".

We will assume for this document, that the address 172.15.11.23 is a well-known address for git servers in the network defined by Authority "example", and port 22 is known to be SSH.

3.7.1.3. Determine Network Requirements

Once the tenant and service have been determined, a lookup for network requirements can be determined.

Example Network Requirements


          SERVICE: github
            Access Policies:
              Tenants Allowed: engineering
              Tenants Denied: release.engineering
            Quality Policy: latency < 40ms
            Security Policy:None
Figure 10

The above definition for github defines an example network requirement. Access policies determine which tenants are allowed, and if any specifically denied. The Quality policy defines the service level experience requirements. Secure Vector Routing exchanges tenants, services, and security policies using character strings in SVR Metadata. Access and quality policies are defined and used locally within a router and logically tied to the service. The implementation of quality and access policy controls are site specific. For example, VLAN based subnets may have different meanings at various locations. Also, QoS management schemes may be different for different network areas.

3.7.1.4. Picking a Peer Path

As stated previously, the East Router has three peer paths that can reach the destination based on L3 reachability. The next step is to apply the network requirements to see which of the peer paths remain. Our policy requires latency to be less than 40 msecs, which eliminates East's LTE pathway from consideration. The remaining two pathways, MPLS and Internet, are both possible. We will choose MPLS as it has the lowest latency, offering the user the best experience.

Many different criteria can be used in selecting a peer pathway. In practice, how busy a peer path is and its capacity result in new sessions routing to 2nd best options. Often simple load balancing is used. In cases where there are higher costs (such as LTE or 5G networking), these may be held in reserve for backup or disaster recovery. The actual algorithms for picking peer pathways are outside the scope of this protocol.

3.7.1.5. Allocate Source NAT if Necessary

In this github example, there is a source NAT at the East Router on the MPLS interface to the datacenter. This by design, allows all of the remote branch sites to use overlapping addresses, and is very common in larger networks. Routers that perform source NAT have two options: use the interface address and allocate a new source port, or use an IP address pool and allocate full IP addresses for each session. Either way, this allocated address only needs to be placed into SVR Metadata, as the existing packet address will be translated to Waypoint Addresses shortly. The egress SVR router will apply the source NAT.

3.7.1.6. Allocation of Ports

The next step is to allocate new ports for the SVR session. The ports being allocated must not be in use, and should not have been used recently to avoid any issues with middleboxes. See Section 4.2.

The range of ports that can be used may be site specific and tied to policies that exist in upstream firewalls or middleboxes. For these reasons, the actual pool of available addresses is provisioned on every SVR router. The East router has ports 8000 to 24000 available for both the source and destination ports. In this example we will allocate an even source port of 8000, and an odd destination port of 8001. Note that both source and destination ports are allocated by the router that initiates SVR Metadata.

3.7.1.7. Session State and SVR Metadata Construction

The router now has reached a point where it can forward the packet. It has valid network requirements, available peer paths, and has available SVR ports. The next step is to create and save all session state information for subsequent packet processing. A session UUID is created for end-to-end tracking of sessions. The table below refers to SVR Metadata TLVs and specific contents that are documented in Section 7.

Session State Table Entry


State Information & Mappings to SVR Metadata Fields

              SVR Metadata TLV                       |------TLV------|
Category       -Field              VALUE             Type   Len  Hdr
--------      ------------------   ----------------
Header                                                       12
Header TLVs
              Security ID          1                   16     4    4
              Path Metrics                             26    10    4
               -Tx Color           5
               -Tx TimeValue       4200 MSecs
               -Rx Color           3
               -Rx TimeVlue        3950 MSecs
               -Drop               No
               -Prev Color Count   950 Packets
                                                            ---  ---
                            Total Header Length = 34 (26+8)  26    8

Payload TLVs
               Forward Context                         2     13    4
               - Source IP Addr     10.0.0.1
               - Dest IP Addr       172.15.11.23
               - Protocol           TCP
               - Source Port        6969
               - Dest Port          22
               Tenant Name          engineering         7    11    4
               Service Name         github             10     6    4
               UUID                 e9b083df-d922.....  6    16    4
               Source Router Name   East Router        14    11    4
               Source NAT Address   203.0.113.1        25     4    4
               Security Policy      NONE               15     4    4
               Peer Path                               19    22    4
               - Source Addr        203.0.113.1
               - Dest Addr          203.0.113.89
                                                            ---  ---
                         Total Payload Length = 119 (87+32)  87   32

                                 To West     Fr West
              Allocated Ports     Router      Router
               -Source Port        8000        8001
               -Dest Port          8001        8000

              Session HMAC Key    [Peer Key at session start]
Figure 11

The required SVR Metadata attributes that must be inserted in a first packet of a new sessions are defined in Section 4.3.1, include Security ID, Forward Context, Tenant Name, Service Name, Session UUID, Source Router Name, Security Policy, Peer Pathway ID.

Two optional SVR Metadata attributes Path Metrics and Source NAT Address are both included in this example. This is documented in Section 7.3.7 and Section 7.4.13.

The order of the TLVs is arbitrary, but header TLVs must be before any payload TLVs. If a TLV is received that is unknown to a peer, it MUST ignore it. In this example, the header length including the two header TLVs is 34, and the 8 payload TLV's are 119 bytes long.

The Session HMAC Key is state information retained by the router. The Session HMAC Key is set to the current Peer Key at session initiation. This key is used for the life of a session.

3.7.1.8. Encryption of SVR Metadata

The next step is to encrypt the SVR Metadata block as defined in Section 4.4. In our example, our provisioned security definitions include AES256 for SVR Metadata encryption. AES has a 128-bit block size for all key lengths. In our example, the SVR Metadata payload TLVs are 119 bytes large. Padding will be added during encryption to make it 128 bytes (or 9 bytes of padding). In addition, to make the encrypted data stateless, we must also include a 16 byte initialization vector directly after the encrypted block. The resultant encrypted SVR Metadata block is 178 bytes and looks like this:

SVR Metadata Block

      +--------------+--------------+---------+----------------+
      +   SVR        |   SVR        |         |                |
      | Metadata     | Metadata     | Padding | Initialization |
      | Header       | Payload TLVs |         |    Vector      |
      | (Unecrypted) |              |         |                |
      | 34 Bytes     | 119 Bytes    | 9 Bytes |  16 Bytes      |
      +--------------+--------------+---------+----------------+
      |<---Clear---->|<---Encrypted Portion-->|

      |<--------------178 Byte SVR Metadata Block------------->|
Figure 12
3.7.1.9. Insert SVR Metadata

The SVR Metadata block is inserted into the packet directly after the L4 Header. The total length of this specific SVR Metadata block is 178 bytes, 34 of which are header bytes and 119 for payload TLVs. If there is data in the payload portion of the IP Packet, the payload data is moved down to make room for the SVR Metadata. The packet structure will now look like:

SVR Metadata Added

      Packet with SVR Metadata inserted
      +---------------------+--------------+-----------+------------+
      | IP Header           | TCP Header   |  SVR      |  PAYLOAD   |
      |    SRC=10.0.1.1     |   Sport=6969 | Metadata  |    Data    |
      |    DST=172.15.11.23 |   Dport=22   | 178 Bytes | (optional) |
      +---------------------+--------------+-----------+------------+
Figure 13

The transport addresses in the packet are updated to use the selected peer path.

Transport Addresses Updated

      Final Transformed Packet with SVR Metadata inserted
      +---------------------+--------------+-----------+------------+
      | IP Header           | TCP Header   |  SVR      |  PAYLOAD   |
      |    SRC=203.0.113.1  |   Sport=8000 | Metadata  |    Data    |
      |    DST=203.0.113.89 |   Dport=8001 | 178 Bytes | (optional) |
      +---------------------+--------------+-----------+------------+
Figure 14
3.7.1.10. Signing SVR Packet

The packet containing SVR Metadata is now signed with a HMAC signature (See Section 3.5). The HMAC signature is placed at the very end of the packet, extending the packet size by the signature's length. The IP header is excluded from the signature. The current SVR Metadata Key is used (see Section 5.1.9) for signing and verifying the authenticity of the packet. In this case the HMAC is 16 bytes.

HMAC Signature Added

    Packet with SVR Metadata inserted
    +-------------------+--------------+-----------+---------+-------+
    | IP Header         | TCP Header   | Encrypted | PAYLOAD | HMAC  |
    |  SRC=203.0.113.1  |   Sport=8000 |   SVR     |   Data  |  16   |
    |  DST=203.0.113.89 |   Dport=8001 | Metadata  |         | Bytes |
    +-------------------+--------------+-----------+---------+-------+
                        |                                    |
                        |<=========HMAC Signed Data=========>|
Figure 15
3.7.1.11. Sending the First Packet

The packet length and checksum is corrected, and the packet is transmitted. The sending side will include the same exact SVR Metadata on every packet until a packet in the opposite direction (reverse direction) arrives with reverse SVR Metadata indicating a complete handshake. For TCP, the SYN packet contains SVR Metadata, and typically a SYN-ACK from the server side responds with SVR Metadata, and there is no further SVR Metadata inserted in a session.

        Client ---->    TCP SYN w/SVR Metadata  ----> Server
        Client <---- TCP SYN-ACK w/SVR Metadata <---- Server
Figure 16

For UDP, SVR Metadata can be inserted in packets until there is a reverse flow packet with SVR Metadata, except for unidirectional flows as noted in Section 3.5.7.

3.7.2. West First Packet Processing

If a packet arrives at the West Router having the West Routers Waypoint (interface address) as a destination address (i.e., the packet was sent to the router, and not to a destination beyond the router) the packet may likely contain SVR Metadata. When this occurs, the following steps are taken.

3.7.2.1. Verify Source Address is a Waypoint

Packets arriving on the routers must be verified to be valid before they are processed (see Section 4.6.2). These simple checks can eliminate any potential attack vectors. If the packet fails authentication or validation the packet MAY be dropped or responded to with an ICMP Destination Unreachable packet.

In the example case we are using, there are only three source addresses that could be possible:

Possible Source Addresses

            203.0.113.1      MPLS Peer Pathway
            198.51.80.2      Internet Peer Pathway
            169.254.231.106  LTE Peer Pathway
Figure 17
3.7.2.2. Verify SVR Metadata Block

The very first and most efficient test is to verify that the SVR Metadata is present is to look for header magic number (see Section 4.6.1).

The next verification step is to check the HMAC signature (see Section 4.5.1). If the signature is invalid, the packet should be dropped and a security event noted. If valid, processing continues.

The unencrypted portions of the SVR Metadata header should be verified for reasonableness. The Header Length and Payload Length must be less than the SVR Metadata block size.

3.7.2.3. Parse SVR Metadata and Save State and Translations

The next step is to decrypt the SVR Metadata (See Section 4.6.2.2). If there are any reasons why the SVR Metadata block cannot be decrypted, or the decryption fails, the packet is dropped.

The payload TLVs can now be parsed and the necessary state and translations loaded into memory. If there is a failure to parse all TLV's, the packet is dropped.

Next the SVR Metadata block and HMAC signatures are removed from the packet.

3.7.2.4. Restore Addresses and Route Packet

The SVR Metadata information is used to restore the original context to the packet. The packet is then recursively processed exactly like the first packet described in Section 3.7.1 with a few differences. The Context, Tenant, Service, Security Policy and Session UUID strings are used from the SVR Metadata (as opposed to locally determining them) eliminating these steps. These are then used for applying policy and routing decisions locally. The end result is the packet may go through another SVR Peer Pathway or be delivered via standard networking techniques. In this example, the West Router delivers the packet to the Server LAN.

When the packet is forwarded to another SVR Peer, there are some differences. The Tenant, Service, Session UUID, Security Policy and the original 5-tuple addresses are all cloned. This provides consistent data across a multi-hop SVR network. It should be noted that the SVR Metadata must be decrypted at every SVR Router and then re-encrypted because the Waypoint addresses are different for each selected peer pathway. Payload decryption however, is only encrypted at the first SVR router and decrypted at the last SVR router.

3.7.2.5. Detection of a Looping Session

Because every hop between SVR Routers utilizes the same session UUID, a looping first packet is easy to detect. There MUST never be two sessions with the same UUID. Any session that loops must be dropped. By detecting looping packets during the first packet transmitted, subsequent packets can be dropped on ingress by the SVR Router that detected the looping behavior. SVR routers must also decrement the TTL and operate in all ways like a traditional router to prevent looping packets that are not detected by SVR.

When a packet arrives with SVR Metadata after the SVR Metadata handshake has been completed, it is assumed to be an update and not classified as looping. Updates can be used to change any attribute, but most commonly to change a peer pathway for a session. See Section 6.1.

3.7.3. Return Packet Path Pre-Established

After processing the first forward packet at both East and West routers, both the East and West routers have established packet forwarding rules and translations for both directions. This means that eastbound rules and westbound rules are all established and installed. The router is thus capable now of recognizing 5-tuples in either direction and acting on the packets without consulting routing tables. This is known as fast path processing.

3.7.4. Sending Reverse SVR Metadata

On a session-by-session basis, SVR Routers must know the status of a SVR Metadata handshake. If a packet for a session arrives and the SVR Metadata handshake is not complete, the SVR Router must insert SVR Metadata for the session. This will continue until there is verification that the SVR Peer has received the information. As stated previously, for TCP SYN this is normally the first reverse packet which is a TCP SYN/ACK. The purpose of reverse SVR Metadata is:

  • To indicate to the sender that it can stop sending SVR Metadata (Completion of the SVR Metadata handshake).
  • Provide backward information about the service for routing of future instances.

In this example, the reverse SVR Metadata includes:

Reverse SVR Metadata Response


          Reverse SVR Metadata Response
State Information & Mappings to SVR Metadata Fields

              SVR Metadata TLV                      |------TLV------|
Category       -Field              VALUE             Type   Len  Hdr
--------      ------------------   ----------------
Header                                                       12
Header TLVs
              Security ID          1                   16     4    4
              Path Metrics                             26    10    4
               -Tx Color           3
               -Tx TimeValue       4100 MSecs
               -Rx Color           5
               -Rx TimeVlue        4050 MSecs
               -Drop               No
               -Prev Color Count   1950 Packets
                                                            ---  ---
                           Total Header Length = 34 (26+8)   26    8

Payload TLVs
               Reverse Context                         4     13    4
               - Source IP Addr     203.0.113.1
               - Dest IP Addr       172.15.11.23
               - Protocol           TCP
               - Source Port        7891
               - Dest Port          6969
               Peer Path                               19    22    4
               - Source Addr        203.0.113.89
               - Dest Addr          203.0.113.1
                                                            ---  ---
                          Total Payload Length = 43 (35+8)   35    8

                                   To East     From East
               Allocated Ports     Router       Router
               - Source Port        8001         8000
               - Dest Port          8000         8001

              Session HMAC Key   [Peer key used by remote peer]
Figure 18

See Section 4.3 for required and optional TLVs in reverse SVR Metadata.

One optional SVR Metadata attribute is included in this example for the pathway metrics. This is documented in Section 7.3.7.

The Session HMAC Key is state information retained by the router. The Session HMAC Key is set to the Peer Key version specified in the SVR Metadata (Security ID). This is most like the current Peer Key but it is possible re-keying could create a race condition. To solve this problem, the Session State for routers terminating SVR sessions uses the Peer Key indicated by the initiator. This key is used for the life of a session.

One of the benefits of SVR is the complete tracking of sessions end-to-end. In this example, the SVR Metadata state located in the SVR router contains all addresses used. The forward context provides the egress SVR router with the addresses being used pre-NAT, and the source NAT information. The reverse context would likewise supply the ingress SVR destination NAT addresses. Knowing the Waypoint Addresses used along with the ports used provides complete end-to-end visibility of each session.

This SVR Metadata will be encrypted, inserted, and an HMAC checksum will be computed and attached as per the previous example. The reverse packet in this example will have 34 bytes of header data, and 43 bytes of payload data, 5 bytes of padding, and a 16 byte initialization vector resulting in a SVR Metadata block that is 98 bytes long.

3.7.5. Subsequent Packet Processing

As soon as an SVR peer receives a packet of a session from another SVR peer and there is no SVR Metadata, the SVR Handshake is complete, and it can stop sending SVR Metadata. This work for both the East Router and the West Router. Both will transmit SVR Metadata until they receive a packet without SVR Metadata.

3.7.6. Session Termination

No SVR Metadata is sent upon normal session termination. The router can monitor the TCP state machine and have a guard timer after seeing a FIN/ACK or RST exchange. After the guard timer, the session can be removed from the system. If a new session arrives during this period (a TCP SYN), then it will cause immediate termination of the existing session. In addition, all protocols also have an associated inactivity timeout, after which the session gets terminated if no packets flow in either direction. Should an existing session send a packet after the inactivity timeout, it will be processed as a new session.

3.7.7. Unidirectional/Asymmetric Flows

When there are unidirectional flows, or path asymmetry (e.g. TCP sequence numbers advance with no reverse packets observed), and there is end-to-end communication, one can stop sending SVR Metadata. For UDP asymmetry, the sending router will send a maximum of 20 packets with SVR Metadata; if no reverse packets are seen during that time, the receiving peer router generates and sends a disable SVR Metadata packet to the originating router to complete the SVR Metadata handshake. (Note: The total number of packets sent with SVR Metadata without receiving a reverse packet is an implementation detail and is not a strict requirement.

3.7.8. Multi-Hop Session Ladder Diagram

The diagram below shows a typical normal TCP session flowing between a client and server through routers in a network.

Ladder Diagram for Session Initiation with SVR Metadata:


        Client . . . . . . . . . . . . . . . . . . . . . . Server
          |                                                   |
          +         RouterA       RouterB      RouterC        |
          |            |             |            |           |
          +----SYN---->|             |            |           |
          |            |--SYN[MD1]-->|            |           |
          |            |             |--SYN[MD2]->|           |
          |            |             |            |----SYN--->|
          |            |             |            |           |
          |            |             |            |<--SYN/ACK-|
          |            |             |<--SYN/ACK--|           |
          |            |<--SYN/ACK---|   [RMD2]   |           |
          |<--SYN/ACK--|    [RMD1]   |            |           |
          |            |             |            |           |
          |            |             |            |           |
          |<=== Session Packets Flow with No SVR Metadata ===>|
Figure 19

Note that each router constructs SVR Metadata for the next chosen peer in the routed pathway as depicted by MD1 and MD2 in the above diagram. Upon receipt of first reverse packet, reverse SVR Metadata RMD2 and RMD1 is inserted. Each router allocates its own transport addresses (Waypoints) for each session. The context, service name, tenant name, and session UUID sent are unchanged between all routers, and can be used for determining routing policies to apply. The session UUID is the same in MD1, MD2, RMD1, and RMD2 in the above diagram.

Likewise, the diagram below shows a session teardown sequence for a typical TCP session.

Ladder Diagram for Session Teardown SVR Metadata:


        Client . . . . . . . . . . . . . . . . . . . . . . Server
          |                                                   |
          +         RouterA       RouterB      RouterC        |
          |            |             |            |           |
          +---FIN----->|             |            |           |
          |            |-----FIN---->|            |           |
          |            |             |----FIN---->|           |
          |            |             |            |-----FIN-->|
          |            |             |            |           |
          |            |             |            |<--FIN/ACK-|
          |            |             |<--FIN/ACK--|           |
          |            |<--FIN/ACK---|            |           |
          |<--FIN/ACK--|             |            |           |
          |            |             |            |           |
          |            |             |            |           |
Figure 20

No SVR Metadata is sent or required when sessions terminate. Each router keeps its state information for a programmed length of time in case a FIN/ACK is delayed or dropped, then the state information is removed.

4. SVR Protocol Definition

This section provides the normative requirements for SVR Metadata to achieve interoperability.

4.1. SVR Session Definitions and Types

SVR implementations MUST support TCP, UDP, and ICMP. SVR implementations SHOULD support UDP Unicast. Sessions are characterized by having an initial first packet that is a unique to an SVR router. Often this is described as a unique 5-tuples as seen by the router. Sessions start when the first packet is processed, and end when either the L4 protocol indicates the session is completed (TCP FIN/FIN ACK, RST) or there has been no activity for a length of time (UDP, ICMP, UDP Unicast, point-to-point ethernet).

SVR is always OPTIONAL. SVR implementations can choose when to use SVR on a session-by-session basis. SVR implementations MUST support non-SVR traffic.

4.2. SVR Metadata Insertion

4.2.1. SVR Metadata Packet Location

SVR implementations MUST insert SVR Metadata into packets directly after the L4 header, even if the resulting increase in packet size would cause the packet to require fragmentation. For Ethernet point-to-point and ICMP error messages, IP Headers and L4 headers MUST be created, and if associated with an existing session, MUST share the exact transport 5-tuples (SVR Waypoints and Ports) as the session the ICMP error message relates to. The SVR Metadata MUST be in the very first packet of a new session (TCP or UDP bidirectional flow) to have any role in path selection or security. SVR Metadata SHALL be sent in any subsequent packet in any direction to change or update the networking requirements. The SVR Metadata is inserted into the payload portion of a packet to guarantee it makes it unchanged through the network. Packet lengths and checksums MUST be adjusted accordingly. TCP sequence numbers MUST NOT be adjusted.

4.2.2. SVR Metadata Prerequisites

A prerequisite for SVR Metadata insertion is that a Peer Pathway MUST be selected relating to a specific session. This is similar to choosing a tunnel between two networks. This Peer Pathway has IP addresses on either side (Waypoint Addresses), and these addresses will always be the transport IP addresses for packets containing SVR Metadata.

4.2.3. SVR Metadata Port Allocation for Sessions

The SVR peer originating the session (client side) MUST allocate both source and destination ports. The ingress side MUST choose even ports for local (source port) and odd ports for remote (destination port). This provides total uniqueness between any two peers, with no negotiation or collision possibilities. The range of ports to use for allocation is provisioned. Ports in use MUST be excluded from allocation. Ports MUST be unallocated when session state is removed. Ports MUST have a 60 second guard time before being reallocated

4.2.4. SVR Metadata on Idle Session

SVR implementations MAY need to send SVR Metadata to a peer at a time when there are no existing packets. In these cases an IP packet MUST be created and inserted into the appropriate existing session with an indication the packet should be dropped. See Section 6.3 for an example. The packet MUST be processed, interpreted, and dropped by the directly adjacent peer and not forwarded to any other SVR peer.

4.2.5. SVR Metadata Packet Structure 

   Existing IP Packet with SVR Metadata inserted
   +------------------+-----------------+----------+----------+----+
   | Existing IP Hdr  | Existing L4 Hdr |  SVR     | PAYLOAD  |HMAC|
   |   Source IP Addr |   Source Port   | Metadata |   Data   |    |
   |   Dest IP Addr   |   Dest Port     | Block    |(optional)|    |
   +------------------+-----------------+----------+----------+----+

   Generated IP Packet with SVR Metadata inserted
   +-------------------+------------------+----------+----+
   | Created  IP Hdr   | Created L4 Hdr   |  SVR     |HMAC|
   |   Source IP Addr  |   Source Port    | Metadata |    |
   |   Dest IP Addr    |   Dest Port      | Block    |    |
   +-------------------+------------------+----------+----+

   ICMP Packet with SVR Metadata inserted
   +-----------------+-----------------+----------+--------+----+
   | Created IP Hdr  | Created UDP Hdr |  SVR     |  ICMP  |HMAC|
   |  Source IP Addr |   Source Port   | Metadata |  MSG   |    |
   |  Dest IP Addr   |   Dest Port     | Block    |        |    |
   +-----------------+-----------------+----------+--------+----+

   Ethernet Packet with SVR Metadata inserted
   +-----------------+-----------------+----------+----------+----+
   | Created IP Hdr  | Created UDP Hdr |  SVR     | Ethernet |HMAC|
   |  Source IP Addr |   Source Port   | Metadata | MSG      |    |
   |  Dest IP Addr   |   Dest Port     | Block    |          |    |
   +-----------------+-----------------+----------+----------+----+
Figure 21

If UDP protocol, the UDP Header MUST be updated to have the correct packet length.

The Layer 4 header (TCP/UDP) MUST have its checksum recalculated per the appropriate procedures.

The IP Packet length field MUST be updated to reflect the number of bytes added for the SVR Metadata block AND the HMAC signature.

The IP Header Checksum MUST be updated after the IP Packet length is adjusted.

If TCP protocol, the TCP Sequence numbers MUST NOT be changed.

4.2.6. Prevention of False Positives

SVR Metadata is sent inside the payload portion of TCP and UDP packets. Given that no byte sequence is truly unique in the payload of a packet, in the scenario where the original payload after the L4 header contained the same byte sequence as the SVR magic number, false positive logic is enacted on the packet. This guarantees downstream SVR routers will not confuse SVR Metadata magic number signatures.

False positives SHALL NOT occur when first packets are processed, since valid SVR Metadata will always be inserted regardless of the contents of the first 8 bytes of the payload. False positive can only occur during existing valid SVR sessions between peers.

To implement false positive logic, SVR implementations MUST insert an empty SVR Metadata header (12 byte header with 0 TLVs). This creates a contract with downstream SVR routers that if the magic number is present, there MUST be valid SVR Metadata that requires processing and removal.

The structure of a false positive SVR Metadata includes just a header of length 12 bytes, with zero header TLVs and zero payload TLVs. The SVR router receiving a packet with false positive SVR Metadata will strip out the SVR Metadata header and any TLV's as is normally expected. The inserted SVR Metadata header has no TLV's and is not encrypted.

SVR Metadata Location


   Received Midstream SVR Packet matching SVR Magic Number
   +-------+--------+-------------------------+
   |IP Hdr | L4 Hdr |0x4c48dbc6ddf6670c ..... |
   +-------+--------+-------------------------+

   Midstream SVR Packet with False Positive SVR Metadata inserted
   +--------+--------+--------+---------------------------+
   |        |        |  SVR   |                           |
   | IP Hdr | L4 Hdr |Metadata| 0x4c48dbc6ddf6670c ...... |
   |        |        |  HDR   |                           |
   +--------+--------+--------+---------------------------+
Figure 22

Insertion of header or payload TLV's is OPTIONAL and at the discretion of the implementation. If adding TLV's, standard procedures MUST be applied including encryption if payload TLV's are added.

4.2.7. TCP to UDP Transformation

TCP to UDP transformation is required when a middlebox blocks certain TCP packets that contain SVR Metadata. SVR implementations typically test Peer Pathways to ensure SVR Metadata insertion into TCP SYN packets will pass through any middleboxes. If TCP SYN packets with SVR Metadata are dropped by a middle box, then TCP packets are transformed to UDP for SVR processing, and restored when exiting SVR processing. The steps to transform TCP to UDP are:

The protocol field in the IP header MUST be changed from 0x06 (TCP) to 0x11 (UDP).

The UDP checksum will write over the sequence number. To save the sequence number, it is copied to the 32-bit checksum/urgent pointer location of the TCP header.

To positively communicate that TCP to UDP transformation has occurred, one must add TLV 12 to the SVR Metadata being transmitted. See Section 7.4.9.

The UDP transformation is for every packet in a session, not just the packets with SVR Metadata. The restoration process is depicted in Section 4.6.3.

4.3. Required and Optional TLVs

4.3.1. New and Moved IP Sessions TLVs

The SVR Metadata TLVs that MUST be inserted in a first forward SVR Metadata packet of a new sessions includes:

Optional SVR Metadata TLV's that MAY be included in forward SVR Metadata are:

The order of the TLVs is arbitrary, but header TLVs must be before any payload TLVs. If a TLV is received that is unknown to a peer, it MUST ignore it.

The SVR Metadata TLVs that MUST be inserted in a first reverse packet of a new sessions include:

Optional SVR Metadata TLV's that MAY be included reverse SVR Metadata are:

4.3.2. ICMP TLVs

The SVR Metadata TLVs that MUST be inserted when returning an ICMP Error include:

Optional SVR Metadata TLV's that MAY be included reverse SVR Metadata are:

4.4. SVR Metadata Encryption

Encryption of SVR Metadata utilizes block mode ciphers. Cipher's MUST have a consistent block size. The cipher to use and its block size MUST be provisioned and known to peers in advance. The provisioning methodology is outside the scope of this document. The SVR Metadata Key used for encryption is specific to all Peer Pathways between any two peers and is obtained using BFD with SVR Metadata (see Section 5.1.9). When data is encrypted with block mode ciphers, the block will be padded with zeros (0x0's) to equal an increment of the block size used by the cipher. An initialization vector allows the decryption to be performed without any state.

SVR Metadata Block


          Cipher      Block Size         IV Size
         -------   -----------------     --------
          AES256   128 Bits(16 Bytes)    16 Bytes
          AES128   128 Bits(16 Bytes)    16 Bytes

      +----------+--------+---------+---------+----------------+
      +   SVR    |        |         |         |                +
      | Metadata | Header | Payload | Padding | Initialization |
      | Header   | TLVs   | TLVs    |         |    Vector      |
      +----------+--------+---------+---------+----------------+
      |<------Clear------>|<--- Encrypted --->|

      |<------------------ SVR Metadata Block ---------------->|
Figure 23

The padding can be computed as the length of the SVR Metadata payload TLVs MOD block size.

4.5. SVR Packet Authentication

4.5.1. HMAC Signatures

Through provisioning (outside the scope of this document), an SVR Authority MUST define if HMAC signatures are to be used. An SVR Authority MUST also define if Time Based HMAC is to be used. AN SVR Authority MUST determine if ALL packets are signed, or just packets containing SVR Metadata. Due to the possibility of replay attacks, it is RECOMMENDED that Time Based HMAC signatures be used on ALL SVR packets. The Session HMAC Key is determined at time of session initialization and defaults to the SVR Metadata Key (see Section 5.1.9).

SVR Peers SHOULD sign all packets with HMAC signatures defined in [RFC2104]. The Session HMAC Key should be used when creating an HMAC signature. When present there MUST be only one HMAC signature in an IP packet even if it fragments across multiple physical IP packets. Time-based HMAC signatures are RECOMMENDED. For time-based HMAC signatures, SVR routers append the current time since epoch (measured in seconds) divided by 2 to the data being signed. SVR routers MUST have clocks synchronized accurately. Methods for synchronizing clocks and measuring any differences or drifts are outside the scope of this document. Minimally NTP [RFC5905] should be implemented. In cases where the current time cannot be relied on, one may need to disable the time based HMAC and use a standard HMAC, but this is NOT RECOMMENDED.

The HMAC signature is always added to the very end of a packet. The size of the HMAC signature depends on which signature is used. Well known HMAC types are used with SVR including SHA1, SHA256-128, and SHA256.

Location of HMAC Checksum


    SVR Packet with SVR Metadata inserted
    +-----------+--------------+---------+----------+-------+
    |IP Header  |  L4 Header   | SVR     | PAYLOAD  | HMAC  |
    |           |              |Metadata |(optional)|       |
    +-----------+--------------+---------+----------+-------+
                |                                   |
                |<======= HMAC Signed Data ========>|

    Subsequent SVR Packet
    +-----------+--------------+---------+-------+
    |IP Header  |  L4 Header   |Payload  | HMAC  |
    |           |              |         |       |
    +-----------+--------------+---------+-------+
                |                        |
                |<== HMAC Signed Data ==>|


       HMAC TYPE          LENGTH OF SIGNATURE
      ------------------  ----------------------
         SHA1             20 Bytes
         SHA256-128       16 Bytes
         SHA256           32 Bytes
Figure 24

4.5.2. HMAC Verification

If HMAC signatures are present in an SVR implementation, SVR implementations MUST verify and remove the signature. Verification provides both authentication of the SVR router that sent the packet, and integrity that the packet has not been modified in any way intentionally, or through transmission errors between two SVR routers.

Through provisioning (outside the scope of this document), an SVR Authority MUST define if HMAC signatures are present. An SVR Authority MUST also define if Time Based HMAC is to be used. AN SVR Authority MUST determine if ALL packets are signed, or just packets containing SVR Metadata. Due to the possibility of replay attacks, it is RECOMMENDED that Time Based HMAC signatures be used on ALL SVR packets. The Session HMAC Key associated with the session state is used for all HMAC signatures and verification.

To verify the HMAC signature, a new signature is generated on the packet and bytewise compared to the signature transmitted in the packet.

HMAC Signature Removed


    SVR Packet with HMAC Signature
    +-----------+--------------+----------+-------+
    |IP Header  |  L4 Header   | PAYLOAD  | HMAC  |
    |           |              |(optional)|       |
    +-----------+--------------+----------+-------+
                |                         |
                |<== Signed Data ========>|


    SVR Packet with HMAC Signature removed
    +-----------+--------------+----------+
    |IP Header  |  L4 Header   | PAYLOAD  |
    |           |              |(optional)|
    +-----------+--------------+----------+
Figure 25

For efficiency reasons, when verifying an Time Based HMAC signature, implementers SHOULD compute the HMAC on the packet (not including the IP header) and save the preliminary result. Then try updating the HMAC signature with the current window value. If this fails to match the signature, one must try updating the preliminary result using the next time window by adding 2 seconds (or previous by subtracting 2). If the time window is determined to be the next time window; it will remain that way for all packets received from a particular peer until it advances with clock time. Keeping an active time window per peer can make this process much more efficient.

If the signature does not match after checking adjacent time windows and newly issued keys, then the packet is dropped and a security event noted.

If the signature matches exactly the signature in the packet, then the packet has been authenticated as being sent by the previous SVR router, and assured that the packets integrity between the two routers is good. The HMAC signature MUST be removed from the packet.

The IP Packet length field MUST be updated to reflect the number of bytes removed.

The IP Header Checksum MUST be updated after the IP Packet length is adjusted.

4.6. Processing SVR Packets with Potential SVR Metadata

Routers MUST process SVR traffic and non-SVR traffic. SVR Routers MUST keep track of sessions that are using SVR. Only sessions setup with SVR may use the procedures described below. Traffic that is using SVR will always originate and terminate on Waypoint addresses (known peer pathways). This provides efficient separation of non-SVR traffic and SVR traffic.

Packets received on known Peer Pathways MUST be assumed to either have SVR Metadata or be packets associated with existing SVR sessions.

4.6.1. Detection of Potential SVR Metadata in Packets

Any packet could arrive at any time with SVR Metadata. DPI MUST be used to scan for the presence of SVR Metadata on every packet. SVR Metadata MAY be expected and required for first packet processing, and the absence of SVR Metadata will result in dropped packets.

The HMAC verification step (defined above) MUST be performed prior to performing any other SVR Metadata verification steps. This prevents attacks by modifying packet on the wire.

If the first 8 bytes of the payload (TCP or UDP) exactly matches the SVR magic number (0x4c48dbc6ddf6670c) it indicates that packet MUST have SVR Metadata. If the first 8 bytes do not match, the packet does not contain SVR Metadata. If SVR Metadata is not present, the packet SHOULD be routed if part of an existing session (see Section 4.6.4). If not part of an existing session the packet MUST be dropped and a security event noted.

4.6.2. Verification of SVR Metadata in Packets

4.6.2.1. TLV Parsing

The SVR Metadata header is parsed (see Section 7.1). If the header length and payload length are both zero, the SVR Metadata is simply removed and the packet is forwarded. Please see Section 4.2.6 for description of false positive SVR Metadata header insertion. The next step is to walk the header TLV's to ensure they are reasonable. If the payload length is zero, then the SVR Metadata can be accepted and processed. Decryption of SVR Metadata is only required when there are payload TLV's.

If a TLV is sent that is unknown to the implementation, the TLV SHOULD be skipped and the TLV MUST be forwarded.

If the SVR Metadata TLVs are not reasonable, the packet MUST be dropped and security events noted.

4.6.2.2. Decryption of SVR Metadata Blocks

If the peers have been provisioned to encrypt SVR Metadata with a specific cipher AND the payload length in the header is non-zero, then the SVR implementation MUST assume that an encrypted SVR Metadata block was transmitted.

To decrypt the encrypted SVR Metadata block, an SVR implementation MUST have the pre-provisioned cipher, block size, and initialization vector size. Once these are known, it is possible based on the payload length in the SVR Metadata header to determine the exact structure of the packet, and how to decrypt it.

Encrypted SVR Metadata Block


        Known in advice: Cipher, Block Size, IV size
        From SVR Metadata Header: Payload TLV size

   +----------+--------+-------+-------+----------------+--~~~
   |  SVR     | Header |Payload|Padding| Initialization | Rest...
   | Metadata | TLVs   |TLVs   |       |    Vector (IV) | of  ...
   | Header   |        |       |       |                | Pkt ...
   +----------+--------+-------+-------+----------------+--~~~
   |<------Clear------>|<- Encrypted ->|

   |<-------------- SVR Metadata Block ---------------->|
Figure 26

The padding is equal to the payload length from the header MOD cipher block size. The "block" is then decrypted assuming that the IV size bytes following the "block" is the Initialization vector.

If the decryption fails, then the packet MUST be assumed invalid and dropped. When this happens a security event is noted.

After the decryption succeeds, the payload TLV's MUST be reviewed for reasonableness and completeness. See Section 4.3 for minimum required TLV's. If there are insufficient TLV's present for the SVR implementation, the packets MUST be dropped and errors noted.

After review of the TLV's, the SVR Metadata is considered valid and accepted by the SVR implementation. The SVR Metadata block is removed from the packet, and the IP header length and checksum MUST be corrected. The packet signatures and decryption provide a very high degree of assurance that the SVR Metadata is authentic and has integrity.

4.6.3. UDP to TCP Transformation

If the received SVR Metadata block contains a TCP SYN Packet TLV (see Section 7.4.9) then the following procedures MUST be performed on EVERY packet of the session. This also signals to the SVR Router that packets flowing in the opposite direction MUST also be UDP transformed. See Section 4.2.7. The steps performed are:

The protocol field in the IP header MUST be changed from 0x11 (UDP) to 0x06 (TCP).

Copy the 32-bit integer in the checksum/urgent pointer location of the TCP header to the sequence number, effectively restoring it.

The TCP Checksum MUST be recalculated.

4.6.4. SVR Session Packets

Any packet that is has a source and destination IP address that maps to a Peer Pathway is an SVR packet. SVR Packets that do not have SVR Metadata are SVR session packets. Each of these MUST have corresponding known session state. If no session state exists, these packets MUST be dropped, or there must be an attempt to restore session state (see Section 2.11).

Packets ingressing to a peer pathway that are part of existing SVR sessions that do not contain SVR Metadata MUST be translated (all 5-tuples, bidirectionally). The source address MUST be replaced with the local Waypoint address associated with the peer pathway. The destination address MUST be replaced with the Waypoint of the SVR Peer chosen. The protocol either remains the same, or is modified if UDP Transformation is required (See Section 4.2.7). The source and destination port fields MUST be replaced with the ports allocated for this SVR session. For efficiency, implementors SHOULD save a single checksum delta as part of the session state because the address/protocol/port modifications will always be identical for each packet of a session.

Packets egressing from a peer pathway must have their addresses restored. SVR session state MUST contain the original packet context 5-tuples for every SVR session. The original Source IP Address MUST be restored. The original Destination IP Address MUST be restored. The original protocol must be restored, and if it is changes from UDP to TCP then one MUST follow the procedures defined in Section 4.6.3. The source port MUST be restored. The destination port MUST be restored.

4.6.5. Tenant/Service Overview

A provisioned SVR Policy SHOULD include both a tenant and service. Absence of an applicable SVR policy SHOULD prevent SVR sessions from being established. Traditional IP routing can be used when SVR policies do not apply.

4.6.5.1. Interpretation of the Service

Services are textual names for sets of CIDR blocks, protocols, and ports. Services map directly to our human understanding of a network use case. Examples include "Zoom" or "Office365".

Service Definition


            svc_name
                protocol: TCP/UDP
                port ranges[]
                CIDR Blocks[]
Figure 27

When a packet arrives with SVR Metadata at an SVR Router the name of the service MUST be in first packet SVR Metadata.

When a first packet arrives without SVR Metadata, the service must be determined through a lookup of the IP destination address, port, and protocol. The resultant string becomes the service name. If this fails to result in a service, the name of the service can be determined by using application recognition techniques. These are omitted from this document, but include HTTP Request Analysis, TLS SNI, and Common Names in certificates.

Services can have associated quality policies and security policies associated with them via provisioning. This is outside the scope of this document.

When egressing an SVR Peer Pathway, the service name can be used to route the packet to another SVR Peer, or to the final destination. If another SVR peer is chosen, the service name MUST be used as provided by the previous SVR peer. When exiting SVR and returning to traditional network routing, the textual service name MUST be resolved to an IP address. SVR supports several options:

Use Destination from Context:
This is the default action. The original destination address will be restored and the packet will be forwarded to the destination.
Destination NAT Based on Local Configuration:
Some provisioned service configurations locally (nearest the destination SVR router) will map the service to one or more local IP addresses through implementation of a destination NAT. This effectively becomes a load balancing algorithm to destination service instances, and is very useful in public clouds.
Resolve Destination using Local DNS:
DNS resolution can be provisioned for services when the IP address is not known. This if often the case with services in private clouds.

Services SHOULD be provisioned to have lists of Tenants that are permitted to use a Service, and tenants that are denied using a service. These access controls are RECOMMENDED.

4.6.5.2. Determination and Interpretation of the Tenant

Tenant is a text string hierarchy delimited by periods. Tenants are logically similar to VLANs, CIDR block subnets, and Security Zones. The entire text string, including the full hierarchy is used to define a tenant, and for policy application, the tenant MAY match right to left in full segments (delimited by periods). The longest match will always be used (the most segments).

Tenants SHOULD be referenced and associated with Services to create a from-to vector. This has the benefits of associating ACLs directly with Destinations. A provisioned SVR Policy SHOULD include both a tenant and service. Absence of a applicable SVR policy prevents SVR sessions from being established. The deny by default approach is RECOMMENDED.

It is RECOMMENDED that a tenant be associated with physical interfaces and logical interfaces (VLANs) as a default for arriving sessions. CIDR block-based tenants SHOULD override these defaults. Tenant definitions directly from clients that self-assert their tenancy SHOULD override all other tenant definitions.

All network interface-based tenant definitions are local to an SVR router. The tenant definitions on ingress to SVR MAY not match those on egress from SVR. This permits the use of different segmentation techniques in different networks.

4.6.6. Payload Encryption

If payload encryption is required, a Security Policy is used to describe all aspects of the agreed upon methods. The Security Policy meaning must be valid and equal at the point of encryption and decryption in multi-hop use cases.

SVR routers generate a Payload Key locally when processing the first packet requiring encryption that conforms to the defined security policy. A FIPS 140-2 approved highly secure DRBG is used to generate keys and IV. It's also highly recommended that NIST SP800.90B be followed for ensuring proper entropy. See [NIST_SP_800-90B]. The OpenSSL function RAND_bytes() can be used to generate a key that is the current number of bits long and conforms to NIST SP-800-90B. This key is included in the first encrypted packet as SVR Metadata. In a post quantum world, the key could be generated from quantum sources.

The SVR Metadata is forwarded hop-by-hop through the network until the packet egresses SVR. The router that terminates the SVR routing MUST extract the Payload Key from the SVR Metadata, and store it as part of the session state information. This key will be used to decrypt all payload packets from this session.

Packets traversing the reverse pathway will use the encryption key received during session establishment. Payload encryption and decryption uses a symmetric key.

The originator of the sessiom can re-key at any time by adding SVR metadata with the new key to a packet within the session. The payload will immediately be encrypted by the new key. Because a session must exist in order for reverse traffic to flow, the originating router can guarantee reverse traffic will use the same encryption key, avoiding race conditions.

In the forward direction, the Security KEY TLV (see Section 7.4.15) contains the key for encryption/decryption in the first packet in each direction. This allows the key for decryption to go end-to-end in multi-hop router cases. The key is safe because SVR Metadata is encrypted hop-by-hop through the network. Thus, each payload encrypted packet is decrypted once at the end of the SVR route. Using a named Security Policy permits implementations to use whatever ciphers are required, as long as they can be named. The default cipher is AES256 with a 256 bit key.

Payload Encryption:


          PeerA           PeerB           PeerC
            |               |               |
    fpkt--->|               |               |
            |               |               |
         Gen Key            |               |
         Encrypt Payload    |               |
         Add Key to MD      |               |
            |               |               |
            |--fpkt w/md--->|               |
            |            Forward            |
            |               |--fpkt w/md--->|
            |               |           Save Key
            |               |               |
            |               |               |--fpkt->
            |               |               |
            |               |               |<-rpkt
            |               |               |
            |               |            Lookup Key
            |               |           from session
            |               |          Encrypt Payload
            |               |<--rpkt -------|
            |            Forward            |
            |<--rpkt -------|               |
            |               |               |
     <-rpkt-|               |               |
            <===ALL PAYLOAD PKTS ENCRYPTED ====>
Figure 28

5. BFD for Peer Pathways

Peer Pathways are similar to Tunnels. They represent virtual transport pathways between routers. BFD is an excellent way to determine reachability, measure quality of a pathway, and to perform authentication and key management.

5.1. SVR Peering and use of BFD

It is RECOMMENDED for every configured or discovered SVR Peer pathway, A UDP BFD session be used to monitor the state of the pathway, and through extensions, measure path quality.

BFD Control messages are sent by each router on each peer path. The BFD message is constructed with appropriate timers for the Peer Pathway which are administratively determined. BFD as defined in [RFC5880] does not support certificates or exchange of public keys. To overcome this, BFD Metadata is used.

BFD Metadata is inserted into existing BFD messages for the following purposes:

  • To determine the Peer Received IP Address.
  • To determine there are NATs on a Peer Pathway.
  • To determine if a routers Peer Received IP Address has changed.
  • To determine MTU size on a pathway.
  • Measure path quality when idle (see Section 7.3.7 for measuring quality on active circuits).
  • Determine if a Peer Pathway has failed to another redundant physical link.
  • To authenticate a peer through certificate exchange.
  • To determine a Peer Key for encryption of SVR Metadata Keys.

BFD Metadata is added to the end of the BFD packet when required. If BFD Metadata is added, the length field in the IP Header, UDP Header, and BFD Control message are all adjusted to be accurate.

BFD Metadata Location:


      BFD Control Packet with BFD Metadata

    +-----------+--------+---------+----------+
    |IP Header  |  UDP   |   BFD   | protobuf |
    |           | Header | Control | BFD      |
    |           |        |  Packet | Metadata |
    +-----------+--------+---------+----------+
                         |                    |
                         |<== BFD Pkt Len  ==>|
Figure 29

In all cases, BFD packets will be defined as BFD Control Packets. When sending MeasureData messages which behave like BFD Echo packets, the Required Min Echo RX Interval (see [RFC5880]) must be greater than zero.

BFD Metadata is described as follows:

BFD Metadata Protobuf Definition:

  syntax = "proto2";
  package pb.bfd;
  import "ip.proto";

  message SessionData {
      required ip.Tuple original_ipTuple  = 1;
      required ip.Tuple received_ipTuple  = 2;
      optional string peername = 3;
      optional string routername = 4;
      optional string routerUUID = 5;
  }

  message MeasureData {
      message Request {
          required uint32 transId = 1;
      }
      message Response {
          required uint32 request_transId  = 1;
          required uint32 response_transId = 2;
      }
      oneof type {
          Request      request   = 1;
          Response     response  = 2;
      }
      optional bool mtu_discovery = 3;
  }

  message NodeInfo {
      required uint32 id = 1;
      required uint64 create_timestamp = 2;
      optional uint64 time_value = 3;
      optional string public_key=4;
      optional uint32 salt=4;
  }

  message SVR_key_data {
      optional NodeInfo node_info = 1;
      optional string encrypted_metadata_key = 2;
      optional int32 metadata_key_index = 3;
  }

  message Metadata {
      optional SessionData     sessionData  = 1;
      optional MeasureData     measure      = 2;
      optional NodeInfo        nodeInfo     = 3;
      optional SVR_key_data    svrkeydata   = 4;
  }
Figure 30

5.1.1. Peer Determination of Received Peer IP Address

The SessionData message can be used to determine the source address of a remote peer router on a Peer Pathway. This is required to establish a peer path. Configuration will be tied to a router hostname, and not a dynamic address associated with a hostname. Remote Peers will create a local address resolution table (i.e. /etc/hosts) to resolve the hostname in configuration to the dynamic IP address. This action can be performed simultaneously with detection of NAT between Peers below.

Determination of Peer Received Address:


     Router-A                Router-B                 Local
    [Addr-A ->               -Addr-B]                  DNS
        |                       |                       |
        |BFD ------------------>|                       |
        | original_ipTuple=A    |                       |
        | hostname="Router-A"   |                       |
        |                       |DNS Update------------>|
        |                       | Router-A: Address A   |
        |                       |                       |
        |                       |                       |

    Router-B has hostname lookup for Router-A
Figure 31

5.1.2. Detection of NAT between Peers using BFD

The SessionData message can optionally be used to detect NATs between two routing peers. Typically, this is performed during initial peer pathway establishment and often grouped together with sending Peer Authorization certificates. Similar to STUN, the IP address of the originating interface is stored in the field SessionData.original_ipTuple. If the router has received any BFD packets from its peer router, it will store the IP address of the received BFD packet in this field. When sending the SessionData BFD Metadata, a router OPTIONALLY places its own name in the peername field. Through the process of comparing addresses within the IP header with addresses used by the router's interfaces, one can detect when there is a NAT on a Peer Pathway.

BFD NAT Detection on Pathway:


     Router-A                  NAT                  Router-B
     Addr-A                   Addr-N                 Addr-B
        |                       |                       |
        |BFD ------------------>|                       |
        | original_ipTuple=A    |                       |
        |                       |                       |
        |                       |BFD ------------------>|
        |                       | original_ipTuple=N    |
        |                       |                       |
        |                       |                       |

                                                NAT Detected
                                                Router-B gets N
                                                address on the wire
                                                and it doesn't match
                                                original_ipTuple
        |                       |                       |
        |                       |                       |
        |                       |<-------------------BFD|
        |                       |    original_ipTuple=B |
        |                       |    received_ipTuple=N |
        |<-------------------BFD|                       |
        |    original_ipTuple=B |                       |
        |    received_ipTuple=N |                       |
        |                       |                       |

    No NAT detected
    Router-A gets B's address
    on the wire which matches
    the original_ipTuple
Figure 32

If a NAT is detected in a Peer Pathway, care must be taken to associate address N with the Peer Pathway to Router-A. Sessions that are traversing this Peer Pathway may require NAT Keep Alive processing. See Section 6.3.

5.1.3. Detection of Routers Address Changes using BFD

Often branch data routers are connected to networks and receive their IP Address dynamically from DHCP, LTE or PPPoE procedures. Although it is rare, sometimes these addresses change unexpectedly. This may be the result of a lease running out, or a router reestablishing connectivity after a failure. When this happens, any peer that was using the old address will lose connectivity to this peer. By including SessionData BFD Metadata, learning the address of the peer and recovery occur very quickly.

BFD Detection on Router Address Change:


     Router-A               DHCP                Router-B
    [Addr-A ->              Server            <-Addr-B]
        |                     |                     |
        |BFD -------------------------------------->|
        | original_ipTuple=A  |                     |
        | received_ipTuple="" |                     |
        |                     |                     |
        |<---------------------------------------BFD|
        |                     |  original_ipTuple=B |
        |                     |  received_ipTuple=A |
        |BFD -------------------------------------->|
        | original_ipTuple=A  |                     |
        | received_ipTuple=B  |                     |

        Both routers have learned each other's IP Address
        and have determined there are no NAT's between them

        |DHCP Lease Exp ----->|                     |
        |<-------New Address C|                     |
        |                     |                     |
        |BFD -------------------------------------->|
        | original_ipTuple=C  |                     |
        | received_ipTuple=B  |                     |
        |<-------------------------------------- BFD|
        |                     |  original_ipTuple=B |
        |                     |  received_ipTuple=C |

        Both routers have the correct IP Address and
        have determined there are no NATs between them
Figure 33

5.1.4. Determining MTU Size with BFD

Knowing the MTU size on a path is important for routers so they can fragment packets when necessary. After a peer pathway is established, a series of BFD MeasureData packets that increase in size can help us find the limit of packet size between peers. To make the BFD packet larger, the lengths are adjusted in the IP header, UDP header, and BFD header. A peer receiving a fragmented BFD request with the MTU Discovery field equal to TRUE simply does not respond.

Often there is an entire network between peers. As such, the MTU size may change over time. It is recommended that MTU be measured routinely and updated, should it change.

BFD MeasureData for Determining Pathway MTU:


     Router-A                                       Router-B
    [Addr-A ->                                     <-Addr-B]
         |                                                |
         |BFD MeasureData (id=1, size 1200)-------------->|
         |BFD MeasureData (id=2, size 1250)-------------->|
         |BFD MeasureData (id=3, size 1300)-------------->|
         |BFD MeasureData (id=4, size 1350)-------------->|
         |BFD MeasureData (id=5, size 1400)-------------->|
         |BFD MeasureData (id=6, size 1450)-------------->|
         |BFD MeasureData (id=7, size 1500)-{fragmented}->|
         |                                                |
         |<----(req_id=1, resp_id=1)-------BFD MeasureData|
         |<----(req_id=2, resp_id=2)-------BFD MeasureData|
         |<----(req_id=3, resp_id=3)-------BFD MeasureData|
         |<----(req_id=4, resp_id=4)-------BFD MeasureData|
         |<----(req_id=5, resp_id=5)-------BFD MeasureData|
         |<----(req_id=6, resp_id=6)-------BFD MeasureData|

         MTU Size = 1450
Figure 34

5.1.5. Measuring Peer Pathway quality using BFD

After a Peer Pathway is authenticated and ready for use, BFD can be used to measure latency and packet loss. This is performed by sending BFD packets with BFD MeasureData Metadata. Both sides of a Peer Pathway can test for quality if desired. The number of packets in a burst is determined by configuration. The frequency of quality tests is also determined by configuration. Quite often routers with a large number of Peer Pathways (such as a data center hub router) may never perform quality tests, and rely solely on observations made by its peer spoke routers.

These quality measurements are only required when circuits are idle. When sessions are traversing a peer path, quality measurements can made for existing sessions using SVR Path Metrics (See Section 7.3.7).

The receiving side generates a response message by re-writing the BFD Metadata request and supplies information if requested. Each "request" generates a "response". Each request has a transaction ID, and so does each response. This solves a problem of exact symmetry where by a peer may not know if a message is a response or a request from a peer.

BFD MeasureData for Measuring Pathway Quality:


     Router-A                                     Router-B
    [Addr-A ->                                   <-Addr-B]
         |                                             |
         |BFD MeasureData (req_id=1)------------------>|
         |BFD MeasureData (req_id=2)------------------>|
         |BFD MeasureData (req_id=3)------------------>|
                 .......
         |BFD MeasureData (req_id=n)------------------>|
         |                                             |
         |<----(req_id=1, resp_id=1)----BFD MeasureData|
         |<----(req_id=3, resp_id=2)----BFD MeasureData|
         |<----(req_id=1, resp_id=3)----BFD MeasureData|
                  ......
         |<----(req_id=N, resp_id=N-1)--BFD MeasureData|

      Latency = Sum of RTT(pkt 1-n)/(2*n)
      Jitter = Std Dev RTT(pkt 1-n)
      Packet Loss = 1-((Pcks_Sent - Pcks_recv)/Pkts_Sent)
Figure 35

Router-B responds to each BFD MeasureData message it receives by responding to the original message and adding a serialized resp_id. To measure latency, the sending (measuring) side (Router-A in this case) can measure the elapsed time between each req_id sent, and its response. Absence of a response counts as a packet lost. The variability in latency provides a method of calculating jitter, and MoS scores can be computed once latency, packet loss, and jitter are known.

Both Router-A and Router-B must send their own BFD MeasureData messages to produce their own quality measurements from their own specific point of view. The actual network quality between these two routers can vary based on direction.

5.1.6. Detection of Path Failover using BFD

If one side of a Peer Pathway fails and there exists a policy to choose an alternate path, BFD NodeInfo Metadata can be used to detect this event. Knowledge of a Peer Pathway failover may be required by routers in certain scenarios.

For redundancy, routers are often grouped into a cluster of active/active nodes. Responsibility for a Peer Pathway may change from one member of a cluster to another. When sending BFD with Metadata, by including the Node ID (instance number in a cluster) and a timestamp of when the Peer router started, one can detect redundancy events at the far end side of a Peer Pathway.

Inclusion of this information is optional. This data can be used by a remote peer to trigger actions where redundancy events impact them.

5.1.7. Peer Authentication Procedures

When a router is initialized, if it does not have a signed authentication certificate that is valid, it must obtain one from a certificate authority (CA). The router will create an elliptic-curve public/private key pair (see [RFC8422]). The public key is used to create an X.509 certificate signing request (CSR) with the common name field set to the router's UUID. Elliptic-curve is used to ensure the X.509 certificate is as small as possible. A certificate signing request is initiated to a known and trusted CA through a secure connection. The CA will digitally sign (ECDSA) the the CSR and return it to the requesting router. The specific details of this process is omitted from this specification, but it is recommended that it follow the procedures and guidelines defined in [RFC4210]. Certificates and Public Keys are stored locally on each router in PEM format defined by [RFC7468].

Router Authentication:


                           RouterA
                         Certificate
            RouterA       Authority
              |                |
       +------+------+         |
       |Cert Missing,|         |
       | Invalid     |         |
       | or Expiring |         |
       +-------------+         |
              |                |
        +-----+------+         |
        |   Create   |         |
        | Curve-P256 |         |
        |  Pub/Priv  |         |
        |  Key Pair  |         |
        +------------+         |
              |                |
        +-----+------+         |
        |   Create   |         |
        | X.509 Cert |         |
        | CN=RouterA |         |
        +------------+         |
              |                |
              +------CSR------>|
                               |
              |<-X.509 Signed--+
Figure 36

The certificate is stored on the router persistently in PEM format. The private key associated with the certificate should be created and stored in a secure non-volatile storage, such as a Trusted Platform Module (TPM).

When establishing a peer pathway, the SVR Routers will authenticate with each other using their public key and UUID. The result will be a symetric Peer Key derived from the routers private and public keys from the X.509 certificate above. UUIDs are used for authentication because router IP Addresses often change. This is true when transport addresses of branch routers are established using DHCP and leases expire.

The diagram below shows two routers, with two peer pathways. BFD Messages with Nodeinfo are sent that contain the X.509 Certificate in the public_key field. The BFD Nodeinfo messages are sent by both routers on all pathways, but only need to be validated one time for each router peer.

Router Authentication:


           RouterA      RouterA      RouterB      RouterB
           Peerpath1    Peerpath2    Peerpath1    Peerpath2
               |           |            |            |
       =============ALL PEER PATHS ARE DISCONNECTED==========
               |           |            |            |
               |--BFD w/X.509 Cert----->|            |
               |           |--BFD w/X.509 Cert------>|
               |           |            |            |
              ....Delay between retransmissions .......
               |           |            |            |
               |--BFD w/X.509 Cert----->|            |
               |           |         RouterA         |
               |           |        Validated        |
               |           |            |            |
               |           |--BFD w/X.509 Cert------>|
               |           |            |            |
               |<----BFD w/X.509 Cert---|            |
            RouterB        |            |            |
           Validated       |            |            |
               |           |<-----BFD w/X.509 Cert---|
               |           |            |            |
      =============ALL PEER PATHS ARE OPERATIONAL==========
               |           |            |            |
              ....Delay between retransmissions .......
               |           |            |            |
               |----BFD---------------->|            |
               |           |-------BFD-------------->|
               |<-------------BFD-------|            |
               |           |<-------------BFD--------|
Figure 37

When a certificate is received from a peer, it must be validated. The validation includes the following checks:

  • Verify the dates are valid.
  • Verify the signature of the Certificate Authority.
  • If revocation list available, verify the certificate has not been revoked.
  • Verify the router name is supported in configuration. Administrative revocation is a primary means of control.

The validation for a peer only needs to be done one time. When a certificate is received from a peer on multiple peer paths, if the certificate is identical to a previously validated certificate, a cached validation response can be used.

When receiving a certificate from a peer router, after validation, the receiving router must extract the peer router's public key and save it. This will be used for validating Peer Key/rekey request's authenticity.

Each router should update its authentication certificate before the current certificate expires utilizing the same exact steps identified herein.

5.1.8. Peer Key-Rekey Procedures

A single Peer Key is used for all paths between two router peers. The key is kept and considered valid until a new key is accepted as a replacement. This key continues to be used through network outages and path failures. If a key is lost, or doesn't appear to function correctly, a new key must be obtained before processing of encrypted BFD traffic.

Anytime a key replacement is desired, or a key is needed, a new salt value is created by the initiator and sent with BFD NodeInfo to a peer. The peer responds by generating a new salt value and responding with a BFD NodeInfo message. Once both salt values are obtained, the Concat KDF calculation can proceed resulting in a symmetric key value for both peering routers.

A Key Derivation Function called Concat KDF (See [NIST_SP_800-56A] is used to calculate an authenticated peer key. This key calculation utilizes the routers authenticated certificates private keys, and as such the key is safe from man in the middle attacks.

OpenSSL has a standard function called ConcatKDF() that can be called to compute this key in a FIPS compliant fashion. The parameters are:

ConcatKDF Function (Part of OpenSSL):


          Peer Key = ConcatKDF(SharedSecret,
                                AlgorithmID,
                                PartyUInfo,
                                PartyVInfo,
                                SuppPubInfo,
                                SuppPrivInfo,
                                KeyDataLen)

          Here's what each parameter represents:

          SharedSecret: The result of an ECDH calculation with the peer
          AlgorithmID: "ECDH"
          PartyUInfo: UUID of the Router
          PartyVInfo: UUID of the Peer Router
          SuppPubInfo: Initiator Salt Concatenated with Responder Salt
          SuppPrivInfo: ""
          KeyDataLen: 256

The Salt values are concatenated octet strings, with initiator salt first followed by responder's salt. For ECDH calculations, please see [ECDH_Key_Exchange].

After a short guard time (1-2 seconds) to allow both peers to complete their calculation, the Peer Key is ready for use, and replaces any pre-existing key. The key is then valid on all peer paths between two peers. Once calculated on one peer path, it can be used immediately on all other paths with the same remote peer.

The key can be immediately used for encrypting BFD Metadata.

Peer Key-Rekeying:


           RouterA      RouterA      RouterB      RouterB
           Peerpath1    Peerpath2    Peerpath1    Peerpath2
               |           |            |            |
            .......NO Current Peer Key Exists............
               |           |            |            |
            Gen Salt       |            |            |
               |           |            |            |
               |--BFD Nodeinfo Req----->|            |
               |           |            |            |
               |           |        Gen Salt         |
               |           |            |            |
               |<----BFD Nodeinfo Req---|            |
               |           |            |            |
              Key          |           Key           |
            Computed       |         Computed        |
               |           |            |            |
             ........Transition Guard Time..............
               |           |            |            |
             ......... 1st Peer Key Exists..............
               |           |            |            |
             ...........At Rekeying Interval............
               |           |            |            |
               |        Gen Salt        |            |
               |           |            |            |
               |           |--BFD NodeinfoReq------->|
               |           |            |            |
               |           |            |         Gen Salt
               |           |            |            |
               |           |<---BFD Node Info Req----|
               |           |            |            |
              Key          |           Key           |
            Computed       |         Computed        |
               |           |            |            |
             ..........Transition Guard Time.............
               |           |            |            |
             ........... 2nd Peer Key Exists.............
Figure 38

The peer key is a symetric key used to encrypt the delivery of the SVR Metadata Key. If the Peer Key is invalid, or expired, or can't be generated, the SVR Routers will not be able share their SVR Metadata Key, which will prevent the routers from functioning. The Concat KDF is a form of ECDH-ES that will only produce a symetric key if there is no man in the middle. If in use, the keys fail to decrypt messages, it is likely a man in the middle exists, and the route peers should remove the Peer Pathway from service.

If a peer sends a BFD NodeInfo to a peer, and there is no response, the peer continues to resend it at periodic intervals. If there is no response after a very long period of time, the peer path can be declared not valid, and removed from service based on administrative timers.

5.1.9. SVR Metadata Key-Rekey

SVR Metadata requires a security association that is not Peer Pathway specific. The interface or source IP Address can not uniquely identify an SVR Peer. For example, there could be many Peer Pathways connected over the public internet. The only way to identify the peer positively is to decrypt the SVR Metadata, and extract the peer name. Each SVR router MUST be able to decrypt SVR Metadata arriving on any interface.

SVR Metadata thus is encrypted specifically for the chosen next hop SVR router. No other SVR router should be able to decrypt the metadata. Thus when sending SVR Metadata, peers must select a key that is directly associated with the chosen next hop SVR router.

To distribute their keys, SVR routers generate a key locally that conforms to the defined security policy. A FIPS 140-2 approved highly secure DRBG is used to generate keys and IV. Its also highly recommended that NIST SP800.90B be followed for ensuring proper entropy. See [NIST_SP_800-90B]. OpenSSL function RAND_bytes() can be used to generate a key that is 256 Bits long and conforms to NIST SP-800-90B.

The key and its index is then shared with all known peers using an Encrypted BFD Metadata that contains SVR_key_data. The Current Peer Key is used to encrypt the 256 Bit SVR Metadata Key calculated above resulting in a 32 Byte Encrypted block and a 16 Byte IV, creating a 48 byte encrypted octet string. The key index is incremented. The encrypted key is then included in a BFD packet (SVR_key_data message), and broadcast to all peers. g The encryption technique is identical to SVR Metadata Encryption but uses the Peer Key as opposed to the SVR Metadata Key. (See Section 3.7.1.8). This also means that SVR Metadata keys are asymmetric. The forward SVR Metadata is encrypted with the Key of the destination router, while reverse SVR Metadata is encrypted with the Key or the originating router.

SVR Metadata Key/Rekeying:


          PeerA       PeerB        PeerC        PeerD
            |            |            |            |
         ...NO Current SVR Metadata Key Exists For A....
            |            |            |            |
         Gen Key         |            |            |
         Inc Indx        |            |            |
            |            |            |            |
            |-BFD w/key->|            |            |
            |<-BFD w/key-|            |            |
            |                         |            |
            |--------BFD w/key------->|            |
            |<-------BFD w/key--------|            |
            |                                      |
            |----------------BFD w/key------------>|
            |<---------------BFD w/key-------------|
            |            |            |            |
         ..........A updates SVR Metadata Key .........
            |            |            |            |
       Gen New Key       |            |            |
         Inc Indx        |            |            |
            |            |            |            |
            |-BFD w/key->|            |            |
            |<-BFD w/key-|            |            |
            |                         |            |
            |--------BFD w/key------->|            |
            |<-------BFD w/key--------|            |
            |                                      |
            |----------------BFD w/key------------>|
            |<---------------BFD w/key-------------|
Figure 39

The above diagram shows Peer A distributing its key and index to Peers B, C, and D followed by a subsequent rekey. Each peer responds with its current key and index to provide a handshake. This is also an efficient way for a router that is restarting to acquire all of its needed keys. If a router needs to send SVR Metadata to a peer and it does not have a key, this procedure can be used to acquire the missing key. Anytime a Peer decides to rekey, it must update all of its peers. When a SVR Metadata Key is shared via BFD, the metadata_key_index field number is extracted and is stored with the key for the peer. When SVR Metadata is to be decrypted, the Security ID field in the metadata provides the key index to use for decryption. Please see Section 7.3.2.

5.1.10. Certificate Revocation/Replacement Procedures

In the event that a router's certificate is about to expire, or needs to be replaced, a new certificate can be added to the system. Once a new certificate file has been loaded into the system, an event is triggered to restart the peer authentication Section 5.1.7 procedure again. The method of loading the certificate is out of the scope of this document.

In the event that a system has become compromised, it may be desirable to revoke its certificate so that it can no longer authenticate with its peers. The management platform of the router is responsible for periodically checking the CRL for any revocations. A notification is sent to any router that has its certificate revoked. Upon receiving this revocation, the router will check its configuration to determine the appropriate behavior.

There SHOULD exist a policy to define the system behavior in the event that a certificate has expired or has been revoked. The default behavior SHOULD be to fail-soft (i.e., providing indication that the certificate is no longer valid and action needs to be taken). Alternatively, if the system is configured to fail-hard, it would remove all of its peering relationships and subsequently would no longer be able to participate in SVR.

6. Additional SVR Metadata Exchanges and Use Cases

SVR Metadata can be inserted and used to share network intent between routers. Below are examples for specific use cases. These examples are illustrative and the use of SVR Metadata is not limited to these use cases alone.

6.1. Moving a Session

To change the pathway of a session between two routers, any SVR Router will reinsert the SVR Metadata described in section Section 3.7.1.7 and transmits the packet on a different peer path, but retains the same Session UUID of the existing session that is being moved.

  • Simultaneously it will update its fast path forwarding tables to reflect the new IP addresses and ports (Waypoints) for the transport. All other aspects of the session remain the same. The presence of middle boxes means that routers on both sides must once again perform NATP detection and update real transmit addresses/ports to ensure that sessions will continue.

After 5 seconds the old path state entries can be removed. By keeping the old and new fast path entries during this 5 second transition, no packets in flight will be dropped. The diagram below shows the sequence for moving sessions around a failed mid-pathway router.

Ladder Diagram for Existing Session Reroute with SVR Metadata:


                   RTR-A      RTR-B      RTR-C      RTR-D
        Client . . . . . . . . . . . . . . . . . . . . . . . . Server
          |          |          |          |          |          |
          |--PUSH--->|          |          |          |          |
          |          |--PUSH-------------->|          |          |
          |          |          |          |--PUSH--->|          |
          |          |          |          |          |--PUSH--->|
          |          |          |          |          |<---ACK---|
          |          |          |          |<---ACK---|          |
          |          |<--------------ACK---|          |          |
          |<---ACK---|          |          |          |          |
          |          |          |          |          |          |
          ......................RTR-C Fails.......................
          |--PUSH--->|          |          |          |          |
          |          |--PUSH--->|          |          |          |
          |          |  [MD1]   |          |          |          |
          |          |          |--PUSH[MD2]--------->|          |
          |          |          |          |          |--PUSH--->|
          |          |          |          |          |<--ACK----|
          |          |          |<-----ACK[RMD2]------|          |
          |          |<--ACK----|          |          |          |
          |<--ACK----|  [RMD1]  |          |          |          |
          |          |          |          |          |          |
          |<==== Session Packets Flow without SVR Metadata =====>|
Figure 40

When router C fails, SVR Metadata MD1, MD2 can be included in the very next packet being sent in either direction. Confirmation that the move was completed is confirmed with reverse SVR Metadata RMD2, RMD1. For established TCP sessions, this is either a PUSH (as shown) or an ACK (Not shown). This can reestablish the SVR session state into a new router (Router B in this example) that previously did not have any involvement in the session. This technique can also be used to modify paths between two routers effectively moving TCP sessions from one transport (MPLS for example) to another (LTE). A session move can be initiated by any router at any time.

Ladder Diagram for Session Reroute Between Peers with SVR Metadata:


                 +-------+              +--------+
                 |       +-----MPLS-----+        |
         Client--| Rtr-A |              | Rtr-B  +----Server
                 |       +------LTE-----+        |
                 +-------+              +--------+

        Client . . . . . . . . . . . . . . . . . . . . . . Server
          |                                                   |
          |         RouterA                    RouterB        |
          |            |                          |           |
          |---PUSH---->|                          |           |
          |            |---PUSH over MPLS-------->|           |
          |            |                          |---PUSH--->|
          ................MPLS has Poor Quality ................
          |            |                          |           |
          |---PUSH---->|                          |           |
          |            |---PUSH over LTE[MD]----->|           |
          |            |                          |---PUSH--->|
          |            |                          |<---ACK----|
          |            |<---ACK over LTE[RMD]-----|           |
          |<---ACK-----|                          |           |
          |            |                          |           |
          |<=== Session Packets Flow without SVR Metadata ===>|
Figure 41

The diagram shows moving an active TCP session from one transport network to another by injecting SVR Metadata (MD) into any packet that is part of the transport in either direction. Reverse SVR Metadata is sent on any packet going in the reverse direction to confirm that the move was successful (RMD).

6.2. Moving Sessions that are Quiescent or One-Way Flows

Certain sessions may be idle or packets may create a one-way information flow (TCP Pushes) with one way acknowledgement (TCP ACKS). In these scenarios, insertion of SVR Metadata into existing packets may not be possible.

After moving a session, if an SVR router determines no packets are received or sent for an active session over an elapsed time of 1 second, the SVR router will generate an SVR Control Message to the peer.

Ladder Diagram for One Way Media Move with SVR Metadata:


                 +-------+              +--------+
                 |       +-----MPLS-----+        |
         Client--| Rtr-A |              | Rtr-B  +----Server
                 |       +------LTE-----+        |
                 +-------+              +--------+

        Client . . . . . . . . . . . . . . . . . . . . . . Server
          |                                                   |
          |         RouterA                    RouterB        |
          |            |                          |           |
          |            |                          |<---PUSH---|
          |            |<---PUSH over MPLS------->|           |
          |<---PUSH----|                          |           |
          |----ACK---->|                          |           |
          |            |------ACK over MPLS------>|           |
          |            |                          |---ACK---->|
          |            X RouterA MPLS FAILS       |           |
          |            X           RouterB MPLS OK|           |
          |            X                          |           |
          ..............RouterA Moves Session to LTE..........
          |            |                          |<---PUSH---|
          |            X<---PUSH over MPLS------->|           |
          |            |                          |<---PUSH---|
          |            X<---PUSH over MPLS------->|           |
          |            |                          |           |
          .......NO Packets at Router A for Moved Session......
          |            |                          |           |
          |            |-----[MD over LTE]------->|           |
          ...............RouterB Moves Session to LTE..........
          |            |                          |<---PUSH---|
          |            |<--PUSH over LTE [RMD]--->|           |
          |<---PUSH----|                          |           |
          |----ACK---->|                          |           |
          |            |------ACK over LTE------->|           |
          |            |                          |---ACK---->|
          |<======== Session Packets Continue flowing =======>|
Figure 42

The SVR Control Message uses the new SVR router interface addresses (Waypoints) and contains the standard first packet SVR Metadata fields with the SVR Control Message TLV added to the header with drop reason "FLOW MOVED". Also added is a TLV attribute with the remaining session time. This is essential to ensure mid-stream routers remove sessions from their tables roughly at the same time. This message will be transmitted once every second for 5 seconds OR reverse SVR Metadata has been received. If no reverse SVR Metadata has been received in 5 seconds the session is torn down. For a quiescent flow, the RMD is a generated SVR Control Message as well as shown below:

Ladder Diagram for Quiescent Moved Session with SVR Metadata:


                 +-------+              +--------+
                 |       +-----MPLS-----+        |
         Client--| Rtr-A |              | Rtr-B  +----Server
                 |       +------LTE-----+        |
                 +-------+              +--------+

        Client . . . . . . . . . . . . . . . . . . . . . . Server
          |                                                   |
          |         RouterA                    RouterB        |
          |            |                          |           |
          |<========== Quiescent Session Established ========>|
          |            |                          |           |
          |            X RouterA MPLS FAILS       |           |
          |            X           RouterB MPLS OK|           |
          |            X                          |           |
          ..............RouterA Moves Session to LTE..........
          |            |                          |           |
          |            |-----[MD over LTE]------->|           |
          |            |                          |           |
          ...............RouterB Moves Session to LTE..........
          |            |                          |           |
          |            |<-----[RMD over LTE]----->|           |
          |            |                          |           |
          |<=========== Quiescent Session Continues =========>|
Figure 43

6.3. NAT Keep Alive

If an SVR Router determines there is one or more NATs on a peer pathway (See Section 2.5, the SVR Peer must maintain the NAT bindings for each active session by sending keep alive SVR Metadata in the direction of the NAT. For keep alive, SVR utilizes a packet that matches the L4 header of the idle session that includes SVR Metadata type 24 with the drop reason set to Keep Alive.

Ladder Diagram for NAT Keep Alive with SVR Metadata:


                   RTR-A       NAT       RTR-B
        Client . . . . . . . . . . . . . . . . . . Server
          |          |          |          |          |
          ...................Existing SVR Session......
          |--PUSH--->|          |          |          |
          |          |--PUSH--->|          |          |
          |          |          |---PUSH-->|          |
          |          |          |          |--PUSH--->|
          |          |          |          |<---ACK---|
          |          |          |<---ACK---|          |
          |          |<--PUSH---|          |          |
          |<--PUSH---|          |          |          |
          .........NO PACKETS EITHER DIRECTION FOR 20 SECS........
          |          |          |          |          |
          |          |--[MD1]-->|          |          |
          |          |          |--[MD1]-->|          |
          |          |          |          |          |
          .........NO PACKETS EITHER DIRECTION FOR 20 SECS........
          |          |          |          |          |
          |          |--[MD1]-->|          |          |
          |          |          |--[MD1]-->|          |
          |          |          |          |          |
Figure 44

The SVR Metadata attributes that MUST be inserted in a keep alive for existing packet sessions includes:

With this minimum set of information, the receiver of this message can verify and update any modifications in a session NAT state. The Session UUID is used to verify all information positively.

6.4. Adaptive Encryption

Unlike a tunnel where all packets must be encrypted, each session in SVR is unique and independent. Most of the modern applications sessions are already using TLS or DTLS. SVR Routers have the capability of encrypting only sessions that are not already encrypted. Below is an example of adaptive encryption. With adaptive encryption, every session begins unencrypted. By analyzing the first 4 packets, the router can determine that encryption is required or not. If the fourth packet is a TLS Client hello message, encryption is NOT required. Any sequence of packets that does not indicate TLS or DTLS setup would immediately toggle encryption on.

Ladder Diagram of Adaptive Encryption with Client Hello:


        Client . . . . . . . . . . . . . . . . . . Server
          |                                           |
          +         RouterA            RouterB        |
          +---SYN----->|                  |           |
          |            |----SYN[MD1]----->|           |
          |            |                  |--SYN----->|
          |            |                  |<--SYN/ACK-|
          |            |<----SYN/ACK------|           |
          |<--SYN/ACK--|    [RMD1]        |           |
          |---ACK----->|                  |           |
          |            |------ACK-------->|           |
          |            |                  |--ACK----->|
          |--Client--->|                  |           |
          |  Hello     |<== ENCRYPTION===>|           |
          |            |   Not Required   |           |
          |            |                  |           |
          |            |-----Client------>|           |
          |            |      Hello       |--Client-->|
          |            |                  |           |
Figure 45

If the fourth packet is not an indication that encryption will be performed by the transport layer, then the ingress SVR Routers must encrypt and the egress SVR router must decrypt the session bidirectionally. This ensures that any data between the SVR Routers is encrypted.

Ladder Diagram of Adaptive Encryption with data:


        Client . . . . . . . .  . . . . . . . . Server
          |                                       |
          +         RouterA        RouterB        |
          +---SYN----->|              |           |
          |            |--SYN[MD1]--->|           |
          |            |              |--SYN----->|
          |            |              |<--SYN/ACK-|
          |            |<--SYN/ACK----|           |
          |<--SYN/ACK--|    [RMD1]    |           |
          |---ACK----->|              |           |
          |            |----ACK------>|           |
          |            |              |--ACK----->|
          |---Data---->|              |           |
          |            |<==ENCRYPT===>|           |
          |            |  Required    |           |
          |            |              |           |
          |            |--Encrypted-->|           |
          |            |   Data       |---Data--->|
Figure 46

Adaptive encryption is part of the security provisioning. Security policies are associated with services, and as such certain applications can mandate encryption; others may allow adaptive encryption, and still others may specify no encryption.

6.5. Packet Fragmentation

When a fragmented packet is presented to a SVR Router, the packet must be completely assembled to be processed. The SVR Router routes IP packets, and as all SVR actions require the entire packet. As such, the HMAC must be applied to the entire packet, and the entire packet must be routed as a whole. Each resulting fragment must be turned into an IP packet with 5-tuples that match the corresponding session to ingress and pass through an SVR. The SVR Router will simply use the same L4 header on all fragments from the session state table (peer pathway and transit ports). a time based HMAC signature is created for the entire packet and appended to the last fragment. Each fragment must also have SVR Metadata inserted that clearly identifies the fragment to the SVR routing peer.

Ladder Diagram Fragmented Packets:


     Client . . . . . . . . . . . . . . . . . . . . . . Server
       |                                                   |
       |         RouterA                    RouterB        |
       |            |                          |           |
       |--Frag 1--->|                          |           |
       |--Frag 3--->|                          |           |
       |--Frag 2--->|                          |           |
       |        +---+----+                     |           |
       |        |Assemble|                     |           |
       |        +---+----+                     |           |
       |            |----Frag 1[L4/MD]-------->|           |
       |            |                          |           |
       |            |----Frag 2[L4/MD]-------->|           |
       |            |                          |           |
       |            |----Frag 3[L4/MD]-------->|           |
       |            |                     +--------+       |
       |            |                     |Assemble|       |
       |            |                     +--------+       |
       |            |                          |--Frag 1-->|
       |            |                          |--Frag 2-->|
       |            |                          |--Frag 3-->|
Figure 47

In the diagram above, Router A collects all the fragments 1 2, and 3. Reassembly is performed. Router A records two things from the inbound fragments: The Original ID, and the largest fragment size received. Router A then proceeds to send the jumbo packet by fragmenting it again, but this time sending each piece inside a packet with a newly created L4 which maps exactly to the peer pathway chosen with ports assigned from the session state table. The fragment size will be the lesser of the smallest MTU on the path OR the largest fragment seen, whichever is smaller. The SVR Metadata header and header TLV's are not encrypted. The packet construction looks like this:

SVR Packet Layout

     Fragment 1
    +-----+-----+----------+----------+---------+
    |     |     |  SVR     |          |         |
    |Peer |Peer | Metadata | Header   | First   |
    |IP   |L4   | Header   | TLV-1,16 | Fragment|
    |HDR  |HDR  | 12 Bytes | 22 Bytes |         |
    +-----+-----+----------+----------+---------+

     Fragment 2
    +-----+-----+----------+----------+---------+
    |     |     |  SVR     |          |         |
    |Peer |Peer | Metadata | Header   | Second  |
    |IP   |L4   | Header   | TLV-1    | Fragment|
    |HDR  |HDR  | 12 Bytes | 14 Bytes |         |
    +-----+-----+----------+----------+---------+

     Fragment 3
    +-----+-----+----------+----------+---------+----------+
    |     |     |  SVR     |          |         |          |
    |Peer |Peer | Metadata | Header   | Third   | PKT      |
    |IP   |L4   | Header   | TLV-1    | Fragment| HMAC     |
    |HDR  |HDR  | 12 Bytes | 14 Bytes |         | SIGNATURE|
    +-----+-----+----------+----------+---------+----------+
Figure 48

The SVR Metadata type 1 inside the SVR fragment will have its own extended ID assigned. This allows a different number of fragments to be between router A and B than the Client and Server have. It also allows independent fragmentation by SVR should it be required. Router B will process the fragments from router A. Router B will look at its egress MTU size, and the largest fragment seen recorded by RouterA and transmitted in SVR Metadata to determine the proper size fragments to send, and the packet is fragmented and sent.

There are no other SVR Metadata fields required. All information about the session state is tied to the 5-tuple peer pathway and transports ports.

The details on packet fragmentation are identical to what is standardly performed in IP fragmentation, exception for the full L4 headers and SVR Metadata insertion.

If a packet traversing an SVR needs to be fragmented by the router for an SVR segment for any reason, including the insertion of SVR Metadata, the initiating router inserts SVR Metadata on the first packet and duplicates the L4 header (either TCP or UDP) on subsequent fragments and inserts SVR Metadata. In this case the Largest Fragment Seen and Original ID field in the SVR Metadata is left blank.

Ladder Diagram Fragmented Packets:


        Client . . . . . . . . . . . . . . . . . . . . . . Server
          |                                                   |
          |         RouterA                    RouterB        |
          |            |                          |           |
          |--Lg Pkt--->|                          |           |
          |            |--------Frag 1[MD]------->|           |
          |            |                          |           |
          |            |----Frag 2[L4 Hdr|MD]---->|           |
          |            |                          |--Lg Pkt-->|
          |            |                          |           |
Figure 49

6.6. ICMP and SVR

There are two types of ICMP messages. There are messages associated with specific packet delivery network errors. This includes:

  • Type 3: Destination Unreachable
  • Type 11: Time Exceeded (TTL)

These messages have information from the packet that generated the error by including the IP header + 8 bytes in the ICMP message (See [RFC0792]. It is important to deliver the ICMP message back to the origin. For these ICMP messages, the router MUST determine what active session the ICMP message belongs to by parsing the IP header information inside the ICMP message. Once a session is found, the ICMP message is transported across the SVR and reverse SVR Metadata is applied by having its destination address changed to the Waypoint Addresses of the routers.

SVR Metadata type 20 and 21 are used to send the source of the ICMP error backward through the networks. See Section 7.3.4 and Section 7.3.5 for information about these SVR Metadata formats. This repeats until the ICMP packet arrives at the initial SVR router. At this point the ICMP packet is recreated and the source address is changed to the address communicated through SVR Metadata type 20 and 21.

SVR Fragment Packet Layout

    +------------+------------+----------------+--------------+
    |            |            |    SVR         |              |
    |  IP HEADER | UDP HEADER | Metadata 20/21 | ICMP Packet  |
    +------------+------------+----------------+--------------+
Figure 50

ICMP over SVR for Network Failures


    Client . . . . . . . . . . . . . . . . . . . . . . .No Network
      |                                                  Found
      |         RouterA                    RouterB          |
      |            |                          |             |
      |----PKT---->|                          |             |
      |            |------PKT[MD]------------>|             |
      |            |                          |<--ICMP------|
      |            |                          |  (Router B) |
      |            |<--UDP[ICMP[RMD]]---------|             |
      |<--ICMP-----|                          |             |
      | (Client)   |                          |             |
      |            |                          |             |
Figure 51

The first ICMP message is directed to Router B. Router B examines the ICMP error to find the session, and forwards backwards to the correct Waypoint for Router A. Router A recreates the ICMP message, and sends to the Client. The address of where the error was detected is in

The second type of ICMP message is not related to any specific sessions. These types of messages include ICMP ECHO for example. These are always encapsulated as UDP, and a session is created for the ICMP message. The identifier field in ICMP and the IP addresses are used as the 5-tuple session key. This includes:

  • Type 8:ECHO Request (Ping)

ICMP over SVR for Information


        Client . . . . . . . . . . . . . . . . . . . . . . . Target
          |                                                     |
          |             RouterA             RouterB             |
          |                |                   |                |
          |--ICMP ECHO---->|                   |                |
          |                |---UDP[ICMP ECHO]->|                |
          |                |       [MD1]       |                |
          |                |                   |---ICMP ECHO--->|
          |                |                   |<--ECHO RESP----|
          |                |<--UDP[ECHO RESP]--|                |
          |                |       [RMD1]      |                |
          |<--ECHO RESP----|                   |                |
Figure 52

The ICMP message creates a session on Router A directed towards Router B. SVR Metadata MD1 is inserted just like any UDP session to establish the return pathway for the response. Reverse SVR Metadata is inserted into the ECHO Response, effectively creating an ICMP session. Subsequent identical ICMP messages will utilize this path without SVR Metadata being inserted. This session state MUST be guarded with an inactivity timer and the state deleted.

6.7. State Recovery Examples

It is exceedingly rare, but there are cases where session state can be lost. Well written applications generally self correct for any networking changes or interruptions. There are however applications with long lived nearly idle sessions (for example Session Initiation Protocol on idle handsets). In these situations recovering state is required.

Every SVR session has one or more SVR routers that have the full session state. Below is a set of techniques to reobtain the session state either from a peer or through regeneration and replacement.

The simplest scenario is when the Ingress SVR router loses state. In this scenario, it simple creates a new session for the old existing session but has the exact parameters of the original session. When this packet with first packet SVR Metadata reaches the egress SVR, the session state tables are updated, allowing two way end-to-end packet processing.

This is secured against attack because the first packet SVR Metadata is both signed and encrypted.

State Recovering Ingress Router with Active Session


      Client . . . . . . . . . . . . . . . . . . . . . . . Server
        |                                                     |
        |        Ingress      Middle        Egress            |
        |          SVR         BOX           SVR              |
        |           |           |             |               |
      <================Existing Bi-Directional Session=========>
        |           |           |             |               |
        |         State         |             |               |
        |         Lost          |             |               |
        |           |           |             |               |
        |---PKT---->|           |             |
        |         Create        |             |               |
        |          New          |             |               |
        |        Session        |             |               |
        |           |--PKT[MD]->|             |               |
        |           |           |--PKT[MD]--->|               |
        |           |           |           Update            |
        |           |           |          Existing           |
        |           |           |           Session           |
        |           |           |             |----PKT------->|
Figure 53

The next scenario is when the Ingress SVR loses session state, and the client application is idle. There is data from the server that can't be delivered. If a packet arrives from the server at the Egress SVR the length of time the client has been inactive is reviewed. If longer than the defined inactivity timer (provisioned, but defaults to 5 seconds), Session Health Check (see Section 7.3.8.) SVR Metadata will be inserted into the packet. The Ingress SVR responds by generating a packet (UDP) with the same L3 and L4 information as the session, and adds SVR Control Message to respond (see Section 7.3.6). If the Ingress SVR needs state for the session, it sets the drop reason in SVR Metadata to type=6, delete session. This causes the very next packet from the server to include first packet SVR Metadata. The session will be treated as a new session. This data is used to restore all aspects of the session.

State Recovering Ingress Router Client Inactive


      Client . . . . . . . . . . . . . . . . . . . . . . Server
        |                                                     |
        |        Ingress        Middle        Egress          |
        |          SVR           BOX           SVR            |
        |           |             |             |             |
      <================Existing Bi-Directional Session=======>
        |           |             |             |             |
        |         State           |             |             |
        |         Lost            |             |             |
        |           |             |             |<----PKT-----|
        |           |             |             |             |
        |           |             |       Client Inactivity   |
        |           |             |         Timer Exceeded    |
        |           |             |             |             |
        <---<--< SEND SESSION HEALTH CHECK METADATA <---<---<-
        |           |             |             |             |
        |           |             |<---PKT[MD]--|             |
        |           |<--PKT[MD]---|             |             |
        |       No State          |             |             |
        |           |             |             |             |
        >---->---> SEND SVR Control Metadata Drop=6 >--->---->-
        |           |             |             |             |
        |           |-GenPKT[MD]->|             |             |
        |           |             |-GenPKT[MD]->|             |
        |           |             |             |             |
        |           |             |         Clear State       |
        |           |             |             |             |
        |           |             |          Send First       |
        |           |             |       PKT SVR Metadata    |
        |           |             |          Next PKT         |
        |           |             |             |             |
        <--<- ON NEXT PACKET SEND FIRST PACKET METADATA <---<-
        |           |             |             |             |
        |           |             |             |<---PKT------|
        |           |             |<--PKT[MD]---|             |
        |           |<--PKT[MD]---|             |             |
        |           |             |             |             |
        |    New Session  State   |             |             |
        |           |             |             |             |
        =======Treat as a new session from this point =========
Figure 54

If an Egress router loses state for a session, it must reobtain the state from a peer. In the example shown below the Ingress SVR detects upon receipt of a packet that the server has not responded for more than the inactivity timer. The packet that arrived is then augmented with Session Health Check SVR Metadata (see Section 7.3.8). If the egress router MUST reply to the session health check by generating a UDP packet with SVR Control Message SVR Metadata (see Section 7.3.6). If it requires state, it must set the drop reason to a type=2, indicating SVR Metadata needs to be sent. The next packet from the client will include all of the first packet SVR Metadata which is used to restore the mission state information.

State Recovering Egress Router


      Client . . . . . . . . . . . . . . . . . . . . . . Server
        |                                                     |
        |        Ingress        Middle        Egress          |
        |          SVR           BOX           SVR            |
        |           |             |             |             |
      <================Existing Bi-Directional Session=======>
        |           |             |             |             |
        |           |             |           State           |
        |           |             |            Lost           |
        |           |             |             |             |
        |--PKT----->|             |             |<---PKT------|
        |           |----PKT----->|             |             |
        |           |             |----PKT----->|             |
        |           |             |             |             |
        |           |             |          Pkts Dropped     |
        |--PKT----->|             |             |             |
        |       Inactivity        |             |             |
        |       Exceeded          |             |             |
        |           |             |             |             |
        >--->---> SEND SESSION HEALTH CHECK METADATA >--->---->
        |           |             |             |             |
        |           |---PKT[MD]-->|             |             |
        |           |             |--PKT[MD]--->|             |
        |           |             |          No State         |
        |           |             |             |             |
        <----<---- SEND SVR CONTROL MESSAGE TYPE=2 <----<----<-
        |           |             |             |             |
        |           |             |<-GenPKT[MD]-|             |
        |           |<-GenPKT[MD]-|             |             |
        |--PKT----->|             |             |             |
        |           |             |             |             |
        -->---> SEND FIRST PACKET METADATA IN NEXT PACKET ---->
        |           |             |             |             |
        |           |---PKT[MD]-->|             |             |
        |           |             |--PKT[MD]--->|             |
        |           |             |          Session          |
        |           |             |        State Restored     |
        |           |             |             |---PKT------>|
        |           |             |             |             |
Figure 55

The most likely loss of state occurs in middle boxes. Often the middle box will either stop routing packets in one direction, both directions, or modify the UDP or TCP ports without notice.

In this instance, we do not know for certain where the state was lost, so we attempt to recover it from our SVR peer by including Session Health Check SVR Metadata in a packet of the session. SVR Peers must respond to this packet, so no response indicates there is a middle box or network problem.

To restore the session, the session state is cleared and the next packet is treated as a first packet. A full SVR Metadata exchange between peers is completed as documented in Section 3.7.1. Both Ingress and Egress SVRs can detect that there is an existing session with the exact same addresses and ports and simply replace the session state.

State Recovering of Middlebox


      Client . . . . . . . . . . . . . . . . . . . . . . Server
        |                                                     |
        |        Ingress        Middle        Egress          |
        |          SVR           BOX           SVR            |
        |           |             |             |             |
      <================Existing Bi-Directional Session=======>
        |           |             |             |             |
        |           |          State            |             |
        |           |           Lost            |             |
        |           |             |             |             |
        |--PKT----->|             |             |<---PKT------|
        |           |----PKT----->|             |             |
        |           |             |<---PKT------|             |
        |           |          Packets          |             |
        |           |          Dropped          |             |
        |        Inactivty        |             |             |
        |        Exceeded         |             |             |
        |           |             |             |             |
        |---PKT---->|             |             |             |
        |           |             |             |             |
        |           |             |             |             |
        |           |---PKT[MD]-->|             |             |
        |           |             |             |             |
        |      No Response        |             |             |
        |           |             |             |             |
        |      Re Allocate Ports  |             |             |
        |   Update Session State  |             |             |
        |           |             |             |             |
        |--PKT----->|             |             |             |
        |           |             |             |             |
        ---> SEND FIRST PACKET METADATA, KEEP OLD SESSIONID--->
        |           |             |             |             |
        |           |--PKT[MD]--->|             |             |
        |           |             |--PKT[MD]--->|             |
        |           |             |          Update           |
        |           |             |          Session          |
        |           |             |             |---PKT------>|
        |           |             |             |             |
Figure 56

7. SVR Metadata Format and Composition

The format of SVR Metadata has both Header attributes as well as Payload attributes. Header attributes are always guaranteed to be unencrypted. These headers may be inspected by intermediate network elements but can't be changed. Header attributes do not have a forward or reverse direction. Header attributes are used for router and peer pathway controls.

Payload attributes optionally can be encrypted by the sender. Payload attributes are associated with sessions, and as such have a forward and reverse direction. For encryption, the pre-existing security association and key sharing is outside the scope of this document. Each SVR attribute defined will indicate whether it is a header attribute (unencrypted) or payload attribute (optionally encrypted). There are no attributes that can exist in both sections.

7.1. SVR Metadata Header

The SVR Metadata header is shown below. A well-known "cookie" (0x4c48dbc6ddf6670c in network byte order byte order) is built into the header which is used in concert with contextual awareness of the packet itself to determine the presence of SVR Metadata within a packet. This is an eight-byte pattern that immediately follows the L4 header and is an indicator to a receiving router that a packet contains SVR Metadata. NOTE: Normal IP traffic will never have the Waypoint Address as its destination. If a packet arrives at a SVR Router Waypoint it has to have SVR Metadata or be associated with an active SVR session. Please see Section 2.11 for a discussion of state recovery techniques.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                                                               |
  +                             Cookie                            +
  |                                                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |Version|   Header Length       |         Payload Length        |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |     Header TLVs ...           |       Payload TLVs ...        |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 57
Cookie (8 bytes):
The fingerprint of SVR Metadata. This value is used to determine the existence of SVR Metadata within a packet.
Version (4-bits):
Field representing the version of the SVR Metadata header. The current version of SVR Metadata is 0x1.
Header Length (12-bits):
Length of the SVR Metadata header including any added Header TLV attributes that are guaranteed to be unencrypted. When there are no Header TLVs, the value Header Length is 12 Bytes or OxC.
Payload Length (2 bytes):
Length of data following the SVR Metadata header, not including the size of the header. This data could be encrypted. The value of this field is the number of bytes in the Payload TLV's. If there are no TLV's the value is zero.

7.1.1. False Positives

Given that no byte sequence is truly unique in the payload of a packet, in the scenario where the original payload after the L4 header contained the same byte sequence as the cookie, false positive logic is enacted on the packet. If the SVR Metadata HMAC signature can't verify that the SVR Metadata is valid, then a false positive SVR Metadata header is added to the packet to indicate that the first eight bytes of the payload matches the cookie.

The structure of a false positive SVR Metadata includes just a header of length 12 bytes, with zero header TLVs and zero payload TLVs. The receiving side of a packet with false positive SVR Metadata will strip out the SVR Metadata header.

In the scenario where a router receives a false positive SVR Metadata header but intends to add SVR Metadata to the packet, the false positive SVR Metadata header is modified to contain the newly added attributes. Once attributes are added, the SVR Metadata header is no longer considered to be false positive.

7.1.2. Forward and Reverse Attributes

Payload SVR Metadata attributes may be valid in the forward direction, the reverse direction, or both. If not valid, it is ignored quietly by the receiving side.

7.2. TLVs for Attributes

All SVR Metadata attributes are expressed as Tag Length Values or TLV's. This includes Header and Payload TLVs. It is recommended that Payload TLVs be encrypted, but not mandatory. When debugging networks, or if mid-stream routers need to consult the TLV's, they can be transmitted in clear text. The entire SVR Metadata block is signed, and thus the integrity of the data can be verified. No midstream router or middlebox can modify any aspect of the SVR Metadata. Doing so will invalidate the signature, and the SVR Metadata will be dropped.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |            Type               |           Length              |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |          Variable Length Values .....                         |
  \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
Figure 58
Type (2 bytes):
Type of data that follows. Each of different Header and Payload TLV's are defined below.
Length (2 bytes):
Number of bytes associated with the length of the value (not including the 4 bytes associated with the type and length fields).

7.3. Header Attributes

7.3.1. Fragment

When a packet is fragmented to insert SVR Metadata, a new fragmentation mechanism must be added to prevent fragmentation attacks and to support reassembly (which requires protocol and port information). If a packet is received that IS a fragment, and it must transit through a SVR Metadata signaled pathway, it must also have this SVR Metadata attached to properly bind the fragment with the correct session.

All fragments will have a SVR Metadata header and the fragment TLV added to the guaranteed unencrypted portion of the SVR Metadata header. If the original packet already has a SVR Metadata header on it, the fragment TLV will be added to it. See [RFC0791] for information about IP Fragmentation. For a detailed example of packet fragmentation in SVR please see Section 6.5

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |            Type = 1           |           Length = 10         |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                         Extended ID                           |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |       Original ID             |Flags|    Fragment Offset      |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |    Largest Seen Fragment      |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 59
TLV:
Type 1, Length 10.
Extended ID (4 bytes):
Uniquely identifies a packet that is broken into fragments This ID is assigned by the SVR that is processing fragmented packets. IPv6 uses a 32-bit Extended ID, and IPv4 uses a 16-bit ID. We use the same algorithm for fragmenting packets for both IPv6 and IPv4, therefore we chose a 32-Bit Extended ID. .
Original ID (2 bytes):
Original identification value of the L3 header of a received packet that is already fragmented.
Flags (3-bits):

Field used for identifying fragment attributes. They are (in order, from most significant to least significant):

  • bit 0: Reserved; must be zero.
  • bit 1: Don't fragment (DF).
  • bit 2: More fragments (MF).
Fragment Offset (13-bits):
Field associated with the number of eight-byte segments the fragment payload contains.
Largest Seen Fragment (2 bytes):
Each SVR router keeps track of the largest fragment processed from each interface. This allows the router to make inferences about the MTU size when fragmenting packets in the opposite direction. This information is used along with a given egress network interface MTU to determine the fragment size of a reassembled packet.

7.3.2. Security ID

A versioning identifier used to determine which security key version should be used when handling features dealing with security and authenticity of a packet.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |           Type = 16           |            Length = 4         |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                        Security Key Version                   |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 60
TLV:
Type 16, Length 4.
Security Key Version (4 bytes):
This is a four-byte security key version identifier. This is used to identify the algorithmic version used for SVR Metadata authentication and encryption.

7.3.3. Disable Forward SVR Metadata

An indication that forward SVR Metadata should be disabled. This is sent in the reverse SVR Metadata to acknowledge receipt of the SVR Metadata. This is the second part of the SVR Metadata handshake.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |           Type = 18           |         Length = 0            |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 61
TLV:
Type 18, Length 0.

No other data is required. The specific session that is being referred to is looked up based on the 5-tuple address of the packet. See SVR Metadata handshake in Section 2.4.

7.3.4. IPv4 ICMP Error Location Address

This is exclusively used to implement ICMP messages that need to travel backwards through SVR pathways. See Section 6.6 for more information. The IPv4 address of the source of the ICMP message is placed into SVR Metadata. This SVR Metadata travels in the reverse direction backwards to the originating SVR, which restores the information and sends an ICMP message to the originator of the packet.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |           Type = 20           |          Length = 4           |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                        Source Address                         |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 62
TLV:
Type 20, Length 4.
Source Address (4 bytes):
Original IPv4 source address of the originating router.

7.3.5. IPv6 ICMP Error Location Address

This is exclusively used to implement ICMP messages that need to travel backwards through SVR pathways. See Section 6.6 for more information. The IPv6 address of the source of the ICMP message is placed into SVR Metadata. This SVR Metadata travels in the reverse direction backwards to the originating SVR, which restores the information and sends an ICMP message to the originator of the packet.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |           Type = 21           |          Length = 16          |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                                                               |
  +                                                               +
  |                                                               |
  +                        Source Address                         +
  |                                                               |
  +                                                               +
  |                                                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 63
TLV:
Type 21, Length 16.
Source Address (16 bytes):
Original IPv6 source address of the originating router.

7.3.6. SVR Control Message

The SVR Control Message is used for protocol specific purposes that are limited to a single peer pathway. This message is sent by an SVR router to a peer. This SVR Metadata is always sent in a UDP message originating by the SVR control plane.

Keep Alive:
When an SVR peer is behind a NAT device and the SVR peer has active sessions, the SVR peer will generate a "Keep Alive" often enough (i.e., 20 seconds) to prevent the firewall from closing a pinhole. This message is generated completely by the SVR router, and directed to the SVR peer for a session. The UDP address and ports fields must exactly match the session that has been idle longer than the provisioned time.
Turn On SVR Metadata:
When a packet is received, and there is missing SVR Session State, the correction procedure may involve sending this request to a peer SVR router that has the information. Please see Section 2.11 for more information.
Turn Off SVR Metadata:
Disable SVR Metadata on a specific 5-tuple. In certain cases, the SVR peer may continue so send SVR Metadata because there are no reverse flow packets or because SVR Metadata was enabled to recover from a loss of state. This message is not part of the normal SVR Metadata handshake and only has a scope of a single peer pathway.
Delete Session:
The session associated with the flow spec used by this generated packet should be deleted. This provides an administrative and error correcting capability to remove a session when required.
Session State Exists:
In response to a Session Health Check request (see Section 7.3.8 to indicate that state for a session exists. 
   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |          Type = 24            |           Length = 1          |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |  Drop Reason  |
  +-+-+-+-+-+-+-+-+
Figure 64
TLV:
Type 24, Length 1.
Drop Reason (1 byte):

Reason why this packet should be dropped.

  • 0 = Unknown. This value is reserved and used for backwards compatibility.
  • 1 = Keep Alive. A packet that is dropped by the receiving node. Used only to keep NAT pinholes alive on middleboxes.
  • 2 = Enable SVR Metadata. Begin sending SVR Metadata on the peer pathway for the 5-tuple matching this control packet.
  • 3 = Disable SVR Metadata. Stop sending SVR Metadata on the peer pathway for a 5-tuple matching this control packet.
  • 6 = Delete Session. Delete any state for the session associated with this SVR Metadata.
  • 8 = Session Health Check indicates state exists, and is valid.

7.3.7. Path Metrics

This SVR Metadata type is used to allows peers to measure and compute inline flow metrics for a specific peer pathway and a chosen subset of traffic. class. The flow metrics can include jitter, latency and packet loss. This is an optional SVR Metadata type.

When a peer sends this SVR Metadata, it provides the information for the period of time to the peer.

When a peer receives this SVR Metadata type 26, it responds with SVR Metadata type 26.

After several exchanges, each side can compute accurate path metrics for the traffic included. This SVR Metadata can be sent at any time, but is normally sent when SVR Metadata is being sent for other reasons. The SVR Metadata includes "colors" which represent blocks of packets. Packet loss and latency can be determined between routers using this in line method. Using colors to measure inline flow performance is outside the scope of this document. Please refer to [RFC9341]

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |           Type = 26           |           Length = 10         |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  | Tx Co |                Transmit TimeValue                     |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  | Rx Co |                Received TimeValue                     |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |D|   Previous Rx Color Count   |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 65
TLV:
Type 26, Length 10.
Transmit Color (4-bits):
Current color of a transmitting node.
Transmit Time Value (28-bits):
Current time value in milliseconds at time of marking. This time value represents the amount of time which has elapsed since the start of a transmit color.
Received Color (4-bits):
Most recently received color from a remote node. This represents the color last received from a specific peer.
Receive Time Value (28-bits):
Cached time value in milliseconds from adjacent node adjusted by the elapsed time between caching of the value and current time. This time value is associated with the received color.
Drop Bit (1-bit):
Should this packet be dropped. This is required if a packet is being sent solely to measure quality on an otherwise idle link.
Previous Rx Color Count (15-bits):
Number of packets received from the previous color block. This count is in reference to the color previous to the current RX color which is defined above.

7.3.8. Session Health Check

This SVR Metadata is used to request a session state check by a peer. The peer responds upon receipt with a generated packet with SVR Metadata confirming presense of SVR Metadata. This SVR Metadata type can be included in an existing packet to check that the peer has session state. The peer will always respond with a generated packet that includes a forced drop SVR Metadata attribute. See Section 6.7 for an example.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |            Type = 46          |           Length = 1          |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |   Type = 1    |
  +-+-+-+-+-+-+-+-+
Figure 66
TLV:
Type 46, Length 1.
TYPE: (1=Request, 2=Request/Timeout)
Request to verify session state with backward SVR Metadata. Type 1 indicates session state is available, Type 2 indicates session state is available but will be cleared and replaced upon receipt of state from a peer. Type 2 is used when a middle box closes pinholes that must be recovered.

7.4. Payload Attributes

Payload attributes are used for session establishment and SHOULD be encrypted to provide privacy. Encryption can be disabled for debugging.

7.4.1. Forward Context IPv4

The context contains a five-tuple associated with the original addresses, ports, and protocol of the packet. This is also known as the Forward Session Key.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |             Type = 2          |           Length = 13         |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                        Source Address                         |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                     Destination Address                       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |         Source Port           |      Destination Port         |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |   Protocol    |
  +-+-+-+-+-+-+-+-+
Figure 67
TLV:
Type 2, Length 13.
Source Address (4 bytes):
Original IPv4 source address of the packet.
Destination Address (4 bytes):
Original IPv4 destination address of the packet.
Source Port (2 bytes):
Original source port of the packet.
Destination Port (2 bytes):
Original destination port of the packet.
Protocol (1 byte):
Original protocol of the packet.

7.4.2. Forward Context IPv6

A five-tuple associated with the original addresses, ports, and protocol of the packet for IPv6.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |           Type = 3            |          Length = 37          |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                                                               |
  +                                                               +
  |                                                               |
  +                         Source Address                        +
  |                                                               |
  +                                                               +
  |                                                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                                                               |
  +                                                               +
  |                                                               |
  +                       Destination Address                     +
  |                                                               |
  +                                                               +
  |                                                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |          Source Port          |        Destination Port       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |   Protocol    |
  +-+-+-+-+-+-+-+-+
Figure 68
TLV:
Type 3, Length 37.
Source Address (16 bytes):
Original IPv6 source address of the packet.
Destination Address (16 bytes):
Original IPv6 destination address of the packet.
Source Port (2 bytes):
Original source port of the packet.
Destination Port (2 bytes):
Original destination port of the packet.
Protocol (1 byte):
Original protocol of the packet.

7.4.3. Reverse Context IPv4

Five-tuple associated with the egress (router) addresses, ports, and protocol of the packet. Forward context and reverse context session keys are not guaranteed to be symmetrical due to functions which apply source NAT, destination NAT, or both to a packet before leaving the router.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |             Type = 4          |           Length = 13         |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                        Source Address                         |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                     Destination Address                       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |         Source Port           |      Destination Port         |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |   Protocol    |
  +-+-+-+-+-+-+-+-+
Figure 69
TLV:
Type 4, Length 13.
Source Address (4 bytes):
Egress IPv4 source address of the packet.
Destination Address (4 bytes):
Egress IPv4 destination address of the packet.
Source Port (2 bytes):
Egress source port of the packet.
Destination Port (2 bytes):
Egress destination port of the packet.
Protocol (1 byte):
Original protocol of the packet.

7.4.4. Reverse Context IPv6

Five-tuple associated with the egress (router) addresses, ports, and protocol of the packet. Forward and reverse session keys are not guaranteed to be symmetrical due to functions which apply source NAT, destination NAT, or both to a packet before leaving the router.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |           Type = 5            |          Length = 37          |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                                                               |
  +                                                               +
  |                                                               |
  +                         Source Address                        +
  |                                                               |
  +                                                               +
  |                                                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                                                               |
  +                                                               +
  |                                                               |
  +                       Destination Address                     +
  |                                                               |
  +                                                               +
  |                                                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |          Source Port          |        Destination Port       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |   Protocol    |
  +-+-+-+-+-+-+-+-+
Figure 70
TLV:
Type 5, Length 37.
Source Address (16 bytes):
Egress IPv6 source address of the packet.
Destination Address (16 bytes):
Egress IPv6 destination address of the packet.
Source Port (2 bytes):
Egress source port of the packet.
Destination Port (2 bytes):
Egress destination port of the packet.
Protocol (1 byte):
Original protocol of the packet.

7.4.5. Session UUID

Unique identifier of a session. The UUID MUST be conformant to [RFC9562]This is assigned by the peer that is initiating a session. Once assigned, it is maintained through all participating routers end-to-end.

The UUID is used to track sessions across multiple routers. The UUID also can be used to detect a looping session. The UUID SVR Metadata field is required for all session establishment.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |            Type = 6           |           Length = 16         |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                                                               |
  +                                                               +
  |                                                               |
  +                              UUID                             +
  |                                                               |
  +                                                               +
  |                                                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 71
TLV:
Type 6, Length 16.
UUID (16 bytes):
Unique identifier of a session.

7.4.6. Tenant Name

An alphanumeric ASCII string which dictates what tenancy the session belongs to.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |            Type = 7           |       Length = variable       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                 Name (1 - n bytes) ....                       |
  \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
Figure 72
TLV:
Type 7, Length variable.
Name (variable length):
The tenant name represented as a string.

7.4.7. Service Name

An alphanumeric string which dictates what service the session belongs to.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |            Type = 10          |       Length = variable       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                  Service Name (1-n bytes) .....               |
  \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
Figure 73
TLV:
Type 10, Length variable.
Name (variable length):
The service name represented as a string.

7.4.8. Session Encrypted

Indicates if the session is having its payload encrypted by the SVR router. This is different from having the SVR Metadata encrypted. The keys used for payload encryption are dependent on the Security Policy defined for a session.

This field is necessary because often traffic is already encrypted before arriving at an SVR router (making DPI a poor choice). Also in certain use cases, re-encryption may be required. This SVR Metadata TLV is always added when SVR encrypts the payload.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |            Type = 11          |           Length = 0          |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 74
TLV:
Type 11, Length 0.

7.4.9. TCP SYN Packet

Indicates if the session is being converted from TCP to UDP to enable passing through middle boxes that are TCP session stateful. A SVR implementation must verify that SVR Metadata can be sent inside TCP packets through testing the Peer Pathway. If the data is blocked, then all TCP sessions must be converted to UDP sessions, and restored on the destination peer.

Although this may seem redundant with the Forward Context that also has the same originating protocol, this refers to a specific peer pathway. In a multi-hop network, the TCP conversion to UDP could occur at the second hop. It's important to restore the TCP session as soon as possible after passing through the obstructive middlebox.

When TCP to UDP conversion occurs, no bytes are changed other than the protocol value (TCP->UDP). Because the UDP message length and checksum sit directly on top of the TCP Sequence Number, the Sequence number is overwritten. The Sequence number is saved by copying it to the TCP Checksum. The Checksum is recalculated upon restoration of the packet. The packet integrity against bit loss or malicious activity is provided through the HMAC signature.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |            Type = 12          |           Length = 0          |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 75
TLV:
Type 12, Length 0.

Note: This type does not contain any value as its existence in SVR Metadata indicates a value.

7.4.10. Source Router Name

An alphanumeric string which dictates which source router the packet is originating from. This attribute may be present in all forward SVR Metadata packets if needed.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |            Type = 14          |       Length = variable       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |               Router Name (1-n bytes) ....                    |
  \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
Figure 76
TLV:
Type 14, Length variable.
Name (variable length):
The router name represented as a string.

7.4.11. Security Policy

An alphanumeric string containing the Security Policy to use for a particular session. This is used only when payload encryption is being performed. The Security Policy describes the specifics about Ciphers used for payload encryption.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |            Type = 15          |       Length = variable       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                        SECURITY POLICY                        |
  \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
Figure 77
TLV:
Type 15, Length variable.
KEY (variable length):
The session key to use for encryption/decryption for this packet and future packets in a session.

7.4.12. Peer Pathway ID

An ASCII string which dictates which router peer pathway has been chosen for a packet. This name is the hostname or IP address of the egress interface of the originating router. This can be used to determine the peer pathway used exactly when there may be multiple possibilities. This enables association of policies with specific paths.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |            Type = 19          |       Length = variable       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |            Peer Pathway Name (1-n bytes) ....                 |
  \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
Figure 78
TLV:
Type 19, Length variable.
Name (variable length):
The peer pathway name which is represented as a string.

7.4.13. IPv4 Source NAT Address

Routers may be provisioned to perform source NAT functions while routing packets. When a source NAT is performed by an SVR Peer, this SVR Metadata TLV MUST be included. When the far end router reconstructs the packet, it will use this address as the source address for packets exiting the SVR.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |            Type = 25          |           Length = 4          |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                     IPv4 Source Nat Address                   |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 79
TLV:
Type 25, Length 4.
Source Address (4 bytes):
Source NAT address of the packet.

7.4.14. Remaining Session Time

After a path failure, it may become necessary to transmit a SVR Control Message when there are one-way flows waiting for a packet to be transmitted. In these cases, the SVR Metadata includes an attribute defining the remaining session time so intermediate routers creating new session entries will expire the session at the correct time.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |            Type = 42          |           Length = 4          |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                     Remaining Session Time (seconds)          |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 80
TLV:
Type 42, Length 4.
Remaining Session Time (4 bytes):
Number of seconds remaining on a session packet guard time. This ensures accurate guarding of sessions that have been moved.

7.4.15. Security Encryption Key

An alphanumeric string containing the cryptographic key to use for a payload encryption of a particular session. This is used only when payload encryption is being performed. The key is encrypted in SVR Metadata hop-by-hop through a network, preventing any party from obtaining the key. The router terminating the session utilizes this key to decrypt payload portions of packets. This prevents re-encryption penalties associated with multi-hop routing scenarios.

To rekey a session, this SVR Metadata can be included in any subsequent packet with the new key to use. When rekeying, the SVR that initiated the encrypted session must compute a new key, and include the key as SVR Metadata.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |            Type = 46          |       Length = variable       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                        SECURITY KEY                           |
  \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
Figure 81
TLV:
Type 46, Length variable.
KEY (variable length):
The session key to use for encryption/decryption for this packet and future packets in a session.

8. Security Considerations

8.1. HMAC Authentication

HMAC signatures are REQUIRED for the packets that contain SVR Metadata to guarantee the contents were not changed, and that the router sending it is known to the receiver. Any HMAC algorithm can be used, from SHA128, or SHA256 as long as both sides agree. HMAC is always performed on the layer 4 payload of the packet. The signature is placed at the end of the existing packet.

8.2. Replay Prevention

Optional HMAC signatures are RECOMMENDED for every packet. This prevents any mid-stream attempts to corrupt or impact sessions that are ongoing. This also helps detect and correct lost state at egress SVR routers. See Section 2.11. The signature must include all of the packet after Layer 4, and include a current time of day to prevent replay attacks. The signature is placed at the end of the existing packet.

Both the sending and receiving routers must agree on these optional HMAC signatures, details of which are outside the scope of this document.

8.3. Payload Encryption

Payload encryption can use AES-CBC-128 or AES-CBC-256 ciphers which can be configured. Since these are block-ciphers, the payload should be divisible by 16. If the actual payload length is divisible by 16, then the last 16 bytes will be all 0s. If the actual payload is not divisible by 16, then the remaining data will be padded and the last byte will indicate the length.

8.4. DDoS and Unexpected Traffic on Waypoint Addresses

Waypoint addresses could be addressed by any client at any time. IP packets that arrive on the router's interface will be processed with the assumption that they MUST contain SVR Metadata OR be part of an existing established routing protocol.

Routers will only accept SVR Metadata from routers that they are provisioned to speak with. As such an ACL on incoming source addresses is limited to routers provisioned to communicate. All other packets are dropped.

When a packet is received the "cookie" in the SVR Metadata header is reviewed first. If the cookie isn't correct, the packet is dropped.

The HMAC signature is checked. If the signature validates, the packet is assumed to be good, and processing continues. If the HMAC fails, the packet is dropped.

These methods prevent distributed denial of service attacks on the Waypoint Addresses of routers.

9. IANA Considerations

This document does not require any IANA involvement.

10. Acknowledgements

The authors would like to thank Anya Yungelson, Scott McCulley, and Chao Zhao for their input into this document.

The authors would like to thank Tony Li for his extensive support and help with all aspects of this document.

The authors want to thank Ron Bonica, Kireeti Kompella, and other IETFers at Juniper Networks for their support and guidance.

11. Normative References

[ECDH_Key_Exchange]
Nakov, S., "Practical Cryptography for Developers", ISBN 978-619-00-0870-5, Publisher Sofia, , <https://cryptobook.nakov.com/asymmetric-key-ciphers/ecdh-key-exchange>.
[NIST_SP_800-56A]
Barker, E., Chen, L., Roginsky, A., Vassilev, A., and R. Davis, "Recommendation for Pair-Wise Key-Establishment Schemes Using Discrete Logarithm Cryptography", ISBN NIST Special Publication 800-56A Rev3, Publisher National Security Agency, , <https://csrc.nist.gov/pubs/sp/800/56/a/r3/final>.
[NIST_SP_800-90B]
Turan, M., Barker, E., Kelsey, J., McKay, K., Baish, M., and M. Boyle, "Recommendation for the Entropy Sources Used for Random Bit Generation", ISBN NIST Special Publication 800-90B, Publisher National Institute of Standards and Technology, , <https://csrc.nist.gov/pubs/sp/800/90/b/final>.
[RFC0791]
Postel, J., "Internet Protocol", STD 5, RFC 791, DOI 10.17487/RFC0791, , <https://www.rfc-editor.org/info/rfc791>.
[RFC0792]
Postel, J., "Internet Protocol", STD 5, RFC 792, DOI 10.17487/RFC0792, , <https://www.rfc-editor.org/info/rfc792>.
[RFC2104]
Krawczyk, H., Bellare, M., and R. Canetti, "HMAC: Keyed-Hashing for Message Authentication", RFC 2104, DOI 10.17487/RFC2104, , <https://www.rfc-editor.org/info/rfc2104>.
[RFC2119]
Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <https://www.rfc-editor.org/info/rfc2119>.
[RFC9562]
Leach, P., Mealling, M., and R. Salz, "A Universally Unique IDentifier (UUID) URN Namespace", RFC 9562, DOI 10.17487/RFC9562, , <https://www.rfc-editor.org/info/rfc9562>.
[RFC4210]
Adams, C., Farrell, S., Kause, T., and T. Mononen, "Internet X.509 Public Key Infrastructure Certificate Management Protocol (CMP)", RFC 4210, DOI 10.17487/RFC4210, , <https://www.rfc-editor.org/info/rfc4210>.
[RFC5880]
Katz, D. and D. Ward, "Bidirectional Forwarding Detection (BFD)", RFC 5880, DOI 10.17487/RFC5880, , <https://www.rfc-editor.org/info/rfc5880>.
[RFC5758]
Dang, Q., Santesson, S., Moriarty, K., Brown, D., and T. Polk, "Internet X.509 Public Key Infrastructure: Additional Algorithms and Identifiers for DSA and ECDSA", RFC 5758, DOI 10.17487/RFC5758, , <https://www.rfc-editor.org/info/rfc5758>.
[RFC5905]
Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, "Network Time Protocol Version 4: Protocol and Algorithms Specification", RFC 5905, DOI 10.17487/RFC5905, , <https://www.rfc-editor.org/info/rfc5905>.
[RFC6062]
Perreault, S., Ed. and J. Rosenberg, "Traversal Using Relays around NAT (TURN) Extensions for TCP Allocations", RFC 6062, DOI 10.17487/RFC6062, , <https://www.rfc-editor.org/info/rfc6062>.
[RFC9300]
Farinacci, D., Fuller, V., Meyer, D., and D. Lewis, "The Locator/ID Separation Protocol (LISP)", RFC 9300, DOI 10.17487/RFC9300, , <https://www.rfc-editor.org/info/rfc9300>.
[RFC7468]
Josefsson, S. and S. Leonard, "Textual Encodings of PKIX, PKCS, and CMS Structures", RFC 7468, DOI 10.17487/RFC7468, , <https://www.rfc-editor.org/info/rfc7468>.
[RFC8174]
Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <https://www.rfc-editor.org/info/rfc8174>.
[RFC9341]
Fioccola, G., Ed., Capello, A., Cociglio, M., Castaldelli, L., Chen, M., Zheng, L., Mirsky, G., and T. Mizrahi, "Alternate-Marking Method for Passive and Hybrid Performance Monitoring", RFC 9341, DOI 10.17487/RFC9341, , <https://www.rfc-editor.org/info/rfc9341>.
[RFC8422]
Nir, Y., Josefsson, S., and M. Pegourie-Gonnard, "Elliptic Curve Cryptography (ECC) Cipher Suites for Transport Layer Security (TLS) Versions 1.2 and Earlier", RFC 8422, DOI 10.17487/RFC8422, , <https://www.rfc-editor.org/info/rfc8422>.
[RFC8445]
Keranen, A., Holmberg, C., and J. Rosenberg, "Interactive Connectivity Establishment (ICE): A Protocol for Network Address Translator (NAT) Traversal", RFC 8445, DOI 10.17487/RFC8445, , <https://www.rfc-editor.org/info/rfc8445>.
[RFC8489]
Petit-Huguenin, M., Salgueiro, G., Rosenberg, J., Wing, D., Mahy, R., and P. Matthews, "Session Traversal Utilities for NAT (STUN)", RFC 8489, DOI 10.17487/RFC8489, , <https://www.rfc-editor.org/info/rfc8489>.
[RFC8986]
Filsfils, C., Ed., Camarillo, P., Ed., Leddy, J., Voyer, D., Matsushima, S., and Z. Li, "Segment Routing over IPv6 (SRv6) Network Programming", RFC 8986, DOI 10.17487/RFC8986, , <https://www.rfc-editor.org/info/rfc8986>.

Authors' Addresses

Abilash Menon
Maia Tech
100 Summit Drive
Burlington, MA 01803
United States of America
Patrick MeLampy
Retired
1024 Main St.
Dustable, MA 01827
United States of America
Michael Baj
Juniper Networks
10 Technology Part Dr.
Westford, MA 01886
United States of America
Patrick Timmons
Maia Tech
100 Summit Drive
Burlington, MA 01803
United States of America
Hadriel Kaplan
Juniper Networks
10 Technology Park Dr.
Westford, MA 01886
United States of America