Rapid.space

86 avenue de la République Paris 75011 France titouan.soulard@rapid.space

Operations and Management Area Autonomic Networking Integrated Model and Approach This document analyzes challenges for using the GeneRic Autonomic Signaling Protocol (GRASP) in the context of deployment of services on machines (servers, Virtual Machines, etc) without the network elements, such as routers, needing any support for GRASP by themselves. It analyses issues regarding the discovery mechanism, including discovery across subnets and operation across administrative boundaries, and provides terminology and general considerations for the development of future extensions, profiles, and architectural refinements.

Introduction The GeneRic Autonomic Signaling Protocol (GRASP) was specified as a common signaling substrate to decentralize the logic of network management . It provides mechanisms for discovery, negotiation, and synchronization between Autonomic Service Agents, without making assumptions about the devices employed or the underlying security layer. Experience with autonomic networking and GRASP already prompted interest in adapting the protocol for use in a broader range of environments, for instance with the emergence of the Constrained GeneRic Autonomic Signaling Protocol (cGRASP) . In the meanwhile, there is a lack of standard for the deployment of Autonomic Functions when such functions are not run on Autonomic Nodes being network elements, but on computers inside the network, taking no active role in networking. These kind of infrastructures where multiple administrative domain co-exist are especially encountered by cloud providers, with Autonomic Nodes being servers, end hosts and virtualized or cloud-native infrastructure. In such configuration, routers and other network elements are not always under the control of the same administrator, and do not necessarily include support for GRASP. Such environments where router cooperation is not possible raises issues related to discovery, with Autonomic Nodes being unable to relay their discovery packets outside of the local subnet (ie. through routers). Using GRASP outside Autonomic Networks with ubiquitous GRASP support would offer several potential benefits. As such, it would enable a unified discovery and negotiation mechanism for autonomic and intent-driven functions for any network of machines. This would avoid the need for new protocols to be defined and implemented on the network elements themselves, including, henceforth allowing more consistent deployment models, and reduced need for ad-hoc signaling solutions at different layers or in different parts of the system. Since basic capabilities such as discovery, signaling, and negotiation are needed in many non-networking scenarios, GRASP also seems a natural candidate for reuse. The purpose of this document is to provide a problem statement for GRASP usage without cooperation from network elements, also called "GRASP through routers", with a view towards an autonomic cloud. It aims at identifying the technical and operational challenges that arise when GRASP is employed on general-purpose hosts or virtualized infrastructure. The goal is not to specify new mechanisms or profiles, but to offer a clear problem space and terminology and to identify areas where future work on extensions, profiles, or architectural refinements would be useful to effectively apply GRASP outside of its original network-centric context.

Network architecture description This section defines the substrate network architecture that prompts potential new developments for GRASP. The main consideration is the deployment of Autonomic Service Agents and Autonomic Functions across heterogeneous network environments, where protocol support cannot be assumed for all devices in the network. The main idea for GRASP through routers is to create a GRASP domain in which nodes are not linked by a fully contiguous GRASP path, or where they lack direct connectivity in both directions. This means that a given node A might be able to send packets directly to node B, but node B cannot establish a connection to node A. Such cases happen, for example, when NAT is involved in the networking path.

The base subnet: a GRASP-aware environment For the purpose of this document, we define a base subnet as a set of Autonomic Nodes with contiguous support for the GRASP hop-by-hop discovery mechanism. In other words, any part of the network not able to connect to the other parts over link-local discovery protocol defines another base subnet. A base subnet can use any, or multiple, addressing schemes, including, but not limited to, the recommended IPv6 scheme, because such addressing schemes (most notably IPv4) are found in practice on many large-scale networks. The idea behind this concept is that since link-local connectivity is expected between adjacent nodes within a subnet, any system administrator can independently operate a fully GRASP-aware domain on a set of nodes within a base subnet without prior network configuration. It should be noted that within this scope, the protocol remains agnostic to the presence of non-participating nodes: any machine not supporting GRASP will simply ignore the traffic and, in turn, be ignored by the protocol.

GRASP domains transcend administrative domains The goal of this document is to address the question of how to create a GRASP bridge between two or more base subnets. In the standard GRASP case, this is eased by the fact that boundary routers have support for the protocol itself, which is the key assumption this document aims to break. Said otherwise, GRASP as defined in always assumes a single base subnet (ie. any node can receive discovery from any other). Breaking free from this use case calls for questionning at least three assumptions:

1. the whole routing chain implements support for the GRASP protocol, especially GRASP relaying of discovery messages;
2. all communicating nodes use the same network-layer protocol (addressing scheme);
3. a path always exists between two nodes in both directions (for node A to node B and from node B to node A).

Cases breaking any of these three base assumptions can occur when some elements in the network, being them servers, routers or any other computers, reside in different administrative domains, and are operated by independent providers. In which case, a GRASP operator may not be aware of the full network configuration in another base subnet as its own, and notice unexpected discovery failures. A typical case is cloud networks, where services are deployed in complex, multi-region environments, and the client has low to no control over the oftentime numerous networking elements forming the end-to-end path. It should be noted that while the initial GRASP specification mandates that cross-domain communication should be performed through the use of two separate GRASP instances, this document does not try to replace nor describe such mechanism. Instead, it states the existing difference between a trust domain and a GRASP-contiguous domain, called base subnet. In other words, it extends a GRASP domain to broader cases, and does not try to initiate communication in between domains: two different trust domains should still operate independent GRASP instances, as required by the specification to communicate with one another.

Reference case #1: GRASP-unaware routers To illustrate how these assumptions might break, two reference (example) networks can be built. The first one involves two base subnets interconnected by one or more routers, which are assumed to be GRASP-unaware. In such case, base assumption (1) is broken: a discovery message cannot be relayed from one subnet to the other through the router.

An example network architecture for GRASP through routers: since router #2 does not support the GRASP protocol, discovery packets are confined to the boundary of subnets #1 and #2

Reference case #2: NAT in the discovery route Another interesting reference case to look at is that where two subnets are interconnected through a NAT. In this case, assumption (3) is broken: devices outside of the NAT area cannot directly address devices inside the NAT area. This can happen even if the router has builtin support for GRASP, even though in most cases, both issues will happen simultaneously.

An example network architecture for GRASP through routers: NAT between inside and outside base subnet prevent discovery messages from the outside to the inside of the NAT

Identified challenges This section examines potential issues encountered when using GRASP, at any stage of an Autonomic Node lifecycle, within the network architecture described above, taking into account the three desribed base assumptions. GRASP being built around two distinct mechanisms: discovery and negotiation, both are examined in order to describe protocol usage within the reference network architecture. Synchronisation, which uses the same mechanism as negotiation, will be examined the same way. One notable exception to this rule is synchronization flooding, which is from a networking perspective closer to the discovery mechanism than to negotiation or synchronization, and will be treated as such in this document.

Negotiation, discovery and NAT Negotiation is used by a Autonomic Node to request an objective from another; as such, it is purely an end-to-end procedure, and uses unicast packets. This feature makes the negotiation mechanism work even when border routers do not have intrinsic support for GRASP (assumption (1) is lifted), since a viable unicast path is always assumed to exist between any two nodes in such case. However, an issue arises within a network where unicast is not guaranteed between all nodes. Such can happen when there is a network protocol mismatch or NAT in the way between two nodes; in which case, negotiation can fail altogether. The failure will be seen as a complete or unidirectional confinement to the local base subnet. Unidirectional confinement means that negotiation can reach the destination node and be acknowledged (since replies are sent on the same socket), but negotiation in the other direction, sent from the former destination node, will fail. Such unidirectional confinement are very likely when NAT is involved, and may be hard to detect. On the other hand, complete discovery failures due to addressing protocol mismatch will usually be easy to find, even without sending a single packet: the base subnets will be completely invisible to each other in this case. While negotiation will fail in cases described, it is also true of discovery. During discovery, unidirectional confinement will even, in most cases, lead to complete failure, because discovery is a multicast process and requires the destination node to be able to connect back to the initiator or relaying node. An exception to this is when unicast is used for discovery, in which case the node answers over the same socket, and the discovery process becomes closer to negociation.

GRASP-unaware routers and hop-by-hop relaying While no issue arises from the negotiation procedure when border routers lack support for GRASP, the other mechanism, discovery (as well as the similar flooding mechanism), used to locate Autonomous Service Agents (ASAs) supporting a given objective, is not as simple. It relies not only on simple unicast, but for many parts requires multicast , as well as a specific protocol for domain-wide flooding through hop-by-hop link-local forwarding between adjacent nodes. Such a behavior should be analyzed in details to understand in which ways it may fail within the described architecture. The core issue is that the discovery mechanism causes domain fragmentation at the boundary of subnets in environment with only partial GRASP support, more specifically when routers at the boundary of a subnet do not have explicit GRASP support. In other words, this is a different case of confinement to a base subnet, also caused by routers. In use cases as described, this reliance on hop-by-hop forwarding creates a fragile discovery topology: in a heterogeneous network where GRASP support is not ubiquitous, a single legacy router or non-compliant switch acts as an barrier to discovery probes. Because link-local multicast is confined to the immediate segment, the discovery mechanism cannot get through non-GRASP nodes and reach other ASAs in the domain. Consequently, messages being confined to a subnet will lead to issues in the deployment of the required range of services. If a DNS server is to be discovered, on a given subnet, using GRASP, machines from all other subnets will fail to autoconfigure, although the DNS server is by itself reachable through routers, and manual configuration would have succeeded. Extending GRASP to support such use case would therefore benefit the development of autonomic networking as a whole, since it extends cases where manual configuration is avoidable. As well as issues related to the passing of messages, the network risks fragmentation into multiple subdomains without explicit division in cases where GRASP messages are blocked at subnet boundaries. Servers residing behind non-participating routers remain invisible to the wider group, and global configuration state distributed via flooding becomes inconsistent across the domain. Without a fallback mechanism or a method to bypass these gaps, the protocol remains hard to use in a global infrastructure.

Mandatory security and the Autonomic Control Plane As defined in the GRASP specification, every solution must specify a security mechanism to be used by GRASP. The main security mechanism, part of the Autonomic Control Plane (ACP) , is dependent on the establishment of an out-of-band hop-by-hop link-local overlay network, which, in itself, cannot work within the described architecture, for the same reasons that the discovery mechanism. Since the security mechanism is not a part of the GRASP specification per se, this document does not aim at improving the ACP further to support use cases described in the network architecture description. It simply emphasizes that such mechanism cannot be used as-is within this architecture, and that new security substrates should be developped, even though standardizing such mechanism may not always be needed, depending on the use case.

Proposed architecture To get GRASP working within the network architecture, the key challenge is to solve again the issue of relaying packets, which routers are usually reluctant to do, and for which the GRASP hop-by-hop discovery mechanism was created. Multiple solutions are examined, from the worst case where none of the three base assumptions hold, and the GRASP operator has no control whatsoever on the network, to the best scenario where there is an issue with only one of the three base assumptions.

Solving the discovery problem The simplest way in which a GRASP infrastructure can fail is when there only is no support for GRASP on some of the border routers. In this case, no NAT is used, and all nodes use the same network protocol. An approach to fixing such network, showing only a discovery problem, involves added initial knowledge. To consider the most extreme case, if all nodes knew their peers beforehand, GRASP could unicast discovery packets to each peer every time. This approach is possible within GRASP's bounds, but ends up in a non-reconfigurable network.

Introducing relay nodes Instead, initial knowledge can be given to at least one node within each subnet, which will be tasked to relay GRASP messages to other subnets; we call such node a relay node. A relay node has two important parameters: a list of peer nodes and an optional local interface. With such parameters, a relay node's task can be summarized as follows: it listens for incoming multicast or unicast GRASP messages, and reemits these messages both as unicast messages addressed to all peer nodes, and as multicast messages on the local interface, if defined and if the message does not come from this same interface. There are two possible approaches to configuring the peers: either local interface and list of peers are initially present on the relay, being them manually added, or configured through another protocol not part of the GRASP ecosystem, or the relay is autoconfigured from the local domain through negociation of a GRASP objective. This new objective, defined to allow dynamic configuration of the relay peers and local interface, may also be used to enable the automatic discovery of a relay node within the local domain. Such objective therefore facilitate zero-configuration deployment, allowing nodes to automatically discover and configure their local relay in cases where there is one relay per subnet. For scenarios where there is only one global relay node, it must also be possible to configure such relay manually. Peer nodes known by a relay node can be of two kinds: other relay nodes, in which case the relay node proxies part of the router function of forwading multicast to peers, or non-relay nodes, making the relay node an "omniscient" node with great network knowledge.

One relay per subnet The first case described, where relaying is done between relay nodes, relies on having one relay node inside each subnet, which simply needs to be configured with addresses of the other relay nodes within other reachable subnets. A tradeoff to this solution is partial centralization, at the subnet level: if the relay node becomes unreachable, querying other subnets becomes impossible. To send traffic outside of the current subnet, a node therefore acts as in a standard GRASP configuration, by sending traffic multicasted to peers, such peers including the relay node. The flow is pictured in figure , and upon leaving node (1), a packet goes through the relay node, which unicasts it (2) to another relay node within a different subnet, where it gets sent as multicast on the local interface (3). In order to keep things simple, we assume that there is at most one relay node within each subnet. Support for more that one such node could be developped later to introduce redundancy, but it is hard in such case to decide which is responsible for what, and to avoid sending a packet multiple times, creating useless network traffic.

A typical traffic flow for solving the discovery problem, which includes adding one relay node for each base subnet.

Omniscient relays Another configuration for a relay is to have a list of non-relay nodes to send packets to. This is especially suitable in networks where only few nodes are outside of a big base subnet, or when almost all machines are within their own base subnet (isolated machines with no contiguous GRASP discovery path). In such cases, a single machine can have discovery knowledge, and know how to address all other nodes within the network. Such a machine would be able to use the same "indirect unicast" mechanism, where a node addresses (over unicast) to a central node, within or outside of the subnet, that node then relaying the GRASP message, also over unicast, to all of the nodes it knows. This goes a level of centralization further compared to the one relay per subnet case, and the relay node would have to acquire knowledge of the other nodes using an ad-hoc mechanism or pre-configuration. The flow for omniscient relays is simple, and consists of two unicast packets: GRASP packets get sent to the relay node, which unicasts them to all other nodes. To make this configuration work, similarly to the relay-to-relay case, a mechanism is needed for sending relayed multicasts to remote relays or nodes.

A typical traffic flow for solving the discovery problem, where there is only one relay node globally.

Reverse discovery Non-routing relays, being them one-per-subnet or omniscient, only solve the discovery issue when the network has no other issues that GRASP-unaware routers. Other base assumptions, where multiple base subnets use different addressing schemes, or NAT, require a different solution. One such possible solution is to put active relays inside the routers: the router linking the two base subnets would decode GRASP packet and re-encodes them differently, as suggested in . This is a complex solution generally, which becomes completely unfeasible whenever the subnets reside in different administrative domains. Therefore, in order to solve the issue with unidirectional networks, the relay may implement a mechanism where, instead of only relaying incoming discovery or negotiation packets, it keeps them for some time, and can serve them back to clients asking for them. In other word, this new process reverses completely the direction in which GRASP works: instead of passive nodes waiting for packets, some nodes become active and start polling. We call this process "reverse discovery". This mechanism must include a process for limiting the number of requested packets, for instance by tracking down which packets were already delivered to a given node via this mechanism. Such reverse discovery mechanism would solve completely any issue with NAT-ing, and make networks where packets can only go one way (from node A to node B), fully GRASP-aware by means of the relay. Assumes that node A sends discovery to a relay within its subnet, and node B polls this relay every five minutes. Even if node B is on the outside of NAT, and could not receive the message sent by node A because both border router does not support GRASP, and relay within A's network could not connect to relay within B's network (which needs not to exist in this case), the packet will be received by polling the relay in A's.

Operational Considerations

Loop avoidance While the Generic Autonomic Signaling Protocol (GRASP) provides built-in loop count and timeout mechanisms to ensure that the discovery process ends and mitigate loops, the relaying mechanism does not inherently benefit from such safeguards. Consequently, loop prevention relies on the network administrator to avoid misconfigurations such as pointing a relay node to forward messages back to another relay connected to the origin.

Scalability In environments containing large numbers of relay nodes, especially, in the one-per-subnet architecture, when each relay node has multiple neighbors, message relaying of all discovery and flooding messages can introduce significant traffic overhead as the network grows. This can lead to excessive link utilization, impacting convergence time, or even normal network operations. To avoid such issues, relay nodes may limit the objective they allow to relay using an extra, optional parameter. Relays may also use caching policies for objectives to suppress redundant discovery traffic across large-scale deployments. Finally, if none of these strategy are enough, networking devices, relays and nodes themselves may define rate-limiting policies for GRASP traffic.

Security Considerations Once relay nodes are acknowledged as a potential solution, and since the reference network architecture allows operation outside of the ACP, questions arise about the security mechanism used for relay nodes. Within a base subnet, nodes can trust a local relay because of the intrinsic subnet's security mechanism, being it the ACP or another ad-hoc mechanism. This leaves two trust concerns to be examined: how do relays establish trust between each other, and once they acknowledge themselves, what kind of mechanism secures communication between these relays. Similar to the GRASP specification, no explicit security mechanism is enforced, but relays must communicate with peers nodes using a secure channel compatible with GRASP requirements. Regarding bootstrapping, in a standard ACP-based GRASP deployment, it is usually handled by the Bootstrapping Remote Secure Key Infrastructure (BRSKI) , with aditions from being especially useful for deployment in the suggested network environments. Application statements for the use of this mechanism between relays, with a central registrar being defined inside or outside of the cross-domain network, can be developed. However, this mechanism can be burdensome for such simple case of peer-to-peer communication between nodes. Therefore, out-of-band, including manual, installation of TLS certificates, or any other authentication mechanism, can be used for relay authentication.

IANA Considerations This document is informational and has no IANA actions.

Normative References Informative References

Acknowledgments Part of this draft is based on an discussion with , who provided a lot of preliminary ideas. I hope I did not betray the discussion too much while drafting. I am also grateful to for early guidance on the IETF procedures, to Bilal Ghani and Hakima Chaouchi for their kind pre-submission review, and to , and all of the ANIMA Working Group for their valuable feedback.