]> Everything OPS & Arrcus

benoit@everything-ops.net

Ciena

paitken@ciena.com

Deutsche Telekom

holger.keller@telekom.de

ZTE Corporation

liu.yao71@zte.com.cn

NTT

Veemweg 23 3771 MT Barneveld Netherlands paolo@ntt.net

Operations and Management OPSAWG IPFIX ordered encapsulation MPLS label stack tunneling The IP Flow Information Export (IPFIX) protocol allows the same Information Element to appear multiple times in a Template Record. RFC 7011 states that multiple occurrences of the same Information Element SHOULD follow the logical order of their treatments by the Metering Process; however, no protocol mechanism exists for the Exporting Process to explicitly signal to the Collecting Process that this ordering is guaranteed. Without such a guarantee, the Collecting Process cannot safely interpret repeated Information Elements as ordered observations. This document describes the problem, presents use cases, analyzes candidate solutions, and specifies a mechanism -- a new Ordered Set type -- that establishes an end-to-end ordering guarantee: the Metering Process records repeated Information Element values in the order the corresponding observations are made, the Exporting Process preserves and signals this guarantee, and the Collecting Process interprets the n-th occurrence of an Information Element as corresponding to the n-th observation by the Metering Process at the Observation Point. This document updates RFC 7011, RFC 5476, and RFC 6183. About This Document Status information for this document may be found at .

Introduction The IPFIX protocol ("Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow Information") exports Flow information from an Exporting Process to one or more Collecting Processes. An (Options) Template Record defines the structure of a Data Record by specifying an ordered list of Information Elements. When the same Information Element appears more than once in a Template Record, Section 8 of states:

• If an Information Element is required more than once in a Template, the different occurrences of this Information Element SHOULD follow the logical order of their treatments by the Metering Process.

The use of SHOULD rather than MUST means that an Exporting Process is not required to guarantee this ordering. Furthermore, Section 4.2 and Section 4.3 of each explicitly state:

• the order of the Information Elements in the Template Records is not guaranteed.

Nevertheless, the Metering Process SHOULD record repeated Information Element values in the order in which the corresponding observations are made, as this is the natural basis for any ordering guarantee. It is important to note that IPFIX is fundamentally an export protocol: defines the behavior of the Exporting Process and the Collecting Process in detail, but places very few normative constraints on the Metering Process. The IPFIX architecture ("Architecture for IP Flow Information Export") explicitly scopes IPFIX as governing "the export of measured IP Flow information from an IPFIX Exporting Process to a Collecting Process" and defines the Metering Process as an internal component of the IPFIX Device -- alongside the Exporting Process, but not a protocol peer (see Figure 2 of ). An end-to-end ordering guarantee requires that all three components act consistently: the Metering Process records values in the order in which they are observed at the Observation Point, the Exporting Process preserves that order during export, and the Collecting Process interprets the received ordering accordingly. There is consequently no IPFIX protocol mechanism by which the Exporting Process can signal to the Collecting Process that the ordering of repeated Information Elements is guaranteed to reflect the Metering Process observation order -- even when the Metering Process correctly records those observations in order. Without such a signal, a Collecting Process cannot safely interpret the n-th occurrence of an Information Element as corresponding to the n-th observation made by the Metering Process at the Observation Point. This limitation has several practical consequences, of which the following are the most prominent:

1. Information Element Proliferation: position-specific Information Elements accumulate in the IANA IPFIX registry to work around the missing ordering guarantee (see ).
2. Multi-Layer Encapsulation Ambiguity: without an ordering guarantee, the Collecting Process cannot determine which encapsulation layer a given Information Element occurrence corresponds to (see ).

The ordering problem also arises in the Packet Sampling (PSAMP) protocol family , which uses IPFIX as its export protocol and relies on IPFIX IEs to describe packet selection and observation metadata. PSAMP defines a Composite Selector as an ordered sequence of Primitive Selectors applied in succession to a Packet Stream. Section 6.5.1 of already requires that selectorId Information Elements in the Selection Sequence Report Options Template MUST be listed in the order the corresponding Primitive Selectors are applied -- making the ordering requirement for Options Template Records older than this document. However, this requirement cannot be enforced at the IPFIX protocol level: the Options Template Set (Set ID 3) of provides no mechanism for the Exporting Process to signal to the Collecting Process that the ordering of repeated selectorId occurrences is guaranteed. The Ordered Options Template Set (Set ID 5) defined in this document closes this gap. ("Export of Structured Data in IP Flow Information Export (IPFIX)") has sometimes been cited as a solution to this problem, through the "ordered" semantic defined for its basicList and subTemplateMultiList structured data types. However, as analyzed in detail in , the "ordered" semantic is an export ordering guarantee only -- it does not assert that the order reflects the Metering Process observation order at the Observation Point. Furthermore, structured data still falls under the SHOULD of Section 8 of and does not upgrade it to a MUST. therefore does not solve the problem described in this document. This document specifies a mechanism to address these use cases by providing an explicit ordering guarantee for repeated Information Elements in an IPFIX (Options) Template Record. The scope of this document is deliberately limited to this specific problem. Other potential IPFIX improvements are explicitly out of scope.

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Terminology This document uses the IPFIX-specific terminology defined in , the PSAMP-specific terminology defined in , and the IPFIX Mediation terminology defined in ("IP Flow Information Export (IPFIX) Mediation: Framework"). As in , the first letter of each IPFIX-specific term is capitalized. The following terms are used as defined in :

Collecting Process:

The process in a Collector that receives and processes IPFIX Messages.

Data Record:

A record that contains values of the parameters corresponding to a Template Record.

Data Set:

A Set that contains one or more Data Records defined by the same Template Record.

Exporting Process:

The process in an Exporter that sends IPFIX Messages to a Collecting Process.

Flow Record:

A record that contains information about a specific Flow.

Information Element:

A protocol-independent description of an attribute that has been observed or metered.

Metering Process:

The process that generates Flow Records from packet observations at an Observation Point.

Observation Domain:

The largest set of Observation Points for which Flow information can be aggregated by a Metering Process.

Observation Point:

A location in the network where IP packets can be observed.

Options Template Record:

A Template Record that defines the structure of an Options Data Record.

Options Template Set:

A collection of one or more Options Template Records that have been grouped together in an IPFIX Message.

Set:

A collection of records that have a similar structure, prefixed by a header.

Template Record:

A record that defines the structure of a Data Record.

Template Set:

A collection of one or more Template Records that have been grouped together in an IPFIX Message.

Transport Session:

In SCTP, the Transport Session is the SCTP association. In TCP, the Transport Session is the TCP connection. In UDP, the Transport Session is the UDP session, uniquely identified by the combination of IP addresses and UDP ports used.

Note: A Flow Record is a specific type of Data Record. Options Data Records, which carry metadata rather than Flow measurements, are also Data Records but not Flow Records. The following term is modified from :

Set ID:

A field in the Set header that identifies the Set type. Value 2 identifies a Template Set, value 3 an Options Template Set, values 4 and 5 identify Ordered Template Sets and Ordered Options Template Sets respectively as defined in this document, and values 256 and above identify Data Sets. Values 6-255 are reserved for future use.

The following terms are used as defined in :

Composite Selector:

An ordered composition of Selectors, in which the output Packet Stream issuing from one Selector forms the input Packet Stream to the succeeding Selector.

Primitive Selector:

A Selector that applies exactly one selection method and is the elemental component of a Composite Selector.

Selection Sequence:

A specific combination of an Observation Point and a Composite Selector, identified by a selectionSequenceId Information Element.

The following term is used as defined in :

IPFIX Mediator:

An IPFIX Device that receives IPFIX Messages from one or more Exporting Processes or other IPFIX Mediators, optionally transforms them through one or more Intermediate Processes, and forwards the results to one or more Collecting Processes.

The following new terms are defined in this document:

Ordered Template Set:

A Set with Set ID 4 that carries Ordered Template Records. It uses the same wire format as the Template Set (Set ID 2) defined in , with the addition that the ordering of repeated Information Elements in the corresponding Data Records is guaranteed to reflect the Metering Process observation order at the Observation Point.

Ordered Options Template Set:

A Set with Set ID 5 that carries Ordered Options Template Records. It uses the same wire format as the Options Template Set (Set ID 3) defined in , with the same ordering guarantee as an Ordered Template Set.

Ordered Template Record:

A Template Record transmitted in an Ordered Template Set (Set ID 4). The order of Information Elements in an Ordered Template Record is significant: when the same Information Element appears more than once, its occurrences MUST reflect the order in which the corresponding values are observed by the Metering Process at the Observation Point.

Ordered Options Template Record:

An Options Template Record transmitted in an Ordered Options Template Set (Set ID 5), with the same ordering guarantee as an Ordered Template Record.

Ordered Data Set:

A Data Set whose Template ID refers to an Ordered Template Record or an Ordered Options Template Record. The ordering of fields in each Data Record within the Set reflects the Metering Process observation order.

Problem Statement & Use Cases

The Ordering Ambiguity The IPFIX protocol allows any Information Element to appear multiple times in a Template Record. Section 8 of provides the following guidance and example:

• If an Information Element is required more than once in a Template, the different occurrences of this Information Element SHOULD follow the logical order of their treatments by the Metering Process. [...] For example, when exporting the two source IP addresses of an IPv4-in-IPv4 packet, the first sourceIPv4Address Information Element occurrence should be the IPv4 address of the outer header, while the second occurrence should be the address of the inner header.

The use of SHOULD rather than MUST means that a conformant Exporting Process is not required to respect this ordering. Even when the Exporting Process does follow the ordering by convention, there is no protocol field by which it can communicate this intent to the Collecting Process. Furthermore, the ordering of Information Elements may be affected by optimizations within the Metering Process itself (for example, software that processes packet headers from different encapsulation layers in parallel rather than in strict observation order) or by optimizations that intervene between the Metering Process and the Exporting Process. Section 2.2 ("IPFIX Limitations") of notes that "some encoding optimizations are based on the permutation of Information Element order", which conflicts with any strict ordering requirement applied universally to all existing IPFIX Sets.

Information Element Proliferation The absence of an ordering guarantee has led implementors to define position-specific Information Elements, effectively polluting the IANA IPFIX registry with entries that exist solely to work around the lack of an ordering mechanism. The MPLS label stack is the most prominent example. Rather than using a single label stack entry Information Element appearing multiple times in ordered fashion (such as the one defined in ), the IPFIX Information Elements registry defines ten separate Information Elements:

• mplsTopLabelStackSection (IPFIX IE 70)
• mplsLabelStackSection2 (IPFIX IE 71)
• mplsLabelStackSection3 (IPFIX IE 72)
• mplsLabelStackSection4 (IPFIX IE 73)
• mplsLabelStackSection5 (IPFIX IE 74)
• mplsLabelStackSection6 (IPFIX IE 75)
• mplsLabelStackSection7 (IPFIX IE 76)
• mplsLabelStackSection8 (IPFIX IE 77)
• mplsLabelStackSection9 (IPFIX IE 78)
• mplsLabelStackSection10 (IPFIX IE 79)

This approach has some drawbacks:

• It consumes ten Information Element identifiers where one could suffice.
• It artificially limits the exportable label stack depth to ten entries.
• It cannot be generalized to other stacked or layered structures without defining further sets of numbered Information Elements.

The same pattern extends beyond the label stack entries. A cluster of Information Elements exists for attributes of the top-of-stack label only: mplsTopLabelType (IPFIX IE 46), mplsTopLabelIPv4Address (IPFIX IE 47), mplsTopLabelPrefixLength (IPFIX IE 91), mplsTopLabelIPv6Address (IPFIX IE 140), mplsTopLabelTTL (IPFIX IE 200), mplsTopLabelExp (IPFIX IE 203), and postMplsTopLabelExp (IPFIX IE 237). These exist because no general mechanism allows expressing "the attribute of the label at stack position N." With ordered export, a single repeated Information Element per attribute would suffice. A Metering Process observing MPLS traffic may observe a label stack of arbitrary depth. An operator may wish to export the entire label stack as a sequence of MPLS label stack entries, where the first occurrence corresponds to the top of the stack, the second to the next label, and so on. With the mechanism specified in this document, the Exporting Process can use a single label stack entry Information Element (see ) repeated N times in an Ordered Template Record and signal that the sequence reflects the top-to-bottom order of the label stack as observed by the Metering Process. This eliminates the need for the ten numbered label-stack Information Elements. Since N is chosen by the Exporting Process when it defines the Template, label stacks of depth greater than ten can be exported in full. This lifts the artificial cap inherent in the positional naming scheme. N is however fixed for a given Template: exporting label stacks of different depths requires a Template for each depth. evaluates a list data type that avoids this.

Ordering Ambiguity in PSAMP Options Template Records The ordering requirement extends beyond Template Records: it also applies to Options Template Records used by the Packet Sampling (PSAMP) protocol ("A Framework for Packet Selection and Reporting") ("Sampling and Filtering Techniques for IP Packet Selection") ("Packet Sampling (PSAMP) Protocol Specifications"). This is the primary use case for the Ordered Options Template Set (Set ID 5) specified in this document. PSAMP defines a Composite Selector as "an ordered composition of Selectors, in which the output Packet Stream issuing from one Selector forms the input Packet Stream to the succeeding Selector" (). The word "ordered" in the definition is architecturally significant: a Composite Selector is a sequence, not a set. The ordering matters semantically. Section 8.1 of demonstrates that filtering followed by sampling and sampling followed by filtering produce different statistical results:

• Filter then Sample: the filter creates population subsets with potentially different intensities, allowing the sampling rate to be tuned per subset. Packets not matching the filter never load the Sampling function.
• Sample then Filter: the same sampling rate is applied to the entire stream before filtering; packets ultimately discarded by the filter still consume sampling capacity.

A Composite Selector is exported using the Selection Sequence Report Options Template defined in Section 6.5.1 of . When the Selection Sequence consists of multiple Primitive Selectors, the selectorId Information Element appears once per Primitive Selector in the same Options Template Record. Section 6.5.1 of is explicit:

• "the selectorId's MUST be identified in the order they are used."

Section 6.5.1 of illustrates this with a concrete example: Selection Sequence 7 (Filter selectorId 5, then Sampler selectorId 10) and Selection Sequence 9 (Sampler selectorId 10, then Filter selectorId 5) are distinct, produce different results, and are differentiated solely by the order of the two selectorId occurrences in the Options Template Record. RFC 5476 therefore already requires ordered Options Template Records, and a PSAMP-aware Collecting Process can infer this guarantee from receipt of the Selection Sequence Report Options Template. However, IPFIX-layer components that do not implement PSAMP -- forwarders, middleboxes, or generic analysis tools -- have no means to apply that constraint. The Ordered Options Template Set (Set ID 5) specified in this document expresses the ordering guarantee at the IPFIX protocol level, making it visible to any IPFIX component regardless of PSAMP awareness.

Multi-Layer Encapsulated Traffic Modern networks increasingly carry traffic with multiple encapsulation layers. SRv6 deployments, in particular, may result in packets with two or three nested IPv6 headers, each with its own source and destination addresses. When an Exporting Process exports fields from such packets, repeating Information Elements such as sourceIPv6Address and destinationIPv6Address, the Collecting Process has no way to determine which occurrence corresponds to which encapsulation layer unless an ordering guarantee is explicitly provided. This problem is described in , which proposes several mechanisms to address the specific case of encapsulation layer identification. That draft brings renewed attention to an ordering ambiguity that has been present in IPFIX since was first published. The present document addresses the underlying, more general problem of ordered Information Element export, of which multi-layer encapsulation is one important use case. When a packet carries multiple IP encapsulation layers -- for example, an SRv6 packet where a Provider Edge adds an outer IPv6 header encapsulating an inner IPv6 packet from the customer -- an Exporting Process may wish to export fields from multiple layers. With the mechanism specified in this document, the Exporting Process can include destinationIPv6Address twice in an Ordered Template Record and signal that the first occurrence corresponds to the outermost encapsulation layer and the second to the innermost, reflecting the order observed by the Metering Process. This use case covers IPv4-in-IPv4, IPv4-in-IPv6, IPv6-in-IPv6, and any other IP-in-IP encapsulation scenario, as well as SRv6 traffic with nested Segment Routing Headers as described in ("Export of Segment Routing over IPv6 Information in IP Flow Information Export (IPFIX)"). Further examples include:

• MAC-in-MAC (IEEE 802.1ah Provider Backbone Bridging): both sourceMacAddress and destinationMacAddress repeat -- once for the outer backbone MAC (B-MAC) and once for the inner customer MAC (C-MAC).
• VXLAN ("Virtual eXtensible Local Area Network (VXLAN): A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks") and GENEVE ("Geneve: Generic Network Virtualization Encapsulation"): outer IP transport addresses (UDP/IP encapsulation) and inner tenant IP addresses both map to Information Elements such as sourceIPv4Address and destinationIPv4Address; without an ordering guarantee the Collecting Process cannot distinguish transport layer from tenant layer.
• GRE: outer and inner IP header fields repeat for the same Information Elements, with the same ambiguity.

Without the mechanism specified in this document, unambiguous export of these layered fields would require defining new position-specific Information Elements, perpetuating the same registry proliferation already seen with MPLS. The mechanism specified in this document addresses the common case where all encapsulation layers are exported consecutively, with the n-th occurrence of an Information Element corresponding to the n-th layer. The explicit layer-marking approach of is complementary and better suited to sparse export of layers and to Collecting Processes that do not support Ordered Sets.

New Ordered Set Specifications The solution is the New Ordered Set Type: the definition of two new Ordered Set types, Ordered Template Set (Set ID 4) and Ordered Options Template Set (Set ID 5). The rationale is as follows:

1. Explicit protocol semantics: A dedicated Set type provides an unambiguous, per-Set signal that ordering is guaranteed. The Collecting Process requires no additional correlation or inference to determine whether a given Template is ordered.
2. Clean protocol design: Ordered data is a distinct semantic concept that warrants a first-class protocol signal. A new Set type is the natural extension point within the IPFIX message format, consistent with the principle that Set IDs 4 through 255 are reserved in for exactly this purpose.
3. Backwards compatibility: The existing Set IDs 2, 3, and 256 to 65535 are unaffected. Exporting Processes that do not require ordered export continue to use the existing Set IDs without modification.

Analysis of alternative approaches is provided in .

Wire Format The Ordered Template Set (Set ID 4) uses exactly the same wire format as the Template Set (Set ID 2) defined in Section 3.4.1 of . The Ordered Options Template Set (Set ID 5) uses exactly the same wire format as the Options Template Set (Set ID 3) defined in Section 3.4.2 of . The Set ID value is the sole distinguishing element: it signals to the Collecting Process that the ordering of repeated Information Elements in corresponding Data Records is guaranteed. No new header fields, flags, or record structures are introduced. Implementations that already support Set ID 2 and Set ID 3 can support Set ID 4 and Set ID 5 by reusing the same parsing code and adding the ordering semantic to the Template Records so identified.

Template Withdrawal The Template Withdrawal mechanism defined in Section 8.1 of applies to Ordered Template Records and Ordered Options Template Records, with the update to the Set ID of a Template Withdrawal specified in . An Ordered Template Record is withdrawn by sending an Ordered Template Set (Set ID 4) containing a Template Record with the target Template ID and a Field Count of 0. An Ordered Options Template Record is withdrawn by sending an Ordered Options Template Set (Set ID 5) with a Field Count of 0. A Template Withdrawal whose Set ID does not correspond to the Set ID in which the Template Record was received is ignored by the Collecting Process, which SHOULD log the error, as specified in Section 8.1 of for a Template Withdrawal of a Template that is not presently stored. This does not apply to the All Templates Withdrawal and the All Options Templates Withdrawal of Section 8.1 of , which also withdraw the Ordered Template Records and the Ordered Options Template Records of the Observation Domain, respectively. Upon receipt of a Template Withdrawal, the Collecting Process MUST cease treating Data Sets referencing the withdrawn Template ID as Ordered Data Sets and MUST discard all stored information about that Template Record. This applies to the IPFIX transport state only; it does not retroactively alter the ordering interpretation of Data Records already received and decoded under that Template ID. A withdrawn Ordered Template ID MAY be reused by the Exporting Process within the same Transport Session and Observation Domain, consistent with Section 8.1 of . If the Template ID is subsequently registered via an unordered Template Set (Set ID 2), the Collecting Process MUST treat it as unordered. If registered again via an Ordered Template Set (Set ID 4), the ordering guarantee applies anew. The same applies for Options Template Records (Set IDs 3 and 5). Over UDP, Section 8.4 of applies unchanged: Template Withdrawals are not used, and a Template ID is reused after the waiting period of that section. Whether the reused Template is ordered is determined by the Set ID of the Template Record sent for that Template ID. For the purpose of Section 8.1 of , the Set type in which a Template Record is received is part of the Template definition. A Template Record received for an already-allocated Template ID in a Set of a different Set type (for example, in Set ID 4 for a Template ID previously received in Set ID 2) is therefore different from the already-received Template, and is handled as specified in that section. Over SCTP and TCP, changing a Template from unordered to ordered, or from ordered to unordered, therefore requires withdrawing it first and then sending the new Template Record.

Normative Behavior The Exporting Process MUST preserve the Metering Process observation order when constructing Data Records for an Ordered Data Set. The observation order is from outermost to innermost for encapsulated traffic and from top to bottom for label stacks, as specified in . A Collecting Process identifies an Ordered Data Set by tracking the Set ID under which the corresponding Template Record was received. If the Template was received in an Ordered Template Set (Set ID 4) or an Ordered Options Template Set (Set ID 5), all subsequent Data Sets referencing that Template ID MUST be treated as Ordered Data Sets. If the Template was received in a Template Set (Set ID 2) or an Options Template Set (Set ID 3), the ordering of repeated Information Elements in the corresponding Data Records is not guaranteed. Section 3.4.1 of requires Template IDs to be unique within an Observation Domain and Transport Session. A given Template ID can therefore appear in only one Set type -- it cannot be registered as both an Ordered Template Set (Set ID 4) and a Template Set (Set ID 2), nor as both an Ordered Options Template Set (Set ID 5) and an Options Template Set (Set ID 3). There is consequently no ambiguity in the Collecting Process's per-Template-ID tracking of the ordering guarantee. The ordering guarantee is part of the state kept for a Template and is scoped like the Template, per Transport Session and Observation Domain. The Collecting Process MUST interpret the n-th occurrence of an Information Element in a Data Record from an Ordered Data Set as corresponding to the n-th observation of that Information Element by the Metering Process at the Observation Point. This ordering guarantee applies independently to each Information Element: the n-th occurrence of IE X corresponds to the n-th observation of IE X, and the n-th occurrence of IE Y corresponds to the n-th observation of IE Y. The guarantee does not impose any relative ordering between occurrences of different Information Elements. This guarantee covers the ordering of exported occurrences only. The Exporting Process is expected to export all observations consecutively; where only a sparse subset of layers is exported, the Collecting Process cannot determine whether exported occurrences are contiguous from the first observation. For sparse export scenarios, the explicit layer-marking approach of is more appropriate. The behavior of IPFIX Mediators ("Flow Selection Techniques") that receive Ordered Template Sets or Ordered Options Template Sets is specified in .

Updates to RFC 7011 The following updates are required. The following text in Section 8 of : OLD:

• If an Information Element is required more than once in a Template, the different occurrences of this Information Element SHOULD follow the logical order of their treatments by the Metering Process. For example, if a selected packet goes through two hash functions, and if the two hash values are sent within a single Template, the first occurrence of the hash value should belong to the first hash function in the Metering Process. For example, when exporting the two source IP addresses of an IPv4-in-IPv4 packet, the first sourceIPv4Address Information Element occurrence should be the IPv4 address of the outer header, while the second occurrence should be the address of the inner header. Collecting Processes MUST properly handle Templates with multiple identical Information Elements.

NEW:

• If an Information Element is required more than once in a Template transmitted in a Template Set (Set ID 2) or Options Template Set (Set ID 3), the different occurrences of this Information Element SHOULD follow the logical order of their treatments by the Metering Process. For example, if a selected packet goes through two hash functions, and if the two hash values are sent within a single Template, the first occurrence of the hash value should belong to the first hash function in the Metering Process. For example, when exporting the two source IP addresses of an IPv4-in-IPv4 packet, the first sourceIPv4Address Information Element occurrence should be the IPv4 address of the outer header, while the second occurrence should be the address of the inner header. Collecting Processes MUST properly handle Templates with multiple identical Information Elements. If the Template Record is transmitted in an Ordered Template Set (Set ID 4) or an Ordered Options Template Set (Set ID 5), the Metering Process MUST record the repeated Information Element values in the order in which the corresponding observations are made at the Observation Point, the Exporting Process MUST preserve and follow this order when constructing the corresponding Data Records, and the Collecting Process MUST interpret the n-th occurrence as corresponding to the n-th observation by the Metering Process at the Observation Point. The Metering Process MUST record observations from outermost to innermost for encapsulated traffic and from top to bottom for label stacks.

The following sentence in Section 4.2 of : OLD:

• Since the Metering Process Reliability Statistics Options Template contains two identical timestamp Information Elements, and since the order of the Information Elements in the Template Records is not guaranteed, the Collecting Process interprets the time interval of ignored packets as the range between the two values; see Section 5.2 for wraparound considerations.

NEW:

• Since the Metering Process Reliability Statistics Options Template contains two identical timestamp Information Elements, and since the order of the Information Elements in Options Template Records transmitted in Sets with Set ID 3 is not guaranteed, whereas the order in Options Template Records transmitted in Sets with Set ID 5 is guaranteed to reflect the Metering Process observation order, the Collecting Process interprets the time interval of ignored packets as the range between the two values; see Section 5.2 for wraparound considerations.

The following sentence in Section 4.3 of : OLD:

• Since the Exporting Process Reliability Statistics Options Template contains two identical timestamp Information Elements, and since the order of the Information Elements in the Template Records is not guaranteed, the Collecting Process interprets the time interval of dropped packets as the range between the two values; see Section 5.2 for wraparound considerations.

NEW:

• Since the Exporting Process Reliability Statistics Options Template contains two identical timestamp Information Elements, and since the order of the Information Elements in Options Template Records transmitted in Sets with Set ID 3 is not guaranteed, whereas the order in Options Template Records transmitted in Sets with Set ID 5 is guaranteed to reflect the Metering Process observation order, the Collecting Process interprets the time interval of dropped packets as the range between the two values; see Section 5.2 for wraparound considerations.

The following sentence in Section 3.3.2 of : OLD:

• Values from 4 to 255 are reserved for future use.

NEW:

• Values 4 and 5 are used for Ordered Template Sets and Ordered Options Template Sets, respectively, as specified in this document. Values from 6 to 255 are reserved for future use.

The following normative text is added to Section 3.3.2 of , after the sentence "The Length value MUST be used to determine the position of the next Set.": NEW:

• A Collecting Process that receives a Set with an unknown Set ID SHOULD log a warning to assist operators in detecting configuration or capability mismatches between Exporting and Collecting Processes.

The following sentence in Section 8.1 of : OLD:

• The Set ID field MUST contain the value 2 for Template Set Withdrawal or the value 3 for Options Template Set Withdrawal.

NEW:

• The Set ID field MUST contain the value 2 for Template Set Withdrawal, the value 3 for Options Template Set Withdrawal, the value 4 for Ordered Template Set Withdrawal, or the value 5 for Ordered Options Template Set Withdrawal, that is, the Set ID of the Set in which the Template Record was received.

Updates to RFC 5476 Section 6.5.1 of already mandates that selectorId occurrences in a Selection Sequence Report Interpretation Options Template Record MUST be listed in the order the corresponding Primitive Selectors are applied. However, predates the Ordered Options Template Set mechanism defined in this document, and therefore uses the regular Options Template Set (Set ID 3), which provides no protocol-level signal to the Collecting Process that the ordering is guaranteed. The following sentence in Section 6.5.1 of : OLD:

• If multiple Selectors are contained in the Selection Sequence Report Interpretation, the selectorId's MUST be identified in the order they are used.

NEW:

• If multiple Selectors are contained in the Selection Sequence Report Interpretation, the selectorId's MUST be identified in the order they are used. The corresponding Options Template Record MUST be transmitted in an Ordered Options Template Set (Set ID 5) as defined in [this document], so that the Collecting Process receives an explicit signal that the ordering of selectorId occurrences reflects the Composite Selector application order.

Updates to RFC 6183 defines the behavior of IPFIX Mediators but predates the Ordered Template Set mechanism defined in this document. This document adds the following normative requirement to Mediator behavior: An IPFIX Mediator that receives Ordered Template Sets (Set ID 4) or Ordered Options Template Sets (Set ID 5) MUST preserve the ordering guarantee when forwarding. The Mediator MUST forward the corresponding Template Records in Ordered Template Sets or Ordered Options Template Sets, and MUST preserve the observation order of fields in the corresponding Data Records. An IPFIX Mediator that cannot preserve the ordering guarantee MUST NOT forward the corresponding Template Records in Ordered Template Sets or Ordered Options Template Sets. Such an IPFIX Mediator MAY forward them in Template Sets (Set ID 2) or Options Template Sets (Set ID 3), in which case no ordering guarantee is signaled to the Collecting Process, or it MAY discard them and the corresponding Data Sets. An IPFIX Mediator preserves the ordering guarantee when, for every Ordered Data Record it exports, the n-th occurrence of an Information Element still corresponds to the n-th observation of that Information Element by the Metering Process that produced the original Data Record. A Mediator can therefore remove an Information Element entirely, remove trailing occurrences of it, or add Information Elements that are not repeated. Removing a leading or intermediate occurrence, or reordering occurrences, does not preserve the guarantee. A Mediator that aggregates Data Records preserves the guarantee only if the aggregated Data Records have identical values for all occurrences of the repeated Information Elements.

Information Elements for Other Layers This document defines no new IPFIX Information Elements. The generic Information Elements for the MPLS label stack and its attributes are defined in . For other protocol layers, no new IPFIX IEs are required: sourceMacAddress (IPFIX IE 56) and destinationMacAddress (IPFIX IE 80) already serve MAC-in-MAC; and IP-level IPFIX IEs (sourceIPv4Address, destinationIPv4Address, and their IPv6 equivalents) already serve VXLAN, GENEVE, GRE, and IP-in-IP encapsulation. In each of these cases, the existing IPFIX IEs can simply be repeated in an Ordered Template Record. The use of repeated Information Elements in Ordered Template Records does not immediately obsolete the existing positional IPFIX IEs. Implementations that do not support ordered export (Set IDs 4 and 5) continue to require the positional IPFIX IEs. Once ordered export is widely deployed, the positional MPLS IPFIX IEs (see ) may be deprecated in a future document.

Operational Considerations This section follows the guidelines of ("Guidelines for Considering Operations and Management of New Protocols and Protocol Extensions") for documenting operational and management considerations in IETF specifications. This document is a protocol extension to ; operational and management considerations that are not specific to ordered export are inherited from and are not repeated here.

Deployment and Coexistence Ordered export is additive, with one exception. Existing Set IDs 2 and 3 are unaffected, and Exporting Processes continue to operate without modification, except that an Exporting Process implementing this document is required to use an Ordered Options Template Set (Set ID 5) for a PSAMP Selection Sequence Report Interpretation Options Template Record that contains multiple selectorId occurrences (see ). The order of the selectorId occurrences is the order in which the Selectors are applied, and only Set ID 5 signals that order to the Collecting Process. For all other Template Records, deployment can proceed incrementally: ordered export can be enabled on a per-Template basis toward specific Collecting Processes, without disrupting any existing IPFIX infrastructure. Operators upgrading PSAMP Exporting Processes SHOULD first ensure that the Collecting Processes receiving the Selection Sequence Report Interpretation support Set ID 5; otherwise the Selection Sequence information may be silently discarded (see ). Operators SHOULD enable ordered export only toward Collecting Processes that are known to support Set ID 4 and Set ID 5. Deployment against unaware Collecting Processes may result in silent loss of entire Ordered Sets, as discussed in .

Configuration Except for the Selection Sequence Report Interpretation Options Template Record described in , an Exporting Process is configured to use an Ordered Template Set (Set ID 4) or an Ordered Options Template Set (Set ID 5) for specific Template Records. No protocol-level negotiation is currently defined. Operators are responsible for ensuring that the Collecting Process is capable of processing the selected Set IDs before enabling ordered export. Ordered export may allow simplification of the Information Element registry: where a deployment previously required multiple position-specific Information Elements (e.g., mplsTopLabelStackSection through mplsLabelStackSection10), a single repeated Information Element in an Ordered Template Record may suffice. This can reduce the number of Information Element identifiers consumed in the IPFIX Information Elements registry . It does not reduce the size of Template Records, because each occurrence of an Information Element has its own Field Specifier.

Verification of Correct Operation End-to-end verification of ordered export requires confirming correct behavior across all three components of the ordering chain:

• Metering Process: records repeated Information Element values in the order they are observed. This is internal to the implementation and is not directly observable on the wire.
• Exporting Process: transmits Ordered Template Sets (Set ID 4) or Ordered Options Template Sets (Set ID 5) and constructs the corresponding Data Records with field values in Metering Process observation order. This can be verified by inspecting the IPFIX Messages on the wire.
• Collecting Process: tracks which Template IDs were received in Ordered Template Sets and interprets the n-th occurrence of a repeated Information Element as the n-th Metering Process observation. Correct interpretation can be verified by comparing collected data against known traffic (e.g., known MPLS label stack or encapsulation layer ordering in a test environment).

Fault Management The primary fault scenario for ordered export is a mismatch between the Exporting Process and the Collecting Process: the Exporting Process sends Ordered Sets that the Collecting Process does not understand. Section 3.3.2 of mandates that "the Length value MUST be used to determine the position of the next Set," providing the mechanism for a Collecting Process that does not support Set ID 4 or 5 to skip the unknown Set and continue processing normally. However, does not specify what a Collecting Process MUST or SHOULD do upon receipt of an unknown Set ID -- whether to skip silently, log the event, or terminate the Transport Session. As a result, an RFC 7011-compliant Collecting Process may silently discard the Set without any notification to the Exporting Process or to the operator. This document addresses this gap; see . Operators SHOULD monitor for unexpected loss of Flow Records that may indicate silent Set discard at an unaware Collecting Process. If suspected, ordered export SHOULD be disabled for that destination until the Collecting Process is confirmed to support Set ID 4 and Set ID 5.

Performance The ordered export mechanism specified in this document introduces no additional overhead on the wire: Ordered Template Sets and Ordered Options Template Sets use the same wire format as their Set ID 2 and Set ID 3 counterparts. The only additional cost is the Collecting Process maintaining per-Template-ID state to track whether each Template was received in an Ordered or unordered Set, which is negligible. The Metering Process, Exporting Process, and IPFIX Mediators may incur an implementation cost to preserve observation order when recording repeated Information Element values, depending on their internal architecture. This is an implementation consideration and is outside the scope of this document.

Security Considerations This document does not change the trust model of the IPFIX protocol. The security considerations of apply. An Exporting Process or an IPFIX Mediator that uses an Ordered Template Set (Set ID 4) or an Ordered Options Template Set (Set ID 5) but fails to preserve the Metering Process observation order will cause the Collecting Process to misinterpret the exported data. In particular, analytics systems that rely on the ordering to identify encapsulation layers or label stack positions may produce incorrect results. Operators SHOULD ensure that the Exporting Process, the Metering Process and any IPFIX Mediator in the path are correctly coordinated before using ordered export. The Set ID now determines how the Collecting Process interprets the order of repeated Information Elements. A modification of the Set ID in transit (2 to 4, 3 to 5, or the reverse) would change this interpretation without otherwise corrupting the message. The use of TLS or DTLS as described in Section 11 of protects against such modification.

IANA Considerations

IPFIX Set IDs Registry This document requests IANA to allocate the following values in the "IPFIX Set IDs" subregistry of the "IP Flow Information Export (IPFIX) Entities" registry :

Value	Set Identifier	Reference
4	Ordered Template Set	This document
5	Ordered Options Template Set	This document

After this allocation, IANA is requested to record the remaining range as "6-255, Unassigned".

References Normative References In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This document specifies the export of packet information from a Packet SAMPling (PSAMP) Exporting Process to a PSAMP Collecting Process. For export of packet information, the IP Flow Information eXport (IPFIX) protocol is used, as both the IPFIX and PSAMP architecture match very well, and the means provided by the IPFIX protocol are sufficient. The document specifies in detail how the IPFIX protocol is used for PSAMP export of packet information. [STANDARDS-TRACK] This document describes a framework for IP Flow Information Export (IPFIX) Mediation. This framework extends the IPFIX reference model specified in RFC 5470 by defining the IPFIX Mediator components. This document is not an Internet Standards Track specification; it is published for informational purposes. This document specifies the IP Flow Information Export (IPFIX) protocol, which serves as a means for transmitting Traffic Flow information over the network. In order to transmit Traffic Flow information from an Exporting Process to a Collecting Process, a common representation of flow data and a standard means of communicating them are required. This document describes how the IPFIX Data and Template Records are carried over a number of transport protocols from an IPFIX Exporting Process to an IPFIX Collecting Process. This document obsoletes RFC 5101. The Intermediate Flow Selection Process is the process of selecting a subset of Flows from all observed Flows. The Intermediate Flow Selection Process may be located at an IP Flow Information Export (IPFIX) Exporter or Collector, or within an IPFIX Mediator. It reduces the effort of post-processing Flow data and transferring Flow Records. This document describes motivations for using the Intermediate Flow Selection process and presents Intermediate Flow Selection techniques. It provides an information model for configuring Intermediate Flow Selection Process techniques and discusses what information about an Intermediate Flow Selection Process should be exported. Informative References This memo defines the IP Flow Information eXport (IPFIX) architecture for the selective monitoring of IP Flows, and for the export of measured IP Flow information from an IPFIX Device to a Collector. This memo provides information for the Internet community. This document specifies a framework for the PSAMP (Packet SAMPling) protocol. The functions of this protocol are to select packets from a stream according to a set of standardized Selectors, to form a stream of reports on the selected packets, and to export the reports to a Collector. This framework details the components of this architecture, then describes some generic requirements, motivated by the dual aims of ubiquitous deployment and utility of the reports for applications. Detailed requirements for selection, reporting, and exporting are described, along with configuration requirements of the PSAMP functions. This memo provides information for the Internet community. This document describes Sampling and Filtering techniques for IP packet selection. It provides a categorization of schemes and defines what parameters are needed to describe the most common selection schemes. Furthermore, it shows how techniques can be combined to build more elaborate packet Selectors. The document provides the basis for the definition of information models for configuring selection techniques in Metering Processes and for reporting the technique in use to a Collector. [STANDARDS-TRACK] This document specifies an extension to the IP Flow Information Export (IPFIX) protocol specification in RFC 5101 and the IPFIX information model specified in RFC 5102 to support hierarchical structured data and lists (sequences) of Information Elements in data records. This extension allows definition of complex data structures such as variable-length lists and specification of hierarchical containment relationships between Templates. Finally, the semantics are provided in order to express the relationship among multiple list elements in a structured data record. [STANDARDS-TRACK] This document defines the data types and management policy for the information model for the IP Flow Information Export (IPFIX) protocol. This information model is maintained as the IANA "IPFIX Information Elements" registry, the initial contents of which were defined by RFC 5102. This information model is used by the IPFIX protocol for encoding measured traffic information and information related to the traffic Observation Point, the traffic Metering Process, and the Exporting Process. Although this model was developed for the IPFIX protocol, it is defined in an open way that allows it to be easily used in other protocols, interfaces, and applications. This document obsoletes RFC 5102. This document describes Virtual eXtensible Local Area Network (VXLAN), which is used to address the need for overlay networks within virtualized data centers accommodating multiple tenants. The scheme and the related protocols can be used in networks for cloud service providers and enterprise data centers. This memo documents the deployed VXLAN protocol for the benefit of the Internet community. Network virtualization involves the cooperation of devices with a wide variety of capabilities such as software and hardware tunnel endpoints, transit fabrics, and centralized control clusters. As a result of their role in tying together different elements of the system, the requirements on tunnels are influenced by all of these components. Therefore, flexibility is the most important aspect of a tunneling protocol if it is to keep pace with the evolution of technology. This document describes Geneve, an encapsulation protocol designed to recognize and accommodate these changing capabilities and needs. This document introduces new IP Flow Information Export (IPFIX) Information Elements (IEs) to identify a set of information related to Segment Routing over IPv6 (SRv6) such as data contained in a Segment Routing Header (SRH), the SRv6 control plane, and the SRv6 Endpoint behavior that traffic is being forwarded with. ZTE Corporation ZTE Corporation Everything OPS & Arrcus Ciena Deutsche Telekom ZTE Corporation Everything OPS Cisco Old Dog Consulting Nokia Blue Fern Consulting ZTE New Protocols or Protocol Extensions are best designed with due consideration of the functionality needed to operate and manage them. Retrofitting operations and management considerations is suboptimal. The purpose of this document is to provide guidance to authors and reviewers on what operational and management aspects should be addressed when defining New Protocols or Protocol Extensions. This document obsoletes RFC 5706, replacing it completely and updating it with new operational and management techniques and mechanisms. It also introduces a requirement to include an "Operational Considerations" section in new RFCs in the IETF Stream. IANA

Solution Space This section analyzes candidate solutions. The analysis is presented to inform discussion and justify the specification in .

• Note: The detailed analysis in this section is intended to support Working Group discussion. Once the Working Group has reached consensus on the solution, this section may be condensed into a summary.

Updating RFC 7011 SHOULD to MUST Alone (Rejected) The most minimal approach would be to update Section 8 of by strengthening the existing SHOULD to a MUST, requiring all Exporting Processes to guarantee that repeated Information Elements follow the Metering Process observation order. The fundamental limitation of this approach is that it constrains the Exporting Process but provides no signal to the Collecting Process. Even if every Exporting Process were compliant with the updated requirement, the Collecting Process would still have no per-Template indication of whether a given Template Record was sent by an updated Exporting Process (ordering guaranteed) or by an older implementation that predates the update (ordering not guaranteed). Without such a signal, the Collecting Process cannot safely interpret the n-th occurrence of a repeated Information Element as the n-th Metering Process observation. Two additional considerations reinforce the rejection. First, Sections 4.2 and 4.3 of explicitly state that "the order of the Information Elements in the Template Records is not guaranteed"; a SHOULD-to-MUST change in Section 8 alone would leave these statements in direct conflict with the new requirement. Second, Section 2.2 ("IPFIX Limitations") of notes that "some encoding optimizations are based on the permutation of Information Element order"; a universal MUST would prohibit such optimizations across all Templates containing repeated Information Elements, regardless of whether ordering is semantically significant. This approach is therefore rejected.

RFC 6313 Structured Data with the "ordered" Semantic (Rejected) This option is analyzed in detail because has been cited as a solution to the ordering problem (see Introduction). The analysis below shows that it does not provide the required guarantee and is therefore rejected. defines structured data types -- basicList, subTemplateList, and subTemplateMultiList -- and associates them with list semantics. One defined semantic is:

• The "ordered" structured data type semantic specifies that elements from the list in the structured data are ordered.

Using this mechanism, an Exporting Process could encode a sequence of ordered MPLS label values (see ) as a basicList with the "ordered" semantic: , , ) ]]> Similarly, multi-layer encapsulation fields could be encoded in a subTemplateMultiList with the "ordered" semantic. The fundamental limitation of this approach is that the "ordered" semantic is an export ordering guarantee, not a Metering Process observation ordering guarantee. Section 4.4.6 of states only that "elements from the list in the structured data are ordered" -- it does not state that the order reflects the sequence in which the Metering Process makes its observations. Furthermore, structured data, when carried in an IPFIX Template Record, still falls under the general rule of Section 8 of : "the different occurrences of this Information Element SHOULD follow the logical order of their treatments by the Metering Process." does not upgrade this SHOULD to a MUST, nor does it bind the "ordered" semantic to Metering Process observation order. Since IPFIX is an export protocol, governing the wire format between the Exporting Process and the Collecting Process, it cannot by itself make claims about the internal behavior of the Metering Process. A Collecting Process that receives a basicList or subTemplateMultiList with the "ordered" semantic therefore cannot safely conclude that element N corresponds to the N-th observation by the Metering Process. This is precisely the guarantee this document requires, and does not provide it. This approach is therefore rejected.

New IPFIX Version 11 (Rejected) A new IPFIX version implies a broad overhaul of the protocol and would carry community expectations far beyond the narrow scope of this document. The risk of scope creep is significant. Beyond scope, a version bump would be far more disruptive than new Set IDs. A Collecting Process that receives an IPFIX Message with an unknown version number has no choice but to discard the entire message. This means that introducing a new IPFIX version would cause every existing Collecting Process to reject all IPFIX Messages -- including those carrying unordered Data Sets -- until it is updated to support the new version. The deployment impact would be severe and disproportionate to the problem being solved. By contrast, new Set IDs are additive. Section 3.3.2 of reserves Set IDs 4 through 255 for future use and mandates that "the Length value MUST be used to determine the position of the next Set." A Collecting Process that does not support Set ID 4 or 5 can therefore use the Set Length field to skip the unknown Set and continue processing the remainder of the IPFIX Message normally. All prior IPFIX extensions -- including -- have been deployed without requiring a version bump. This approach is therefore rejected.

Marker Information Element (Evaluated) Define a new Information Element whose presence in an (Options) Template Record indicates that the Information Elements following it (until the next marker) are to be interpreted as ordered. This approach is similar to the encapLayerTop and encapLayer2 Information Elements proposed in for encapsulation layer identification. Advantages:

• A Collecting Process that does not understand the marker Information Element MAY discard only that one Information Element (per Section 9 of ), retaining the other fields in the Data Record.

Disadvantages:

• The semantic of other Information Elements becomes dependent on the presence and position of the marker Information Element, creating an implicit dependency between IPFIX IEs that is not expressible in the IPFIX Information Model .
• The approach is less clean than a dedicated Set type from a protocol design perspective.

Options Template Indicator (Evaluated) Define a new Options Template Record with a scope of Template ID, containing a bitmap Information Element that identifies, for each field in the referenced Template Record, whether that field belongs to an ordered sequence. This is analogous to the flowKeyIndicator Information Element (IPFIX IE 173), which uses an Options Template to convey which fields of a referenced Template Record are Flow Keys. Advantages:

• Does not require changes to the Data Set format.
• The Options Template Record can be transmitted once per Template and cached by the Collecting Process.

Disadvantages:

• Requires the Collecting Process to correlate the Options Template record with the Data Template before it can correctly interpret Data Records, adding a level of indirection and a potential for synchronization issues.
• More complex to implement than a dedicated Set type.

New Ordered Set Type Define new Set IDs to indicate that the ordering of Information Elements within the corresponding Data Records is guaranteed to reflect the Metering Process observation order. This approach extends the existing Set ID structure of :

Set ID	Name
2	Template Set (existing)
3	Options Template Set (existing)
4	Ordered Template Set (new)
5	Ordered Options Template Set (new)

Ordered Template Records (Set ID 4) and Ordered Options Template Records (Set ID 5) use the same wire format as their existing counterparts (Set ID 2 and Set ID 3 respectively), with the addition of the ordering semantic on the corresponding Data Records. Advantages:

• Provides an explicit, unambiguous per-Set signal that ordering is guaranteed, without relying on the Collecting Process to parse an in-band marker or to maintain an out-of-band correlation table.
• No structured data complexity is required; existing flat Template and Data Record formats are reused.
• Backwards compatible: existing Set IDs 2 and 3 are unaffected and continue to be used for cases where ordering is not guaranteed or not relevant.
• Generalizable to any stacked or layered protocol structure without requiring new position-specific Information Element definitions.

Disadvantages:

• Requires both Exporting Process and Collecting Process updates to support the new Set IDs.
• A Collecting Process that does not support Set ID 4 or 5 may discard the entire Set upon receipt, resulting in a complete loss of the Data Records in that Set without any log information. Section 3.3.2 of does not specify the behavior of a Collecting Process upon receipt of an unknown Set ID.

New Ordered List Data Type (Evaluated) Define a new structured data type, for example an orderedBasicList, based on the basicList of , whose semantic signals to the Collecting Process that the elements reflect the Metering Process observation order. does not provide this guarantee (see ), but a new data type could. The position of each element could also be encoded, for example as (position, value) pairs, so that a Collecting Process can detect elements that were not exported (for instance, positions 1 and 3 exported but not 2). Two cases of repeated Information Elements illustrate the difference with Ordered Template Sets:

1. Information Elements of a single technology, with an unbounded number of occurrences, such as the SRv6 Segment List of or the MPLS label stack. A list carries a variable number of elements in a single Template Record.
2. The same Information Element originating from different technologies, such as destinationIPv6Address in the outer and inner headers of a tunnel. The Metering Process is configured for specific encapsulation protocols, so the number of occurrences is known, or bounded. Repeated Information Elements in an Ordered Template Record cover this case with a fixed number of occurrences.

Advantages:

• A variable number of elements in a single Template Record, instead of one Template Record per number of occurrences.
• Position encoding makes non-consecutive elements explicit.

Disadvantages:

• Requires a new data type, and Exporting Process and Collecting Process support for it.
• A list covers one Information Element. Pairing elements of different lists (for example the source and destination addresses of the same layer) requires the position to be defined as shared across lists.

This approach is not specified in this document.

Migration from Positional to Repeated Information Elements This appendix is informative. It provides guidance for migrating existing deployments from positional Information Elements to repeated Information Elements used with the ordered export mechanism specified in this document. The mapping for the MPLS Information Elements is provided in . Positional Information Elements were defined as a workaround for the absence of an ordering guarantee in IPFIX (see ). With ordered export, a single generic Information Element repeated N times in an Ordered Template Record replaces N positional Information Elements, where the N-th occurrence corresponds to the N-th observation at the Observation Point. Migration requires both Exporting Process and Collecting Process updates to support Ordered Template Sets (Set ID 4) or Ordered Options Template Sets (Set ID 5). Until ordered export is universally deployed, both positional and generic forms must be supported concurrently.

Changes Note to the RFC Editor: please remove this appendix before publication.

Changes from 00 to 01

• Moved the solution-space analysis to and dropped the "Recommended" label.
• Performance: added the Exporting Process and IPFIX Mediators to the implementation-cost paragraph.
• Editorial: renumbered sections, marked "[this document]" as a placeholder in the RFC 5476 update.

Changes from 01 to 02

• Moved the generic MPLS Information Elements, their IANA requests and the MPLS migration table to .
• Removed the erroneous Q-in-Q examples.
• Updated Section 8.1 of : the Set ID of a Template Withdrawal
• Updated Template lifecycle (Section 4.2, 4.3).
• Mediators (Section 4.3, 4.6): reworded the rule
• Section 4.3: stated the observation direction (outermost to innermost, top to bottom).
• Section 5.2: replaced the Template Record size claim with the Information Element registry simplification.
• Editorial changes

Acknowledgements The authors thank Yao Liu and Taoran Zhou for their work on , which identified the multi-layer encapsulation use case and motivated the broader problem statement addressed in this document.