Network Working Group T. Graf Internet-Draft Swisscom Intended status: Standards Track B. Claise Expires: 4 January 2025 Huawei A. Huang Feng INSA-Lyon 3 July 2024 Support of Observation Timestamp in YANG-Push Notifications draft-tgraf-netconf-yang-push-observation-time-02 Abstract This document extends YANG-Push Notifications with the YANG objects observation timestamp and point-in-time for streaming update YANG- Push notifications. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 4 January 2025. Copyright Notice Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Graf, et al. Expires 4 January 2025 [Page 1] Internet-Draft YANG Notifications Observation Time July 2024 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Terminologies . . . . . . . . . . . . . . . . . . . . . . . . 3 3. Extend the Streaming Update Notifications . . . . . . . . . . 3 4. YANG-Push capabilities . . . . . . . . . . . . . . . . . . . 7 5. The "ietf-yp-observation" YANG Module . . . . . . . . . . . . 7 5.1. Data Model Overview . . . . . . . . . . . . . . . . . . . 7 5.2. YANG Module . . . . . . . . . . . . . . . . . . . . . . . 10 6. Implementation Status . . . . . . . . . . . . . . . . . . . . 13 6.1. 6WIND VSR . . . . . . . . . . . . . . . . . . . . . . . . 13 7. Operational Considerations . . . . . . . . . . . . . . . . . 13 7.1. Time Series Late Indexing . . . . . . . . . . . . . . . . 13 8. Security Considerations . . . . . . . . . . . . . . . . . . . 13 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13 9.1. URI . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 9.2. YANG module name . . . . . . . . . . . . . . . . . . . . 14 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 14 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 14 11.1. Normative References . . . . . . . . . . . . . . . . . . 14 11.2. Informative References . . . . . . . . . . . . . . . . . 15 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 15 1. Introduction To correlate network data among different Network Telemetry planes as described in Section 3.1 of [RFC9232] or among different YANG-Push subscription types defined in Section 3.1 of [RFC8641], observation timestamp describes when the state change was observed or when the data was accounted. This is essential for understanding the timely relationship among these different planes and YANG push subscription types. With [I-D.tgraf-netconf-notif-sequencing] the delay between the YANG Notification export and the arrival at the downstream system storing the data can be measured. With observation timestamp described in this document, the delay between the time of the network observation and the data export of the YANG push publisher process can be measured as well, extending the delay measurement scope from the time the network observation and storing the data. When the time bucket length in a time series database and the periodical YANG push subscription time is identical, the eventTime of the netconf notification message header is used for indexing, there is variable delay between the observation timestamp and the eventTime, and the anchor-time as described in Section 4.2 of [RFC8641] is close to the time bucket boundaries, a time bucket is going to have periodically 0 or 2 measurements indexed instead of 1. Graf, et al. Expires 4 January 2025 [Page 2] Internet-Draft YANG Notifications Observation Time July 2024 Leading to inconsistent accounting errors in the time series database. This problem is resolved by using the observation timestamp instead of the eventTime for time series database indexing. By extending YANG-Push Notifications with the observation timestamp YANG object and a point-in-time enumeration for streaming update YANG-Push notifications it is ensured that the observation timestamp is always present with the best available time, it can be therefore used unconditionally in the data processing chain and with the point- in-time enumeration there is semantic describing at which point in time the observation timestamp was observed. 2. Terminologies The following terms are defined in [RFC8639] and are not redefined here: Publisher Receiver Subscription In addition, this document defines the following terms: Timestamp: Describes when the metric(s) (reported in YANG-Push subscriptions) were observed for a given Subscription ID and notification message. Point-in-time: Describes at which point in time the metric(s) reported in YANG-Push subscriptions) were observed for a given Subscription ID and notification message. 3. Extend the Streaming Update Notifications Besides the Subscription ID as described in Section 3.7 of [RFC8641], the following network observation time metadata objects are part of "push-update" and "push-change-update" notifications. timestamp: States the measurement observation time for the "push- update" notification in a "periodical" and for the "push-change- update" notification in a "on-change" subscription. Graf, et al. Expires 4 January 2025 [Page 3] Internet-Draft YANG Notifications Observation Time July 2024 By comparing the observation timestamp of two "push-update" notifications in a periodical subscription, the collector can deduce the actual cadence of the measurements, and compare it with the subscription configuration. In case of an "on-change" subscription it states the time when the network state change was observed. point-in-time: The enumeration states at which point in time the value of the observation timestamp was observed. Choices are: current-accounting states for "periodical" subscriptions, the point-in-time where the metrics are being polled and observed. initial-state states for "on-change sync on start" subscriptions, the initial point in time when the subscription was established and the state was observed. state-changed states for "on-change" and "on-change sync on start" subscriptions, the point in time when the state change was observed after the subscription was established. Figure 1 provides an example timeline of events to describe observation timestamp and point-in-time for "on-change sync on start" subscriptions. At "T2" when the "on-change sync on start" subscription is established, the timestamp of "T2" is used in observation timestamp and "point-in-time" is set to "initial-state". At "T3" and "T4" the timestamp of "T3" respectively "T4" is used in observation timestamp and point-in-time is set to "state-changed". Timeline ----------------------------------------------------------------------> | (T1) Interface | (T2) YANG-Push | (T3) Interface | (T4) Interface | state changed | "on-change sync | state changed | state changed | to "Up". | on start" | to "Down". | to "Up". | | subscription for | | | | interface state | | | | is established. | | v v v v Figure 1: Example timeline for On-Change Sync on Start Subscription Graf, et al. Expires 4 January 2025 [Page 4] Internet-Draft YANG Notifications Observation Time July 2024 ========== NOTE: '\' line wrapping per RFC 8792) =========== { "ietf-notification:notification": { "eventTime": "2023-03-25T08:30:11.22Z", "ietf-notification-sequencing:sysName": "example-router", "ietf-notification-sequencing:sequenceNumber": 1, "ietf-yang-push:push-change-update": { "id": 2222, "ietf-yp-observation:timestamp": \ "2023-03-25T08:30:11.22Z", "ietf-yp-observation:point-in-time": \ "state-changed", "datastore-contents": { "yang-patch": { "patch-id": "patch_54", "comment": "Changing encoding to JSON and increasing \ the period to 10 minutes", "edit": [ { "edit-id": "id_change_1", "operation": "merge", "target": "/ietf-subscribed-notifications\:subs\ criptions/subscription[id=2222]", "value": { "ietf-subscribed-notifications:encoding": \ "ietf-subscribed-notifications:encode-json", "ietf-yang-push:periodic": { "period": 60000 } } } ] } } } } } Figure 2: JSON Push Example for a push-change-update notification message Figure 3 provides an example timeline of events to describe point-in- time for "periodical" subscriptions. The timestamp of observation timestamp in "T2" is set at the start of the polling of the YANG data store to "T2" and point-in-time is set "current-accounting". At "T3" the timestamp of observation timestamp of "T3" is used. At "T4" the timestamp of observation timestamp of "T4" is used. Graf, et al. Expires 4 January 2025 [Page 5] Internet-Draft YANG Notifications Observation Time July 2024 Timeline ----------------------------------------------------------------------> | (T1) YANG-Push | (T2) YANG-Push | (T3) YANG-Push | (T4) YANG-Push | "periodical" | "periodical" | "periodical" | "periodical" | subscription | current | current | current | for interface | accounting | accounting. | accounting. | counter is | | | | established | | | v v v v Figure 3: Example timeline for Periodical Subscription Figure 4 provides an example of a JSON encoded, [RFC8259], "push- change-update" notification message over HTTPS-based [I-D.ietf-netconf-https-notif] or UDP-based [I-D.ietf-netconf-udp-notif] transport with [I-D.tgraf-netconf-notif-sequencing] as extension. ========== NOTE: '\' line wrapping per RFC 8792) =========== { "ietf-notification:notification": { "eventTime": "2023-03-25T08:30:11.22Z", "ietf-notification-sequencing:sysName": "example-router", "ietf-notification-sequencing:sequenceNumber": 1, "ietf-yang-push:push-update": { "id": 6666, "ietf-yp-observation:timestamp": \ "2023-03-25T08:30:11.22Z", "ietf-yp-observation:point-in-time": \ "current-accounting", "datastore-contents": { "ietf-interfaces:interfaces": [ { "interface": { "name": "eth0", "type": "iana-if-type:ethernetCsmacd", "oper-status": "up", "mtu": 1500 } } ] } } } } Figure 4: JSON Push Example for a push-update notification message Graf, et al. Expires 4 January 2025 [Page 6] Internet-Draft YANG Notifications Observation Time July 2024 4. YANG-Push capabilities YANG-related system capabilities can be discovered by subscribing to the datastore defined in [RFC9196]. This document augments the "ietf-notification-capabilities" defined in Section 5 of [RFC9196] so that the YANG-Push receiver can learn the capabilities defined in this documents through the datastore. The definition of this YANG augmentation is specified in Section 5. 5. The "ietf-yp-observation" YANG Module This YANG module augments the "ietf-yang-push" module with the observation timestamp and point-in-time in the "push-update" and "push-change-update" streaming update notifications. 5.1. Data Model Overview 5.1.1. Tree View This "ietf-yp-observation" YANG module augments the "ietf-yang-push" YANG module specified in Section 5 of [RFC8641] adding the "timestamp" and the "point-in-time" leaves and augments "ietf- notification-capabilities" defined in [RFC9196] to add a system level capability. module: ietf-yp-observation augment /yp:push-update: +--ro timestamp? yang:date-and-time +--ro point-in-time? enumeration augment /yp:push-change-update: +--ro timestamp? yang:date-and-time +--ro point-in-time? enumeration augment /sysc:system-capabilities/notc:subscription-capabilities: +--ro yang-push-observation-supported? inotifseq:notification-support {yang-push-observation-timestamp}? 5.1.2. Full Tree View The following YANG tree diagram [RFC8340] shows the complete notification defined in the "ietf-yang-push" module: Graf, et al. Expires 4 January 2025 [Page 7] Internet-Draft YANG Notifications Observation Time July 2024 module: ietf-yang-push rpcs: +---x resync-subscription {on-change}? +---w input +---w id sn:subscription-id notifications: +---n push-update | +--ro id? sn:subscription-id | +--ro datastore-contents? | +--ro incomplete-update? empty | +--ro ypot:timestamp? yang:date-and-time | +--ro ypot:point-in-time? enumeration +---n push-change-update {on-change}? +--ro id? sn:subscription-id +--ro datastore-changes +--ro incomplete-update? empty +--ro ypot:timestamp? yang:date-and-time +--ro ypot:point-in-time? enumeration The following YANG tree diagram [RFC8340] shows the complete "ietf- system-capabilities" module with the augmentations defined in "ietf- yp-observation" module: Graf, et al. Expires 4 January 2025 [Page 8] Internet-Draft YANG Notifications Observation Time July 2024 module: ietf-system-capabilities +--ro system-capabilities +--ro datastore-capabilities* [datastore] | +--ro datastore | | -> /yanglib:yang-library/datastore/name | +--ro per-node-capabilities* [] | +--ro (node-selection)? | | +--:(node-selector) | | +--ro node-selector? | | nacm:node-instance-identifier | +--ro notc:subscription-capabilities | +--ro notc:max-nodes-per-update? uint32 | +--ro notc:periodic-notifications-supported? | | notification-support | +--ro (notc:update-period)? | | +--:(notc:minimum-update-period) | | | +--ro notc:minimum-update-period? uint32 | | +--:(notc:supported-update-period) | | +--ro notc:supported-update-period* uint32 | +--ro notc:on-change-supported? | | notification-support {yp:on-change}? | +--ro notc:minimum-dampening-period? uint32 | | {yp:on-change}? | +--ro notc:supported-excluded-change-type* union | {yp:on-change}? +--ro notc:subscription-capabilities +--ro notc:max-nodes-per-update? uint32 +--ro notc:periodic-notifications-supported? | notification-support +--ro (notc:update-period)? | +--:(notc:minimum-update-period) | | +--ro notc:minimum-update-period? uint32 | +--:(notc:supported-update-period) | +--ro notc:supported-update-period* uint32 +--ro notc:on-change-supported? | notification-support {yp:on-change}? +--ro notc:minimum-dampening-period? uint32 | {yp:on-change}? +--ro notc:supported-excluded-change-type* union | {yp:on-change}? +--ro inotifseq:sysname-sequence-supported? | notification-support {sysname-sequence}? +--ro ypot:yang-push-observation-supported? inotifseq:notification-support {yang-push-observation-timestamp}? Graf, et al. Expires 4 January 2025 [Page 9] Internet-Draft YANG Notifications Observation Time July 2024 5.2. YANG Module The YANG module has one leave augmenting the model of Subscription to YANG Notifications [RFC8639]. file "ietf-yp-observation@2024-06-18.yang" module ietf-yp-observation { yang-version 1.1; namespace "urn:ietf:params:xml:ns:yang:ietf-yp-observation"; prefix ypot; import ietf-yang-types { prefix yang; reference "RFC 6991: Common YANG Data Types"; } import ietf-yang-push { prefix yp; reference "RFC 8641: Subscription to YANG Notifications for Datastore Updates"; } import ietf-system-capabilities { prefix sysc; reference "RFC 9196: YANG Modules Describing Capabilities for Systems and Datastore Update Notifications"; } import ietf-notification-capabilities { prefix notc; reference "RFC 9196: YANG Modules Describing Capabilities for Systems and Datastore Update Notifications"; } import ietf-notification-sequencing { prefix inotifseq; reference "draft-tgraf-netconf-notif-sequencing-05: YANG Notifications Sequencing"; } organization "IETF NETCONF (Network Configuration) Working Group"; contact "WG Web: WG List: Authors: Thomas Graf Benoit Claise Graf, et al. Expires 4 January 2025 [Page 10] Internet-Draft YANG Notifications Observation Time July 2024 Alex Huang Feng "; description "Defines YANG-Push event notification header with the observation time in streaming update notifications. Copyright (c) 2024 IETF Trust and the persons identified as authors of the code. All rights reserved. Redistribution and use in source and binary forms, with or without modification, is permitted pursuant to, and subject to the license terms contained in, the Revised BSD License set forth in Section 4.c of the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info). This version of this YANG module is part of RFC XXXX; see the RFC itself for full legal notices."; revision 2024-06-18 { description "First revision"; reference "RFC XXXX: Support of YANG-Push Notifications Observation Time"; } feature yang-push-observation-timestamp { description "This feature indicates the YANG-push Notifications support the observation timestamps in streaming update notifications."; } grouping yang-push-observation { description "This grouping adds the observation timestamp for the observed metrics."; leaf timestamp { type yang:date-and-time; description "This is the time when metrics were observed."; } leaf point-in-time { type enumeration { enum current-accounting { description "For periodical subscriptions, the point-in-time where the metrics are being polled and observed."; } enum initial-state { description "For 'on-change sync on start' subscriptions, the Graf, et al. Expires 4 January 2025 [Page 11] Internet-Draft YANG Notifications Observation Time July 2024 initial point in time when the subscription was established and the state was observed."; } enum state-changed { description "For 'on-change sync on start' subscriptions, the point in time when the state change was observed after the subscription was established."; } } description "This describes at which point in time the metrics were observed"; } } // Event notifications augment "/yp:push-update" { description "This augmentation adds the observation timestamp of the accounted metrics in the push-update notification."; uses ypot:yang-push-observation; } augment "/yp:push-change-update" { description "This augmentation adds the observation timestamp of the event in the push-change-update notification."; uses ypot:yang-push-observation; } // Event capabilities augment "/sysc:system-capabilities/notc:subscription-capabilities" { description "Add system level capabilities"; leaf yang-push-observation-supported { if-feature "yang-push-observation-timestamp"; type inotifseq:notification-support; description "Specifies whether the publisher supports exporting observation-timestamp and point-in-time in notifications."; reference "RFC XXXX: YANG Notifications Observation Time"; } } } Graf, et al. Expires 4 January 2025 [Page 12] Internet-Draft YANG Notifications Observation Time July 2024 6. Implementation Status Note to the RFC-Editor: Please remove this section before publishing. 6.1. 6WIND VSR 6WIND implemented this document for a YANG Push publisher on UDP-based Transport for Configured Subscriptions [I-D.ietf-netconf-udp-notif] in their VSR platform. 7. Operational Considerations 7.1. Time Series Late Indexing Since the timespan between network state change and subscription establishment as exampled in timeline with "T1" and "T2" in Figure 1 for "on-change sync on start" subscriptions could be very long when the network node uptime and thus the state change was long time ago and the subscription is establish very recent, a time series database may discard the notification message at ingestion since the observation timestamp is in the very past and therefore would have potential performance impact on the time series database indexing. This can be circumvent by using the eventTime for indexing or a change to "on-change" subscriptions instead. 8. Security Considerations The security considerations for the YANG notifications subscription mechanism are described in [RFC8641]. This document adds no additional security considerations. 9. IANA Considerations This document describes the URI used for the IETF XML Registry and registers a new YANG module name. 9.1. URI IANA is requested to add this document as a reference in the following URI in the IETF XML Registry [RFC3688]. URI: urn:ietf:params:xml:ns:yang:ietf-yp-observation Registrant Contact: The IESG. XML: N/A; the requested URI is an XML namespace. Reference: RFC5277; RFC-to-be Graf, et al. Expires 4 January 2025 [Page 13] Internet-Draft YANG Notifications Observation Time July 2024 9.2. YANG module name This document registers the following YANG module in the YANG Module Names Registry [RFC6020], within the "YANG Parameters" registry: name: ietf-yp-observation namespace: urn:ietf:params:xml:ns:yang:ietf-yp-observation prefix: ypot Reference: RFC-to-be 10. Acknowledgements The authors would like to thank Rob Wilton, Nick Corran, Pierre Francois Ahmed Elhassany, Jean Quilbeuf and Marco Tollini for their review and valuable comments. 11. References 11.1. Normative References [RFC3688] Mealling, M., "The IETF XML Registry", BCP 81, RFC 3688, DOI 10.17487/RFC3688, January 2004, . [RFC6020] Bjorklund, M., Ed., "YANG - A Data Modeling Language for the Network Configuration Protocol (NETCONF)", RFC 6020, DOI 10.17487/RFC6020, October 2010, . [RFC8340] Bjorklund, M. and L. Berger, Ed., "YANG Tree Diagrams", BCP 215, RFC 8340, DOI 10.17487/RFC8340, March 2018, . [RFC8639] Voit, E., Clemm, A., Gonzalez Prieto, A., Nilsen-Nygaard, E., and A. Tripathy, "Subscription to YANG Notifications", RFC 8639, DOI 10.17487/RFC8639, September 2019, . [RFC8641] Clemm, A. and E. Voit, "Subscription to YANG Notifications for Datastore Updates", RFC 8641, DOI 10.17487/RFC8641, September 2019, . Graf, et al. Expires 4 January 2025 [Page 14] Internet-Draft YANG Notifications Observation Time July 2024 [RFC8792] Watsen, K., Auerswald, E., Farrel, A., and Q. Wu, "Handling Long Lines in Content of Internet-Drafts and RFCs", RFC 8792, DOI 10.17487/RFC8792, June 2020, . [RFC9196] Lengyel, B., Clemm, A., and B. Claise, "YANG Modules Describing Capabilities for Systems and Datastore Update Notifications", RFC 9196, DOI 10.17487/RFC9196, February 2022, . 11.2. Informative References [I-D.ietf-netconf-https-notif] Jethanandani, M. and K. Watsen, "An HTTPS-based Transport for YANG Notifications", Work in Progress, Internet-Draft, draft-ietf-netconf-https-notif-15, 1 February 2024, . [I-D.ietf-netconf-udp-notif] Zheng, G., Zhou, T., Graf, T., Francois, P., Feng, A. H., and P. Lucente, "UDP-based Transport for Configured Subscriptions", Work in Progress, Internet-Draft, draft- ietf-netconf-udp-notif-13, 27 June 2024, . [I-D.tgraf-netconf-notif-sequencing] Graf, T., Quilbeuf, J., and A. H. Feng, "Support of Hostname and Sequencing in YANG Notifications", Work in Progress, Internet-Draft, draft-tgraf-netconf-notif- sequencing-06, 29 June 2024, . [RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data Interchange Format", STD 90, RFC 8259, DOI 10.17487/RFC8259, December 2017, . [RFC9232] Song, H., Qin, F., Martinez-Julia, P., Ciavaglia, L., and A. Wang, "Network Telemetry Framework", RFC 9232, DOI 10.17487/RFC9232, May 2022, . Authors' Addresses Graf, et al. Expires 4 January 2025 [Page 15] Internet-Draft YANG Notifications Observation Time July 2024 Thomas Graf Swisscom Binzring 17 CH-8045 Zurich Switzerland Email: thomas.graf@swisscom.com Benoit Claise Huawei Email: benoit.claise@huawei.com Alex Huang Feng INSA-Lyon Lyon France Email: alex.huang-feng@insa-lyon.fr Graf, et al. Expires 4 January 2025 [Page 16]