Page History

...

This page documents the existing Yang Parser used in ONAP and OpenDayLight and will investigate if they can be used for it fulfill the needs of the C&PS.

Overview

The Yang parser used in ONAP (CCSDK / SDNC) was developed (and still is) a OpenDayLight Library.

...

Additional Resources (still to be examined)

https://github.com/PANTHEONtech/lighty-core

_{*Although this documentation link is to the latest ODL doc revision, it is very outdated and the code examples need significant updates, see findings in Mini-PoC below (bug reported: https://jira.opendaylight.org/browse/DOCS-126)}

...

The documentation mentioned above lists many modules but the code examples do not clarify which exactly are needed to parse a Yang model and Yang file data files in java code.

To be able to use yangtools teh project needs to use the mdsal 'binding-parent' pom
The module yang-parser-impl contains all code required to parse yang files
SLF4J has been added as the implementation requires a logger enabled

Code Block

language	xml

Code Block

language	xml
theme	Midnight
title	pom.xml
collapse	true

    <properties>
		        <maven.compiler.version>3.8.1</maven.compiler.version>
		        <maven.compiler.release>11</maven.compiler.release>
		<org.opendaylight.        <odl.yangtools.version>5.0.3<5</org.opendaylightodl.yangtools.version>
	    </properties>
	
	    <build>
		<plugins>
			<plugin>
				        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
				                <artifactId>maven-compiler-plugin</artifactId>
				                <version>${maven.compiler.version}</version>
				<configuration>
					                <configuration>
                    <release>${maven.compiler.release}</release>
				                </configuration>
			            </plugin>
		</plugins>
	        </plugins>
    </build>

	    <dependencies>
		<dependency>
			        <dependency>
            <groupId>org.opendaylight.yangtools</groupId>
			            <artifactId>yang-parser-api</artifactId>
			<version>${org.opendaylight            <version>${odl.yangtools.version}</version>
		        </dependency>

		<dependency>
			        <dependency>
            <groupId>org.opendaylight.yangtools</groupId>
			            <artifactId>yang-parser-impl</artifactId>
			<version>${org.opendaylight            <version>${odl.yangtools.version}</version>
		        </dependency>

		<dependency>
			        <dependency>
            <groupId>org.opendaylight.yangtools</groupId>
			<artifactId>yang            <artifactId>yang-model-util</artifactId>
			<version>${org.opendaylight            <version>${odl.yangtools.version}</version>
		        </dependency>

		<dependency>
			        <dependency>
            <groupId>org.opendaylight.yangtools</groupId>
			            <artifactId>yang-data-codec-xml</artifactId>
			<version>${org.opendaylight            <version>${odl.yangtools.version}</version>
		        </dependency>

		<dependency>
			<groupId>org.slf4j</groupId>
			<artifactId>slf4j-api</artifactId>
			<version>1.6.1</version>
		</dependency>

		<dependency>
			<groupId>org.slf4j</groupId>
			<artifactId>slf4j-log4j12</artifactId>
			        <dependency>
            <groupId>org.opendaylight.yangtools</groupId>
            <artifactId>yang-data-codec-gson</artifactId>
            <version>${odl.yangtools.version}</version>
        </dependency>
        <!-- SLF4J API -->
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.6.1</version>
		        </dependency>
	</dependencies>

Documentation Code Updates

The sample code provided in the documentation is faulty (using == for assigning?!) and is using some long deprecated and even removed classes and methods.

Bug reported: https://jira.opendaylight.org/browse/DOCS-126

Code Block

language	java
theme	Midnight
title	Corrected Code
linenumbers	true
collapse	true

    static YangParserFactory PARSER_FACTORY;

    static {        <!-- LOG4J -->
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
        final Iterator<YangParserFactory> it = ServiceLoader<version>1.load(YangParserFactory.class).iterator();6.1</version>
        if (!it.hasNext()) {</dependency>
    </dependencies>

Note. SLF4J has also been added as the implementation requires an enabled logger

Documentation Code Updates

The sample code provided in the documentation is faulty (using == for assigning?!) and is using some long deprecated and even removed classes and methods.

Bug reported: https://jira.opendaylight.org/browse/DOCS-126

Code Block

language	java
theme	Midnight
title	Corrected Code
linenumbers	true
collapse	true

    static YangParserFactory PARSER_FACTORY;

    static {
        final Iterator<YangParserFactory> it = ServiceLoader.load(YangParserFactory.class).iterator(        throw new IllegalStateException("No YangParserFactory found");
        }
        PARSER_FACTORY = it.next();
    }

    ...

        File file = new File(ClassLoader.getSystemClassLoader().getResource("example.yang").getFile());
        YangTextSchemaSource source = YangTextSchemaSource.forFile(file);
if (!it.hasNext()) {
          
  throw new IllegalStateException("No YangParserFactory found");
  final YangParser yangParser = PARSER_FACTORY.createParser(StatementParserMode.DEFAULT_MODE   }
        PARSER_FACTORY = it.next();
    }

    yangParser.addSource(source);...

        SchemaContextFile schemaContextfile = yangParser.buildEffectiveModel();
 new File(ClassLoader.getSystemClassLoader().getResource("example.yang").getFile());
        schemaContext.getModules()YangTextSchemaSource source = YangTextSchemaSource.forFile(file);
        
        final YangParser yangParser = PARSER_FACTORY.createParser(StatementParserMode.DEFAULT_MODE);
        yangParser.addSource(source);
        SchemaContext schemaContext = yangParser.buildEffectiveModel();

        schemaContext.getModules()

This is the kind of object (module) that gets created:This is the kind of object (module) that gets created:

Generated Java Object Structures per Yang concept

...

Description	Yang	Java Object View	Notes	XML Validation	JSON Validation
Datatypes and basic constraints
Basic String	`leaf response-code {` `type string;` `}`		`TypeStatement TypeAwareDeclaredStatement.getType()`	Yes	Yes
Mandatory Basic String	`leaf response-code {925px` `type string;` `mandatory "true";` `}`		`Optional<MandatoryStatement> MandatoryStatementAwareDeclaredStatement.getMandatory()`	No	No
Limited String	`leaf pnf-name {` `type string {` `length "0..256";` `}`		`Optional<LengthConstraint> AbstractLengthRestrictedType.getLengthConstraint()`	Yes	Yes
typedef (String) with pattern	`typedef dotted-quad {` `type string {` `pattern` `'(([0-9] ...';` `}` `}` `leaf address {` `type dotted-quad;` `mandatory "true";` `}`		`List<PatternConstraint> RestrictedStringType.getPatternConstraints()`	Yes	Yes
Limited uint64	`leaf cid {` `type uint64 {` `range "0..503";` `}` `}`		`org.opendaylight.yangtools.yang.model.util.type.RestrictedUint64Type`	Yes	Yes
boolean with default value	`leaf blacklisted {` `type boolean;` `default 1;` `}`		`org.opendaylight.yangtools.yang.model.util.type.DerivedBooleanType`	N/A	N/A
Unique
Unique	`list server {` `key "name";` `unique "ip port";` `leaf name {` `type string;` `}` `leaf ip {` `type dotted-quad;` `}` `leaf port {` `type uint32;` `}}`		`org.opendaylight.yangtools.yang.model.api.stmt.UniqueStatement`	No	No
Choice
Choice	`choice transfer-method {` `leaf transfer-interval {` `type uint64 {` `range "15..2880";` `}` `units minutes;` `}` `leaf transfer-on-commit {` `type empty;` `}}`		`org.opendaylight.yangtools.yang.model.api.stmt.ChoiceStatement`	N/A	N/A
Must
Must	`leaf ifType {` `type enumeration {` `enum ethernet;` `enum atm;}}` `leaf ifMTU {` `type uint32;}` `must "ifType != 'ethernet' or "` `+ "(ifType = 'ethernet' and ifMTU = 1500)" {error-message 466px"An ethernet MTU must be 1500";}`		`org.opendaylight.yangtools.yang.model.api.stmt.ErrorMessageStatement`	No	No
When
When	leaf a { type boolean; } leaf b { type string; when "../a = 'true'"; }		`org.opendaylight.yangtools.yang.model.api.stmt.WhenStatement`	No	No
Extension
Extension declaration	`extension store-state-ext {` `argument duration;` `description "An extension to enablestate-storage for any attribute. Use duration to specify how long: nnn hnnn<h\|d\|yy>";` `}`	Image RemovedImage Added		N/A
Extension usage	`leaf attribute-with-temporal-storage {` `type string;` `cm-notify-api:store-state-ext "3 d3d"; // store state 3 days` `}`	Image Removed	extension is stored as 'UnknownNode' and refers back to the extension declaration	N/A
Augmentation
	Image Added	`org.opendaylight.yangtools.yang.model.api.stmt.UnrecognizedStatement` which extends `UnknownStatement:` public interface UnknownStatement<A> extends DeclaredStatement<A> { default @Nullable A getArgument() { return argument(); } }	N/A
Augmentation
	`augment "server" {` `when "port = '8787'";` `leaf enable-debug {` `type boolean;` `}` `}`	Image Added	The additional leaf just appears in the SchemaTree (without any reference that it is an augmentation)Image Removed	N/A
RPC
rpc	rpc nbrlist-change-notification { description "RAN Neighbor List change notification to configure RuntimeDB"; input { : } output { : } }	Image RemovedImage Added		N/A
rpc input	input { leaf fap-service-number-of-entries-changed { type uint64; description "Number of cells for which neighbor list has changed"; } list fap-service { key "alias"; leaf alias { type string { length "1..64"; } } leaf cid { type string { length "0..52"; } } uses x-0005b9-lte-g; leaf lte-cell-number-of-entries { type uint64; description "Number of cells in a neighbor list that was changed"; } list lte-ran-neighbor-list-in-use-lte-cell-changed { key "plmnid cid"; uses lte-ran-neighbor-list-in-use-lte-cell-g; description "Changed/Modified List of cells in a neighbor list for this fap service"; } } }	Image RemovedImage Added		N/A
rpc output	output { uses cm-notification-response; }	Image RemovedImage Added		N/A

Yang Data Parsing and

...

Validation

XML Parsing

Code Block

language	java
theme	Midnight
title	XML Parsing Example
linenumbers	true
collapse	true

		SchemaContext schemaContext = ... (see previous snippets)
        final Module module = schemaContext.findModules("ultraconfig-interfaces").iterator().next();
        QName qName = QName.create(module.getQNameModule(),"interfaces");
        final Optional<DataSchemaNode> node = module.findDataChildByName(qName);
        if (node.isPresent()) {
            final InputStream resourceAsStream = ClassLoader.getSystemClassLoader().getResourceAsStream("example2data.xml");
            final XMLStreamReader reader = UntrustedXML.createXMLStreamReader(resourceAsStream);
            final NormalizedNodeResult result = new NormalizedNodeResult();
            final NormalizedNodeStreamWriter streamWriter = ImmutableNormalizedNodeStreamWriter.from(result);
            final XmlParserStream xmlParser = XmlParserStream.create(streamWriter, schemaContext, node.get() );
            xmlParser.parse(reader);
            final NormalizedNode<?, ?> transformedInput = result.getResult();
        }

...

Code Block

language	java
theme	Midnight
title	JSON Parsing Example
linenumbers	true
collapse	true

		SchemaContext schemaContext = ... (see previous snippets)
        JSONCodecFactory jsonCodecFactory = JSONCodecFactorySupplier.DRAFT_LHOTKA_NETMOD_YANG_JSON_02.getShared(schemaContext);
        final NormalizedNodeResult result = new NormalizedNodeResult();
        final NormalizedNodeStreamWriter streamWriter = ImmutableNormalizedNodeStreamWriter.from(result);
        final JsonParserStream jsonParser = JsonParserStream.create(streamWriter, jsonCodecFactory);
        final InputStream resourceAsStream = ClassLoader.getSystemClassLoader().getResourceAsStream("example2data.json");
        final JsonReader jsonReader = new JsonReader(new InputStreamReader(resourceAsStream));
        jsonParser.parse(jsonReader);
        final NormalizedNode<?, ?> transformedInput = result.getResult();

JSON Validation Findings

As expected the parsing of string, originating form XML or JSON is done by the same code and the results are identical to those for XML Data Parsing

Conclusions

 = JsonParserStream.create(streamWriter, jsonCodecFactory);
        final InputStream resourceAsStream = ClassLoader.getSystemClassLoader().getResourceAsStream("example2data.json");
        final JsonReader jsonReader = new JsonReader(new InputStreamReader(resourceAsStream));
        jsonParser.parse(jsonReader);
        final NormalizedNode<?, ?> transformedInput = result.getResult();

JSON Validation Findings

As expected the parsing of string, originating form XML or JSON is done by the same code and the results are identical to those for XML Data Parsing

Conclusion

Pros

EXTENSIVE - The YangTools model parser is comprehensive and covers all possible Yang Language elements we might require
AVAILABLE - Extensively used throughout ONAP and ODL projects (able to convert to Java objects, code & API). These parsers are already used in many ONAP platform projects.
MATURE CODE - Mature code, dates back to 2013 with contributions from many companies including Cisco and Redhat and Pantheon Tech. (No need for licenses, these are also open source Yang parser). ODL is open source.
VALIDATION - Testing with parsing of JSON and XML data with validated for a (parsed) model included. Files to objects and vice versa is possible. Model violation & compilation validation is available.
OBJECTIVES - Meets our two high-level requirements & objectives. Parse models from SDC into Java objects relate to persistence of data. Parsing of documents compliant to those schemas.

Cons

LEARNING CURVE - Due to its completeness it also is a complicated piece of software which will take some time to get familiar with.
DOCUMENTATION - Documentation out of date, this page hopes to address that somewhat
The YangTools model parser is comprehensive and cover all disposable Yang Language elements we might require
complicated, no clear interface
documentation out of data,

Space shortcuts

Page tree

Versions Compared

Old Version 41

New Version Current

Key

Overview

Documentation Code Updates

Documentation Code Updates

Generated Java Object Structures per Yang concept

Datatypes and basic constraints

Unique

Choice

Must

When

Extension

Augmentation

Augmentation

RPC

Yang Data Parsing and

Validation

XML Parsing

JSON Validation Findings

Conclusions

JSON Validation Findings

Conclusion