AAI Tutorial-Making and Testing a Schema Change - Dublin

1. Set up a development environment as described in AAI Developer Environment Setup - Dublin
   
   
2. This document will show editing schema files with Eclipse, but Eclipse is not required.
   
   
3. In this example, we will add 2 new attributes to the cloud-region node type and a new node type called "new-widget" which will be a child node of "cloud-region"
   
   
4. Adding a new attribute to "cloud-region"
5. Open aai/schema-service, aai-schema/src/main/resources/oxm/v16/aai_oxm_v16.xml
   
   
   

6. Add the following attributes between "inMaint" and "resourceVersion" in the Complex object:

   <xml-element default-value="false" java-attribute="inMaint" name="in-maint" required="true" type="java.lang.Boolean"> <xml-properties> <xml-property name="defaultValue" value="false"/> <xml-property name="description" value="Used to indicate whether or not cloud-region object is in maintenance mode."/> </xml-properties> </xml-element> <xml-element java-attribute="newAttributeForDemo" name="new-attribute-for-demo" required="true" type="java.lang.String"> <xml-properties> <xml-property name="description" value="Example new attribute for cloud-region instance. " /> </xml-properties> </xml-element> <xml-element java-attribute="numberAttributeForDemo" name="number-attribute-for-demo" required="true" type="java.lang.Integer"> <xml-properties> <xml-property name="description" value="Example number attribute for cloud-region instance. " /> </xml-properties> </xml-element> <xml-element java-attribute="resourceVersion" name="resource-version" type="java.lang.String"> <xml-properties> <xml-property name="description" value="Used for optimistic concurrency. Must be empty on create, valid on update and delete." /> </xml-properties> </xml-element>

   Do not try to add a new attribute with ‘xml-key=”true”’ because it creates complications when the key value is already set!

   Existing classes in the schema will already have an xml-key set.

   Compare with the "NewWidget" example below. As a new class in the schema, NewWidget must define a new attribute with the xml-key set.

   
   

7. Save the file, and rebuild the libraries and microservices:

   here's my example file: aai_oxm_v16.xml
   1. Rebuild schema-service first:
   2. $ cd ~/LF/AAI/schema-service
      $ mvn versions:set -DnewVersion=0.0.1-TEST-SNAPSHOT
      $ mvn clean install
      Should result in BUILD SUCCESS
   3. $ cd ~/LF/AAI/graphadmin
   4. $ mvn clean install -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
      Should result in BUILD SUCCESS
   5. $ cd ~/LF/AAI/resources
   6. $ mvn clean install -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
      Should result in BUILD SUCCESS
   7. $ cd ~/LF/AAI/traversal
      mvn clean install -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT

   8. Should result in BUILD SUCCESS

      Sometimes you may be tempted to use "-DskipTests" in the mvn command line to save some time, but it is known that some kinds of schema mistakes are detected only at run-time, not compile-time, so running the tests is a good indicator.

8. Run GenTester in the graphadmin repository:

   $ (cd graphadmin/ && mvn -PrunAjsc -Dstart-class=org.onap.aai.schema.GenTester -Daai.schema.version=0.0.1-TEST-SNAPSHOT -DskipTests -Dcheckstyle.skip=true -DAJSC_HOME=$HOME/src/aai/graphadmin -DBUNDLECONFIG_DIR=src/main/resources) You should see the following output:     ---- NOTE --- about to open graph (takes a little while)--------; -- loading schema into JanusGraph -- loading schema into JanusGraph -- Loading new schema elements into JanusGraph -- -- graph commit -- graph shutdown # You can check the logs to see if the graph elements were created successfully and grep for your property cd graphadmin/logs/createDBSchema/; grep 'complex-name' metrics.log 2018-12-06T04:43:18.061+0000|2018-12-06T04:43:42.960+0000|449b8583-54c1-42bf-8ada-5b75c52d7a13||main      ||AAI|AAI-TOOLS|AAI|main|COMPLETE|0|||INFO||127.0.1.1|24899|localhost||org.onap.a ai.dbgen.SchemaGenerator||||||||co=DBGenTester:Creating PropertyKey: [complex-name], [String], [SINGLE] 2018-12-06T04:43:18.061+0000|2018-12-06T04:43:42.960+0000|449b8583-54c1-42bf-8ada-5b75c52d7a13||main      ||AAI|AAI-TOOLS|AAI|main|COMPLETE|0|||INFO||127.0.1.1|24899|localhost||org.onap.a ai.dbgen.SchemaGenerator||||||||co=DBGenTester:Add index for PropertyKey: [complex-name]

9. Start the "resources" microservice
   1. Resources runs on port 8447.  Go to the resources directory
      $ cd ~/LF/AAI/resources
   2. Set the debug port to 9447
      $ export MAVEN_OPTS="-Xms1024m -Xmx5120m -XX:PermSize=2024m -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=9447,server=y,suspend=n"
   3. Start the microservice
      $ mvn -pl aai-resources -P runAjsc -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
      
      
10. Start the "traversal" microservice
    1. Traversal runs on port 8446.  Go to the traversal directory
       $ cd ~/LF/AAI/traversal
    2. Set the debug port to 9446
       $ export MAVEN_OPTS="-Xms1024m -Xmx5120m -XX:PermSize=2024m -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=9446,server=y,suspend=n"
    3. Start the microservice
       $ mvn -pl aai-traversal -PrunAjsc -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
       Should see something like this: Traversal Microservice Started
       
       
11. Start the "graphadmin" microservice
    1. Graphadmin runs on port 8449.  Go to the graphadmin directory
       $ cd ~/LF/AAI/graphadmin
    2. Set the debug port to 9449
       $ export MAVEN_OPTS="-Xms1024m -Xmx5120m -XX:PermSize=2024m -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=9449,server=y,suspend=n"
    3. Start the microservice
       $ mvn -PrunAjsc -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
       Should see something like this: GraphAdmin Microservice Started

12. Get an example cloud-region object, postman: Cloud-Region Example.postman_collection.json

    

    You should see the new attributes on the example object, as highlighted in the red rectangle above

13. Copy and paste the example object. Note that the following attributes need to be removed from your new object, if present in the example data:
    1. model-invariant-id (this is a foreign key to SDC model data, which will not exist in this tutorial)
    2. model-customization-id (this is a foreign key to SDC model data, which will not exist in this tutorial)
    3. model-version-id (this is a foreign key to SDC model data, which will not exist in this tutorial)
    4. resource-version (this value must be empty when creating a new object)

14. PUT your new object data to persist the new attributes.  Use this postman collection during the next 2 steps as well.

    

15. Check the object by doing a GET: GET Cloud-Region (no depth param) in postman collection
    
16. Add the depth parameter: GET Cloud-Region (depth = all) in postman collection
    
    
    
    
    1. This indicates that the schema has been updated with the new attributes and they are making it to the database
       
       
17. Clear the cloud region and then delete it by using "Clear Cloud-Region" and "Delete Cloud-Region"
    
    1. note: you will need to perform a GET each time and update the resource version of the cloud-region in the XML payload for the clear, and do the GET again and update the resource-version QueryParam to perform
       the delete
       
       

18. Run PUT Cloud-Region - missing attr.  This will try to PUT the object without the required attribute we defined in the schema, and the response will look like this:
    
    

    <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <Fault> <requestError> <serviceException> <messageId>SVC3000</messageId> <text>Invalid input performing %1 on %2 (msg=%3) (ec=%4)</text> <variables> <variable>PUT</variable> <variable>v16/cloud-infrastructure/cloud-regions/cloud-region/example-cloud-owner-val-44086/example-cloud-region-id-val-67393</variable> <variable>Invalid input performing %1 on %2:Missing required property: new-attribute-for-demo</variable> <variable>ERR.5.2.3000</variable> </variables> </serviceException> </requestError> </Fault>

19. Adding a new node type.  First, add the container under the top-level inventory category.  Adding "newWidgets" to "Network" so the URI will be "aai/v16/network/new-widgets/new-widget/{new-widget-name}"

    <java-type name="Network"> <xml-properties> <xml-property name="description" value="Namespace for network inventory resources."/> </xml-properties> <xml-root-element name="network"/> <java-attributes> <xml-element java-attribute="logicalLinks" name="logical-links" type="inventory.aai.onap.org.v16.LogicalLinks"/> <xml-element java-attribute="sitePairSets" name="site-pair-sets" type="inventory.aai.onap.org.v16.SitePairSets"/> <xml-element java-attribute="vpnBindings" name="vpn-bindings" type="inventory.aai.onap.org.v16.VpnBindings"/> <xml-element java-attribute="vplsPes" name="vpls-pes" type="inventory.aai.onap.org.v16.VplsPes"/> <xml-element java-attribute="multicastConfigurations" name="multicast-configurations" type="inventory.aai.onap.org.v16.MulticastConfigurations"/> <xml-element java-attribute="vces" name="vces" type="inventory.aai.onap.org.v16.Vces"/> <xml-element java-attribute="vnfcs" name="vnfcs" type="inventory.aai.onap.org.v16.Vnfcs"/> <xml-element java-attribute="l3Networks" name="l3-networks" type="inventory.aai.onap.org.v16.L3Networks"/> <xml-element java-attribute="networkPolicies" name="network-policies" type="inventory.aai.onap.org.v16.NetworkPolicies"/> <xml-element java-attribute="genericVnfs" name="generic-vnfs" type="inventory.aai.onap.org.v16.GenericVnfs"/> <xml-element java-attribute="lagLinks" name="lag-links" type="inventory.aai.onap.org.v16.LagLinks"/> <xml-element java-attribute="newvces" name="newvces" type="inventory.aai.onap.org.v16.Newvces"/> <xml-element java-attribute="pnfs" name="pnfs" type="inventory.aai.onap.org.v16.Pnfs"/> <xml-element java-attribute="physicalLinks" name="physical-links" type="inventory.aai.onap.org.v16.PhysicalLinks"/> <xml-element java-attribute="ipsecConfigurations" name="ipsec-configurations" type="inventory.aai.onap.org.v16.IpsecConfigurations"/> <xml-element java-attribute="routeTableReferences" name="route-table-references" type="inventory.aai.onap.org.v16.RouteTableReferences"/> <xml-element java-attribute="instanceGroups" name="instance-groups" type="inventory.aai.onap.org.v16.InstanceGroups"/> <xml-element java-attribute="zones" name="zones" type="inventory.aai.onap.org.v16.Zones"/> <xml-element java-attribute="configurations" name="configurations" type="inventory.aai.onap.org.v16.Configurations"/> <xml-element java-attribute="forwardingPaths" name="forwarding-paths" type="inventory.aai.onap.org.v16.ForwardingPaths"/> <xml-element java-attribute="collections" name="collections" type="inventory.aai.onap.org.v16.Collections"/> <xml-element java-attribute="vlanTags" name="vlan-tags" type="inventory.aai.onap.org.v16.VlanTags"/> <xml-element java-attribute="connectivities" name="connectivities" type="inventory.aai.onap.org.v16.Connectivities"/> <xml-element java-attribute="lanPortConfigs" name="lan-port-configs" type="inventory.aai.onap.org.v16.LanPortConfigs"/> <xml-element java-attribute="networkResources" name="network-resources" type="inventory.aai.onap.org.v16.NetworkResources"/> <xml-element java-attribute="newWidgets" name="new-widgets" type="inventory.aai.onap.org.v16.NewWidgets" /> <xml-element java-attribute="siteResources" name="site-resources" type="inventory.aai.onap.org.v16.SiteResources"/> <xml-element java-attribute="sdwanVpns" name="sdwan-vpns" type="inventory.aai.onap.org.v16.SdwanVpns"/> <xml-element java-attribute="devices" name="devices" type="inventory.aai.onap.org.v16.Devices"/> <xml-element java-attribute="wanPortConfigs" name="wan-port-configs" type="inventory.aai.onap.org.v16.WanPortConfigs"/> <xml-element java-attribute="extAaiNetworks" name="ext-aai-networks" type="inventory.aai.onap.org.v16.ExtAaiNetworks"/> </java-attributes> </java-type>

    
    
    

20. Set up the "NewWidgets" java type:

    <java-type name="NewWidgets"> <xml-properties> <xml-property name="description" value="Collection of new Widgets" /> </xml-properties> <xml-root-element name="new-widgets" /> <java-attributes> <xml-element container-type="java.util.ArrayList" java-attribute="newWidget" name="new-widget" type="inventory.aai.onap.org.v16.NewWidget" /> </java-attributes> </java-type>

21. Set up the NewWidget java type:

    <java-type name="NewWidget"> <xml-root-element name="new-widget" /> <java-attributes> <xml-element java-attribute="newWidgetName" name="new-widget-name" required="true" type="java.lang.String" xml-key="true"> <xml-properties> <xml-property name="description" value="e.g.,awesome-new-widget, terrific-new-widget" /> </xml-properties> </xml-element> <xml-element java-attribute="newWidgetType" name="new-widget-type" required="true" type="java.lang.String"> <xml-properties> <xml-property name="description" value="Type of new Widget, e.g., fantastic, amazing" /> </xml-properties> </xml-element> <xml-element java-attribute="resourceVersion" name="resource-version" type="java.lang.String"> <xml-properties> <xml-property name="description" value="Used for optimistic concurrency. Must be empty on create, valid on update and delete." /> </xml-properties> </xml-element> <xml-element java-attribute="modelInvariantId" name="model-invariant-id" type="java.lang.String"> <xml-properties> <xml-property name="description" value="the ASDC model id for this resource or service model." /> <xml-property name="visibility" value="deployment" /> <xml-property name="requires" value="model-version-id" /> <xml-property name="dbAlias" value="model-invariant-id-local" /> </xml-properties> </xml-element> <xml-element java-attribute="modelVersionId" name="model-version-id" type="java.lang.String"> <xml-properties> <xml-property name="description" value="the ASDC model version for this resource or service model." /> <xml-property name="visibility" value="deployment" /> <xml-property name="requires" value="model-invariant-id" /> <xml-property name="dbAlias" value="model-version-id-local" /> </xml-properties> </xml-element> <xml-element java-attribute="newWidgetId" name="new-widget-id" type="java.lang.String"> <xml-properties> <xml-property name="description" value="ID of the newWidget" /> </xml-properties> </xml-element> <xml-element java-attribute="relationshipList" name="relationship-list" type="inventory.aai.onap.org.v16.RelationshipList" /> </java-attributes> <xml-properties> <xml-property name="description" value="New widgets are the best widgets. Our new widgets are really, really great."/> <xml-property name="indexedProps" value="new-widget-id,new-widget-name,model-invariant-id,model-version-id" /> <xml-property name="uniqueProps" value="new-widget-id" /> <xml-property name="container" value="new-widgets" /> <xml-property name="namespace" value="network" /> <xml-property name="searchable" value="new-widget-name,new-widget-id" /> </xml-properties> </java-type>

22. Save the file, and rebuild the libraries and microservices:

    here's my example file:  aai_oxm_v16.xml
    1. Rebuild schema-service first:
    2. $ cd ~/LF/AAI/schema-service
       $ mvn versions:set -DnewVersion=0.0.1-TEST-SNAPSHOT
       $ mvn clean install
       Should result in BUILD SUCCESS
    3. $ cd ~/LF/AAI/graphadmin
    4. $ mvn clean install -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
       Should result in BUILD SUCCESS
    5. $ cd ~/LF/AAI/resources
    6. $ mvn clean install -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
       Should result in BUILD SUCCESS
    7. $ cd ~/LF/AAI/traversal
       mvn clean install -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
    8. Should result in BUILD SUCCESS
       
       

23. Run GenTester in the graphadmin repository:

    $ (cd graphadmin/ && mvn -PrunAjsc -Dstart-class=org.onap.aai.schema.GenTester -Daai.schema.version=0.0.1-TEST-SNAPSHOT -DskipTests -Dcheckstyle.skip=true -DAJSC_HOME=$HOME/src/aai/graphadmin -DBUNDLECONFIG_DIR=src/main/resources) # You should see the following output:     ---- NOTE --- about to open graph (takes a little while)--------; -- loading schema into JanusGraph -- loading schema into JanusGraph -- Loading new schema elements into JanusGraph -- -- graph commit -- graph shutdown #You can check the logs to see if the graph elements were created successfully and grep for your propertycd logs/createDBSchema/; grep 'new-widget-name' metrics.log Creating PropertyKey: [new-widget-name], [String], [SINGLE] [DEV: 2017-Jul-28 08:44:12,287][INFO ][main ]Creating PropertyKey: [new-widget-name], [String], [SINGLE] Add index for PropertyKey: [new-widget-name] [DEV: 2017-Jul-28 08:44:12,289][INFO ][main ]Add index for PropertyKey: [new-widget-name] Creating PropertyKey: [new-widget-type], [String], [SINGLE] [DEV: 2017-Jul-28 08:44:12,291][INFO ][main ]Creating PropertyKey: [new-widget-type], [String], [SINGLE] No index added for PropertyKey: [new-widget-type] [DEV: 2017-Jul-28 08:44:12,501][INFO ][main ]No index added for PropertyKey: [new-widget-type] PropertyKey [resource-version] already existed in the DB. PropertyKey [model-invariant-id] already existed in the DB. PropertyKey [model-version-id] already existed in the DB. Creating PropertyKey: [new-widget-id], [String], [SINGLE] [DEV: 2017-Jul-28 08:44:12,501][INFO ][main ]Creating PropertyKey: [new-widget-id], [String], [SINGLE]

24. Start the "resources" microservice
    1. Resources runs on port 8447.  Go to the resources directory
       $ cd ~/LF/AAI/resources
    2. Set the debug port to 9447
       $ export MAVEN_OPTS="-Xms1024m -Xmx5120m -XX:PermSize=2024m -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=9447,server=y,suspend=n"
    3. Start the microservice
       $ mvn -pl aai-resources -P runAjsc -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
       Should see something like this: Resources Microservice Started
       
       
25. Start the "traversal" microservice
    1. Traversal runs on port 8446.  Go to the traversal directory
       $ cd ~/LF/AAI/traversal
    2. Set the debug port to 9446
       $ export MAVEN_OPTS="-Xms1024m -Xmx5120m -XX:PermSize=2024m -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=9447,server=y,suspend=n"
    3. Start the microservice
       $ mvn -pl aai-traversal -P runAjsc -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
       Should see something like this: Traversal Microservice Started
       
       
26. Start the "graphadmin" microservice
    1. Graphadmin runs on port 8449.  Go to the graphadmin directory
       $ cd ~/LF/AAI/graphadmin
    2. Set the debug port to 9449
       $ export MAVEN_OPTS="-Xms1024m -Xmx5120m -XX:PermSize=2024m -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=9449,server=y,suspend=n"
    3. Start the microservice
       $ mvn -PrunAjsc -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
       Should see something like this: GraphAdmin Microservice Started
27. Get an example newWidget using "New Widget Example" (find these examples in this postman collection: New-Widget Example.postman_collection.json)
    
28. Copy the new widget and PUT it: (PUT New Widget)
    
    
29. Success!  We have updated an existing object type and added a new object type.  Next, we will set up an edge between the CloudRegion and the NewWidget
    
    
30. Open DbEdgeRules_v16.json
    File is schema-service/aai-schema/src/main/resources/onap/dbedgerules/v16/DbEdgeRules_v16.json
    

31. Add a new EdgeRule, allowing an edge between the CloudRegion and NewWidget:

    {             "from": "cloud-region",             "to": "tenant",             "label": "has",             "direction": "OUT",             "multiplicity": "One2Many",             "contains-other-v": "${direction}",             "delete-other-v": "NONE",             "SVC-INFRA": "!${direction}",             "prevent-delete": "${direction}"         },         {             "from": "cloud-region",             "to": "new-widget",             "label": "has",             "direction": "OUT",             "multiplicity": "One2Many",             "contains-other-v": "NONE",             "delete-other-v": "NONE",             "SVC-INFRA": "NONE",             "prevent-delete": "NONE"         }, This means cloud-region connects to new-widget, edge label is "has", it's an OUT edge, one cloud region can have multiple edges to new-widgets, cloud-region does not contain new-widget, new-widget will not be deleted when the cloud-region connected to it is deleted, it is not SVC_INFRA type, and having an edge to a new-widget will not prevent deletion of the cloud-region.

    
    
    

32. Save the file, and rebuild schema-service, resources, and traversal.  Start resources and traversal microservices.
    
    
33. Create a cloud region with a relationship to the new-widget that was created earlier: (PUT Cloud-Region related to New-Widget)
    
    
34. GET the new-widget object: (GET New Widget)
    
    
    
    
35. Notice that when Getting the new-widget object it shows the relationship to the cloud-region.
    
    We have successfully modified the schema and edge rules to update an existing type, create a new type, and define an edge relationship which allows for a new-widget type to be connected to a cloud-region.  

Attachments