Skip to end of metadata
Go to start of metadata

This guide will illustrate setting up an A&AI development environment for R3 Casablanca in Ubuntu 16.04.  

DRAFT - this guide is in process of being updated - thanks for your patience (smile)

NOTE - Please update the changes where you see v11 with v14 in order to test the schema change. 

(For alternative setup, see also  AAI-2049 - Getting issue details... STATUS  and https://gerrit.onap.org/r/gitweb?p=aai%2Ftest-config.git;hb=e64f08ac8242a7db6eb3238ee0ad1a30c8aecf0d;f=local-setup )

For this exercise, I set up a new instance of Ubuntu in Virtualbox and gave it 16G RAM, 200GB dynamically allocated storage, and 3 processors.

  1. install openjdk 8
    1. sudo apt install openjdk-8-jdk

  2. Install single node hadoop/janusgraph
    1. $ wget http://github.com/JanusGraph/janusgraph/releases/download/v0.2.0/janusgraph-0.2.0-hadoop2.zip
    2. $ unzip janusgraph-0.2.0-hadoop2.zip
    3. $ cd janusgraph-0.2.0-hadoop2/
    4. $ ./bin/janusgraph.sh start, # make sure you are not a root user as elasticsearch cannot be run as root.  Response looks like:
      Forking Cassandra...
      Running `nodetool statusthrift`... OK (returned exit status 0 and printed string "running").
      Forking Elasticsearch...
      Connecting to Elasticsearch (127.0.0.1:9200)...... OK (connected to 127.0.0.1:9200).
      Forking Gremlin-Server...
      Connecting to Gremlin-Server (127.0.0.1:8182).... OK (connected to 127.0.0.1:8182).
      Run gremlin.sh to connect.
    5. you can verify whether everything is running by executing  
      ./bin/janusgraph.sh status

      Gremlin-Server (org.apache.tinkerpop.gremlin.server.GremlinServer) is running with pid 9835
      Elasticsearch (org.elasticsearch.bootstrap.Elasticsearch) is running with pid 9567
      Cassandra (org.apache.cassandra.service.CassandraDaemon) is running with pid 9207


  3. Install haproxy
    1. $ sudo apt-get -y install haproxy
    2. $ <path-to-haproxy>/haproxy -v
      HA-Proxy version 1.6.3 2015/12/25
      Copyright 2000-2015 Willy Tarreau <willy@haproxy.org>
    3. Install the attached haproxy.cfg file in /etc/haproxy

      1. $ wget https://wiki.onap.org/download/attachments/45306687/haproxy.cfg

      2. $ sudo cp haproxy.cfg /etc/haproxy
      3. $ sudo mkdir /usr/local/etc/haproxy
    4. Install the attached aai.pem file in /etc/ssl/private

      1. $ wget https://wiki.onap.org/download/attachments/45306687/aai.pem

      2. $ sudo cp aai.pem /etc/ssl/private/aai.pem
      3. $ sudo chmod 640 /etc/ssl/private/aai.pem
      4. $ sudo chown root:ssl-cert /etc/ssl/private/aai.pem 
    5. Add these hostnames to the loopback interface in /etc/hosts: 

      1. 127.0.0.1 localhost aai-traversal.api.simpledemo.openecomp.org aai-resources.api.simpledemo.openecomp.org

    6. $ sudo service haproxy restart

  4. follow the initial setup instructions in Setting Up Your Development Environment e.g.
    1. $ sudo apt-get install git
    2. $ sudo apt-get install npm
    3. $ sudo apt-get install maven
    4. $ sudo apt-get install docker.io
    5. $ wget https://git.onap.org/oparent/plain/settings.xml
    6. $ mkdir ~/.m2
    7. cp settings.xml ~/.m2
    8. If you get an error on some of the repos saying that oparent is unresolvable, using the example settings.xml file should solve this problem: Setting Up Your Development Environment#MavenExamplesettings.xml
  5. Set up repos:
    1. $ mkdir -p ~/src/aai
    2. $ cd ~/src/aai ; for f in aai-common resources traversal graphadmin logging-service ; do git clone ssh://<username>@gerrit.onap.org:29418/aai/$f; done
  6. cd ~/src/aai ; for f in aai-common resources traversal graphadmin logging-service ; do
    (cd $f ; git checkout casablanca) done | tee checkoutlog.txt
  7. Janus Setup (part 1) 
    Modify both janus-cached.properties and janus-realtime.properties to the following (for all MS’s that will connect to the local Cassandra backend)
    storage.backend=cassandra
    storage.hostname=localhost
    storage.cassandra.keyspace=onap # or different keyspace name of your choosing
    • ~/src/aai/resources/aai-resources/src/main/resources/etc/appprops/janusgraph-cached.properties
    • ~/src/aai/resources/aai-resources/src/main/resources/etc/appprops/janusgraph-realtime.properties
    • ~/src/aai/traversal/aai-traversal/src/main/resources/etc/appprops/janusgraph-cached.properties
    • ~/src/aai/traversal/aai-traversal/src/main/resources/etc/appprops/janusgraph-realtime.properties
    • ~/src/aai/graphadmin/src/main/resources/etc/appprops/janusgraph-cached.properties
    • ~/src/aai/graphadmin/src/main/resources/etc/appprops/janusgraph-realtime.properties
  8. Build all the modules:
    1. $ cd ~/src/aai ; for f in aai-common resources traversal graphadmin logging-service ; do
      (cd $f ; mvn -DskipTests clean install) done | tee log.txt 2>&1
    2. $ grep -e "SUCCESS" -e "FAILURE" log.txt

      [INFO] aai-aai-common ..................................... SUCCESS [ 24.040 s]
      [INFO] aai-schema-ingest .................................. SUCCESS [ 30.066 s]
      [INFO] aai-annotations .................................... SUCCESS [ 0.887 s]
      [INFO] aai-core ........................................... SUCCESS [02:39 min]
      [INFO] aai-schema ......................................... SUCCESS [ 25.144 s]
      [INFO] aai-auth ........................................... SUCCESS [ 1.415 s]
      [INFO] aai-utils .......................................... SUCCESS [ 1.004 s]
      [INFO] BUILD SUCCESS
      [INFO] aai-resources ...................................... SUCCESS [ 3.428 s]
      [INFO] aai-resources ...................................... SUCCESS [ 45.262 s]
      [INFO] BUILD SUCCESS
      [INFO] aai-traversal ...................................... SUCCESS [ 4.050 s]
      [INFO] aai-traversal ...................................... SUCCESS [01:01 min]
      [INFO] BUILD SUCCESS
      [INFO] BUILD SUCCESS
      [INFO] aai-logging-service ................................ SUCCESS [ 8.281 s]
      [INFO] Common Logging API ................................. SUCCESS [ 2.548 s]
      [INFO] EELF Logging Implementation ........................ SUCCESS [ 3.552 s]
      [INFO] Common Logging Distribution ........................ SUCCESS [ 0.575 s]
      [INFO] BUILD SUCCESS
  9. Janus setup (part 2)
    1. Run this on the local instance on your first time running AAI and whenever using new keyspace or after wiping the data.
    2. Download createDbSchemaLocal.sh to your dev machine.  Adjust your build version accordingly, as of the casablanca release the version is 1.0.1
      1. $ wget https://wiki.onap.org/download/attachments/45306687/createDbSchemaLocal.sh
      2. $ export PROJECT_HOME=~/src/aai/graphadmin/target/aai-graphadmin-1.0.2-SNAPSHOT-build
      3. $ sh createDbSchemaLocal.sh
        You should see: 
        ---- NOTE --- about to open graph (takes a little while)--------;
        -- Loading new schema elements into JanusGraph --
        -- graph commit
        -- graph shutdown
  10. Start the "resources" microservice
    1. Resources runs on port 8447.  Go to the resources directory
      $ cd ~/src/aai/resources
    2. Set the debug port to 9447
      $ export MAVEN_OPTS="-Xms1024m -Xmx5120m -XX:PermSize=2024m -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=9447,server=y,suspend=n"
    3. Start the microservice - adjust your build version accordingly, as of the casablanca release the version is 1.3.4, current casablanca branch is 1.3.5
      $ java -DAJSC_HOME=aai-resources -DBUNDLECONFIG_DIR=src/main/resources/ -jar aai-resources/target/aai-resources-1.3.4-SNAPSHOT.jar
      Should see something like this: Resources Microservice Started 

  11. Verify the resources microservice (this example uses curl from commandline)
    1. $ sudo apt-get install jq  # for pretty output
    2. $ wget https://wiki.onap.org/download/attachments/45306687/test-complex
    3. $ wget https://wiki.onap.org/download/attachments/45306687/data-complex.json
    4. $ sh ./test-complex 2>&1 | tee log.txt
    5. Confirm log.txt contains:

      > GET /aai/v14/cloud-infrastructure/complexes HTTP/1.1
      {
        "requestError": {
          "serviceException": {
            "messageId": "SVC3001",
            "text": "Resource not found for %1 using id %2 (msg=%3) (ec=%4)",
            "variables": [
              "GET",
              "cloud-infrastructure/complexes",
              "Node Not Found:No Node of type complex found at: cloud-infrastructure/complexes",
              "ERR.5.4.6114"
            ]
          }
        }
      }
      
      

      then followed by:

      > PUT /aai/v14/cloud-infrastructure/complexes/complex/clli2 HTTP/1.1
      > GET /aai/v14/cloud-infrastructure/complexes/complex/clli2 HTTP/1.1
      {
        "physical-location-id": "clli2",
        "data-center-code": "example-data-center-code-val-6667",
        "complex-name": "clli2",
        "identity-url": "example-identity-url-val-28399",
        "resource-version": "1543408364646",
        "physical-location-type": "example-physical-location-type-val-28399",
        "street1": "example-street1-val-28399",
        "street2": "example-street2-val-28399",
        "city": "example-city-val-28399",
        "state": "example-state-val-28399",
        "postal-code": "example-postal-code-val-28399",
        "country": "example-country-val-28399",
        "region": "example-region-val-28399",
        "latitude": "1111",
        "longitude": "2222",
        "elevation": "example-elevation-val-28399",
        "lata": "example-lata-val-28399"
      }
      
      

      and finishes with:

      > DELETE /aai/v14/cloud-infrastructure/complexes/complex/clli2?resource-version=1543408364646 HTTP/1.1
      > GET /aai/v14/cloud-infrastructure/complexes HTTP/1.1
      {
        "requestError": {
          "serviceException": {
            "messageId": "SVC3001",
            "text": "Resource not found for %1 using id %2 (msg=%3) (ec=%4)",
            "variables": [
              "GET",
              "cloud-infrastructure/complexes",
              "Node Not Found:No Node of type complex found at: cloud-infrastructure/complexes",
              "ERR.5.4.6114"
            ]
          }
        }
      }
      
      
  12. Alternately, verify the resources microservice (this example uses Postman utility for Google Chrome)
    1. Use basic auth, user = AAI, pw = AAI
    2. Set the X-TransactionId header (in the example below, the value is 9999)
    3. Set the X-FromAppId header (in the example below, the value is jimmy-postman)
    4. Perform a GET of https://127.0.0.1:8443/aai/v11/network/vces
    5. You should see an error as below, 404 Not Found, ERR.5.4.6114.  This indicates that the service is functioning normally:



  13. Start the "traversal" microservice
    1. Traversal runs on port 8446.  Go to the traversal directory
      $ cd ~/src/aai/traversal
    2. Set the debug port to 9446
      $ export MAVEN_OPTS="-Xms1024m -Xmx5120m -XX:PermSize=2024m -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=9446,server=y,suspend=n"
    3. Start the microservice - adjust your build version accordingly, as of the casablanca release the version is 1.3.3, current version in the casablanca branch is 1.3.4
      $ java -DAJSC_HOME=aai-traversal -DBUNDLECONFIG_DIR=src/main/resources/ -jar aai-traversal/target/aai-traversal-1.3.3-SNAPSHOT.jar
      Should see something like this: Traversal Microservice Started

  14. Verify the traversal microservice by executing attached postman AAI API calls
    1. Set up the widget models
      This will set up the postman to add widget models: Add Widget Models.postman_collection.json
    2. Create a runner using this file: models.csv
    3. Run the test runner
    4. Add a named query called "getComponentList" (this named query is used by VID): NamedQuery.postman_collection.json
    5. Add objects: Add Instances for Named Query.postman_collection.json (when using vXX in place of v11, replace the xmlns "http://org.openecomp.aai.inventory/v11" with "http://org.onap.aai.inventory/vXX" in the Body of the PUT request)
    6. Execute named-query: Execute Named Query.postman_collection.json
      You should see something like the following:
  15. Your A&AI instance is now running, both the resources and traversal microservices are working properly with a local janus graph. 
  16. Next: Tutorial: Making and Testing a Schema Change in A&AI in Casablanca Release

Attachments

  File Modified
File aai.pem Nov 27, 2018 by Jimmy Forsyth
File Add Instances for Named Query.postman_collection.json Nov 27, 2018 by Jimmy Forsyth
File Add Widget Models.postman_collection.json Nov 27, 2018 by Jimmy Forsyth
File createDbSchemaLocal.sh Nov 27, 2018 by Jimmy Forsyth
File data-complex.json Nov 28, 2018 by Keong Lim
File Execute Named Query.postman_collection.json Nov 27, 2018 by Jimmy Forsyth
File haproxy.cfg Nov 27, 2018 by Jimmy Forsyth
PNG File image2017-7-26_11-6-11.png Nov 27, 2018 by Jimmy Forsyth
PNG File image2017-7-26_16-17-19.png Nov 27, 2018 by Jimmy Forsyth
PNG File image2017-7-26_16-23-12.png Nov 27, 2018 by Jimmy Forsyth
PNG File image2017-7-26_16-58-5.png Nov 27, 2018 by Jimmy Forsyth
File models.csv Nov 27, 2018 by Jimmy Forsyth
File NamedQuery.postman_collection.json Nov 27, 2018 by Jimmy Forsyth
File test-complex Nov 28, 2018 by Keong Lim

  • No labels

7 Comments

  1. Keong Lim and Pavel Paroulek I updated this for the casablanca release (i'm done thru step 8) and it works for me - if either of you gets a minute overnight, could you give it a try and see if you get a good result?

    1. Tested in my fresh Ubuntu machine, which has only 8GB RAM, 100GB HDD and 4 processors. I had it upgraded from Ubuntu 14.04.5 LTS to Ubuntu 16.04.5 LTS.

      In step 2b, the "unzip" command was not yet available, so needed to do:

      • sudo apt-get install unzip

      Updated step 2e with example response from janusgraph status

      Updated step 3c with wget command to download haproxy.cfg from wiki page.

      • As an aside, should that haproxy.cfg be using user root and group root? the default one has user haproxy and group haproxy, which seems like a minimal privilege setup for security
      • Rearranged the mkdir to be under 3c haproxy steps

      Updated step 3d with wget command to download aai.pem from wiki page.

      • rearranged following steps under 3d as related to aai.pem
      • Got error "chown: invalid group: ‘root:ssl-cert’" on the chown step, leaving mine as root:root

      Split up step 4 into separate steps for Setting up dev env, then setup repos

      • added wget for maven settings.xml

      For my step 5 repos, I chose to use anonymous https method instead, e.g.

      For my step 6 build, I preferred to include tests rather than skipping them, i.e.

      • mvn clean install
      • compressed the steps into a for loop, added example output from the grep to check all results

      Updated step 7b with wget createDbSchemaLocal

      • should the output say "inmemory" or should it be "cassandra" after step 7a?

      Insert step 9 to verify resources with curl (alternative to using Postman)

      • uploaded shell script and data file
      • added wget to download files and expected log file output

      Updated step 11 with Casablanca version of traversal JAR


      1. For 7b it should look like it does after my edit.  I realized that we had an out-of-order issue there, and if somebody did the build before editing the files that it would use the inmemory graph instead of local cassandra.  I created moved the file edits above the build and I think it's better now.

        I like the curl steps you added, very nice.

  2. Note that this page uses "HA-Proxy version 1.6.3 2015/12/25".

    There was a report of graphadmin problems when using "HA-Proxy version 1.7.11-1ppa1~xenial 2018/04/30".


  3. Linked Krzysztof Kurczewski work on  AAI-2049 - Getting issue details... STATUS  as alternative setup.

  4. Jimmy Forsyth I could successfully complete step upto 14 e. In the last step i.e 14 f, i am not getting the desired output saying "no node found" . Even though when i am doing GET of global customer id and service instance it does exists..

    Could you please help me tracing what must be wrong.

  5. Jimmy Forsyth In step 14e, I ran the runner named  "Add Instances for Named Query", but I got the error message saying "Error making edge to target node : Node of type service-instance. Could not find object at:/business/customers/customer/aai_demo_for_onap_community/service-subscriptions/service-subscription/vDNS/service-instances/service-instance/37b8cdb7-94eb-468f-a0c2-4e3c3546578e". I checked whether any service-instance existed or not using the url of https://ip_address:8443/aai/v11/service-design-and-creation/services, then I got the same error which said "Node Not Found:No Node of type service found at: service-design-and-creation/services".I wonder in which step the service-instance should be created.In the previous steps, the outcomes were all success.I am very confused with this issue.Could you please help me resolve this problem.Thanks in advance.