These instructions have been verified on OSX High Sierra 10.13.4 using Docker for Mac Version 18.05.0-ce-rc1-mac63 (24246), which is latest edge version, stable channel will work as well.
Requirements:
- Recent Docker for Mac (tested with latest edge and stable release)
- Docker for Mac has to be sized to allow running all SDC containers, see instructions.
- Basic development utils for building the source code (Git, java, maven etc..)
Instructions:
Make sure your docker is up and running, and is fully updated. Once docker is up and running (you can pull images and run them, ie docker pull ubuntu && docker run -it ubuntu /bin/bash), set resources for docker to something like this(these work on my MacBook Pro 15 Retina):
Save settings and restart docker.
Building the source code
Change directory to SDC project and run:
mvn clean install -Pdocker
This will build entire project and all docker containers. Note that this might take a while.
Preparing the configuration files
These instructions are taken from: Deploying SDC on a Linux VM for Development and slightly modified to facilitate for OSX specifics.
- Inside SDC project root directory, create new folder called data
- Add above created data directory to gitignore, as we don't want this to be included with any commits.
- Inside data directory, do following:
- create opt/config/
- place the following files in this data/opt/config/ folder:
- Configuration files
- Inside data directory, create environments directory
- Copy the file from here: https://git.onap.org/sdc/tree/sdc-os-chef/environments/Template.json into data/environments/AUTO.json
- Edit AUTO.json file, replace yyy with your en0 IP address (your laptop's IP address), and update name and description xxx with AUTO
#brew install iproute2mac # uncomment if needed for following commands IP=`ip route get 8.8.8.8 | awk '/src/{ print $7 }'` sed -i '' -e"s/yyy/${IP}/g" ./AUTO.json sed -i '' -e"s/xxx/AUTO/g" ./AUTO.json
Example file looks like this after editing:
{ "name": "AUTO", "description": "OpenSource-AUTO", "cookbook_versions": { "Deploy-SDandC": "= 1.0.0" }, "json_class": "Chef::Environment", "chef_type": "environment", "default_attributes": { "disableHttp": false, "CS_VIP": "159.107.152.117", "BE_VIP": "159.107.152.117", "ONBOARDING_BE_VIP": "159.107.152.117", "FE_VIP": "159.107.152.117", "ES_VIP": "159.107.152.117", "KB_VIP": "159.107.152.117", "DCAE_BE_VIP": "159.107.152.117", "DCAE_FE_VIP": "159.107.152.117", "interfaces": { "application": "eth0", "private": "eth1" }, "ECompP": { "ecomp_rest_url": "http://portal.api.simpledemo.onap.org:8989/ONAPPORTAL/auxapi", "ueb_url_list": "10.0.11.1,10.0.11.1", "app_secret": "XftIATw9Jr3VzAcPqt3NnJOu", "app_key": "x9UfO7JsDn8BESVX", "inbox_name": "ECOMP-PORTAL-INBOX", "ecomp_redirect_url": "http://portal.api.simpledemo.openecomp.org:8989/ECOMPPORTAL/login.htm", "app_topic_name": "ECOMP-PORTAL-OUTBOX-SDC1", "decryption_key": "AGLDdG4D04BKm2IxIWEr8o==" }, "UEB": { "PublicKey": "iPIxkpAMI8qTcQj8", "SecretKey": "Ehq3WyT4bkif4zwgEbvshGal", "fqdn": ["10.0.11.1", "10.0.11.1"] }, "Nodes": { "CS": [ "159.107.152.117" ], "BE": "159.107.152.117", "ONBOARDING_BE": "159.107.152.117", "FE": "159.107.152.117", "ES": [ "159.107.152.117" ], "KB": "159.107.152.117" }, "Plugins": { "DCAE": { "dcae_discovery_url": "159.107.152.117", "dcae_source_url": "159.107.152.117" }, "WORKFLOW": { "workflow_discovery_url": "159.107.152.117", "workflow_source_url": "159.107.152.117" } }, "VnfRepo": { "vnfRepoPort": "8702", "vnfRepoHost": "192.168.50.5" } }, "override_attributes": { "FE": { "http_port": "8181", "https_port": "9443" }, "BE": { "http_port": "8080", "https_port": "8443" }, "ONBOARDING_BE": { "http_port": "8081", "https_port": "8445" }, "elasticsearch": { "cluster_name": "SDC-ES-", "ES_path_home": "/usr/share/elasticsearch", "ES_path_data": "/usr/share/elasticsearch/data", "num_of_replicas": "0", "num_of_shards": "1" }, "cassandra": { "concurrent_reads": "32", "num_tokens": "256", "data_dir": "/var/lib/cassandra/data", "hinted_handoff_enabled": "true", "cassandra_user": "asdc_user", "cassandra_password": "Aa1234%^!", "concurrent_writes": "32", "cluster_name": "SDC-CS-", "multithreaded_compaction": "false", "cache_dir": "/var/lib/cassandra/saved_caches", "log_file": "/var/lib/cassandra/log/system.log", "phi_convict_threshold": "8", "commitlog_dir": "/var/lib/cassandra/commitlog", "socket_read_timeout": "20000", "socket_connect_timeout": "20000", "titan_connection_timeout": "10000" } } }
Running containers:
Change directory to SDC project and export WORKSPACE env variable:
export WORKSPACE=$(pwd)
and you can start locally built containers now with:
sdc-os-chef/scripts/docker_run.sh -e AUTO -l
It takes a while to start them all and init cassandra, elastic and import normatives, but after sometime you should have:
All containers will be exposed (as per Docker for OSX) on localhost and your machine IP (en0 interface).
Troubleshooting:
- All scripts are located here: sdc-os-chef/scripts/, in case something isnt going right, modify first line /bin/bash to /bin/bash -xv for additional debugging.
- Make sure docker_run.sh command is executed only after WORKSPACE env var is defined, otherwise scripts will use / as path for data directory (look for files in /data and /opt)
11 Comments
Bharath Thiruveedula
Dejan Kitic, this is quite helpful, thank you!
One question, for testing distribution of network services, did you setup DMaaP too ? I couldn't find latest/proper documentation to setup DMaaP.
Dejan Kitic
No, I didn't, sorry.
Viswanath Kumar Skand Priya
Hi Dejan Kitic,
The instructions 99% worked for me. However I had to setup CS_HOST_IP manually in docker_run.sh inorder to get it through. Else it was trying to contact 127.0.0.1 port 9160 and was failing.
Viswanath Kumar Skand Priya
Dejan Kitic,
BTW I'm not able to bring up SDC simulator docker in mac. Docker4Mac doesn't mount /data properly. Any thoughts on this one?
Hitting with "docker: Error response from daemon: Mounts denied: EOF." error. I tried exposing /data & /etc/localtime in preferences, but no luck.
Viswanath Kumar Skand Priya
I was able to run the simulator natively and documented the steps here : Building & Running SDC Simulator natively in OSX
Mitchell Stafford
newbie fail ;-(
Following instructions from Building the Source Code above.
Is there something else that needs to be installed first?
mvn clean
install
-Pdocker
5. initial WARNINGs
6. Build error on the catalog.be
Viswanath Kumar Skand Priya
Make sure you have right settings.xml on your ~/.m2 repo
Mitchell Stafford
thanks. Actually the main problem that was encountered was the file too big message
For reference, I modified the pom.xml file in each of the following directories, and then finally the build was successful.
catalog-be / asdctool / ui-ci / test-apis-ci with the following line in the <configuration></configuration> section
Prathamesh Morde
To SDC experts, I am trying to run SDC locally on MacOS. I have followed all the instructions from this page but somehow I am getting this error after running this script
sdc-os-chef
/scripts/docker_run
.sh -e AUTO -l
It looks like it fails to start sdc-cassandra-init due to which other containers like (sdc-BE, sdc-FE etc are not able to start). This is the error I am getting
docker run sdc-cassandra-init...
[2019-03-07T21:40:51+00:00] INFO: Started chef-zero at chefzero://localhost:1 with repository at /root/chef-solo
One version per cookbook
[2019-03-07T21:40:51+00:00] INFO: Started chef-zero at chefzero://localhost:1 with repository at /root/chef-solo
One version per cookbook
[2019-03-07T21:40:51+00:00] INFO: Forking chef instance to converge...
[2019-03-07T21:40:51+00:00] INFO: Forking chef instance to converge...
Starting Chef Client, version 13.8.5
[2019-03-07T21:40:51+00:00] INFO: *** Chef 13.8.5 ***
[2019-03-07T21:40:51+00:00] INFO: *** Chef 13.8.5 ***
[2019-03-07T21:40:51+00:00] INFO: Platform: x86_64-linux-musl
[2019-03-07T21:40:51+00:00] INFO: Platform: x86_64-linux-musl
[2019-03-07T21:40:51+00:00] INFO: Chef-client pid: 14
[2019-03-07T21:40:51+00:00] INFO: Chef-client pid: 14
[2019-03-07T21:40:51+00:00] INFO: The plugin path /etc/chef/ohai/plugins does not exist. Skipping...
[2019-03-07T21:40:51+00:00] INFO: The plugin path /etc/chef/ohai/plugins does not exist. Skipping...
[2019-03-07T21:40:52+00:00] INFO: Setting the run_list to ["role[cassandra-actions]"] from CLI options
[2019-03-07T21:40:52+00:00] INFO: Setting the run_list to ["role[cassandra-actions]"] from CLI options
[2019-03-07T21:40:52+00:00] INFO: Run List is [role[cassandra-actions]]
[2019-03-07T21:40:52+00:00] INFO: Run List is [role[cassandra-actions]]
[2019-03-07T21:40:52+00:00] INFO: Run List expands to [cassandra-actions::01-createCsUser, cassandra-actions::03-schemaCreation, cassandra-actions::04-importComformance]
[2019-03-07T21:40:52+00:00] INFO: Run List expands to [cassandra-actions::01-createCsUser, cassandra-actions::03-schemaCreation, cassandra-actions::04-importComformance]
[2019-03-07T21:40:52+00:00] INFO: Starting Chef Run for 1eef96bea083
[2019-03-07T21:40:52+00:00] INFO: Starting Chef Run for 1eef96bea083
[2019-03-07T21:40:52+00:00] INFO: Running start handlers
[2019-03-07T21:40:52+00:00] INFO: Running start handlers
[2019-03-07T21:40:52+00:00] INFO: Start handlers complete.
[2019-03-07T21:40:52+00:00] INFO: Start handlers complete.
resolving cookbooks for run list: ["cassandra-actions::01-createCsUser", "cassandra-actions::03-schemaCreation", "cassandra-actions::04-importComformance"]
[2019-03-07T21:40:54+00:00] INFO: Loading cookbooks [cassandra-actions@0.0.0]
[2019-03-07T21:40:54+00:00] INFO: Loading cookbooks [cassandra-actions@0.0.0]
Synchronizing Cookbooks:
- cassandra-actions (0.0.0)
Installing Cookbook Gems:
Compiling Cookbooks...
================================================================================
Recipe Compile Error in /root/chef-solo/cache/cookbooks/cassandra-actions/recipes/03-schemaCreation.rb
================================================================================
NoMethodError
-------------
undefined method `+' for nil:NilClass
Cookbook Trace:
---------------
/root/chef-solo/cache/cookbooks/cassandra-actions/recipes/03-schemaCreation.rb:21:in `block in from_file'
/root/chef-solo/cache/cookbooks/cassandra-actions/recipes/03-schemaCreation.rb:15:in `from_file'
Relevant File Content:
----------------------
/root/chef-solo/cache/cookbooks/cassandra-actions/recipes/03-schemaCreation.rb:
14:
15: template "titan.properties" do
16: sensitive true
17: path "/tmp/sdctool/config/titan.properties"
18: source "titan.properties.erb"
19: mode "0755"
20: variables({
21>> :DC_NAME => node['cassandra']['datacenter_name']+node.chef_environment,
22: :cassandra_ip => node['Nodes']['CS'].first,
23: :cassandra_pwd => node['cassandra'][:cassandra_password],
24: :cassandra_usr => node['cassandra'][:cassandra_user],
25: :titan_connection_timeout => node['cassandra']['titan_connection_timeout']
26: })
27: end
28:
29:
30: template "/tmp/sdctool/config/configuration.yaml" do
System Info:
------------
chef_version=13.8.5
platform=alpine
platform_version=3.8.1
ruby=ruby 2.5.2p104 (2018-10-18 revision 65133) [x86_64-linux-musl]
program_name=chef-solo worker: ppid=8;start=21:40:51;
executable=/usr/bin/chef-solo
Running handlers:
[2019-03-07T21:40:55+00:00] ERROR: Running exception handlers
[2019-03-07T21:40:55+00:00] ERROR: Running exception handlers
Running handlers complete
[2019-03-07T21:40:55+00:00] ERROR: Exception handlers complete
[2019-03-07T21:40:55+00:00] ERROR: Exception handlers complete
Chef Client failed. 0 resources updated in 04 seconds
[2019-03-07T21:40:55+00:00] FATAL: Stacktrace dumped to /root/chef-solo/cache/chef-stacktrace.out
[2019-03-07T21:40:55+00:00] FATAL: Stacktrace dumped to /root/chef-solo/cache/chef-stacktrace.out
[2019-03-07T21:40:55+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2019-03-07T21:40:55+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2019-03-07T21:40:55+00:00] ERROR: undefined method `+' for nil:NilClass
[2019-03-07T21:40:55+00:00] ERROR: undefined method `+' for nil:NilClass
[2019-03-07T21:40:55+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
[2019-03-07T21:40:55+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
Anyone aware of this ?
James Guistwite
I got past this error by adding datacenter_name: "SDC-CS-" into the AUTO.json. Found the change in https://git.onap.org/sdc/tree/sdc-os-chef/environments/Template.json
James Guistwite
Rather than using the IP address of my host laptop, I substitute "host.docker.internal" for "yyy". Docker will automatically create a DNS (or hosts file - not sure) where this address resolves to the IP of the host machine. This allows me to run both on the corporate network and at home without constantly updating the AUTO.json file.