You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Current »


Install and run clearwater automated  test suite


  1. Install the clearwater live testsuite on a VM
    1. could use the robot framework since its lightly loaded unless you want a VM that will be around if you want to re-install ONAP
  2. Get the public IPs for :
    1. bono-o (PROXY): 
    2. ellis-0 (ELLIS):
    3. sprout-0 (GEMINI) : 
    4. MOMENTO_HTTP: <same as sprout>
    5. MOMENTO_SID: <same as sprout>
  3. Get the domain from the preload data
    1. vimstest.onap.org
  4. The default Clearwater SIGNUP_CODE/password is "secret"
  5. Run the clearwater-live-tests

ubuntu@vm1-robot:~/clearwater-live-test$ rake test[vimstest.onap.org] SIGNUP_CODE=secret ELLIS=<ellis-0 IP> PROXY=<bono-0 IP> GEMINI=<sprout-0 IP> MEMENTO_SIP=<sprout-0 IP> MOMENTO_HTTP=<sprout-0 IP> ELLIS_API_KEY=secret



6. the automated test report is displayed in real time with Passed/Failed

7. The default configuration should have 

Clearwater Test Script:
 
ubuntu@vm1-robot:~/clearwater-live-test$ rake test[bdftest.openecomp.org] SIGNUP_CODE=secret ELLIS=172.99.116.54 PROXY=104.130.138.183 GEMINI=162.209.77.90 MEMENTO_SIP=162.209.77.90 MOMENTO_HTTP=162.209.77.90 ELLIS_API_KEY=secret
Basic Call - Mainline (TCP) - (5555551016, 5555551013) Passed
Basic Call - SDP (TCP) - (5555551013, 5555551002) Passed
Basic Call - Tel URIs (TCP) - (5555551001, 5555551015) Passed
Basic Call - Unknown number (TCP) - (5555551003, 5555551008) Passed
Basic Call - Rejected by remote endpoint (TCP) - (5555551015, 5555551003) Passed
Basic Call - Messages - Pager model (TCP) - (5555551000, 5555551004) Passed
Basic Call - Pracks (TCP) - (5555551015, 5555551017) Passed
Basic Registration (TCP) - (5555551010) Passed
Multiple Identities (TCP) - (5555551011, 5555551019) Passed
Call Barring - Outbound Rejection (TCP) - (5555551000, 5555551016) Passed
Call Barring - Allow non-international call (TCP) - Skipped (No PSTN support)
   - Call with PSTN=true to run test
Call Barring - Reject international call (TCP) - Skipped (No PSTN support)
   - Call with PSTN=true to run test
Call Barring - Inbound Rejection (TCP) - (5555551017, 5555551003) Passed
Call Diversion AS - Unconditional (TCP) - Skipped (No Call Diversion AS hostname provided)
   - Call with CDIV_AS=<hostname>
Call Diversion AS - Busy (TCP) - Skipped (No Call Diversion AS hostname provided)
   - Call with CDIV_AS=<hostname>
Call Diversion AS - No answer (TCP) - Skipped (No Call Diversion AS hostname provided)
   - Call with CDIV_AS=<hostname>
Call Diversion - Not registered (TCP) - (5555551009, 5555551010, 5555551011) Passed
Call Diversion - Not reachable (not registered) (TCP) - (5555551001, 5555551005, 5555551004) Passed
Call Diversion - Not reachable (408) (TCP) - (5555551014, 5555551011, 5555551003) Passed
Call Diversion - Not reachable (503) (TCP) - (5555551002, 5555551010, 5555551001) Passed
Call Diversion - Not reachable (500) (TCP) - (5555551015, 5555551012, 5555551001) Passed
Call Diversion - Busy (TCP) - (5555551006, 5555551001, 5555551007) Passed
Call Diversion - Unconditional (TCP) - (5555551007, 5555551010, 5555551008) Passed
Call Diversion - No answer (TCP) - (5555551004, 5555551001, 5555551000) Passed
Call Diversion - Bad target URI (TCP) - (5555551003, 5555551011) Passed
Call Diversion - Audio-only call (TCP) - (5555551014, 5555551016, 5555551013, 5555551010) Passed
Call Diversion - Audio-video call (TCP) - (5555551000, 5555551010, 5555551015, 5555551018) Passed
Call Waiting - Accepted (TCP) - (5555551012, 5555551003, 5555551019) Passed
Call Waiting - Cancelled (TCP) - (5555551011, 5555551013, 5555551012) Passed
CANCEL - Mainline (TCP) - (5555551009, 5555551000) Passed
Filtering - Accept-Contact (TCP) - (5555551009, 5555551003) Passed
Filtering - Accept-Contact no match (TCP) - (5555551005, 5555551001) Passed
Filtering - Accept-Contact negated match (TCP) - (5555551005, 5555551006) Passed
Filtering - RFC3841 example (TCP) - (5555551014, 5555551002) Passed
Filtering - Reject-Contact no match (TCP) - (5555551003, 5555551007) Passed
Filtering - Reject-Contact match (TCP) - (5555551008, 5555551001) Passed
Gemini - INVITE - VoIP device answers (TCP) - (5555551008, 5555551007, 1235555551007@bdftest.openecomp.org) Failed
Endpoint threw exception:
- undefined method `include?' for nil:NilClass
   - /home/ubuntu/clearwater-live-test/lib/tests/gemini.rb:72:in `block (2 levels) in <top (required)>'
Terminating other threads after failure
Gemini - INVITE - Mobile device answers (TCP) - (5555551006, 5555551015, 1235555551015@bdftest.openecomp.org) Failed
Endpoint threw exception:
- undefined method `include?' for nil:NilClass
   - /home/ubuntu/clearwater-live-test/lib/tests/gemini.rb:165:in `block (2 levels) in <top (required)>'
Terminating other threads after failure
Gemini - INVITE - VoIP device rejects (TCP) - (5555551016, 5555551011, 1235555551011@bdftest.openecomp.org) Failed
Endpoint threw exception:
- undefined method `include?' for nil:NilClass
   - /home/ubuntu/clearwater-live-test/lib/tests/gemini.rb:259:in `block (2 levels) in <top (required)>'
Terminating other threads after failure
Gemini - INVITE - Mobile device rejects (TCP) - (5555551017, 5555551005, 1235555551005@bdftest.openecomp.org) Failed
Endpoint threw exception:
- undefined method `include?' for nil:NilClass
   - /home/ubuntu/clearwater-live-test/lib/tests/gemini.rb:347:in `block (2 levels) in <top (required)>'
Terminating other threads after failure
Gemini - INVITE - Mobile device rejects with a 480 (TCP) - (5555551015, 5555551008, 1235555551008@bdftest.openecomp.org) Failed
Endpoint threw exception:
- undefined method `include?' for nil:NilClass
   - /home/ubuntu/clearwater-live-test/lib/tests/gemini.rb:443:in `block (2 levels) in <top (required)>'
Terminating other threads after failure
Endpoint threw exception:
- undefined method `include?' for nil:NilClass
   - /home/ubuntu/clearwater-live-test/lib/tests/gemini.rb:458:in `block (2 levels) in <top (required)>'
Gemini - INVITE - Both reject, choose mobile response (TCP) - (5555551012, 5555551010, 1235555551010@bdftest.openecomp.org) Failed
Endpoint threw exception:
- undefined method `include?' for nil:NilClass
   - /home/ubuntu/clearwater-live-test/lib/tests/gemini.rb:535:in `block (2 levels) in <top (required)>'
Terminating other threads after failure
Gemini - INVITE - Both reject, choose VoIP response (TCP) - (5555551004, 5555551016, 1235555551016@bdftest.openecomp.org) Failed
Endpoint threw exception:
- undefined method `include?' for nil:NilClass
   - /home/ubuntu/clearwater-live-test/lib/tests/gemini.rb:610:in `block (2 levels) in <top (required)>'
Terminating other threads after failure
Gemini - INVITE - Successful call with GR (TCP) - (5555551005, 5555551001, 1235555551001@bdftest.openecomp.org) Passed
Gemini - INVITE - Failed call with GR (TCP) - (5555551009, 5555551017, 1235555551017@bdftest.openecomp.org) Passed
Gemini - INVITE - Successful call with Accept-Contact (TCP) - (5555551007, 5555551001, 1235555551001@bdftest.openecomp.org) Failed
Endpoint threw exception:
- sip:1235555551001@bdftest.openecomp.org timed out waiting for new incoming call
   - /home/ubuntu/.bundle/ruby/2.3.0/bundler/gems/quaff-f92372cac3e1/lib/endpoint.rb:65:in `rescue in incoming_call'
   - /home/ubuntu/.bundle/ruby/2.3.0/bundler/gems/quaff-f92372cac3e1/lib/endpoint.rb:62:in `incoming_call'
   - /home/ubuntu/clearwater-live-test/lib/tests/gemini.rb:808:in `block (2 levels) in <top (required)>'
Terminating other threads after failure
Gemini - INVITE - Failed call with Accept-Contact (TCP) - (5555551013, 5555551017, 1235555551017@bdftest.openecomp.org) Failed
Endpoint threw exception:
- sip:1235555551017@bdftest.openecomp.org timed out waiting for new incoming call
   - /home/ubuntu/.bundle/ruby/2.3.0/bundler/gems/quaff-f92372cac3e1/lib/endpoint.rb:65:in `rescue in incoming_call'
   - /home/ubuntu/.bundle/ruby/2.3.0/bundler/gems/quaff-f92372cac3e1/lib/endpoint.rb:62:in `incoming_call'
   - /home/ubuntu/clearwater-live-test/lib/tests/gemini.rb:864:in `block (2 levels) in <top (required)>'
Terminating other threads after failure
Gemini - SUBSCRIBE - Mobile Notifies (TCP) - (5555551013, 5555551006, 1235555551006@bdftest.openecomp.org) Failed
Endpoint threw exception:
- undefined method `include?' for nil:NilClass
   - /home/ubuntu/clearwater-live-test/lib/tests/gemini.rb:926:in `block (2 levels) in <top (required)>'
Terminating other threads after failure
Gemini - SUBSCRIBE - Joint 408 (TCP) - (5555551019, 5555551009, 1235555551009@bdftest.openecomp.org) Failed
Endpoint threw exception:
- undefined method `include?' for nil:NilClass
   - /home/ubuntu/clearwater-live-test/lib/tests/gemini.rb:988:in `block (2 levels) in <top (required)>'
Terminating other threads after failure
GRUU - REGISTER - two bindings with and without GRUU (TCP) - (5555551010) Passed
GRUU - REGISTER - binding suggested GRUU (TCP) - (5555551009) Passed
GRUU - REGISTER - instance ID requires escaping (TCP) - (5555551011) Passed
GRUU - REGISTER - three bindings with GRUUs (TCP) - (5555551012) Passed
GRUU - Call - first endpoint GRUU as target (TCP) - (5555551014, 5555551010) Passed
GRUU - Call - second endpoint GRUU as target (TCP) - (5555551017, 5555551009) Passed
GRUU - Call - only GRUU as target (TCP) - (5555551005, 5555551015) Passed
GRUU - Call - AoR as target (TCP) - (5555551003, 5555551000) Passed
GRUU - Call - unknown GRUU as target (TCP) - (5555551006, 5555551000) Passed
GRUU - Call - unknown GRUU as target - no GRUUs assigned (TCP) - (5555551010, 5555551011) Passed
GRUU - Call - Reject-Contact interop (TCP) - (5555551006, 5555551003) Passed
GRUU - Call - Accept-Contact interop (TCP) - (5555551001, 5555551012) Passed
GRUU - Call - AoR with other param as target (TCP) - (5555551015, 5555551007) Passed
GRUU - Call - GRUU with other param as target (TCP) - (5555551011, 5555551008) Passed
ISC Interface - Terminating (TCP) - Skipped (No hostname given)
   - Call with HOSTNAME=<publicly accessible hostname/IP of this machine>
ISC Interface - Terminating (UDP AS) (TCP) - Skipped (No hostname given)
   - Call with HOSTNAME=<publicly accessible hostname/IP of this machine>
ISC Interface - Terminating Failed (TCP) - Skipped (No hostname given)
   - Call with HOSTNAME=<publicly accessible hostname/IP of this machine>
ISC Interface - Third-party Registration (TCP) - Skipped (No hostname given)
   - Call with HOSTNAME=<publicly accessible hostname/IP of this machine>
ISC Interface - Third-party Registration - implicit registration (TCP) - Skipped (No hostname given)
   - Call with HOSTNAME=<publicly accessible hostname/IP of this machine>
ISC Interface - Redirect (TCP) - Skipped (No hostname given)
   - Call with HOSTNAME=<publicly accessible hostname/IP of this machine>
ISC Interface - B2BUA (TCP) - Skipped (No hostname given)
   - Call with HOSTNAME=<publicly accessible hostname/IP of this machine>
Live Call - Dial out to a real number (TCP) - Skipped (No live number given)
   - Call with LIVENUMBER=<number>
Memento - Incorrect Password (TCP) - Skipped (No memento hostnames provided)
   - Call with MEMENTO_SIP=<SIP hostname> and MEMENTO_HTTP=<HTTP hostname>
Memento - Wrong Call List (TCP) - Skipped (No memento hostnames provided)
   - Call with MEMENTO_SIP=<SIP hostname> and MEMENTO_HTTP=<HTTP hostname>
Memento - Basic Call (TCP) - Skipped (No memento hostnames provided)
   - Call with MEMENTO_SIP=<SIP hostname> and MEMENTO_HTTP=<HTTP hostname>
Memento - Unknown Number (TCP) - Skipped (No memento hostnames provided)
   - Call with MEMENTO_SIP=<SIP hostname> and MEMENTO_HTTP=<HTTP hostname>
Memento - Rejected Call (TCP) - Skipped (No memento hostnames provided)
   - Call with MEMENTO_SIP=<SIP hostname> and MEMENTO_HTTP=<HTTP hostname>
Memento - Cancelled Call (TCP) - Skipped (No memento hostnames provided)
   - Call with MEMENTO_SIP=<SIP hostname> and MEMENTO_HTTP=<HTTP hostname>
Memento - Privacy Call (TCP) - Skipped (No memento hostnames provided)
   - Call with MEMENTO_SIP=<SIP hostname> and MEMENTO_HTTP=<HTTP hostname>
Memento - Barred Call (TCP) - Skipped (No memento hostnames provided)
   - Call with MEMENTO_SIP=<SIP hostname> and MEMENTO_HTTP=<HTTP hostname>
Memento - Busy Call Forwarding (TCP) - Skipped (No memento hostnames provided)
   - Call with MEMENTO_SIP=<SIP hostname> and MEMENTO_HTTP=<HTTP hostname>
Nonce-Count - Mainline (TCP) - Skipped (No nonce-count support)
   - Call with NONCE_COUNT=true to run test
Nonce-Count - Reject Re-Use (TCP) - Skipped (No nonce-count support)
   - Call with NONCE_COUNT=true to run test
Off-net calls - tel: URI (TCP) - Skipped (No off-net number given)
   - Call with OFF_NET_TEL=<a number set up in ENUM/BGCF to route to port 5072 on this machine>
Off-net calls - sip: URI (TCP) - Skipped (No off-net number given)
   - Call with OFF_NET_TEL=<a number set up in ENUM/BGCF to route to port 5072 on this machine>
SUBSCRIBE - reg-event (TCP) - (5555551011) Passed
SUBSCRIBE - reg-event with a GRUU (TCP) - (5555551017) Passed
SUBSCRIBE - Subscription timeout (TCP) - (5555551003) Passed
SUBSCRIBE - Registration timeout (TCP) - (5555551012, 5555551010) Passed
11 failures out of 88 tests run
    Gemini - INVITE - VoIP device answers (TCP) at 2017-07-13 00:35:20 +0000
    Gemini - INVITE - Mobile device answers (TCP) at 2017-07-13 00:35:25 +0000
    Gemini - INVITE - VoIP device rejects (TCP) at 2017-07-13 00:35:30 +0000
    Gemini - INVITE - Mobile device rejects (TCP) at 2017-07-13 00:35:35 +0000
    Gemini - INVITE - Mobile device rejects with a 480 (TCP) at 2017-07-13 00:35:41 +0000
    Gemini - INVITE - Both reject, choose mobile response (TCP) at 2017-07-13 00:35:46 +0000
    Gemini - INVITE - Both reject, choose VoIP response (TCP) at 2017-07-13 00:35:51 +0000
    Gemini - INVITE - Successful call with Accept-Contact (TCP) at 2017-07-13 00:36:36 +0000
    Gemini - INVITE - Failed call with Accept-Contact (TCP) at 2017-07-13 00:37:10 +0000
    Gemini - SUBSCRIBE - Mobile Notifies (TCP) at 2017-07-13 00:37:14 +0000
    Gemini - SUBSCRIBE - Joint 408 (TCP) at 2017-07-13 00:37:19 +0000
26 tests skipped
Error logs, including Call-IDs of failed calls, are in the 'logfiles' directory




  • No labels