Motivation

The most obvious solution for taking full advantage of CSIT is to add the test cases under the same repository with the functionalities that they are testing (instead of having them in separate centralized CSIT repository as we currently have). This would have the following benefits:

• CSIT could be triggered by any commit to the project repo
• CSIT tests the code (or specifically, docker images that have been built) from the committed branch
• CSIT could have a vote on the commit based on the result of the test run
• If the implementation changes require changes in CSIT tests (to cover new functionality or to pass in the first place), that could be handled within the same commit
• ideally, local verification would become less complex (no need to work between CSIT repo and project repo) 

Issues

Given the fact that CSITs are currently a very colourful collection of various suites with different scopes and strategies, the transition of CSITs to project repositories is not necessarily trivial:

• CSIT suites that test components from multiple project repositories at the same time 
  • such CSIT tests may have to be separated using additional simulators, or
  • project repository structures themselves may have to be reconsidered
• Jenkins templates may have to be redesigned to support unified approach for triggering review branch-specific docker build, CSIT execution and voting chain as part of review verification
• CSITs would become blockers for merging code
  • local pre-commit verification should be supported better by common CSIT tools

  • are all projects and suites mature enough to deal with that?

Technical details to be decided

• Should we keep separate docker build and CSIT jobs and just chain them into review verification, or should we try to incorporate docker building and CSIT execution into existing review jobs?
  • Either way, JJB templates will probably have to be touched
• Should we still have common CSIT scripts (run-csit.sh etc) in CSIT repo and related procedures (setup, tests, teardown and related result collection) as the basis of project-specific test execution?  

Project status and readiness at the end of Guilin