References
CPS-2055
-
Getting issue details...
STATUS
<optional other relevant references, Jiras, Confluence pages, external links>
Assumptions
<optional, assumptions are like decision made up front ie. everyone agrees on the answer but they are important to mention>
| # | Assumption | Notes |
|---|---|---|
| 1 | Decisions will be agreed with stakeholders |
Issues & Decisions
| Issue | Notes | Decision | |
|---|---|---|---|
| 1 | Acceptance criteria for bug tickets | What should we do if a bug not reproducible? | |
| 2 | Standard Bug Reporting Template | The minimal information needed for all bug tickets | |
| 3 | NCMP Bug Reports | There is specific information needed for reproducing NCMP bugs, e.g. number of CM-handles | |
| 4 | Performance Bug Reports | Performance issues are much more difficult to reproduce than functional issues | |
| 5 | Handling of sensitive information such as logs and heap dumps | Need to protect IPR of commercial contributors | |
| 6 | Bug reporting guidelines | A wiki page showing how to write a good bug report |
<Note. use green for closed issues, yellow for important ones if needed>
Background
It has been observed that some bugs (particularly performance-related ones) can take weeks to resolve, and often cannot be reproduced by the CPS dev team.
This proposal is to improve ways of working around bugs to reduce time spent.
Bug Reporting Template
Many bug tickets lack basic information such as what version was tested.
It is proposed that in addition to CPS FSA Template, a Bug Reporting Template be created. At a minimum, the following information should be provided:
- Full description of the issue
- Affected version(s)
- Expected behaviour
- Actual behaviour
- Impact - this is important for setting priority
- Steps to reproduce - ideally a Minimal reproducible example
- (Optional) Attached artifacts: Screenshots, Logs, Test data, etc.
NCMP Bugs
It is observed that there has been much confusion about the number of CM-handles the bug reporters are testing with. In many tickets, phrases such as "80k deployment" is used, but in some cases this was 6k CM-handles, and 20k for others!
- How many CM-handles are registered/queried?
Performance Bugs
- Available resources in the deployment (memory and CPU cores, number of application instances)
- What is the load on the system (how many concurrent operations, etc.)?
- Measured CPU and memory consumption
- For Out Of Memory Errors (OOMEs): Attached heap dump - I think this should be a requirement moving forward
Acceptance Criteria for Bugs
We need to agree on reasonable criteria, e.g.
- All information in the Bug Reporting Template is provided, especially steps to reproduce
- Bug reporter tests with the latest released version?
- But what do we do when a bug is not reproducible by us?
Sharing artifacts such as Logs in the Open Source
Some bugs being reported often require sharing information such as logs, heap dumps, etc. This needs to be done in a way that IPR is protected, since CPS is developed in the open source.