Jira ticket reference :
-
POLICY-4109Getting issue details...
STATUS
Problem: Policy Framework includes database-managed statistics, which are currently not being utilized as we have chosen to prioritize the use of Prometheus that offers seamless integration with Grafana for data visualization and analysis.
Deciding to remove the unused code will streamline our codebase and improve maintainability. Even unused code can sometimes have dependencies or unintended consequences that may affect other parts of the system, therefore a careful review and testing are needed.
Affected DB tables:
MariaDB [(none)]> SHOW DATABASES; | MariaDB [policyadmin]> SHOW TABLES; | MariaDB [policyadmin]> SELECT * FROM `pdpstatistics`; | SELECT * FROM `sequence` | jpapdpstatistics_enginestats; |
---|
Affected Components:
- Removing model and statistics service from PAP
Models PdpStatistics , JpaPdpStatistics
Pap PdpStatisticsService (*)
Api ApiStatisticsManager (Spring)
Distribution DistributionStatisticsManager (Prometheus)
Apex-pdp ApexPolicyStatisticsManager (Prometheus)
Drools-pdp PolicyStatsManager (drools/metrics/Metric.java)
Xacml XacmlPdpStatisticsManager (**) - Removing Statistics endpoints: PAP API DISTRIBUTION DROOLS-PDP XACML-PDP
- Removing DB-Migrator sequence and pdpstatistics tables
CISIT Runs after removing PAP statistics code:
PAP CSIT
APEX CSIT
XACML CSIT
Simulator error
policy-pap | org.onap.policy.pap.main.PolicyPapException: Failed to register PDP. No matching PdpGroup/SubGroup Found - PdpStatus(super=PdpMessage(messageName=PDP_STATUS, requestId=7364a2c2-e85f-4f10-8827-3628282d0e00, timestampMs=1677767874574, name=drools-apps.c8e40082-75bb-414a-a278-fcdec702f80c, pdpGroup=defaultGroup, pdpSubgroup=null), pdpType=drools, state=PASSIVE, healthy=HEALTHY, description=null, policies=[], deploymentInstanceInfo=null, properties=null, statistics=PdpStatistics(pdpInstanceId=drools-apps.c8e40082-75bb-414a-a278-fcdec702f80c, timeStamp=2023-03-02T14:37:54.574896Z, generatedId=null, pdpGroupName=defaultGroup, pdpSubGroupName=null, policyExecutedCount=0, policyExecutedSuccessCount=0, policyExecutedFailCount=0, policyDeployCount=0, policyDeploySuccessCount=0, policyDeployFailCount=0, policyUndeployCount=0, policyUndeploySuccessCount=0, policyUndeployFailCount=0, engineStats=null), response=null)
policy-xacml-pdp | [2023-03-06T10:07:13.183+00:00|INFO|HttpClient|DMAAP-source-POLICY-PDP-PAP] GET http://message-router:3904/events/POLICY-PDP-PAP/31197308-8836-4e7d-9dd5-a905cb5a665f/policy-xacml-pdp (anonymous) ...
policy-pap | [2023-03-06T10:07:13.265+00:00|ERROR|PdpStatusMessageHandler|DMAAP-source-POLICY-HEARTBEAT] Operation Failed
policy-pap | org.onap.policy.pap.main.PolicyPapException: Failed to register PDP. No matching PdpGroup/SubGroup Found - PdpStatus(super=PdpMessage(messageName=PDP_STATUS, requestId=fa94a3cc-489c-4540-b3f5-029350f0a574, timestampMs=1678097233050, name=xacml-2aba354c-5a42-4940-9975-4efb1fee0fb2, pdpGroup=defaultGroup, pdpSubgroup=null), pdpType=xacml, state=PASSIVE, healthy=HEALTHY, description=null, policies=[], deploymentInstanceInfo=null, properties=null, statistics=PdpStatistics(pdpInstanceId=null, timeStamp=+55146-10-09T17:23:27Z, generatedId=null, pdpGroupName=defaultGroup, pdpSubGroupName=null, policyExecutedCount=0, policyExecutedSuccessCount=0, policyExecutedFailCount=0, policyDeployCount=0, policyDeploySuccessCount=0, policyDeployFailCount=0, policyUndeployCount=0, policyUndeploySuccessCount=0, policyUndeployFailCount=0, engineStats=null), response=null)
policy-apex-pdp | [2023-03-06T10:08:09.761+00:00|INFO|CambriaSimplerBatchPublisher|pool-2-thread-1] cambria reply ok (104 ms):{"serverTimeMs":0,"count":1}
policy-pap | [2023-03-06T10:08:09.855+00:00|WARN|ErrorPacket|DMAAP-source-POLICY-HEARTBEAT] Error: 1146-42S02: Table 'policyadmin.statistics_sequence' doesn't exist
policy-pap | [2023-03-06T10:08:09.858+00:00|ERROR|TableGenerator|DMAAP-source-POLICY-HEARTBEAT] HHH000351: Could not read or init a hi value
policy-pap | java.sql.SQLSyntaxErrorException: (conn=307) Table 'policyadmin.statistics_sequence' doesn't exist
3 Comments
Suresh Charan
Francesco Davide Lapenta
In policy-distribution component, the statistics are based on the prometheus. We have reference to db based statistics (savePdpStatisticsInDb) only in testsuites/stability/src/main/resources/setup/config/papParameters.yaml . I think, removing this parameter configuration will be enough. Please correct me, If my understanding is wrong.
Francesco Davide Lapenta
Correct Suresh Charan the /metrics endpoint is based on the Prometheus.
There is still the statistics endpoint that we can remove:
https://github.com/onap/policy-distribution/blob/master/main/src/main/java/org/onap/policy/distribution/main/rest/DistributionRestController.java
Francesco Davide Lapenta
So If Prometheus counters are working correctly, then we don't need the /statistics endpoints since Prometheus generate /metrics endpoint automatically.
I agree that apex-pdp, drools-pdp and xacml-pdp don't need changes