Problems/Issues encountered or steps required when using ONAP Dublin after a fresh installation

Portal HTTP 500

PORTAL-571 - Getting issue details... STATUS  

DMaaP message router pods in ContainerConfigError

Restart kubelet container in affected hosts


Policy PODs in CrashLoopBackOff

POLICY-1640 - Getting issue details... STATUS

Most of SO PODs in CrashLoopBackOff

See  SO-1674 - Getting issue details... STATUS  and https://lists.onap.org/g/onap-discuss/message/16509 

workaround
1. Connect to MariaDB container
2. mysql -uroot
3. use mysql;
4. SET PASSWORD FOR 'root'@'localhost' = PASSWORD('password');

5. cd docker-entrypoint-initdb.d
6. Run scripts 01-… to 05-…


Wait for SO PODs to become in Running state

Remove duplicate objects from A&AI

Context of issue is described here

For instance, we had many duplicate Logical-links and we could not fetch/delete them.

So, we have to first get their Vertex IDs so that we can manually delete them from Graph database.

Fetching Vertex IDs
curl -X GET \
'https://172.30.0.90:30233/aai/v16/network/logical-links?format=id' \
-H 'Accept: application/json' \
-H 'Authorization: Basic QUFJOkFBSQ==' \
-H 'Content-Type: application/json' \
-H 'Postman-Token: 5f14e474-f704-4f48-859e-ac1977e7243c' \
-H 'X-FromAppId: Postman Application' \
-H 'X-TransactionId: Postman REST Transaction' \
-H 'cache-control: no-cache'

An example of an output is shown the official documentation


Upon fetching all the vertex IDs that need to be manually removed, for each one of them we execute the following command (replace the graph-admin POD and VERTEX_ID placeholders as per your environment)

Manually Remove Objects from A&AI
kubectl exec -it <graph-admin-pod> -- bash -c 'gosu aaiadmin ./scripts/forceDeleteTool.sh -action DELETE_NODE -userId testId1 -vertexId <VERTEX_ID>'

For example, here is one of the executions to delete a duplicate entry of a logical-link

Command Output
root@onap-rancher-daily:/home/ubuntu# kubectl exec -it dev-aai-aai-graphadmin-c8b9c58c5-kh7t7 -- bash -c 'gosu aaiadmin ./scripts/forceDeleteTool.sh -action DELETE_NODE -userId testId1 -vertexId 749696'

Defaulting container name to aai-graphadmin.
Use 'kubectl describe pod/dev-aai-aai-graphadmin-c8b9c58c5-kh7t7 -n onap' to see all of the containers in this pod.

Wed Jul 3 08:20:16 UTC 2019 Starting ./scripts/forceDeleteTool.sh
NOTE - if you are deleting data, please run the dataSnapshot.sh script first or
at least make a note the details of the node that you are deleting.
---- NOTE --- about to open graph (takes a little while)--------

ForceDelete called by: userId [testId1] with these params: [ -action DELETE_NODE -userId testId1 -vertexId 749696]
>>> Found Vertex with VertexId = 749696, properties:
[resource-version|1560082563941]
[in-maint|false]
[last-mod-source-of-truth|prh]
[aai-created-ts|1560082563941]
[aai-last-mod-ts|1560082563941]
[source-of-truth|prh]
[aai-uri|/network/logical-links/logical-link/167772160-3-0]
[aai-uuid|5686dadd-d40d-454f-9018-5a2afb8656ba]
[link-name|167772160-3-0]
[link-type|attachment-point]
[aai-node-type|logical-link]
No OUT edges were found for this vertex.
No IN edges were found for this vertex.
Found 0 descendant nodes. Note - forceDelete does not cascade to child nodes, but they may become unreachable after the delete.

Found total of 0 edges incident on this node.


Are you sure you want to do this delete? (y/n): y

User [testId1] has confirmed this delete request.
>>>>>>>>>> Removed node with vertexId = 749696
Failed to run the tool ./scripts/forceDeleteTool.sh successfully
command terminated with exit code 1
  • No labels