...

TLDR: Tracing has been added for A1 Policy Management Service. By default tracing is disabled. To enable it change there are two ways:

A) System Property

Change the flag otel.sdk.disabled to false in the application.yaml (New Delhi) 

otel:
  sdk:     
    disabled: ${ONAP_SDK_DISABLED:false}
    south: ${ONAP_TRACING_SOUTHBOUND:true}   
  instrumentation:
    spring-webflux:
      enabled: ${OTEL_INSTRUMENTATION_SPRING_WEBFLUX_ENABLED:true}

...

true

1. otel.sdk.disabled: enable/disable tracing all toghether
2. otel.sdk.south: if ONAP_SDK_DISABLED is false then we can enable/disable southbound tracing
3. otel.instrumentation.spring-webflux.enabled: if ONAP_SDK_DISABLED is false we can enable/disable northbound tracing

B) Enviroment Variable

Have the environment variables, this way you don't need to change the application.yaml and rebuild the docker image

ONAP_SDK_DISABLED=false
ONAP_TRACING_SOUTHBOUND=true
OTEL_INSTRUMENTATION_SPRING_WEBFLUX_ENABLED=true

1. ONAP_SDK_DISABLED: enable/disable tracing all toghether
2. ONAP_TRACING_SOUTHBOUND: if ONAP_SDK_DISABLED is false then we can enable/disable southbound tracing
3. OTEL_INSTRUMENTATION_SPRING_WEBFLUX_ENABLED: if ONAP_SDK_DISABLED is false we can enable/disable northbound tracing

Possible Combinations 

So we can have the following combinations:

...

a) A docker compose with a1pms, a1-osc-simulator, and jaeger that acts as a collector and exporter. Note: onap/ccsdk-oran-a1policymanagementservice:1.7.0-SNAPSHOT is built locally by doing "mvn clean install", you can use the nexus hosted image changing the prefix.

version: '3.7'
services:
  a1_policy_management:
    container_name: a1-pms
    image: onap/ccsdk-oran-a1policymanagementservice:1.7.0-SNAPSHOT
    ports:
      - "8433:8433"
      - "8081:8081"
    volumes:
      - ./application_configuration.json.nosdnc:/opt/app/policy-agent/data/application_configuration.json:ro
    networks:
      - jaeger-example
    depends_on:
      - jaeger
    environment:
      - ONAP_SDK_DISABLED=false
      - ONAP_TRACING_SOUTHBOUND=true
      - OTEL_INSTRUMENTATION_SPRING_WEBFLUX_ENABLED=true
      - ONAP_OTEL_SAMPLER_JAEGER_REMOTE_ENDPOINT=http://jaeger:14250
      - ONAP_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
      - ONAP_OTEL_EXPORTER_PROTOCOL=grpc
      - ONAP_OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=grpc

  a1-sim-OSC:
    image: "nexus3.o-ran-sc.org:10002/o-ran-sc/a1-simulator:2.1.0"
    container_name: a1-sim-OSC
    ports:
      - "30001:8085"
      - "30002:8185"
    environment:
      - A1_VERSION=OSC_2.1.0
      - REMOTE_HOSTS_LOGGING=1
      - ALLOW_HTTP=true
    networks:
      - jaeger-example

  jaeger:
    image: jaegertracing/all-in-one:latest
    container_name: jaeger
    ports:
      - "16686:16686"
      - "14250:14250"
      - "14268:14268"
      - "4317:4317"
      - "4318:4318"
    environment:
      - JAEGER_DISABLED=true
      - LOG_LEVEL=debug
      - COLLECTOR_OTLP_ENABLED=true
    networks:
      - jaeger-example

networks:
  jaeger-example:
    driver: bridge

...

e) http://localhost:16686/ Load Jaeger UI, a1-pms traces, and a sample of the last call would be:
Image Removed
Image Added


Steps Taken and Challenges:

...

OTEL_INSTRUMENTATION_SPRING_WEBFLUX_ENABLED=true from the documentation we can use this flag to disable the automatic spring instrumentation and we keep a separate manual flag ONAP_TRACING_SOUTHBOUND for the AsyncRestClient requests made to the southbound.

System property: otel.instrumentation.[name].enabled
Environment variable: OTEL_INSTRUMENTATION_[NAME]_ENABLED
Note: When using OPENTELEMETRY (Evrything starting with otel) environment variables, dashes (-) should be converted to underscores (_). For example, to suppress traces from spring-webflux library, set OTEL_INSTRUMENTATION_SPRING_WEBFLUX_ENABLED to false

Full Tracing:
Image Added

Only Southbound Tracing Output:
Image Added

Only Northbound Tracing Output:

Image Added