Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: removed task descriptions

Wiki to track the design requirements for Helm generator to support 

Jira
serverONAP JIRA
serverId425b2b0a-557c-3c0c-b515-579789cceedb
keyDCAEGEN2-2694

USECASES

  1. Build helm chart generator taking following inputs (values.yaml) and templates to generate chart package and perform lint for consolidated charts.

Templates:

(Include dependent charts along with main chart so that the validation can be done)

                Helm lint (checks for syntax) – Initial validation can be just done with lint.

                Validation needs to be configured (default – enabled and can be disabled)

2. Separate Values.yaml into separate templates and verify dynamic values.yaml generation (and parameter substitution)

  • Configuration/parameters required common across MS
  • Configuration/parameters to be templatized and values sourced from ComponentSpec
  • Optional configuration/parameters to be templatized and included based on flags/properties from ComponentSpec

The generator must consolidate these separate base values.template and created required values.yaml

                TEST: Generated charts must be validated in ONAP lab K8S environment

Refer Sample Chart Yaml mapping from component spec and Sample Values yaml mapping from component spec in the requirements document.

3. Identify Component-spec schema changes for ENV setting mapping

4. Identify Component-spec schema changes for Service mapping (and nodeport)

5. Build helm chart generator taking as inputs template directory and template list file to be used for chart generation and perform lint for consolidated charts.

Note: Use base/default template if corresponding template not found on specified template directory

6. Support MAPPING requirement – ENV SETTING (refer REQ DOC for details)

7. Support MAPPING requirement – CMPv2 Certificates (refer REQ DOC for details)

8. Support MAPPING requirement – Postgres (refer REQ DOC for details)

9. Support MAPPING requirement – Policy Sidecar (refer REQ DOC for details)

10. Support SERVICE MAPPING based on spec file (refer REQ DOC for details)

11. Support MAPPING requirement – ConfigMap support (refer REQ DOC for details)

12. Support MAPPING requirement – DMAAP Secure Topic/Feed (refer REQ DOC for details)

13. Create user guide for the tool detailing all command-line options/override

14. Submit code to ONAP; ensure compliance to ONAP coding standard and test coverage requirement (atleast 80%) and verify the library build/pushed to ONAP:nexus

15. Integrate tool into MOD/Runtime or MOD2/CatalogService

16. Verify E2E for ONAP DCAE MS spec file (TCA w/policy, PM-Mapper, VES, and validate corresponding charts generated in ONAP lab if components can be successfully deployed

17. Add distribution support in tool. Additional configuration support needed for below parameters either in tool property file or CMD line options

DistributionEnabled

DistributionURL

DistributionUsername

DistributionPwd

DistributionFormat - tgz or as directory

18. Provide REST interface to support HELM generation

Attached is the draft use cases for Helm Generator 

View file
nameMOD-HelmGenerator-Usecase_v0.2.docx
height250

REQUIREMENTS

Attached is the draft requirements for Helm Generator

View file
nameMOD-HelmGenerator-Requirements_v0.2.docx
height250

Reference 

Discussed ppt slides  

Table of Contents

REQUIREMENTS

1. ENV SETTING SUPPORT

Component Spec

Code Block
languagejs
themeMidnight
"auxilary": {
.
.
"helm": {
 	"applicationEnv": {
    "PMSH_PG_URL": "dcae-pmsh-pg-primary",
    "PMSH_PG_USERNAME": {
    	"secretUid": "pgUserCredsSecretUid",
    	"key": "login"
    },
    "PMSH_PG_PASSWORD": {
         "secretUid": "pgUserCredsSecretUid",
         "key": "password"
    }
    }
}
.
.
}


Values.yaml specification


applicationEnv:
  PMSH_PG_URL: dcae-pmsh-pg-primary
  PMSH_PG_USERNAME:
     secretUid: pgUserCredsSecretUid
     key: login
  PMSH_PG_PASSWORD:
     secretUid: pgUserCredsSecretUid
     key: password

Note: Text in blue should be mapped from component-spec. If using secret UID, its responsibility of MS developer to include them also on values.yaml

Example

  - uid: &pgUserCredsSecretUid pg-user-creds
    name: &pgUserCredsSecretName '{{ include "common.release" . }}-pmsh-pg-user-creds'
    type: basicAuth
    externalSecret: '{{ ternary "" (tpl (default "" .Values.postgres.config.pgUserExternalSecret) .) (hasSuffix "pmsh-pg-user-creds" .Values.postgres.config.pgUserExternalSecret) }}'
    login: '{{ .Values.postgres.config.pgUserName }}'
    password: '{{ .Values.postgres.config.pgUserPassword }}'
    passwordPolicy: generate

2. CONFIG-MAP SUPPORT 

Component Spec

Code Block
languagepy
themeMidnight
  "config_map_volume": {
      "type": "object",
      "properties": {
        "config_volume": {
          "type": "object",
          "name": {
            "type": "string"
          }
        },
        "container": {
          "type": "object",
          "bind": {
            "type": "string"
          },
          "mode": {
            "type": "string"
          }
        }
      },
      "required": ["config_volume", "container"]
    },


Example:

                     

Code Block
languagepy
themeMidnight
"volumes": [{      
	"config_volume": {        
		"name": "dcae-external-repo-configmap-schema-map"      
		},      
	"container": {        
		"bind": "/opt/app/VESCollector/etc/externalRepo/"      
	}    
}, 
{      
	"config_volume": {        
		"name": "dcae-external-repo-configmap-sa88-rel16"      
		},      
	"container": {       
		 "bind": "/opt/app/VESCollector/etc/externalRepo/3gpp/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/"      
	}
  }
],

    
https://git.onap.org/dcaegen2/collectors/ves/tree/dpo/spec/vescollector-componentspec.json


Values.yaml specification


Code Block
languageyml
themeMidnight
externalVolumes:
  - name: dcae-external-repo-configmap-schema-map
    type: configmap
    mountPath: /opt/app/VESCollector/etc/externalRepo/
    optional: true (default)
  - name: '{{ include "common.release" . }}-another-example'   //dcae-external-repo-configmap-sa88-rel16
    type: configmap
    mountPath: /opt/app/VESCollector/etc/externalRepo/3gpp/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI
    optional: false  //If set to false, the configMap must be present in order for the microservice's pod to start. Defaults to true. 


3. CMVP2 Certificates support

Component Spec

Code Block
languageyml
themeMidnight

         "tls_info": {
          "description": "Component information to use tls certificates",
          "type": "object",
          "properties": {
            "cert_directory": {
              "description": "The path in the container where the component certificates will be placed by the init container",
              "type": "string"
            },
            "use_tls": {
              "description": "Boolean flag to determine if the application is using tls certificates",
              "type": "boolean"
            },
            "use_external_tls": {
              "description": "Boolean flag to determine if the application is using tls certificates for external communication",
              "type": "boolean"
            }
          },
          "required": [
            "cert_directory","use_tls"
          ],
          "additionalProperties": false
        },


Example:

Code Block
languagetext
themeMidnight
    "tls_info":{
	"cert_directory":"/opt/app/dcae-certificate/",
	"use_tls":true,
        "use_external_tls": true
     }

https://git.onap.org/dcaegen2/collectors/ves/tree/dpo/spec/vescollector-componentspec.json

Values.yaml specification


Code Block
languageyml
themeMidnight
# CMPv2 certificate
certificates:
  - mountPath: /opt/app/dcae-certificate/external
    commonName: dcae-ves-collector  --> from spec
    dnsNames:
      - dcae-ves-collector --> from spec
    keystore:
      outputType:
        - jks
      passwordSecretRef:
        name: ves-cmpv2-keystore-password  --> TBD
        key: password
        create: true

requirement.yaml

Code Block
languageyml
themeMidnight
     - name: certManagerCertificate
       version: ~8.x-0
       repository: '@local'

templates/certificates.yaml

Code Block
languageyml
themeMidnight
{{ if and .Values.certDirectory .Values.global.cmpv2Enabled .Values.global.CMPv2CertManagerIntegration }}
{{ include "certManagerCertificate.certificate" . }}
{{ end }}


4. POLICY SIDECAR SUPPORT

Component Spec

Code Block
languagejava
themeMidnight

    "policy_info": {
      "type": "object",
      "properties": {
        "policy":
        {
          "type": "array",
          "items":
          {
            "type": "object",
            "properties":
            {
              "node_label":
              {
                "type": "string"
              },
              "policy_id":
              {
                "type": "string"
              },
              "policy_model_id":
              {
                "type": "string"
              }
            },
            "required": ["node_label", "policy_model_id"]
          }
        }
      },
      "additionalProperties": false
    }
  }


Example: 

            

Code Block
languagejava
themeMidnight
	"policy_info":{
		"policy":[
			{
			    "node_label":"tca_policy_00",
			    "policy_model_id":"onap.policies.monitoring.cdap.tca.hi.lo.app"
				"policy_id":"tca_policy_id_10",
			},
			{
				"node_label":"tca_policy_11",
				"policy_id":"tca_policy_id_11",
				"policy_model_id":"onap.policies.monitoring.cdap.tca.hi.lo.app"
			}
		]
	}

Values.yaml specification


#dcaePolicySyncImage: onap/org.onap.dcaegen2.deployments.dcae-services-policy-sync:1.0.1  → From base template
policies:
  duration: 300  → default

  policyRelease: onap
  policyID: |
    '["tca_policy_id_11","tca_policy_id_10"]'  → coming from spec file

5. POSTGRES SUPPORT

Component Spec

Code Block
languagejava
themeMidnight
        "databases": {
          "description": "The databases the application is connecting to using the pgaas",
          "type": "object",
          "additionalProperties": {
            "type": "string",
            "enum": [
              "postgres"
            ]
          }
        },


  • Need secret suffix or retrieve from spec-name?

Values.yaml specification

Code Block
languageyml
themeMidnight
#################################################################
# Secrets Configuration.
#################################################################
secrets:
  - uid: pg-user-creds
    name: '{{ include "common.release" . }}-pmsh-pg-user-creds'
    type: basicAuth
    externalSecret: '{{ ternary "" (tpl (default "" .Values.postgres.config.pgUserExternalSecret) .) (hasSuffix "pmsh-pg-user-creds" .Values.postgres.config.pgUserExternalSecret) }}'
    login: '{{ .Values.postgres.config.pgUserName }}'
    password: '{{ .Values.postgres.config.pgUserPassword }}'
    passwordPolicy: generate


postgres:
  nameOverride: dcae-pmsh-postgres
  service:
    name: dcae-pmsh-postgres
    name2: dcae-pmsh-pg-primary
    name3: dcae-pmsh-pg-replica
  container:
    name:
      primary: dcae-pmsh-pg-primary
      replica: dcae-pmsh-pg-replica
  persistence:
    mountSubPath: pmsh/data
    mountInitPath: pmsh
  config:
    pgUserName: pmsh
    pgDatabase: pmsh
    pgUserExternalSecret: '{{ include "common.release" . }}-pmsh-pg-user-creds'

Note: applicationEnv setting if required should be mapped from spec as-is (req#1). Example above contains <pmsh> part of secret name and PG name which should be mapped to component-name from spec file


Requirement.yaml

Code Block
themeMidnight
titlerequirement yaml content
  - name: postgres
    version: ~8.x-0
    repository: '@local'
    condition: postgres.enabled


6. DMAAP – Secure Topic/Feed (WIP)

Component Spec

            TBD

Values.yaml specification

Code Block
languageyml
themeMidnight
#################################################################
# Secrets Configuration.
#################################################################
secrets:
  - uid: &aafCredsUID aafcreds
    type: basicAuth
    login: '{{ .Values.aafCreds.identity }}'
    password: '{{ .Values.aafCreds.password }}'
    passwordPolicy: required


# AAF Credentials
aafCreds:
  identity: dcae@dcae.onap.org
  password: demo123456!

credentials:
- name: AAF_USER
  uid: *aafCredsUID
  key: login
- name: AAF_PASSWORD
  uid: *aafCredsUID
  key: password

      Note: applicationConfig should use same names as defined under credentials

Example:

  enable_tls: true
  aaf_identity: ${AAF_USER}
  aaf_password: ${AAF_PASSWORD}
  streams_publishes:
      ves-3gpp-fault-supervision:
      type: kafka
      aaf_credentials:
        username: ${AAF_USER}
        password: ${AAF_PASSWORD}
      kafka_info:
        bootstrap_servers: message-router-kafka:9092
        topic_name: SEC_3GPP_FAULTSUPERVISION_OUTPUT

7. SERVICE MAPPING 

Component Spec

Code Block
languagejs
themeMidnight
"auxilary": {
.
.
"helm": {
      "services": [
        {
          "type": "NodePort",
          "name": "dcae-ves-collector",
          "ports": [
             {
                "name": "http",
                "port": 8443,
                "plain_port": 8080,
                "port_protocol": "http",
                "nodePort": 17,
                "useNodePortExt": true
             }
          ]
       }
      ]
    }
  }
.
.
}


  • Schema change required need to determine if nodeport vs clusterip
    • Require type/name/ports 
      • type - Nodeport or ClusterIPO
      • ports - list of objects mapped from spec as-is
      • constraints for ports can be added later

https://git.onap.org/dcaegen2/platform/tree/mod/component-json-schemas/component-specification/dcae-cli-v2/component-spec-schema.json

Values.yaml specification

Code Block
languageyml
themeMidnight
service:
  type: ClusterIP
  name: dcae-tcagen2
  ports:
    - port: 9091
      name: http

OR 

Code Block
languageyml
themeMidnight
global:
  nodePortPrefix: 302
  nodePortPrefixExt: 304

	# service configuration
service:
  type: NodePort
  name: dcae-ves-collector
  ports:
    - name: http
      port: 8443
      plain_port: 8080
      port_protocol: http
      nodePort: 17
      useNodePortExt: true

OR

Based on https://gerrit.onap.org/r/c/oom/+/121390

Code Block
languageyml
themeMidnight
service:
  type: NodePort
  name: dcae-ves-collector
  has_internal_only_ports: true
  ports:
    - name: http
      port: 8443
      plain_port: 8080
      port_protocol: http
      nodePort: 17
      useNodePortExt: true
    - name: metrics
      port: 4444
      internal_only: true

REVISED V3 SPEC

ComponentV3 SchemaV2 SchemaWith CMPV2With PostgresWith Policy
VESCollectorvescollector-componentspec-v3-helmvescollector-componentspecvescollector-componentspec-cmpv2-v3-helmvescollector-componentspec-postgres-v3-helm
TCAgen2tcagen2_spec-v3-helmtcagen2_spec

tcagen2_spec-policy-v3-helm
PRH prh-componentspec-v3-helm (pending test)prh-componentspec


hv_vescollectorhv-ves-collector-componentspec-v3-helm (pending test)hv-ves-collector.componentspec


PM-Mapperpmmapper-component-spec-v3-helm (need to update publisher and subscriber and pending test)pmmapper-component-spec


DataFileCollector (DFC)datafile-component-spec-v3-helm (need to update publisher and subscriber and pending test)datafile-component-spec



REFERENCE

Discussed ppt slides  Helm_deployment.pptx

MOD-HelmGenerator-Requirements_v0.2.docx

MOD-HelmGenerator-Usecase_v0.2.docx

...