...
This only needs to be done one time and, at the moment, all modifications are done manually (they can be automated via scripting in the future when the needs come up).
Under oom project check out the patch:
git fetch https://haok@gerrit.onap.org/r/a/oom refs/changes/67/25467/11 && git checkout FETCH_HEAD
or you can follow the change details to update yaml files.
Change details
Edit SDNC Templates
The following is the list of SDNC deployment templates which need to be modified for an SDN-C cluster deployment.
under directory {$OOM}/kubernetes/sdnc
# | Files under | Changed/Added fields and values |
---|
1 | values.yaml | Defined a new variable "mysql: mysql:5.6" |
under directory {$OOM}/kubernetes/sdnc/templates
# | file name | Changed/Added fields and values |
---|
0 | (Create this file) sdnc-configmap.yaml | Code Block |
---|
| apiVersion: v1
kind: ConfigMap
metadata:
name: mysql
namespace: "{{ .Values.nsPrefix }}-sdnc"
labels:
app: mysql
data:
master.cnf: |
# Apply this config only on the master.
[mysqld]
log-bin
[localpathprefix]
master
slave.cnf: |
# Apply this config only on slaves.
[mysqld]
super-read-only
[localpathprefix]
slave |
|
1 | db-deployment.yaml | Field | New value | Old value |
---|
.apiVersion | apps/v1beta1 1 | extensions/v1beta1 | .kind | StatefulSet | Deployment | .spec.serviceName | "dbhost" 2 | N/A | .spec.replicas | 2 | N/A | .spec.initContainers | Code Block |
---|
title | Init Containers |
---|
collapse | true |
---|
| initContainers:
- name: init-mysql
image: {{ .Values.image.mysql }}
imagePullPolicy: {{ .Values.pullPolicy }}
command:
- bash
- "-c"
- |
set -ex
# Generate mysql server-id from pod ordinal index.
[[ `hostname` =~ -([0-9]+)$ ]] || exit 1
ordinal=${BASH_REMATCH[1]}
echo [mysqld] > /mnt/conf.d/server-id.cnf
# Add an offset to avoid reserved server-id=0 value.
echo server-id=$((100 + $ordinal)) >> /mnt/conf.d/server-id.cnf
# Copy appropriate conf.d files from config-map to emptyDir.
if [[ $ordinal -eq 0 ]]; then
cp /mnt/config-map/master.cnf /mnt/conf.d/
else
cp /mnt/config-map/slave.cnf /mnt/conf.d/
fi
volumeMounts:
- name: conf
mountPath: /mnt/conf.d
- name: config-map
mountPath: /mnt/config-map
- name: clone-mysql
image: gcr.io/google-samples/xtrabackup:1.0
command:
- bash
- "-c"
- |
set -ex
# Skip the clone if data already exists.
[[ -d /var/lib/mysql/mysql ]] && exit 0
# Skip the clone on master (ordinal index 0).
[[ `hostname` =~ -([0-9]+)$ ]] || exit 1
ordinal=${BASH_REMATCH[1]}
[[ $ordinal -eq 0 ]] && exit 0
# Clone data from previous peer.
ncat --recv-only sdnc-dbhost-$(($ordinal-1)).dbhost.{{ .Values.nsPrefix }}-sdnc 3307 | xbstream -x -C /var/lib/mysql
# Prepare the backup.
xtrabackup --prepare --target-dir=/var/lib/mysql
volumeMounts:
- name: sdnc-data
mountPath: /var/lib/mysql
subPath: mysql
- name: conf
mountPath: /etc/mysql/conf.d
|
| N/A | .spec.containers | Code Block |
---|
title | Containers |
---|
collapse | true |
---|
| containers:
- env:
- name: MYSQL_ROOT_PASSWORD
value: openECOMP1.0
- name: MYSQL_ROOT_HOST
value: '%'
image: {{ .Values.image.mysqlServer }}
imagePullPolicy: {{ .Values.pullPolicy }}
name: sdnc-db-container
volumeMounts:
- mountPath: /var/lib/mysql
name: sdnc-data
subPath: mysql
- name: conf
mountPath: /etc/mysql/conf.d
ports:
- containerPort: 3306
resources:
requests:
cpu: 500m
memory: 1Gi
livenessProbe:
exec:
command: ["mysqladmin", "ping"]
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
readinessProbe:
tcpSocket:
port: 3306
initialDelaySeconds: 5
periodSeconds: 10
- name: xtrabackup
image: gcr.io/google-samples/xtrabackup:1.0
env:
- name: MYSQL_ROOT_PASSWORD
value: openECOMP1.0
- name: MYSQL_ROOT_HOST
value: '%'
ports:
- name: xtrabackup
containerPort: 3307
command:
- bash
- "-c"
- |
set -ex
cd /var/lib/mysql
# Determine binlog position of cloned data, if any.
if [[ -f xtrabackup_slave_info ]]; then
# XtraBackup already generated a partial "CHANGE MASTER TO" query
# because we're cloning from an existing slave.
mv xtrabackup_slave_info change_master_to.sql.in
# Ignore xtrabackup_binlog_info in this case (it's useless).
rm -f xtrabackup_binlog_info
elif [[ -f xtrabackup_binlog_info ]]; then
# We're cloning directly from master. Parse binlog position.
[[ `cat xtrabackup_binlog_info` =~ ^(.*?)[[:space:]]+(.*?)$ ]] || exit 1
rm xtrabackup_binlog_info
echo "CHANGE MASTER TO MASTER_LOG_FILE='${BASH_REMATCH[1]}',\
MASTER_LOG_POS=${BASH_REMATCH[2]}" > change_master_to.sql.in
fi
# Check if we need to complete a clone by starting replication.
if [[ -f change_master_to.sql.in ]]; then
echo "Waiting for mysqld to be ready (accepting connections)"
until mysql --user=root --password=$MYSQL_ROOT_PASSWORD -h 127.0.0.1 -e "SELECT 1"; do sleep 1; done
echo "Initializing replication from clone position"
# In case of container restart, attempt this at-most-once.
mv change_master_to.sql.in change_master_to.sql.orig
mysql --user=root --password=$MYSQL_ROOT_PASSWORD -h 127.0.0.1 <<EOF
$(<change_master_to.sql.orig),
MASTER_HOST='sdnc-dbhost-0.dbhost.{{ .Values.nsPrefix }}-sdnc',
MASTER_USER="root",
MASTER_PASSWORD=$MYSQL_ROOT_PASSWORD,
MASTER_CONNECT_RETRY=10;
START SLAVE;
EOF
fi
# Start a server to send backups when requested by peers.
exec ncat --listen --keep-open --send-only --max-conns=1 3307 -c \
"xtrabackup --backup --slave-info --stream=xbstream --host=127.0.0.1 --user=root --password=$MYSQL_ROOT_PASSWORD"
volumeMounts:
- name: sdnc-data
mountPath: /var/lib/mysql
subPath: mysql
- name: conf
mountPath: /etc/mysql/conf.d
resources:
requests:
cpu: 100m
memory: 100Mi |
| Code Block |
---|
title | Containers |
---|
collapse | true |
---|
| containers:
- env:
- name: MYSQL_ROOT_PASSWORD
value: openECOMP1.0
- name: MYSQL_ROOT_HOST
value: '%'
image: {{ .Values.image.mysqlServer }}
imagePullPolicy: {{ .Values.pullPolicy }}
name: sdnc-db-container
volumeMounts:
- mountPath: /etc/localtime
name: localtime
readOnly: true
- mountPath: /var/lib/mysql
name: sdnc-data
ports:
- containerPort: 3306
readinessProbe:
tcpSocket:
port: 3306
initialDelaySeconds: 5
periodSeconds: 10 |
| .spec.volumes | Code Block |
---|
title | Dynamic Volume Claim |
---|
collapse | true |
---|
| volumeClaimTemplates:
- metadata:
name: sdnc-data
annotations:
volume.beta.kubernetes.io/storage-class: "{{ .Values.nsPrefix }}-sdnc-data"
spec:
accessModes: ["ReadWriteMany"]
resources:
requests:
storage: 1Gi |
| Code Block |
---|
title | Static Volume Claim |
---|
collapse | true |
---|
| volumes:
- name: localtime
hostPath:
path: /etc/localtime
- name: sdnc-data
persistentVolumeClaim:
claimName: sdnc-db |
|
|
2 | sdnc-deployment.yaml
| Field | New value | Old value |
---|
.apiVersion | apps/v1beta1 1 | extensions/v1beta1 | .kind | StatefulSet | Deployment | .spec.serviceName | "sdnhostcluster" 2 | N/A | .spec.replicas | 3 | N/A | .spec.podManagementPolicy | "Parallel" 3 | N/A | .spec.containers.ports 4 | - containerPort: 2550 - containerPort: 8080 | N/A | .spec.containers.volumeMounts 5.a | - mountPath: /opt/onap/sdnc/bin/startODL.sh name: sdnc-startodl - mountPath: /opt/opendaylight/current/deploy name: sdnc-deploy | N/A | .spec.volumes 5.b | - name: sdnc-deploy hostPath: path: /home/ubuntu/cluster/deploy - name: sdnc-startodl hostPath: path: /home/ubuntu/cluster/script/cluster-startODL.sh | N/A |
|
3 | web-deployment.yaml | Field | New value | Old value |
---|
.apiVersion | apps/v1beta1 1 | extensions/v1beta1 | .kind | StatefulSet | Deployment | .spec.serviceName | "sdnc-portal" 2 | N/A | .spec.replicas | 3 | N/A | .spec.podManagementPolicy | "Parellel" 3 | N/A |
|
4 | sdnc-pv-pvc.yaml | Changed from Static volume mounts (PV and PVCs) to Dynamic volume mounts New Value | Old Value |
---|
Code Block |
---|
title | Dynamic Volume Provisioner |
---|
collapse | true |
---|
| kind: Service
apiVersion: v1
metadata:
name: nfs-provisioner
namespace: "{{ .Values.nsPrefix }}-sdnc"
labels:
app: nfs-provisioner
spec:
ports:
- name: nfs
port: 2049
- name: mountd
port: 20048
- name: rpcbind
port: 111
- name: rpcbind-udp
port: 111
protocol: UDP
selector:
app: nfs-provisioner
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: nfs-provisioner
namespace: "{{ .Values.nsPrefix }}-sdnc"
spec:
replicas: 1
strategy:
type: Recreate
template:
metadata:
labels:
app: nfs-provisioner
spec:
containers:
- name: nfs-provisioner
image: quay.io/kubernetes_incubator/nfs-provisioner:v1.0.8
ports:
- name: nfs
containerPort: 2049
- name: mountd
containerPort: 20048
- name: rpcbind
containerPort: 111
- name: rpcbind-udp
containerPort: 111
protocol: UDP
securityContext:
capabilities:
add:
- DAC_READ_SEARCH
- SYS_RESOURCE
args:
- "-provisioner=sdnc/nfs"
env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: SERVICE_NAME
value: nfs-provisioner
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: export-volume
mountPath: /export
volumes:
- name: export-volume
hostPath:
path: /dockerdata-nfs/{{ .Values.nsPrefix }}/sdnc/data
---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: "{{ .Values.nsPrefix }}-sdnc-data"
namespace: "{{ .Values.nsPrefix }}-sdnc"
provisioner: sdnc/nfs |
| Code Block |
---|
title | Static volume |
---|
collapse | true |
---|
| kind: PersistentVolume
metadata:
name: "{{ .Values.nsPrefix }}-sdnc-db"
namespace: "{{ .Values.nsPrefix }}-sdnc"
labels:
name: "{{ .Values.nsPrefix }}-sdnc-db"
spec:
capacity:
storage: 2Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
hostPath:
path: /dockerdata-nfs/{{ .Values.nsPrefix }}/sdnc/data
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: sdnc-db
namespace: "{{ .Values.nsPrefix }}-sdnc"
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 2Gi
selector:
matchLabels:
name: "{{ .Values.nsPrefix }}-sdnc-db" |
|
|
5 | all-services.yaml
| - Add a headless service of sdnc 6
Code Block |
---|
---
apiVersion: v1
kind: Service
metadata:
name: sdnhostcluster
namespace: onap-sdnc
labels:
app: sdnc
annotations:
service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
spec:
ports:
- name: "sdnc-cluster-port"
port: 2550
clusterIP: None
selector:
app: sdnc
sessionAffinity: None
type: ClusterIP |
|
(Find sdnhost service and add expose one more port under .spec.ports) Code Block |
---|
- name: "sdnc-jolokia-port-8080"
port: 9090
targetPort: 8080
nodePort: {{ .Values.nodePortPrefix }}00 |
|
...
On the node where you have configured nfs server, run the following:
# | Purpose | Command and Example |
---|
1 | find the node name | Expand |
---|
title | find nfs server node |
---|
| Run command "ps -ef|grep nfs", you should - node with nfs server runs nfsd process:
ubuntu@sdnc-k8s:~$ ps -ef|grep nfs root 3473 2 0 Dec07 ? 00:00:00 [nfsiod] root 11072 2 0 Dec06 ? 00:00:00 [nfsd4_callbacks] root 11074 2 0 Dec06 ? 00:00:00 [nfsd] root 11075 2 0 Dec06 ? 00:00:00 [nfsd] root 11076 2 0 Dec06 ? 00:00:00 [nfsd] root 11077 2 0 Dec06 ? 00:00:00 [nfsd] root 11078 2 0 Dec06 ? 00:00:00 [nfsd] root 11079 2 0 Dec06 ? 00:00:03 [nfsd] root 11080 2 0 Dec06 ? 00:00:13 [nfsd] root 11081 2 0 Dec06 ? 00:00:42 [nfsd] ubuntu@sdnc-k8s:~$
- node with nfs client runs nfs svc process:
ubuntu@sdnc-k8s-2:~$ ps -ef|grep nfs ubuntu 5911 5890 0 20:10 pts/0 00:00:00 grep --color=auto nfs root 18739 2 0 Dec06 ? 00:00:00 [nfsiod] root 18749 2 0 Dec06 ? 00:00:00 [nfsv4.0-svc] ubuntu@sdnc-k8s-2:~$
|
kubectl get node Expand |
---|
| ubuntu@sdnc-k8s:~$ kubectl get node NAME STATUS ROLES AGE VERSION sdnc-k8s Ready master 6d v1.8.4 sdnc-k8s-2 Ready <none> 6d v1.8.4 ubuntu@sdnc-k8s:~$ |
|
2 | set label on the node | kubectl label nodes <NODE_NAME_FROM_LAST_STEP> disktype=ssd Expand |
---|
| ubuntu@sdnc-k8s:~$ kubectl label nodes sdnc-k8s disktype=ssd node "sdnc-k8s" labeled |
|
3 | check the label has been set on the node | kubectl get node --show-labels |
4 | update nfs-provisioner pod template to force it running on the nfs server node | In sdnc-pv-pv.yaml file, add “spec.template.spec.nodeSelector” for pod “nfs-provisioner” Expand |
---|
title | An example of the nfs-provisioner pod with nodeSelector |
---|
| Image Modified
|
|
Create New Script: cluster-startODL.sh
...