...
In the most of the cases, in tosca model template there is compatibility in field types. Only two types (java.util.Date and Boolean) are not compatible between Eclipse-Link and Hibernate.
Note about precision in DATETIME (https://mariadb.com/kb/en/datetime/): MariaDB can store microseconds with a precision between 0 and 6. If no microsecond precision is specified, then 0 is used by default.
Code Block | ||||
---|---|---|---|---|
| ||||
@Embeddable @Data public class ExampleKey implements Serializable { private static final long serialVersionUID = 1L; @Column(name = "name", length = 120) private String name; @Column(name = "version", length = 20) private String version; } @Entity @Table(name = "Example") @Data public class JpaExample implements Serializable { private static final long serialVersionUID = 1L; @EmbeddedId @VerifyKey @NotNull private ExampleKey key; @Column private Boolean primed; @Column(name = "timeStamp", precision = 3) @Temporal(TemporalType.TIMESTAMP) @NotNull private Date timeStamp; } |
...
MariaDB [controlloop]> select * from ExampleObjListEc;
+-------------+---------+
| name | version |
+-------------+---------+
| ExampleList | 1.0.0 |
+-------------+---------+
MariaDB [controlloop]> select * from JpaExampleObjListEc_EXAMPLES;
+-------------+---------+------------+---------------+--------+-----------+
| name | version | child_name | child_version | primed | timeStamp |
+-------------+---------+------------+---------------+--------+-----------+
| ExampleList | 1.0.0 | example1 | 1.0.0 | NULL | NULL |
| ExampleList | 1.0.0 | example2 | 1.0.0 | NULL | NULL |
+-------------+---------+------------+---------------+--------+-----------+
...
List of Objects using
...
ElementCollection (hacked)
In this scenario JpaExampleLob class is not entity and neither embeddedOneToMany is not used in tosca model template. The example below shows an alternative to ElementCollection. In this example there is a field names collision.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
@Entity @Table(name = "ExampleObjMapExampleObjListEc") @Data public class JpaExampleObjMapJpaExampleObjListEc implements Serializable { private static final long serialVersionUID = 1L; @EmbeddedId @VerifyKey @NotNull private ExampleKey key; @OneToMany(fetch = FetchType.EAGER, cascade = CascadeType.ALL, orphanRemoval = true)@ElementCollection @Lob private Map<@NotNull String, @NotNull JpaExample> List<@NotNull JpaExampleLob> examples; } |
Representation of the example in YAML of what we could save using JpaExampleObjMap entity.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
ExampleObjMap:
name: ExampleObjMap
version: 1.0.0
examples:
MyKey1:
name: example1
version: 1.0.0
MyKey2:
name: example2
version: 1.0.0 |
Eclipse-Link
Hibernate
MariaDB [controlloop]> SHOW CREATE TABLE JpaExampleObjListEc_EXAMPLES;
+------------------------------MariaDB [controlloop]> describe ExampleObjMap;
+----------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------+--------------+------+-----+---------+-------+
| Table EXAMPLES | longblob | YES | | Create NULLTable | |
| name | varchar(120) | NO | PRI | NULL | |
| version | varchar(20) | NO | PRI | NULL | |
+----------+--------------+------+-----+---------+-------+
After saving the example:
MariaDB [controlloop]> select * from ExampleObjMap;
+--------------------------+---------------+---------+
| EXAMPLES | name | version |
+--------------------------+---------------+---------+
| .... <binary code> .... | ExampleObjMap | 1.0.0 |
+--------------------------+---------------+---------+
Removing @Lob Annotation
The keys name and version of the JpaExampleObjMap as the same as JpaExample, so Eclipse-Link creates a wrong ExampleObjMap_Example table.
MariaDB [controlloop]> describe ExampleObjMap;
+---------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------+--------------+------+-----+---------+-------+
| name | varchar(120) | NO | PRI | NULL | |
| version | varchar(20) | NO | PRI | NULL | |
+---------+--------------+------+-----+---------+-------+
MariaDB [controlloop]> SHOW CREATE TABLE ExampleObjMap_Example;
+-----------------------+-------------------------------------------------------------------------------------------------------------+
| Table | Create Table JpaExampleObjListEc_EXAMPLES | CREATE TABLE `JpaExampleObjListEc_EXAMPLES` (
`name` varchar(120) NOT NULL,
`version` varchar(20) NOT NULL,
`examples` longblob DEFAULT NULL,
KEY `FK1a58hldurq1910ne2hbm9a7af` (`name`,`version`),
CONSTRAINT `FK1a58hldurq1910ne2hbm9a7af` FOREIGN KEY (`name`, `version`)
REFERENCES `ExampleObjListEc` (`name`, `version`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 |
+-----------------------+-------+------------------------------------------------------------------------------------------------------+| ExampleObjMap_Example | CREATE TABLE `ExampleObjMap_Example` (
`name` varchar(120) NOT NULL,
After saving an example:
MariaDB [controlloop]> select * from JpaExampleObjListEc_EXAMPLES;
+-------------+---------+---------------------------+
| name | version | examples |
+-------------+---------+---------------------------+
| ExampleList | 1.0.0 | `version` varchar(20) NOT NULL,
`EXAMPLES_KEY` varchar(255) DEFAULT NULL,
PRIMARY KEY (`name`,`version`),
.... <binary code> .... |
| ExampleList | 1.0.0 | .... <binary code> .... |
+-------------+---------+---------------------------+
Map of Objects using OneToMany
OneToMany is not used in tosca model template. The example below shows an alternative to ElementCollection. In this example there is a field names collision.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
@Entity
@Table(name = "ExampleObjMap")
@Data
public class JpaExampleObjMap implements Serializable {
private static final long serialVersionUID = 1L;
@EmbeddedId
@VerifyKey
@NotNull
private ExampleKey key;
@OneToMany(fetch = FetchType.EAGER, cascade = CascadeType.ALL, orphanRemoval = true)
@Lob
private Map<@NotNull String, @NotNull JpaExample> examples;
} |
Representation of the example in YAML of what we could save using JpaExampleObjMap entity.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
ExampleObjMap:
name: ExampleObjMap
version: 1.0.0
examples:
MyKey1:
name: example1
version: 1.0.0
MyKey2:
name: example2
version: 1.0.0 |
Eclipse-Link
MariaDB [controlloop]> describe ExampleObjMap;
+----------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------+--------------+------+-----+---------+-------+
| EXAMPLES | longblob | YES | | NULL | |
| name | varchar(120) | NO | PRI | NULL | |
| version | varchar(20) | NO | PRI | NULL | |
+ CONSTRAINT `FK_ExampleObjMap_Example_name` FOREIGN KEY (`name`, `version`)
REFERENCES `ExampleObjMap` (`name`, `version`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 |
+-----------------------+--------------+------+-----+---------+-------+
After saving the example:
MariaDB [controlloop]> select * from ExampleObjMap;
+--------------------------+---------------+-----------------+
| EXAMPLES | name | version |
+--------------------------+
Using these tables I have got this error: Cannot add or update a child row: a foreign key constraint fails (`controlloop`.`ExampleObjMap_Example`, CONSTRAINT `FK_ExampleObjMap_Example_name` FOREIGN KEY (`name`, `version`) REFERENCES `ExampleObjMap` (`name`, `version`))
Error Code: 1452
Map of Objects using OneToMany (fixed)
The example below shows an alternative to ElementCollection with the fix to avoid field names collision.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
@Entity
@Table(name = "ExampleObjMap")
@Data
public class JpaExampleObjMap implements Serializable {
private static final long serialVersionUID = 1L;
@EmbeddedId
@VerifyKey
@NotNull
private ExampleKey key;
@OneToMany(cascade = CascadeType.ALL, fetch = FetchType.EAGER)
@JoinTable(
joinColumns = {@JoinColumn(name = "parent_name", referencedColumnName = "name"),
@JoinColumn(name = "parent_version", referencedColumnName = "version")},
inverseJoinColumns = {@JoinColumn(name = "child_name", referencedColumnName = "name"),
@JoinColumn(name = "child_version", referencedColumnName = "version")})
private Map<@NotNull String, @NotNull JpaExample> examples;
} |
...
---------------+---------+
| .... <binary code> .... | ExampleObjMap | 1.0.0 |
+--------------------------+---------------+---------+
Removing @Lob Annotation
The keys name and version of the JpaExampleObjMap as the same as JpaExample, so Eclipse-Link creates a wrong ExampleObjMap_Example table.
MariaDB [controlloop]> describe ExampleObjMap;
+---------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------+--------------+------+-----+---------+-------+
| name | varchar(120) | NO | PRI | NULL | |
| version | varchar(20) | NO | PRI | NULL | |
+---------+--------------+------+-----+---------+-------+
MariaDB [controlloop]> SHOW CREATE TABLE ExampleObjMap_Example;
+-----------------------+-------------------------------------------------------------------------------------------------------------+
| Table | Create Table |
+-----------------------+-------------------------------------------------------------------------------------------------------------+
| ExampleObjMap_Example | CREATE TABLE `ExampleObjMap_Example` (
`parent_name``name` varchar(120) NOT NULL,
`parent_version``version` varchar(20) NOT NULL,
`child`EXAMPLES_name`KEY` varchar(120255) NOTDEFAULT NULL,
PRIMARY `child_version`KEY varchar(20) NOT NULL`name`,`version`),
`EXAMPLES_KEY` varchar(255) DEFAULT NULL,
PRIMARY KEY (`parent_name`,`parent_version`,`child_name`,`child_version`),CONSTRAINT `FK_ExampleObjMap_Example_name` FOREIGN KEY (`name`, `version`)
KEY `FK_ExampleObjMap_Example_child_name` (`child_name`,`child_version`),
CONSTRAINT `FK_ExampleObjMap_Example_child_name` FOREIGN KEY (`child_name`, `child_version` REFERENCES `ExampleObjMap` (`name`, `version`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 REFERENCES `Example` (`name`, `version`),
CONSTRAINT `FK_ExampleObjMap_Example_parent_name` FOREIGN KEY (`parent_name`, `parent_version`)
REFERENCES `ExampleObjMap` (`name`, `version`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 |
+------|
+-------------------------+-------------------------------------------------------------------------------------------------------------+
After saving the example:
MariaDB [controlloop]> select * from ExampleObjMap;
+---------------+---------+
| name | version |
+---------------+---------+
| ExampleObjMap | 1.0.0 |
+---------------+---------+
Using these tables I have got this error: Cannot add or update a child row: a foreign key constraint fails (`controlloop`.`ExampleObjMap_Example`, CONSTRAINT `FK_ExampleObjMap_Example_name` FOREIGN KEY (`name`, `version`) REFERENCES `ExampleObjMap` (`name`, `version`))
Error Code: 1452
Map of Objects using OneToMany (fixed)
The example below shows an alternative to ElementCollection with the fix to avoid field names collision.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
@Entity
@Table(name = "ExampleObjMap")
@Data
public class JpaExampleObjMap implements Serializable {
private static final long serialVersionUID = 1L;
@EmbeddedId
@VerifyKey
@NotNull
private ExampleKey key;
@OneToMany(cascade = CascadeType.ALL, fetch = FetchType.EAGER)
@JoinTable(
joinColumns = {@JoinColumn(name = "parent_name", referencedColumnName = "name"),
@JoinColumn(name = "parent_version", referencedColumnName = "version")},
inverseJoinColumns = {@JoinColumn(name = "child_name", referencedColumnName = "name"),
@JoinColumn(name = "child_version", referencedColumnName = "version")})
private Map<@NotNull String, @NotNull JpaExample> examples;
} |
Eclipse-Link
MariaDB [controlloop]> SHOW CREATE TABLE ExampleObjMap_Example;
+------MariaDB [controlloop]> select * from Example;
+--------+-----------+----------+---------+
| PRIMED | timeStamp | name | version |
+--------+-----------+----------+---------+
| NULL | NULL | example1 | 1.0.0 |
| NULL | NULL | example2 | 1.0.0 |
+--------+-----------+----------+---------+MariaDB [controlloop]> select * from ExampleObjMap_Example;
+---------------+----------------+------------+---------------+--------------+
| parent_name | parent_version | child_name | child_version | EXAMPLES_KEY |
+---------------+----------------+------------+---------------+--------------+
| ExampleObjMap | 1.0.0+
| Table | Create Table | example1 | 1.0.0 | MyKey1 |
| ExampleObjMap | 1.0.0 | example2 | 1.0.0 | MyKey2 |
+---------------+----------------+------------+---------------+--------------+
Hibernate
MariaDB [controlloop]> SHOW CREATE TABLE ExampleObjMap_Example;
+-----------------------+-------------------------------------------------------------------------------------------------------------+
| Table | Create Table |
+-----------------------+-------------------------------------------------------------------------------------------------------------+
| ExampleObjMap_Example | CREATE TABLE `ExampleObjMap_Example` (+
| ExampleObjMap_Example | CREATE TABLE `ExampleObjMap_Example` (
`parent_name` varchar(120) NOT NULL,
`parent_version` varchar(20) NOT NULL,
`child_name` varchar(120) NOT NULL,
`child_version` varchar(20) NOT NULL,
`parent`EXAMPLES_name`KEY` varchar(120255) NOTDEFAULT NULL,
PRIMARY KEY (`parent_version` varchar(20) NOT NULLname`,`parent_version`,`child_name`,`child_version`),
`child KEY `FK_ExampleObjMap_Example_child_name` varchar(120) NOT NULL(`child_name`,`child_version`),
`child_version` varchar(20) NOT NULL,
`examples_KEY` varchar(255) NOT NULL,
CONSTRAINT `FK_ExampleObjMap_Example_child_name` FOREIGN KEY (`child_name`, `child_version`)
PRIMARY KEY (`parent_name`,`parent_version`,`examples_KEY`),
UNIQUE KEY `UK_ma8t20i58tt5ilqjrk2ged242` (`child_name`,`child_version`REFERENCES `Example` (`name`, `version`),
CONSTRAINT `FK2n1x5g38x55wejkvnvhqin980``FK_ExampleObjMap_Example_parent_name` FOREIGN KEY (`parent_name`, `parent_version`)
REFERENCES `ExampleObjMap` (`name`, `version`),
CONSTRAINT `FKstwyuf69lrj4tjpafwriuwp3h` FOREIGN KEY (`child_name`, `child_version`)
ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 REFERENCES `Example` (`name`, `version`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 |
+-|
+------------------------------+-------------------------------------------------------------------------------------------------------------+
...
MariaDB [controlloop]> select * from ExampleObjMap;
+---------------+---------+
| name | version |
+---------------+---------+
| ExampleObjMap | 1.0.0 |
+---------------+---------+
MariaDB [controlloop]> select * from Example;
+--------+--+---------+--------+--+---------+
| namePRIMED | timeStamp | versionname | primed | timeStampversion |
+--------+--+---------+--------+--+---------+
| NULL | NULL | example1 | 1.0.0 |
| NULL | NULL |
| example2 | 1.0.0 | NULL | NULL |
+--------+--+---------+--------+--+---------+
MariaDB [controlloop]> select * from ExampleObjMap_Example;
+---------------+----------------+------------+---------------+--------------+
| parent_name | parent_version | child_name | child_version | examplesEXAMPLES_KEY |
+---------------+----------------+------------+---------------+--------------+
| ExampleObjMap | 1.0.0 | example1 | 1.0.0 | MyKey1 |
| ExampleObjMap | 1.0.0 | example2 | 1.0.0 | MyKey2 |
+---------------+----------------+------------+---------------+--------------+ | 1.0.0 | example1 | 1.0.0 | MyKey1 |
| ExampleObjMap | 1.0.0 | example2 | 1.0.0 | MyKey2 |
+---------------+----------------+------------+---------------+--------------+
Hibernate
MariaDB [controlloop]> SHOW CREATE TABLE ExampleObjMap_Example;
+-----------------------+-------------------------------------------------------------------------------------------------------------+
| Table | Create Table |
+-----------------------+-------------------------------------------------------------------------------------------------------------+
| ExampleObjMap_Example | CREATE TABLE `ExampleObjMap_Example` (
`parent_name` varchar(120) NOT NULL,
`parent_version` varchar(20) NOT NULL,
`child_name` varchar(120) NOT NULL,
`child_version` varchar(20) NOT NULL,
`examples_KEY` varchar(255) NOT NULL,
PRIMARY KEY (`parent_name`,`parent_version`,`examples_KEY`),
UNIQUE KEY `UK_ma8t20i58tt5ilqjrk2ged242` (`child_name`,`child_version`),
CONSTRAINT `FK2n1x5g38x55wejkvnvhqin980` FOREIGN KEY (`parent_name`, `parent_version`)
REFERENCES `ExampleObjMap` (`name`, `version`),
CONSTRAINT `FKstwyuf69lrj4tjpafwriuwp3h` FOREIGN KEY (`child_name`, `child_version`)
REFERENCES `Example` (`name`, `version`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 |
+-----------------------+-------------------------------------------------------------------------------------------------------------+
After saving the example:
MariaDB [controlloop]> select * from ExampleObjMap;
+---------------+---------+
| name | version |
+---------------+---------+
| ExampleObjMap | 1.0.0 |
+---------------+---------+
MariaDB [controlloop]> select * from Example;
+----------+---------+--------+-----------+
| name | version | primed | timeStamp |
+----------+---------+--------+-----------+
| example1 | 1.0.0 | NULL | NULL |
| example2 | 1.0.0 | NULL | NULL |
+----------+---------+--------+-----------+
MariaDB [controlloop]> select * from ExampleObjMap_Example;
+---------------+----------------+------------+---------------+--------------+
| parent_name | parent_version | child_name | child_version | examples_KEY |
+---------------+----------------+------------+---------------+--------------+
| ExampleObjMap | 1.0.0 | example1 | 1.0.0 | MyKey1 |
| ExampleObjMap | 1.0.0 | example2 | 1.0.0 | MyKey2 |
+---------------+----------------+------------+---------------+--------------+
Performance issue
Using current version, for each operation save/read/delete of the service template below, control-loop runtime application access to 14 tables and handles 51 rows.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
name: ToscaServiceTemplateSimple
version: 1.0.1
tosca_definitions_version: tosca_simple_yaml_1_3
data_types:
onap.datatypes.ToscaConceptIdentifier:
derived_from: tosca.datatypes.Root
properties:
name:
type: string
required: true
version:
type: string
required: true
node_types:
org.onap.policy.clamp.controlloop.Participant:
version: 1.0.1
derived_from: tosca.nodetypes.Root
properties:
provider:
type: string
requred: false
org.onap.policy.clamp.controlloop.ControlLoopElement:
version: 1.0.1
derived_from: tosca.nodetypes.Root
properties:
provider:
type: string
requred: false
participantType:
type: onap.datatypes.ToscaConceptIdentifier
requred: true
startPhase:
type: integer
required: false
constraints:
- greater_or_equal: 0
metadata:
common: true
description: check wiki documentation about start phase
stopped simultaneously
org.onap.policy.clamp.controlloop.ControlLoop:
version: 1.0.1
derived_from: tosca.nodetypes.Root
properties:
provider:
type: string
requred: false
elements:
type: list
required: true
entry_schema:
type: onap.datatypes.ToscaConceptIdentifier
org.onap.policy.clamp.controlloop.K8SMicroserviceControlLoopElement:
version: 1.0.1
derived_from: org.onap.policy.clamp.controlloop.ControlLoopElement
properties:
chart:
type: string
required: true
configs:
type: list
required: false
requirements:
type: string
requred: false
templates:
type: list
required: false
entry_schema:
values:
type: string
requred: true
topology_template:
node_templates:
org.onap.k8s.controlloop.K8SControlLoopParticipant:
version: 2.3.4
type: org.onap.policy.clamp.controlloop.Participant
type_version: 1.0.1
description: Participant for K8S
properties:
provider: ONAP
org.onap.domain.database.HelloWorld_K8SMicroserviceControlLoopElement:
# Chart from any chart repository configured on helm client.
version: 1.2.3
type: org.onap.policy.clamp.controlloop.K8SMicroserviceControlLoopElement
type_version: 1.0.0
description: Control loop element for the K8S microservice for Hello World
properties:
provider: ONAP
participantType:
name: org.onap.k8s.controlloop.K8SControlLoopParticipant1
version: 2.3.4
chart:
chartId:
name: dummy
version: 0.1.0
releaseName: test
namespace: test
org.onap.domain.database.PMSH_K8SMicroserviceControlLoopElement:
# Chart from local file system
version: 1.2.3
type: org.onap.policy.clamp.controlloop.K8SMicroserviceControlLoopElement
type_version: 1.0.0
description: Control loop element for the K8S microservice for PMSH
properties:
provider: ONAP
participantType:
name: org.onap.k8s.controlloop.K8SControlLoopParticipant2
version: 2.3.4
startPhase: 1
chart:
chartId:
name: dcae-pmsh
version: 8.0.0
namespace: onap
releaseName: pmshms
repository:
repoName: chartmuseum
protocol: http
address: 10.152.183.120
port: 80
userName: onapinitializer
password: demo123456!
overrideParams:
global.masterPassword: test
org.onap.domain.database.Local_K8SMicroserviceControlLoopElement:
# Chart installation without passing repository name
version: 1.2.3
type: org.onap.policy.clamp.controlloop.K8SMicroserviceControlLoopElement
type_version: 1.0.0
description: Control loop element for the K8S microservice for local chart
properties:
provider: ONAP
participantType:
name: org.onap.k8s.controlloop.K8SControlLoopParticipant
version: 2.3.4
chart:
chartId:
name: nginx-ingress
version: 0.9.1
releaseName: nginxms
namespace: test
repository:
repoName: nginx-stable
org.onap.domain.sample.GenericK8s_ControlLoopDefinition:
version: 1.2.3
type: org.onap.policy.clamp.controlloop.ControlLoop
type_version: 1.0.0
description: Control loop for Hello World
properties:
provider: ONAP
elements:
- name: org.onap.domain.database.HelloWorld_K8SMicroserviceControlLoopElement
version: 1.2.3
- name: org.onap.domain.database.PMSH_K8SMicroserviceControlLoopElement
version: 1.2.3
- name: org.onap.domain.database.Local_K8SMicroserviceControlLoopElement
version: 1.2.3
|
Multi templates
Right now using Spring repositories it is possible to create more than one service template, as service templates in policy-models/tosca. It works in both Eclipse-Link and Hibernate.
How to create additional service template (just add different name and version into the yaml file):
...
- Move to Hibernate with EclipseLink tables is probably possible.
- The presence of longblob types used to store whole objects is an issues, it is readable only by Java language or by any other language compatible with Java binary code. I am not sure if a db-migrator can handle that.
- Document databases store all information for a given object in a single instance in the database, and every stored object can be different from every other. Each operation save/read/delete of a service template, could be done using one access to a document. So, using Cassandra/MongoDB /Cassandra will solve all issues.
JpaRepository
and and CassandraRepository
/MongoRepository
extendCrudRepository
and expose the capabilities of the underlying persistence technology in addition to the rather generic persistence technology-agnostic interfaces like e.g.CrudRepository
.
...