I believe that OMSA can be realized using open source service mesh technology projects such as istio.io, linkerd and openResty + Consul etc.. In my view, it would be good to identify various open source projects and provide a comparison. Before that, I feel that it is necessary to list down requirements from ONAP perspective (personally, I don't believe that ONAP has any additional requirements beyond what service mesh technology requirements are, but it is good to list them down). What do you think? If you have done some analysis already, it is good to post it here and we can discuss further.
I believe that OMSA can be built on open source projects, but in my opinion, OMSA is more than service mesh. service mesh is mainly aimed to build a reliable and secure microservices communication infrastructure while OMSA covers every aspect we need for a Carrier-Grade ONAP platform, including Microservice development, deployment, orchestration, communication, governance and monitoring.
Specifically, for service, I have already done some investigation on Istio, Istio has a well-designed architecture and backup of some big companies such as Google and IBM, but it's not production-ready yet. MSB project plans to integrate Istio with ONAP in Beijing release, but it's mostly for investigation rather than a promised delivery. After it's verified, we will introduce Istio into ONAP.
I guess istio supports Policy and security portion of it. It also supports API management.
From monitoring bucket in your picture perspective, is the idea is to use fluentd (for logging), OpenTracing (for tracing) and Prometheus for metrics/monitoring? If so, that is good. these are CNCF projects and using them comprehensively for ONAP is very good.
What do you have in mind for Governance - Configuration management?
Yes, that's the plan. I prefer to integrate CNCF projects and leverage them to complete the OMSA. For configuration, I see that you already have proposed using distributed kv storage, which can be a good start point. Have you ever take a look at Spring Cloud Config ?
Yes, it is good to go with CNCF projects and glad that you concur.
I have looked at the Spring Cloud Config. Since, none of the projects are using spring framework, we have suggested to go with cfg4J which has Consul backend in our "Distributed KV Store" proposal.
With service mesh support , is there any plan to have DMaaP communication also enabled through the sidecar ? Also I think the REST end points registration of each MS need to be done with API Gateway seprately ? Or is it taken care by the sidecar ?
DMaaP communication is not considered yet at this point. First, we may want to think about what's the additional benefits brought by letting the async message pass through sidecar?
REST endpoints registration don't need to be done separately. Service mesh has a centralized control plane and sidecar can get service endpoints from it.
Envoy proxy that is used by istio has support for multiple protocols. As far as I know it has support for HTTP, gRPC, MySQL and Kafka. BTW, DMaaP uses Kafka inside. So, DMaaP should work across Envoy proxies.
In case of istio, my understanding is that service registration and service discovery happens among side cars with no support from application containers.
Thanks Srini, In the diagram above, the External API GW is another instance of Envoy or is it a separate module which will handle the API routing to the right Service end point realized through a group of Pods with side cars?
Hi Manoj, Good question. I need to spend some time to understand whether Envoy is capable of doing this. I would assume so. But need to spend some time to confirm.
This is the vision of OMSA, some of the functionalities have already been implemented and used in Amsterdam, such as service orchestration by OOM project, service registration/discovery/sync communication by MSB project, and some are still missing. I hope that for the next step we can define all the functionalities in OMSA and achieve it with all the ONAP project together.
HuabingZhao I have 2 questions: 1 . Does each microservice expose two different types of API? i.e. one on sidebar and one for regular clients. If so, how much implementation and maintenance overhead do they have? 2. In the diagram, the Async Message Bus seems serving as the adapter/control layer dealing with the run-time environments. Is this consistent with other ONAP components in term of how they are deployment and controlled? I just joined so have missed previous discussion and trying to catch up. Thanks.
Not really, ideally sidecar should be transparent to the microservice implementation. MSB gives the flexibility that a microservice can expose its endpoints at a different URL other than the real implementation, and it can be exposed at more than URLs.
For question 2:
No, Async message bus is one way of communication between microservices. The red arrows in the diagram cause confusions, I removed them.
Thanks Huabing. There are still things that need to be clarified and/or covered explicitly in the Architecture. I'd like to start a discussion with the team here sometime next week. I don't know who has been coordinate the meeting. HuabingZhao is it you? If not exist yet, we should have a mail-list for regular communication on this.
20 Comments
Srinivasa Addepalli
Hi Huabing and Manoj,
I believe that OMSA can be realized using open source service mesh technology projects such as istio.io, linkerd and openResty + Consul etc.. In my view, it would be good to identify various open source projects and provide a comparison. Before that, I feel that it is necessary to list down requirements from ONAP perspective (personally, I don't believe that ONAP has any additional requirements beyond what service mesh technology requirements are, but it is good to list them down). What do you think? If you have done some analysis already, it is good to post it here and we can discuss further.
Thanks
Srini
HuabingZhao
Hi Srinivasa,
I believe that OMSA can be built on open source projects, but in my opinion, OMSA is more than service mesh. service mesh is mainly aimed to build a reliable and secure microservices communication infrastructure while OMSA covers every aspect we need for a Carrier-Grade ONAP platform, including Microservice development, deployment, orchestration, communication, governance and monitoring.
Specifically, for service, I have already done some investigation on Istio, Istio has a well-designed architecture and backup of some big companies such as Google and IBM, but it's not production-ready yet. MSB project plans to integrate Istio with ONAP in Beijing release, but it's mostly for investigation rather than a promised delivery. After it's verified, we will introduce Istio into ONAP.
Srinivasa Addepalli
Hi Huabing,
I guess istio supports Policy and security portion of it. It also supports API management.
From monitoring bucket in your picture perspective, is the idea is to use fluentd (for logging), OpenTracing (for tracing) and Prometheus for metrics/monitoring? If so, that is good. these are CNCF projects and using them comprehensively for ONAP is very good.
What do you have in mind for Governance - Configuration management?
HuabingZhao
Yes, that's the plan. I prefer to integrate CNCF projects and leverage them to complete the OMSA. For configuration, I see that you already have proposed using distributed kv storage, which can be a good start point. Have you ever take a look at Spring Cloud Config ?
Srinivasa Addepalli
Yes, it is good to go with CNCF projects and glad that you concur.
I have looked at the Spring Cloud Config. Since, none of the projects are using spring framework, we have suggested to go with cfg4J which has Consul backend in our "Distributed KV Store" proposal.
Manoj Nair
Hi Huabing,
With service mesh support , is there any plan to have DMaaP communication also enabled through the sidecar ? Also I think the REST end points registration of each MS need to be done with API Gateway seprately ? Or is it taken care by the sidecar ?
HuabingZhao
DMaaP communication is not considered yet at this point. First, we may want to think about what's the additional benefits brought by letting the async message pass through sidecar?
REST endpoints registration don't need to be done separately. Service mesh has a centralized control plane and sidecar can get service endpoints from it.
Srinivasa Addepalli
Hi Manoj,
Envoy proxy that is used by istio has support for multiple protocols. As far as I know it has support for HTTP, gRPC, MySQL and Kafka. BTW, DMaaP uses Kafka inside. So, DMaaP should work across Envoy proxies.
In case of istio, my understanding is that service registration and service discovery happens among side cars with no support from application containers.
Srinivasa Addepalli
BTW, Envoy also has very good extensible mechanism to add new protocol plugins.
Manoj Nair
Thanks Srini, In the diagram above, the External API GW is another instance of Envoy or is it a separate module which will handle the API routing to the right Service end point realized through a group of Pods with side cars?
HuabingZhao
Logically it's another component, for implementation, it can be Envoy, nginx or other implementation.
Srinivasa Addepalli
Hi Manoj, Good question. I need to spend some time to understand whether Envoy is capable of doing this. I would assume so. But need to spend some time to confirm.
Arash Hekmat
Hey guys,
Is this architecture used in any microservice development in ONAP currently?
HuabingZhao
Hi Arash,
This is the vision of OMSA, some of the functionalities have already been implemented and used in Amsterdam, such as service orchestration by OOM project, service registration/discovery/sync communication by MSB project, and some are still missing. I hope that for the next step we can define all the functionalities in OMSA and achieve it with all the ONAP project together.
Kevin Wan
Is there weekly/regular discussion on the mS architecture? any mail-list for this sub-team? I like to join the discussion.
HuabingZhao
We can leverage the MSB weekly meeting time slot and zoom bridge for the discussion if you folks are comfortable with that.
Kevin Wan
HuabingZhao I have 2 questions: 1 . Does each microservice expose two different types of API? i.e. one on sidebar and one for regular clients. If so, how much implementation and maintenance overhead do they have? 2. In the diagram, the Async Message Bus seems serving as the adapter/control layer dealing with the run-time environments. Is this consistent with other ONAP components in term of how they are deployment and controlled? I just joined so have missed previous discussion and trying to catch up. Thanks.
HuabingZhao
For question 1:
Not really, ideally sidecar should be transparent to the microservice implementation. MSB gives the flexibility that a microservice can expose its endpoints at a different URL other than the real implementation, and it can be exposed at more than URLs.
For question 2:
No, Async message bus is one way of communication between microservices. The red arrows in the diagram cause confusions, I removed them.
Kevin Wan
Thanks Huabing. There are still things that need to be clarified and/or covered explicitly in the Architecture. I'd like to start a discussion with the team here sometime next week. I don't know who has been coordinate the meeting. HuabingZhao is it you? If not exist yet, we should have a mail-list for regular communication on this.
HuabingZhao
I would be glad to coordinate the meeting if you guys are ok with that.
Let's use the next MSB meeting for that if the time works for you and others.